Is there a way to pull graphQL API data using Azure Data Factory

Vipin Sumathi 36

Is there a way to pull GraphQL API data using Azure Data Factory ?. I am trying to pull data from a third party application which uses GraphQL API . Is there a option in Azure Data Factory to do that?

SUNOJ KUMAR YELURU 14,886 Reputation points MVP

2021-08-18T00:18:16.853+00:00

To follow-up, Please let us know if you have further query on this.
Please don’t forget to Accept the answer
HimanshuSinha-msft 19,476 Reputation points Microsoft Employee

2021-08-19T16:23:01.04+00:00

Hello @Vipin Sumathi ,
We haven’t heard from you on the last response from @SUNOJ KUMAR YELURU and was just checking back to see if you have a resolution yet .In case if you have any resolution please do share that same with the community as it can be helpful to others . Otherwise, will respond back with the more details and we will try to help .
Thanks
Himanshu
Feldman, Ilya 1 Reputation point

2021-10-28T19:51:19.623+00:00

I don't believe that the answer provided is sufficient. I am facing a similar challenge connecting to the GraphQL API, and I see no material available online on this. If anyone has been able to get this accomplished successfully, please share your success!
Alex Gazzard 1 Reputation point

2022-03-30T14:59:25.957+00:00

Hello @Feldman, Ilya , @Vipin Sumathi , @SUNOJ KUMAR YELURU . I'm also facing the same challenge of trying to connect a GraphQL API to an Azure Data Factory, did any of you manage to achieve this in the end? If so would you be able to share how please? Thanks, Alex
Karol Kolodziejczyk 11 Reputation points

2022-08-25T12:18:10.837+00:00

Hello, actually you can do that as mentioned in the answer using HTTP/REST connector and POST method. You just need to configure Authorization header correctly and put the query in the body of the request.
David Steketee (US) 6 Reputation points

2022-09-08T18:44:05.087+00:00

When you submit your query, do you just dump the query into the body? Or do you embed the query in additional JSON?, e.g. { "query" : "query text" }
Karol Kolodziejczyk 11 Reputation points

2022-09-12T11:30:57.707+00:00

In the body it needs to be full JSON schema like your sample { "query" : "query text" }, matching with the schema of your GraphQL API.
David Steketee (US) 6 Reputation points

2022-09-12T13:10:00.477+00:00

Excellent. This is what I also discovered after sniffing the traffic from a C# app I wrote. I also had to replace internal quotes inside the query with \u0022.

Accepted answer

SUNOJ KUMAR YELURU 14,886 Reputation points MVP

2021-08-17T02:32:15.063+00:00

Hi @Vipin Sumathi

Microsoft Graph uses the HTTP method on your request to determine what your request is doing. The API supports the following methods.

Data factory offers a generic HTTP connector and a specific REST connector, allowing you to do retrieve data from HTTP endpoints by using GET or POST methods.
refer - https://stackoverflow.com/questions/35551533/using-azure-data-factory-to-get-data-from-a-rest-api

If the Answer is helpful, please click Accept Answer and up-vote, this can be beneficial to other community members.
Please sign in to rate this answer.
Emily Groves 1 Reputation point

2022-11-11T04:02:44.043+00:00

This is a generic answer about connecting using an http linked service. The OP asked specifically about setting up to query an api using graphQL.

If anyone has any experience of using synapse copy activity to get data using a graphQL api, I (and others it looks like) would be grateful.

I’m the meantime, I’ll keep working on mine and I’ll pop back here to share if I get it working.

Emily

Alex Gazzard 1 Reputation point

2022-11-11T10:35:02.767+00:00

Hi Emily,

I've actually got this working now, sorry for not updating the thread.

The way this works is:

Create a new dataset in ADF and select HTTP.

Then you need to select JSON as the format of your data type.

Next, create a new HTTP-linked service.

Enter your GraphQL endpoint into the Base URL field.

Set authentication type as anonymous

Create your Auth header, save and publish.

You then need to add your Copy data activity to the pipeline.

In the Source tab of the copy data activity select your new dataset.

Change the request method to Post.

Then in the request body, enter your GraphQL query. You will need to enclose the query like so: {query:"{Test(last: 10, filter: {test_date: \"2022-11-11\"}){edges{node {Test}}}}"}

To test this works, you can then use the Preview data button.

Hope this helps!

Alex

Emily Groves 1 Reputation point

2022-11-11T11:05:53.223+00:00

I had all of that in place already which is great, except I have moved my Authorization Header to the linked service (it was in the dataset).

I have encapsulated my qraphQL query (that works using postman) with {query:" at the beginning and "} at the end.
I'm getting 400 error. So the server doesn't like what I'm sending it.
I tried adding a content-type:GraphQL additional header, didn't make any difference.
Any ideas?

Aqshata Akshay Ajnadkar 0 Reputation points

2024-01-04T11:45:04.06+00:00

Your content type shouldn't be GraphQL. I made the same mistake. Content-Type should be application/json.

By any chance, anyone got there GraphQL query working in azure data factory?
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

2 additional answers

Emily Groves 1 Reputation point

2022-11-11T10:38:01.63+00:00

Fantastic, thanks Alex
I'll have a look right now :)
Please sign in to rate this answer.
Naveen Mittapally 1 Reputation point

2023-01-03T14:17:17.047+00:00

Hi @Emily Groves , Do you have any luck in connecting to a GraphQL endpoint using COPY activity. I am stuck with 400 error. But when i am using 'WEB' activity , i am able to establish the connection successfully. Quite strange!!
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.
Naveen Mittapally 1 Reputation point

2023-01-03T14:19:58.663+00:00

Hi @ShaikMaheer-MSFT , Can you please help me in resolving this issue?
Please sign in to rate this answer.
Naveen Mittapally 1 Reputation point

2023-01-03T15:36:05.183+00:00

Hi @ShaikMaheer-MSFT Never mind! I am able to fix the issue

Luka Ostojic 1 Reputation point

2023-02-06T12:14:13.9133333+00:00

Hi Naveen, how you resolved the 400 error ?
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

Is there a way to pull graphQL API data using Azure Data Factory

2 additional answers

Your answer