Convert Python api call to Data Factory Activity and pull data

Question

Following code is used to get data from an API. When I am running following code through jupyter notebooks, it brings the data. But I want to implement the same in Data factory. Not sure how parameters are passed and which activity to use(Rest or Http or Web). Please suggest on this.

Answer 1

Hi @Krishnamohan Nadimpalli ,

Thanks for using Microsoft Q&A forum and posting your query.

As per my understanding, you would like to get from a REST API with query parameters to filter data using Azure data factory. Please correct me if my understanding is wrong.

For your requirement, you can use Web activity to get the data and pass query parameters using pipeline variables or pipeline parameters as discussed in the SO thread - Where to specify REST API parameter for GET method in ADF Web Activity

For example as below:

In case if you just want to get the data and map few fields from the API response to your desired sink then use REST connector in COPY ACTIVITY Source

Please refer to this thread for more info on how to use REST for passing query parameters. You can declare pipeline variables or parameters for API parameters and use them in relative URL as above using dynamic expression.

Ref thread - how to pass variables to Azure Data Factory REST url's query stirng

An advantage of REST connector is that you can configure pagination rules in case if your API returns result in pages.

Hope this info helps.

---

Please don’t forget to Accept Answer and Yes for "was this answer helpful" wherever the information provided helps you, this can be beneficial to other community members.