Share via

Extracting data from REST API (JSON) in ADF to Azure Data Lake

RJ 1 Reputation point
2021-06-23T05:22:33.02+00:00

Hi

I have ADF (Azure Data Factory) calling two REST API's in JSON.

  1. Customer IDs
  2. Customer Data. - Customer ID.

Task is to extract the Customer data (JSON2) by passing Customer ID (JSON1) as key, and store it in Parque file format in a Data Lake.

please suggest me possible solutions.

Thank you in advance.

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,459 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,151 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,524 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. MartinJaffer-MSFT 26,081 Reputation points
    2021-06-23T21:47:48.51+00:00

    Hello @RJ and welcome to Microsoft Q&A.

    It sounds like you are saying you need to make 2 calls, one to fetch some metadata for use in making the second call.

    In this case, you can use a Lookup activity for the first call, and use a portion of the output in the second. The second would be a Copy activity with REST as source and Data Lake Parquet as sink.

    Please tell me if I misunderstood.

    1 person found this answer helpful.
    0 comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.