can we use airflow connector to microsoft azure for data extraction

kiran kumar challa 1 Reputation point
2022-10-14T22:30:27.47+00:00

can we use airflow connector to microsoft azure for data extraction

i want to get the data from microsoft azure data lake using apache airflow connector. Can we ?

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,426 questions
{count} votes

1 answer

Sort by: Most helpful
  1. MartinJaffer-MSFT 26,061 Reputation points
    2022-10-17T23:26:33.747+00:00

    Hello @kiran kumar challa ,
    Thanks for the question and using MS Q&A platform.

    As I understand, you want to connect your Apache Airflow to Azure Data Lake.
    I found Azure connectors here. Specifically, a Data Lake Hook. It doesn't mention whether this is for Data Lake Gen1 or Data Lake Gen2. You will want Gen2.

    If that doesn't work, you can try the HDFS, as Data Lake should be compatible with it.

    Data Lake Gen 2, is a storage account with Hierarchical Namespace enabled. If your Hierarchical Namespace is disabled, try the WASB hook.

    Please do let me if you have any queries.

    Thanks
    Martin


    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
      • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators
    0 comments No comments