Pipeline in ADF for external tables

Question

Hello Team,

I am trying to create a pipeline to transfer the data (dynamically) from

Azure blob storage---> External tables in dedicated sql pool

I need to design the ADF pipeline. Please let me know the supporting document.

Thanks

RK

Answer 1

@Rohit Kulkarni - Thanks for the question and using MS Q&A platform.

To transfer data from Azure Blob Storage to External tables in Dedicated SQL Pool using Azure Data Factory, you can follow the below steps:

1. Create a new pipeline in Azure Data Factory.
2. Add a new "Copy Data" activity to the pipeline.
3. In the "Source" tab of the "Copy Data" activity, select "Azure Blob Storage" as the source data store.
4. Configure the connection to your Azure Blob Storage account and select the blob container and file(s) you want to copy data from.
5. In the "Sink" tab of the "Copy Data" activity, select "Azure Synapse Analytics" as the sink data store.
6. Configure the connection to your Dedicated SQL Pool and select the external table(s) you want to copy data to.
7. Map the source and sink columns by selecting the appropriate columns from the source and sink schemas.
8. Configure any additional settings for the "Copy Data" activity, such as data transformation or filtering.
9. Save and publish the pipeline.

You can refer to the following Microsoft documentation for more detailed information on how to create a pipeline to transfer data from Azure Blob Storage to External tables in Dedicated SQL Pool using Azure Data Factory:

https://docs.microsoft.com/en-us/azure/data-factory/connector-azure-blob-storage#copy-data-from-azure-blob-storage-to-azure-synapse-analytics

I hope this helps! Let me know if you have any further questions.