Copy data from azure select server to adls

Shambhu Rai 1,411 Reputation points
2023-12-02T19:10:06.3566667+00:00

Hi expert, How to copy data from SQL server db tables to in paraquat format in adls using one copy pipeline

Azure Data Explorer
Azure Data Explorer
An Azure data analytics service for real-time analysis on large volumes of data streaming from sources including applications, websites, and internet of things devices.
506 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,668 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,070 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,121 questions
0 comments No comments
{count} votes

2 answers

Sort by: Most helpful
  1. Konstantinos Passadis 17,456 Reputation points MVP
    2023-12-02T21:04:29.4866667+00:00

    Hello @Shambhu Rai

    You need two linked services :

    https://learn.microsoft.com/en-us/azure/data-factory/concepts-linked-services

    Create two Datasets for each one , source and destination :

    https://learn.microsoft.com/en-us/azure/data-factory/concepts-datasets-linked-services?tabs=data-factory

    Finally create a new Copy Data Activity Pipeline

    https://learn.microsoft.com/en-us/azure/data-factory/connector-sql-server?tabs=data-factory


    I hope this helps!

    Kindly mark the answer as Accepted and Upvote in case it helped!

    Regards

    0 comments No comments

  2. Javier Villegas 900 Reputation points MVP
    2023-12-02T23:16:07.4933333+00:00

    Hello @Shambhu Rai

    If you use SQL Server 2022 you can use Data Virtualization and CETAS to write directly parquet files into adls

    https://cloudblogs.microsoft.com/sqlserver/2022/10/05/data-virtualization-with-polybase-for-sql-server-2022/

    Regards