Querying Azure Table from Azure Data Factory

Saravanan Ponnaiah 21 Reputation points
2022-06-03T12:44:27.603+00:00

Unlike data sources like Azure SQL or Azure Blob, there is no data connector available for Azure Table Store to use in Dataflow in ADF. So, currently we are copying data from Azure Table store to Azure Blob as csv. Then we query the data from blob in dataflow. For now, the records in Azure Table is minimum, so copy data to csv is pretty much faster. But in long run as the records grow to huge volume, copy complete data from Azure Table to Blob will not be an efficient activity. Is there any better approach to query & filter data from Azure Table within dataflow?

Azure Table Storage
Azure Table Storage
An Azure service that stores structured NoSQL data in the cloud.
157 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,643 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Suba Balaji 11,186 Reputation points
    2022-06-05T05:22:33.3+00:00

    Hi @Saravanan Ponnaiah ,

    Thanks for posting your question on Microsoft Q&A platform.

    Azure table storage connectors (source/sink) aren't available for use in mapping data flow.

    Using copy activity you can query from Table storage.

    Please check the performance tips here to make the copy activity load faster.

    Kindly revert if you are looking for any other detail in this regard.

    Thanks.

    0 comments