Share via

Auto refresh regular Synapse dedicated SQL pool tables

pmnguye2 21 Reputation points
2020-11-09T11:18:09.713+00:00

We're using Synapse Analytics with dedicated SQL pool. We try to create some regular tables from some parquet files from Azure Data Lake Storage Gen2 to use materialzied view. Is there anyway we can have it auto refreshed when new data coming into the ADLS?

Azure Data Lake Storage
Azure Data Lake Storage

An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.

Azure Synapse Analytics
Azure Synapse Analytics

An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.

0 comments

Answer accepted by question author

MartinJaffer-MSFT 26,161 Reputation points
2020-11-10T20:49:35.5+00:00

Hello @pmnguye2 and welcome to Microsoft Q&A.

I think you might want to look into Event Triggers. Even triggers can be set to start a pipeline whenever a blob / gen2 file is created. You can then have the pipeline tell Synapse / SQL pool to refresh the table.

The refresh could be done by using Copy activity to load data into SQL pool, in the pipeline triggered by the Event trigger.
Or if you have some other method set up, the pipeline could call on a stored proc to pull the load.

Was this answer helpful?

0 additional answers

Sort by: Most helpful
Most helpful Newest Oldest

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.