How can we load a table data from dedicated pool to spark Pool

Question

hi Community,
Is there a way through which we can load the whole table data from dedicated pool to spark pool datalake in Synapse?
I have a requirement where i am currently loading data in spark datalake tables directly from csv files. But i am thinking rather than using files for reading i would first load the data of my csv files into dedicated pool and from there into spark datalake?
so in future if we need to add new data into our table we can add that in Synapse dedicated pool and those should reflect in spark tables also.
****is ETL possible here?****

Answer 1

Hi @Devender ,

Thank you for posting query in Microsoft Q&A Platform.

As per my understanding, it seems you are looking for possibilities of taking data from dedicated SQL database table to delta tables in Synapse. Please correct me if I am wrong.

You can consider using dataflows in Azure data factory or Azure Synapse analytics. Or you can consider writing pyspark code in Synaspe notebooks as well.

When you use delta tables as in sink in dataflows. make sure to select inline dataset option.

Hope this helps. Please let me know if any further queries.

---------------

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification