CI CD for Synapse spark pool lake database objects

saurabh 331 Reputation points
2023-04-12T23:01:42.3033333+00:00

How can one promote lake database objects from dev synapse workspace to higher environments using azure devops . For instance, for promoting sql serverless or dedicated pool, one can use extensiona to extract and publish dacpac’s which will create the database objects in higher environments. Is it possible to achieve the same for lake database objects created in spark pool? If yes, please share reference implementation links.

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,498 questions
{count} votes

Accepted answer
  1. BhargavaGunnam-MSFT 27,896 Reputation points Microsoft Employee
    2023-04-13T20:24:25.81+00:00

    Hello @Saurabh, Please correct me If my understanding is wrong. You want to publish the lake database objects created on the lake database from your dev Synapse workspace to higher environments via Azure DevOps. If this is your requirement, You can publish the schema here using the Azure synapse pipeline deployment task for the workspace. Ex: In my dev, I have created a new lake database and a table. I pushed the changes to my UAT with the release pipeline using the synapse deployment task. User's image

    User's image

    User's image

    Please note: When you create a lake database, it does not create any of the underlying structures. We need to load the data into lake database using pipelines to begin querying it. The synchronization of lake database objects between storage and Synapse is one-directional. https://learn.microsoft.com/en-us/azure/synapse-analytics/data-integration/data-integration-data-lake https://learn.microsoft.com/en-us/azure/synapse-analytics/database-designer/concepts-lake-database I hope this helps. Please let me know if you have any further questions.

    2 people found this answer helpful.

0 additional answers

Sort by: Most helpful