Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Applies to: ✅ Warehouse in Microsoft Fabric
This article explains how Git integration and deployment pipelines work for warehouses in Microsoft Fabric. Learn how to set up a connection to your repository, manage your warehouses, and deploy them across different environments. Source control for Fabric Warehouse is currently a preview feature.
You can use both Git integration and Deployment pipelines for different scenarios:
- Use Git and SQL database projects to manage incremental change, team collaboration, and commit history in individual database objects.
- Use deployment pipelines to promote code changes to different pre-production and production environments.
Git integration
Git integration in Microsoft Fabric enables developers to integrate their development processes, tools, and best practices directly into the Fabric platform. It allows developers who are developing in Fabric to:
- Backup and version their work
- Revert to previous stages as needed
- Collaborate with others or work alone by using Git branches
- Apply the capabilities of familiar source control tools to manage Fabric items
For more information on the Git integration process, see:
- What is Microsoft Fabric Git integration?
- Basic concepts in Git integration
- Get started with Git integration
Set up a connection to source control
From the Workspace settings page, you can easily set up a connection to your repo to commit and sync changes.
- To set up the connection, see Get started with Git integration. Follow instructions to Connect to a Git repo to either Azure DevOps or GitHub as a Git provider.
- Once connected, your items, including warehouses, appear in the Source control panel.
- After you successfully connect the warehouse instances to the Git repo, you see the warehouse folder structure in the repo. You can now execute future operations, like creating a pull request.
Database projects for a warehouse in Git
The following image is an example of the file structure of each warehouse item in the repo:
When you commit the warehouse item to the Git repo, the warehouse is converted to a source code format, as a SQL database project. A SQL project is a local representation of SQL objects that comprise the schema for a single database, such as tables, stored procedures, or functions. The folder structure of the database objects is organized by Schema/Object Type. Each object in the warehouse is represented with a .sql file that contains its data definition language (DDL) definition. Warehouse table data and SQL security features aren't included in the SQL database project.
Shared queries are also committed to the repo and inherit the name that they are saved as.
Deployment pipelines
You can also use deployment pipelines to deploy your warehouse code across different environments, such as development, test, and production. Deployment pipelines don't expose a database project.
Use the following steps to complete your warehouse deployment by using the deployment pipeline.
- Create a new deployment pipeline or open an existing deployment pipeline. For more information, see Get started with deployment pipelines.
- Assign workspaces to different stages according to your deployment goals.
- Select, view, and compare items, including warehouses, between different stages, as shown in the following example.
- Select Deploy to deploy your warehouses across the Development, Test, and Production stages.
For more information about the Fabric deployment pipelines process, see Introduction to deployment pipelines.
Limitations in source control
- You must export or migrate SQL security features by using a script-based approach. Consider using a post-deployment script in a SQL database project. You can configure this script by opening the project with the SQL Database Projects extension available in Visual Studio Code.
Limitations in Git integration
- Currently, if you use
ALTER TABLEto add a constraint or column in the database project, the deployment process drops and recreates the table, which results in data loss. To preserve the table definition and data, consider the following workaround:- Create a new copy of the table in the warehouse by using
CREATE TABLEandINSERT,CREATE TABLE AS SELECT, or Clone table. - Modify the new table definition with new constraints or columns, as desired, by using
ALTER TABLE. - Delete the old table.
- Rename the new table to the name of the old table by using sp_rename.
- Modify the definition of the old table in the SQL database project in the exact same way. The SQL database project of the warehouse in source control and the live warehouse should now match.
- Create a new copy of the table in the warehouse by using
- Currently, don't create a Dataflow Gen2 with an output destination to the warehouse. A new item named
DataflowsStagingWarehouseappears in the repository and blocks committing and updating from Git. - Fabric Git integration doesn't support the SQL analytics endpoint item.
- Cross item dependencies, item sequencing, and synchronization gaps between the SQL analytics endpoint and warehouse impact the "branching out to a new or existing workspace" and "switching to a different branch" workflows during development and continuous integration.
Limitations for deployment pipelines
- Currently, if you use
ALTER TABLEto add a constraint or column in the database project, the deployment process drops and recreates the table, which results in data loss. - Currently, don't create a Dataflow Gen2 with an output destination to the warehouse. A new item named
DataflowsStagingWarehouseappears in the deployment pipeline and blocks deployment. - Fabric Deployment pipelines don't support the SQL analytics endpoint item.
- Cross item dependencies, item sequencing, and synchronization gaps between the SQL analytics endpoint and warehouse impact Fabric Deployment Pipelines workflows.
Unsupported scenarios
The following CI/CD workflows aren't officially supported when warehouses in different workspaces have different collations. Even though these operations might succeed without errors, they can result in metadata errors.
In all of these scenarios, if a collation mismatch occurs, use the Python script scripts/dw-collation-error-update-tmsl/pbi_interactive.py in the Fabric toolbox GitHub repository to update the dataset (TMSL) collation to match the warehouse collation.
| Scenario | Description | Risk |
|---|---|---|
| Deployment pipelines | Promoting warehouse content through pipeline stages (for example, Dev → Test → Prod) where the target warehouse was created with a different collation than the source isn't supported. | Deployment might succeed, but the dataset collation isn't updated to match the target warehouse collation. |
| Branching out to a new or existing workspace | Using Git integration to branch out from an existing workspace to a new or existing workspace where the warehouse has a different collation isn't supported. | Warehouse content is synced, but the collation metadata isn't reconciled. |
| Switching branches on a workspace | Switching to a branch that was associated with a warehouse of a different collation on a Git-connected workspace isn't supported. | Synced content might carry over collation assumptions that don't match the current warehouse. |
| Merging changes between workspaces through branches | Merging Git branches across workspaces where the warehouses have different collations isn't supported. | Merge might succeed at the Git level, but the resulting dataset collation doesn't reflect the target warehouse's collation. |