Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
This article explains how deployment pipeline integration works for Spark job definitions (SJDs) in Microsoft Fabric. Learn how to use deployment pipelines to deploy your SJDs across different environments/workspaces and synchronize changes between them.
Important
This feature is in preview.
Deploy a Spark job definition
Complete the following steps to deploy SJDs using deployment pipelines:
Sign in to the Fabric portal.
Create a new deployment pipeline or open an existing one. For more information, see Get started with deployment pipelines.
Assign workspaces to different stages based on your deployment needs.
Select items to deploy, including SJDs. Use the Compared to source column to identify differences between source and target stages.
Select Deploy to deploy selected items to the target stage.
Optionally, you can create a deployment rule to overwrite the default bindings between the SJD and other items in the target stage.
If all the items are deployed, the bindings between the SJD and other items are automatically created in the target stage. However, if the lakehouse isn't deployed, the SJD links to the source stage lakehouse. To modify this behavior, create a new deployment rule. Select Deployment rules, then select the SJD item to create a rule.
You can create separate deployment rules for the default lakehouse and additional lakehouses. Choose from the three options: Same as source Lakehouse, N/A (No lakehouse), or other to manually enter the lakehouse information.
To overwrite the default binding, provide the LakehouseId, LakehouseName, and the WorkspaceId for the target lakehouse. Make sure the LakehouseId, LakehouseName, and LakehouseWorksapceId values correspond to each other. You can get the LakehouseId and LakehouseWorkspaceId values from the item URL link.
Once the deployment rule is updated, rerun the deployment to apply the changes.
