Define artifact settings in Databricks Asset Bundles
This article describes how to override the settings for artifacts in Databricks Asset Bundles. See What are Databricks Asset Bundles?
In Azure Databricks bundle configuration files, you can override the artifact settings in a top-level artifacts
mapping with the artifact settings in a targets
mapping, for example:
# ...
artifacts:
<some-unique-programmatic-identifier-for-this-artifact>:
# Artifact settings.
targets:
<some-unique-programmatic-identifier-for-this-target>:
artifacts:
<the-matching-programmatic-identifier-for-this-artifact>:
# Any more artifact settings to join with the settings from the
# matching top-level artifacts mapping.
If any artifact setting is defined both in the top-level artifacts
mapping and the targets
mapping for the same artifact, then the setting in the targets
mapping takes precedence over the setting in the top-level artifacts
mapping.
To demonstrate how this works in practice, in the following example, path
is defined in the top-level artifacts
mapping, which defines all of the settings for the artifact (ellipses indicate omitted content, for brevity):
# ...
artifacts:
my-artifact:
type: whl
path: ./my_package
# ...
When you run databricks bundle validate
for this example, the resulting graph is:
{
"...": "...",
"artifacts": {
"my-artifact": {
"type": "whl",
"path": "./my_package",
"...": "..."
}
},
"...": "..."
}
In this example, path
is defined both in the top-level artifacts
mapping and in the artifacts
mapping in targets
. In this example, path
in the artifacts
mapping in targets
takes precedence over path
in the top-level artifacts
mapping, to define the settings for the artifact (ellipses indicate omitted content, for brevity):
# ...
artifacts:
my-artifact:
type: whl
path: ./my_package
targets:
dev:
artifacts:
my-artifact:
path: ./my_other_package
# ...
When you run databricks bundle validate
for this example, the resulting graph is (ellipses indicate omitted content, for brevity):
{
"...": "...",
"artifacts": {
"my-artifact": {
"type": "whl",
"path": "./my_other_package",
"...": "..."
}
},
"...": "..."
}