Issues with Automatic CSV Uploads to Databricks Volume Using dbxservice
Anonymous
I initially created an import folder manually, which successfully stored CSV files as volumes in Databricks. After deleting this manually created import folder, I expected the pipeline to automatically recognize the new import folder created during the pipeline execution and process the CSV files as artifacts. However, the pipeline is not uploading the CSV files to the Databricks volume as expected. Additionally, I encountered an error related to an undefined variable job_mi_id when trying to use the Databricks CLI command to copy files.
Sign in to answer