Share via

Issues with Automatic CSV Uploads to Databricks Volume Using dbxservice

Anonymous
2025-03-27T10:28:28.8966667+00:00

I initially created an import folder manually, which successfully stored CSV files as volumes in Databricks. After deleting this manually created import folder, I expected the pipeline to automatically recognize the new import folder created during the pipeline execution and process the CSV files as artifacts. However, the pipeline is not uploading the CSV files to the Databricks volume as expected. Additionally, I encountered an error related to an undefined variable job_mi_id when trying to use the Databricks CLI command to copy files.

0 comments No comments

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.