Extra files in each folder when saving file from azure databricks to blob

Question

Hi,
I'm using databricks to mount to my blob storage with this function:

dbutils.fs.mount()

So I am fetching data and saving data from/to blob. I have partitioned my path so the path looks like <container_name>/year/month/day/<blob_file>. When I save my csv file it creates additional files in my partitions, that is /year/month/day. Below is a snapshot of how it looks like in folder month:

Why is it creating those extra files and is it possible to avoid these additional files?

Answer 1

Hi @Alexis Kan ,

Thank you for posting query in Microsoft Q&A Platform.

Are you using blob end point here to mount? Please note, if we use blob then its expected behavior. Kindly check below link. Please share your code to understand more and try same at my end to help better.
https://stackoverflow.com/questions/52872654/why-is-an-empty-file-with-the-name-of-folder-inside-a-azure-blob-storage-contain