How to handle scenario of the file getting overwritten in a Blob Storage account

Litty Kuriakose 106 Reputation points
2022-11-13T17:51:39.893+00:00

I generated a SAS url for a container and given to an application to transfer files. Since the application cannot transfer files with dynamic name ,it transferred files with the same name of container. When the file is created in blob, the files are moved to another container with a filename (container name _last modified date of file in blob) using an ADF pipeline using blob create trigger to handle file overwrite issue. The application transferred 2 files with an interval difference of 1 sec .The first trigger job failed on the metadata activity saying the file doesnt exist ( I think overwrite within that time) .The second trigger went through and moved the file to the second container with correct name . So 1 file is missed in this transfer.

Please suggest a way to handle this issue

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,073 questions
{count} votes