Want to restrict the snapshot files created automatically in azure storage accoutn when adf pipeline runs .

Question

Want to restrict the snapshot files created automatically in azure storage accoutn when adf pipeline runs .

Amar Agnihotri 891

Hi,

First of all sorry for not posting a clear question about the issue because i was not able to understand how to frame this question. Actually when my adf copy pipeline runs, some files in storage account get created automatically and when i saw those files i found that these are some kind of snapshot files that are generating automatically. I want to restrict those files to be created automatically from next run. Please tell me the way to stop that.

These are the files -

and when i clicked i found this inside -

This is the data inside files -

Thanks

Pratik Somaiya 4,126 Reputation points

2022-04-04T10:54:32.393+00:00

Hello @Amar Agnihotri : Can you describe what the Copy activity is trying to achieve, what are the source and sink etc?
Amar Agnihotri 891 Reputation points

2022-04-04T12:08:15.12+00:00

Hi @Pratik Somaiya

here is the pipeline .

There are 2 containers in blob. rapport and Rapport-archive. There is a rapport file in aws s3 bucket . Now the workflow of pipeline is -
It copies old file from rapport to rapport-archive with yesterdáy's date appended to the file name . Noiw it deleted all the files from rapport container . Now it copies new rapport file from aws s3 bucket to rapport folder. The pipeline is scheduled by a trigger and i found that whenever it runs some random files are generated in the storage account which has already been mentioned.

Hope you understand the flow now ,
Amar Agnihotri 891 Reputation points

2022-04-04T12:13:07.88+00:00

If you can see in the snap. These are actually containers which are being created automatically and inside the container there is a csv file.

Pratik Somaiya 4,126 Reputation points

2022-04-04T10:54:32.393+00:00

Hello @Amar Agnihotri : Can you describe what the Copy activity is trying to achieve, what are the source and sink etc?
Amar Agnihotri 891 Reputation points

2022-04-04T12:08:15.12+00:00

Hi @Pratik Somaiya

here is the pipeline .

There are 2 containers in blob. rapport and Rapport-archive. There is a rapport file in aws s3 bucket . Now the workflow of pipeline is -
It copies old file from rapport to rapport-archive with yesterdáy's date appended to the file name . Noiw it deleted all the files from rapport container . Now it copies new rapport file from aws s3 bucket to rapport folder. The pipeline is scheduled by a trigger and i found that whenever it runs some random files are generated in the storage account which has already been mentioned.

Hope you understand the flow now ,
Amar Agnihotri 891 Reputation points

2022-04-04T12:13:07.88+00:00

If you can see in the snap. These are actually containers which are being created automatically and inside the container there is a csv file.

Answer 1

ShaikMaheer-MSFT 35,796 Microsoft Employee

Hi @Amar Agnihotri ,

Thank you for posting query in Microsoft Q&A Platform.

From the screenshots I could see these are log files which generated to keep track of deleted files audit. Kindly check your delete activity configuration, it seems logging is enabled in Delete activity. Please remove that logging to avoid this.

Hope this helps. Please let us know if any further queries.

-----------

Please consider hitting Accept Answer button. Accepted answers help community as well.

Amar Agnihotri 891 Reputation points

2022-04-07T07:15:09.117+00:00

@ShaikMaheer-MSFT Thanks

Want to restrict the snapshot files created automatically in azure storage accoutn when adf pipeline runs .

0 additional answers

Activity