how to log a list of files passed to delete activity into a new file stating if file is deleted or not

Gulhasan.Siddiquee 101 Reputation points

I have 100 files in a container ,I want to delete only 10 files . So I used filter activity ,as output of filter activity I am getting 12 files names . That 12 list of files I am passing to deletes activity and put last modified date condition over there so only 10 files are getting deleted .

Now I want to log these 12 files name in single log file stating which files are deleted and which are not.

I tried to use "logging setting" of delete activity but it is creating multiple log files, one for each file name passed to delete activity. I think this is because I am using delete activity inside for each.

Here is snippet of my pipeline.

User's image

Azure SQL Database
Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
2,315 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
3,828 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
8,535 questions
Microsoft Entra ID
Microsoft Entra ID
A Microsoft Entra identity service that provides identity management and access control capabilities. Replaces Azure Active Directory.
17,612 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. AnnuKumari-MSFT 28,091 Reputation points Microsoft Employee

    Hi Gulhasan.Siddiquee ,

    Thankyou for using Microsoft Q&A platform and thanks for posting your question here.

    As I understand your query, you are trying to delete only few selected files from the folder. Please let me know if my understanding is incorrect about the query.

    Instead of iterating through the files that you want to delete, you can either use 'wildcard file pattern' to delete the files in case your filenames which you want to delete shares a common pattern , or else, if the filenames doesn't have any matching pattern, you can use 'list of files' option where you need to point to a file which contains all the filenames you want to delete:

    User's image

    For implementation, kindly checkout the below video where it's about copying the files, but same logic goes with deleting as well:

    Wildcard filters in copy activity of ADF

    List of files in copy activity of adf

    Hope it helps. Kindly accept the answer if it's helpful. Thankyou