Hi @ashish ranjan ,
Thank you for posting query in Microsoft Q&A Platform.
Below are the detailed steps of implementation, that helps to compare source and sink file names and filter out missing files and then loop them to copy.
Here we are using two get meta data activities to get file names from source and sink locations. After that we are using filter activity to filter the file names which are missing in sink location. Writing expression correctly in Filter activity is important thing here. Once we filter the file names then we can use for each activity to loop them and inside for each activity use copy activity to copy file to destination.
Below are the settings used in Filter activity to filter file names which are missing in sink location.
Hope this helps. Please let us know if any further queries.
Please consider hitting
Accept Answer button. Accepted answers help community as well.
@Nandan Hegde Hi There,
Reason is that the amount of data we have is huge around 1 TB so going for full load everyday will take huge time.
So thats why we are searching for solution like file is not available in Destination then those files will proceed and copy activity happen.
Hi @ShaikMaheer-MSFT we have one issue here instead of checking metadata in destination folder...can we compare with list of values present in one sql table so thay we can compare filename with ADLS folder to SQL FileName column???