Hi @Anmol Ganju , Please go through below posts, it addresses your ask. Thanks
https://stackoverflow.com/questions/70667287/how-to-load-only-newly-added-file-in-azure-sql-db
This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
Hi, I have a folder structure in ADLS Gen2 in container source that goes as below:
/table1/2021/01/20/file1.parquet
/table1/2021/02/20/file2.parquet
/table1/2022/01/10/file3.parquet
/table1/2022/01/12/file4.parquet
/table2/2021/01/12/file5.parquet
.
.
.
.
Like wise I have multiple tables..
I want to copy the content of latest file as per current datetime to the Azure SQL Database, How this can be achived, we are looking for only copying the latest file to azuresqldatabase.
Hi @Anmol Ganju , Please go through below posts, it addresses your ask. Thanks
https://stackoverflow.com/questions/70667287/how-to-load-only-newly-added-file-in-azure-sql-db
Hi @Anmol Ganju ,
Thank you for posting query in Microsoft Q&A Platform.
You can use GetMetaData activity to get Last Modified Date Time vale of file. So you should consider using it for all your files and then try to compare each other then finally filter our latest file to copy.
Similar kind of logic implemented in below video. Kindly check that to get an idea of doing it.
https://www.youtube.com/watch?v=QHeG36oqkPQ
Hope this will help. Please let me know if any concerns.
---------------------
Please consider hitting Accept Answer
button. Accepted answers helps community as well.