Issue with copy data from Azure Synapse Link for Dataverse using ADF Dataflow
Hi there,
We use Export to Data Lake option to copy the data from D365 and want to transform to Azure Synapse Link for Dataverse because Export to Data Lake option is obsoleting from 1st Nov 2024.
We have configured Azure Synapse link for Dataverse in UAT environment as per below screenshot.
It has created the new container with different timestamp folders.
Above folders contain tables and data information.
I have created a data pipeline in ADF to get the custtable data using dataflow and give the path till highlighted in above screenshot. It works fine till here but as I make changes in D365, a new folder with different timestamp created with the data and modify rows only as can see in below screenshot.
I want to get custtable or any specific entity data using ADF Dataflow but here the issue is I need to define the full path till timestamp and this timestamp folder create whenever any update happen in the entity, so this is not the fix path. How I get the custtable data everyday but how to know the path?
here is the Dataflow code: