Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,365 questions
Azure Synapse: Data Flow Failed
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(246.4, 55.00000000000001%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
Pravin007
1
Reputation point
Hi,
We have developed a data flow which reads parquet files from ADLS, and then, after a little bit of transformation, it stores them in ADLS and Azure SQL. The data flow has been running successfully but occasionally fails with the following error message.
Even though the file is present in ADLS, the data flow still fails. If I rerun the same failed data flow, it runs successfully.
Azure Synapse Analytics
{count} votes
-
PRADEEPCHEEKATLA 90,631 Reputation points Moderator2022-05-10T08:52:17.28+00:00 Hello @Pravin007 ,
Thanks for the question and using MS Q&A platform.
How often are you experiencing this error message?
This could be an intermittent issue. I would suggest you re-try again and check if that works.
In case if you getting this error message quite often, please do let me know.
-
Pravin007 1 Reputation point
2022-05-10T09:37:21.733+00:00 Hi @PRADEEPCHEEKATLA ,
This issue occurs every day or every couple of days. Retrying the activity after a while works, but pipelines are scheduled to run every two hours and this data flow activity failure affects pipeline automatic execution and every time it fails it requires manual intervention to rerun.
So can you please provide us any permanent solution on this?
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(224.00000000000003, 77%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERA%3C/text%3E%3C/svg%3E)
Ryan Abbey 1,186 Reputation points2022-05-11T03:08:39.333+00:00 What's the process for provisioning the source file? Does this data flow do any checks on what it should process?
The most likely issue is that the file doesn't exist at the time of process but is subsequently available when you go to re-run, so is the source file deleted/moved at any point prior to being reloaded?
Sign in to comment
Sign in to answer