Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,639 questions
MY Data Flow is failing by saying 'The spark job of Dataflow completed, but the runtime state is either null or still InProgress.' Can some one please help me to resolve this ?
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(198.4, 96%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKG%3C/text%3E%3C/svg%3E)
Kandi, Ganesh
5
Reputation points
Azure Data Factory
{count} votes
-
Kandi, Ganesh 5 Reputation points
2023-03-03T09:48:51.2766667+00:00 @Amira Bedhiafi Thanks for your reply. I don't see any errors and warnings in the monitor . is there any other reason for this could cause my data flow execution error ?
Appreciated your efforts in advance.
Thanks,
Ganesh Kandi
-
Amira Bedhiafi 33,631 Reputation points Volunteer Moderator2023-03-03T14:57:23.58+00:00 What is the status of your job in Spark UI ?
-
Amira Bedhiafi 33,631 Reputation points Volunteer Moderator
2023-03-07T00:16:41.5566667+00:00 Try to check the Data Flow activity's monitoring page in Azure Data Factory and see if there are any error messages or warnings that may provide additional information about the issue(memory , resource constraints...)
Verify also with the Spark logs through the Azure portal or using the Azure Storage Explorer.
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(134.4, 9%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKM%3C/text%3E%3C/svg%3E)
KranthiPakala-MSFT 46,642 Reputation points Microsoft Employee Moderator2023-03-07T00:18:34.9933333+00:00 @Kandi, Ganesh - If you are still facing the issue, could you please share failed data flow activity ID as well as failed pipeline runID? Also please confirm what is the IR region if it is custom Azure IR and also what is the ADF region?
Are you noticing this issue with debug run or trigger run?Thank you
-
Kandi, Ganesh 5 Reputation points
2023-03-07T11:36:56.69+00:00 Hello @KranthiPakala-MSFT
Thanks for your reply.
As there are some security reason , I can not share the required details. this issues is coming through the debug run.
If there is any chance of connecting through teams, please share your email ID . so that I can schedule some time to discuss on this .
Thanks in advance,
Ganesh Kandi
-
KranthiPakala-MSFT 46,642 Reputation points Microsoft Employee Moderator
2023-03-17T06:18:14.49+00:00 @Kandi, Ganesh Thanks for your response. As the information cannot be shared here, I recommend filing a support ticket for deeper investigation on this problem. Filing a support ticket will help support engineer to have a call with you and and take a deeper look into the required logs. In case fi you don't have a support plan, please let me know
-
KranthiPakala-MSFT 46,642 Reputation points Microsoft Employee Moderator
2023-03-20T18:29:17.35+00:00 @Kandi, Ganesh Following up to see if you have got a chance to file a support ticket for deeper investigation. If so, could you please share the SR# with us here please.
Thank you
Sign in to comment
Sign in to answer