Azure Synapse Mapping Dataflow - Job failed due to reason: Not started.

Question

Azure Synapse Mapping Dataflow - Job failed due to reason: Not started.

Hello,

since yesterday we've got an issue with two mapping Data Flows in Azure Synapse:

{"message":"Job failed due to reason: Not started. Details:java.lang.Throwable: Not started\n\tat com.microsoft.dataflow.FlowCode$$anonfun$run$1$$anonfun$1.apply$mcV$sp(FlowRunner.scala:175)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$run$1$$anonfun$1.apply(FlowRunner.scala:168)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$run$1$$anonfun$1.apply(FlowRunner.scala:168)\n\tat scala.util.Try$.apply(Try.scala:192)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$run$1.apply$mcV$sp(FlowRunner.scala:168)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$run$1.apply(FlowRunner.scala:160)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$run$1.apply(FlowRunner.scala:160)\n\tat scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)\n\tat scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)\n\tat scala.concurrent.impl.ExecutionContextImpl$AdaptedForkJoinTask.exec(ExecutionContextImpl.scala:121)\n\tat scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)\n\tat scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:133","failureType":"UserError","target":"Data flow1","errorCode":"DFExecutorUserError"}

Is there any issue with the debug clusters or something like that? Also from time to time the Pipes and Dataflow just don't start and Queue forever.

BhargavaGunnam-MSFT 10,411 Reputation points Microsoft Employee

2022-08-15T18:22:19.313+00:00

Hello @Christopher Geckert ,

Thanks for the question and using MS Q&A platform.

Could you please share the pipeline run ids for this issue?
BhargavaGunnam-MSFT 10,411 Reputation points Microsoft Employee

2022-08-16T22:48:36.06+00:00

Hello @Christopher Geckert ,
The product team identified the problem, and they are working on it. I will keep you posted once the fix is applied.
Thank you for being so patient.
Christopher Geckert 91 Reputation points

2022-08-17T08:07:54.84+00:00

Hello,
@BhargavaGunnam-MSFT did you already change something? We now encounter:
{"message":"Job failed due to reason: None.get. Details:java.util.NoSuchElementException: None.get\n\tat scala.None$.get(Option.scala:347)\n\tat scala.None$.get(Option.scala:345)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$com$microsoft$dataflow$FlowCode$$recurseLineageNode$1.apply(FlowRunner.scala:517)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$com$microsoft$dataflow$FlowCode$$recurseLineageNode$1.apply(FlowRunner.scala:515)\n\tat scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)\n\tat scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)\n\tat com.microsoft.dataflow.FlowCode.com$microsoft$dataflow$FlowCode$$recurseLineageNode(FlowRunner.scala:515)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$com$microsoft$dataflow$FlowCode$$recurseLineageNode$1.apply(FlowRunner.scala:518)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$com$microsoft$dataflow$FlowCode$$recurseLineageNode$1.apply(FlowRunner.scala:515)\n\tat scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)\n\tat scala.collection.mutable.ArrayBuffer.foreach(A","failureType":"UserError","target":"Data flow1","errorCode":"DFExecutorUserError"}

Pipeline Run ID: fbba4dcf-ce6b-4ffe-ad30-ab745bb311d8
Dataflow RunID: 993b49e2-6fa5-4e35-94ba-273714bcb527
Sandra Aerts 6 Reputation points

2022-09-15T11:18:34.47+00:00

Is this the same issue? Today in my Data Factory:

failed: {"StatusCode":"DFExecutorUserError","Message":"Job failed due to reason: None.get","Details":"java.util.NoSuchElementException: None.get\n\tat scala.None$.get(Option.scala:347)\n\tat scala.None$.get(Option.scala:345)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$com$microsoft$dataflow$FlowCode$$recurseLineageNode$1.apply(FlowRunner.scala:547)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$com$microsoft$dataflow$FlowCode$$recurseLineageNode$1.apply(FlowRunner.scala:545)\n\tat scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)\n\tat scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)\n\tat com.microsoft.dataflow.FlowCode.com$microsoft$dataflow$FlowCode$$recurseLineageNode(FlowRunner.scala:545)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$com$microsoft$dataflow$FlowCode$$recurseLineageNode$1.apply(FlowRunner.scala:548)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$com$microsoft$dataflow$FlowCode$$recurseLineageNode$1.apply(FlowRunner.scala:545)\n\tat scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)\n\tat scala.collection.mutable.ArrayBuffer.foreach(A"}

The underlying data flow succeeded. The pipeline holds nothing else, just this 1 dataflow. Debug mode.
BhargavaGunnam-MSFT 10,411 Reputation points Microsoft Employee

2022-09-15T21:19:20.563+00:00

Hello @Sandra Aerts ,
Sorry, This is not the same error. The product team fixed the issue of the error message: ""message":"Job failed due to reason: Not started"

BhargavaGunnam-MSFT 10,411 Reputation points Microsoft Employee

2022-08-15T18:22:19.313+00:00

Hello @Christopher Geckert ,

Thanks for the question and using MS Q&A platform.

Could you please share the pipeline run ids for this issue?
BhargavaGunnam-MSFT 10,411 Reputation points Microsoft Employee

2022-08-16T22:48:36.06+00:00

Hello @Christopher Geckert ,
The product team identified the problem, and they are working on it. I will keep you posted once the fix is applied.
Thank you for being so patient.
Christopher Geckert 91 Reputation points

2022-08-17T08:07:54.84+00:00

Hello,
@BhargavaGunnam-MSFT did you already change something? We now encounter:
{"message":"Job failed due to reason: None.get. Details:java.util.NoSuchElementException: None.get\n\tat scala.None$.get(Option.scala:347)\n\tat scala.None$.get(Option.scala:345)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$com$microsoft$dataflow$FlowCode$$recurseLineageNode$1.apply(FlowRunner.scala:517)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$com$microsoft$dataflow$FlowCode$$recurseLineageNode$1.apply(FlowRunner.scala:515)\n\tat scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)\n\tat scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)\n\tat com.microsoft.dataflow.FlowCode.com$microsoft$dataflow$FlowCode$$recurseLineageNode(FlowRunner.scala:515)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$com$microsoft$dataflow$FlowCode$$recurseLineageNode$1.apply(FlowRunner.scala:518)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$com$microsoft$dataflow$FlowCode$$recurseLineageNode$1.apply(FlowRunner.scala:515)\n\tat scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)\n\tat scala.collection.mutable.ArrayBuffer.foreach(A","failureType":"UserError","target":"Data flow1","errorCode":"DFExecutorUserError"}

Pipeline Run ID: fbba4dcf-ce6b-4ffe-ad30-ab745bb311d8
Dataflow RunID: 993b49e2-6fa5-4e35-94ba-273714bcb527
Sandra Aerts 6 Reputation points

2022-09-15T11:18:34.47+00:00

Is this the same issue? Today in my Data Factory:

failed: {"StatusCode":"DFExecutorUserError","Message":"Job failed due to reason: None.get","Details":"java.util.NoSuchElementException: None.get\n\tat scala.None$.get(Option.scala:347)\n\tat scala.None$.get(Option.scala:345)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$com$microsoft$dataflow$FlowCode$$recurseLineageNode$1.apply(FlowRunner.scala:547)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$com$microsoft$dataflow$FlowCode$$recurseLineageNode$1.apply(FlowRunner.scala:545)\n\tat scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)\n\tat scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)\n\tat com.microsoft.dataflow.FlowCode.com$microsoft$dataflow$FlowCode$$recurseLineageNode(FlowRunner.scala:545)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$com$microsoft$dataflow$FlowCode$$recurseLineageNode$1.apply(FlowRunner.scala:548)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$com$microsoft$dataflow$FlowCode$$recurseLineageNode$1.apply(FlowRunner.scala:545)\n\tat scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)\n\tat scala.collection.mutable.ArrayBuffer.foreach(A"}

The underlying data flow succeeded. The pipeline holds nothing else, just this 1 dataflow. Debug mode.
BhargavaGunnam-MSFT 10,411 Reputation points Microsoft Employee

2022-09-15T21:19:20.563+00:00

Hello @Sandra Aerts ,
Sorry, This is not the same error. The product team fixed the issue of the error message: ""message":"Job failed due to reason: Not started"

Answer 1

BhargavaGunnam-MSFT 10,411 Microsoft Employee

Hello @Christopher Geckert ,

The product team identified the problem and they are working on it. I will keep you posted once the fix is applied.

Latest Update on Aug 17th, 2022: This issue is being hot fixed by the product team today. If you ever notice this issue, please feel free to let us know. Thank you for your patience.

BhargavaGunnam-MSFT 10,411 Reputation points Microsoft Employee

2022-08-20T03:06:22.39+00:00

Hello @Christopher Geckert ,

I just wanted to check if the issue is resolved and have any further questions. Please let me know.
Alessandro 62 Reputation points

2022-10-14T09:45:05.907+00:00

Details: Job failed due to reason: None.get

Dataflow is starting.
Data is written to sink correctly.
Dataflow throws error.
Alessandro 62 Reputation points

2022-10-14T12:10:16.367+00:00

Okay after hours of trying out what could be the problem, it seems like the "new" cast block is not working with an assert together, leading to the problem that sink to error path do not work correctly.
After switching the cast to derivedColumn, the not successful transformed data rows are written successfully to error path.
Seems like the cast operation is the problem.
https://learn.microsoft.com/en-us/azure/data-factory/data-flow-cast

Azure Synapse Mapping Dataflow - Job failed due to reason: Not started.

1 answer

Activity