Hello,
since yesterday we've got an issue with two mapping Data Flows in Azure Synapse:
{"message":"Job failed due to reason: Not started. Details:java.lang.Throwable: Not started\n\tat com.microsoft.dataflow.FlowCode$$anonfun$run$1$$anonfun$1.apply$mcV$sp(FlowRunner.scala:175)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$run$1$$anonfun$1.apply(FlowRunner.scala:168)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$run$1$$anonfun$1.apply(FlowRunner.scala:168)\n\tat scala.util.Try$.apply(Try.scala:192)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$run$1.apply$mcV$sp(FlowRunner.scala:168)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$run$1.apply(FlowRunner.scala:160)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$run$1.apply(FlowRunner.scala:160)\n\tat scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)\n\tat scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)\n\tat scala.concurrent.impl.ExecutionContextImpl$AdaptedForkJoinTask.exec(ExecutionContextImpl.scala:121)\n\tat scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)\n\tat scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:133","failureType":"UserError","target":"Data flow1","errorCode":"DFExecutorUserError"}
Is there any issue with the debug clusters or something like that? Also from time to time the Pipes and Dataflow just don't start and Queue forever.
Hello @Christopher Geckert ,
The product team identified the problem, and they are working on it. I will keep you posted once the fix is applied.
Thank you for being so patient.
Hello,
@BhargavaGunnam-MSFT did you already change something? We now encounter:
{"message":"Job failed due to reason: None.get. Details:java.util.NoSuchElementException: None.get\n\tat scala.None$.get(Option.scala:347)\n\tat scala.None$.get(Option.scala:345)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$com$microsoft$dataflow$FlowCode$$recurseLineageNode$1.apply(FlowRunner.scala:517)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$com$microsoft$dataflow$FlowCode$$recurseLineageNode$1.apply(FlowRunner.scala:515)\n\tat scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)\n\tat scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)\n\tat com.microsoft.dataflow.FlowCode.com$microsoft$dataflow$FlowCode$$recurseLineageNode(FlowRunner.scala:515)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$com$microsoft$dataflow$FlowCode$$recurseLineageNode$1.apply(FlowRunner.scala:518)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$com$microsoft$dataflow$FlowCode$$recurseLineageNode$1.apply(FlowRunner.scala:515)\n\tat scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)\n\tat scala.collection.mutable.ArrayBuffer.foreach(A","failureType":"UserError","target":"Data flow1","errorCode":"DFExecutorUserError"}
Pipeline Run ID: fbba4dcf-ce6b-4ffe-ad30-ab745bb311d8
Dataflow RunID: 993b49e2-6fa5-4e35-94ba-273714bcb527
Is this the same issue? Today in my Data Factory:
failed: {"StatusCode":"DFExecutorUserError","Message":"Job failed due to reason: None.get","Details":"java.util.NoSuchElementException: None.get\n\tat scala.None$.get(Option.scala:347)\n\tat scala.None$.get(Option.scala:345)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$com$microsoft$dataflow$FlowCode$$recurseLineageNode$1.apply(FlowRunner.scala:547)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$com$microsoft$dataflow$FlowCode$$recurseLineageNode$1.apply(FlowRunner.scala:545)\n\tat scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)\n\tat scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)\n\tat com.microsoft.dataflow.FlowCode.com$microsoft$dataflow$FlowCode$$recurseLineageNode(FlowRunner.scala:545)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$com$microsoft$dataflow$FlowCode$$recurseLineageNode$1.apply(FlowRunner.scala:548)\n\tat com.microsoft.dataflow.FlowCode$$anonfun$com$microsoft$dataflow$FlowCode$$recurseLineageNode$1.apply(FlowRunner.scala:545)\n\tat scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)\n\tat scala.collection.mutable.ArrayBuffer.foreach(A"}
The underlying data flow succeeded. The pipeline holds nothing else, just this 1 dataflow. Debug mode.
Hello @Sandra Aerts ,
Sorry, This is not the same error. The product team fixed the issue of the error message: ""message":"Job failed due to reason: Not started"
Sign in to comment