Azure Synapse DataFlow - NoSuchElementException
Hello all,
lately we are facing issues in Azure Synapse when running pipelines with data flows, that have previously run without problems. The pipelines run for the expected time and then fail at the sink, throwing the error java.util.NoSuchElementException: None.get.
Any ideas?
Whole message:
{"message":"Job failed due to reason: None.get. Details:java.util.NoSuchElementException: None.get\n\tat scala.None$.get(Option.scala:347)\n\tat scala.None$.get(Option.scala:345)\n\tat com.microsoft.datafactory.dataflow.MetricsUtility$$anonfun$getMetricsForSink$5$$anonfun$apply$18.apply(MetricsUtility.scala:345)\n\tat com.microsoft.datafactory.dataflow.MetricsUtility$$anonfun$getMetricsForSink$5$$anonfun$apply$18.apply(MetricsUtility.scala:342)\n\tat scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:130)\n\tat scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:130)\n\tat scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:236)\n\tat scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)\n\tat scala.collection.mutable.HashMap.foreach(HashMap.scala:130)\n\tat com.microsoft.datafactory.dataflow.MetricsUtility$$anonfun$getMetricsForSink$5.apply(MetricsUtility.scala:342)\n\tat com.microsoft.datafactory.dataflow.MetricsUtility$$anonfun$getMetricsForSink$5.apply(MetricsUtility.scala:296)\n\tat scala.collection.immutable.List.","failureType":"UserError","target":"XXX","errorCode":"DFExecutorUserError"}