Azure Synapse DataFlow - NoSuchElementException

TanjaM 1 Reputation point
2023-01-02T15:41:18.51+00:00

Hello all,

lately we are facing issues in Azure Synapse when running pipelines with data flows, that have previously run without problems. The pipelines run for the expected time and then fail at the sink, throwing the error java.util.NoSuchElementException: None.get.

Any ideas?

Whole message:
{"message":"Job failed due to reason: None.get. Details:java.util.NoSuchElementException: None.get\n\tat scala.None$.get(Option.scala:347)\n\tat scala.None$.get(Option.scala:345)\n\tat com.microsoft.datafactory.dataflow.MetricsUtility$$anonfun$getMetricsForSink$5$$anonfun$apply$18.apply(MetricsUtility.scala:345)\n\tat com.microsoft.datafactory.dataflow.MetricsUtility$$anonfun$getMetricsForSink$5$$anonfun$apply$18.apply(MetricsUtility.scala:342)\n\tat scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:130)\n\tat scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:130)\n\tat scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:236)\n\tat scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)\n\tat scala.collection.mutable.HashMap.foreach(HashMap.scala:130)\n\tat com.microsoft.datafactory.dataflow.MetricsUtility$$anonfun$getMetricsForSink$5.apply(MetricsUtility.scala:342)\n\tat com.microsoft.datafactory.dataflow.MetricsUtility$$anonfun$getMetricsForSink$5.apply(MetricsUtility.scala:296)\n\tat scala.collection.immutable.List.","failureType":"UserError","target":"XXX","errorCode":"DFExecutorUserError"}

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,175 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,219 questions
{count} votes

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.