Pipeline fails with DFExecutorUserError: None.get error

anand 1 Reputation point
2022-12-22T05:17:28.567+00:00

I have created a Dataflow which takes data from CSV file from BlobStorage and insert into a table into Dataverse. While debugging it I am getting following error
Error: DFExecutorUserError
Job failed due to reason: None.get

Here is detailed error
Operation on target Data flow1 failed: {"StatusCode":"DFExecutorUserError","Message":"Job failed due to reason: None.get","Details":"java.util.NoSuchElementException: None.get\n\tat scala.None$.get(Option.scala:347)\n\tat scala.None$.get(Option.scala:345)\n\tat com.microsoft.datafactory.dataflow.MetricsUtility$$anonfun$getMetricsForSink$5.apply(MetricsUtility.scala:307)\n\tat com.microsoft.datafactory.dataflow.MetricsUtility$$anonfun$getMetricsForSink$5.apply(MetricsUtility.scala:296)\n\tat scala.collection.immutable.List.foreach(List.scala:392)\n\tat com.microsoft.datafactory.dataflow.MetricsUtility$.getMetricsForSink(MetricsUtility.scala:296)\n\tat com.microsoft.datafactory.dataflow.MetricsUtility$.metricsToPayload(MetricsUtility.scala:464)\n\tat com.microsoft.datafactory.dataflow.AdmsClient.getMonitoringPayloadInternal(AdmsClient.scala:485)\n\tat com.microsoft.datafactory.dataflow.AdmsEventListener$$anon$1$$anonfun$run$1$$anonfun$apply$mcV$sp$8$$anonfun$apply$1.apply(AdmsEventListener.scala:90)\n\tat com.microsoft.datafactory.dataflow.AdmsEventListener$$anon$1$$anonfun$run$1$$anonfun$apply$mcV$sp$8$$anonfun$apply$1.apply(AdmsEventListener.scala:90)\n\tat sca"}

I also tried to export data as CSV from Sink and import into Dataverse , which is working fine. Same data I am not able to insert via pipeline.
Also not getting any relevant information regarding the error.

Failed Pipeline RunId: 75ffbe2f-eb38-4920-94a1-1e9c26a70c6d
Azure Data Factory Region : West Europe

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,487 questions
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. BhargavaGunnam-MSFT 25,881 Reputation points Microsoft Employee
    2022-12-22T19:38:22.527+00:00

    Hello @anand ,

    Welcome to the MS Q&A platform.

    A similar issue has been discussed here.

    The error message you are getting could be related to a schema mismatch. Can you please follow the below steps?

    1)Reimport the projections from the source.

    2) Check if all datatypes in the sink are the same as the destination.

    I hope this helps. Please let us know if you still have any issues