This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(224.00000000000003, 28.999999999999996%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ED%3C/text%3E%3C/svg%3E)
Hello, I'm facing an issue that is driving me mad. I get a DFExecutionUserError None.get without any other information when trying to execute a pipeline in Live mode. This error doesn't happens when I run it outside the live mode or in debug mode.
The pipeline executes a DataFlow that filters some information from a datalake, modifies a column using a Derive Column action and then it sinks it to an API REST.
I have checked both this thread and this thread. I have checked and updated the schema from the source both in the Dataset and in the source. I have also checked as true the Validation Schema mark, so I'm quite sure that there is not a mismatch between the schema of the source and the expected schema. Since the sink is not a database, but an API, there is not an expected schema and the API is not receiving any request.
This is the complete error I receive:
{
"StatusCode": "DFExecutorUserError",
"Message": "Job failed due to reason: None.get",
"Details": "java.util.NoSuchElementException: None.get
at scala.None$.get(Option.scala:347)
at scala.None$.get(Option.scala:345)
at com.microsoft.datafactory.dataflow.MetricsUtility$$anonfun$getMetricsForSink$5.apply(MetricsUtility.scala:306)
at com.microsoft.datafactory.dataflow.MetricsUtility$$anonfun$getMetricsForSink$5.apply(MetricsUtility.scala:295)
at scala.collection.immutable.List.foreach(List.scala:392)
at com.microsoft.datafactory.dataflow.MetricsUtility$.getMetricsForSink(MetricsUtility.scala:295)
at com.microsoft.datafactory.dataflow.MetricsUtility$.metricsToPayload(MetricsUtility.scala:463)
at com.microsoft.datafactory.dataflow.AdmsClient.getMonitoringPayloadInternal(AdmsClient.scala:485)
at com.microsoft.datafactory.dataflow.AdmsEventListener$$anon$1$$anonfun$run$1$$anonfun$apply$mcV$sp$8$$anonfun$apply$1.apply(AdmsEventListener.scala:90)
at com.microsoft.datafactory.dataflow.AdmsEventListener$$anon$1$$anonfun$run$1$$anonfun$apply$mcV$sp$8$$anonfun$apply$1.apply(AdmsEventListener.scala:90)
at sca"
}
And this is the output from the run dataflow element in my pipeline:
{
"runStatus": {
"computeAcquisitionDuration": 194181,
"dsl": null,
"profile": null,
"metrics": null,
"executorVersion": "20221220.6"
},
"effectiveIntegrationRuntime": "AutoResolveIntegrationRuntime (West Europe)",
"billingReference": {
"activityType": "executedataflow",
"billableDuration": [
{
"meterType": "General",
"duration": 0.5193346866666667,
"unit": "coreHour",
"sessionType": "JobCluster"
}
]
},
"reportLineageToPurview": {
"status": "NotReported"
}
}
Thanks in advance for any help you may provide.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(220.8, 4%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESM%3C/text%3E%3C/svg%3E)
Hi @davidsg ,
Welcome to Microsoft Q&A platform and thanks for using Azure Services.
We are sorry about the inconvenience you are experiencing.
Please follow the below mentioned steps :
It could be an issue with the schema mismatch between sink and the database.
Follow the given best practices to avoid this issue:
Please let us know if this helped in your case. If not please share more details about Source, Sink, Transformations used so that we can reproduce the mapping and check.
Thank you!
None of that helped. I have already tried everyone of those suggestions and it is still not working.
I have simplified the pipeline to only a source and a sink and it still gives the same error in the same circunstances. Only in the live mode and only in the pipeline execution.
Hi @davidsg ,
Could you please share the image of the pipeline with configuration?
I will try to reproduce and check.
I would also recommend you to raise support ticket if this is very urgent and impacting your work.
Hi @davidsg ,
We haven't heard back from you. In case you already found a solution, would you please share it here with the community? Otherwise, let us know and we will continue to engage with you on the issue.
The problem is that I have simplified the pipeline to only a source and a sink as I already said and the error keeps happening.
Hi @davidsg ,
Thank for the reply.
I tried to reproduce and could not get the error.
I would recommend you to create a support ticket with Microsoft so that our product team can directly engage with you and provide solution after deeper investigation. Here is how you can do that: Create an Azure support request.
If you have a support plan you may file a support ticket to get to the root of this, else please let us know so that we may help you with other option to unblock you.
Just for the record. This problema was solved by the MS support after opening a ticket with them. The executorVersion that solved the problem was the 20230109.