error while flattening json using data flows in synapse

Question

I am facing error while reading json and performing data flattening using data flow in azure synapse.

Error json_flattening {"message":"Job failed due to reason: at Sink 'sink1': Failed to execute dataflow with internal server error, please retry later. If issue persists, please contact Microsoft support for further assistance. Details:org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 1.0 failed 1 times, most recent failure: Lost task 0.0 in stage 1.0 (TID 4) (vm-0e533693 executor 1): ExecutorLostFailure (executor 1 exited caused by one of the running tasks) Reason: Container from a bad node: container_1722209626697_0001_01_000002 on host: vm-0e533693. Exit status: 143. Diagnostics: [2024-07-28 23:44:12.016]Container killed on request. Exit code is 143\n[2024-07-28 23:44:12.017]Container exited with a non-zero exit code 143. \n[2024-07-28 23:44:12.017]Killed by external signal\n.\nDriver stacktrace:\n\tat com.microsoft.dataflow.FileStoreExceptionHandler$.extractRootCause(FileStoreExceptionHandler.scala:36)\n\tat com.microsoft.dataflow.transformers.DefaultFileWriter.$anonfun$write$16(FileStore.scala:1203)\n\tat scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)\n\tat scala.util.Try$.apply(Try.scala:213)\n\tat com.microsoft.dataflow.transformers.DefaultFileWriter.$anonfun$write$15(FileStore.scala:1175)\n\t","failureType":"UserError","target":"json_flattening","errorCode":"DF-Executor-InternalServerError"}

Answer 1

Hi @Anand

Thanks for the question and using MS Q&A platform.

As I understand that you are facing an error message while flattening JSON data in your Azure Synapse Data Flow.

Please refer to the below similar thread that might help you in resolving your issue.

Thread link: https://learn.microsoft.com/en-us/answers/questions/1161175/df-executor-internalservererror-error-in-mapping-d

If the issue still persists, please do let us know. Glad to help.

I hope this information helps.