Pipeline fails at data flow

Question

I have created a 'copy data' and 'data flow' and have added them to my pipeline.
When debugging the pipeline, copy data runs successfully, however, it fails at data flow with error:
{"message":"at Sink 'OutputStreamName': java.util.NoSuchElementException: None.get. Details:at Sink 'OutputStreamName': java.util.NoSuchElementException: None.get","failureType":"UserError","target":"DataFlowName","errorCode":"DFExecutorUserError"}
Few things I want to highlight,
the copy operation is from CosmosDB to SQLDB and both the datasets are schema-less.

Answer 1

Hi @Rose Kipgen ,

Welcome to Microsoft Q&A and thanks for reaching out.

We can use "Insert Explicit Structure" option in "Derived Column" transformation in Mapping dataflow that separates the fields from array in JSON and then they can be mapped to SQL columns in Sink as shown in GIF below.

Hope this helps! Do let us know for queries.