Azure data pipeline fails when parsing array data from cosmos db

Francis Gesora 1 Reputation point

I use Aure data factory to run data pipelines from cosmosDB to mysql db. all datatypes work well, except for array data types which fail with the error message:

Operation on target Copy_XxxCollection failed: ErrorCode=UserErrorSchemaMappingCannotInferSinkColumnType,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Data type of column 'xxxx' can't be inferred from 1st row of data, please specify its data type in mappings of copy activity or structure of DataSet.,Source=Microsoft.DataTransfer.Common,'

I have enabled "map complex values to string" whcih doesnt seemt to fix. My arrays asically hold user defined values and the number of values in a single json object depends on the user - one useer could define 3 values, another could define 10 etc.

Is there any way to map the arrays successfully to a single sql field?

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
8,471 questions
{count} votes