Obvious behavior of dynamic mapping in ADF

Question

I have a single copy activity that moves data from a source parquet file to an azure sql database. When I used auto mapping and tried to change the name of source or sink field it failed, however when I copy the json mapping generated and change one of the fields in the json string and use dynamic mapping @json(myjsonmapping), the activity succeeded.

Why this unexpected behabior.

Answer 1

Hello Bens Bernard,

Welcome to the Microsoft Q&A forum.

From your issue, it seems like auto mapping attempts to map source and sink fields by names and data types, but if the names don't match or if you're trying to change the field names, it may fail because it can't find a corresponding field in the destination.

When you use dynamic content with @json(myjsonmapping), you're providing a JSON string that explicitly defines the mapping, which allows for more flexibility and control over the field names and how they're mapped. This can be particularly useful when the source and sink schema do not match or when you need to transform the data as it moves from source to sink.

Please see the below reference documents:

https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-schema-and-type-mapping

https://stackoverflow.com/questions/51412683/azure-data-factory-copy-activity-failed-mapping-strings-from-csv-to-azure-sql

I hope this helps. Please let me know if you have any further questions.