Obvious behavior of dynamic mapping in ADF

Bens Bernard 0 Reputation points
2024-05-29T06:44:25.6133333+00:00

I have a single copy activity that moves data from a source parquet file to an azure sql database. When I used auto mapping and tried to change the name of source or sink field it failed, however when I copy the json mapping generated and change one of the fields in the json string and use dynamic mapping @json(myjsonmapping), the activity succeeded.

Why this unexpected behabior.

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,891 questions
{count} votes

2 answers

Sort by: Most helpful
  1. Deleted

    This answer has been deleted due to a violation of our Code of Conduct. The answer was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.


    Comments have been turned off. Learn more

  2. BhargavaGunnam-MSFT 28,111 Reputation points Microsoft Employee
    2024-05-29T18:57:46.0166667+00:00

    Hello Bens Bernard,

    Welcome to the Microsoft Q&A forum.

    From your issue, it seems like auto mapping attempts to map source and sink fields by names and data types, but if the names don't match or if you're trying to change the field names, it may fail because it can't find a corresponding field in the destination.

    When you use dynamic content with @json(myjsonmapping), you're providing a JSON string that explicitly defines the mapping, which allows for more flexibility and control over the field names and how they're mapped. This can be particularly useful when the source and sink schema do not match or when you need to transform the data as it moves from source to sink.

    Please see the below reference documents:

    https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-schema-and-type-mapping

    https://stackoverflow.com/questions/51412683/azure-data-factory-copy-activity-failed-mapping-strings-from-csv-to-azure-sql

    I hope this helps. Please let me know if you have any further questions.

    0 comments No comments