Need some ADF Guidence on Mapping Please

Question

Hello! @Nasreen Akter @MartinJaffer-MSFT
We don't have Data Flows in the Government Cloud yet, sadly. An issue came up today where my Manager would like to know the following. IF we change our json file and add a new field in it such as

we just had to add hanfordID this morning to this json file. I use the Copy Activity and the json file as a Source and SQL DB as the sink. Ok, today I had to totally remap over 100 fields in in the json file just to get this field in the mapping; it was very time consuming. My Manager thinks surely ADF has a faster way to map when a json file (schema) changes instead of doing it over manually (to map to the sink). Is there a better way? I need to report this back to him. He is saying with Logic Apps it is transparent.

Thanks so much for your opinion...
Mike Kiser

Answer 1

Hi @Mike Kiser ,

In the ADF CopyActivity --> Mapping, if you hover over the source side, you will see "+" sign after each attributes. You will able to add array/object/node in any position. That way you can avoid remapping all the fields in the CopyActivity. Hope this helps! :)

Answer 2

If you are technically savvy, you could automate this process (home-brewed, not a part of Data Factory).

1. Write a script/function that can read the pipeline definition, locate the appropriate copy activity, insert new stuff into the mapping.
2. Get a list of pipelines / copy activities in need of update.
3. Write a script, that for each pipeline in the list, fetches the definition, uses the aforementioned function on the definition, then saves the altered definition back to the source (Data Factory / Git).
4. Publish changes and test.