Column mapping in Copy Activity

Question

Hi all,

I facing a problem ,in copy activity i have 100 columns coming from source side amd 200 columns from sink source side , i need to map only 100 columns , but when i import then i need to delete those columns which left unmapped plus add to this i every week or day i need to add some columns then i again i need to do the same thing import and delete those who left unmapped so its a very length process. Kind of a headache....

any idea how to overcome this plus how to main consistency when i am mapping more columns their sequence etc things. ,even for code reviewer its should be easy to check mapping of source to sink (less time should consume)

i will be very happing if you can help with any solid thing coz i need to use this thing in actual prod. env

currently mapping is a very length process when i add even 1 column it's becomes frustrating.

Thanks, in advance.

Answer 1

Hi Shreyash Choudhary,

Thank you for posting query in Microsoft Q&A Platform.

If column names are exactly same in source and sink they get auto map. But usually, its rare case to have such incident. I would recommend you have mappings dynamically by saving your mapping information into some file or table. That way you no need to do manually mapping again and again. Mapping will be taken dynamically from your mapping file or table.

In below video, I explained how to have dynamic mappings. Kindly follow same.

Dynamic Column mapping in Copy Activity in Azure Data Factory

Hope this helps. Please let me know if any further queries.

---

Please consider hitting Accept Answer button. Accepted Answers help community as well.