Hi 19B01A04F3 P ,
Welcome to Microsoft Q&A platform and thanks for posting your query here.
As per my understanding you are trying to fetch only few selected columns of csv, which are mentioned in a config table using ADF pipeline. Please let me know if my understanding is incorrect.
Kindly try any one of the below approaches to achieve the requirement:
- You can use mapping section of copy activity to customize the schema as per the requirement and deselect the unnecessary ones. If not manually, then you can create the json having the needed columns and use it for dynamic schema mapping. Here is more details on that: Dynamic Column mapping in Copy Activity in Azure Data Factory
- Copy the csv data into SQL table and write the desired select query removing the unwanted columns.
Hope it helps. Kindly accept the answer by clicking on Accept answer
button. Thankyou