ErrorCode=UserErrorInvalidColumnMappingColumnNotFound

Question

ErrorCode=UserErrorInvalidColumnMappingColumnNotFound

Lucy Liu 10

I'm having the a similar issue copying from csv to azure sql db as this post.

I'm getting the same error, however the behavior is a little different:

The column indicated in the error is added to the Source under Additional Columns section and exists in the Sink table (both uppercase).
The copy activity is nested in a ForEach activity that loops through about 70 files that pulls data into corresponding tables. Tables are auto created in the DB if it does not already exist. The failure occurs on only 1 file.
Even stranger is that when I check the sink table in sql the expected data is copied.

I've even tried deleting the sink table completely and having the pipeline auto create a new table. The error persists even when the table is created and the data is copied successfully.

Any help to resolve this error is appreciated.

Thanks!

Chandra Boorla 14,585 Reputation points Microsoft External Staff Moderator

2024-07-16T05:09:40.1666667+00:00

Hi @Lucy Liu

Just checking in to see if the below answer helped. If this answers your query, do click " Accept the answer” for the same, which might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

1 answer

Your answer

Chandra Boorla 14,585 Reputation points Microsoft External Staff Moderator

2024-07-16T05:09:40.1666667+00:00

Hi @Lucy Liu

Just checking in to see if the below answer helped. If this answers your query, do click " Accept the answer” for the same, which might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

Answer 1

Chandra Boorla 14,585 Microsoft External Staff Moderator

Hi @Lucy Liu

Thanks for the question and using MS Q&A platform.

Are you doing explicit mapping (Import Schemas in Mapping section of Copy) or auto mapping from source to sink? If you are doing auto mapping, could you please try explicit mapping by clicking import schemas in Mapping section of your copy activity. You will be experiencing this error message when you pass mapping without Import Schemas:

You are able to resolve this error message when you pass mapping with Import Schemas:

If the issue still persists, please do let us know. Glad to help.

Hope this information helps!

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Lucy Liu 10 Reputation points

2024-07-11T13:21:55.34+00:00

My copy activity is nested in a ForEach loop that loops through a number of different source files with different sink tables. I'm unable to import schema on the single source. The copy activity works on all of the source files and sink tables except for this one.

ForEach loop based on import metadata:

In the ForEach loop I'm copying the source data if file size > 0:
Chandra Boorla 14,585 Reputation points Microsoft External Staff Moderator

2024-07-12T07:49:43.29+00:00
Hi @Lucy Liu

Thank you for providing more context. I understand that importing schema for a single source file might not be feasible in your scenario.

In that case, here are some other possible causes for the ErrorCode=UserErrorInvalidColumnMappingColumnNotFound error:

Column name mismatch: Double-check that the column names in the source file and sink table match exactly, including case sensitivity. Even though you mentioned that the column exists in both the source and sink, it's essential to verify this.

Column data type mismatch: Ensure that the data types of the columns in the source file and sink table are compatible. If there's a mismatch, it might cause the error.

ForEach loop variable: Since the ForEach loop is based on import metadata, check if the loop variable is correctly set and used in the Copy Activity. Verify that the variable is correctly referencing the source file and sink table for each iteration.

Dynamic column mapping: Consider using dynamic column mapping in the Copy Activity to handle column name differences between the source and sink. This might help resolve the issue.
For more details, please refer: https://www.youtube.com/watch?v=b27gmOufge4

To further troubleshoot the issue, could you please provide more details about the source file and sink table structures, including the column names and data types? Additionally, please confirm that the ForEach loop variable is correctly set and used in the Copy Activity. I appreciate your cooperation in providing more information to help us troubleshoot the issue.

Hope this helps! Let me know if you have any further questions.
Chandra Boorla 14,585 Reputation points Microsoft External Staff Moderator

2024-07-15T06:43:56.87+00:00

Hi @lucy liu

Following up to see if the above answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Share via

ErrorCode=UserErrorInvalidColumnMappingColumnNotFound

1 answer

Your answer