Mongodb - ADF connection error

Kiran Joka 0

I am facing the below issue while connecting to MongoDB. In the dataset, I am able to see preview data, but in the pipeline copy activity, I am getting the below error.

Error: Failure happened on 'Source' side. 'Type=System.InvalidCastException,Message=Invalid cast from 'System.Int64' to 'System.Nullable`1[[System.Int32, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]]'.,Source=mscorlib,'

Vinodh247-1375 11,396 Reputation points

2024-04-29T09:59:05.0533333+00:00

Hi Kiran Joka,

Thanks for reaching out to Microsoft Q&A.

usually occurs when the order of columns in your data mapping does not match the order with the source. Check if the order of each column in your data mapping aligns with that of your source schema. Also check if there is any driver incompatibility in your linked service with monogodb.

https://learn.microsoft.com/en-us/azure/data-factory/connector-mongodb?tabs=data-factory#linked-service-properties

Please 'Upvote'(Thumbs-up) and 'Accept' as an answer if the reply was helpful. This will benefit other community members who face the same issue.
Kiran Joka 0 Reputation points

2024-04-29T10:41:06.5466667+00:00

I can be able to see the data in dataset level. When I click on preview data in pipeline getting error message.
Vinodh247-1375 11,396 Reputation points

2024-04-29T12:32:58.5833333+00:00

in the dataset, I am able to see preview data, but in the pipeline copy activity, I am getting the below error

When I click on preview data in pipeline getting error message.

could you confirm if the issue is when you try to preview or copy activity. are you getting the same error on both the iterations?
Kiran Joka 0 Reputation points

2024-04-29T12:37:32.2466667+00:00

In copy activity I am getting the error, I can preview the data in dataset.
Kiran Joka 0 Reputation points

2024-04-29T12:39:46.9066667+00:00

Copy activity screenshot

2 answers

ShaikMaheer-MSFT 38,126 Reputation points Microsoft Employee

2024-04-29T16:51:57.2733333+00:00

Hi Kiran Joka,

Thank you for posting query in Microsoft Q&A Platform.

From the error message, it seems error is related to data. Preview data will fetch the partial data whereas copy activity loads data from source to sink and while loading if there is any data compatibility issue, then it will throw error.

It seems in your data somewhere null is getting converted to Int as per source or sink table formats and resulting in error. Kindly check your data for same.

Please note, under copy activity settings you can try skip in compatible rows option to avoid this error. But this skips the non-compatible rows.

Hope this helps. Please let me know how it goes. Thank you.
Please sign in to rate this answer.
Kiran Joka 0 Reputation points

2024-04-29T17:15:13.09+00:00

I tried to use skip in compatible rows, but it's not working.

ShaikMaheer-MSFT 38,126 Reputation points Microsoft Employee

2024-04-30T16:41:48.31+00:00

Hi Kiran Joka, Then it seems some issue with data. Kindly check your data correctly if there are any nulls and see if that helps. Thank you.

ShaikMaheer-MSFT 38,126 Reputation points Microsoft Employee

2024-05-07T04:11:07.6566667+00:00

Hi Kiran Joka, Just checking if you get chance to share updates?
Sign in to comment
Kiran Joka 0 Reputation points

2024-05-07T04:29:41.76+00:00

Looks like issue is with driver version(In Linked Service) and special character in the connection string(when we add to key vault)
Please sign in to rate this answer.
Vinodh247-1375 11,396 Reputation points

2024-05-08T14:42:06.76+00:00

ShaikMaheer-MSFT
Sign in to comment

Share via

Mongodb - ADF connection error

2 answers