ADF copy data to cosmosdb producing errors

I created a copydata pipline in azure data factory to move data from a blob to cosmos mongo api database. All necessary dataset connections connected successfully .But while i debug or execute i get the errors below relating to the sink. Is there a way to resolve this issue.

Failure happened on 'Sink' side. ErrorCode=MongoDbFailedWithMongoDbServerError,

'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=>

The operation failed with server error.

ConnectionId: '{ ServerId : { ClusterId : 1,

EndPoint : "Unspecified/cscdevmongocosmosdbconv-canadacentral.mongo.cosmos.azure.com:10255" },

LocalValue : 5,

ServerValue : "936468714" }'.,

Source=Microsoft.DataTransfer.Runtime.MongoDbAtlasConnector,''

Type=MongoDB.Driver.MongoBulkWriteException`1[[MongoDB.Bson.BsonDocument,

MongoDB.Bson,

Version=2.10.4.0,

Culture=neutral,

PublicKeyToken=15b1115599983c50]],

Message=A bulk write operation resulted in one or more errors.

Error=13, Details='Insert error.

BhargavaGunnam-MSFT 26,136 Reputation points Microsoft Employee

2023-01-23T20:40:02.3866667+00:00

Hello @Shittu Adesoji (NHQ-AC-CONSULTANT),

Welcome to the MS Q&A platform.

Per the error message, you seem to have an issue performing the bulk copy.

The issue is most likely with the Cosmos mongo API DB throughput. Can you please increase the value to a higher number and see if it helps in the bulk copy?

I hope this helps. Please let me know if you have any further questions.
BhargavaGunnam-MSFT 26,136 Reputation points Microsoft Employee

2023-01-25T17:29:43.45+00:00

Hello @Shittu Adesoji (NHQ-AC-CONSULTANT),

I am just checking to see if you got a chance to look into my above response.
Shittu Adesoji (NHQ-AC-CONSULTANT) 5 Reputation points

2023-01-27T21:24:41.1866667+00:00

I did increase the throughput for the cosmos db resource from 4000 - 5000 . I still get the error. Pipeline is only moving 16 record s from json file. RU for collection is 400
Shittu Adesoji (NHQ-AC-CONSULTANT) 5 Reputation points

2023-01-28T04:06:29.3266667+00:00

I did increase the throughput for the cosmos db resource from 4000 - 5000 . I still get the error. Pipeline is only moving 16 record s from json file. RU for collection is 400
BhargavaGunnam-MSFT 26,136 Reputation points Microsoft Employee

2023-01-28T04:17:39.3433333+00:00

Hello @Shittu Adesoji (NHQ-AC-CONSULTANT),

Thanks for the reply. Can you try using the mongo Db connector in ADF, which uses a different driver, and see if you can copy it?

Shittu Adesoji (NHQ-AC-CONSULTANT) 5

Still gives similar error only exception driver that is used ...see below

Operation on target Copy_bj0 failed: Failure happened on 'Sink' side. ErrorCode=MongoDbFailedWithMongoDbServerError,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=>The operation failed with server error. ConnectionId: '{ ServerId : { ClusterId : 1, EndPoint : "Unspecified/<databasename>l.mongo.cosmos.azure.com:10255" }, LocalValue : 3, ServerValue : "932206126" }'.,Source=Microsoft.DataTransfer.Runtime.MongoDbAtlasConnector,''Type=MongoDB.Driver.MongoBulkWriteException`1[[MongoDB.Bson.BsonDocument, MongoDB.Bson, Version=2.10.4.0, Culture=neutral, PublicKeyToken=15b1115599983c50]],Message=A bulk write operation resulted in one or more errors.
  Error=13, Details='Insert error.

BhargavaGunnam-MSFT 26,136 Reputation points Microsoft Employee

2023-01-30T22:11:43.17+00:00

Hello @Shittu Adesoji (NHQ-AC-CONSULTANT),

Thanks for the details. As the next steps, I would suggest filing a support request, and a support engineer can take a deeper look into the logs to troubleshoot the issue further. If you don't have a support plan, I can enable one-time free support for you to work closely on this matter.

I am looking forward to hearing from you.

2 answers

Shittu Adesoji (NHQ-AC-CONSULTANT) 5 Reputation points

2023-01-31T22:01:39.74+00:00

Issue resolved and related to privilege provision .Extra contributor role for resource added .
Please sign in to rate this answer.

1 person found this answer helpful.
BhargavaGunnam-MSFT 26,136 Reputation points Microsoft Employee

2023-01-31T23:37:46.96+00:00

Hello @Shittu Adesoji (NHQ-AC-CONSULTANT),

Glad you were able to resolve the issue, and thank you for sharing the resolution, as it helps the community members looking for answers to similar questions.

Jared Livingston 0 Reputation points

2023-11-27T13:29:49.1966667+00:00

Can you specify what the steps were to fix this issue?
Sign in to comment
Shittu Adesoji (NHQ-AC-CONSULTANT) 5 Reputation points

2023-01-31T22:03:56.6166667+00:00

Extra contributor role for resource added to current roles.
Please sign in to rate this answer.

0 comments No comments
Sign in to comment