Not able to write the data in cosmos DB

Question

Hi ,

I am facing the issue in loading the data in cosmos db. I am able to test the connection and preview the data in ADF but not able to write the data and SPN has contributor role to cosmosdb through which I am trying to load.

Not getting the error details properly. could someone please help on it.

"Failure happened on 'Sink' side. ErrorCode=CosmosDbSqlApiOperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=CosmosDbSqlApi operation Failed. ErrorMessage: Object reference not set to an instance of an object.{"name":"CreateItemAsync","id":"fefb9cb6-0ae8-4bb9-b8a3-515f34e06f82","caller info":{"member":"OperationHelperWithRootTraceAsync","file":"ClientContextCore.cs","line":219},"start time":"09:13:08:285","duration in milliseconds":669.6081,"data":{"Client Configuration":{"Client Created Time Utc":"2025-04-23T21:13:05.3637439Z","NumberOfClientsCreated":7,"User Agent":"cosmos-netstandard-sdk/3.20.0|3.19.3|6|X64|Microsoft Windows 10.0.20348 |.NET Framework 4.8.4739.0|N|F 00000001|","ConnectionConfig":{"gw":"(cps:110, urto:10, p:False, httpf: False)","rntbd":"(cto: 5, icto: -1, mrpc: 30, mcpe: 65535, erd: False, pr: ReuseUnicastPort)","other":"(ed:False, be:True)"},"ConsistencyConfig":"(consistency: Session, prgns:[])"}},"children":[{"name":"ItemSerialize","id":"11e946d2-a3d2-4f64-b2a5-0a28d937c582","caller info":{"member":"ExtractPartitionKeyAndProcessItemStreamAsync","file":"ContainerCore.Items.cs","line":931},"start time":"09:13:08:285","duration in milliseconds":0.0327},{"name":"Get PkValue From Stream","id":"f3c4cded-8b9b-42c5-b3b8-c824d3996409","caller info":{"member":"GetPartitionKeyValueFromStreamAsync","file":"ContainerCore.Items.cs","line":1037},"start time":"09:13:08:285","duration in milliseconds":288.34020000000004,"children":[{"name":"Get Collection Cache","id":"d2b804f4-8aaa-4563-bdde-31c6707838d7","caller info":{"member":"GetCollectionCacheAsync","file":"DocumentClient.cs","line":546},"start time":"09:13:08:285","duration in milliseconds":0.0085,"children":[{"name":"Waiting for Initialization of client to complete","id":"0b505e11-846e-476e-bb86-f7be8ac334e0","caller info":{"member":"EnsureValidClientAsync","file":"DocumentClient.cs","line":1416},"start time":"09:13:08:285","duration in milliseconds":0.0071}]},{"name":"Read Collection","id":"68108ece-9a1d-48f6-b06a-13988700b175","caller info":{"member":"ReadCollectionAsync","file":"ClientCollectionCache.cs","line":68},"start time":"09:13:08:301","duration in milliseconds":258.58590000000004,"data":{"Client Side Request Stats":{"Id":"AggregatedClientSideRequestStatistics","ContactedReplicas":[],"RegionsContacted":[],"FailedReplicas":[],"AddressResolutionStatistics":[],"StoreResponseStatistics":[],"HttpResponseStats":[{"StartTimeUTC":"2025-04-23T21:13:08.5669414Z","DurationInMs":0,"RequestUri":"https://cosmos-cap-de-dev-westus2.documents.azure.com/dbs/CFARCAPAPIDev/colls/XXXXXX","ResourceType":"Collection","HttpMethod":"GET","ActivityId":"f65aa706-1a16-497f-b11f-acb95da18917","StatusCode":"OK"}]}}}]},{"name":"Get Collection Cache","id":"8487fee9-90eb-4901-81ef-c9e2a7435341","caller info":{"member":"GetCollectionCacheAsync","file":"DocumentClient.cs","line":546},"start time":"09:13:08:566","duration in milliseconds":0.0012000000000000001}]}.,Source=Microsoft.DataTransfer.ClientLibrary.CosmosDbSqlApiV3,''Type=Microsoft.Azure.Cosmos.CosmosNullReferenceException,Message=Object reference not set to an instance of an object.{"name":"CreateItemAsync","id":"fefb9cb6-0ae8-4bb9-b8a3-515f34e06f82","caller info":{"member":"OperationHelperWithRootTraceAsync","file":"ClientContextCore.cs","line":219},"start time":"09:13:08:285","duration in milliseconds":669.6081,"data":{"Client Configuration":{"Client Created Time Utc":"2025-04-23T21:13:05.3637439Z","NumberOfClientsCreated":7,"User Agent":"cosmos-netstandard-sdk/3.20.0|3.19.3|6|X64|Microsoft Windows 10.0.20348 |.NET Framework 4.8.4739.0|N|F 00000001|","ConnectionConfig":{"gw":"(cps:110, urto:10, p:False, httpf: False)","rntbd":"(cto: 5, icto: -1, mrpc: 30, mcpe: 65535, erd: False, pr: ReuseUnicastPort)","other":"(ed:False, be:True)"},"ConsistencyConfig":"(consistency: Session, prgns:[])"}},"children":[{"name":"ItemSerialize","id":"11e946d2-a3d2-4f64-b2a5-0a28d937c582","caller info":{"member":"ExtractPartitionKeyAndProcessItemStreamAsync","file":"ContainerCore.Items.cs","line":931},"start time":"09:13:08:285","duration in milliseconds":0.0327},{"name":"Get PkValue From Stream","id":"f3c4cded-8b9b-42c5-b3b8-c824d3996409","caller info":{"member":"GetPartitionKeyValueFromStreamAsync","file":"ContainerCore.Items.cs","line":1037},"start time":"09:13:08:285","duration in milliseconds":288.34020000000004,"children":[{"name":"Get Collection Cache","id":"d2b804f4-8aaa-4563-bdde-31c6707838d7","caller info":{"member":"GetCollectionCacheAsync","file":"DocumentClient.cs","line":546},"start time":"09:13:08:285","duration in milliseconds":0.0085,"children":[{"name":"Waiting for Initialization of client to complete","id":"0b505e11-846e-476e-bb86-f7be8ac334e0","caller info":{"member":"EnsureValidClientAsync","file":"DocumentClient.cs","line":1416},"start time":"09:13:08:285","duration in milliseconds":0.0071}]},{"name":"Read Collection","id":"68108ece-9a1d-48f6-b06a-13988700b175","caller info":{"member":"ReadCollectionAsync","file":"ClientCollectionCache.cs","line":68},"start time":"09:13:08:301","duration in milliseconds":258.58590000000004,"data":{"Client Side Request Stats":{"Id":"AggregatedClientSideRequestStatistics","ContactedReplicas":[],"RegionsContacted":[],"FailedReplicas":[],"AddressResolutionStatistics":[],"StoreResponseStatistics":[],"HttpResponseStats":[{"StartTimeUTC":"2025-04-23T21:13:08.5669414Z","DurationInMs":0,"RequestUri":"https://cosmos-cap-de-dev-westus2.documents.azure.com/dbs/CFARCAPAPIDev/colls/XXXXXXX","ResourceType":"Collection","HttpMethod":"GET","ActivityId":"f65aa706-1a16-497f-b11f-acb95da18917","StatusCode":"OK"}]}}}]},{"name":"Get Collection Cache","id":"8487fee9-90eb-4901-81ef-c9e2a7435341","caller info":{"member":"GetCollectionCacheAsync","file":"DocumentClient.cs","line":546},"start time":"09:13:08:566","duration in milliseconds":0.0012000000000000001}]},Source=Microsoft.Azure.Cosmos.Direct,'"

Thanks in Advance,

Venkat

Answer 1

Hello Venkateswarlu Pottapalli,

Welcome to the Microsoft Q&A and thank you for posting your questions here.

I understand that you are unable to write the data in cosmos DB.

Regarding the error and information provided, for best practices to resolve the issue. Ensure the data includes the necessary partition key and is correctly formatted. Also, the error message indicates a null reference exception, you will need review your code to ensure that all objects are properly instantiated before use. Also, verify the connection string and ensure the service principal (SPN) has the necessary permissions - https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-azure-cosmos-db and if you're copying large amounts of data, try reducing the write batch size in the copy activity sink. This can help avoid issues related to request size limits.

Secondly, for better performance and reliability, use Direct mode for connectivity in your Cosmos DB client configuration. Make sure you're using the latest version of the Azure Cosmos DB SDK. Older versions might have bugs or compatibility issues - https://learn.microsoft.com/en-us/azure/cosmos-db/nosql/best-practice-dotnet

I hope this is helpful! Do not hesitate to let me know if you have any other questions or clarifications.

---

Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful.