Error writing to cosmosdb sink

Question

I'm getting the following error trying to write to a cosmosdb sink. I've tried decreasing the batch size to 1 and the problem persists. Any thoughts on how to solve this problem?

Thank you!

{
"StatusCode": "DFExecutorUserError",
"Message": "Job failed due to reason: Errors encountered in bulk import API execution. PartitionKeyDefinition: {\"paths\":[\"/product_id\"],\"kind\":\"Hash\"}, Number of failures corresponding to exception of type: com.microsoft.azure.documentdb.DocumentClientException = 1; FAILURE: com.microsoft.azure.documentdb.DocumentClientException: {\"Errors\":[\"Request size is too large\"]}\nActivityId: 6efc5f73-b85b-11eb-a5d6-8560775d5c0f, Request URI: /apps/837b86bc-7029-4526-9f8c-86dc293a7b49/services/d8ea2608-02d9-4bbd-bf9b-04341fac787a/partitions/8ee291e7-b52f-4e2c-bf24-115193243591/replicas/132657744298919356p//, StatusCode: RequestEntityTooLarge, SubStatusCode: , \nRequestStartTimeUTC: \"19 May 2021 04:34:00.850\", RequestEndTimeUTC: \"19 May 2021 04:34:00.911\", Number of regions attempted: 1\nStoreResponseStatistics{requestStartTimeUTC=\"19 May 2021 04:34:00.850\", requestResponseTimeUTC=\"19 May 2021 04:34:00.911\", storeReadResult=StoreReadResult{statusCode=413, storePhysicalAddress=https://cdb-ms-prod-eastus2-fd4.documents.azure.com:14612/a",
"Details": "Errors encountered in bulk import API execution. PartitionKeyDefinition: {\"paths\":[\"/product_id\"],\"kind\":\"Hash\"}, Number of failures corresponding to exception of type: com.microsoft.azure.documentdb.DocumentClientException = 1; FAILURE: com.microsoft.azure.documentdb.DocumentClientException: {\"Errors\":[\"Request size is too large\"]}\nActivityId: 6efc5f73-b85b-11eb-a5d6-8560775d5c0f, Request URI: /apps/837b86bc-7029-4526-9f8c-86dc293a7b49/services/d8ea2608-02d9-4bbd-bf9b-04341fac787a/partitions/8ee291e7-b52f-4e2c-bf24-115193243591/replicas/132657744298919356p//, StatusCode: RequestEntityTooLarge, SubStatusCode: , \nRequestStartTimeUTC: \"19 May 2021 04:34:00.850\", RequestEndTimeUTC: \"19 May 2021 04:34:00.911\", Number of regions attempted: 1\nStoreResponseStatistics{requestStartTimeUTC=\"19 May 2021 04:34:00.850\", requestResponseTimeUTC=\"19 May 2021 04:34:00.911\", storeReadResult=StoreReadResult{statusCode=413, storePhysicalAddress=https://cdb-ms-prod-eastus2-fd4.documents.azure.com:14612/apps/837b86bc-7029-4526-9f8"
}

Answer 1

Hello @Scott Alvis ,
Thanks for the ask and using the Microsoft Q&A platform .
Since you have not mentioned what is the source , I am assumeing that you may be hitting anyone of the limits called out here

Please do let me know how it goes .
Thanks
Himanshu
Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members

Answer 2

The size of 2MB is small , Can it be adjusted to 10MB in the future?