Copy data to blob storage fails with UserErrorFailedFileOperation,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Upload file failed at path

Anonymous
2022-09-01T10:40:16.89+00:00

When copying data from cosmos DB to Blob storage I get an error:
"UserErrorFailedFileOperation,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Upload file failed at path orchestrationsarch/20220406temp\20220406bigFile.json.,Source=Microsoft.DataTransfer.Common,''Type=Microsoft.Azure.Documents.RequestTimeoutException,Message=Request timed out."

This is my sink configuration:
236790-image.png

The action fails just after a short time which is odd because the result set of the query is not that big. Also, I don't know why there is a backslash in the path: "orchestrationsarch/20220406temp\20220406bigFile.json"
236915-image.png

Can anyone tell me what could be wrong?

Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,415 questions
Azure Cosmos DB
Azure Cosmos DB
An Azure NoSQL database service for app development.
1,437 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,487 questions
{count} votes

3 answers

Sort by: Most helpful
  1. Anonymous
    2022-09-19T08:33:57.857+00:00

    The path doesn't really matter, slash and backslash nither. I'm able to access any other file in the container that was created by this pipeline. I just don't know why the outcome of this pipeline is so random. Sometimes it passes with no problem, sometimes it just waits from 1 minute to several hours.

    A successful run for a large amount of data:
    242290-image.png

    A failed run for no particular reason:
    242350-image.png

    The error message is misleading, I don't know if this is a problem on the CosmosDB or the Blob side. I added logging to the copy activity, but it doesn't show anything useful. Is there a place where I can find detailed logs from the pipeline runs?

    1 person found this answer helpful.

  2. DevWeb 26 Reputation points
    2022-10-31T15:27:14.177+00:00

    Do we have a solution for this issue? I am experiencing the same problem right now. The source storage container has many folders, it appeared that the one folder that has more nested folders (4 level) was choking here with the error message and the rest of the folder paths got copied without a problem.

    255758-image.png

    1 person found this answer helpful.

  3. Lokesh Rajpoot 36 Reputation points
    2023-04-19T11:20:03.9366667+00:00

    Hi @Anonymous i was also facing this same issue and that is resolved now the issue was on the source data there are some data that were not satisfying the condition. I was applying date casting and there were some data 202300501 that raised error in casting as DateTime after handling this my pipeline is fixed now Thanks

    1 person found this answer helpful.