This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(128, 8%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
When copying data from cosmos DB to Blob storage I get an error:
"UserErrorFailedFileOperation,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Upload file failed at path orchestrationsarch/20220406temp\20220406bigFile.json.,Source=Microsoft.DataTransfer.Common,''Type=Microsoft.Azure.Documents.RequestTimeoutException,Message=Request timed out."
This is my sink configuration:
The action fails just after a short time which is odd because the result set of the query is not that big. Also, I don't know why there is a backslash in the path: "orchestrationsarch/20220406temp\20220406bigFile.json"
Can anyone tell me what could be wrong?
Hello @Anonymous ,
Can you please confirm if you are able to preview data on the path which you have selected in the file path?
And also does it work when you provide full path as hard coded?
Hello @Anonymous ,
Follow up to see if you have had a chance to see the previous response. We need the following information to understand/investigate this issue further.
Please take a look at my "answer", by mistake I replied there.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(124.80000000000001, 25%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ELR%3C/text%3E%3C/svg%3E)
Hi @Anonymous i was also facing this same issue and that is resolved now the issue was on the source data there are some data that were not satisfying the condition. I was applying date casting and there were some data 202300501 that raised error in casting as DateTime after handling this my pipeline is fixed now Thanks
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(131.20000000000002, 51%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGS%3C/text%3E%3C/svg%3E)
Hi @Lokesh Rajpoot I am also facing the same issue . But I have to choose only those files which are modified 2 months ago or earlier .So I have to apply Date casting .
Did you remove the Date Casting ?How did you handled it ? Can u please let me know.
here is Screenshot of my pipeline.
you can cast this out of the foreach activity. If your watermark table is on the source that supports sql query you can easily do this in lookup activity as well.
The path doesn't really matter, slash and backslash nither. I'm able to access any other file in the container that was created by this pipeline. I just don't know why the outcome of this pipeline is so random. Sometimes it passes with no problem, sometimes it just waits from 1 minute to several hours.
A successful run for a large amount of data:
A failed run for no particular reason:
The error message is misleading, I don't know if this is a problem on the CosmosDB or the Blob side. I added logging to the copy activity, but it doesn't show anything useful. Is there a place where I can find detailed logs from the pipeline runs?
Hello @Anonymous ,
Apologize for the delay in response.
Did you resolve the issue or still experiencing the error message?
The problem is still there.
Hello @Anonymous ,
This issue looks strange. For a deeper investigation and immediate assistance on this issue, if you have a support plan you may file a support ticket.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(115.19999999999999, 66%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ED%3C/text%3E%3C/svg%3E)
Do we have a solution for this issue? I am experiencing the same problem right now. The source storage container has many folders, it appeared that the one folder that has more nested folders (4 level) was choking here with the error message and the rest of the folder paths got copied without a problem.
I've created a support ticket to escalate this issue. Several MS departments are working on this because they have never encountered such a problem :P If I have any news I will post them here.
Seems like a Microsoft bug to me. In my case, the File Path was broken down into 2 parts where you have 3, e.g. dataset().container / dataset().destination. I also have the '\' separator in the Directory part. But I got the same error, and it always failed at a specific location inside the specified Container/Directory structure, yet other containers or other directories under the same container has no issue. I have opened a ticket with Microsoft since 11/1/2022 and so far there has been no progress.
I changed the sink to another CosmosDB container and the result was almost identical:
'Type=Microsoft.Azure.Documents.RequestTimeoutException,Message=Request timed out.
ActivityId: 19318520-e983-47be-b2fb-663ef123142e, Request URI: /dbs/BUcpAA==/colls/BUcpAKjiORc=/docs, RequestStats: , SDK: documentdb-dotnet-sdk/2.5.1 Host/64-bit MicrosoftWindowsNT/6.2.9200.0,Source=Microsoft.Azure.Documents.Client,''Type=System.Threading.Tasks.TaskCanceledException,Message=A task was canceled.,Source=mscorlib,'
I think it proves that the Blob storage and the destination path are not the problems.
I was testing my pipeline with different amounts of data and found out that anything above 300-400 MB would crush it. Then I changed my pipeline so it would run for every hour of the day instead of the whole day and it worked. I suggest you do the same and maybe it will also work for you.
The problematic Blob storage path does not contain a large amount of data in my case. It does have many nested folder structures (6 levels) but a small number of small-sized files. I will have to figure out a workaround for this problem though. Thanks for the suggestion.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(316.8, 21%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJB%3C/text%3E%3C/svg%3E)
I was getting the same error, I eventually figured it out when using the 'preview data' button surfaced the underlying error. Turns out my postgres table had an unusual datatype from postgis (geometry) which the underlying library used in the pipeline did not recognise. I solved it by changing my copy operation from targeting a table to being a query, and explicitly casting the geometry column using ::TEXT.
Probably not going to help you with cosmos though :|
In my case, it was a Microsoft bug as they are trying to cope with blob storage with metadata and if you have metadata specified as null (the API supports it but the portal site does not), the error is bound to occur in ADF copy activity.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(35.2, 78%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETX%3C/text%3E%3C/svg%3E)
I have similar error message, but Failure happened on 'Source' side. The source is query from serverless sql pool's external delta tables. My walkaround is to set retry to 5 (default is 0) in General tab of copy activity. Sometimes it succeeded after 4 trials/several hours.