question

ImranMondal-3977 avatar image
0 Votes"
ImranMondal-3977 asked MartinJaffer-MSFT answered

Data Factory Copy data Blob - To _Table Storage

Hi Team,

I am running a data factory job that loads data from Blob Storage to Table storage, I am using 200 Data integration Unit.

I am getting the below error.

Operation on target Copy_8fe failed: Failure happened on 'Sink' side. ErrorCode=FailedStorageOperation,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=A storage operation failed with the following error 'Element 0 in the batch returned an unexpected response code.'.,Source=,''Type=Microsoft.WindowsAzure.Storage.StorageException,Message=Element 0 in the batch returned an unexpected response code.,Source=Microsoft.WindowsAzure.Storage,StorageExtendedMessage=The server is busy.
RequestId:172be358-f002-0057-1ada-273ff6000000
Time:2021-04-02T16:05:34.6782152Z,,'


Please help me resolve this issue

azure-data-factoryazure-table-storage
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

MartinJaffer-MSFT avatar image
0 Votes"
MartinJaffer-MSFT answered

Hello again @ImranMondal-3977 .

It sounds like you are being throttled by Table Storage. 200 Data integration units sound like enough parallelism to cause this, assuming each unit moves 100 entities per second, or all of them move 10 entities per second into the same partition. See table storage performance limits here.

Maximum request rate per storage account 20,000 transactions per second, which assumes a 1-KiB entity size
Target throughput for a single table partition (1 KiB-entities) Up to 2,000 entities per second

In your case, I think reducing the Data integration units might actually be the solution. If you still need more throughput, options include using more storage accounts, bigger storage accounts, or changing architecture.


5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.