IOT Hub File Upload Limit

David Gonzalez D 61

Hi, i have an IOT Hub with 50 000 devices connected, this devices are sending files every five minutes but the problem is i frecquently receive this responses from IOT Hub

The request has been throttled. Wait 10 seconds and try again. Operation type: FileUploadRequest

I have an S3 IOT Hub service plan with 3 instances, i read the documentation and i found the file upload limit is 83.33 in S3 service plan per instance so my limit is 250 uploads

Is it possible to increase the upload limit to support 50 000 sendings?

Thank you very much

1 answer

QuantumCache 20,031 Reputation points

2022-09-14T22:54:36.58+00:00

Hello @David Gonzalez D

Your S3 each unit has some limits, 5000 file uploads per minute per unit. Please let us know if this is not what you are looking at?

Document: Operation throttles

What is the time you are looking to finish 50 000 sending?

Is every device exceeding the 10 files concurrency upload limit?
Please sign in to rate this answer.
David Gonzalez D 61 Reputation points

2022-09-14T23:02:55.55+00:00

Hi, when documentation says 50000/min/unit it means that each connected device can upload 5000 files per minute? or it means the whole instance??

Is every device exceeding the 10 files concurrency upload limit?
May be, sometimes there are segmented files in multiple threads let me check if this is the issue.

David Gonzalez D 61 Reputation points

2022-09-14T23:04:14.717+00:00

What is the time you are looking to finish 50 000 sending?
Less than five minutes, is there possible?

Thanks for support.

David Gonzalez D 61 Reputation points

2022-09-14T23:07:35.607+00:00

Hi, when documentation says 50000/min/unit it means that each connected device can upload 5000 files per minute? or it means the whole instance??

Is every device exceeding the 10 files concurrency upload limit?
May be, sometimes there are segmented files in multiple threads let me check if this is the issue.

What is the time you are looking to finish 50 000 sending?
Less than five minutes, is there possible?

QuantumCache 20,031 Reputation points

2022-09-14T23:45:26.923+00:00

Hello @David Gonzalez D

Hi, when documentation says 50000/min/unit it means that each connected device can upload 5000 files per minute? or it means the whole instance??

No, No, I mean the limits are per Instance of IoT Hub. The total files reaching the IoT Hub instance should be in limit of 5K per minute per Unit(instance)( either from single Device or from multiple devices, we count the Total files at IoT Hubs doorstep per minute)

What is the time you are looking to finish 50 000 sending? Less than five minutes, is there possible?

3 Units x (5000 per minute) = 15000 per minute for all 3 units. [This is the capability]

5 minutes x 15000 = 75,000 messages : from all the devices (combined count)

And not more than 10 concurrent files from each device.

Correct me if i am wrong in the calculation.

QuantumCache 20,031 Reputation points

2022-09-14T23:50:49.55+00:00

Latency
IoT Hub strives to provide low latency for all operations. However, due to network conditions and other unpredictable factors it cannot guarantee a certain latency. When designing your solution, you should:

Avoid making any assumptions about the maximum latency of any IoT Hub operation.

Provision your IoT hub in the Azure region closest to your devices.

Consider using Azure IoT Edge to perform latency-sensitive operations on the device or on a gateway close to the device.

Multiple IoT Hub units affect throttling as described previously, but do not provide any additional latency benefits or guarantees.

If you see unexpected increases in operation latency, contact Microsoft Support or Leave a new question on this forum and we will help you.

David Gonzalez D 61 Reputation points

2022-09-15T14:56:11.113+00:00

Hi, 5 minutes x 15000 = 75,000 messages : from all the devices (combined count), yes te calculation is correct but just to make sure, if i try to do an upload and it fails because of throthling, this try count as an upload operation? or count until the file is uploaded? I mean, if 50 000 devices try to upload a file at the same time, 15 000 devices upload correct and 35 000 fail then i retry, this 35 000 fails count as an upload op???

And not more than 10 concurrent files from each device.
Correct i did an adjust to prevent that and it reduced the fail messages

QuantumCache 20,031 Reputation points

2022-09-15T15:11:12.583+00:00

Hello @David Gonzalez D ,

That was great to see resolving one of the issues!!!!

And not more than 10 concurrent files from each device.
Correct i did an adjust to prevent that and it reduced the fail messages

QuantumCache 20,031 Reputation points

2022-09-15T15:14:29.34+00:00

Hello @David Gonzalez D ,

if 50 000 devices try to upload a file at the same time, 15 000 devices upload correct and 35 000 fail then i retry, this 35 000 fails count as an upload op???

I would like to verify if the quota is already hit ? otherwise your Azure IoT Hub configuration is huge and it must respect the said Quota limit and should allow the 50,000 file uploads.
And you already said the file size is within the limit/
Sign in to comment