Azure IoT Hub - Message Routing: Custom Endpoints Quota

Question

Azure IoT Hub - Message Routing: Custom Endpoints Quota

Amir Zuker (CV) 75

Hi,

TL;DR: Azure IoT Hub has a limit of maximum 10 custom endpoints in its message routing, can we increase it somehow?

We're using Azure IoT Hub and its message routing. We have different kinds of telemetry sent from devices and we define several routes with different custom endpoints using separate event-hubs for that matter.

There's a limit though - a total of only 10 maximum custom endpoints, and we need more than that. (see attached picture)

We're already up to date with all IoT Hub quotas. By the way, the custom endpoints aren't listed as far as I can tell,

I know that we can consider other methods like using Event Grid or Azure Service Bus and doing filter-based routing there instead. However, Event Hubs should fit more nicely in our scenario, plus we need body-level (payload) filtering, and not only on message properties, and IoT hub message routing seems to be only one supporting that.

Long story short, can we increase the limit of 10 maximum custom endpoints somehow?

Amir Zuker.

Answer accepted by question author

0 additional answers

Your answer

Answer 1

AshokPeddakotla-MSFT 36,006 Moderator

@Amir Zuker (CV) Welcome to Microsoft Q&A forum! Thanks for raising this query.

can we increase the limit of 10 maximum custom endpoints somehow?

Unfortunately, the limit of 10 custom endpoints in Azure IoT Hub is a hard limit and cannot be increased.

I know that we can consider other methods like using Event Grid or Azure Service Bus and doing filter-based routing there instead. However, Event Hubs should fit more nicely in our scenario, plus we need body-level (payload) filtering, and not only on message properties, and IoT hub message routing seems to be only one supporting that.

I understand that you are aware of the Event Grid or Azure Service Bus to route messages from IoT Hub.

Alternatively, you can also create multiple IoT hubs and distribute your devices across them, each hub with its own set of custom endpoints.

I am also checking internally with our team in case if it is possible to increase this limit in any way. Stay tuned for further updates.

Amir Zuker (CV) 75 Reputation points

2023-02-16T16:23:44.4533333+00:00

@AshokPeddakotla-MSFT Thank you for the prompt response.

Multiple IoT hubs will complicate our current solution substantially, I appreciate your effort in trying to see further if there's any way to do that, hopefully something will turn up.

Thanks!
Sander van de Velde | MVP 36,951 Reputation points MVP Volunteer Moderator

2023-02-16T22:01:59.4+00:00

Hello @Amir Zuker (CV) ,

Check the Device Provisioning Service allocation policies for linking the device to one of the IoT Hubs.

Perhaps the "Evenly weighted distribution" is perfect for your needs.
AshokPeddakotla-MSFT 36,006 Reputation points Moderator

2023-02-17T07:53:21.5066667+00:00
Amir Zuker (CV) I further verified with our team and found that currently it is not possible to increase the hard limit.

Below is the response and I hope it will help you.

Here are some suggestions for customers who are requesting an increase in multiple endpoints:

The first suggestion is to split the devices in multiples hub, for example, devices for Dev, Test and Prod could be placed in 3 distinct IoTHubs.

Routing to an Event Hub which then splits the messages to 3 or more other Event Hubs or additional readers. This solution may increase the data latency.

Using Event Grid where they can have up to 500 endpoints per IoT Hub. Better for more granular messages than regular telemetry, but another possibility.

Event Grid is a possibility depending on their ordering needs. If they require strict in-order processing Event Grid will not be a viable solution, but otherwise it may work. With complex in-order routing needs, our general recommendation is to use a function. They do introduce some latency, though it’s usually quite minimal when you aren’t doing much processing. To reach the flexibility they’re looking for this is the path that makes the most sense.

Do let us know if it helps.

If the suggestions answers your query, do click 'Accept Answer' and 'Yes' if this answer helpful. And, if you have any further query do let us know.
Amir Zuker (CV) 75 Reputation points

2023-02-19T15:47:55.0233333+00:00
@Sander van de Velde | MVP and @AshokPeddakotla-MSFT -

Thanks for digging deeper and for all the useful information, unfortunately none of the options really apply in our case.

DPS or multiple IoT Hubs - the problem isn't the number of devices, it is relevant even if we have only 1 device. A device can publish numerous d2c messages representing different events. The event-type is in the body (payload) and not a message property, and we need to route and filter according to that

Event Grid or Azure Service Bus - doesn't let us route or filter according to the event type in the body. It can support filtering/forwarding and such but only according to message properties (via the subscription), and the event-type is currently in the body. Plus, even if we had it as a message property, this approach increases the cost as it includes additional managed services only because we can't have more than 10 custom endpoints.

Event Hub that distributes to other Event Hubs - there isn't such a built-in capability in Event Hubs AFAIK. If we talk about Stream Analytics or our own intermediate compute - we wanted to avoid that (increased latency, complexity, cost, etc.). And again, using Event Grid or ASB doesn't help because it's data in the body and not a message property. [please update if I missed something here] I guess there's nothing much to do about it, it's a hard limit that cannot be changed right now.
Seems like we're going to have to set up something more costly that involves additional hops and services for that matter. If we can move the event-type to be a message property, at least we won't have to include our own compute or Stream Analytics for that matter.

Thanks again for your time and effort, much appreciated!

Amir Zuker.
Amir Zuker (CV) 75 Reputation points

2023-02-19T16:14:01.4866667+00:00

There was an editing problem so my comment was duplicated, please ignore this one, thanks.
Sander van de Velde | MVP 36,951 Reputation points MVP Volunteer Moderator

2023-02-19T22:34:34.22+00:00

Hello @Amir Zuker (CV) ,

your additional comments make your case more clear.

It seems the number of event types you use far exceeds the default routing capabilities of the services you mentioned.

I agree with most observations, this is a typical challenge in designing IoT solutions..

I then suggest double-checking Azure Stream Analytics due to the flexibility, excellent filtering, and rerouting it offers. The cost increase is probably compensated by the lesser amount of development hours and increased functionality offered extra.

Share via

Azure IoT Hub - Message Routing: Custom Endpoints Quota

0 additional answers

Your answer