Azure IoT Hub trigger for Azure Functions breaks until I touch EventHub partition storage blobs

Question

I have an Azure Function which uses and IoT Hub trigger. You can see the function decorations below, but it's pretty close to sample code. IoT hub is running on the B1 tier, with four partitions. The function has its own consumer group, to avoid any conflicts with desktop debuggers, VS Code's device monitoring plugin etc. It works OK, but after a few hours, or whenever I restart the service it no longer receives any events from the Event Hub endpoint. Maybe these two conditions are the same, because the service gets restarted due to Azure reallocating resources. After multiple round of refinement, this is the simplest procedure I have found to make it work again:

1. Using the Azure Portal, explore the azure-webjobs-eventhub container corresponding to the Event Hub.
2. For each partition in the corresponding consumer group
   a. Open the blob for that partition of that consumer group in the editor.
   b. If the JSON defines an offset other than null, make a change then undo the change so that the UI will allow it to be saved in its unmodified state.

If I do this while there are two devices sending telemetry messages every 5 seconds, I observe that all the backlogged messages from one of the devices pour in after a write one of the blobs. When I write the other, the same happens for the messages from the other device. After this procedure, everything works as it should for a few more hours.

This is obviously not a fix I would be comfortable using in production. It seems to indicate some underlying bug or misconfiguration. Where to now?

Although all tutorials talk about using this approach to responding to IoT Hub events in Azure Functions, I note that Event Grid is another, newer, approach. Is there any reason to think it will be more reliable than Event Hub endpoints?

 [FunctionName("IotHubTrigger")]
    public static async Task Run(
        [EventHubTrigger("messages/events", Connection = "IoTHubEndpoint", ConsumerGroup = "%IoTConsumerGroup%")]EventData message,
        [SignalR(HubName = SignalRHubName)]IAsyncCollector<SignalRMessage> signalRMessages,      
        ILogger log)
    {
//my code here

}

Answer 1

Are you using a Gen2 Storage account by any chance?