Stream Analytics Job within the private subnet is not reading input from IoT Hub on the same private VNET

Question

Hello! I've setup a private virtual network and resources for IoT Hub and Stream Analytics. IoT Hub is setup on a private endpoint within subnet1 of my private VNET and Stream Analytics is made to run within the same private VNET from another subnet.
Stream Analytics input connection test to IoT Hub works fine and am able to preview the data received on IoT Hub. However when the SA job is run, I dont see any IoT Hub input messages consumed by Stream Analytics - I could confirm from the monitoring metrics that Stream Analytics does not show any Input events and watermark delay is increasing. IoT Hub metrics show the telemetry data sent.
Pls note: I've not setup the stream analytics on a dedicated v2 cluster fearing the high monthly expense. Could anyone in the community suggest on this topic pls?

Answer 1

Hi @Vishwanath Hariharan Greetings! Welcome to Microsoft Q&A forum. Thank you for posting this question here.

One of the possible reasons why you might be receiving data could be the usage of the same Consumer group. IoT Hub allows you to add consumer groups for endpoints to provide independent views into the event stream that enable apps and Azure services to independently consume data from the same Event Hub endpoint. It is recommended that there's only one active receiver on a partition within a consumer group. Inspect your Azure Stream Analytics input section to see if the Consumer Group is set to $Default Please refer the below image for reference.

Add a new Consumer Group on your IoT Hub and set it to the Stream Analytics job consumer group. Wait a few minutes and see if the events start.

Another reason why the Stream analytics job might be failing could be due to multiple parallel jobs running on Stream Analytics cluster. This might be the reason why the Watermark delay keeps increasing. If you suspect this could be the cause, please refer the article Analyze Stream Analytics job performance by using metrics and dimensions for debugging steps on how to overcome this issue.

Hope this helps! If you need further assistance on this issue, please reach us through the comments below and we would be glad to assist you.

---

If the response helped, please do click Accept Answer and Yes for the answer provided. Doing so would help other community members with similar issue identify the solution. I highly appreciate your contribution to the community.