Read From IOT Hub to DataBricks

Question

If two jobs are Run concurrently to read data from the IOTHUB job getting aborted. even I tried with sequential timings but still, the job is failing it throws the below exception.

Please suggest

Answer 1

Hi @Imran Mondal ,

I have verified with EventHubs team and confirmed that this issue could happen if the same consumer group is being used in the concurrent jobs. It has been called out in the connector’s FAQ document that a unique consumer group should be used for each consuming application.

Here is a stackoverflow thread where a similar issue has been discussed with possible resolution: https://stackoverflow.com/questions/54750779/reusing-an-event-hub-stream-for-multiple-queries-in-azure-data-bricks

Hope this helps.

----------

Please don’t forget to Accept Answer and Up-Vote wherever the information provided helps you, this can be beneficial to other community members.