Databricks job timeout sending message to service bus with spark 3.2.1 and azure-messaging-servicebus sdk

Martano, Giovanni 1 Reputation point
2022-05-13T10:32:19.103+00:00

Hi, we have developed a scala job using azure-messaging-servicebus sdk for java (ver 7.8.0) to send messages on service bus topic. All work fine as expected using an azure cloud databricks cluster 9.1 LTS (with Spark 3.1.2 and scala 2.12), but running the same job on 10.4 LTS cluster (with Spark 3.2.1, Scala 2.12) the job fail with the following timeout error:

IllegalStateException: Timeout on blocking read for 245600000000 NANOSECONDS
at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:123)
at reactor.core.publisher.Mono.block(Mono.java:1731)

Any suggestion about this issue, how we can solve it? For production environment we need to deploy on 10.4LTS cluster type. Thanks in advance

Azure Service Bus
Azure Service Bus
An Azure service that provides cloud messaging as a service and hybrid integration.
548 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
1,938 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. KranthiPakala-MSFT 46,422 Reputation points Microsoft Employee
    2022-05-16T22:21:40.577+00:00

    Hello @Martano, Giovanni ,

    Thanks for the question and using MS Q&A platform.

    Seems like you have already reached out in SDK GitHub platform for this issue and one of our engineers have suggested the workarounds to overcome it. I'm sharing the link here as it can be beneficial to other community members who reads this thread.

    Here is the GitHub link where this issue has been discussed with possible solutions - [BUG] Send message to service bus using ServiceBusSenderClient get timeout]1

    202482-image.png

    Thank you

    ----------

    Please do consider clicking on "Accept Answer" and "Upvote" on the post that helps you, as it can be beneficial to other community members.

    0 comments