question

MartanoGiovanni-1808 avatar image
0 Votes"
MartanoGiovanni-1808 asked KranthiPakala-MSFT edited

Databricks job timeout sending message to service bus with spark 3.2.1 and azure-messaging-servicebus sdk

Hi, we have developed a scala job using azure-messaging-servicebus sdk for java (ver 7.8.0) to send messages on service bus topic. All work fine as expected using an azure cloud databricks cluster 9.1 LTS (with Spark 3.1.2 and scala 2.12), but running the same job on 10.4 LTS cluster (with Spark 3.2.1, Scala 2.12) the job fail with the following timeout error:


IllegalStateException: Timeout on blocking read for 245600000000 NANOSECONDS
at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:123)
at reactor.core.publisher.Mono.block(Mono.java:1731)



Any suggestion about this issue, how we can solve it? For production environment we need to deploy on 10.4LTS cluster type. Thanks in advance

azure-databricksazure-service-bus
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

I have noted that the log report a previous error (non fatal) caused by the same library (reactor)

22/05/13 13:04:23 ERROR Schedulers: Scheduler worker in group main failed with an uncaught exception
java.lang.NoSuchMethodError: reactor.util.retry.Retry.generateCompanion(Lreactor/core/publisher/Flux;)Lorg/reactivestreams/Publisher;
at reactor.core.publisher.FluxRetryWhen.subscribe(FluxRetryWhen.java:68)
at reactor.core.publisher.FluxRetryWhen.subscribeOrReturn(FluxRetryWhen.java:83)
at reactor.core.publisher.Mono.subscribe(Mono.java:4385)
at reactor.core.publisher.MonoWhen$WhenCoordinator.subscribe(MonoWhen.java:160)
at reactor.core.publisher.MonoWhen.subscribe(MonoWhen.java:99)

0 Votes 0 ·

1 Answer

KranthiPakala-MSFT avatar image
0 Votes"
KranthiPakala-MSFT answered KranthiPakala-MSFT edited

Hello @MartanoGiovanni-1808,

Thanks for the question and using MS Q&A platform.

Seems like you have already reached out in SDK GitHub platform for this issue and one of our engineers have suggested the workarounds to overcome it. I'm sharing the link here as it can be beneficial to other community members who reads this thread.

Here is the GitHub link where this issue has been discussed with possible solutions - [BUG] Send message to service bus using ServiceBusSenderClient get timeout][1]

202482-image.png

Thank you



Please do consider clicking on "Accept Answer" and "Upvote" on the post that helps you, as it can be beneficial to other community members.



image.png (73.5 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.