Azure Databricks - InvalidAbfsRestOperationException

Sharma, Abhinav 21

I am trying to run Azure Databricks Notebook using ADB job clusters. The notebook language is pyspark. I am trying to access a container located in azure storage account. Container contains text files.

Notebook code uses command spark.read.format('text).load('filePath). While trying to read data from azure storage account container, command gets stuck and after few hours, ~5-6 hours, it gets timed out and throw below exception.

"Py4JJavaError: An error occurred while calling o1778.load. : Status code: -1 error code: null error message: InvalidAbfsRestOperationExceptionjava.net.SocketTimeoutException: connect timed out"

Can anyone please let me know whether this exception pertains to network, storage account, spark cluster or some other resource and what could be possible solution.

Please let me know if any more information is required on this.

HimanshuSinha-msft 19,476 Reputation points Microsoft Employee

2022-12-27T22:57:09.17+00:00

Hello @Anonymous ,
Since its is failing after 5-6 hrs , its needs a closer look and hence I think you should work with the MS support . if you have a support plan you may file a support ticket, else could you please send an email to azcommunity@microsoft.com with the below details, so that we can create a one-time-free support ticket for you to work closely on this matter.
Subscription ID:
Subject : Attn Himanshu
Please let me know once you have done the same.
Thanks
Himanshu
Sharma, Abhinav 21 Reputation points

2022-12-29T06:03:42.31+00:00

Hi @HimanshuSinha-msft ,

Thank you for response, it is helpful.
As per your suggestion, we will raise a support ticket with Microsoft and take it further. We have a separate team for that. They will look into it.

Thank You.
HimanshuSinha-msft 19,476 Reputation points Microsoft Employee

2023-01-05T17:50:16.19+00:00

Hello @Anonymous ,
Can you please the SR# ?
Thanks
Himanshu

Share via

Azure Databricks - InvalidAbfsRestOperationException

Your answer