How to Fix Synapse Spark pool Job Execution Error

Question

How to Fix Synapse Spark pool Job Execution Error

Kate Zhang 0

When I ran a notebook with below code

%%pyspark
df = spark.read.load('abfss://******@sasyoccutableaudev.dfs.core.windows.net/Report.ACCOUNT.parquet', format='parquet')
display(df.limit(10))

I got Py4JJavaError: An error occurred while calling o1659.load. : Status code: -1 error code: null error message: InvalidAbfsRestOperationExceptionjava.net.UnknownHostException: sasyoccutableaudev.dfs.core.windows.net

Santhi Swaroop Naik Bukke 595 Reputation points

2023-01-20T20:41:53.5566667+00:00

This error message is indicating that there is a problem with the hostname specified in the Azure Data Factory pipeline. The hostname "sasyoccutableaudev.dfs.core.windows.net" is not resolving, resulting in a "java.net.UnknownHostException" error. This could be caused by a number of issues, such as a typo in the hostname, a DNS resolution issue, or an issue with the network connection. It would be helpful to check the configuration of the pipeline and ensure that the hostname is spelled correctly and that the necessary DNS and network configurations are in place.
Santhi Swaroop Naik Bukke 595 Reputation points

2023-01-20T21:33:21.0466667+00:00

This error message is indicating that there is a problem with the hostname specified in the Azure Data Factory pipeline. The hostname "sasyoccutableaudev.dfs.core.windows.net" is not resolving, resulting in a "java.net.UnknownHostException" error. This could be caused by a number of issues, such as a typo in the hostname, a DNS resolution issue, or an issue with the network connection. It would be helpful to check the configuration of the pipeline and ensure that the hostname is spelled correctly and that the necessary DNS and network configurations are in place.
Bhargava-MSFT 31,261 Reputation points Microsoft Employee Moderator

2023-01-24T00:53:58.9766667+00:00

Hello @Kate Zhang,

I am checking to see if you got a chance to look into my above response. Please let us know if you have any further questions.

1 answer

Your answer

Santhi Swaroop Naik Bukke 595 Reputation points

2023-01-20T20:41:53.5566667+00:00

This error message is indicating that there is a problem with the hostname specified in the Azure Data Factory pipeline. The hostname "sasyoccutableaudev.dfs.core.windows.net" is not resolving, resulting in a "java.net.UnknownHostException" error. This could be caused by a number of issues, such as a typo in the hostname, a DNS resolution issue, or an issue with the network connection. It would be helpful to check the configuration of the pipeline and ensure that the hostname is spelled correctly and that the necessary DNS and network configurations are in place.
Santhi Swaroop Naik Bukke 595 Reputation points

2023-01-20T21:33:21.0466667+00:00

This error message is indicating that there is a problem with the hostname specified in the Azure Data Factory pipeline. The hostname "sasyoccutableaudev.dfs.core.windows.net" is not resolving, resulting in a "java.net.UnknownHostException" error. This could be caused by a number of issues, such as a typo in the hostname, a DNS resolution issue, or an issue with the network connection. It would be helpful to check the configuration of the pipeline and ensure that the hostname is spelled correctly and that the necessary DNS and network configurations are in place.
Bhargava-MSFT 31,261 Reputation points Microsoft Employee Moderator

2023-01-24T00:53:58.9766667+00:00

Hello @Kate Zhang,

I am checking to see if you got a chance to look into my above response. Please let us know if you have any further questions.

Answer 1

Bhargava-MSFT 31,261 Microsoft Employee Moderator

Hello @Kate Zhang,

Welcome to the MS Q&A platform.

On the storage account container level, can you please switch the Authentication method to "Access key" and see if it helps?

User's image

Share via

How to Fix Synapse Spark pool Job Execution Error

1 answer

Your answer