Getting error with error code when trying to run synapse pipeline having python notebook on spark cluster BBC_REQUEST_FAILURE_INTERNALSERVERERROR

Bhatt, Ramachandra 0 Reputation points
2023-11-09T05:27:28.71+00:00

Hi team,

it was working till yesterday. today morning when I try to run synapse notebook written in python as part of pipeline using spark serverless pool, I get below error

Error details

Error details

This application failed due to the total number of errors: 1.

Error code 1

BBC_REQUEST_FAILURE_INTERNALSERVERERROR

Message

Cluster creation failed due to an unexpected system error from a runtime related dependency.

Source

Dependency

No other information. not sure what is the issue. could you please guide.

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,985 questions
{count} votes

1 answer

Sort by: Most helpful
  1. PRADEEPCHEEKATLA-MSFT 90,146 Reputation points Microsoft Employee
    2023-11-09T08:34:51.7233333+00:00

    @Bhatt, Ramachandra - Thanks for the question and using MS Q&A platform.

    I'm sorry to hear that you're having trouble running your Synapse notebook on a Spark serverless pool. The error message you received indicates that the cluster creation failed due to an unexpected system error from a runtime related dependency. This error code is not specific to any particular issue, so it's difficult to determine the root cause of the problem without more information.

    To troubleshoot this issue, I recommend checking the logs available in the Logs tab at the bottom part of the Monitor tab in Synapse Studio. From the Activities section, select Apache Spark Applications and find your Spark job from the list. Inspect the logs available in the Logs tab at the bottom part of this page. This may provide more information about the error and help you identify the root cause.

    For more details, refer to Use Synapse Studio to monitor your Apache Spark applications.

    If you are still unable to resolve the issue, please provide more information about your Synapse notebook code snippet which you are using and the Spark pool version you are using. This will help me better understand the issue and provide more specific guidance.

    0 comments No comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.