I am getting error while running the spark job within the azure synapse analytics workspace

Shanmugam S 6 Reputation points
2021-09-24T14:13:59.063+00:00

getting this error when running the spark job with sparkpool creation within the notebook and the job is not successfully executed

Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Got exception: org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsRestOperationException Operation failed: "This request is not authorized to perform this operation using this permission.", 403, HEAD

Azure Monitor
Azure Monitor
An Azure service that is used to collect, analyze, and act on telemetry data from Azure and on-premises environments.
2,066 questions
Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
990 questions
.NET
.NET
Microsoft Technologies based on the .NET software framework.
1,138 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
3,092 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
1,380 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. ShaikMaheer-MSFT 31,551 Reputation points Microsoft Employee
    2021-09-25T06:09:36.9+00:00

    Hi @Shanmugam S ,

    Welcome to Microsoft Q&A Platform. Thank you for posting query here.

    You may be using managed identity or user account to run your notebook which doesn't has permissions to the workspace.

    When you are working on synapse workspace with the managed identity you would need to give Storage Blob Data contributor permission to the workspace that represents the managed identity permission

    Please check below link to know more details about Grant permissions to workspace managed identity,
    https://learn.microsoft.com/en-us/azure/synapse-analytics/security/how-to-grant-workspace-managed-identity-permissions

    Please check below link to know more about Azure Synapse workspace managed identity,
    https://learn.microsoft.com/en-us/azure/synapse-analytics/security/synapse-workspace-managed-identity

    Kindly check below blog also where similar issue discussed.
    https://techcommunity.microsoft.com/t5/azure-synapse-analytics/notebook-this-request-is-not-authorized-to-perform-this/ba-p/1712566

    Requesting you to grant Storage Blob Data contributor for your managed identity or user account and see.

    Hope this will help you. Please let us know how it goes and if any further queries. Thank you.

    ------------------------------

    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
    • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators