How to fix PERMISSION_DENIED: request not authorized SQLSTATE: 42501

Question

I have onboarded the SPN to the Databricks Account and Workspace. Added the account client id and secret to scope. I can read data from my source. I have unity catalog schema and catalog but I cannot write to the container directly or to the schema. EXTERNAL LOCATION has been created. GRANTS of ALL PRIVLEDGES on the SCHEMA and CATALOG have been made. SPN has contributor on the container. Databricks Connector has contributor to the container.

Answer 1

Hi Andres de la Garza,
Thanks for contacting to Microsoft QA.
permission denied / SQL state 42501 means uc can see your catalog/schema, but the identity that actually touches adls can’t write to the storage path, or you’re writing to abfss directly instead of a uc object. “contributor” on the storage account/container is management-plane only and won’t let you write files.

ignore the “databricks connector has contributor” bit uc uses the storage credential’s spn to hit adls, not that connector role.

likely to follow

1. Use a UC‑enabled cluster
   Access mode: Single User or Shared with UC; recent DBR.
2. Don’t write to raw abfss:// paths
   Pick one path:
   • Managed tables → make sure the metastore root or your catalog/schema has a managed location, then create tables there.
   • Files / external tables → create a Storage Credential with your SPN, then an External Location on the target folder, then a Volume on that location. Write under /Volumes/<catalog>/<schema>/<volume>/...
3. Grant data‑plane rights on storage to the SPN used in the Storage Credential
   • Assign Storage Blob Data Contributor on the storage account or container.
   • If HNS ACLs are in use: ensure execute (x) on every parent folder and rwx on the target folder.
   • If the credential pulls from Key Vault: allow the SPN to get/list that secret.
4. Grant UC privileges (schema/catalog grants alone don’t write files)
   • Catalog: USE CATALOG
   • Schema: USE SCHEMA, CREATE
   • External location: WRITE FILES (and READ FILES if needed)
   • Volume: WRITE
   Note: ALL PRIVILEGES on schema/catalog does not imply WRITE FILES to storage.
5. Validate what you configured

      VALIDATE EXTERNAL LOCATION <name>;
      DESCRIBE EXTERNAL LOCATION <name>;
      SHOW GRANTS ON EXTERNAL LOCATION <name>;
      SHOW GRANTS ON VOLUME <name>;
      SHOW GRANTS ON SCHEMA <catalog>.<schema>;
   

6. Quick checks if it still fails
   • The external location’s path exactly matches where you’re writing (no extra/missing subfolders).
   • The SPN on the storage credential is the same principal you granted RBAC/ACLs to.
   • The catalog/schema really have a managed location if you’re creating managed tables.
   • Cluster is running in UC mode; avoid direct path access—stick to Volumes or UC locations.