401 Unauthorized When Remounting ADLS/Blob Storage in Azure Databricks with Valid Service Principal

Question

Hello,

I recently generated a new client secret for our App Registration, as the previous one had expired. I then updated our Databricks Secret Scope using the following command:

databricks secrets put --scope <secret-scope> --key <secret-key> --string-value <new secret value>

As documented in Microsoft Learn – Mounting cloud object storage on Azure Databricks, I unmounted the existing mount point that was using the old secret and attempted to remount it using the updated credentials.

However, I am now encountering a 401 Unauthorized error when remounting.

Error Details:

AADSTS7000215: Invalid client secret provided. Ensure the secret being sent in the request is the client secret value, not the client secret ID.

What I’ve Already Tried:

Verified that the correct client ID, client secret value (not the secret ID), and tenant ID are being used.

• Confirmed the credentials are valid by successfully authenticating via the Azure CLI and also via POST-Request to get a Token.

Regenerated the client secret to ensure it is active and not expired.

Double-checked the syntax and parameters of the mount command.

• Restarted the Databricks cluster to clear any potential caching issues.
• Even when hardcoding the secrets in the notebook, the error persist like the mount-Function use another secret.

Despite these steps, the issue persists.

Could someone please help identify why Databricks still returns a 401 Unauthorized error, even though authentication with the same Service Principal and secret works correctly outside of Databricks?

Thanks in advance for any guidance.

Answer 1

Hi @Ndjamen, P0462JN (601-ITC-Herden GmbH)
Thanks for your patience. Yes, the following two links are highly relevant and helpful for your issue:

1. Microsoft Q&A – How do I mitigate AADSTS7000215:
   https://learn.microsoft.com/en-us/answers/questions/1000888/how-do-i-mitigate-aadsts7000215-invalid-client-sec
   This explains the root cause and confirms that using the client secret value (not the ID) is key.
2. Stack Overflow – Rare AADSTS7000215 error from Microsoft: https://stackoverflow.com/questions/62424076/rare-aadsts7000215-error-from-microsoft-invalid-client-secret-is-provided
   This highlights that in rare cases, a full service principal credential reset is needed to resolve lingering authentication issues.

If this is helpful, please click Accept Answer and kindly upvote it so that other people who faces similar issue may get benefitted from it.

Let me know if you have any further Queries.

Answer 2

I have good news—The problem has been solved.

The issue was caused by leftover mount points from other storage accounts. Even though we where working with a different storage account, it seems that Databricks was somehow using secrets from the existing mount points (probably because it was mounted with the same service principal).
To fix it, I unmounted all the mount points from every storage account. When remounting them again, everything worked fine.