An error occurred when using odbc and managed identity to link azure databricks in the. net core

Question

Environmental description:

The app service has already been configured with managed identity, and the relevant permissions for this managed identity have been configured in Azure databricks. When the app service uses code to access Azure databricks, the following error message will occur：

ERROR [28000] [Simba][DriverOAuthSupport] (8719) Can't access to Azure Metadata Service, check if you run on VM with MI: OAuthAPIErr

According to the reference document, the use of the app service failed

The reference document states that Azure VM needs to be configured, but how to use managed identity to authorize access and use Azure databricks in the app service.

https://learn.microsoft.com/en-us/azure/databricks/integrations/odbc/authentication

Answer 1

@ZhiHong Zhang (BEYONDSOFT CONSULTING INC) - I'm glad that you were able to resolve your issue with the help of support and I'm posting the solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to accept the answer .

Ask: An error occurred when using odbc and managed identity to link azure databricks in the. net core?

Solution: The issue is resolved with the help of support ticket and here is the resolution:

ODBC does not support connecting a web app to a Databricks cluster via managed identity authentication, especially in a Docker environment.

If I missed anything please let me know and I'd be happy to add it to my answer, or feel free to comment below with any additional information.

If you have any other questions, please let me know. Thank you again for your time and patience throughout this issue.

---

Please don’t forget to Accept Answer and Yes for "was this answer helpful" wherever the information provided helps you, this can be beneficial to other community members.

Answer 2

As you mentioned above, the basic permission settings have been determined to be set according to the document. After some testing, the same issue was still found.

But starting from step 5 in the document, it is necessary to configure Azure VM, but the current condition is that it is in the web app service, not Azure VM. And the current operating environment is. net core. All ODBC versions in the document must be at least 2.7.7. This is completely certain. When excluding external conditions and running in the code, I even take out Auth-ClientId and repeatedly use the set value. Then you will get this error:

Error [28000] [Simba] [DriverOAuthSupport] (8719) Can't access Azure metadata service, check if you run on VM with MI: OAuthAPIErr

And the speed is very fast, and the error is clearly pointed out as running in the VM. However, we are currently using a web app service and deploying it in a container style. Based on my guess, this authentication method did not recognize MI at all, so it immediately returned an exception.

I don't know if there are any examples that can be provided to me to correctly access Azure databricks and use query statements in the. net core environment of C # code. If there are no examples, would you like to inform me that they are not currently supported. Because there is a reference below indicating that some SDKs do not support it.

https://databricks-bi-artifacts.s3.us-east-2.amazonaws.com/simbaspark-drivers/odbc/2.8.0/SimbaSparkODBC-2.8.0.1002-Debian-64bit.zip

https://learn.microsoft.com/en-us/azure/databricks/dev-tools/azure-mi-auth

https://learn.microsoft.com/en-us/azure/databricks/dev-tools/auth/