Query azure databricks table from azure function app

Question

Query azure databricks table from azure function app

Hi,

I have a table Invoice in Azure Databricks. I would like run a select query on this table and get the result in my azure function. My function app is .net and consumption plan.

I am unable to figure how to achieve this. Any help will be much appreciated.

Thanks

Answer 1

MughundhanRaveendran-MSFT 12,126 Microsoft Employee

@Karpagam Gurumurthy ,

Thanks for reaching out to Q&A.

You can install Simba Spark ODBC Driver to query Azure Databricks via ODBC connection. However you wont be able to install ODBC driver due to the functions sandbox limitation

You could use Windows Containers on Azure App Service in which you have control over what drivers or custom software to install.

To run Functions inside the container you can use the following Windows Container for Azure Functions v3 : mcr.microsoft.com/azure-functions/dotnet:3.0-nanoserver-1809 and run it App Service: https://learn.microsoft.com/en-us/azure/app-service/quickstart-custom-container?pivots=container-windows&tabs=dotnet

Please note that, you won’t have the portal experience.

I hope this helps!

Please 'Accept as answer' and ‘Upvote’ if it helped so that it can help others in the community looking for help on similar topics.

Karpagam Gurumurthy 41 Reputation points

2022-02-07T19:51:47.117+00:00

Thanks for the quick response. This is exactly what I was looking for w.r.t using ODBC drivers to connect to azure databricks.
Karpagam Gurumurthy 41 Reputation points

2022-02-09T20:07:26.49+00:00

Hi @MughundhanRaveendran-MSFT ,

I will not be able to go for the App service. I got a suggestion to use databricks sql endpoints? From what I understand, to use databricks SQL endpoints, I would still require ODBC driver.

I can call Rest API but they do not have the capability to return the result back to azure functions. One option is to write the result to storage and access read from there in my function.

Could you please let me know if my understanding is correct?
MughundhanRaveendran-MSFT 12,126 Reputation points Microsoft Employee

2022-02-14T05:12:11.557+00:00

@Karpagam Gurumurthy , If you write the result to the storage, it can be read using Storage blob trigger or Storage queue trigger functions.
Pin 1 Reputation point

2022-12-05T11:14:26.947+00:00

Were you able to get to a solution to this problem ? I am also looking at a potential solution for a similar scenario. SQL Endpoints -- to be consumed at Azure Function apps.
How is the performance overall if there is this additional layer of writing and reading from storage.

Query azure databricks table from azure function app

0 additional answers

Activity