Deploy ASP.NET Core Web API with Simba Spark Driver for Databricks to Azure App Serivce

Thomas Ingenhorst 21 Reputation points
2022-01-25T08:32:12.197+00:00

Hi all,

I have a .NET Core Web API which creates a connection to the SQL endpoint of Azure Databricks with the official Simba ODBC driver. Locally everything works fine. I can see my System DSN entry and I use this driver to create the connection and I can succesfully load data from Databricks via SQL ODBC endpoint.

Now I want to deploy my solution to our Azure App Service (windows / .NET 5 / 64 bit) but I get the following error:

168159-image.png

I think the problem is the missing Simba Spark Driver in my Azure App Service?? Is there a way to install this custom driver as an extension? Or is there any other option without using a VM??

Thanks,
Thomas

.NET Machine learning
.NET Machine learning
.NET: Microsoft Technologies based on the .NET software framework.Machine learning: A type of artificial intelligence focused on enabling computers to use observed data to evolve new behaviors that have not been explicitly programmed.
159 questions
Azure App Service
Azure App Service
Azure App Service is a service used to create and deploy scalable, mission-critical web apps.
7,912 questions
0 comments No comments
{count} votes

Accepted answer
  1. Andriy Bilous 11,431 Reputation points MVP
    2022-01-26T20:31:13.827+00:00

    Hello @Thomas Ingenhorst

    Unfortunately you can't install Databricks Simba Spark Driver in my Azure App Service driver when using App Service Windows.

    One possibility is to use Windows or Linux Containers on Azure App Service in which you have control over what drivers or custom software to install.
    Please check the quick start to run a Windows or Linux Container in App Service:
    https://learn.microsoft.com/en-us/azure/app-service/app-service-web-get-started-windows-container
    https://learn.microsoft.com/en-us/azure/app-service/configure-custom-container?pivots=container-linux

    Example of Linux App Service Container Dockerfile:

    # Install Databricks ODBC driver.  
    RUN apt update && apt install -y unixodbc unixodbc-dev freetds-dev sqsh tdsodbc unzip libsasl2-modules-gssapi-mit  
    RUN curl -sL https://databricks.com/wp-content/uploads/drivers-2020/SimbaSparkODBC-2.6.16.1019-Debian-64bit.zip -o databricksOdbc.zip && unzip databricksOdbc.zip  
    RUN dpkg -i SimbaSparkODBC-2.6.16.1019-Debian-64bit/simbaspark_2.6.16.1019-2_amd64.deb  
    RUN export ODBCINI=/etc/odbc.ini ODBCSYSINI=/etc/odbcinst.ini SIMBASPARKINI=/opt/simba/spark/lib/64/simba.sparkodbc.ini  
    

    https://stackoverflow.com/questions/64314613/install-hive-odbc-driver-on-azure-function-app-in-app-service-plan
    https://stackoverflow.com/questions/37441052/odbc-or-oledb-database-drivers-under-azure-app-service


0 additional answers

Sort by: Most helpful

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.