How to export Databricks dataframe(Pandas or Scala) to Synapse analyitcs SQL table using Active Directory Password

Pradeep Thotakura 1 Reputation point
2021-07-09T09:59:44.493+00:00

Hi Team,

I am new to the Azure, I need your expertise in Moving Dataframe from Databricks to Synapse Analytics table. We have implemented a classification model in Databricks and predicted the values and created the dataframe with Actual and predicted values. Now we need to move that dataframe to Synapse Analytics with Active Directory Password in JDBC connection. I have tried many ways but those are not working out. Could you please help me how to move data.

Thanks in Advance.

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,320 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
1,901 questions
{count} votes

1 answer

Sort by: Most helpful
  1. PRADEEPCHEEKATLA-MSFT 76,361 Reputation points Microsoft Employee
    2021-07-15T05:52:03.44+00:00

    Hello @Pradeep Thotakura ,

    Thanks for the sharing the details.

    Unfortunately, AAD credentials as basic authentication is not supported in Azure Synapse Analytics.

    If you want to use user identities (and not SPN, for which there seems to be support - OAuth 2.0 with a service principal). The username/password doc should apply to sql auth support (sql users created within sql itself).

    The following sections describe each connection’s authentication configuration options.

    114779-image.png

    The Spark driver can connect to Azure Synapse using JDBC with a username and password or OAuth 2.0 with a service principal for authentication.

    I had clearly explained the both the methods here: Azure databricks to Azure synapse Service Principal Authentication.

    Hope this helps. Do let us know if you any further queries.

    ---------------------------------------------------------------------------

    Please "Accept the answer" if the information helped you. This will help us and others in the community as well.