Can a .NET C# application connect and query from an Azure Databricks or Azure Synapse Spark Delta Lake table?

Question

Is there a recommended way for a .NET C# application to connect and query from an Azure Databricks or Azure Synapse Spark Delta Lake table? Perhaps, there is a framework available we can use?

Can a .NET C# app connect via JDBC or ODBC connection to the delta lake database and query from a table?
Ref: https://stackoverflow.com/questions/66750474/simba-odbc-connection-to-delta-table-read-data-from-delta-format-tables-using

If there is none yet, alternatively, is there a recommended way for a .NET C# application to connect and query from a Data Lake parquet file?
Ref: https://www.reddit.com/r/dotnet/comments/9enqz7/open_source_c_net_libraries_for_reading_and/

Thank you.

Answer 1

Hi @Catameo, Fred (Taguig City 1634) ,

Here is a NuGet package available for Delta support in C# : .Net for Apache Spark Extensions

Could you please try this and see if it helps to achieve your requirement.

And also Synapse supports authoring your Spark applications in .NET. Here are a couple of links that might be useful:

• Use .NET for Apache Spark - Azure Synapse Analytics | Microsoft Learn
• Synapse/Hitchhikers Guide to DotNET for Apache Spark.ipynb at main · Azure-Samples/Synapse · GitHub
• Synapse/Hitchikers Guide to Delta Lake - CSharp.ipynb at main · Azure-Samples/Synapse · GitHub

The notebook linked in the second bullet above shows you how you can do basic IO and the third bullet above shows you how you can read a Delta Lake table.

Hope this info helps. Let us know how it goes.

Thanks

----------

Please don’t forget to Accept Answer and Up-Vote wherever the information provided helps you, this can be beneficial to other community members.

Answer 2

Also you can use any SQL Server client to query the Data Lake through Synapse Serverless SQL Pool.