Is it possible to directly connect Azure OpenAI to Azure Databricks tables?

Question

I am hoping to develop a chatbot application with the Azure OpenAI service in Azure Databricks. To do so, I would like the gpt model to have access to the Azure Databricks tables. Rather than making the LLM generate or modify a SQL query to indirectly search for data in Databricks tables, is it possible to directly connect the LLM to said tables?

Answer 1

Hi Cheung, Clarence,

No, Azure OpenAI models (like GPT-4) cannot directly connect to Azure Databricks tables. They are stateless language models and do not have native, persistent connections to external databases or data sources like Databricks.

Detailed Explanation:

1.Azure OpenAI Nature (LLM Scope)

Azure OpenAI (e.g., GPT-4) is a stateless text generation service. It:

·       Receives prompts (text),

·       Returns completions (text),

·       Has no memory between calls (unless implemented via session),

·       And cannot initiate external connections (e.g., to Databricks, SQL, REST APIs).

2.Typical Pattern for Data Access (LangChain or Custom)

To integrate Azure OpenAI with Databricks, you'd follow an architecture like this:

Step-by-step process:

·       User Input → GPT-4 Prompt

·       GPT-4 returns a SQL query or instruction.

·       A middleware (Python, LangChain, Azure Function, or Databricks notebook) executes that query on your Databricks table.

·       The result is then passed back to GPT-4 as context for further responses.

You may build this with:

·       LangChain + Azure OpenAI + Databricks SQL connector

·        Databricks REST API + GPT prompt templating

·        Notebooks with Azure Functions / Flask serving GPT interactions

3.What’s Not Possible

Azure OpenAI:

·       Cannot "see" the data in Databricks unless you provide it via prompt or API.

·       Cannot "connect" to JDBC, Delta Tables, Spark, or anything like that by itself.

What You Can Do Instead:

Use LangChain with Azure OpenAI + Databricks SQL

from langchain.chains import SQLDatabaseChain
from langchain.sql_database import SQLDatabase
from langchain.chat_models import AzureChatOpenAI
# Connect to Databricks SQL
db = SQLDatabase.from_uri("databricks+connector://<your_connection_string>")
llm = AzureChatOpenAI(deployment_name="gpt-4", temperature=0)
chain = SQLDatabaseChain.from_llm(llm, db)
result = chain.run("your prompt here eg: Show me the top 5 customers by revenue")

Hope this helps. Do let us know if you any further queries.

---

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Thank you!