Connecting an On-Premises SQL Server Database with Azure OpenAI Services

Question

Hello,

We want to integrate our on-premises SQL Server database with Azure OpenAI services to enable a feature in our web application that allows querying the database using natural language (English).

Could you please suggest any documentation or video tutorials to help us achieve this requirement?

Thanks,

Avinash Vyas

Answer 1

If you want to do this today in SQL 2022, you would have to roll quite a bit on your own. For the calls to Open AI, you can implement REST calls with help of CLR procedures. To make use of the data that comes back, you would have to implement your own vector-handling.

If you can be little patient, there are several new features in this space in the upcoming SQL 2025. There is a built-in stored procedure for the REST calls, and there is also vector support, so that you can match the response to the user query with what you have previously have received for whatever you had in your database.

SQL 2025 is currently in private preview. You can read more about it here: https://www.microsoft.com/en-us/sql-server/blog/2024/11/19/announcing-microsoft-sql-server-2025-apply-for-the-preview-for-the-enterprise-ai-ready-database/ This article includes a link to sign up for the preview. Note that your company will need to sign an NDA to be accepted.

Answer 2

Hello Avinash Vyas, per the documentation on using Azure OpenAI with your own data, you'll either have to use a supported data store to be connected directly or upload your files using Azure OpenAI Studio or the ingestion API. In the documentation, I do not see on-premises SQL Server as a supported data store and hence, you would only have to use the ingestion API to bring the content into the service. Source: https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/use-your-data?tabs=ai-search%2Ccopilot#developing-with-azure-openai-on-your-data

Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.