Applies to: 
SQL Server 2025 (17.x)
Azure SQL Database
Azure SQL Managed Instance
SQL database in Microsoft Fabric
This article contains frequently asked questions about vectors and embeddings in the SQL Database Engine.
For samples and examples, visit the SQL AI Samples repository.
Can I create a retrieval-augmented generation (RAG) solution completely in T-SQL?
Yes, you can create a Retrieval-Augmented Generation (RAG) solution driven by native functionality in the SQL Database Engine. You can use T-SQL to implement the necessary data retrieval and processing logic, while also integrating with external AI services for the generation aspect. Vectors can be stored natively in SQL engine and connections to LLMs that provide natural language understanding capabilities are possible via sp_invoke_external_rest_endpoint.
Why would I create a RAG solution completely in T-SQL?
If you want to improve an existing application without having to re-architect it to support AI capabilities, use the SQL engine built-in features to implement AI functionalities directly within your database queries. You only need to update your T-SQL code to incorporate AI features, rather than making extensive changes to your application architecture.
Are there any end-to-end samples using Azure SQL or Fabric SQL for RAG?
Sure, you can find end-to-end samples for RAG using Azure SQL and Fabric SQL here:
Can I have RAG working on structured data, like columns and rows?
If you need to work with structured data, you can still leverage RAG by combining it with other techniques, such as using embeddings to represent your structured data in a way that can be understood by the AI model. This allows you to perform retrieval and generation tasks on structured data while still benefiting from the capabilities of RAG.
Why does sending a full, complex schema to an LLM lead to poor SQL generation, and how can I fix it?
If you have a complex and large database schema, with hundreds of tables and views, it's better to use a multi-agent approach to help reduce the noise and allow AI models to focus on specific areas of the schema. A full description along with a working end-to-end sample is available here:
Can I connect to Azure OpenAI using Managed Identity?
Yes, you can connect to Azure OpenAI using Managed Identity. This allows you to securely authenticate and access the Azure OpenAI Service without needing to manage credentials directly. For more information, see:
Is my data used by Microsoft for training models?
No. Data isn't used by Microsoft for training models. For more information, see Responsible AI documentation.
What data does the Azure OpenAI Service process?
For details on how data provided by you to Azure Direct Models in Microsoft Foundry are processed, see Data, privacy, and security for Azure OpenAI Service. An "Azure Direct Model" is an AI model designated and deployed as an “Azure Direct Model” in Foundry, and includes Azure OpenAI models.
How can I protect my data from unauthorized AI Agent access?
Azure SQL and SQL Server provide extensive support for fine-grained access security:
- Get started with Database Engine permissions: Control access to database objects at a granular level using permissions.
- Use stored procedures that perform specifically-authorized operations within guardrails. Grant EXECUTE permissions to an agent only as needed, instead of granting direct access to the underlying tables. In this way, agents interact with the database deterministically, using pre-written T-SQL statements.
- Row-Level Security (RLS): Control access to rows in a table based on the characteristics of the user executing a query. You can see RLS in action in this video.
- Dynamic data masking: Limit the exposure of sensitive data by masking it to non-privileged users.
- Always Encrypted: Protect sensitive data by encrypting it at rest and in transit, ensuring that only authorized users can access the unencrypted data.
For more information on auditing in the SQL Database Engine, see: