Hybrid search using vectors and full text in Azure AI Search
Hybrid search is a combination of full text and vector queries that execute against a search index containing both searchable plain text content and generated embeddings. For query purposes, hybrid search is:
- A single query request that includes both
searchandvectorsquery parameters - Executing in parallel
- With merged results in the query response, scored using Reciprocal Rank Fusion (RRF)
This article explains the concepts, benefits, and limitations of hybrid search. Watch this embedded video for an explanation and short demos of how hybrid retrieval contributes to high quality chat-style and copilot apps.
How does hybrid search work?
In Azure AI Search, vector fields containing embeddings can live alongside textual and numerical fields, allowing you to formulate hybrid queries that execute in parallel. Hybrid queries can take advantage of existing functionality like filtering, faceting, sorting, scoring profiles, and semantic ranking in a single search request.
Hybrid search combines results from both full text and vector queries, which use different ranking functions such as BM25, HNSW, and EKNN. A Reciprocal Rank Fusion (RRF) algorithm merges the results. The query response provides just one result set, using RRF to rank the unified results.
Structure of a hybrid query
Hybrid search is predicated on having a search index that contains fields of various data types, including plain text and numbers, geo coordinates if you want geospatial search, and vectors for a mathematical representation of a chunk of text. You can use almost all query capabilities in Azure AI Search with a vector query, except for pure text client-side interactions such as autocomplete and suggestions.
A representative hybrid query might be as follows (notice that the vector queries have placeholder values for brevity):
POST https://{{searchServiceName}}.search.windows.net/indexes/hotels-vector-quickstart/docs/search?api-version=2024-07-01
content-type: application/JSON
{
"count": true,
"search": "historic hotel walk to restaurants and shopping",
"select": "HotelId, HotelName, Category, Description, Address/City, Address/StateProvince",
"filter": "geo.distance(Location, geography'POINT(-77.03241 38.90166)') le 300",
"facets": [ "Address/StateProvince"],
"vectors": [
{
"value": [ <array of embeddings> ]
"k": 7,
"fields": "DescriptionVector"
},
{
"value": [ <array of embeddings> ]
"k": 7,
"fields": "Description_frVector"
}
],
"queryType": "semantic",
"queryLanguage": "en-us",
"semanticConfiguration": "my-semantic-config"
}
Key points include:
searchspecifies a single full text search query.vectorsfor vector queries, which can be multiple, targeting multiple vector fields. If the embedding space includes multi-lingual content, vector queries can find the match with no language analyzers or translation required.selectspecifies which fields to return in results, which should be text fields that are human readable.filterscan specify geospatial search or other include and exclude criteria, such as whether parking is included. The geospatial query in this example finds hotels within a 300-kilometer radius of Washington D.C.facetscan be used to compute facet buckets over results that are returned from hybrid queries.queryType=semanticinvokes semantic ranker, applying machine reading comprehension to surface more relevant search results. Semantic ranking is optional. If you aren't using that feature, remove the last three lines of the hybrid query.
Filters and facets target data structures within the index that are distinct from the inverted indexes used for full text search and the vector indexes used for vector search. As such, when filters and faceted operations execute, the search engine can apply the operational result to the hybrid search results in the response.
Notice how there's no orderby in the query. Explicit sort orders override relevanced-ranked results, so if you want similarity and BM25 relevance, omit sorting in your query.
A response from the above query might look like this:
{
"@odata.count": 3,
"@search.facets": {
"Address/StateProvince": [
{
"count": 1,
"value": "NY"
},
{
"count": 1,
"value": "VA"
}
]
},
"value": [
{
"@search.score": 0.03333333507180214,
"@search.rerankerScore": 2.5229012966156006,
"HotelId": "49",
"HotelName": "Swirling Currents Hotel",
"Description": "Spacious rooms, glamorous suites and residences, rooftop pool, walking access to shopping, dining, entertainment and the city center.",
"Category": "Luxury",
"Address": {
"City": "Arlington",
"StateProvince": "VA"
}
},
{
"@search.score": 0.032522473484277725,
"@search.rerankerScore": 2.111117362976074,
"HotelId": "48",
"HotelName": "Nordick's Valley Motel",
"Description": "Only 90 miles (about 2 hours) from the nation's capital and nearby most everything the historic valley has to offer. Hiking? Wine Tasting? Exploring the caverns? It's all nearby and we have specially priced packages to help make our B&B your home base for fun while visiting the valley.",
"Category": "Boutique",
"Address": {
"City": "Washington D.C.",
"StateProvince": null
}
}
]
}
Why choose hybrid search?
Hybrid search combines the strengths of vector search and keyword search. The advantage of vector search is finding information that's conceptually similar to your search query, even if there are no keyword matches in the inverted index. The advantage of keyword or full text search is precision, with the ability to apply optional semantic ranking that improves the quality of the initial results. Some scenarios - such as querying over product codes, highly specialized jargon, dates, and people's names - can perform better with keyword search because it can identify exact matches.
Benchmark testing on real-world and benchmark datasets indicates that hybrid retrieval with semantic ranker offers significant benefits in search relevance.
The following video explains how hybrid retrieval gives you optimal grounding data for generating useful AI responses.
See also
Outperform vector search with hybrid retrieval and ranking (Tech blog)