Edit

Relevance scoring in hybrid search using Reciprocal Rank Fusion (RRF)

  • Article

Reciprocal Rank Fusion (RRF) is an algorithm that evaluates the search scores from multiple, previously ranked results to produce a unified result set. In Azure AI Search, RRF is used whenever there are two or more queries that execute in parallel. Each query produces a ranked result set, and RRF is used to merge and homogenize the rankings into a single result set, returned in the query response. Examples of scenarios where RRF is always used include hybrid search and multiple vector queries executing concurrently.

RRF is based on the concept of reciprocal rank, which is the inverse of the rank of the first relevant document in a list of search results. The goal of the technique is to take into account the position of the items in the original rankings, and give higher importance to items that are ranked higher in multiple lists. This can help improve the overall quality and reliability of the final ranking, making it more useful for the task of fusing multiple ordered search results.

How RRF ranking works

RRF works by taking the search results from multiple methods, assigning a reciprocal rank score to each document in the results, and then combining the scores to create a new ranking. The concept is that documents appearing in the top positions across multiple search methods are likely to be more relevant and should be ranked higher in the combined result.

Here's a simple explanation of the RRF process:

  1. Obtain ranked search results from multiple queries executing in parallel.

  2. Assign reciprocal rank scores for result in each of the ranked lists. RRF generates a new @search.score for each match in each result set. For each document in the search results, the engine assigns a reciprocal rank score based on its position in the list. The score is calculated as 1/(rank + k), where rank is the position of the document in the list, and k is a constant, which was experimentally observed to perform best if it's set to a small value like 60. Note that this k value is a constant in the RRF algorithm and entirely separate from the k that controls the number of nearest neighbors.

  3. Combine scores. For each document, the engine sums the reciprocal rank scores obtained from each search system, producing a combined score for each document. 

  4. The engine ranks documents based on combined scores and sorts them. The resulting list is the fused ranking.

Only fields marked as searchable in the index, or searchFields in the query, are used for scoring. Only fields marked as retrievable, or fields specified in select in the query, are returned in search results, along with their search score.

Parallel query execution

RRF is used anytime there's more than one query execution. The following examples illustrate query patterns where parallel query execution occurs:

  • A full text query, plus one vector query (simple hybrid scenario), equals two query executions.
  • A full text query, plus one vector query targeting two vector fields, equals three query executions.
  • A full text query, plus two vector queries targeting five vector fields, equals 11 query executions

Scores in a hybrid search results

Whenever results are ranked, @search.score property contains the value used to order the results. Scores are generated by ranking algorithms that vary for each method. Each algorithm has its own range and magnitude.

The following chart identifies the scoring property returned on each match, algorithm, and range of scores for each relevance ranking algorithm.

Search method Parameter Scoring algorithm Range
full-text search @search.score BM25 algorithm No upper limit.
vector search @search.score HNSW algorithm, using the similarity metric specified in the HNSW configuration. 0.333 - 1.00 (Cosine), 0 to 1 for Euclidean and DotProduct.
hybrid search @search.score RRF algorithm Upper limit is bounded by the number of queries being fused, with each query contributing a maximum of approximately 1 to the RRF score. For example, merging three queries would produce higher RRF scores than if only two search results are merged.
semantic ranking @search.rerankerScore Semantic ranking 0.00 - 4.00

Semantic ranking doesn't participate in RRF. Its score (@search.rerankerScore) is always reported separately in the query response. Semantic ranking can rerank full text and hybrid search results, assuming those results include fields having semantically rich content.

Number of ranked results in a hybrid query response

By default, if you aren't using pagination, the search engine returns the top 50 highest ranking matches for full text search, and the most similar k matches for vector search. In a hybrid query, top determines the number of results in the response. Based on defaults, the top 50 highest ranked matches of the unified result set are returned.

Often, the search engine finds more results than top and k. To return more results, use the paging parameters top, skip, and next. Paging is how you determine the number of results on each logical page and navigate through the full payload.

Full text search is subject to a maximum limit of 1,000 matches (see API response limits). Once 1,000 matches are found, the search engine no longer looks for more.

For more information, see How to work with search results.

Diagram of a search scoring workflow

The following diagram illustrates a hybrid query that invokes keyword and vector search, with boosting through scoring profiles, and semantic ranking.

Diagram of prefilters.

A query that generates the previous workflow might look like this:

POST https://{{search-service-name}}.search.windows.net/indexes/{{index-name}}/docs/search?api-version=2023-11-01
Content-Type: application/json
api-key: {{admin-api-key}}
{
   "queryType":"semantic",
   "search":"hello world",
   "searchFields":"field_a, field_b",
   "vectorQueries": [
       {
           "kind":"vector",
           "vector": [1.0, 2.0, 3.0],
           "fields": "field_c, field_d"
       },
       {
           "kind":"vector",
           "vector": [4.0, 5.0, 6.0],
           "fields": "field_d, field_e"
       }
   ],
   "scoringProfile":"my_scoring_profile"
}

See also