Skip to content

Hybrid search and reranking

Combine keyword search (BM25) with vector search, then rerank the merged candidates with a cross-encoder. This is the highest-leverage RAG upgrade.

Stable: this rarely changes. Reviewed 17 Oct 2026.

Key ideas

BM25 catches exact terms: product codes, names, error messages. Vectors catch paraphrases.
Merge result lists with Reciprocal Rank Fusion (RRF); it needs no score normalisation.
Retrieve wide (50 to 100 candidates), rerank, keep the top 5 to 10.
Rerankers: Cohere Rerank, Voyage, open cross-encoders (bge-reranker), or an LLM as reranker for small sets.

In code

python

def rrf(result_lists, k=60):
    scores = {}
    for results in result_lists:
        for rank, doc_id in enumerate(results):
            scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank + 1)
    return sorted(scores, key=scores.get, reverse=True)

candidates = rrf([bm25_ids, vector_ids])[:50]
top = reranker.rank(query, candidates)[:8]

Common mistakes

Reranking too few candidates; the reranker cannot recover what retrieval missed.