SciFact, a public retrieval benchmark, has a test claim that reads "CCL19 is absent within dLNs." CCL19 is a chemokine and dLNs are draining lymph nodes. One abstract in the corpus answers it. I embedded the claim with nomic-embed-text and asked pgvector for the 100 nearest abstracts, and the right one was not among them. A BM25 index put it second, because CCL19 is a rare word and that abstract contains it.

Hybrid search exists for queries like this one. You run the vector search and the keyword search, then merge the two lists with Reciprocal Rank Fusion, in a single SQL statement. pgvector has no keyword ranking of its own, so the keyword half comes from somewhere else in Postgres, and that choice decided most of the result.

The recipe most tutorials show ranks the keyword half with Postgres' built-in ts_rank, and on both datasets I tried, fusing it with vector search scored worse than vector search alone. With a BM25 index in its place, the same fusion scored best of all.

How the two halves rank#

Vector search compares meanings. The question and each document become points in one space, and the nearest points come back. That forgives typos and paraphrases, and it knows nothing special about a gene name. To the embedding model a rare identifier is a few tokens, and an abstract full of related biology can sit closer than the one that names it.

Keyword search compares words. BM25, the ranking function search engines have used for about thirty years, scores a document higher when it contains the query's words and then adjusts three things. A rare word counts more than a common one (that is inverse document frequency), and the tenth occurrence of a word adds less than the first, so repetition stops paying. A long document is pulled down a little, since it contains more words by chance.

Postgres' own full-text search matches well and ranks poorly. ts_rank and ts_rank_cd only look at how often and how close the words appear inside one document. There is no statistic across the corpus, so "the" and "CCL19" weigh the same once stop words are removed. Length only counts if you pass a normalization flag, and the flag divides by this document's length with no idea what is typical for the rest. Tiger's write-up lists the same three gaps.

Merging the lists by adding their scores does not work, because a cosine distance and a BM25 score have unrelated scales and the scale moves with every query. Reciprocal Rank Fusion uses positions only. A document gets 1 / (k + rank) from each list it appears in, and the sums give the order. The formula comes from a 2009 SIGIR paper by Cormack, Clarke and Büttcher, where k = 60 was fixed in a pilot run and never changed. Tiger's hybrid search tutorial and the dev.to example below both still use 60.

The usual recipe joins the words with AND#

The version you find in most guides has a generated tsvector column with a GIN index beside the embedding, one CTE per half, and a FULL OUTER JOIN that adds the reciprocal ranks. The keyword half parses the question with websearch_to_tsquery or plainto_tsquery and ranks with ts_rank or ts_rank_cd. This dev.to post is a tidy example, and this one does the fusion in Python.

Both parsers join the words with AND. In a search box, where people type a few words, that is what they expect. With a question in natural language, a document has to contain every word left after stop words before it matches at all. SciFact's test claims average 12.5 words, and with AND the keyword half returned nothing for 274 of the 300. NFCorpus questions average 3.3 words, and 131 of its 323 still came back empty.

So I measured the recipe twice, once as written and once with the words joined by OR. The change is one line:

-- OR the lexemes plainto_tsquery produces, so a document needs any of them
SELECT replace(plainto_tsquery('english', $1)::text, ' & ', ' | ')::tsquery;

Fusion with ts_rank scored below vector search alone#

The data is two public BEIR datasets with relevance labels: SciFact, with 5,183 scientific abstracts and 300 test claims, and NFCorpus, with 3,633 medical documents and 323 test questions. Everything ran in one PostgreSQL 18.6 instance with pgvector 0.8.4, ParadeDB's pg_search 0.25.11 and Tiger's pg_textsearch 1.5.1. nDCG@10 scores how well the top ten is ordered against the labels, where 1 is perfect, and recall@100 is the share of relevant documents found anywhere in the top 100.

Fusing ts_rank_cd with vector search scored 0.594 on SciFact, against 0.703 for the vector half on its own, and 0.324 against 0.347 on NFCorpus. Recall went up on both, so the fused list does find more relevant documents somewhere in its top 100. It orders the top ten worse, though, and the top ten is what a RAG pipeline puts in the prompt.

On its own (the second row), ts_rank_cd scores 0.330 on SciFact, under half of BM25's 0.688 on the same matches, which makes its order a weak signal next to the vector list. RRF gives both lists the same weight, and the weak one pushes good vector hits down.

With BM25 as the keyword half, fusion helps on both datasets. It scored 0.727 and 0.735 on SciFact, 3 to 5 percent above vector search, and 0.356 and 0.358 on NFCorpus, about 3 percent above. The two BM25 extensions landed within a few thousandths of each other, as two implementations of one formula should. BM25 on its own also lands where the literature puts it, at 0.688 and 0.325 here, against 0.665 to 0.686 for SciFact and 0.325 for NFCorpus in the published BEIR reference runs.

This is what each method cost per query on SciFact, at the median and the 95th percentile:

These are server execution times on a warm cache, without the network or the embedding call. The fused BM25 query costs about as much as vector search alone, because its keyword half takes a fraction of a millisecond. The ts_rank version is the slow one. With OR most documents match at least one word, and ts_rank_cd scores every match before it can sort them, since the GIN index can find matches but has no way to rank them.

The query with a BM25 index#

The table keeps the text and the embedding, plus a generated tsvector for Postgres' own matching in filters. Each BM25 extension then adds its own index type.

CREATE TABLE docs (
  id        text PRIMARY KEY,
  content   text NOT NULL,
  tsv       tsvector GENERATED ALWAYS AS (to_tsvector('english', content)) STORED,
  embedding vector(768) NOT NULL
);
 
CREATE INDEX docs_hnsw ON docs USING hnsw (embedding vector_cosine_ops);

-- needs shared_preload_libraries = 'pg_textsearch'
CREATE EXTENSION pg_textsearch;
CREATE INDEX docs_bm25 ON docs USING bm25 (content)
  WITH (text_config = 'english');

-- needs shared_preload_libraries = 'pg_search'
-- CASCADE also installs pgvector, which pg_search 0.25 depends on
CREATE EXTENSION pg_search CASCADE;
CREATE INDEX docs_bm25 ON docs USING bm25
  (id, (content::pdb.simple('stemmer=english')))
  WITH (key_field = 'id');

The English stemmer on the pg_search index keeps the comparison fair. Without it "runs" does not match "running", and both text_config = 'english' and to_tsvector('english', ...) stem.

The fused query takes the question's text as $1 and its embedding as $2, both computed once by the application.

SELECT coalesce(v.id, k.id) AS id,
       coalesce(1.0 / (60 + v.r), 0) + coalesce(1.0 / (60 + k.r), 0) AS score
FROM (SELECT id, row_number() OVER (ORDER BY d) AS r
      FROM (SELECT id, embedding <=> $2::vector AS d FROM docs
            ORDER BY embedding <=> $2::vector LIMIT 100) s) v
FULL JOIN
     (SELECT id, row_number() OVER (ORDER BY d) AS r
      FROM (SELECT id, bm25_get_current_score() AS d FROM docs
            ORDER BY content <@> to_bm25query($1, 'docs_bm25') LIMIT 100) s) k
  ON v.id = k.id
ORDER BY score DESC, id
LIMIT 10;

SELECT coalesce(v.id, k.id) AS id,
       coalesce(1.0 / (60 + v.r), 0) + coalesce(1.0 / (60 + k.r), 0) AS score
FROM (SELECT id, row_number() OVER (ORDER BY d) AS r
      FROM (SELECT id, embedding <=> $2::vector AS d FROM docs
            ORDER BY embedding <=> $2::vector LIMIT 100) s) v
FULL JOIN
     (SELECT id, row_number() OVER (ORDER BY d DESC) AS r
      FROM (SELECT id, pdb.score(id) AS d FROM docs
            WHERE content ||| $1 ORDER BY pdb.score(id) DESC LIMIT 100) s) k
  ON v.id = k.id
ORDER BY score DESC, id
LIMIT 10;

Each half sorts and limits inside its own subquery. That way it hands over its best 100, and the row_number() window carries its own ORDER BY because SQL makes no promise about the order rows arrive in. The outer SELECT takes the id through coalesce, so a document found by only one half still has one, and the final ORDER BY breaks ties on id so every run returns the same list. In pg_search, ||| matches any of the words, which is the OR behavior websearch_to_tsquery did not give. The halves are subqueries in FROM and not CTEs, for a reason the next section gets to.

Three things made the same query 15 times slower#

My first version of the fused BM25 query took 25.7 ms at the median. The version above takes 1.6 ms and returns the same rows. Three separate problems were in the way, and none of them raised an error.

The planner skipped HNSW. On a table of 5,000 rows Postgres estimated that reading every vector and sorting was cheaper than walking the graph, and picked that plan. It took 16.6 ms at the median, while the forced HNSW scan took 1.5 ms. The estimate depends on table size, so a bigger table may well get the index without help. On a small one, in a test or a new product, check EXPLAIN before trusting a latency number, because this one chose a plan eleven times slower.

pg_textsearch scored every row twice. Its README puts the score in the select list as content <@> to_bm25query(...). In 1.5.1 that ran the BM25 index scan and then scored each of the 100 returned rows again from its text, which took 15.5 ms against 0.5 ms for the same query returning only ids. The extension has a planner hook meant to swap that expression for bm25_get_current_score(), a function that reads the score the index scan already computed, and in my runs the swap did not happen. Calling the function myself brought the query to 0.3 ms. It is not in the README, so I would check that it still exists after an upgrade.

The CTE was materialized. Postgres folds a CTE into the main query when the CTE is used once and calls no volatile function. bm25_get_current_score() is declared VOLATILE, so a WITH k AS (...) around it stayed a separate step. The planner hook could not reach inside it, and the 15 ms came back. With the same half written as a subquery in FROM, the whole fused query went from 20.8 ms to 2.2 ms on the first test claim.

All three showed up in EXPLAIN (ANALYZE, BUFFERS) on the fused query, which is the plan to read before trusting any latency number for hybrid search.

Which BM25 you get depends on where Postgres runs#

When I wrote the book's chapter on what is coming next, in February, two teams were building BM25 for Postgres. By October there are at least five options, and where your database runs decides most of the choice.

pg_search and pg_textsearch both name their access method bm25, so whichever goes second into a database fails with "access method bm25 already exists". They can share a server with one database each, which is how I measured them. Both also need an entry in shared_preload_libraries, so on a managed service you get the extension your provider ships, or none, and its extension list is the first thing to check. Chapter 12 of the book keeps the same kind of table for pgvector, pgvectorscale, pgai and the other AI extensions, provider by provider.

Tuning k moved the score by a hundredth#

In RRF, k sets how fast credit falls off down a list. A small k gives most of the credit to the first few ranks, and a large one flattens the curve so that agreement between the two lists counts for more than position. For the pg_textsearch fusion, nDCG@10 by k came out like this:

k = 10 beat 60 on both, by 0.011 on SciFact and 0.004 on NFCorpus. That is small enough that the default costs little. The keyword half moved the result far more. With ts_rank_cd as that half, the best k on SciFact was 1, which is RRF nearly ignoring the second list.

Bruch, Gai and Ingber argue that a weighted sum of normalized scores beats RRF once the weight is tuned on a few labeled queries. I tried it. I min-max normalized each list, tuned the weight on SciFact's 809 training claims and NFCorpus' dev questions, then scored the test sets. It reached 0.737 and 0.360, the same as RRF at k = 10. With ts_rank_cd it did lift the fusion to 0.709 on SciFact, by putting 0.8 of the weight on the vector list. With labeled queries, tuning either one gets you to the same place, and without them RRF with a BM25 half needs nothing tuned.

Fusion does not win on every query either. Against whichever half happened to be better for a given query, which you cannot know in advance, the pg_textsearch fusion came out ahead on 25 SciFact claims and behind on 71. On NFCorpus it was ahead on 58 and behind on 125, and below both halves on 14. The fair comparison is with vector search alone, and there it was ahead on 67 SciFact claims and behind on 38, and ahead on 112 NFCorpus questions and behind on 83. That is where the average gain comes from. On SciFact, 11 claims had their abstract missing from the vector half's 100 and found by fusion, at ranks from 7th to 65th. CCL19 was the 7th.

In one of the losses, for "DMRT1 is a sex-determining gene that is epigenetically regulated by the MHM region", pg_textsearch puts the right abstract second, the vector half does not have it in its 100, and after fusion it sits 36th. The other way round, one claim misspells metastases as "matasteses". The vector half puts the right abstract first. BM25 cannot match the misspelled word at all, so fusion drops the abstract to 23rd. A document found by one half gets at most half the credit of a document both halves found somewhere, and RRF cannot tell which half was right for this query.

Where it stops#

Both datasets are small, 5,183 and 3,633 documents. I ran everything on a laptop with Docker limited to 1 GB of memory, which was too small for the 57,000-document FiQA set I had planned. The quality comparison holds at this size. The latency numbers are for tables that fit in memory, and a table of millions of rows probably spends its time elsewhere, starting with HNSW cache misses.

There was one embedding model, nomic-embed-text, picked because it runs locally. A stronger model raises the vector baseline and leaves BM25 less to add, so run the same evaluation with your own model before you decide a second index is worth it.

Both corpora are English, and so is the stemming. For Turkish, Postgres ships a turkish text search configuration and the BM25 extensions take whatever tokenizer you give them. I have not measured how either copes with Turkish suffixes.

Hybrid search also inherits a filtering problem. If the query has WHERE tenant_id = $3 as well, an HNSW scan can return fewer than 100 rows after the filter. That is pgvector's most discussed issue, and it needs a post of its own.

How I measured, so you can argue with it#

One container ran ParadeDB's paradedb/paradedb:0.25.11-pg18 image (PostgreSQL 18.6, pgvector 0.8.4, pg_search 0.25.11), with pg_textsearch 1.5.1 installed from Tiger's release package. pg_textsearch and pg_search lived in two databases on that server because of the bm25 name clash. The machine is an Apple M2, and Docker had 8 CPUs and 1 GB of memory. The settings were shared_buffers = 256MB, maintenance_work_mem = 256MB, jit = off and hnsw.ef_search = 200, with HNSW at pgvector's defaults. Every vector number in the tables uses the HNSW index, forced with enable_seqscan = off for those queries, because on tables this small the planner preferred an exact scan. The exact scan scored the same 0.703 on SciFact and 0.347 on NFCorpus.

The data is BEIR's SciFact and NFCorpus as published. Each document is its title and text joined by a newline, embedded once through Ollama 0.34.4 with nomic-embed-text and the search_document: prefix. Questions used search_query:, and I raised num_ctx to 8192 so nothing was truncated. Every method returned its top 100 from one SQL statement per question.

Scores come from pytrec_eval, as nDCG@10 and recall@100 on each dataset's test split, with a query that returned nothing counted as zero. The SQL fusion matched the same RRF computed in Python on 599 of the 600 test questions, and the one difference is a tie broken the other way. Latency is the Execution Time from EXPLAIN (ANALYZE, TIMING OFF), the median of three runs per question after one run to warm the cache, with percentiles taken over the test questions. I measured all of it on 2 and 3 October 2026.

import pytrec_eval
 
def score(runs, qrels):
    """runs: {query: [(doc, score), ...]} best first; qrels: {query: {doc: grade}}"""
    as_run = {q: {d: float(len(r) - i) for i, (d, _) in enumerate(r)} for q, r in runs.items()}
    ev = pytrec_eval.RelevanceEvaluator(qrels, {"ndcg_cut.10", "recall.100"})
    res = ev.evaluate({q: r for q, r in as_run.items() if q in qrels})
    n = len(qrels)
    ndcg = sum(res.get(q, {}).get("ndcg_cut_10", 0.0) for q in qrels) / n
    recall = sum(res.get(q, {}).get("recall_100", 0.0) for q in qrels) / n
    return ndcg, recall
 
def rrf(a, b, k=60):
    out = {}
    for q in set(a) | set(b):
        s = {}
        for lst in (a.get(q, []), b.get(q, [])):
            for i, (d, _) in enumerate(lst):
                s[d] = s.get(d, 0.0) + 1.0 / (k + i + 1)
        out[q] = sorted(s.items(), key=lambda x: (-x[1], x[0]))[:100]
    return out

import requests
 
def embed(texts, prefix):
    r = requests.post("http://localhost:11434/api/embed", json={
        "model": "nomic-embed-text",
        "input": [prefix + t for t in texts],
        "options": {"num_ctx": 8192},
    })
    r.raise_for_status()
    return r.json()["embeddings"]
 
# documents: embed([title + "\n" + text, ...], "search_document: ")
# questions: embed([text, ...], "search_query: ")

FROM paradedb/paradedb:0.25.11-pg18
USER root
COPY pg-textsearch-postgresql-18_1.5.1-1_arm64.deb /tmp/pgts.deb
RUN dpkg -i /tmp/pgts.deb && rm /tmp/pgts.deb
USER postgres
# run with: -c shared_preload_libraries=pg_search,pg_cron,pg_textsearch

The book covers the longer version of this, search and RAG on one Postgres, and a free sample chapter is there if you want to see how it reads first.