Skip to content
Tutorials Center
My learning

Topics

Lesson 15 of 36 ยท Indexing and ANN Search

Tuning and Benchmarking Vector Indexes

Learn a repeatable method to benchmark ANN indexes: ground truth, recall-versus-cost curves, picking the cheapest setting for a target recall, and checking worst-case queries.

Every ANN index has dials: M, efSearch, nlist, nprobe, number of bits, quantization level. Documentation gives starting values, but the right values depend on your vectors, your hardware and how accurate you need to be. Guessing wastes money (too much recall costs latency and RAM) or quality (too little recall silently loses answers).

This lesson gives you a repeatable method: build ground truth, sweep the dials, plot recall against cost, pick the cheapest setting that meets a target, and check the worst queries, not just the average. The code uses a deterministic cost measure (distance computations) so results are the same on every machine; in a real project you would also time queries on your server.

The method

flowchart TD
    A["1. Sample real vectors and real queries"] --> B["2. Compute exact top-k (ground truth)"]
    B --> C["3. Sweep index parameters"]
    C --> D["4. Record recall and cost for each setting"]
    D --> E["5. Keep the Pareto frontier"]
    E --> F["6. Pick the cheapest setting with recall >= target"]
    F --> G["7. Re-check on held-out queries and the worst queries"]

Seven steps from data to a chosen setting.

  1. Use realistic data. Random vectors behave differently from real embeddings. Use a sample of your own chunks and real (or realistic) user questions. A few hundred queries are enough for stable averages.
  2. Ground truth with a flat index. Exact search over the same vectors, same metric, same k you will use in production.
  3. Sweep. Try a grid of values; build time and memory matter too, so record them.
  4. Measure recall@k and cost (latency per query, throughput, memory).
  5. Pareto frontier. A setting is dominated if another has both higher recall and lower cost. Discard dominated settings.
  6. Pick by requirement: "recall at least 0.95 at the lowest cost".
  7. Validate on queries you did not tune on.

A benchmark harness

We reuse the IVF index idea from the IVF lesson and sweep two dials, nlist (a build-time setting) and nprobe (a query-time setting). Cost is the number of distance computations per query: centroid comparisons plus scanned vectors.

Python Sweep nlist and nprobe, keep the Pareto frontier

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Read the output from top to bottom. The frontier lists only the useful settings, ordered by cost: each one buys more recall for more work. The "cheapest setting" line answers the question you actually have. In this run it is nlist=256, nprobe=4: recall 0.967 for 586 distance computations, 2.9% of exact search. The last line is the habit that saves you from surprises: the average of 0.967 hides at least one query with recall 0.3, and 3% of queries fall below 0.8. If those queries are important (rare topics, long questions), raise the target or look at what they have in common.

What to measure besides recall

Metric Why it matters
Latency percentiles (p50, p95, p99) Users feel tail latency; averages hide it. Measure under realistic concurrency.
Throughput (queries per second) Decides how many servers you need.
Memory RAM per million vectors, including the graph/lists and any rescoring copies.
Build and update time Initial load and incremental inserts.
Filtered search Rerun the benchmark with real metadata filters; selective filters can wreck an index tuned without them.
Recall per query Min and low percentiles, not only the mean.

Time measurement needs care: warm the cache first, discard the first runs, keep the machine otherwise idle, use the real batch size, and report medians of several repetitions.

Typical dials and their effects

Index Build-time dials Query-time dial Raise it to...
HNSW M, efConstruction efSearch gain recall at the cost of RAM, build time, latency (HNSW)
IVF nlist nprobe gain recall at the cost of latency
IVF-PQ / PQ m, bits per code nprobe, rescoring factor trade memory against recall (compression)
LSH bits, tables probes (if multi-probe) see LSH

Prefer tuning query-time dials: you can change them per request or per tenant without rebuilding. Fix build-time dials with a generous margin, because changing them means re-indexing.

From index recall to answer quality

Index recall measures fidelity to exact search, not usefulness. After picking a setting, run your full retrieval evaluation with the ANN index and with a flat index on the same questions. If the two metrics are within noise, extra index recall is wasted. If they differ, spend more on the index. Remember that the embedding model and chunking choices usually move quality more than the last few points of ANN recall.

A short checklist

  • Benchmark on your own vectors and queries, never only on public datasets.
  • Fix the metric and k first; they define ground truth.
  • Report recall with its cost, always as a pair.
  • Re-run the benchmark when data size grows 5 to 10 times, when you change the embedding model, or when you add filters.
  • Keep the benchmark script in the repository so it is repeatable.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What is ground truth in an ANN benchmark?

    Recall is measured against exact search on the same vectors, metric and k.
  2. 2. What does it mean for a setting to be on the Pareto frontier?

    Dominated settings are strictly worse; the frontier is the set of reasonable tradeoffs.
  3. 3. Why look at per-query recall and not only the mean?

    A mean of 0.95 may include queries at 0.5; tail behavior decides whether users notice.
  4. 4. Which kind of dial is easiest to change in production?

    Query-time dials apply per request without rebuilding the index.
  5. 5. Your benchmark used no metadata filters, but production queries filter to 1% of the data. What should you do?

    Selective filters can sharply reduce recall of an index that was tuned on unfiltered queries.

Summary

  • Tune with a method: realistic data, exact ground truth, parameter sweep, recall-versus-cost, Pareto frontier.
  • Choose the cheapest setting that meets a recall target, then validate on held-out queries and on the worst queries.
  • Track latency percentiles, throughput, memory, build time and filtered-query behavior, not recall alone.
  • Prefer query-time dials, keep the benchmark in version control, and re-run it when data, model or filters change.
  • Index recall is only a means; confirm that answer-level retrieval metrics do not suffer.

Next: Loading and Cleaning Documents