Skip to content
Tutorials Center
My learning

Topics

Lesson 3 of 38 ยท Getting Started

The Recommender Pipeline: Candidates and Ranking

See how real recommenders are built in stages: candidate generation, scoring, filtering, re-ranking and serving, with a runnable two-stage example in Python.

A single clever model rarely runs a whole recommendation product. A catalog of ten million items cannot be scored by an expensive model for every request, because the page must load in a fraction of a second. So real systems are pipelines: cheap stages narrow millions of items to hundreds, then expensive stages order those hundreds carefully.

Understanding the pipeline early gives you a map for every later technique. When someone says "we use matrix factorization", the next question is "for which stage?".

The stages

Two-stage recommendation

Catalog Millions of items
Candidate generation Hundreds, fast and cheap
Ranking Careful scoring
The page Ten items shown

Each stage trades recall for speed.

1 / 2
  1. Candidate generation (retrieval). Several cheap sources each propose items: "similar to what you watched", "popular in your country", "new from creators you follow". Goal: do not miss good items (high recall) while keeping the list small.
  2. Scoring / ranking. A richer model uses many features (user, item, context) to predict how likely each candidate is to be clicked, watched or bought.
  3. Filtering. Remove items already consumed, out of stock, age-restricted, or hidden by the user. Often done before scoring to save work.
  4. Re-ranking. Adjust the sorted list for diversity, freshness, fairness or business rules.
  5. Serving and logging. Return the list, and log what was shown (not only what was clicked) so the next model can learn from it.

A two-stage example

We will use a fake catalog of 5,000 items to show why staging matters. Stage one is a cheap dot product on small vectors; stage two is a slightly more expensive scoring function applied only to the shortlist.

Python Cheap retrieval, careful ranking

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The second stage changes the order using information (age, price) that the retrieval stage ignored. If we had run rich_score on all 5,000 items the result would be similar but 50 times slower; at 10 million items it would be impossible.

Multiple candidate sources

Real systems rarely trust one retriever. They merge several and let the ranker decide.

Python Merging candidate sources

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Round-robin merging is only a simple strategy. In practice, each candidate also carries its source as a feature, so the ranker can learn that "from followed creators" is a strong signal.

What each stage optimizes

Stage Cost per item Main goal Typical methods
Retrieval microseconds Recall Nearest neighbors on embeddings, co-occurrence tables, popularity lists
Ranking milliseconds Precision at the top Gradient-boosted trees, neural networks with many features
Re-ranking small Diversity, rules, fairness MMR, calibration, business constraints

You will implement versions of the retrieval methods in chapters 3 to 5, and discuss ranking models and serving in chapters 6 and 8.

Offline versus online

Everything above runs online, while the user waits. But models are trained offline, in batches (nightly or hourly), and the results (embeddings, similarity tables) are loaded into fast storage. This split matters for evaluation too: you test a model offline on historical data first, then confirm with an online experiment (A/B test). Both are discussed in the evaluation chapter.

Logging only clicks is a classic mistake. If you do not store which items were shown, you cannot tell "not clicked" from "never seen", and your next model will learn from a biased picture.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why not score every item in a ten-million-item catalog with the best model for each request?

    Expensive models are applied only to a shortlist produced by a cheap retrieval stage.
  2. 2. What is the primary goal of candidate generation?

    The ranker can only reorder what retrieval gives it, so retrieval must not lose good items.
  3. 3. Which stage typically applies diversity or business rules to the sorted list?

    Re-ranking adjusts the final order for diversity, freshness, fairness or constraints.
  4. 4. Why should you log impressions (items shown) as well as clicks?

    Without impressions you cannot compute click-through rate or learn what users reject.

Summary

  • Production recommenders are pipelines: retrieval, filtering, ranking, re-ranking, serving.
  • Retrieval is cheap and aims for recall; ranking is expensive and aims for precision at the top.
  • Several candidate sources are merged and labelled so the ranker can weigh them.
  • Models train offline; scores are served online.
  • Always log what was shown, not only what was clicked.

Next: Your First Recommender: A Popularity Baseline