Skip to content
Tutorials Center
My learning

Topics

Lesson 26 of 38 ยท Deep and Hybrid Models

Two-Tower Models and Embeddings

Build a two-tower retrieval model: a user tower and an item tower, in-batch softmax training, fast dot-product search, and item features that help brand-new items.

Production recommenders must choose a handful of items out of millions in a few milliseconds. The architecture that makes this possible is the two-tower model (also called a dual encoder). One neural network, the user tower, turns everything known about the user into a vector. A second network, the item tower, turns everything known about an item into a vector of the same size. The score is the dot product of the two vectors.

Because the towers never look at each other, you can compute every item vector in advance, store them in a nearest-neighbor index, and answer a request with one user-tower call plus one fast index lookup. This is the standard design for the candidate generation stage of the recommendation pipeline.

What goes into each tower

A tower is just a function from features to an embedding. Inputs can be anything you can encode as numbers:

  • User tower: the ids of recently used items (or their average embedding), device, country, time of day, long-term interests.
  • Item tower: the item id, category or genre, text or image embeddings, age of the item, price.

Matrix factorization is a degenerate two-tower model where each tower is a single embedding lookup by id. Real towers are deeper networks and take rich features. The big practical win is that item features let the model embed an item it has never seen in training.

flowchart LR
  UF["user features<br/>history, context"] --> UT["user tower"]
  IF["item features<br/>id, genre, text"] --> IT["item tower"]
  UT --> UE["user embedding"]
  IT --> IE["item embedding"]
  UE --> DOT["dot product"]
  IE --> DOT
  DOT --> S["score"]

The towers meet only at the final dot product.

Training with in-batch negatives

Training data is a list of positive pairs (user, item they used). The model needs negatives too. A cheap and very effective trick is in-batch negatives: take a batch of B positive pairs; for user k the item k is the positive, and the other Bโˆ’1 items in the same batch serve as negatives. The loss is a softmax cross-entropy over the B ร— B score matrix, where the correct answer for row k is column k.

Two cautions: popular items appear as negatives more often than their real frequency would imply for rare items (practitioners correct the logits with log(sampling probability)), and the same item appearing twice in a batch creates a false negative.

The next example uses linear towers so the gradients fit on a few lines. The item tower sees a genre code (6 numbers) plus a one-hot id; the user tower sees the user's history, with the target item removed so the model cannot cheat.

Python A tiny two-tower model

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Both embedding tables come out of the training loop as plain arrays. Notice the division of labor at serving time: item_emb is built offline once, and online you only compute one row of user_emb and take a top-k of its dot products with item_emb.

Retrieval: exact and approximate

With 60 items you can multiply the user vector by every item vector. With 50 million you cannot, so production systems use approximate nearest neighbor (ANN) indexes (HNSW graphs, IVF clusters, product quantization). The serving lesson covers them. Note that this works only because the score is a plain dot product, which is why the tower design does not include a learned interaction network.

A new item with no history

An id-only embedding cannot represent an item that was never in the training data. The feature-based item tower can: its genre part was trained on many other items. The block below reuses the trained towers (the training loop is repeated in compact form), then builds the embedding of a brand-new item that has only genre features and an all-zero id, and checks which users it would be shown to.

Python Embedding an item that has never been seen

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A random choice would give about one third; here every share is 100%, which means the genre columns of the item tower learned what each group likes, so the model can place a new item sensibly before a single interaction exists.

Design tips

  • Normalize the embeddings (cosine similarity) and use a learned temperature in the softmax; this stabilizes training and makes index search better behaved.
  • Same dimensionality, different weights. Towers can share layers or embeddings (for example, the history items in the user tower reuse the item-id embeddings).
  • Sampling-bias correction. Subtract log q(item) (the probability an item is drawn as a negative) from its logit when using in-batch negatives.
  • Freshness. Refresh item embeddings and the index regularly; the user tower runs live so its output always reflects recent behavior.
  • Re-ranking. The first-stage dot product is coarse. A heavier model that sees user and item together (an NCF-style network) re-ranks the few hundred retrieved items.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why can a two-tower model retrieve from millions of items quickly?

    The towers meet only at the dot product, so item vectors are computed offline and indexed.
  2. 2. In in-batch negative training with batch size B, how many negatives does each positive pair get?

    Each row of the BxB score matrix uses the other items of the batch as negatives.
  3. 3. What advantage do item features give over id-only embeddings?

    Feature columns such as genre or text were learned from other items, so a new item with those features gets a useful vector.
  4. 4. Why remove the target item from the history in the user tower input during training?

    If the target is part of the input, the model can match it trivially and learns nothing about the user.

Summary

  • A two-tower model embeds users and items with separate networks and scores with a dot product.
  • Item vectors are precomputed and searched with an ANN index; the user tower runs per request.
  • In-batch softmax gives cheap negatives; correct for sampling bias and duplicates.
  • Item and user features let the model handle new items and rich context.
  • Use it for candidate generation, then re-rank with a heavier model.

Next: Hybrid Recommenders