Skip to content
Tutorials Center
My learning

Topics

Lesson 24 of 38 ยท Deep and Hybrid Models

Neural Collaborative Filtering

Replace the dot product with a neural network: embeddings, generalized matrix factorization, an MLP interaction function, negative sampling and a NumPy implementation.

Matrix factorization scores a pair with one fixed formula, the dot product of two vectors. Neural collaborative filtering (NCF) keeps the embeddings but lets a small neural network learn how to combine them. The idea became a standard template: look up an embedding for the user, another for the item, and feed both into a network that outputs the probability of an interaction.

This lesson builds the pieces step by step, trains a small network in plain NumPy so nothing is hidden, and then looks at what NCF really gains (and where it does not beat a well-tuned factorization model).

The architecture

NCF has three parts:

  1. Embedding layers. Two lookup tables turn user ids and item ids into dense vectors, exactly as in matrix factorization.
  2. An interaction function. In classic MF it is a dot product. In NCF it can be:
    • GMF (generalized matrix factorization): the element-wise product of the two vectors followed by learned weights. With all weights equal to 1 it is the dot product.
    • MLP: concatenate the two vectors and pass them through hidden layers with a non-linearity such as ReLU.
    • NeuMF: run a GMF branch and an MLP branch with separate embeddings and join their outputs in a final layer.
  3. An output layer. A sigmoid gives P(interaction), trained with binary cross-entropy against observed interactions (label 1) and sampled non-interactions (label 0).
flowchart LR
  U["user id"] --> EU["user embedding"]
  I["item id"] --> EI["item embedding"]
  EU --> C["concatenate"]
  EI --> C
  C --> H["hidden layer (ReLU)"]
  H --> O["sigmoid output"]
  O --> L["P(interaction)"]

A simple NCF with an MLP interaction function.

Building the training set

Implicit data has only positives, so each epoch you create negatives by sampling items the user has not interacted with. A common ratio is 4 negatives per positive. The label is 1 for the positive and 0 for the sampled ones.

Python Positives and sampled negatives

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

New negatives are drawn each epoch, so the network sees many different "not interested" items over time and does not memorize one fixed sample.

Training an NCF in NumPy

The code below implements both interaction functions with manual backpropagation and Adam. It uses the same synthetic taste-group data as the BPR lesson. The MLP model is: concatenate embeddings, one ReLU hidden layer, one sigmoid output. The GMF model multiplies embeddings element-wise and applies a learned weight vector.

Python GMF vs MLP interaction functions

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Both neural models learn the group structure and beat popularity. Do not expect the MLP to win by a big margin: on this toy data the signal is a dot-product-like structure, which GMF already represents exactly. The fair summary from the research literature is that an MLP is able to represent more complicated interactions, but a carefully tuned dot-product model is hard to beat and is much cheaper to serve.

Why NCF is not always better

A well-known study re-ran the NCF experiments against properly tuned dot-product baselines and found the baselines competitive or better. The reasons are worth remembering:

  • A dot product is a strong inductive bias. It also allows fast retrieval: you can search for the nearest item vectors with an index. An MLP interaction must be evaluated per pair, which is expensive over a large catalog. That is why large systems use the MLP only as a re-ranker over a short candidate list (see two-tower models).
  • Learning a dot product with an MLP is hard. A network needs many units and much data to imitate a multiplication.
  • Small gains need careful evaluation. Use time-based splits and the same hyperparameter budget for every model.

Where neural models help is when you add side features (item text, images, user context): the network takes them as extra inputs, which a plain factorization model cannot do.

Practical notes

  • Frameworks. In real projects you would write this in PyTorch or Keras and let autograd compute the gradients. The manual version above is only for understanding.
  • Regularization. Use weight decay or dropout; embeddings for rare ids overfit easily.
  • Negative sampling ratio. 4 to 10 negatives per positive is a common range; tune it.
  • Initialization. Small random embeddings; pretraining GMF and MLP separately and then combining them (as NeuMF proposed) can help.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. In NCF, what replaces the fixed dot product of matrix factorization?

    NCF keeps embeddings but learns how to combine them with a neural network.
  2. 2. With implicit data, how are label-0 examples created for training NCF?

    Only positives are observed, so negatives are sampled each epoch (for example 4 per positive).
  3. 3. What is GMF?

    GMF generalizes the dot product by weighting each dimension of the element-wise product.
  4. 4. Why are MLP-based scorers usually kept for re-ranking instead of retrieval?

    A dot product can be indexed for nearest-neighbor search; an arbitrary MLP over a pair cannot.

Summary

  • NCF looks up user and item embeddings and feeds them to a neural interaction function.
  • GMF generalizes the dot product, an MLP learns arbitrary interactions, NeuMF combines both.
  • Training uses binary cross-entropy with positives and sampled negatives.
  • Tuned dot-product models are strong baselines and far cheaper to serve.
  • Neural models shine with side features and as re-rankers.

Next: Autoencoders for Recommendation