Skip to content
Tutorials Center
My learning

Topics

Lesson 25 of 38 ยท Deep and Hybrid Models

Autoencoders for Recommendation

Treat each user's history as a vector, compress and reconstruct it: denoising autoencoders for recommendations, the closed-form linear model EASE, and runnable NumPy code.

Matrix factorization learns one vector per user. An autoencoder takes a different view: a user is simply their row of the interaction matrix, a long vector with a 1 for every item they used. A neural network squeezes that vector into a small code (the encoder) and then tries to rebuild the original from the code (the decoder). Items that the network reconstructs with a high score although the user never touched them become recommendations.

Two ideas come from this family: AutoRec-style neural autoencoders, and a surprisingly strong linear cousin called EASE that you can solve in a few lines of linear algebra. You will build both here.

From reconstruction to recommendation

Suppose a user's history is x = [1, 0, 1, 0, 0, 1]. The autoencoder outputs xฬ‚ = [0.9, 0.5, 0.8, 0.1, 0.0, 0.7]. The second item has a score of 0.5 although the user never used it. The network inferred it from the pattern "people who have items 1, 3 and 6 usually also have item 2". That inferred score is the recommendation score.

The danger is the trivial solution: if the code is as large as the input, the network can copy the input and learn nothing useful. Three standard remedies exist:

  • A bottleneck (hidden layer much smaller than the item count).
  • Denoising: randomly zero out part of the input and ask the network to restore the full vector. Only item-item patterns help with that, not copying.
  • Regularization of the weights, or, as in EASE, forbidding the identity mapping outright.

How an autoencoder recommends

User history A 0/1 vector over all items
Encode Squeeze into a small code
Decode Rebuild every item score
Rank unseen Highest scores win

Noise in the input forces real patterns to be learned.

1 / 2

A denoising autoencoder in NumPy

We reuse the three-taste-group data from the previous lessons. The model is xฬ‚ = W2 ยท relu(W1 ยท xฬƒ + b1) + b2, where xฬƒ is the input with 30% of the ones randomly dropped. The loss is the squared error against the clean vector. The network is small enough to train with hand-written gradients.

Python Denoising autoencoder for recommendations

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Training uses the corrupted input, but scoring uses the clean history, so the network does what it practiced: restore missing entries. Mask out items the user already has before ranking (the recall_at_10 function does it), otherwise the top of every list is the user's own history.

EASE: a linear autoencoder with a closed form

If you remove the hidden layer and the non-linearity, the model becomes xฬ‚ = x ยท B, where B is an item-by-item weight matrix. Fitting B to reproduce X would give the identity matrix, which is useless. EASE (Embarrassingly Shallow Autoencoder) adds one constraint: the diagonal of B must be zero, so an item cannot predict itself. With L2 regularization, the exact solution is:

TEXT
G = Xแต€X + ฮป I          P = Gโปยน
B = I โˆ’ P / diag(P)    (column-wise division), then set diag(B) = 0

That is one matrix inversion, no gradient descent, no learning rate. For catalogs of tens of thousands of items it is fast and often competitive with deep models on implicit-feedback benchmarks.

Python EASE in five lines

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The regularization strength ฮป matters a lot: tiny values overfit and large values flatten everything toward popularity. In this run ฮป = 100 scores best (recall about 0.55) and both smaller and larger values are worse; in a real project choose ฮป on a validation split. Notice also what B is: an item-item similarity matrix learned from data, so EASE connects to the item-based neighborhood methods of the collaborative filtering chapter (see the track overview), with weights fitted instead of computed by a fixed formula.

Practical notes

  • Input dropout is the key knob for neural autoencoders. Too little and they copy; too much and they cannot see the pattern.
  • Mult-VAE. A variational autoencoder with a multinomial likelihood is a popular deep variant; it models the user's whole history as a probability distribution over items.
  • Memory. EASE inverts an item-by-item matrix, so it is practical up to a few tens of thousands of items. Beyond that, restrict the catalog or use a low-rank method.
  • Both models are user-history based: a new user needs no retraining, their history vector is simply the input.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why does a denoising autoencoder zero out part of the input?

    If the network can see the full input it can copy it. Hiding some entries makes it infer them from the rest.
  2. 2. How does an autoencoder turn reconstructions into recommendations?

    High reconstructed scores for unseen items mean the pattern says the user would likely use them.
  3. 3. What makes EASE avoid learning the identity mapping?

    With diag(B) = 0 an item cannot reconstruct itself, so it must be predicted from other items.
  4. 4. Which is true about EASE?

    EASE solves a regularized least-squares problem exactly with an inverse of the item Gram matrix.

Summary

  • Autoencoders reconstruct a user's interaction vector; scores for unseen items are recommendations.
  • Bottlenecks, input corruption and regularization stop the network from copying.
  • EASE is a linear autoencoder with zero diagonal and an exact closed-form solution.
  • Its weight matrix is a learned item-item similarity.
  • New users need no retraining because their history is the input.

Next: Two-Tower Models and Embeddings