Skip to content
Tutorials Center
My learning

Topics

Lesson 22 of 38 · Matrix Factorization

ALS and Implicit Feedback

Learn alternating least squares (ALS) for implicit data such as clicks and plays: preferences, confidence weights, the closed-form solves, and a working NumPy version.

Most real recommender data has no star ratings. You have clicks, plays, purchases and watch time. A missing entry does not mean "dislike": it usually means "never saw it". SGD on observed ratings, as in the previous lesson, does not fit this setting, because there are no observed negatives to learn from.

Alternating least squares (ALS) with confidence weights is the standard answer. It treats every cell as a weak statement ("probably not interested") and every interaction as a stronger one ("probably interested"), then solves for the factors with linear algebra instead of gradient steps. It was introduced for implicit feedback by Hu, Koren and Volinsky, and open-source libraries such as implicit and Spark MLlib are built around it.

Preference and confidence

Turn raw interaction counts r(u,i) into two matrices:

  • Preference p(u,i): 1 if the user interacted (r > 0), otherwise 0.
  • Confidence c(u,i) = 1 + α · r(u,i): how sure we are about that preference. An unobserved cell still has confidence 1, a song played ten times has a much higher one.

The model predicts the preference with a dot product, p̂(u,i) = x_u · y_i, and minimizes:

TEXT
sum over ALL (u,i) of  c(u,i) (p(u,i) - x_u · y_i)²  +  λ ( Σ|x_u|² + Σ|y_i|² )

Note "all": the sum includes the zeros. That looks expensive (users times items terms), but the algebra below avoids visiting the zeros one by one.

Implicit data in two steps

Interaction counts Plays, clicks, buys
Preference 0 or 1 Did anything happen?
Confidence 1 + α·r How sure are we?

Zeros stay in the loss, just with low weight.

1 / 2

The closed-form solve

If the item factors Y (a matrix with one row per item) are fixed, the loss is a plain weighted least-squares problem for each user separately. The solution for user u is:

TEXT
x_u = (YᵀY + Yᵀ (C_u − I) Y + λ I)⁻¹  Yᵀ C_u p(u)

where C_u is the diagonal matrix of that user's confidences. The trick is that C_u − I is non-zero only for items the user interacted with. So YᵀY is computed once per half-step and shared by every user, and each user only adds a small correction from their own few items. Cost per user depends on how many interactions they have, not on the catalog size.

The item step is identical with the roles swapped. Because each user's solve does not depend on any other user's, the work parallelizes perfectly, which is the main reason ALS is popular for big data.

A working implementation

The data below is synthetic: 120 users in three taste groups listen to 60 items, and each user has 20% of their plays hidden for testing. We factor the remaining plays and check how often a hidden item appears in the user's top 10, compared with a "most popular" baseline.

Python ALS with confidence weights

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

ALS recovers the three taste groups from interactions alone and beats the global popularity ranking (about 0.34 against 0.23 here), because the popular items are not the same for every group. The metric used here, recall at k, is explained in Precision and Recall at K.

Choosing α, λ and the number of factors

  • α (confidence scale). Large α trusts the observed interactions heavily relative to the unobserved zeros. Typical values range from 1 to 100 or more; try a log scale (1, 10, 40). Because raw counts have heavy tails, many practitioners use 1 + α · log(1 + r) instead.
  • λ. Plays the same role as before: it stops users with a handful of events from getting wild vectors.
  • Factors. 32 to 256 in production; the toy example uses 6.
  • Iterations. Loss usually flattens after 10 to 20 sweeps.

The next block shows how α changes the quality on the same data. It repeats the setup in compact form.

Python Effect of the confidence scale alpha

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In this run α = 10 scores best and both extremes are a little worse (recall values of roughly 0.39, 0.38, 0.46 and 0.42 for the four settings). On such a tiny dataset the differences are noisy, so do not read a trend into single rows. The point is the workflow: sweep α on a log scale, score each setting on validation data, and keep the best.

Practical notes

  • New users in a batch system. A user's vector is one closed-form solve from the item matrix, so you can compute a vector for a new user in milliseconds without retraining ("fold-in").
  • Use sparse matrices. The toy code builds dense arrays. Real libraries store only non-zero cells and run the per-user solves in parallel.
  • Conjugate gradient. For large factor counts, replace the exact solve by a few conjugate-gradient steps; the implicit library does this.
  • Evaluation. Hold out interactions by time, exclude already-seen items from candidates, and report ranking metrics, not RMSE.

Check your understanding

Quick check · 4 questions

of 4 answered You scored / 4 · perfect!

  1. 1. In ALS for implicit feedback, what does an unobserved cell represent?

    Unobserved cells stay in the loss with preference 0 and baseline confidence 1, because we are only weakly sure the user dislikes them.
  2. 2. What does the confidence value c = 1 + α·r express?

    More interactions mean more evidence of preference, scaled by α.
  3. 3. Why does ALS parallelize so well?

    With the item factors fixed, every user vector is its own small least-squares problem.
  4. 4. Why can YᵀY be computed once per half-step?

    YᵀY is shared by all users; each user adds only a correction from their own interacted items.

Summary

  • Implicit feedback gives preference (0 or 1) and confidence (1 + α·r); zeros stay in the loss.
  • ALS alternately fixes one side and solves the other in closed form as ridge regression.
  • YᵀY is shared, so cost per user depends on that user's interactions, not on catalog size.
  • Tune α, λ and the factor count on validation data and judge with ranking metrics.
  • Fold-in gives new users a vector without retraining.

Next: BPR and Pairwise Ranking