Skip to content
Tutorials Center
My learning

Topics

Lesson 38 of 38 · Projects and Practice

Recommender Systems Practice Set

Eight worked practice problems on similarity, prediction, matrix factorization updates, implicit confidence, ranking metrics, A/B sample size, MMR and cold start, with runnable solutions.

The best way to find out what you really understood is to compute things by hand. This practice set has eight problems that mirror the chapters of the track. For each one, try to solve it on paper or in your own code first. Then open the editor, which contains a runnable solution, compare the numbers, and change the inputs to see how the answer moves.

Problem 1: cosine similarity between items

Three users rated items A, B and C. User 1: A=5, B=4, C=1. User 2: A=4, B=5, C=2. User 3: A=1, B=2, C=5. Which pair of items is the most similar by cosine similarity of their rating columns?

Python Problem 1 solution

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Items A and B get similar ratings from every user, so they form the most similar pair. C moves in the opposite direction. Remember the caveat from similarity measures: on ratings, adjusted (mean-centered) cosine would be more careful about user generosity.

Problem 2: predict a rating with user-based CF

Use the matrix above, but suppose user 3's rating of item A is unknown. Predict it as the similarity-weighted average of the other users' ratings of A, where similarity is the cosine of the users' rows over the items rated by both (use items B and C).

Python Problem 2 solution

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The prediction lands between the two neighbors' ratings (5 and 4), pulled toward the more similar user. Notice that it is a plain average of two high ratings, even though user 3 seems to dislike what users 1 and 2 like: with so little data the neighborhood is poor. Shrinkage and mean-centering are the remedies.

Problem 3: one SGD step of biased matrix factorization

Global mean μ = 3.5, b_u = 0.2, b_i = −0.1, p_u = [0.5, 0.1], q_i = [0.4, 0.3], true rating 5, learning rate 0.05, regularization 0.02. Compute the prediction, the error and the updated b_u, b_i, p_u, q_i.

Python Problem 3 solution

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A common bug is to update p_u first and then use the new p_u to update q_i. Use the old value for both, as in the SGD lesson.

Problem 4: confidence weights for implicit data

A listener played song 1 once, song 2 ten times and never played song 3. With α = 20, give the preference and the confidence of each cell, both for raw counts and for log1p counts. How many times larger is the confidence for song 2 versus song 1 in each case?

Python Problem 4 solution

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Song 3 keeps confidence 1 and preference 0: the zero stays in the loss with a low weight, as explained in ALS and Implicit Feedback. Log scaling tames the effect of heavy repeat plays.

Problem 5: ranking metrics on one list

A model recommends [d, a, f, c, b]. The user's relevant items are {a, b, e}. Compute precision@5, recall@5, reciprocal rank, average precision@5 and binary NDCG@5.

Python Problem 5 solution

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Item e was never recommended, so recall cannot reach 1. Move item b to the first position and rerun to see how each metric reacts; precision and recall will not change, but the rank-aware metrics will. See ranking metrics.

Problem 6: how many users does the A/B test need?

Your current click-through rate is 4%. You want to detect an absolute lift of 0.4 percentage points (4.0% to 4.4%). Use the rule of thumb n ≈ 16 · p(1−p) / δ² per arm. How many users do you need per arm? What if you only care about a lift of 0.8 points?

Python Problem 6 solution

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Halving the effect you want to detect roughly quadruples the sample size, because δ is squared in the denominator. That is why small improvements need large experiments. See Offline Evaluation and A/B Testing.

Problem 7: one MMR step

Three candidate items have relevance scores 0.9 (item X, genre Action), 0.85 (item Y, genre Action) and 0.7 (item Z, genre Comedy). Item X is already in the list. With λ = 0.5 and a similarity of 1 for the same genre and 0 otherwise, which item does MMR pick next? What if λ = 0.9?

Python Problem 7 solution

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

With a low λ the similarity penalty beats the small relevance gap, so the Comedy item enters the list; with a high λ the more relevant Action item wins. This is the accuracy-diversity knob from Beyond Accuracy.

Problem 8: design question (cold start)

A new streaming service launches with 500 titles and no viewing history. Outline a recommendation plan for the first week, the first month and the first year. There is no code here, but you can check your answer against this outline:

  • First week: editorial picks and popularity by country; onboarding questions (favorite genres); content-based "similar titles" from metadata; log every impression and click.
  • First month: item-item "people also watched" once there are enough sessions; simple hybrid of popularity, content and co-views; exploration slots for new titles; per-segment metrics for new users.
  • First year: matrix factorization or a two-tower model on the collected logs; ANN retrieval plus a ranker; A/B tests with guardrails; fairness and diversity monitoring.

Compare with Cold Start and the feedback-loop lesson.

Check your understanding

Quick check · 5 questions

of 5 answered You scored / 5 · perfect!

  1. 1. In biased MF, which value should be used to update q_i after computing the new p_u?

    Both updates are gradient steps at the same point, so they use the same old values.
  2. 2. A top-5 list has hits at positions 2 and 5 of 3 relevant items. What is recall@5?

    Hits / relevant items = 2/3.
  3. 3. Why does detecting half the effect size need about four times the users?

    n ≈ 16 p(1−p)/δ², and δ is squared.
  4. 4. Which preference and confidence does an unplayed song get in implicit ALS?

    Unobserved cells stay in the loss with preference 0 and the baseline confidence of 1.
  5. 5. In MMR, what does a smaller λ favor?

    Smaller λ gives more weight to the similarity penalty.

Summary

  • You practiced similarity, neighborhood prediction, an SGD update, implicit confidence, ranking metrics, sample size, MMR and cold-start planning.
  • Re-derive each answer by changing the inputs and predicting the result before running the code.
  • Keep building: return to the two projects and extend them with the ideas from the exercises.

Next: Recommender Systems track overview