Skip to content
Tutorials Center
My learning

Topics

Lesson 50 of 65 ยท Case Studies: Classic Designs

Design a News Feed

Design a social news feed: posting, fan-out on write and on read, the celebrity problem, hybrid feeds, ranking, caching and pagination, with runnable Python models.

A news feed is the scrolling list of posts from the people you follow, shown newest or most relevant first. Facebook, Instagram, X and LinkedIn all have one. The task sounds like "select posts from followed users, sort, show", but at hundreds of millions of users that query is far too slow to run on every page open. The whole design is a choice about when to do the work.

Requirements

  • Users publish posts (text, images, video links) and follow other users.
  • Opening the app shows a feed of recent posts from followed accounts, ranked.
  • Feed loads in under 500 ms; new posts appear within seconds.
  • Scale: 300 million daily users, each opens the feed 10 times a day and follows 200 accounts on average.

Estimates: 3 billion feed reads per day, about 35,000 per second, with peaks near 100,000. Posts are far fewer, maybe 100 million a day (about 1,200 per second). Reads outnumber writes roughly 30 to 1, and any design must make the feed read cheap.

Two basic strategies

Fan-out on read (pull). When Alice opens her feed, query the latest posts of all 200 accounts she follows, merge them and return the top 20. Writing a post is cheap (one insert) but reading is expensive: 200 lookups per feed view.

Fan-out on write (push). When Bob posts, immediately insert the post ID into a precomputed feed list for each of his followers. Reading is one cheap lookup of Alice's own list. Writing costs one insert per follower.

Python Push and pull feeds compared

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Push read is 1 operation, pull read is 200. But one post by a user with 1,000 followers costs 1,001 writes, and a celebrity with 50 million followers would trigger 50 million writes for a single post.

The celebrity problem and the hybrid model

Real systems use a hybrid:

  • Normal users (say under 10,000 followers): fan out on write.
  • Celebrities: do not fan out. When Alice reads, merge her precomputed inbox with the recent posts of the few celebrities she follows, pulled at read time.
flowchart LR
    P["New post"] --> SVC["Post service"]
    SVC --> DB[("Post store")]
    SVC --> FQ["Fan-out queue"]
    FQ --> W["Fan-out workers"]
    W --> F[("Feed cache per user")]
    R["Feed request"] --> FS["Feed service"]
    FS --> F
    FS --> CP["Celebrity posts (pull)"]
    FS --> RK["Ranking"]

Fan-out happens in the background through a queue, so posting stays fast.

Python Hybrid feed: push for normal users, pull for celebrities

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Ranking

A chronological feed is simple but a ranked one keeps users engaged. A typical pipeline: candidate generation (the few hundred posts from inboxes and celebrities), scoring (a model predicts clicks, likes and watch time, using freshness, the author's relationship strength and the post's early engagement), then filtering and diversification (hide seen or blocked content, avoid ten posts from the same author). Scoring must be fast, so precompute features and cap the candidate set.

Python A simple feed score

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Storage, caching and pagination

  • Post store: a sharded database or wide-column store keyed by post ID; media lives in object storage behind a CDN.
  • Feed cache: keep each active user's feed (just post IDs, a few hundred) in an in-memory store like Redis. Inactive users' feeds are not built; they are generated on login.
  • Hydration: the feed holds IDs, then the service batch-fetches post bodies, author profiles and counters from caches.
  • Pagination: use a cursor (the last post's ID or timestamp), not an offset, so a new post does not shift the page.
  • Counts (likes, comments) are updated asynchronously and may lag slightly.
  • Deletes and privacy changes must remove items from existing feeds or filter them at read time.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is the main cost of fan-out on write?

    Every follower's feed list is updated, so write cost grows with follower count.
  2. 2. How does the hybrid model treat celebrities?

    Pulling a few celebrity timelines at read time avoids millions of writes per post.
  3. 3. Why use cursor pagination for feeds?

    A cursor marks a position by post identity, so inserting new items does not move it.
  4. 4. What does a feed cache usually store per user?

    IDs are small; bodies and counters are fetched from their own caches.

Summary

  • Feed reads dominate, so do work at write time: fan-out on write with precomputed lists.
  • Pure push breaks for accounts with huge audiences; use a hybrid with pull for celebrities.
  • Fan out through a queue so posting stays fast; cache feeds as ID lists; hydrate at read.
  • Rank with freshness, affinity and engagement, then filter and diversify.
  • Paginate with cursors and handle deletes and privacy at read time.

Next: Design a Notification System