Design a News Feed

Requirements

A news feed (home timeline) shows a user the recent posts of every account they follow, newest or most relevant first. The user timeline (one author's own posts) is a much simpler sub-problem that falls out of the same design.

Functional

• A user publishes a post (text, optional media).
• A user follows and unfollows other users.
• A user opens their home feed and scrolls through it with pagination.
• A user views a single author's timeline.
• Followers can optionally receive push notifications for new posts.

Non-functional

• Fast reads: feed load p99 under ~200 ms; reads far outnumber writes.
• Fast publish: the author should get an acknowledgement in well under a second, even if delivery to followers finishes later.
• Highly available: a stale feed is acceptable, an error page is not. Eventual consistency of a few seconds is fine.
• Skewed traffic: some accounts have tens of millions of followers; most have a few hundred.

Capacity Estimates

Assume 300 M daily active users, 500 M new posts per day, each user follows ~200 accounts on average, and the median post is delivered to ~10 active followers (follower counts are heavily skewed, so the average delivery fan-out is higher than the median).

The two headline numbers are the 600 K timeline inserts per second (writes are amplified by follower count) and the ~5 TB feed cache, which is large but affordable across a Redis cluster. Storing post IDs rather than full posts is what keeps that cache small.

API Design

POST /api/v1/posts
  Body:  { "text": "hello", "mediaIds": ["m_91"], "visibility": "public" }
  201:   { "postId": "1849275532", "createdAt": "2026-10-30T12:00:00Z" }

GET /api/v1/feed?limit=20&cursor=<opaque>
  200:   { "items": [ { "postId": "...", "author": {...}, "text": "...", "media": [...],
                       "likeCount": 12 } ],
           "nextCursor": "<opaque>" }

GET /api/v1/users/{userId}/posts?limit=20&cursor=<opaque>
POST   /api/v1/users/{userId}/follow
DELETE /api/v1/users/{userId}/follow

• Use cursor pagination (encode the last seen post ID / score), never page numbers: new posts arrive constantly and offsets would skip or duplicate items.
• Post IDs should be time-sortable (Snowflake-style: timestamp + worker + sequence) so a chronological merge is just a sort on ID.

Data Model

CREATE TABLE posts (
  post_id     BIGINT PRIMARY KEY,      -- Snowflake ID, sorts by time
  author_id   BIGINT NOT NULL,
  body        TEXT,
  media_keys  TEXT,
  created_at  TIMESTAMP NOT NULL
);
CREATE INDEX idx_author_time ON posts (author_id, post_id DESC);

CREATE TABLE follows (
  follower_id BIGINT,
  followee_id BIGINT,
  created_at  TIMESTAMP,
  PRIMARY KEY (follower_id, followee_id)
);

Keep two copies of the follow graph: one partitioned by follower ("whom do I follow?" for read-time merges) and one by followee ("who follows me?" for fan-out).

High-Level Design

flowchart LR
  C[Client] --> LB[Load Balancer]
  LB --> WA[Write API]
  LB --> RA[Read API]
  WA --> PS[(Post Store)]
  WA --> MQ[[Fan-out Queue]]
  MQ --> FO[Fan-out Workers]
  FO --> G[(Social Graph)]
  FO --> FC[(Feed Cache)]
  FO --> N[Notification Service]
  RA --> TS[Timeline Service]
  TS --> FC
  TS --> PS
  TS --> U[(User Info Cache)]
  C --> CDN[CDN for Media]

Publish path: the Write API stores the post, enqueues a fan-out job, and returns immediately. Workers look up the author's followers and prepend the post ID into each follower's cached feed.

Read path: the Timeline Service reads the user's list of post IDs from the feed cache (one call), then hydrates them with multi-gets against the post and user caches, applies filters (blocked users, deleted posts) and returns a page. Media URLs point to the CDN.

Deep Dives

Fan-out on write (push)

When a post is created, push its ID into the precomputed feed of every follower.

• Reads are O(1): opening the feed is a single list read from cache plus hydration.
• Writes are O(followers): a user with 1,000 followers costs 1,000 inserts. That is fine for the median user and is done asynchronously by queue workers.
• Wasted work: most followers will never open the app before the post scrolls out of their feed. Limit fan-out to users active in the last N days and rebuild others on demand.
• Trim lists: cap each feed at a few hundred to ~800 entries; older pages fall back to the slower pull path.

Fan-out on read (pull)

Store nothing per reader. At read time, fetch the follow list, query each followee's recent posts, and k-way merge them by post ID.

• Writes are O(1) and there is no wasted delivery to inactive users.
• Reads are expensive: 200 followees means 200 lookups (scatter-gather across shards) on every feed open, and tail latency is set by the slowest shard.
• Works well for inactive users, for the long tail of feed history, and for accounts that are followed by huge numbers of people.

The celebrity problem and the hybrid model

An account with 50 M followers turns one post into 50 M cache writes. That takes minutes, delays every other job in the queue, and creates odd effects such as replies appearing before the original post in some feeds.

The standard fix is a hybrid:

1. Mark accounts above a follower threshold (say 100 K) as celebrities.
2. Normal authors are fanned out on write as usual.
3. Celebrity posts are not fanned out. They are written only to the author's own timeline, which is heavily cached because millions read it.
4. At read time, the Timeline Service takes the precomputed feed and merges in the recent posts of the (usually few) celebrities the reader follows, then sorts.

Refinements: make the threshold dynamic (based on follower count and posting rate), and still push celebrity posts to their most engaged followers if product wants instant delivery for them.

Feed hydration and ranking

The cache holds only IDs. A read page of 20 items triggers a multi-get for 20 posts and their authors, preferably from in-memory object caches. Hydration is also where you drop posts from blocked or deleted accounts, so you never need to rewrite millions of feeds when someone blocks a user.

For a ranked feed, treat the cached IDs (plus celebrity and recommended posts) as candidates, compute features (author affinity, engagement velocity, recency, media type), score them with a model, and return the top N. Cache the ranked page briefly so scrolling is consistent; re-rank on pull-to-refresh.

Edits, deletes and new follows

• Delete: mark the post deleted in the post store; hydration filters it out. Lazy cleanup avoids touching millions of lists.
• Unfollow: filter by the current follow set at read time, and optionally purge asynchronously.
• New follow: backfill a few recent posts from the new followee into the follower's feed so the change is visible immediately.

Scaling & Reliability

• Shard the feed cache by user_id with consistent hashing; replicate each shard. If a feed is lost, rebuild it with the pull path, so the cache is never the source of truth.
• Shard the post store by post_id, and the author index by author_id. Keep recent posts in memory; move old posts to cheaper storage.
• Queue-based fan-out (Kafka) absorbs bursts, allows replays and lets you prioritize: run separate queues for normal and high-follower authors so one viral post does not delay everyone.
• Idempotent workers: inserting the same post ID twice must be harmless (dedupe by ID when trimming), because queues redeliver.
• Hot keys: celebrity timelines and viral posts are read by millions; replicate them across cache nodes and add a small in-process cache in the Timeline Service.
• Graceful degradation: if ranking or hydration of some fields fails, return a chronological feed with fewer decorations rather than an error.
• Multi-region: serve reads locally; replicate posts asynchronously. Users rarely notice a few seconds of cross-region delay.

Trade-offs

Interview Tips

• Put numbers on fan-out early; the amplification from followers is what drives the design.
• Explain push and pull separately, show where each breaks, then introduce the hybrid as the resolution.
• Say clearly that the feed cache stores IDs and can be rebuilt; it is not the system of record.
• Mention cursor pagination and time-sortable IDs; both are common follow-up questions.
• Address deletes, blocks and unfollows with read-time filtering instead of mass rewrites.
• If asked about ranking, frame it as candidate generation + scoring + caching, and keep model details brief unless probed.

References

• System Design Primer: Design the Twitter timeline and search
• System Design Primer: Cache and cache-update strategies
• ByteByteGo System Design 101
• High Scalability: The architecture Twitter uses to deal with 150M active users
• Meta Engineering: TAO, the power of the graph
• Awesome Scalability (curated case studies)