Design a Video Streaming Platform

Requirements

Functional

• Creators upload videos (up to several GB) from web and mobile.
• Uploaded videos are processed into multiple resolutions and become watchable.
• Viewers stream videos on any device with smooth playback, seeking, and resume.
• Video metadata: title, description, thumbnails, view counts; basic search and recommendations exist but are out of scope here.

Non-functional

• Smooth playback: start-up under ~2 s, rebuffering rare, quality adapts to the network.
• Massive read skew: views dwarf uploads, and a small fraction of videos gets most views.
• Durability: an uploaded original must never be lost.
• Processing latency: a typical video playable within minutes of upload (low resolutions first).
• Cost efficiency: bandwidth and storage are the dominant costs, so every design choice is weighed against them.

Capacity Estimates

Assume 200 M daily viewers each watching 5 videos of ~5 minutes, 500 K uploads per day averaging 300 MB, and an average delivered bitrate of 3 Mbps.

Ten-plus terabits per second of egress cannot come from a few data centres; it must be pushed to the edge. Storage grows by petabytes per month, which motivates tiered storage for old, rarely watched videos.

API Design

POST /api/v1/videos                      // create upload session
  Body: { "title": "...", "sizeBytes": 314572800, "contentType": "video/mp4" }
  201:  { "videoId": "v_8812", "uploadId": "up_31", "partSize": 8388608,
          "partUrls": ["https://upload.example.com/...part=1&sig=...", ...] }

PUT  <partUrl>                           // client uploads each 8 MB part directly to object storage
POST /api/v1/videos/{videoId}/complete   // { "uploadId": "up_31", "parts": [{ "n": 1, "etag": "..." }] }
GET  /api/v1/videos/{videoId}            // metadata + status: UPLOADING | PROCESSING | READY | FAILED
GET  /api/v1/videos/{videoId}/play       // { "manifestUrl": "https://cdn.example.com/v_8812/master.m3u8?token=..." }
POST /api/v1/videos/{videoId}/progress   // { "positionSec": 312 }  (resume watching, analytics)

Uploads go straight to object storage through pre-signed, chunked (multipart) URLs, so API servers never proxy gigabytes of video and a failed chunk is retried on its own.

Data Model

CREATE TABLE videos (
  video_id     BIGINT PRIMARY KEY,
  owner_id     BIGINT NOT NULL,
  title        VARCHAR(200),
  status       VARCHAR(16) NOT NULL,   -- UPLOADING, PROCESSING, READY, FAILED
  duration_sec INT,
  manifest_key VARCHAR(255),
  created_at   TIMESTAMP NOT NULL
);

CREATE TABLE renditions (
  video_id   BIGINT,
  codec      VARCHAR(16),              -- h264, vp9, av1
  height     INT,                      -- 240, 480, 720, 1080, 2160
  bitrate_kb INT,
  PRIMARY KEY (video_id, codec, height)
);

High-Level Design

flowchart LR
  CR[Creator] -->|"multipart upload"| RAW[(Raw Object Store)]
  CR --> API[Video API]
  API --> META[(Metadata DB)]
  RAW -->|"upload complete event"| Q[[Job Queue]]
  Q --> ORC[Pipeline Orchestrator]
  ORC --> W[Transcode Workers]
  W --> ENC[(Encoded Segment Store)]
  ORC --> META
  ENC --> CDN[CDN Edge Caches]
  V[Viewer Player] --> API
  V -->|"manifest and segments"| CDN
  V --> AN[[Playback Analytics]]

Upload path: create session → upload parts directly to the raw bucket → complete → an event triggers the processing pipeline → status moves to READY when the first renditions are done.

Playback path: the player asks the API for a signed manifest URL, then fetches the manifest and video segments from the nearest CDN edge. The origin is only touched on cache misses.

Deep Dives

Upload pipeline

• Resumable multipart upload: the file is split into fixed parts (5-16 MB). The client tracks which parts succeeded, so a dropped mobile connection resumes instead of restarting.
• Validation first: after completion, a lightweight job probes the container (format, codec, duration, corruption) and runs content-safety and copyright checks before spending compute on transcoding.
• Durability: the original is stored with high redundancy (multi-AZ object storage) and kept as the "mezzanine" source for future re-encodes with better codecs.
• Status updates flow back via the metadata DB, and the creator is notified when the video is ready.

Transcoding as a parallel DAG

Encoding a 1-hour 4K video serially can take hours. Instead, the orchestrator builds a directed acyclic graph of tasks:

1. Split the source into chunks of a few seconds, aligned to keyframes (GOP boundaries).
2. Encode each chunk for each target rendition in parallel across a worker fleet (e.g. 240p to 4K, H.264 for compatibility, VP9/AV1 for efficiency).
3. Assemble/package the encoded chunks into streaming segments and write the manifests.
4. Side tasks in parallel: extract audio tracks, generate thumbnails, produce captions, compute a perceptual quality score.

• Workers pull tasks from a queue; failed tasks are retried individually, and idempotent outputs (deterministic object keys) make retries safe.
• Prioritize: encode a low resolution first so the video becomes playable quickly, then add higher renditions. Fresh uploads from popular creators can jump the queue.
• Spot/preemptible instances cut cost since chunk tasks are short and restartable.

Per-title encoding: a static cartoon looks fine at a much lower bitrate than a sports match. Analysing complexity and picking a custom bitrate ladder per video saves significant bandwidth at equal perceived quality, which matters when egress is the largest bill.

Adaptive bitrate streaming (HLS / DASH)

• Each rendition is cut into segments of ~2-6 seconds. A master manifest (.m3u8 for HLS, .mpd for DASH) lists all renditions and their bandwidths; each rendition has a media playlist listing its segments.
• The player measures throughput and buffer level, and before each segment picks the best rendition it can sustain. It switches up or down at segment boundaries without interrupting playback.
• Start with a modest rendition for fast start-up, then climb. Buffer-based algorithms prevent oscillation better than throughput-only ones.
• Everything is plain HTTP GETs of static files, which is exactly why CDNs can cache it.
• CMAF packaging lets HLS and DASH share the same fragmented-MP4 segments, halving storage for dual-format delivery.

CDN and content placement

• Pull CDN for the long tail: an edge fetches a segment from origin on first request, then caches it. Simple and storage-efficient.
• Push / pre-positioning for predictable hits: popular titles (new episode releases) are copied to edge caches during off-peak hours before demand arrives. Netflix's Open Connect appliances, installed inside ISPs, are the canonical example.
• Tiered caching: edge → regional mid-tier → origin shield, so a miss at hundreds of edges results in one request to origin.
• Signed URLs/tokens with short expiry protect content; DRM (Widevine, FairPlay, PlayReady) is applied at packaging time for premium catalogues.
• Multi-CDN with client-side or DNS steering improves availability and lets you route by price and performance.

Scaling & Reliability

• Metadata DB: shard by video_id, cache hot metadata, and use read replicas; it is small compared with video bytes.
• Storage tiering: move renditions of rarely watched videos to cold storage; keep only a few renditions (or re-encode on demand) for the long tail.
• View counts are aggregated through a stream (Kafka → stream processor) rather than incremented per view in the DB.
• Pipeline resilience: the orchestrator persists DAG state so it can resume after a crash; dead-letter queues capture poison videos for inspection.
• Back pressure: during upload spikes, queue depth grows rather than overwhelming workers; autoscale the encoder fleet on queue lag.
• Playback QoE monitoring: collect start-up time, rebuffer ratio and average bitrate from players to detect bad CDN nodes or encoding regressions.
• Live streaming (stretch): same ABR model but segments are produced in real time with short segments or low-latency HLS, and the encoder ladder is fixed rather than per-title.

Trade-offs

Interview Tips

• State the read/write split and size egress bandwidth first; it justifies the CDN-centric design.
• Show direct-to-storage multipart uploads and explain resumability.
• Describe transcoding as a DAG of chunk tasks and mention "low resolution first" for fast availability.
• Explain ABR concretely: segments, manifests, and a player that chooses renditions per segment.
• Distinguish pull and push CDN strategies and mention ISP-embedded caches.
• Mention cost levers (per-title encoding, modern codecs, cold storage); interviewers value cost awareness here.

References

• System Design Primer: Content delivery network (push vs pull CDNs)
• System Design Primer: Asynchronism and task queues
• Meta Engineering: How Facebook encodes your videos
• Netflix Open Connect
• RFC 8216: HTTP Live Streaming
• Karan Pratap Singh: System Design (Netflix case study)