System Design Interview
Trending topics inspired by Twitter/X Trends and YouTube Trending: hashtag events on a partitioned stream, windowed counting with a count-min sketch and heavy-hitter heaps, trends ranked by velocity against a baseline, a precomputed top 50 per region in a cache, a replayable raw-event archive and bot filtering. A teaching design.
See why a database row per hashtag breaks the moment one tag goes viral.
What was missing: Nothing yet.
Why that's risky: A viral hashtag sends thousands of updates per second to the same row, and they queue behind its lock. Sorting every hashtag on each read gets slower as the table grows.
What gets added: One server that adds one to a hashtag's row on every event, and sorts every row by count when someone opens the trending tab.
Trade-offs: Exact counts and one place to look, paid for with a write on every event and a full sort on every read.
Take counting off the request path and stop hot rows.
What was missing: Every event was a database write, and a viral hashtag turned into one hot row.
Why that's risky: The trending tab still sums an hour of minute rows and sorts every hashtag on each read.
What gets added: An ingest service that only appends events to a stream partitioned by hashtag, counter workers that add up each minute in memory and write one row per hashtag per minute, and a trends service behind a load balancer.
Trade-offs: Counts lag by up to a minute, which nobody notices on a trending list.
Rank by how fast a topic is rising, and serve the answer instead of computing it per read.
What was missing: Trends were computed from scratch on every read, and the most-used hashtags (good morning, every day) always won.
Why that's risky: Raw events live only in the stream. A counting bug or a lost partition can't be repaired by recounting.
What gets added: A count-min sketch and a heap of heavy hitters in each counter, a ranker that scores tags every 30 seconds by how fast they rise over a sliding window against their usual baseline, and the top 50 per region written to a trends store and a cache.
Trade-offs: A count-min sketch can over-count slightly in exchange for fixed memory. That's fine for ranking, not for billing.
Make counts repairable, and stop fake accounts manufacturing a trend.
What was missing: No way to recount after a bug, and a few thousand bot accounts could push any tag to the top.
Why that's risky: One entry point and no limit on how fast a single account or IP can post events.
What gets added: An archiver that writes every raw event to hourly files in object storage, and counters that skip flagged bot accounts and count each account once per hashtag per window.
Trade-offs: Archiving raw events costs storage, but makes every count reproducible.
Survive a breaking-news spike and a bot flood, and see trouble before users do.
What was missing: One load balancer, no per-account limits, and no view of stream lag or freshness.
Why that's risky: Remaining risk is low at this scale. A second region would cover a full regional outage.
What gets added: A redundant load balancer, a rate limiter per account and IP, extra ingest, trends and counter instances, a replicated counts store, and monitoring of stream lag, counter throughput and how old the current trends are.
Trade-offs: Trends are up to 30 seconds old by design, the price of serving them from memory.
You're in the middle of an interview session. Leaving now will end your current attempt.
Explore concept overviews, real-system examples, key tradeoffs, and interview talking points for each roadmap section.
You've conquered this phase. These are the skills you now own:
