System Design Interview
A support bot that answers from a company's help center, inspired by Intercom Fin and Zendesk AI: help articles split into passages and embedded in the background, answers grounded only in retrieved passages, common questions answered from a cache, and low-confidence conversations handed to a human with a summary. A teaching design.
See why stuffing every article into the prompt fails.
What was missing: Nothing yet.
Why that's risky: Large help centers don't fit in the prompt, every question pays for thousands of words, and the model mixes up articles.
What gets added: One server that pastes the company's help articles and the conversation into one huge prompt.
Trade-offs: No retrieval to build, but cost and errors grow with the help center.
Send only the few passages that answer the question.
What was missing: Every question sent the whole help center to the model.
Why that's risky: If retrieval misses the right passage, the bot says it doesn't know; that is better than inventing an answer.
What gets added: Articles are split into passages, embedded by background workers and stored in an index per company. Each question is embedded, the best five passages are retrieved, and the LLM answers only from them, citing the articles.
Trade-offs: Smaller prompts, cheaper and more accurate; retrieval quality now decides answer quality.
Answer the same question once, not ten thousand times.
What was missing: During an outage, thousands of customers asked the same question and each one paid for a full LLM call.
Why that's risky: Two questions that look alike can need different answers ("cancel my order" vs "cancel my plan"); keep the similarity threshold strict.
What gets added: A cache of answers to common questions, keyed by meaning and the version of the articles used; an edit changes the version, so stale answers stop matching.
Trade-offs: Fewer LLM calls, at the cost of a check on every question.
Pass low-confidence conversations to an agent with context.
What was missing: When the bot couldn't help, customers were stuck repeating themselves.
Why that's risky: Too low a threshold floods agents; too high frustrates customers. Track both resolution and handoff rates.
What gets added: When retrieval finds nothing relevant, the model is unsure, or the customer asks for a person, the conversation is queued for handoff. Workers summarise it, open a ticket in the helpdesk, and record it.
Trade-offs: The bot resolves most questions; people handle the rest with full context.
Stop abuse and remove single points of failure.
What was missing: Bots could spam the widget at the company's expense, and the load balancer and databases were single points of failure.
Why that's risky: Strict limits can cut off a genuine customer typing quickly; limit per minute, not per second.
What gets added: Rate limits per visitor and per plan, a second load balancer, sharded and replicated conversation storage, index replicas, more chat capacity, and monitoring of resolution and thumbs-down rates per company.
Trade-offs: Resolution rate is what customers pay for; it is the number to watch.
You're in the middle of an interview session. Leaving now will end your current attempt.
Explore concept overviews, real-system examples, key tradeoffs, and interview talking points for each roadmap section.
You've conquered this phase. These are the skills you now own:
