System Design Interview
Upload a long document and get a summary, inspired by Notion AI and Adobe Acrobat's AI Assistant: files go to object storage, summaries are made in the background by splitting long documents into chunks, summarizing them in parallel and combining the results, and identical files are never summarized twice. A teaching design.
See why waiting on the LLM in the request fails.
What was missing: Nothing yet.
Why that's risky: The LLM takes seconds to minutes, so requests time out, and long documents don't fit in one call at all.
What gets added: One server that takes the upload, sends the whole text to the LLM, waits for the summary and saves both in the database.
Trade-offs: Simple, but every slow summary holds a server thread.
Accept uploads instantly and summarize in the background.
What was missing: Uploads waited on the LLM and file bytes sat in the database.
Why that's risky: A 300-page document still doesn't fit in one LLM call.
What gets added: Files go to object storage and a document row is created; a queue feeds summary workers that call the LLM and save the result. The page polls for status.
Trade-offs: Users wait a few seconds for a summary, but uploads always succeed.
Summarize documents far longer than the model can read.
What was missing: Long documents didn't fit in one LLM call, and one big call was slow.
Why that's risky: Details can be lost between chunks; the overlap and a final pass that sees every chunk summary reduce that.
What gets added: Map-reduce: split workers cut the text into overlapping chunks, chunk workers summarize them in parallel, and a combine step merges the partial summaries into the final one.
Trade-offs: More LLM calls per document, but each one is short and they run at the same time.
Never summarize the same file twice, and keep reads off the database.
What was missing: The same public report was summarized thousands of times, and every status poll hit the database.
Why that's risky: Only identical files match; a re-saved copy with one changed page is summarized again.
What gets added: Each file is hashed on upload; if that content already has a summary, it is reused with no LLM call. A cache holds recent summaries and job status for the polling page.
Trade-offs: Reusing summaries across users is safe only because the summary depends on nothing but the file.
Control LLM spend and remove single points of failure.
What was missing: Free accounts could upload whole libraries at our LLM cost, and the load balancer and database were single points of failure.
Why that's risky: Limits frustrate heavy free users; that is also the upgrade prompt.
What gets added: Rate limits on uploads and pages, a second load balancer, a database replica, more workers for surges, and monitoring of queue age and LLM cost per document.
Trade-offs: Watching cost per document catches prompt changes that quietly double spend.
You're in the middle of an interview session. Leaving now will end your current attempt.
Explore concept overviews, real-system examples, key tradeoffs, and interview talking points for each roadmap section.
You've conquered this phase. These are the skills you now own:
