System Design Interview
Meeting and lecture transcription, inspired by Otter.ai and OpenAI's Whisper API: audio uploaded to object storage, cut into chunks at silences and transcribed in parallel on GPU workers, stitched back together with speaker labels, and made searchable. A teaching design.
See why holding a request open for a long recording fails.
What was missing: Nothing yet.
Why that's risky: An hour of audio takes minutes of GPU time, so the request times out; one long lecture blocks every user behind it.
What gets added: One server with a GPU that transcribes each upload before replying and keeps the audio in the database.
Trade-offs: Simple, but the web tier and the GPU can't scale separately.
Accept uploads instantly and transcribe in the background.
What was missing: Uploads waited on the GPU, and audio sat in the database.
Why that's risky: A three-hour lecture occupies one GPU for many minutes while short meetings queue behind it.
What gets added: Audio goes straight to object storage; a queue feeds GPU workers that transcribe whole recordings and save the transcript.
Trade-offs: Users wait for the result, but uploads never fail.
Make long recordings as fast as short ones.
What was missing: Long recordings tied up one GPU while short ones waited.
Why that's risky: Cutting mid-word garbles text, so cuts happen at silences with a little overlap that the stitcher removes.
What gets added: Split workers cut audio into 30-second chunks at silences; GPU workers transcribe chunks in parallel; stitch workers join them in order and label speakers.
Trade-offs: More coordination, but every GPU stays busy with short, equal-sized work.
Serve reads and search without the database or GPUs.
What was missing: Every view hit the database, and users couldn't search what was said.
Why that's risky: The index is a copy; a deleted recording must be removed from it too.
What gets added: A cache of recent transcripts and job progress, and a search index of every transcript's words with speaker and timestamp, filtered by owner.
Trade-offs: Search adds storage and an indexing step after every transcript.
Absorb top-of-the-hour bursts and remove single points of failure.
What was missing: Free accounts could upload hours of audio at GPU cost, and the load balancer and databases were single points of failure.
Why that's risky: Free users wait longer at peak; paying users could get their own queue.
What gets added: Upload limits per plan, a second load balancer, database and index replicas, more workers for the hourly burst, and monitoring of queue age and accuracy on sampled audio.
Trade-offs: GPUs scale on queue age; keeping a few warm avoids slow cold starts at the top of the hour.
You're in the middle of an interview session. Leaving now will end your current attempt.
Explore concept overviews, real-system examples, key tradeoffs, and interview talking points for each roadmap section.
You've conquered this phase. These are the skills you now own:
