System Design Interview
AI image generation inspired by Midjourney and DALL·E: prompts accepted as queued jobs, a fleet of GPU workers running diffusion, a fast lane for paid plans and a relaxed lane for free ones, live previews from a progress cache, images in object storage behind a CDN, a cached gallery, prompt moderation and image safety checks. A teaching design.
See why holding a request open for a 30-second GPU job doesn't scale.
What was missing: Nothing yet.
Why that's risky: An image takes tens of seconds, so connections time out and the GPU serves one prompt at a time. Image bytes bloat the database, and every page view pulls them through the server.
What gets added: One server with a GPU that generates each image while the request waits, then stores the image bytes in the database.
Trade-offs: Simple, but GPU and web work can't be scaled separately.
Accept prompts instantly, and keep a fleet of GPUs busy from a queue.
What was missing: Generation ran inside the request and images lived in the database.
Why that's risky: Paying users wait behind free users in one line, users see nothing until the image is done, and every poll reads the database.
What gets added: A job API that checks credits, saves the job as queued and returns at once; GPU workers that take jobs from a queue and upload finished images to object storage; and a status service the client polls.
Trade-offs: The client polls instead of waiting, which is the right shape for work that takes seconds to minutes.
Serve paying users first, and show the image forming.
What was missing: One line for everyone, and nothing to see until the image was done.
Why that's risky: Finished images are served from object storage on every view, and galleries query the database directly.
What gets added: Two queues: a fast lane for paid plans that GPUs always serve first, and a relaxed lane served whenever a GPU is free. Workers write a low-resolution preview every few denoising steps to a progress cache, which status polls read instead of the database.
Trade-offs: Relaxed jobs can wait minutes at peak. That is the product: speed is what users pay for.
Serve the much larger viewing traffic cheaply.
What was missing: Every image view hit object storage, and galleries queried the database.
Why that's risky: Any prompt reaches the GPUs, including disallowed ones, one load balancer, and nothing caps a user's request rate.
What gets added: A CDN in front of image storage, thumbnails for gallery grids, and a gallery service with a cache of recent pages and the public showcase feed.
Trade-offs: Images are immutable once generated, so the CDN can cache them indefinitely.
Keep disallowed content off the GPUs and out of galleries, and see queues before users complain.
What was missing: Disallowed prompts used GPU time, nothing checked finished images, and no per-user limits.
Why that's risky: Remaining risk is GPU supply: a viral moment fills the relaxed lane for a while, by design.
What gets added: Prompt moderation before a job is queued, a safety classifier on every finished image, a rate limiter on requests and concurrent jobs per plan, a second load balancer, a replicated jobs database, more GPU workers, and monitoring of queue wait per lane and seconds per image.
Trade-offs: Moderation blocks a few harmless prompts. Appeals and tuning reduce that over time.
You're in the middle of an interview session. Leaving now will end your current attempt.
Explore concept overviews, real-system examples, key tradeoffs, and interview talking points for each roadmap section.
You've conquered this phase. These are the skills you now own:
