Beginner System design concept · How Systems Actually Work · 22 mins read
How Requests Flow Through Systems
Follow a user action from app click to backend response and see why each hop exists.
Domain Name System
Learn how DNS turns human-readable names into IP addresses, how resolution and caching actually work, and why DNS is both a load-balancing tool and a single point of failure.
Intuition
Users type 'example.com', but network packets can only be delivered to IP addresses like 93.184.216.34. Somewhere between the browser and the server, a name must be translated into an address — and this translation has to happen billions of times per second, globally, with answers that change as traffic shifts and servers fail. DNS is the first hop of nearly every request. If DNS is slow, every page load is slow; if DNS is down, your entire site is unreachable even when all your servers are healthy. It is also a real load-balancing and traffic-steering mechanism (geo-routing, weighted records, failover), so understanding its resolution flow, caching behavior, and failure modes is essential for designing systems that are fast and resilient.
Mental Model
DNS is a globally distributed, hierarchical key-value store mapping domain names to records. Resolution walks the hierarchy: a recursive resolver (usually run by your ISP or a public service like Cloudflare 1.1.1.1 or Google 8.8.8.8) queries a root server to find the TLD server (e.g., .com), then the TLD server to find the domain's authoritative nameserver, then the authoritative server to get the actual record. Every layer caches answers for a time-to-live (TTL), so in practice most lookups are served from cache in single-digit milliseconds and never touch the authoritative servers. Think of it like: DNS is like a global postal directory. Asking for an address in a foreign city: you first ask the international directory (root), which points you to the country's directory (TLD), which points you to the city office (authoritative server) that actually holds the street address. Once you've looked it up, you write it in your notebook (cache) and stop asking until your note expires (TTL).
Building Blocks
- Recursive Resolver: The server that does the legwork of a lookup on behalf of the client, typically run by the ISP or a public provider (Cloudflare 1.1.1.1, Google 8.8.8.8). It iteratively queries root, TLD, and authoritative servers, caches the result for the TTL, and answers future identical queries from cache. A warm resolver cache is why most DNS lookups complete in under 10 ms.
- Root, TLD, and Authoritative Servers: The three tiers of the DNS hierarchy. Root servers (13 named logical services, actually hundreds of anycast instances worldwide) direct queries to TLD servers (.com, .org, country codes). TLD servers point to the domain's authoritative nameserver (e.g., Route53 or Cloudflare nameservers for your domain), which holds the actual records. Each tier only knows about the layer below it, which is what makes the system scale.
- Record Types (A, AAAA, CNAME): A records map a name to an IPv4 address; AAAA records map to IPv6. CNAME records alias one name to another name (e.g., www.example.com → example.cdn.net), which then resolves to an A record. A CNAME cannot coexist with other records at the same name, and you cannot put a CNAME at the zone apex in classic DNS — which is why providers invented ALIAS/ANAME pseudo-records.
- TTL and Caching: Every DNS record carries a TTL (time-to-live) telling resolvers how long to cache the answer. Long TTLs (hours) reduce lookup latency and load on authoritative servers but slow down failover — clients may keep sending traffic to a dead IP until the TTL expires. Short TTLs (30–60 seconds) enable faster traffic shifts at the cost of more authoritative queries and slightly higher first-hit latency.
- Anycast: A routing technique where the same IP address is announced from many physical locations, and the network routes each user to the nearest one. Major DNS providers and CDNs use anycast so a query to 1.1.1.1 is answered by whichever of hundreds of worldwide PoPs is closest. Anycast improves latency and provides natural resilience — if one location dies, routing simply shifts to the next nearest.
- DNS-Based Load Balancing: Returning different IPs for the same name to spread or steer traffic: round-robin DNS rotates a list of A records; geo DNS returns region-specific IPs; weighted records shift percentages of traffic. Its limits are fundamental: resolvers and clients cache answers past your control, DNS has no health awareness (it will happily return a dead server's IP until the TTL expires and the record changes), and changes propagate slowly and unevenly.
Definitions
- DNS Resolution
-
The process of translating a domain name into an IP address by walking the DNS hierarchy.
- Full cold resolution path: client → recursive resolver → root server → TLD server → authoritative server → answer.
- Each hop adds latency, but caching collapses most real-world lookups to a single resolver query.
- The resolver queries iteratively: each server answers with a referral to the next layer, not the final answer.
- TTL (Time-To-Live)
-
The number of seconds a DNS record may be cached by resolvers and clients before it must be re-fetched.
- Low TTL (30–300 s) enables fast failover and frequent traffic steering; high TTL (hours) minimizes lookups and latency.
- TTL is the main reason DNS changes are 'propagating' — stale caches keep serving the old answer until expiry.
- Some resolvers and ISPs ignore very low TTLs and enforce a minimum, so you cannot fully control cache behavior.
- A / AAAA Record
-
Records that map a domain name directly to an IPv4 (A) or IPv6 (AAAA) address.
- Multiple A records on one name is the simplest form of load balancing: clients pick from the returned list.
- AAAA records matter increasingly as mobile networks and ISPs go IPv6-first.
- Because these hold literal IPs, changing them is how you move traffic between data centers or providers.
- CNAME Record
-
A record that aliases one domain name to another, deferring the final IP lookup to the target name.
- Canonical use: pointing app.example.com at a CDN hostname like d123.cloudfront.net so the CDN controls the final IPs.
- Adds one extra lookup per cold resolution, which slightly increases latency.
- Cannot exist alongside other record types at the same name, and classic DNS forbids it at the zone apex.
- Authoritative Nameserver
-
The server that holds the definitive records for a domain and answers resolvers directly for that zone.
- Hosted by your DNS provider — Route53, Cloudflare, NS1, Google Cloud DNS — or self-hosted with something like BIND.
- Its availability bounds your availability: if it stops answering, caches save you only until TTLs expire.
- Good providers run it as a globally anycasted, DDoS-hardened service; this is why 'use two DNS providers' is a real resilience strategy.
- Anycast
-
Announcing the same IP from multiple locations so routing directs each user to the nearest instance.
- Used by root servers, public resolvers (1.1.1.1 answers from 300+ locations), and CDNs.
- Failover is a side effect of routing: withdraw the announcement from a dead site and traffic shifts automatically.
- Works best for connectionless, stateless protocols like DNS over UDP; TCP services need more care.
- DNS Propagation
-
The informal term for the delay between changing a DNS record and the change being visible everywhere.
- There is no push mechanism — 'propagation' is just every resolver's cached copy expiring at its own pace.
- Bound roughly by the old record's TTL: a 24-hour TTL means up to a day of mixed old/new answers.
- Standard migration practice: lower the TTL a day or two before a cutover so the change takes effect quickly.
Patterns
- Geo DNS / Latency-Based Routing — When you serve users from multiple regions and want each user directed to the nearest or fastest region.
- DNS Failover with Health Checks — When you need automatic region or endpoint failover without a global load balancer.
- CNAME to a Managed Edge — When delegating traffic steering to a CDN or cloud load balancer that changes its IPs dynamically.
- Weighted DNS for Progressive Rollouts — When you want to shift a percentage of traffic to a new deployment without client changes.
Strategies
- Pick TTLs Per Record Purpose When: Always — TTL choice is a deliberate availability/latency decision, not a default. How: Stable records (mail servers, verification TXT) get long TTLs like 86400 s. Records behind failover or frequent deploys get 30–300 s. Before a planned migration, drop the TTL a day early, wait one old-TTL period, make the change, then raise it again. Example: An e-commerce site keeps its CDN CNAME at 3600 s but its failover-managed origin A records at 60 s, so a region cutover converges within about a minute of cache expiry.
- Treat DNS as a Dependency with Its Own SLO When: When your availability target is 99.9% or higher — your DNS provider's outage becomes your outage. How: Choose a provider with anycast infrastructure and published SLAs, monitor resolution latency and failure rate from multiple regions, and for critical systems consider dual DNS providers (two authoritative providers serving the same zone) to survive a provider-wide incident. Example: After the October 2016 Dyn DDoS attack took down Twitter, Spotify, Reddit, and Netflix domains simultaneously, many large companies adopted dual-provider DNS so a single authoritative provider outage no longer makes them unresolvable.
- Layer DNS Steering Under Real Load Balancing When: When you need both coarse geographic steering and fine-grained, health-aware distribution. How: Use DNS to choose a region (coarse, slow-changing, cached), then a regional load balancer with active health checks to choose a server (fast-changing, immediate failover). DNS handles 'which continent'; the LB handles 'which machine is alive right now'. Example: A global API uses geo DNS to send users to one of three regions, and inside each region a load balancer removes unhealthy instances in seconds — combining DNS's global reach with LB-level reaction speed DNS cannot provide.
- Fail Over at the Client When DNS Can't React When: When TTL-bound DNS failover is too slow for your recovery target. How: Return multiple IPs or endpoints to clients and let the client retry alternates on connection failure. Mobile apps and service meshes commonly do this: they treat DNS answers as hints, not guarantees, and keep working through a stale answer by falling back to cached or secondary endpoints. Example: A mobile app caches the last-known-good API IPs; when the resolved IP times out, it tries the cached alternates before surfacing an error, surviving a stale-DNS window that pure DNS failover would not.
Why DNS Outages Take Down Everything
DNS failures are disproportionately catastrophic because DNS sits on the critical path of every connection and is invisible until it breaks. When an authoritative provider fails, existing caches keep sites reachable until TTLs expire — then resolution starts failing globally within minutes. Two famous examples: the 2016 Mirai-botnet DDoS on Dyn made major sites unresolvable across the US East Coast for hours, and the October 2021 Facebook outage started as a BGP withdrawal that made Facebook's authoritative nameservers unreachable, which then cascaded because Facebook's own internal systems also relied on DNS that was now failing — engineers could not even reach the badge systems to enter the data centers. The lessons generalize: keep DNS dependencies independent of the systems they serve (don't host your status page or outage tooling behind the same DNS), monitor resolution externally, and remember that 'my servers are healthy' means nothing if nobody can find them.
Tradeoffs
| Decision | Upside | Downside |
|---|---|---|
| Low TTL vs High TTL | Low TTLs enable fast failover and frequent traffic steering. High TTLs cut lookup latency (cache hits), reduce authoritative query load, and lower DNS provider costs. | Low TTLs mean more cold lookups, higher latency variance, and heavier authoritative load. High TTLs make failovers and migrations painfully slow — clients can hold a dead IP for hours. |
| DNS-Based Load Balancing vs Anycast/Load Balancers | DNS steering is simple, works for any client, and can route globally without proxies. Anycast and real load balancers react to failures in seconds and balance by actual load. | DNS is cache-bound (no precise control, uneven splits, slow failover) and has zero health awareness. Anycast/LB layers add infrastructure cost, complexity, and another system to operate. |
| Single DNS Provider vs Dual Providers | One provider is simpler to configure and avoids record-sync problems. Two independent authoritative providers keep your domain resolvable through a provider-wide outage or DDoS. | Single-provider setups inherit the provider's blast radius (Dyn 2016). Dual-provider setups must keep zones synchronized and double cost and operational surface area. |
Real World
| System | How it's used |
|---|---|
| Cloudflare 1.1.1.1 | Cloudflare's public recursive resolver is anycasted from 300+ cities, so nearly every user on Earth reaches a resolver within ~10 ms. It emphasizes privacy (no selling of query data, query-name minimization) and speed, and demonstrates how anycast plus aggressive caching makes DNS resolution effectively free on the request path. |
| Amazon Route53 | Route53 is an authoritative DNS service with a 100% availability SLA, offering latency-based routing, geo DNS, weighted records, and health-check-driven failover. Companies use it to direct users to the nearest AWS region and to automate regional failover, accepting the TTL-bound convergence time. |
| Facebook (October 2021 outage) | A misconfigured BGP update withdrew the routes to Facebook's authoritative DNS servers, making facebook.com and instagram.com unresolvable worldwide for ~6 hours. The cascade reached internal tools and physical badge access because they too depended on the now-dead DNS — a canonical lesson in keeping DNS independent from the systems it serves. |
| Dyn (2016 DDoS attack) | The Mirai botnet's DDoS on managed DNS provider Dyn made Twitter, Spotify, Reddit, Netflix, and dozens of other sites unresolvable for hours, even though their own servers were healthy. The incident is the standard argument for dual DNS providers and for treating authoritative DNS as critical availability infrastructure. |
| Netflix | Netflix clients resolve Open Connect appliance hostnames whose DNS answers are chosen to steer each viewer to a nearby cache server holding their content. Because apps and devices retry alternate endpoints when a connection fails, Netflix tolerates stale DNS answers better than a pure browser-based service would. |
Interview
Questions interviewers ask
- Walk me through what happens when a user types a URL into a browser, starting with DNS.
- How would you design DNS for a service deployed in three regions? How do users get routed to the nearest one?
- Your primary region just died. How does traffic fail over, and how long does it take? What limits that speed?
- Why is DNS-based load balancing not enough on its own? What do you layer under it?
What a strong answer covers
Candidate should describe the recursive resolution flow (resolver → root → TLD → authoritative), explain TTL/caching and its effect on failover time, know the main record types (A/AAAA/CNAME), mention anycast for global low-latency resolution, and articulate the limits of DNS load balancing (caching, no health awareness) plus the mitigation (health-checked load balancers, client retry, dual providers).
Common traps
- Claiming DNS failover is instant — ignoring that cached answers survive until TTL expiry.
- Using DNS round-robin as the only load-balancing mechanism with no health-checked layer beneath it.
- Forgetting that DNS itself is a dependency with a blast radius — one provider outage takes the whole site down.
- Putting a CNAME at the zone apex or stacking CNAMEs without mentioning the extra lookup latency.
Quiz
During a cold DNS lookup, what is the correct order of servers a recursive resolver contacts?
- Authoritative → TLD → root
- Root → TLD → authoritative
- TLD → root → authoritative
- Root → authoritative → TLD
The resolver walks the hierarchy top-down: root servers point to TLD servers, TLD servers point to the domain's authoritative nameserver, which returns the final record.
Why does a DNS-based failover take minutes rather than seconds even after you update the record?
- Authoritative servers batch updates hourly
- BGP must re-converge across the internet first
- Resolvers and clients keep serving the cached old IP until the TTL expires
- TCP connections must finish before new lookups are allowed
DNS is a cached system: every resolver and client holds the old answer for up to the record's TTL, so traffic shifts gradually as caches expire — there is no push mechanism.
What does a CNAME record do?
- Maps a name directly to an IPv4 address
- Aliases one domain name to another name, which is then resolved separately
- Encrypts DNS queries between client and resolver
- Maps an IP address back to a domain name
A CNAME defers the answer: www.example.com → example.cdn.net, and the resolver then looks up example.cdn.net to find the actual A/AAAA record.
What is the fundamental limitation of DNS round-robin as a load-balancing mechanism?
- It only supports two servers at a time
- It has no health awareness and cannot control caching, so dead servers keep receiving traffic
- It requires clients to support HTTP/2
- It increases the size of DNS packets beyond the UDP limit
DNS will keep returning a dead server's IP until the record changes and caches expire; it cannot see server health, connection counts, or actual load — which is why health-checked load balancers sit beneath it.
What is the main benefit of anycast for a public DNS resolver like Cloudflare's 1.1.1.1?
- It encrypts queries end to end
- It guarantees every query bypasses the cache for fresh answers
- The same IP is announced from hundreds of locations, so users are routed to the nearest one for low latency and automatic resilience
- It lets one physical server impersonate many IP addresses
Anycast announces one IP from many PoPs; routing delivers each user to the closest instance, giving low latency worldwide and automatic failover when a location withdraws its announcement.
Idempotency
Learn why retries inevitably produce duplicate requests, how idempotency keys turn unsafe operations into safe ones, and why exactly-once delivery is a myth you design around rather than achieve.
This section is part of the full PRISM roadmap, with worked examples, trade-off tables, interview questions and a quiz.
Unlock the full lessonPractice how requests flow through systems in PRISM
Concepts stick when you watch them fail. Build an architecture that depends on how requests flow through systems, push traffic through it in the PRISM simulator, and see the latency and error rates change as you adjust the design.