Most DNS debugging goes wrong because people picture one lookup. There are several, at different layers, each with its own cache and its own idea of the answer.

It starts on your own machine

Your application almost never speaks DNS. It calls getaddrinfo(), and the system resolver, the stub resolver, decides what that means. On Linux that decision comes from /etc/nsswitch.conf, whose hosts: line usually reads files dns or files mymachines resolve [!UNAVAIL=return] dns. Left to right, first answer wins.

files is /etc/hosts, which is why a hosts entry beats every nameserver on the network. Then comes whatever local caching daemon is installed: systemd-resolved listening on 127.0.0.53, nscd on older systems, Docker's embedded resolver on 127.0.0.11 inside a container. Only after that does a query leave the machine, to the nameservers in /etc/resolv.conf.

Three consequences fall out of this immediately. A hosts entry is invisible to anything that does its own DNS instead of calling getaddrinfo, which includes browsers on DNS-over-HTTPS and several Go programs. /etc/resolv.conf inside a container is written by the container runtime and has nothing to do with the host's file. And dig deliberately skips this entire layer: it builds its own query and sends it straight to a nameserver, so it never reads /etc/hosts and never sees your local cache. That single difference explains a good share of "the tool says one thing, the app says another" reports.

The walk: root, TLD, authoritative

Your stub resolver asks a recursive resolver, and the recursive resolver does the actual work. It is the only participant that walks the tree; everything else answers one question about one zone. The design is straight out of RFC 1034, published in 1987 and still structurally accurate.

For a cold lookup of www.example.com:

  1. The resolver asks a root server about com. The root does not know the answer and does not pretend to: it returns a referral, the NS records for .com, plus glue records with their IP addresses so the next step does not need its own lookup.
  2. It asks a .com server about example.com. Another referral, this time to the nameservers listed in the registrar's delegation.
  3. It asks one of those about www.example.com and gets an authoritative answer, marked with the AA flag.
  4. It caches everything it learned along the way and hands the record back to your machine.

Nothing was pushed anywhere. Each server answered a question about the zone it is responsible for, which is why the mental model of records "spreading across the internet" is wrong in a way that leads to real mistakes, an argument we make in full in DNS propagation is a myth.

Modern resolvers do this more carefully than the original design. With QNAME minimisation, the root is asked only about com rather than being handed the full name, so an internal hostname does not leak to an organisation that only operates the .com zone. It is on by default in current Unbound, BIND and Knot Resolver.

The five caches between you and the answer

When a change does not seem to take effect, the useful question is which of these is holding the old value.

CacheLifetimeHow to inspect it
Browserseconds to minutes, browser-specificchrome://net-internals/#dns
Application runtimeJVM and some HTTP clients cache independentlynetworkaddress.cache.ttl in Java
OS stub / local daemonthe record TTL, if it caches at allresolvectl statistics
Recursive resolverthe record TTL, counted down from its own fetchdig @1.1.1.1 example.com and watch the TTL fall
Authoritative servernone, it is the sourcedig @ns1.example.com example.com

The runtime row surprises people most often. A long-running JVM keeps its own resolution cache governed by networkaddress.cache.ttl, and in some configurations it has historically been effectively permanent, so a database failover that every other client follows within a minute leaves one Java service talking to an address that no longer serves anything. Connection pools make this worse by holding the connection open regardless.

The TTL you see in a dig answer from a public resolver is a remainder, not a setting: it counts down from whenever that resolver last fetched the record. Query the authoritative server directly and you get the configured value. The DNS lookup tool queries public resolvers over DoH straight from your tab, so you can compare what two of them currently hold without waiting for either to expire.

/etc/resolv.conf can carry a search list and an ndots option, and together they decide whether a name is tried as written or appended to each search domain first. The default ndots is 1: a name containing at least one dot is tried as an absolute name first.

Kubernetes sets ndots:5. Since api.stripe.com has two dots, fewer than five, it is treated as relative and expanded first:

  • api.stripe.com.default.svc.cluster.local → NXDOMAIN
  • api.stripe.com.svc.cluster.local → NXDOMAIN
  • api.stripe.com.cluster.local → NXDOMAIN
  • api.stripe.com → the answer

That is four queries, and eight once the resolver asks for A and AAAA in parallel, for every external hostname a pod resolves. At scale it is a measurable share of cluster DNS traffic and a recurring cause of latency spikes when CoreDNS is under pressure. Two fixes: write the trailing dot (api.stripe.com., an absolute name, no expansion) or set a lower ndots in the pod's dnsConfig. The value 5 exists so that a bare service name like postgres resolves inside the cluster; if your code uses fully qualified names anyway, you are paying for a feature you do not use.

UDP, 512 bytes and the TCP fallback

DNS runs over UDP port 53 because a query and its answer normally fit in one packet each. The original limit was 512 bytes; anything larger came back with the TC flag set and the client repeated the whole query over TCP.

EDNS0 improved this by letting a client advertise a bigger buffer. Resolvers have converged on around 1232 bytes to stay inside common path MTUs, because fragmented UDP is unreliable and, in the DNS case, a security problem.

What still overflows: long TXT record sets, DKIM public keys, SPF records that grew by acquisition, and DNSSEC signatures. If TCP port 53 is blocked by a firewall someone configured while thinking only about UDP, ordinary A lookups work and mail authentication lookups fail intermittently, which is a memorable afternoon. The counterpart problem, an SPF record that exceeds the ten-lookup limit, is in SPF, DKIM and DMARC explained.

The 512-byte limit also explains a piece of trivia: there are exactly 13 root server addresses because 13 NS records with their glue was the most that fit in one unfragmented UDP response. Behind those 13 names sit well over a thousand anycast instances.

Your browser has its own resolver now

Since around 2020, browsers have shipped DNS-over-HTTPS. Firefox enabled it by default for users in the United States; Chrome uses it automatically when your configured resolver is known to support it. Queries then travel as HTTPS requests on port 443 to a provider chosen in the browser, not in the operating system.

For privacy this is a real improvement: the local network can no longer read or rewrite your lookups. For debugging it adds a resolver that none of your usual tools can see. The symptoms are familiar once you know to look for them. dig returns the new address, the browser keeps loading the old site. A hosts-file override does nothing in the browser. A corporate split-horizon setup stops resolving internal names, because the internal zone exists only on the resolver the browser has just stopped using.

Check the browser's Secure DNS setting before assuming a caching problem, and use chrome://net-internals/#dns to see and clear its own cache. The protocol is RFC 8484 if you want the wire format; the practical point is simply that "the machine's DNS settings" are no longer a single source of truth.

Reading a resolution path

Two commands cover almost every case. dig +trace example.com performs the walk yourself, one referral at a time, starting from the root hints and ignoring your recursive resolver's cache entirely. It shows exactly where a delegation is wrong, which is what a stale glue record or a half-finished nameserver change looks like from the outside.

Then dig @ns1.example.com example.com +norecurse asks the authoritative server directly. If that answer is right and a public resolver's is wrong, you are waiting on a TTL. If the authoritative answer itself is wrong, no amount of waiting helps.

For the reverse direction, a PTR lookup answers the different question of what an address claims to be, and it is worth knowing that forward and reverse are separate zones with separate owners, which our reverse DNS lookup lays out along with the forward-confirmed check that mail servers apply. The two directions disagreeing is normal on cloud hosts and a real problem on a mail server.

Resolver questions

How many servers are involved in resolving one domain name?

For a cold lookup of www.example.com, four: your recursive resolver asks a root server for .com, a .com TLD server for example.com, the authoritative server for the record, and returns it to your stub resolver. That is three round trips from the resolver plus one from your machine. In practice almost all of it is cached, and a resolver that already knows the .com nameservers skips straight to the authoritative query, which is why a warm lookup takes a few milliseconds and a cold one can take a couple of hundred.

Why does my browser resolve a domain differently from dig?

Because they are not asking the same resolver. Chrome and Firefox ship DNS-over-HTTPS and, when it is enabled, send queries straight to their own configured provider over port 443, bypassing the system resolver that dig uses. Both browsers also keep an internal cache with its own lifetime. So dig can show the new IP while the browser holds the old one, and neither is wrong. Check chrome://net-internals/#dns for the browser’s view, and its Secure DNS setting for which resolver it is really using.

How do I flush the DNS cache on macOS, Windows and Linux?

macOS: sudo dscacheutil -flushcache; sudo killall -HUP mDNSResponder. Windows: ipconfig /flushdns. Linux with systemd-resolved: sudo resolvectl flush-caches, and resolvectl statistics shows whether it did anything. None of these touch the cache of your upstream recursive resolver, which is usually the one holding the stale answer, and none of them touch a browser’s internal cache.

Does the hosts file override DNS?

On most systems yes, because /etc/nsswitch.conf lists files before dns and the stub resolver reads /etc/hosts first. It applies only to lookups that go through the system resolver: anything using DNS-over-HTTPS in a browser, or a language runtime that talks to a resolver directly instead of calling getaddrinfo, will ignore it completely. That gap is the reason a hosts-file override sometimes works for curl and does nothing for the browser tab next to it.

What is the ndots:5 problem in Kubernetes?

Kubernetes writes options ndots:5 into a pod’s /etc/resolv.conf, meaning any name with fewer than five dots is first tried against every search domain before being tried as written. Looking up api.stripe.com therefore produces queries for api.stripe.com.default.svc.cluster.local, then .svc.cluster.local, then .cluster.local, and only then the real name: four failed lookups per external hostname, doubled if the resolver queries A and AAAA. Fix it with a trailing dot (api.stripe.com.) or by setting a lower ndots in the pod’s dnsConfig.

Are there really only 13 root servers?

There are 13 root server addresses, named a.root-servers.net through m.root-servers.net, but well over a thousand physical instances behind them, spread across the world by anycast. The number 13 is a leftover from the 512-byte UDP limit: 13 was the most that fit in a single unfragmented response along with their glue records. Twelve independent organisations operate them, and the list your resolver ships with is called root hints, refreshed by a priming query at startup.

What is QNAME minimisation?

It is the practice of sending each server in the chain only the part of the name it needs. Without it, a resolver asking a root server about www.internal.example.com hands the full name to an organisation that only knows about .com. With it, the root is asked only about com, the .com servers only about example.com, and the full name reaches nobody but the authoritative server. It was specified in RFC 7816 in 2016, updated by RFC 9156, and is on by default in current Unbound, BIND and Knot Resolver.

Why does a DNS query fall back to TCP?

Because the answer did not fit. Classic DNS over UDP capped responses at 512 bytes; a server with more to say sets the TC (truncated) flag, and the client repeats the query over TCP. EDNS0 raised the practical limit by letting the client advertise a larger buffer, commonly 1232 bytes today to stay under typical MTUs and avoid IP fragmentation. Large TXT sets, long DKIM keys and DNSSEC signatures are what usually push a response over the edge, so a firewall that blocks DNS over TCP breaks exactly those lookups while ordinary A records keep working.