systemd-resolved · netplan
What is changing /etc/resolv.conf? Find who sets your DNS (Ubuntu 24.04)
Published · HardStacked · 10 min
Your Ubuntu 24.04 server asks a DNS server that no longer exists. You open /etc/resolv.conf and it says 127.0.0.53, or, if you changed the link, a list with the wrong server in it and the line # Too many DNS servers configured, the following entries may be ignored. You edit it, and it comes back. The question has been on askubuntu since 2024-07-26 (question 1521666, 43,902 views on 2026-10-28, three answers, none accepted), and the person who asks puts it in one sentence:
"How can I remove whatever is setting this DNS server, incorrectly?"
Nobody edits your /etc/resolv.conf. It is a link to a file that systemd-resolved replaces whole, every time the DNS of an interface changes. So ask which interface brings that DNS, and where it got it. ls -l names the writer, resolvectl dns names the interface, the DHCP lease names the server that handed it out. When the lease does not match, the DNS was set at runtime, by a person or a script, and it survives a lease renewal and a restart of systemd-resolved until resolvectl revert. The fix is to refuse the DNS from DHCP on that interface. Adding a global DNS removes nothing.
Measured on an Ubuntu 24.04.5 LTS virtual machine (arm64, official cloud image), systemd-resolved 255.4, netplan.io 1.1.2, on 2026-10-28. Same on Ubuntu 26.04.1 LTS (systemd 259.5), except DHCPv4 via instead of DHCP4 via.
01. Nothing edits the file: the lab and the trap
You can reproduce this without touching your real network. We build a virtual interface, demo0, wired to a fake router: a network namespace where dnsmasq does DHCP and nothing else, and hands out three dead DNS servers, 198.51.100.112 to .114. Nobody owns these addresses, like the defunct server of the question (the 198.51.100.x and 192.0.2.x ranges are reserved for documentation, RFC 5737). Safe: no gateway (--dhcp-option=3 is empty), only demo0 is touched, everything is removed at the end. 10.0.2.3 is the normal DNS of the VM (interface enp0s1); on your machine it will be yours. We cut the debconf and needrestart lines of apt-get.
$ sudo apt-get install -y -qq dnsmasq-base inotify-tools >/dev/null
$ sudo ip netns add demo-router
$ sudo ip link add demo0 type veth peer name demo0-r
$ sudo ip link set demo0-r netns demo-router
$ sudo ip -n demo-router addr add 198.51.100.1/24 dev demo0-r
$ sudo ip -n demo-router link set demo0-r up
$ sudo ip netns exec demo-router dnsmasq --conf-file=/dev/null --port=0 --interface=demo0-r --bind-interfaces --dhcp-range=198.51.100.50,198.51.100.99,12h --dhcp-option=3 --dhcp-option=6,198.51.100.112,198.51.100.113,198.51.100.114 --pid-file=/run/demo-dnsmasq.pid --dhcp-leasefile=/run/demo-dnsmasq.leases
$ printf 'network:\n version: 2\n ethernets:\n demo0:\n dhcp4: true\n' | sudo tee /etc/netplan/60-demo.yaml
network:
version: 2
ethernets:
demo0:
dhcp4: true
$ sudo chmod 600 /etc/netplan/60-demo.yaml
$ sudo netplan apply && sleep 5
The trap: the file says 127.0.0.53, nothing abnormal. That is the local resolver of systemd-resolved, and it hides the real servers, kept per interface. resolvectl dns demo0 shows three dead ones. tcpdump on demo0 during a lookup shows ARP requests for 198.51.100.112: the dead server is asked, and example.net resolves anyway.
the trap
$ grep nameserver /etc/resolv.conf
nameserver 127.0.0.53
$ resolvectl dns demo0
Link 12 (demo0): 198.51.100.112 198.51.100.113 198.51.100.114
$ sudo timeout 8 tcpdump -tlni demo0 -c 3 arp and host 198.51.100.112 2>/dev/null & sleep 1; sudo resolvectl flush-caches; getent hosts example.net >/dev/null && echo 'example.net resolves'; wait
ARP, Request who-has 198.51.100.112 tell 198.51.100.72, length 28
example.net resolves
ARP, Request who-has 198.51.100.112 tell 198.51.100.72, length 28
ARP, Request who-has 198.51.100.112 tell 198.51.100.72, length 28
The manual: a query that matches no routing domain goes to all the DNS servers of the interfaces with DefaultRoute= set, plus the global one (systemd-resolved(8)), and the dead one did not hold up the answer. So a dead server does not break public names. It breaks what only it knew, the internal names of a lab. (example.net resolves falls between the ARP lines because two programs write at once.)
02. Who writes it, and the "Too many DNS servers" line
The link points to stub-resolv.conf. systemd-resolved(8) lists four modes for /etc/resolv.conf, detected from where the link points, and says resolved maintains stub-resolv.conf and resolv.conf in /run/systemd/resolve/. The whole directory belongs to systemd-resolve: the writer is named. We cut the two socket lines.
who writes it
$ ls -l /etc/resolv.conf
lrwxrwxrwx 1 root root 39 Oct 17 12:58 /etc/resolv.conf -> ../run/systemd/resolve/stub-resolv.conf
$ ls -l /run/systemd/resolve/
total 8
-rw-r--r-- 1 systemd-resolve systemd-resolve 934 Oct 28 18:44 resolv.conf
-rw-r--r-- 1 systemd-resolve systemd-resolve 920 Oct 28 18:38 stub-resolv.conf
That resolv.conf is the uplink file, the real servers with no stub, the list the person in the question was reading after switching the link to it. Here: the dead three, the warning, and our healthy DNS in 4th position.
the uplink file
$ grep -E '^nameserver|^# Too' /run/systemd/resolve/resolv.conf
nameserver 198.51.100.112
nameserver 198.51.100.113
nameserver 198.51.100.114
# Too many DNS servers configured, the following entries may be ignored.
nameserver 10.0.2.3
glibc reads three servers at most (resolv.conf(5), MAXNS = 3), so the 4th is ignored. Which one lands 4th is not stable: over five readings on 24.04 and 26.04, it was the healthy DNS four times and a dead one once, both seen on the same 24.04. Do not read that line as "your good DNS is ignored".
Editing the file by hand does not stick: your line is gone at the next lease renewal, even if the DNS is the same (grep -c gave 1, then 0), because the file is replaced and its inode number changes. The asker's other lead:
"Other posts that I have read said that /etc/resolv.conf will get fed from: /etc/resolvconf/resolv.conf.d/tail , that file does not exist."
That folder belongs to the resolvconf package (Debian's ships its head and base files there; tail is one you create), and on this 24.04 apt has no resolvconf package to install. Here the writer is the one ls -l names.
03. Which DHCP server handed out that DNS
Ask networkd about the interface. The lease file is named after the interface index.
$ networkctl status demo0 --no-pager | grep -E '^ *(Address|DNS):'
Address: 198.51.100.72 (DHCP4 via 198.51.100.1)
DNS: 198.51.100.112
$ grep -E '^(SERVER_ADDRESS|DNS)=' /run/systemd/netif/leases/$(cat /sys/class/net/demo0/ifindex)
SERVER_ADDRESS=198.51.100.1
DNS=198.51.100.112 198.51.100.113 198.51.100.114
SERVER_ADDRESS is the culprit: a DHCP server, named by its address. If the router was fixed since and the old DNS lingers, networkctl renew demo0 asks for a new lease.
04. The DNS nobody wrote down
Now the case where the lease is innocent. We set a DNS on demo0 by hand, renew, restart systemd-resolved, read the lease.
a DNS set by hand
$ sudo resolvectl dns demo0 198.51.100.200
$ sudo networkctl renew demo0 && sleep 4 && resolvectl dns demo0
Link 12 (demo0): 198.51.100.200
$ sudo systemctl restart systemd-resolved && sleep 2 && resolvectl dns demo0
Link 12 (demo0): 198.51.100.200
$ grep '^DNS=' /run/systemd/netif/leases/$(cat /sys/class/net/demo0/ifindex)
DNS=198.51.100.112 198.51.100.113 198.51.100.114
$ sudo resolvectl revert demo0 && sleep 2 && resolvectl dns demo0
Link 12 (demo0): 198.51.100.112 198.51.100.113 198.51.100.114
The hand-set DNS survives the renewal and the restart. The lease still says .112, .113 and .114: it does not tell you what is active. resolvectl revert removes the hand-set DNS. resolvectl(1) says these commands inform "systemd-resolved or systemd-networkd", and networkd does hold it: in our runs, networkctl status demo0 then showed DNS: 198.51.100.200. It also survives netplan apply (measured on 26.04).
05. Watch the folder, not the file
To see the change as it happens, inotifywait on the file. Two changes, one seen. Then the folder.
file, then folder
$ sudo timeout 6 inotifywait -mq -e modify,attrib,move_self,delete_self /run/systemd/resolve/resolv.conf & sleep 1; sudo resolvectl dns demo0 198.51.100.201; sleep 1; sudo resolvectl dns demo0 198.51.100.202; wait
/run/systemd/resolve/resolv.conf ATTRIB
/run/systemd/resolve/resolv.conf DELETE_SELF
$ sudo timeout 6 inotifywait -mq -e create,moved_to /run/systemd/resolve/ & sleep 1; sudo resolvectl dns demo0 198.51.100.203; sleep 1; sudo resolvectl dns demo0 198.51.100.204; wait
/run/systemd/resolve/ CREATE .#resolv.conf22c36321bad9fea2
/run/systemd/resolve/ CREATE .#stub-resolv.conf3152481f6abd0d30
/run/systemd/resolve/ MOVED_TO resolv.conf
/run/systemd/resolve/ CREATE .#resolv.confe9d8f0efdac74324
/run/systemd/resolve/ CREATE .#stub-resolv.conf0d3055593eb59b9e
/run/systemd/resolve/ MOVED_TO resolv.conf
$ sudo resolvectl revert demo0 && sleep 2 && resolvectl dns demo0
Link 12 (demo0): 198.51.100.112 198.51.100.113 198.51.100.114
On the file: ATTRIB, DELETE_SELF, then nothing for the second change, because the watch follows the old file, deleted. On the folder, every change shows CREATE .#resolv.conf… then MOVED_TO resolv.conf: written next to it, then renamed over. Two tools saw nothing, measured on 26.04: an auditd rule on /etc/resolv.conf, even for a write made through the link (a rule on the target file did catch it), and fatrace 0.19 in /run.
06. The fix: refuse the DHCP DNS on that interface
The second answer of the question starts with "I didn't find what is changing…" and suggests DNS= in /etc/systemd/resolved.conf. We try it, with a documentation address.
a global DNS=
$ sudo mkdir -p /etc/systemd/resolved.conf.d
$ printf '[Resolve]\nDNS=192.0.2.53\n' | sudo tee /etc/systemd/resolved.conf.d/demo-global.conf
[Resolve]
DNS=192.0.2.53
$ sudo systemctl restart systemd-resolved && sleep 2 && resolvectl dns
Global: 192.0.2.53
Link 2 (enp0s1): 10.0.2.3
Link 12 (demo0): 198.51.100.112 198.51.100.113 198.51.100.114
$ sudo rm /etc/systemd/resolved.conf.d/demo-global.conf && sudo systemctl restart systemd-resolved
Global: 192.0.2.53 is added, and demo0 keeps its three dead servers. It adds, it removes nothing. The fix is on the interface. Netplan documents use-dns: "When true, the DNS servers received from the DHCP server will be used and take precedence over any statically configured ones. Currently only has an effect on the networkd back end."
the fix
$ printf 'network:\n version: 2\n ethernets:\n demo0:\n dhcp4: true\n dhcp4-overrides:\n use-dns: false\n' | sudo tee /etc/netplan/60-demo.yaml
network:
version: 2
ethernets:
demo0:
dhcp4: true
dhcp4-overrides:
use-dns: false
$ sudo netplan apply && sleep 5 && resolvectl dns demo0
Link 12 (demo0):
$ (sudo timeout 8 tcpdump -tlni demo0 arp and host 198.51.100.112 2>/dev/null | grep -c who-has) & sleep 1; sudo resolvectl flush-caches; getent hosts example.net >/dev/null && echo 'example.net resolves'; wait
example.net resolves
0
$ grep -E '^nameserver|^# Too' /run/systemd/resolve/resolv.conf
nameserver 10.0.2.3
demo0 has no DNS left, the count of ARP requests toward the dead server is 0, and the uplink file holds the healthy one only. Scope: the networkd back end, the server default. Ubuntu Desktop uses NetworkManager: not covered, not measured. If dhcp4 and dhcp6 are both true, networkd wants the same keys in both overrides.
The three answers. The top one (score 5) says to renew the lease and put the stub mode back: right if the DHCP server was corrected meanwhile, but it finds no one if DHCP keeps giving the same DNS, and a hand-set DNS survives the renew (section 04). The second (score 1) is the global DNS= above. The third (score 0) says /etc/resolv.conf is managed by "systemd-dnsmasq": we found no package and no manual page of that name on 24.04 or 26.04.
07. Clean up
Keep dnsmasq-base if it was installed before (Ubuntu Desktop installs it with NetworkManager) and skip the last apt-get remove in that case. tcpdump stays: Ubuntu ships it.
teardown
$ sudo rm /etc/netplan/60-demo.yaml
$ sudo netplan apply
$ sudo kill $(cat /run/demo-dnsmasq.pid)
$ sudo ip netns del demo-router
$ sudo rm /run/demo-dnsmasq.pid /run/demo-dnsmasq.leases
$ ip link show demo0
Device "demo0" does not exist.
$ sudo apt-get remove -y -qq dnsmasq-base inotify-tools >/dev/null
Not measured: NetworkManager, resolvconf, Debian.