Linux DNS Tuning for Performance and Resilience

DNS Configuration on Linux

We often don’t realize the importance of DNS, or name resolving in our infrastructure. The impact when things go (slightly) wrong is huge. Time to have a good look at improving our DNS configuration.

How DNS resolving works

When your Linux system needs to know the IP address of a particular host, it will use gethostbyname(3) function. This will use the nsswitch configuration stored in /etc/nsswitch.conf. For the related hosts line, it will determine how to do resolving.

Screenshot of /etc/nsswitch.conf file for DNS resolving — *The order specified, determines how resolving will tried for each lookup*

It is common to find the usage of the word hosts, which refers to the /etc/hosts file, a static list to be configured by the system administrator. It is then often followed by the word dns, which specifies it can use DNS queries to get the answer. Here our journey begins to query nameservers. To know which nameservers should be used, the /etc/resolv.conf file is consulted. Each nameserver is prepended with the word nameserver, followed by the IPv4 or IPV6 address of a DNS resolver.

Users who use systemd, might actually have a “resolve” in their nsswitch.conf configuration, pointing to the systemd-resolved service. It performs cache and aggregation of DNS related settings.

Timeouts and settings

Most Linux administrators, have a minimal configuration stored in the /etc/resolv.conf file. To counter the impact of an unreachable DNS system, we can do a few things.

Define multiple nameservers

The easiest option is adding more nameservers. If you have defined just one, you are fully relying on the availability of that particular system. Also when that particular nameserver has a higher load of queries to process, your services will be affected.

Note: Most Linux distributions use /etc/resolv.conf directly. Some use /etc/resolvconf.conf, or /etc/systemd/resolved.conf when using systemd-resolvd. So check carefully what your system is using before making adjustments.

Limit DNS timeout

In the resolver configuration, we can change the timeout of DNS queries. If we don’t get an answer within a specified amount of time, we continue using the next system. By default, this can be as long as 30 seconds! That means if a nameserver is not available, it will take a while before it tries the next one. And typically it won’t remind that this nameserver is down, so other DNS requests may have to experience the timeout as well. So change your configuration file and set it to a much lower time.

options timeout:1

Maximum DNS resolve attempts

Typically it doesn’t make sense to repeat the same request multiple times to the same system. This is especially true when the related system is down. Restrict the number of attempts:

options attempts:3

Divide requests

It doesn’t always make sense if all systems use the same system. The name resolver allows you to rotate the requests. So the first request might go to 10.0.0.1, while the second goes to 10.0.0.2.

options rotate

There is a little caveat: rotation can have a negative impact. The performance of a busy nameserver is typically better, as it can serve data from its cache. So the busier the nameserver, the quicker it usually is to respond to requests. Therefore use this setting only when you are sure that all involved nameservers are fairly busy.

Local caching for Speed

The quickest network packet is the one which does not need to travel the network. For most systems a lot of repetition is involved in the tasks it is doing. This also applies for DNS requests. Very often a request will be made for a host we recently already contacted. Linux systems do not cache DNS requests by default. This means that a lot of traffic is sent to the network, for nothing!

We can counter the repetition of requests towards our nameservers, by using caching. These tools can run locally, and cache both positive and negative matches. In other words, it has a different timer for both types. If a name or IP address can’t be found, it may be caching that result shorter (or longer). This improves the caching table and hit rates on the cache, while limiting the bad impact of a “miss”.

dnsmasq

One of the options to consider is the toolkit dnsmasq. It can provide DHCP and DNS services for smaller networks. It is also a great candidate to run just a local DNS cacher. After installation, configure it so that it just does DNS resolving. Then provide it with some nameservers, so it can externally request the answers. Last step is pointing your normal resolver configuration to 127.0.0.1, so dnsmasq will be the middle man to deal with DNS requests.

Arch Linux - pacman -Ss dnsutils

These utilities might be handy during configuration, testing, and troubleshooting.

drill
dig
host