Systemd features to secure units and services

Introduction

Securing a service means that one has to know what the underlying system functions and resources it uses. If we turn that around, it is as useful to know what it does not need. This information alone can be used to set boundaries of a service.

Systemd provides a huge set of features that may be used to set boundaries. Some define functions that can be used with the help of an allow list. Not on the list? Then the related action will be denied. Other functions limit what a process can see, such information about other processes, other users, or even a limited view of the file system.

Why securing our services?

To enable most of these features, we actually need the right set of permissions. Often it requires root permissions to lower our capabilities. Or in other words, we tell the process to give away some power.

You may wonder why we should invest the time in lowering the capabilities of a process. The most important reason is to stay out of trouble later on. For example, a system configuration that today is secure and fully patched, might be a weak spot in the matter of a day.

By configuring our processes to run with the bare minimum of permissions, we may be protected against a software vulnerability. After all, if a particular function is not used under normal conditions, it can also not be misused if we restrict access to it.

Organization of a systemd unit

To better understand what there is to secure, let’s have a look at common service running on the web or network: nginx.

systemctl cat nginx.service

The output may look like this:

[Unit]
Description=A high performance web server and a reverse proxy server
Documentation=man:nginx(8)
After=network-online.target remote-fs.target nss-lookup.target
Wants=network-online.target

[Service]
Type=forking
PIDFile=/run/nginx.pid
ExecStartPre=/usr/sbin/nginx -t -q -g 'daemon on; master_process on;'
ExecStart=/usr/sbin/nginx -g 'daemon on; master_process on;'
ExecReload=/usr/sbin/nginx -g 'daemon on; master_process on;' -s reload
ExecStop=-/sbin/start-stop-daemon --quiet --stop --retry QUIT/5 --pidfile /run/nginx.pid
TimeoutStopSec=5
KillMode=mixed

[Install]
WantedBy=multi-user.target

This unit has three sections: Unit, Service, and Install.

[Unit]

First part is the [Unit], which defines the basics of the unit, including a description. It tells the unit if it has any dependencies. As there is no point running a web server if the network is not available yet, it is defining the network-online.target as its dependencies. It also tells the service manager that in what order to start up, in this case after the network is online, file systems are available, and host and name resolution services are up and running.

[Service]

The most interesting section is the second one. Within the [Service] definition we define what we are going to do. Not just what we start, but also what we expect. Things like where we should store the process ID, if we have preparation steps before starting our main program, and even what to do in the event things go wrong. This section is also the place where we can do security tuning of our unit.

[Install]

The latest section, [Install], defines at what stage this service should be started. In this case the common multi-user target, similar to what before was runlevel 3.

No restrictions by default

If we look the example above, we there are actually no restrictions defined. In other words, the process can do almost all it wants, unless it is restricted by file permissions, a security framework like AppArmor or SELinux, or by a generic kernel feature.

Why aren’t systemd units secured by default?

Every system has its own needs and its own resources. The developer of the software, nginx in this case, does not know how you will be using the software. Even though the developer typically knows what system functions are used (or not), it may have limited resources to define a good basic set of required permissions. For a small script it may be easy, but a network service with many available modules, that is another story.

Next in line is the package maintainer, the one who creates package and includes the basic unit file. This maintainer has typically less knowledge about the inner workings of the software, and struggles with the same question: what will the user exactly be doing with the software and what access or resources does it need? To make sure that the software works, he or she will be defining a basic unit file. That leaves you, as the system administrator, to do the security hardening of the service.

Available security features

So knowing that most systemd units have no security hardening by default, it is time to have a look at the available features that systemd has to offer. Since there is not a “one-fits-all” approach, we will be looking first at the available features. This is also the good and the bad part, there are many options available. Covering them all in one article would make it into a small book. Instead, have a look at the unit settings. That is (becoming) a long list.

Let’s take a step back, and cover a few important things related to these security features before we start making changes.

Not all features may be available to your system

Some of the features did show up at a later systemd version that your system is running. So it’s good to check first what version you have on your system.

systemctl --version

This output will also show another important part: systemd may not be compiled with all available features. If it has support, it is prepended with a plus sign, otherwise a minus.

Another important part is the Linux kernel itself. While some features may be present in systemd, under the hood it usually makes use of what the Linux kernel has to offer. So if a particular mount option is not supported by the kernel, then systemd can’t make use of it. Sometimes there are workaround, but often it means that implementing this feature has to be skipped.

Implementing requires some knowledge

Another limitation of implementing the available features is the amount of knowledge about a process, how the Linux kernel works, and what this means in relation to the systemd features. Where possible, we cover this here in other articles.

Good to know: There is a list of abbreviations in case you are not familiar with a specific term. In addition, there is a list of definitions to explain what some things refer to. If a subject requires more explanation that just a single line, often there is an article. Use the glossary or search functionality.

Changing systemd units

The location of a unit can be easily displayed by using systemctl with the subcommand cat. The first line will show this.

systemctl cat nginx.service

Making changes to these files is not recommended. These vendor-supplied files are meant to be clean and may be overwritten. Instead, we use the ‘drop-in’ functionality, by using the edit subcommand. This will create a separate configuration file (override), and they are merged by systemd.

systemctl edit nginx.service

Upon running this command, your defined system editor will open. Read the comments carefully, as only changes between the comments at the top and in the middle will be applied. To add a specific feature to our Service section, it may look like this:

### Editing /etc/systemd/system/nginx.service.d/override.conf
### Anything between here and the comment below will become the new contents of the file

[Service]
ProtectSystem=strict

### Lines below this comment will be discarded

### /lib/systemd/system/nginx.service

After saving the file, you may validate the file using systemctl cat before. The changes should be reflected and are listed below the original configuration.

Applying systemd security features to actual services

With this foundation, we can start tuning actual services. Here is a good start: harden nginx with systemd