Linux security guide: the extended version

Feeling overwhelmed with the resources available to secure your Linux system? This security guide will provide you the points where to start.

With this Linux security guide, we walk step-by-step through the options, tools, and resources. After reading this article, you will be able to make educated decisions about what Linux security defenses to implement for your systems.

You will be introduced to the right tools that help you automate and test your improvements.

Instead, related articles and resources will be available in the text. The goal is to make this guide into a go-to article for when you need to secure your Linux installation.

Goals

Learn how to select the right Linux distribution
Determine which basic security controls are mandatory for each system
Select the right security tools for Linux systems
How to repeat system hardening on more than one system
Know which automation steps can be taken

Security steps

Select the right Linux distribution

The first step for Linux security is selecting a “healthy” Linux distribution. There are many distributions, each with their own advantages. You may select the distribution on personal preferences, like who maintains it, how commercial it is, or which uses your preferred package manager. We suggest using a distribution that is common, has a clear release schedule, and releases security bulletins.

Let’s have a look at some examples.

Arch Linux

The Arch Linux distribution is cutting edge, using the latest software components available. It uses pacman to keep the system up-to-date. As its focus is always to run the latest versions, this distribution is great for research and desktop usage. It may be less suitable for business-critical servers or beginners to Linux. Arch Linux has a plus on security, as it has an extensive wiki with security topics, including security tips for hardening an Arch Linux system.

CentOS, Fedora, and RHEL

These two Linux distributions have a different relation to Red Hat, yet are very similar. For personal use or less critical business activities, these distributions may be a good choice. The package manager on CentOS won’t show security related updates. Additional care is needed to keep systems up-to-date to prevent using any vulnerable software.

For Fedora, you can choose to show security updates only and keep a system properly patched (even automatic security updates). Fedora is considered to be an experimental Linux distribution by Red Hat, making it more vulnerable to early adopter errors.

Those with a budget might go for Red Hat Enterprise Linux. RHEL is a good choice if you like the company itself and need stability. As you may expect, Red Hat provides security bulletins and does also contribute to the Linux kernel. It does also do development on security-related products like SELinux. The downside of this distribution is due to stability, most tools are not the latest and the greatest.

Debian and Ubuntu

Debian is well-known for its security, both good and bad. It made some mistakes in the past, like a change that resulted in a weak random number generator. At the same time, it is very keen on having a solid platform to package software and push out security updates when they are available.

Ubuntu is based on Debian and evolved over the years. It is supported by Canonical, and their long-term support (LTS) versions are appreciated by businesses. Both Debian and Ubuntu have the ability to run the AppArmor security framework. They also have people contributing to the Linux kernel like Red Hat does.

Other distributions

There are a lot other Linux distributions and even options like macOS and the BSD family (OpenBSD, NetBSD, FreeBSD, DragonFlyBSD, etc). To learn about more options, we can recommend DistroWatch for a more extensive overview of all the available options.

Choose security during installation

The installation process is the fundament of your system. It will have a great influence on your security posture later on. Typically you want to start safe from the start, as bolting on security defenses on an existing system will consume more time and risk. Doing the right things from the start makes things easier, increasing security and possibly even privacy.

Separate file systems

When possible separate your file systems, so each of the mount points can be tuned. Most likely you won’t need device files on your /tmp partition, which is one of the many things you can do to harden the file system.

Use the minimal installation option

Most Linux distributions have the option to select a minimal installation. Each package has the chance to have a security vulnerability at one time, so only install what is really needed. It also reduces installation time and makes the footprint of the system smaller, resulting in quicker upgrades and better performance.

Encrypt where possible

Some Linux distributions allow you to use LVM with an encrypted file system. This is a great option to protect your data when a disk or complete system is stolen. It also helps when returning disks to a vendor and knowing that vendor will never be able to see any data on the disk.

Use a strong passphrase to protect the data. If you deploy encryption for multiple systems, consider using a passphrase with an additional string that makes each system unique. For example, something that can be easily looked up by you or your colleagues, yet hard to guess. For example a customer ID, or date of installation. Using the hostname might be less suitable if that is printed on the system itself.

Privacy tip: DNS

Most home networks are using DHCP to automatically assign an IP address, the default gateway, and the DNS servers. Have a look at what DNS servers are assigned. This might be room for improvement when it comes to privacy. Instead of using the DNS servers of the big companies, consider setting up your own system. DNS is light on resources and can be great to improve privacy for all systems on your network. It even allows you to monitor for infected systems by reviewing the DNS server logs.

Performing a security audit

System hardening is great, but how do you know you did all that is needed? That is why you perform a technical health scan of your system. We call this process a technical security audit.

To perform a security audit you will need the right toolkit. Important is to learn some of the tools and their main purpose. After all, there is no silver bullet in security. Fortunately, Linux provides a lot of security tools. Here is a top 3 for security auditing and vulnerability scanning.

Lynis
Nmap
OpenVAS

Get familiar with at least these three tools. Each of them provides valuable security insights. Also, they will be of great assistance during system hardening and to keep the system secured in the future.

Ongoing audits with Lynis

The first tool in alphabetic order is Lynis. This is an open source security scanner which runs on the host itself. It checks the configuration of the system, like a health scan for your body. Any room for improvement is shared in a report. Lynis can also detect vulnerable packages and other weaknesses. A huge benefit of this tool is that it is very light on requirements and resources. In less than 1 minute you can perform a scan. So have this tool installed on your systems.

Port scanning with Nmap

Next in line is Nmap. This versatile port scanner is another battle-tested tool. It helps you to detect open network ports and perform different types of scans. This includes vulnerability scanning and complements the previous tool. Nmap can perform both a check on the local system or do it from the network. It is a powerful tool, which also means it has many options. This can be overwhelming when using it for the first time. Using the tool with some examples from around the web will definitely help.

Vulnerability scanning with OpenVAS

Finally, there is OpenVAS. This is an open source vulnerability scanner. It has many tests to perform vulnerability scanning. Like Nmap, it performs its magic by scanning the network. Based on the available systems and services, it will look more specifically at available vulnerabilities. As this tool primarily focuses on vulnerability management, it takes a bit more time to set up than the other two. You will need a dedicated system with enough resources and have it update its database first.

Don’t run once: schedule

These three tools combined provide a good basis to detect weaknesses and continuously perform security checkups of your systems. Schedule Lynis to run daily on the machine. Have it email the results, or store the output on a central system. Use a daily difference check to show what has been changed. Do the same for nmap, to ensure no unwanted ports are opened unexpectedly. If you are using a mixed environment, nmap has also a way to show Linux systems only.

System hardening for Linux systems

Now it is time to start hardening the system. System hardening is the process of adding new defenses and removing weak spots in existing defenses. We remove these weak spots by minimization, detection of vulnerabilities, and adjusting weak configuration defaults.

If you are new to Linux security, then read the article How to secure a Linux system. It provides a good introduction to these principles.

Principle of minimization

Every piece of unneeded ballast on the system should be removed (as much possible). This includes users, processes, files, directories, empty log files, etc. Sure, don’t overdo it, as it may result in a broken system. Often you can remove a lot without any impact to the system.

For example when atd is running, but you don’t use that scheduler, then simply remove it. You can find the related package a file belongs to, then remove that package. The same applies to unused user accounts. The article Unused Linux users: delete or keep them? provides background on user management.

Update software packages

The easiest way to kill a lot of vulnerabilities, or software weaknesses, is by installing the related software updates. An update is a patched version of a software package that has a known issue. This could be a critical bug causing data corruption, a new feature to make life easier, or solving a security weakness.

We speak about updates when it slightly changes the behavior of a program. Often these are minor releases (1.2.3 to 1.2.4, or 1.2.8 to 1.3.0). An upgrade is usually bigger, like moving to a new major version, or a new operating system version (Ubuntu 16.04 to 18.04). Both upgrades and updates help with getting better software and make system management easier. While there is always the chance of breaking some functionality or process, this should be no excuse to stop doing proper patch management. Use clusters, snapshots, and backups to your advantage, to allow regular software updates to be installed.

Weak configuration details

Most software is created to work as simple as possible. That often conflicts with the principle of secure by default. So for every package we install, we should carefully look at the impact it has on the system. Ask yourself the following questions:

Does it spawn a new process?
Does it listen on the network interfaces by default?
Does it require authentication in any way?

For each of these questions, we should find the answers and make an educated decision on what that might mean for the system.

Network traffic filtering

Linux allows filtering network traffic with the netfilter functionality. You may be familiar with the tools that use this kernel feature:

iptables
nftables
firewalld
bpfilter

Kernel support for netfilter is usually already enabled by the Linux distribution, so no recompilation is needed.

Even with a network-based firewall enabled, you may benefit from adding a local firewall. It will decrease the chance that a successful attacker may jump between hosts, as traffic streams to (and from) the system can be regulated. It does also allow tools like fail2ban to create a blacklist for that specific server, based on locally gathered intelligence.

Securing authentication on Linux

Linux systems usually have the PAM framework available. The abbreviation stands for Pluggable Authentication Module. It provides a stackable set of authentication modules. This stack then determines who can access the system and any specific conditions that might apply to the session. Not only does it filter out the authorized users, it can set shell specific settings and check for password strength.

Improve your passwords

You can use very weak passwords on Linux systems by default. That might be fine for your personal system, but less suitable for corporate systems. In such case have a look at pam_cracklib or the newer pam_pwquality. With these modules, the minimum password length can be enforced, or the variety in special characters that should be available. If you are used to two-factor authentication, consider using that for your most sensitive systems.

SSH hardening

Most systems use SSH to be managed. So it shouldn’t be a surprise that this access point needs some attention as well. If you run the OpenSSH daemon, perform SSH configuration hardening before deploying the system. If it is already running, then consider who is connecting to it, and restrict access using options like AllowUsers.

Due to the importance of this subject, we have some other related articles to SSH hardening. Determine what would make sense in your situation.

Audit file access and events

One of the least used options on Linux is still the Linux audit system, a kernel component developed by Red Hat. Although setting it up takes a little bit of time (and testing), it is very powerful to detect file changes and reporting the usage of specific system calls. Great for intrusion detection or even as a tool to troubleshoot.

Audit commands by users

Your security policy might be stating that all commands need to be logged. You can use the Linux audit system to capture commands executed by the root user, or any other user for that matter. But there are also other ways to capture some events on the system. In such case, you might want to configure Snoopy, a fairly easy way to capture that information.

Containers

Although container technology isn’t new, the implementation on Linux is fairly young. Containers help to encapsulate a service into its own bubble. This has a few advantages, like having a better picture of what kind of traffic needs to flow between applications. Containers are controlled using control groups and namespaces. This keeps them under control and limits the maximum resources they can use. Within the container, you can only see your own processes and user. From the outside (on the host itself), things look normal. In the end, each container is still a normal process.

While the container technology is taking on, there is still a lot of development going on. Most of it is in the supporting tooling, to make management easier. If you truly want to understand the capabilities of containers, I urge you to have a look under the hood and understand the techniques used. A few of them are:

cgroups
namespaces
seccomp
port filtering
read-only file system

Protect databases

When running a database like MySQL or PostgreSQL engine on your system, additional care should be taken for the system. As usually precious data is being stored, the first step is to check if your backup strategy is properly implemented. A normal file backup might be insufficient for proper database backups. Instead, create a database export with tools like mysqldump or pg_dump.

The next layer of database security includes setting up proper authentication. For example, ensure that the local ‘root’ user needs to authenticate. This is a common finding of Lynis, where simply no password is set. Use the strongest hashing algorithms as possible for authentication options, also if you have a custom web application. In such case use hashing together with salting. Your programming language should have the appropriate libraries to achieve that.

MySQL and MariaDB

my.conf (configuration)
mysql_secure_installation (hardening tool)

PostgreSQL

pg_hba.conf (configuration)

Hardening a web server

Setting up a web server is fairly easy. Protecting it properly against evil bots and attackers is a whole different story. A few things that every web server contain now is being HTTPS-only and restrict access to sensitive files. When possible a web application firewall (WAF) is a great addition to block SQL injections and other common attacks.

As setting up a web server in a secure way is an article in itself, here are some pointers to improve your web server.

Securing nginx configurations: implementing OCSP stapling
Configure HSTS (HTTP Strict Transport Security) for Apache/Nginx
[Optimize SSL/TLS for Maximum Security and Speed]({{ relref “optimize-ssl-tls-for-maximum-security-and-speed.md” >}})

Hardening WordPress

About 25% of the web is powered by WordPress. That is a nice achievement, yet means the software is a target as well. Keep your installation up-to-date and especially update plugins on a regular basis. Remove plugins that are badly maintained and replace them with others. Yes, your website functionality might need to be changed a bit, but that is always better than have to deal with a breach.

More resources

Understanding Linux privilege

System hardening provides additional layers of defense against malicious attempts to breach the system. To properly increase our Linux security defenses, we need to understand how attackers work, and in particular how Linux privilege escalation works. Like Sun Tzu said, you have to understand both your enemy and yourself. Only then you don’t have to fear the battle.

Linux security best practices and hardening guides

It is common to find “best practices” in the world of information security. These are commonly configuration settings and practices that are considered to be safe. For most companies and environments these are suitable. As always, you have to confirm if this also applies to your systems. Hardening guides are usually filled with these best practices.

Security conferences and video presentations

Although there are many conferences, there aren’t many that just focus on Linux security. If you are interested in video materials and presentations, check out the following resources.

Linux security summit
O’Reilly security conference

If you like this article, become part of the community and share it with your network. Got feedback? Use the comments at the bottom.

What additional defenses are you using and did we skip in this article?