Detecting Linux rootkits

Malware, or malicious software is also an issue on Linux systems. Let’s have a look into this threat and what actions you can take.

What is a rootkit?

A rootkit is a set of tools with the goal to hide its presence and to continue providing system access to an attacker. The word rootkit comes from the root user, which is the administrator account on Linux systems and Unix-clones. The kit refers to a toolkit, or a set of tools.

Hiding by manipulation

The tools in the rootkit are typically altered binaries that provide an alternative truth. They will display everything a typical command would do, except those parts that are part of the rootkit itself. Some rootkits also provide an additional backdoor. This way the system can be fully patched and still allow the attacker to enter via the hidden entrance.

Rootkits are usually not installed by a system administrator. In fact, the system administrator is typically the victim of such software. He or she is seeing manipulated results when using common system tools like ls and ps.

Popularity of rootkits

Nowadays rootkits are less popular than they were before. This is partially due to some measures were taken in modern Linux kernel versions. The newer versions increase the difficulty to circumvent some areas like using some of the available system calls. As you may expect, the cat-and-mouse game is not over and backdoors remain very popular. Often the attacker doesn’t even need full root access to misuse a system for other purposes. Helping in a Distributed Denial of Service (DDoS), sending spam, or act as a hop to attack other systems, to name a few.

Rootkit detection

Methods to detect rootkit presence

Since rootkits are malicious, they should be detected as soon as possible. There are different ways to detect them, each with different rates of success. Let’s have a look at the various methods.

Behavioral analysis

Each system has processes running that consume resources like processor time and memory.  As rootkits alter the execution path, its behavior might become visible.

By timing common operations (profiling), it can be possible to detect alterations in the kernel or system tools. You would then compare them with a known-good state. This known-good state can be measured from a similar system, or even from the system at a given time. When it is different at a later stage, then this might be a possible hint and may require additional research.

Another detection method is by creating files tailored to what rootkits may use. When they don’t show up in the output of a system tool or system call, it may indicate a rootkit at work.

Memory analysis

Although rootkits can be masters of illusion, they need to use disk storage, run in memory, or both. If a rootkit is using memory, then memory analysis can be used for detection. An open source memory forensics tool like Volatility can capture the contents of memory and analyze it.

Using signatures

The antivirus industry uses fingerprints or signatures to detect common malware like viruses, worms, and backdoors. Some parts of the rootkit can be detected with the same approach. Many of the rootkits use hard-coded paths or names, making it easier to detect them.

Log file analysis

Suspicious events like daemons crashing could be a first indication of a system break-in. While it may not be directly related with a rootkit, it may be a hint. So monitoring the log files for unexpected events and crashes could be a useful step in the detection process.

Rootkit detection tools

File integrity tools

One method to detect alterations to a system is with the help of file integrity tools. These suites consist of several components:

• File database
• Checksums
• Metadata
• Utilities

The utilities are used to create and check checksums or fingerprints of files. They store the checksum in the database, together with metadata. Examples of metadata include the ownership, permissions, and timestamps of a file. Typically detection occurs at the moment when the current state is compared with an earlier moment in time. Previously it was Tripwire that was known to do this. Nowadays AIDE and Samhain are used for this purpose.

Rootkit scanners

Specialized tools exist to detect traces of rootkits. These rootkit scanners search for common and uncommon files, compare the outputs of different utilities and try to trick a rootkit in revealing itself again. Rootkit Hunter and Chkrootkit are the most known tools.

Rootkits and false positives

Most detection methods are not fool-proof. So-called false positives are common. This means that a finding is raised that actually is not related to the work by a rootkit. Tools try to reduce these false positives as much as possible. With the fine line between malicious software and just innocent system behavior, it is hard to be 100% secure.

Frequently Asked Questions

Is a rootkit harmful to the system?

Yes, it is. A rootkit is malicious software and therefore unwanted malware.

What is the best way to detect a rootkit?

Use a malware scanner combined with file integrity monitoring and accounting of suspicious system calls. AIDE can be used for file integrity monitoring, complemented by the Linux Audit Framework to monitor changes and used system calls.

How can I remove a rootkit?

As a rootkit is a master of illusion, it is almost impossible to remove it and being 100% sure of its complete removal. For that reason, the general advice is to do a fresh installation instead.

Which tools can I use to detect a rootkit?

The detection of rootkits can be done by generic intrusion detection tools, malware scanners, or specific rootkit scanners for Linux .

Tools

Some tools mentioned in this post:

• Rootkit Hunter
• Chkrootkit
• OSSEC
• AIDE (Advanced Intrusion Detection Environment)
• Tripwire

Did you find this article useful? Great! Make our community smarter and share this knowledge.