Detecting Linux rootkits

What is a rootkit?

A rootkit is a set of tools with the goal to hide its presence and to continue providing system access to an attacker. The word rootkit comes from the root user, which is the administrator account on Linux systems and Unix-clones. The kit refers to a toolkit, or a set of tools.

Hiding by manipulation

The tools in the rootkit are typically altered binaries that provide an alternative truth. They will display everything a typical command would do, except those parts that are part of the rootkit itself. Some rootkits also provide an additional backdoor. This way the system can be fully patched and still allow the attacker to enter via the hidden entrance.

Rootkits are usually not installed by a system administrator. In fact, the system administrator is typically the victim of such software. He or she is seeing manipulated results when using common system tools like ls and ps.

Popularity of rootkits

Nowadays rootkits are less popular than they were before. This is partially due to some measures were taken in modern Linux kernel versions. The newer versions increase the difficulty to circumvent some areas like using some of the available system calls. As you may expect, the cat-and-mouse game is not over and backdoors remain very popular. Often the attacker doesn’t even need full root access to misuse a system for other purposes. Helping in a Distributed Denial of Service (DDoS), sending spam, or act as a hop to attack other systems, to name a few.

Rootkit detection

Methods to detect rootkit presence

Since rootkits are malicious, they should be detected as soon as possible. There are different ways to detect them, each with different rates of success. Let’s have a look at the various methods.

Behavioral analysis

Each system has processes running that consume resources like processor time and memory.  As rootkits alter the execution path, its behavior might become visible.

By timing common operations (profiling), it can be possible to detect alterations in the kernel or system tools. You would then compare them with a known-good state. This known-good state can be measured from a similar system, or even from the system at a given time. When it is different at a later stage, then this might be a possible hint and may require additional research.

Another detection method is by creating files tailored to what rootkits may use. When they don’t show up in the output of a system tool or system call, it may indicate a rootkit at work.

Memory analysis

Although rootkits can be masters of illusion, they need to use disk storage, run in memory, or both. If a rootkit is using memory, then memory analysis can be used for detection. An open source memory forensics tool like Volatility can capture the contents of memory and analyze it.

Using signatures

The antivirus industry uses fingerprints or signatures to detect common malware like viruses, worms, and backdoors. Some parts of the rootkit can be detected with the same approach. Many of the rootkits use hard-coded paths or names, making it easier to detect them.

Log file analysis

Suspicious events like daemons crashing could be a first indication of a system break-in. While it may not be directly related with a rootkit, it may be a hint. So monitoring the log files for unexpected events and crashes could be a useful step in the detection process.

Rootkit detection tools

File integrity tools

One method to detect alterations to a system is with the help of file integrity tools. These suites consist of several components:

  • File database
  • Checksums
  • Metadata
  • Utilities

The utilities are used to create and check checksums or fingerprints of files. They store the checksum in the database, together with metadata. Examples of metadata include the ownership, permissions, and timestamps of a file. Typically detection occurs at the moment when the current state is compared with an earlier moment in time. Previously it was Tripwire that was known to do this. Nowadays AIDE and Samhain are used for this purpose.

Rootkit scanners

Specialized tools exist to detect traces of rootkits. These rootkit scanners search for common and uncommon files, compare the outputs of different utilities and try to trick a rootkit in revealing itself again. Rootkit Hunter and Chkrootkit are the most known tools.

Screenshot of rkhunter performing a rootkit scan on Linux system

Rootkit Hunter at work

Rootkits and false positives

Most detection methods are not fool-proof. So-called false positives are common. This means that a finding is raised that actually is not related to the work by a rootkit. Tools try to reduce these false positives as much as possible. With the fine line between malicious software and just innocent system behavior, it is hard to be 100% secure.

 

Frequently Asked Questions

Is a rootkit harmful to the system?

Yes, it is. A rootkit is malicious software and therefore unwanted malware.

What is the best way to detect a rootkit?

Use a malware scanner combined with file integrity monitoring and accounting of suspicious system calls. AIDE can be used for file integrity monitoring, complemented by the Linux Audit Framework to monitor changes and used system calls.

How can I remove a rootkit?

As a rootkit is a master of illusion, it is almost impossible to remove it and being 100% sure of its complete removal. For that reason, the general advice is to do a fresh installation instead.

Which tools can I use to detect a rootkit?

The detection of rootkits can be done by generic intrusion detection tools, malware scanners, or specific rootkit scanners for Linux.

Tools

Some tools mentioned in this post:

 

Did you find this article useful? Great! Make our community smarter and share this knowledge.

One more thing...

Keep learning

So you are interested in Linux security? Join the Linux Security Expert training program, a practical and lab-based training ground. For those who want to become (or stay) a Linux security expert.

See training package




Lynis Enterprise screenshot to help with system hardeningSecurity scanning with Lynis and Lynis Enterprise

Run automated security scans and increase your defenses. Lynis is an open source security tool to perform in-depth audits. It helps with system hardening, vulnerability discovery, and compliance.


Download

2 comments

  • Jason SotoJason Soto

    Hi Michael… I Would add Unhide to this list…. Unhide is a forensic tool to find hidden processes and TCP/UDP ports by rootkits / LKMs or by another hidden technique. .. Te project page is… http://www.unhide-forensics.info/ …..

    Reply
    • Thanks, that is indeed a useful addition for the readers. Being the original author of Rootkit Hunter, I’m familiar with this great tool. So thanks for sharing!

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.