Vulnerability Scanning: The Destiny to Disappointment?

Our digital world is full of hardware and software components. The big difference between the two is the quality. When hardware ships with defects, people will return it and talk badly about it. For software it is fine if things are not perfect from the beginning. It can be improved upon in steps, until most of its users are happy with it. Developers of this software often are some level of pressure. We already know that most of the security vulnerabilities are caused by proper training or lack of quality testing. And even then, it is hard to get everything right.

When we bring a new piece of software into our world, like our notebook or IT environment, we increased our risks. Another target to be exploited. To combat this, we can ensure our software patch management is done properly. The other common option is to actually scan for vulnerabilities and dealing with them, which is vulnerability management.

We get a lot of demo requests for our software, and we take the personal approach. Before we share a trial, we want to learn a little bit about our potential customer. Not to nag them, but to better understand if we are both a good match. Very often we tell such requester we are not a good match. In most of these cases, this is because they are looking for purely a vulnerability scanner. If you worked with Lynis or Lynis Enterprise, you know we perform security audits. Deep health checks on your servers, of which vulnerabilities is just a part.

Pain Points

With all the emails we received over the years, we found out this is the top 3 of “pain points”:

1. Patching - Customer finds it difficult to get a good software patching process in place, or properly work with it.
2. Next step - Customer does not know how to continue at different stages. Like determining what can be done next. Think if of how to move forward with system hardening for further improvement and reduce risk internally and for their customers.
3. Validation - Be able to validate the effectiveness of patching and hardening efforts, and properly report that.

Some people will directly say you will need a vulnerability scanner for that. Your (average) vendor of vulnerability scanner will say that as well. But really, is it a solution? Their focus is broad, to detect many different flaws. At the same time, they leave some stones untouched. Another issue with vulnerability scanners is that they focus on the bad. It is their primary duty to find weaknesses. The more they find, the better the tool is supposed to be. Well, unless you get into false positives.

Negativity In Security Does Not Work

Sometimes we can’t avoid expressing a key performance indicator (KPI) as a negative thing, like the number of issues found. But when the number is always bigger than zero, does it really make sense to report on that? This negative expression of security indicators is only depressing people. It is like saying there is no need to put time into it, as it is always in a bad shape. Consider that if you have a few pounds too much and decide to stop exercising at all (because you think it is of no use anyways). As we know, the problem only becomes worse.

Vulnerability scanners have the negative association in them, as their name already focuses on scanning for the “weak”. It would have been so much better as they were named “Security defense checkers”. But well, the term is around in the field, and so many people are now programmed to hunt vulnerability. Yet, we consider it the way to depression. That is a shame, as the tools definitely can have a good impact on the general security posture of a company. So time to move away from vulnerability scanners only, and look at the positive side!

The Alternative To Vulnerability Management?

If you want to leverage vulnerability scanning the best possible extent, you have to combine it. This could be with good reporting, sharing insights, and reward people for doing good things. Don’t focus just on the “bad”, like vulnerability scanners. Instead, set targets and next steps, reward those who comply with them.

An interesting alternative to vulnerability management is making it part of a bigger strategy. Combine it into a power tool. What this power tool looks like:

• Vulnerability scanner -> Determine any weaknesses, and detect outdated packages
• Security auditing tool -> Discover health status of environment, show compliance, define next step of improvement
• Configuration automation tool -> Implement improvement: push configurations and remediations

So instead of one single tool, go for a combination of tools, each with their own strengths. While you could still count the number of high-risk vulnerabilities, you have more items to score on. Like the percentage of systems covered by the configuration management system. Or the number of systems scanned in the last week by both the vulnerability scanner and auditing tool.

Let’s start with the last category first.

Configuration Management On Linux Systems

Regular readers know we focus on Linux and UNIX systems. What makes these platforms great is the flexibility you have to create something that works for your organization. This starts at the stage of doing configuration management. You can do it manually, or create some nifty shell scripts, or decide to go for a configuration management tool. Over the last years, it is Ansible that got a great share in this market, although Puppet still seems to be leading it. But of course don’t forget the others, like CFEngine and Chef.

In this day and age, automation is key. Where possible you should try to replace yourself with a script. Instead of doing repeating tasks over and over, try to capture them into an easier way of working. Then you can move up to the role at puppeteer, with control over how other pieces in the environment will move and act. Use technologies like PXE boot to deploy systems, Kickstart to install the operating system, and a configuration management tool to maintain the configuration. You should be focusing on the exceptions and the outliers.

Linux Doctor: Security Auditing

Even if you think you have everything under control, things might be wrong. The problem is that we don’t know with a proper health check. This is similar to bringing your car for an inspection, or go to the doctor yourself. Linux can also use a good checkup now and then. This helps in detecting the exceptions much quicker. While many things can be automatically checked with your monitoring tooling, some items never might appear on your radar. That is, until you run an in-depth scan. Do you ever check the integrity of your password file, or any users that no longer should be there?

Want to get convinced there is so much to improve? Give open source tool Lynis a spin.

Automatic Software Patching

If you have your configuration management, system monitoring, and system auditing in place, it is time to ensure you have a good software patch routine. This means you should also have a policy, to make decisions on how you deal with the ongoing stream of updates. A policy should answer things like how often will you be doing software updating, and how to deal with high-risk security updates. But even if you have a great policy, it is the routine of patching that makes the difference.

Still too many system administrators are afraid of rolling out patches. For a good reason, as they get blamed when something breaks. For that reason, you should get systems in place to test updates. One thing you could do is set up a small set of virtual machines which gets patched 24/7. When there is a new patch, those systems will receive them first. They mimic production systems, by running a combination of different roles.

For example, these patch testing systems could be hosting both the mail server configuration, a web server, together with a database engine. If a patch is released for any of these components (like nginx, Postfix, or MySQL), it will be applied there first. This will also apply for generic updates, like common components as OpenSSL, or glibc. And the system will also need to be rebooted automatically, to ensure kernel updates are tested. Extensive system monitoring will then check to see if the system stays up, even after automatic patch and rebooting.

The takeaway is this: automate everything, including those things you rather do manually. Because in the end, you could still do some things manually. And you can, because you saved yourself a lot of time, so you can put in total focus on the things that are rare, or special. This might be your business critical service, or that outdated system which needs to be phased out.

Automation is also a great indicator for your security metrics. Instead of focusing on the negative number of vulnerabilities, share what percentage of systems is automatically managed, monitored, or checked.

Vulnerability Scanning

This article had the focus on vulnerability scanning. Why the other parts? They can reduce the number of vulnerabilities greatly. Again, automation is key, and the only way to solve newly discovered issues quickly. For that same reason, it is a safe bet to focus on continuous auditing and security monitoring. You can’t prevent everything, but you can improve your detection rates. Vulnerability scanning might help here, yet audit tools and event logging will have a greater impact in the end.

Conclusion: stop getting depressed with the negative aspects of security, and vulnerability scanners in particular. Focus on improvement, automation, and reporting. Show the good work we do, leave the bad for the others.