Determine Processes Which Need a Restart with checkrestart/needrestart
Determine which processes need a restart after software patching
Proper software patch management helps reducing weaknesses on your systems. But even if you patched an outdated system, old processes and libraries can continue to run in memory. For example when a library is updated, an active program might still use the old version. To really finish the process of software patching, we have to do more. This includes preparation, performing the update and finally check if we need a restart of software components. In this post we have a look at several options, to properly execute this last part of the process. An introduction into the world of tools like checkrestart and needrestart.
Checkrestart
The first utility to help with the job of finding processes using old files, is checkrestart. It is part of the debian-goodies package and only available for Debian based systems. It uses LSOF (List Open Files) to determine open files and what processes using such resource.
Installation
apt-get install debian-goodies
Requirements
- Debian (or clone)
- Python
- LSOF
- root permissions
Usage
Running the checkrestart command will give an overview of what it discovered and what processes need a restart. It shows the processes using old files and determines what init scripts are related to these processes. Of course, those which it can find. For the others it will display the related processes, so you can manually take action.
Example output
michael-nb ~ # checkrestart lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1000/gvfs Output information may be incomplete. Found 68 processes using old versions of upgraded files (48 distinct programs) (40 distinct packages) Of these, 7 seem to contain init scripts which can be used to restart them: The following packages seem to have init scripts that could be used to restart them: sudo: 3908 /usr/bin/sudo cups-browsed: 1457 /usr/sbin/cups-browsed samba: 965 /usr/sbin/smbd 700 /usr/sbin/smbd 2371 /usr/sbin/nmbd mdm: 1491 /usr/sbin/mdm pulseaudio: 3039 /usr/bin/pulseaudio cups-daemon: 3568 /usr/sbin/cupsd dbus: 2822 /bin/dbus-daemon 3477 /bin/dbus-daemon 2385 /bin/dbus-daemon 2836 /bin/dbus-daemon 507 /bin/dbus-daemon These are the init scripts: service sudo restart service cups-browsed restart service samba-ad-dc restart service smbd restart service samba restart service nmbd restart service mdm restart service pulseaudio restart service cups restart service dbus restart These processes do not seem to have an associated init script to restart them: udisks2: 3162 /usr/lib/udisks2/udisksd policykit-1: 918 /usr/lib/policykit-1/polkitd modemmanager: 783 /usr/sbin/ModemManager blueman: 3255 /usr/bin/blueman-applet network-manager: 912 /usr/sbin/NetworkManager mate-polkit:amd64: 3257 /usr/lib/x86_64-linux-gnu/polkit-mate-authentication-agent-1 xserver-xorg-core: 1498 /usr/bin/Xorg gvfs-daemons: 3143 /usr/lib/gvfs/gvfs-udisks2-volume-monitor 3554 /usr/lib/gvfs/gvfsd-metadata 3321 /usr/lib/gvfs/gvfsd-trash mate-keyring: 2996 /usr/bin/mate-keyring-daemon caja: 3236 /usr/bin/caja system-tools-backends: 3487 /usr/sbin/system-tools-backends gvfs-backends: 3214 /usr/lib/gvfs/gvfs-mtp-volume-monitor 3203 /usr/lib/gvfs/gvfs-afc-volume-monitor 3208 /usr/lib/gvfs/gvfs-gphoto2-volume-monitor marco: 3015 /usr/bin/marco upower: 3272 /usr/lib/upower/upowerd mintupdate: 3821 /usr/lib/linuxmint/mintUpdate/mintUpdate.py mate-screensaver: 3258 /usr/bin/mate-screensaver at-spi2-core: 3472 /usr/lib/at-spi2-core/at-spi-bus-launcher dnsmasq-base: 2142 /usr/sbin/dnsmasq 3543 /usr/sbin/dnsmasq mate-bluetooth: 3261 /usr/bin/mate-bluetooth-applet consolekit: 2476 /usr/sbin/console-kit-daemon gvfs-fuse: 3023 /usr/lib/gvfs/gvfsd-fuse google-chrome-stable: 3646 /opt/google/chrome/chrome 3859 /opt/google/chrome/chrome 3642 /opt/google/chrome/chrome 3683 /opt/google/chrome/chrome 3675 /opt/google/chrome/chrome 3666 /opt/google/chrome/chrome 3612 /opt/google/chrome/chrome 3691 /opt/google/chrome/chrome 5706 /opt/google/chrome/chrome 3694 /opt/google/chrome/chrome 3601 /opt/google/chrome/chrome 3698 /opt/google/chrome/chrome 3621 /opt/google/chrome/chrome 3708 /opt/google/chrome/chrome 3616 /opt/google/chrome/nacl_helper mate-terminal: 3891 /usr/bin/mate-terminal mate-settings-daemon-pulse: 2987 /usr/bin/mate-settings-daemon system-config-printer-gnome: 3252 /usr/share/system-config-printer/applet.py mate-session-manager: 2547 /usr/bin/mate-session isc-dhcp-client: 2024 /sbin/dhclient mintmenu: 3049 /usr/lib/linuxmint/mintMenu/mintMenu.py mate-media-pulse: 3259 /usr/bin/mate-volume-control-applet network-manager-gnome: 3248 /usr/bin/nm-applet 3263 /usr/bin/nm-applet mate-panel: 3055 /usr/lib/mate-panel/clock-applet 3032 /usr/bin/mate-panel 3051 /usr/lib/mate-panel/wnck-applet login: 3909 /bin/su mate-power-manager: 3244 /usr/bin/mate-power-manager
From this output we can see what processes can be restarted, along the related init script. Additionally it shows what processes need also restarting, but lack the related script.
Checkrestart status
The utility does what it should do. However, it is limited to Debian based systems and the code contains a lot of “to do” items. If you are evaluating software components in this article, you definitely want to read more about Needrestart.
Needrestart
Another utility is needrestart. Like checkrestart it determins what processes need a restart, after running a software upgrade. It is part of the package with the same name and written in Perl. It seems to be well-maintained and supports newer technologies like containers (LXC, Docker). The tool hooks into the update process, so for example it can restart services after running a “dpkg upgrade”. It is also possible to run it manually, with specifically the mode to list processes only, provide the option to do restarting (interactively) per process, or do them all automatically.
Package managers
Need support a few package managers out of the box:
- DPKG
- Pacman
- RPM
Needrestart restart uses features of the package manager to determine which related package, or daemon needs a restart. It does so by looking for the related startup script. In the case of systems using DPKG, it actually uses some intelligence from the previously mentioned checkrestart utility. For RPM it leverages the rpmquery utility, for pacman the pacman utility itself.
Installation
Debian / Ubuntu
apt-get install needrestart
Fedora
For our Fedora system we used the following steps to get the tool working. Instead of using Git, you might use a custom package to simplify the handling of dependencies. As we simply use it for testing on our Fedora 21 test server, we install the dependencies and run it from the root home directory.
# cd /root # yum -y -d1 install git perl-Module-ScanDeps perl-Proc-ProcessTable perl-Sort-Naturally perl-Term-ProgressBar-Simple perl-Module-Find.noarch perl-ExtUtils-MakeMaker.noarch # git clone https://github.com/liske/needrestart # cp /root/needrestart/needrestart.conf /etc/needrestart/needrestart.conf # mkdir /etc/needrestart/hook.d # cp /root/needrestart/ex/hooks/* /etc/needrestart/hook.d # perl -I /root/needrestart/perl/lib ./needrestart -r l
Configuration of Needrestart
By default, not much configuration is needed for this utility. It works great out of the box. When adjustments are needed for the behavior of the tool, this can be done via the configuration file /etc/needrestart/needrestart.conf. Some of options that are found in the configuration file, can also be adjusted via the command line (e.g. type of operation).
So what is there to configure? The default restarting mode, what processes should be ignored etc. This way you can change the utility to do its job, while avoiding restarting unwanted parts of the system.
Needrestart in action
So after running security updates, we will run needrestart. In this example, we use it on a Fedora system. Just after applying hundreds of software patches, we run the tool in list mode (-r l):
Needrestart found several processes which needed a restart. The tool even checks if the latest installed kernel is running and discovered an outdated kernel being active. In this case a full system restart would even be better.
Which one to use?
After reviewing them both, go for the needrestart utility. It works on multiple Linux based systems, is well-maintained, has support for newer technologies and does it job very well. As always, we encourage testing first on non-production systems.
Found even a better tool to handle this job? Let us know!
6:00 PM
Hi Micheal, have you run into an issue where after some processes are restarted ssh access hangs for a minute before logging you in?
6:17 PM
Hi Jim, that might be caused by (failing) DNS lookups.
1:55 AM
Ok thanks, I’ll look into it.
12:52 AM
Were you able to find the reason?
10:07 PM
Im sorry I forgot check back with you sooner.
I ended up rebooting the server and the issue cleared up instantly.
I’ve been using need restart since then I and haven’t noticed any issues when processes needed to be restarted.
One thing I did learn was when I’m using screen for my Minecraft server as a different user (no root privileges) and there’s an update that needs to be restarted, I have to log into that user and type screen -r to cancel screen session and then screen back in. That clears up the restart notice from needrestart.