Performance
Troubleshooting CPU usage
Articles and information about troubleshooting system performance issues with focus on CPU usage.
Summary
Got a busy system that comes to a halt due it being too busy? In this article we look at troubleshooting issues related to CPU usage.
Monitoring CPU usage
The tool top might be the most familiar tool to monitor CPU or memory usage. A good alternative is the pidstat tool. It can be using an interval and easily show active processes, followed by a summary.
# pidstat 3
Linux 6.5.0-28-generic (workstation) 20-05-24 _x86_64_ (8 CPU)
13:45:47 UID PID %usr %system %guest %wait %CPU CPU Command
13:45:50 0 638 0,00 0,66 0,00 0,00 0,66 3 irq/204-nvidia
13:45:50 1000 1744 0,33 1,00 0,00 0,00 1,33 6 pulseaudio
13:45:50 1000 2050 1,66 0,33 0,00 0,00 1,99 2 gnome-shell
13:45:50 1000 3767 1,99 1,33 0,00 0,00 3,32 3 firefox
13:45:50 1000 3985 1,00 0,66 0,00 0,00 1,66 7 Isolated Web Co
13:45:50 1000 4277 0,33 0,00 0,00 0,00 0,33 5 WebExtensions
13:45:50 1000 25736 0,00 0,33 0,00 0,00 0,33 5 Isolated Web Co
13:45:50 1000 560859 0,33 0,00 0,00 0,00 0,33 5 Isolated Web Co
13:45:50 1000 657165 0,33 0,00 0,00 0,00 0,33 3 Isolated Web Co
13:45:50 1000 858923 0,33 0,00 0,00 0,00 0,33 5 Isolated Web Co
13:45:50 1000 1235407 0,33 0,00 0,00 0,00 0,33 1 Isolated Web Co
13:45:50 0 1284255 0,00 0,33 0,00 0,00 0,33 7 kworker/7:1-events
13:45:50 0 1284904 0,00 0,33 0,00 0,00 0,33 0 kworker/0:2-pm
13:45:50 0 1285798 0,00 0,33 0,00 0,00 0,33 3 kworker/3:0-events
13:45:50 1000 1286455 1,00 0,00 0,00 0,00 1,00 0 Isolated Web Co
13:45:50 0 1287603 0,00 0,33 0,00 0,00 0,33 0 kworker/0:0-events
13:45:50 UID PID %usr %system %guest %wait %CPU CPU Command
13:45:53 0 638 0,00 0,67 0,00 0,00 0,67 3 irq/204-nvidia
13:45:53 1000 1744 0,00 1,00 0,00 0,00 1,00 6 pulseaudio
13:45:53 1000 2050 0,33 0,00 0,00 0,00 0,33 2 gnome-shell
13:45:53 1000 3767 7,67 1,67 0,00 0,00 9,33 3 firefox
13:45:53 1000 3985 0,33 0,00 0,00 0,00 0,33 5 Isolated Web Co
13:45:53 1000 4277 0,67 0,00 0,00 0,00 0,67 7 WebExtensions
13:45:53 1000 18502 0,33 0,00 0,00 0,00 0,33 5 virt-manager
13:45:53 1000 560859 1,00 0,00 0,00 0,00 1,00 0 Isolated Web Co
13:45:53 1000 656930 0,33 0,00 0,00 0,00 0,33 5 Isolated Web Co
13:45:53 1000 1235407 0,33 0,00 0,00 0,00 0,33 6 Isolated Web Co
13:45:53 1000 1286455 0,67 0,67 0,00 0,00 1,33 7 Isolated Web Co
13:45:53 1000 1287953 0,00 0,33 0,00 0,00 0,33 1 pidstat
^C
Average: UID PID %usr %system %guest %wait %CPU CPU Command
Average: 0 638 0,00 0,67 0,00 0,00 0,67 - irq/204-nvidia
Average: 0 945 0,00 0,11 0,00 0,00 0,11 - libvirtd
Average: 1000 1744 0,11 1,00 0,00 0,00 1,11 - pulseaudio
Average: 1000 2050 0,67 0,11 0,00 0,00 0,78 - gnome-shell
Average: 1000 3767 5,77 1,44 0,00 0,00 7,21 - firefox
Average: 1000 3985 0,55 0,22 0,00 0,00 0,78 - Isolated Web Co
Average: 1000 3989 0,00 0,11 0,00 0,00 0,11 - Isolated Web Co
Average: 1000 4277 0,55 0,00 0,00 0,00 0,55 - WebExtensions
Average: 1000 18502 0,11 0,00 0,00 0,00 0,11 - virt-manager
Average: 1000 25736 0,00 0,11 0,00 0,00 0,11 - Isolated Web Co
Average: 1000 492449 0,11 0,00 0,00 0,00 0,11 - Isolated Web Co
Average: 1000 560859 0,55 0,00 0,00 0,00 0,55 - Isolated Web Co
Average: 1000 656930 0,11 0,00 0,00 0,00 0,11 - Isolated Web Co
Average: 1000 657165 0,11 0,00 0,00 0,00 0,11 - Isolated Web Co
Average: 1000 858923 0,11 0,00 0,00 0,00 0,11 - Isolated Web Co
Average: 1000 1202040 0,11 0,00 0,00 0,00 0,11 - Isolated Web Co
Average: 1000 1235407 0,33 0,00 0,00 0,00 0,33 - Isolated Web Co
Average: 0 1284255 0,00 0,11 0,00 0,00 0,11 - kworker/7:1-events
Average: 0 1284904 0,00 0,11 0,00 0,00 0,11 - kworker/0:2-pm
Average: 0 1285798 0,00 0,11 0,00 0,00 0,11 - kworker/3:0-events
Average: 1000 1286455 0,78 0,55 0,00 0,00 1,33 - Isolated Web Co
Average: 0 1287603 0,00 0,11 0,00 0,00 0,11 - kworker/0:0-events
Average: 1000 1287953 0,00 0,22 0,00 0,00 0,22 - pidstat
Filter by process
To zoom in on a particular process or task, the option -C or -G can be used. It filters on the provided string and looks if that is part of the command name. When needed, a regular expression can be used.
Network
Articles and information about troubleshooting network performance issues and monitoring network statistics
Summary
Network connectivity starts at a device that links the system to a network, and for Linux systems that is no different. Depending on the physical layer, such as Ethernet of Wi-Fi, the transactions between be better or worse. Besides physical limitations, there is congestion and packet loss that may introduce issues. In this section we look at troubleshooting network performance issues, tooling, and examples.
Interfaces
The first step is to learn what links we have towards the network.
Memory
Articles and information about how memory, such as RAM, is being used on Linux systems. Great for system administration and troubleshooting purposes.
Summary
System performance
Articles and tools to troubleshoot Linux system performance issues.
Summary
Swap memory information
Articles and tools to troubleshoot Linux system performance issues with focus on swap memory and its usage.
Summary
Physical RAM is used to store information. Linux divides this RAM into smaller chunks, named memory pages. When there is no more normal memory available, the Linux kernel might need to temporarily store information aside. This is called paging or swap space.
During the process of paging, memory pages will be moved from the RAM to the disk. This way memory is freed up for active processes, while older information is temporarily stored on the disk. A disk is much slower than RAM, so typically you want to avoid paging as much as possible.
Test web server caching with curl
Want to test your web server and see if static files are properly cached? Curl can help and with some scripting even automate the task for you.
Summary
Learn how to use curl to test if your web server is properly caching static files
Understanding what runs on your Linux system (and why)
Linux systems have a lot of processes running by default. Let's dive into how programs are started and how you can see all details of each running process.
Summary
Introduction
Each Linux system has a bunch of processes running. Most of these processes might be familiar to you if you regularly use a command like ps or top to display them. Processes may look like just an item in a list. They are actually complicated pieces of code that are tamed by a memory manager. To truly understand how your system is running, knowledge of process (or memory) management is of great help. So let’s make a jump into the internals of Linux by learning the tools at our disposal.
Linux DNS Tuning for Performance and Resilience
Linux DNS configuration is usually done during the installation of the system. With proper configuration and tuning, you gain performance and stability.
Summary
DNS Configuration on Linux
We often don’t realize the importance of DNS, or name resolving in our infrastructure. The impact when things go (slightly) wrong is huge. Time to have a good look at improving our DNS configuration.
How DNS resolving works
When your Linux system needs to know the IP address of a particular host, it will use gethostbyname(3) function. This will use the nsswitch configuration stored in /etc/nsswitch.conf. For the related hosts line, it will determine how to do resolving.
Tuning auditd: high-performance Linux Auditing
To achieve better performance with a auditd configuration, it needs to be tuned. See performance boosters like events exclusion, rule ordering, and more.
Summary
The Linux Audit framework is a powerful tool to audit system events. From running executables up to system calls, everything can be logged. However, all this audit logging comes at the price of decreased system performance. Let’s have a look at how we can optimize our audit rules.
Performance tips
Good auditd performance will reduce stress on the Linux kernel and lower its impact. Before changing anything to your system, we suggest benchmarking your system performance before and after. This way you can see the benefits of your tuning efforts.
Optimize SSL/TLS for Maximum Security and Speed
Everyone loves secure websites, as long as they are quick. Let's configure our website for maximum security and performance, at the same time.
Summary
Recently we changed our corporate website into a “HTTPS only” version. Most of the content is not secret information, still we have some sensitive areas. The ordering section and downloads, and additional our portal. While some areas were already covered with a lock, we felt it was time to make the jump to cover it all.
Additionally, we believe that we doing everything we can on our website, practicing security hardening ourselves. So that includes buying a SSL certificate, configure our web servers and finally tune it. In this article we share what we learned while doing so.