Making scripts (more) secure and safe
When you create a shell script, many things can go wrong. With a few basics you can catch errors easier and at the same time make your scripts (more) failsafe. The beauty of shell scripting is that with just a few steps this can be achieved!
Empty variables: nounset (-u)
A very typical issue in shell scripts is an incorrect or empty variable. Usually this happens due to a typo, but sometimes also assignments can be wrong. We can let our shell script stop immediately if this happens. This is a great safety measure during development, but also when a shell script is tested and working its duties in production.
Let’s have a look what happens when you use an empty variable normally:
#!/bin/sh
echo "${oops}"
The output is an empty line. Very innocent when it is just some text (not) being displayed. But this can change quickly when the variable is input for a task that uses rm.
To enable this measure, put the following line at the start of your script.
set -o nounset
So usually your script would then look like this:
#!/bin/sh
# safeguards
set -o nounset
echo "${oops}"
When we now run the script, it will give a different output.
./empty-variable.sh: 6: oops: parameter not set
So on line 6 we have a parameter not set, which is named oops. In other words, we used a parameter that had not a value assignment.
We can also define this nounset option by using the shorter set -u. While this is often used in scripts, it may not always be clear what it does. The fully written options are somewhat more self-documenting, especially for others reading your scripts.
Exit upon errors in last command in pipe: pipefail
During the execution of your shell script, a combination of commands will be used. Sometimes single commands, sometimes piped after each other. To quit immediately if something goes wrong in the last part of the pipe, we can use the pipefail option. Let’s have a look at a simple script.
#!/usr/bin/env bash
set -o nounset
set -o pipefail
echo "test 1" | wc -l
echo "test 2" | wcc -l
echo "test 3" | grep test
The pipefail option is usually not available when using /bin/sh as shell
So like before we use the nounset to counter for empty variables. Additionally we also set the pipefail to monitor for the exit state. But what if we didn’t use it? The output would look like this:
1
./error-in-pipe.sh: 7: wcc: not found
test 3
Only the first and third echo give the expected result. Again, an innocent echo of text won’t make a huge difference. But one wrong replacement or failed action, and data or files could be corrupted, moved, or worse.
If we have the pipefail option enabled, then suddenly we see just two lines of output:
1
./error-in-pipe.sh: 7: wcc: not found
So the first echo goes well, but the second does not. The third won’t be performed, as we stop execution when the first (unexpected) issue showed up.
Exit upon errors: errexit (-e)
The next step is catch exit states that are not zero. For example when you try to do something with a file that does not exist.
#!/bin/sh
cat /tmp/randomfilethatdoesnotexist
echo "Happy end!"
When running the script, it will show the following output:
cat: /tmp/randomfilethatdoesnotexist: No such file or directory
Happy end!
As you can see, this will return an error followed by ‘Happy end!’. Typically, we don’t errors and it may have been better to stop upon a serious error to prevent issues.
So let’s add some safeguards and insert the nounset and errexit the option to the script:
#!/bin/sh
set -o errexit
set -o nounset
cat /tmp/randomfilethatdoesnotexist
echo "Happy end!"
The script will now only show the failure and exit immediately after it:
cat: /tmp/randomfilethatdoesnotexist: No such file or directory
Note: the nounset obviously has no influence in this example. It’s just here to show that you can combine them.
Want to learn more about the options that set can provide? Have a look at the GNU Set Builtin page.
Happy scripting!
