The Non-Technical Changelog: Insights of 6 Months Development

Lessons learned between our last and current release

The Lynis project team is proud to announce a new release of our security auditing tool. With months of work and a variety of changes, we bumped up the version to a “zero release” (2.2.0). The technical changelog is included in the download. We consider it to be a stable release, yet ask all to test it first.

Being the original author of Lynis, there is an additional background behind a changelog, which might be even more interesting. With this post, I want to share some of the background going into open source development. We have both our challenges and victories. Let me share some of our insights, in the same “open” spirit we develop our software.

Achievement: 2nd Place at ToolsWatch

One of our biggest lessons to share directly, is that it doesn’t really matter how good software you create, or at what price. People simply expect that the quality is good, and the price is as low as possible. It is hard for most open source projects to be cheap and provide high-quality software without a budget. Instead of focusing on those aspects alone, get noticed and extend your community. We applied this principle, resulting in a second place at the yearly ToolsWatch vote for the best security tools. A nice achievement and another public reference that people trust Lynis. This trust will help the project for the next years, like sharing it on conferences, blogging about it, or sharing it with colleagues and friends.

Long (Overdue) Release

In the last years, we released Lynis very regularly. That is, varying from every few weeks, to a week of eight. This time, it is almost 6 months between the previous release and this one. We reflected on that, to better understand why it took much more time than usual. In essence, it comes down to priorities.

In 2013 we decided to create a commercial offering around Lynis, to ensure the open source tool received a continuous feed in resources (time, expertise, development, promotion). In its turn, the open source version is also the client component in Lynis Enterprise. A better Lynis client benefits both the community and our customers.

Open source should provide value, additional services should provide additional value

But customers don’t pay for something they can get for free. That is the reason why we invest a lot of our time into the development of Enterprise functionality. Happy customers enable us to continue the work on all components. In other words, we set our priorities straight, to ensure long-time success.

There are serious consequences of not releasing an open source project often. One is that people might think it is no longer developed or supported, and then abandon it. They might even speak bad about the project, stating it is useless. We have learned that lesson before, so we know how big the impact can be. It is one of the reasons why we made the jump to GitHub, to show the community things are being developed and continuously updated.

Release open source project often, for community display, damage control, and marketing purposes

Crazy as it might sound, but open source projects needs a similar marketing approach as commercial software. Even if you give away your software for free, its value is low if it is not being used. The more people using your project, the better it becomes over the years. More eyeballs, as they say in marketing.

Due to the long time between releases, we have collected a lot of changes. Several new tests have been added, and even more enhancements than before. To prevent the release coming too big without testing, we did intermediate releases on GitHub. This way the community still has the latest and greatest, with the possibility to test and improve.

Community

Speaking about community, we have the same challenge as many open source projects. It is hard to find long-time contributors. Most have busy jobs and can join in quickly for some improvements, before they are filled with other activities. This is a big lesson for us, and we are doing now the ground work to make it easier to do small commits. Those who do contribute for a longer time are rewarded by a special mentioning in our changelog.

Our first priority after this release is getting our documentation improved. Before we had a single very long document, with tried to capture all the important things. If you want people to use your open source project, give them the chance to quickly try it out.

If you have an open source project, your documentation should provide an outcome

We provide now our “Getting Started” document to target beginning users. Initial review of our analytics showed that it works. Enough reason to split up more documents and turn them into in expected outcomes. Before one clicks on a document, it should be clear what they can expect and what result they will achieve.

Simplicity Sells

During the development of this release we learned that a few things were still to complicated. Our software does not have to be installed, nor has dependencies. You would think people would be able to figure out quickly how to use it, right? We were wrong and decided to improve upon that as well.

We already used colored output in previous versions. It does not sound spectacular as a feature. Still, our guess is that not many shells scripts have it implemented properly. This release we applied coloring on more places, including the first screen people get when running the tool without parameters.

Small changes like this help simplifying the usage of the tool, especially if you never used it before. Another common issue is the overload of options used. In this release we stripped actually some of the parameters from the output, and put them only in the man page and online documentation.

Adding new features is great, removing clutter is even better

New users are easy confused. Some don’t speak the English language, others stare at all the available options. So our focus is now on reducing things on the screen. They can only confuse new users, or they are screen clutter for the typical user. We can’t track parameter usage, otherwise it would have been a great source to determine what other parameters should be left out. What we can do is better devide the use cases of our software. In other words, we are trying to learn better how people really use our software. Not just the environment they are running in, but the how, why, and when.

Users will do unintended things with your software, if you give them room for misinterpretation

We are now interested in how they run the tool, which version, and what they do with the results. We do ask people now, after seeing a recent GitHub request was about the usage of a particular parameter, then parsing the output of the screen. This was not intended, as we actually have a report file with the findings, so you don’t have to scrape screen output. It made us realize that users will do unintended things with your software, if you give them room for misinterpretation. Another lesson is that we should bring it in line. So if on screen something is flagged as a red item, it should be in the report as wlel.

Why Upgrade?

As this is the non-technical changelog, we won’t go into the technical changes. It would still be useful to determine the reasons to upgrade. One of the most interesting facts we learned during this development cycle is that people upgrade for different reasons. We already learned that some people actually refused to upgrade, simply because the packaged version in their Linux repository was outdated. So this is a clear reason why some do not upgrade. Now let’s dive into the reasons we heared why people do actually want to upgrade.

Some of our users wanted to use the latest code, to be sure they were not missing out on features and bug fixes. Running the latest version does not scare them, knowing they can review changes first before pushing them to all systems. Especially those who embedded the GitHub repo in their deployment belong to this category. Some were running the latest code simply because the different in version number.

Don’t confuse users with version numbering of your open source project

We had our previous release tagged 2.1.1, and after a batch of commits we updated the release number on GitHub, without making it into an official release. We used it to determine if people were up-to-date when providing bugs. While great for us, it was confusing for our users. After all, they found version 2.1.8 on GitHub, but still the “outdated” version 2.1.1 on our website. So don’t confuse users with version numbering of your open source project. It will only result in more support questions.

The Need for Packages

For a long time, we resisted the idea of creating Lynis packages. Yes, packages are a good thing and help with system management. At the same time, our tool didn’t need any installation and rolling your own package is not that hard. Last year we continuously heard that people want things packaged. Although we can’t offer it yet, we started with some initial work to get things in motion on this front as well.

More Non-Technical Changelogs?

If you enjoyed this article and found it useful, share it with your (online) friends. I personally appreciate a comment on how it may help your project, and count it as a +1 for doing it more often.

Thanks for being part of the open source community.

Michael

Running Linux, Mac OS X or some other UNIX-based system? Don’t guess your security levels, but check it easily yourself. Within 2 minutes you have your system audited with the free and open source tool Lynis. And if you like it, consider the upgrade to the Enterprise version.