Linux History: How Dot Files Became Hidden Files

The history of hidden files

Ever wondered why there are files on your Linux system, starting with a dot? The short answer: they are shortcuts. The story begins many years ago when the first file systems were created on UNIX. To allow easy navigation, a single file with a dot (.) was added to each directory. Secondly, a double dot file (..) was added to easily move up in the directory structure. As these files had no real data in them, a quick hack was added to the ls binary.

The “hack”

The change made to the ls binary involved checking for the first character. If that was a dot, it should be ignored. And it worked great!

Something which was not anticipated is what we now know as a hidden file. If you create a file starting with a dot, it will become a file on disk, but which is not displayed by default. This behavior is the result of the earlier hack applied to the ls binary.

Are Hidden Files Bad?

So with all these hidden files, we might wonder if their purpose is good or bad. After all, it can declutter your home directory, by showing less files. While this is true, the opposite is true as well. Most utilities create hidden files, and might not really clean things up when unneeded. With utilities scanning your home directory, things slowly will cost more and more time.

More Details

In a Google+ post by Rob Pike, A lesson in shortcuts, the more detailed rationale behind the dot files.

Long ago, as the design of the Unix file system was being worked out, the entries . and .. appeared, to make navigation easier. I’m not sure but I believe .. went in during the Version 2 rewrite, when the file system became hierarchical (it had a very different structure early on). When one typed ls, however, these files appeared, so either Ken or Dennis added a simple test to the program. It was in assembler then, but the code in question was equivalent to something like this:

if (name[0] == ‘.’) continue;

This statement was a little shorter than what it should have been, which is if (strcmp(name, “.”) == 0 || strcmp(name, “..”) == 0) continue;

But hey, it was easy. Two things resulted. First, a bad precedent was set. A lot of other lazy programmers introduced bugs by making the same simplification. Actual files beginning with periods are often skipped when they should be counted. Second, and much worse, the idea of a “hidden” or “dot” file was created. As a consequence, more lazy programmers started dropping files into everyone’s home directory. I don’t have all that much stuff installed on the machine I’m using to type this, but my home directory has about a hundred dot files and I don’t even know what most of them are or whether they’re still needed. Every file name evaluation that goes through my home directory is slowed down by this accumulated sludge.
I’m pretty sure the concept of a hidden file was an unintended consequence. It was certainly a mistake.

So what do you think? Are dot files evil or good?