I commented on Lobsters that
/tmpis usually a bad idea, which caused some surprise. I suppose/tmpsecurity bugs were common in the 1990s when I was learning Unix, but they are pretty rare now so I can see why less grizzled hackers might not be familiar with the problems.I guess that’s some kind of success, but sadly the fixes have left behind a lot of scar tissue because they didn’t address the underlying problem:
↫ Tony Finch/tmpshould not exist.
Not only is this an excellent, cohesive, and convincing argument against the existence of /tmp, it also contains some nice historical context as to why things are the way they are. Even without the arguments against /tmp, though, it just seems entirely more logical, cleaner, and sensible to have /tmp directories per user in per user locations. While I never would’ve been able to so eloquently explain the problem as Finch does, it just feels wrong to have every user resort to the exact same directory for temporary files, like a complex confluence of bad decisions you just know is going to cause problems, even if you don’t quite understand the intricate interplay.
History is a thing.
I could make the same argument for why Windows should not exist.
Yeah, but IMO the “real” tmp directory should be $HOME/tmp and be dynamically linked to /tmp per each user (so root’s /tmp would really be /root/tmp, Jenny’s tmp would be /home/jenny/tmp and so on) to account for software that have /tmp hardcoded.
Really I’m sorta apathetic on the whole thing – /tmp has it’s uses, but is it essential? Probably not – With various asterisks for the details, I could achieve the same goals in several different way, and probably would in a modern software system. Per user /tmp? Um, isn’t that compounding the problem? Yes you take away the part of the problem that falls under the “Shared space” category, and pile on some extra system administration and configuration in its’ place, when really you should be solving this problem using other tools. It’s not the 1970’s anymore, and we have new things – go find them. Do I care enough to kill /tmp? Not really. Honestly, “modern” (read cloud native, not desktop) software shouldn’t be writing to the filesystem at all! Again, mind those asterisks for exceptions, as there are a million of them.
There are plenty of old conventions that I find hacky: sticky bits are definitely one of them. Another thing that’s always bugged me is having application software scattered around the file system rather than keeping package files together.
https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard
IMHO this layout is cumbersome and counterproductive to the goal of organization. Installing really should be as easy as extracting an archive. Rather than having to lump unrelated files together in global directories like /lib /usr/lib /bin /sbin /usr/bin /usr/sbin /opt etc, you could have those files be organized within the package’s directory. Unlike today with files scattered every which way, package resources would stay in the package directory making the relationship between file and package far more obvious. Uninstalling the package could be as intuitive as deleting the directory (like it was in DOS).
Program files should be placed where they belong instead of having to be placed in designated system directories where they can collide and conflict, Also, operating systems facilities for locating these files should be more capable than PATH and/or other environment variables. Instead, files having certain attributes could be indexed by the file system. File system primatives should be robust & compliant with ACID database principals (Atomicity, Consistency, Isolation, Durability). Not only would this clean up the file system, but the binary loader, rather than having to search dozens of directories for each shared object file, could perform much faster database lookups.
This is something I’ve thought about over many years and I even started to implement it in my own linux distro but I ended up giving up because the maintenance burden to change all the software to use new conventions was overwhelming. It’s like trying to move the ocean with a bucket. Even though I felt the new conventions were much better than the old ones, it’s far easier to “stay with the pack”.
Anyway thanks for the article, good food for thought 🙂
It feels like I’ve been complaining about the Linux FHS forever. Sadly, I think most people agree holding their nose and staying the course is better than the amount of work to fix all the design stupidity. And then of course you have those … people … who love to point out that you can always `make your own distro however you’d like`, as if merely wanting something better is an outrageous ask. I wonder if the brilliant minds who came up with that FHS keep their plates in an upstairs bedroom, bowls in the garage, cups in a downstairs closet, and silverware outside in the a shed,
Alfman,
The “cruft” over time happened due to natural reasons, including optimizing storage space.
The /use was supposed to hold sharable read only files, but became the de facto location of applications. And since many now use shared libraries, it made sense to keep them there.
However…
Modern file systems have copy on write and de-duplication facilities. We no longer need to have a central location of libraries, but can host copies with each application “bundle” (or whatever we call them).
Alternatively we can still have a shared location for libraries, but symlink them to app dirs, but that would require tracking back links.
In any case, I agree, the standard Linux file organization needs a makeover.
Unfortunately inertia will make it very difficult.
sukru,
There are a lot of technical solutions we could come up with to keep packages together in the same logical space. Symlinks could work like you say, and it would keep modern shells and linux loaders working as they do today. However what I envisioned is how *NIX could have evolved if file systems were always based on packages from the start. The need for /bin and simply wouldn’t have existed at all. Instead shells and library loaders searching PATH variables inside of /bin directories, we could have more evolved filesystem APIs with faster, more robust, and maybe even more secure indexing capabilities.
Yeah, a naive me attempted to do this myself in the 2000s. It worked to the extent that I could customize every package for my distro. but it was too much work. Other distros like Gobo are also trying to fix layouts. Ultimately it’s the giant distros and their infinite forks that set the momentum overall. Without their involvement things won’t be fixed. And even if they were willing to do it, the compatibility issues would leave us with on foot in the past anyways.
Alfman,
You touched in important point:
Even if none of the other considerations were in place, searching a few directories vs hundreds would make a significant difference for the shell.
Yes modern systems can cache these. But for early UNIX, I think the choice of having a two or three directories to search (/bin, /usr/bin, /usr/local/bin) made a lot of sense.
sukru,
This is exactly why I keep bringing up indexes. In the absence of directories that must contain every program, it would have been natural for operating systems to evolve index capabilities to quickly find programs. Not only would this facilitate shells & DLLs, but OS indexes could have been used to rapidly locate all sorts of information and metadata anywhere in the file system without the need to open/read every file. I can think of all sorts of innovative applications.
I really suggest reading stuff like Brian Kernighan’s memoirs…
Yes, it sucks. But back in the day, it was very common that you had, for example, local configuration and binaries stored remotely. For example, you could have a netbooted device, with main configuration (/etc) living in the central server where your workstation booted from, and then you would have /usr/local/etc for your specific configuration living in your device, and /usr/local/bin for your own applications.
This was designed back when machines sometimes had no or minimum local storage and your main server could have multiple disks of a few megabytes each, so managing partitions and mount points was very important.
It is far from being stupid, even though it sucks from a modern perspective.
Shiunbird,
I don’t think it was ever technically necessary to do that. The problem is that unix originally assumed it would be best to keep all binaries in the same /bin directory.. In the very early days there weren’t that many applications, so package organization wasn’t seen to be a big problem, but it set *nix down a path of cramming more and more files to fit inside a single directory instead of keeping them organized as packages. And then when they could no longer fit more files into the directories due to physical disk constraints, they created more bin directories on different disks. The disk capacity problem was understandable but I still think it was a bad pattern to use in the first place. It would have been far better for unix operating systems to evolve around packages that remained contained to one logical path and could run anywhere regardless of the physical disks they were located on.
Those early decisions went on to shape unix forever, but there were contemporary examples that did not break package layouts. For example mutlidisk setups were the norm under DOS too. DOS did not rely on this kind of layout having application files splayed across disparate directories. Obviously other aspects of DOS weren’t very evolved, but as far as file and package organization goes, I think we went down the wrong path.
@Alfman
Fair enough. However, DOS is perhaps the only system I know of that kept all system binaries under C:\DOS and everything else would be in their own folders.
(can’t recall now if command.com, config.sys and autoexec.bat were in C:\ root)
Windows went full mess mode as well stuff in \Windows, \Windows\System32 and the registry, and then applications that keep some files in their folders, others in AppData (like some sort of auto updater)
I agree with you that I much prefer to have the binary in a folder, configurati0n in a local ini/conf file, but there has been a reason why most systems go “full chaos” for the eyes of the ones that were not part of the decision-making process.
Shiunbird,
I’d like for all operating systems to do away with “cram directories” by sticking more vigorously to the concept of packages. Windows uses a mixed approach. “Program Files” is a good way to keep applications organized. but I’d like to see something similar for “Shared Libraries” rather than stuffing everything in “c:\windows\system32”, that thing is a mess. Microsoft themselves haven’t been setting the best example….far too much of their stuff gets crammed into global directories. It doesn’t have to be this way.
MS made some awful unforced mistakes when it came to 64bit windows. They created arbitrary, unnecessary, unhelpful, and unfriendly chasms between “Program Files” and “Program Files(x86)”, “System32” “SysWOW64”, registry branches, etc. This added complexity and confusion and it was completely unnecessary. No other operating system making the 64bit jump needed to rely on hidden kernel translations to fix up the paths that MS broke this way.
>/tmp security bugs were common in the 1990s when I was learning Unix
These were easily solved with mounting /tmp with noexec, nosuid, nodev. Which you will now find are in all security recommendations and part of standards like CIS. Good systems admins make it so you can’t use /tmp to run scripts etc which is how it was normally used by the old school worms.
I feel this whole discussion is obsolete, given containers(docker etc) and vms which long ago resulted in large monolith systems being broken up into many smaller parts. The whole importance of local escalations are not what they used to be as you have to jump through multiple remote systems that might not even exist long enough for it to matter.
So /tmp is useful as it is IMHO. It does not need to be complicated as it does not mean what it used to mean,
But current /tmp also has some advantages:
If all programs create their temporary storage there, you can quickly delete all the useless stuff if system is low on disk space. And by concept, everything in /tmp can be deleted without compromising the system.
Otherwise you have to check in 1000 places. For example, in SUSE Linux, cups never deletes print jobs. So unless you go and delete /var/spool/cups, you keep every print job you ever had.
Other advantage is that /tmp could be on a different partition with a faster disk. So a system with a faster /tmp directory can work faster, if they run programs, which do lots of caching, saving undo operations, and such.
https://images7.memedroid.com/images/UPLOADED555/66f2c9e913373.webp
This is why we all need selinux.