Ever since I started using computers, I’ve been baffled by the relative clumsiness of installing applications. Whether we are talking the really old days (launching the Rambo game off a tape), the ’90s (running Keen or using installers in Windows 95), or the modern days (still those installers, but now also package management and self-contained applications); it’s all relatively cumbersome, and they all have their downsides. I decided to put my money where my mouth is, and come up with my idealistic, utopian method of installing, running, updating, and uninstalling applications. Introduction
Mac users are always quick to point out the benefits of their self-contained applications; one package to download, no installation procedures, easy to remove. While this seems ideal, there are many problems with the actual Mac OS X implementation of this idea. Applications in Mac OS X are generally not easy to remove at all, because they leave a trail of files around outside of
The Windows world is not much better off – in fact, it is probably worse. Not only does it have to deal with the same problems as OS X, it also has too deal with dreadful installers. Some of them are such usability disasters they make me want to curl up in foetal position and cry. And then, when I’m done crying, I can start all over again because the uninstallation procedure is just as dreadful.
This leaves us with the Linux world. They have the centralised, easy updating application – the update application in for instance Ubuntu is an excellent example of proper balance between providing enough technical information for experts, but still hiding all that fluff from normal users. However, Linux suffers from other problems. Dependency hell, while not nearly as huge a problem as it used to be, still exists to this day. Downloading a package outside of the repositories is a risky business, but it really shouldn’t be. You are completely dependent on your distributor updating his repositories and keeping them clean – nothing is as annoying as knowing there is a new version of Super Awesome Garden Designer Ultimate Edition, only to realise all the distributions except yours already packaged it.
Basically, the proposal you’re about to read will incorporate bits and pieces from all over the operating system world, but will also include some new ideas of my own. The goal of my proposal is to make the installation, updating, and uninstallation of applications as easy as possible, while at the same time providing features advanced users and developers might appreciate.
I started working on this proposal ages ago, and as I went along, I realised I needed to alter a lot of things about the operating system itself (most notably, the filesystem layout) before I could continue working on the proposal itself. Consequently, I have created a new filesystem layout, and most likely a new filesystem – as I am not sure to what extent existing filesystems can accommodate this plan. Consequently, this article starts out with detailing the privilege model required, followed by an explanation and breakdown of the filesystem layout I devised. It will then move on to the importance and benefits of using BFS-like attributes and queries for updating and maintaining applications, ending with a conclusion (I bet you didn’t see that coming).
Privileges
It might seem a little odd to start an article on a utopian method of managing applications with a section on privileges – and you’re right, it is. The reason I’m starting with it anyway is because in my opinion, the privileges a user has is vital in the process of managing applications – or programs, as I will call them from now on (don’t worry, this arbitrary decision will become clear). Don’t pay an awful lot of attention to the names of the directories yet, I will explain those later on.
By default, a user has either User rights, or System rights. When in normal, day-to-day operation, the user and his programs/processes utilise User rights. This means the user and his programs can only write to their home directory (
This is similar to other operating systems, and has the usual and understood benefits: processes and programs launched by a user can only damage his or her own files, and cannot ruin the entire system or other users’ files.
Which brings us to System rights. Users can elevate their rights either on a need, or a perceived-need basis, to System rights, which gives them the ability to change system settings or access content in otherwise restricted directories. With ‘on a need basis’, I mean that programs, processes, or actions initiated in User rights mode, may require System rights at some point (changing a systemwide setting, writing a file to a restricted directory, and so on). With ‘on a perceived-need basis’, I mean that the user can switch to System rights without there being a specific need; this may come in handy when setting up the machine in question, when doing heavy system modifications, or simply when an experienced user wants to know more about the system. This is one of the complaints behind UAC: you can’t tell it to maintain System rights.
Of course, switching to System rights requires the System password at all times.
The reason I settled on the naming convention of User and System rights (instead of things like root or administrator) is because I find them clearer to the user. You either have ‘User-wide’ access (Home and Settings directories) or ‘System-wide’ access. I find ‘root’ to be an inherently meaningless term in computing, and administrator sounds too much like someone who works for the government. And nobody likes government people. In addition, these names happen to fit quite well into the new filesystem layout.
Even though these two permission levels are the default, advanced users are of course free to fine grain permissions, add different permission levels, forbid specific users from elevating permissions, create a System user account that always has access to everything (root, basically), and so on. In addition, advanced users may also sandbox applications at will, by specifically forbidding certain programs or processes from gaining System privileges. This level of control is needed in for instance larger organisations or server environments, but can also prove to be valuable for programmers who want to test drive their latest code in a secure, non-destructive manner.
Of course, this is nothing new, but in my utopian system, this should actually be easy to accomplish. I’ve been wrestling with the Group Policy editor thing in Server 2003 for a while now, and dear lord, I have absolutely no idea what I’m doing. I’m reviewing a tool that allows me to fine grain the rights of processes and programs, but it’s so utterly complicated I’m completely lost. I see the potential – it just needs to be easier to accomplish.
Let’s move on to the real deal.
Filesystem layout
The filesystem layout is actually remarkably straightforward. I hope I can keep this paragraph short, because that would mean it reached its goal – to be self explanatory. The raw data:
There are two main themes behind this layout. The first is to separate the base operating system files from everything else, and tuck them safely away in
The second theme is the separation between between
To prevent the mess Mac OS X has with separating settings files from the program bundles, the system should somehow ‘know’ the two belong together – this shouldn’t be too hard to realise. One option would be to use BFS-like attributes; the Garden Designer settings file could have an attribute attached to it that links it to the Garden Designer application bundle, for instance. This way, when you want to remove Garden Designer from your system, the system can ask you if you want to delete its settings files too.
The
This still leaves one question unanswered: how are we going to manage all these applications? Wasn’t one of my points of criticism on Mac OS X, in the introduction, that it lacked a centralised method of updating applications? Indeed.
Cue BFS-like attributes and live queries.
Harold Kelley would be proud
This is really where the beauty of my utopian system starts to shine. Attributes in the BFS filesystem, combined with the concept of live queries, enabled you to do all sorts of crazy stuff with the files on your BeOS machine. Back in 2005, when I reviewed Zeta R1, I already explained a few interesting uses of BFS live queries:
/Applications that normal users rarely encounter. Over the course of time, this can amount to quite the mess. In addition, Mac OS X provides no way of updating applications in a central way, resulting in each application in OS X having its own updater application; hardly the user-friendly and consistent image Apple tries to adhere to.
/Users/User 1) and their settings directory (/Settings/User 1). Write access to all other parts of the system is restricted – of course programs and processes may read contents in the restricted directories, but they may not write there. Consequently, installing and uninstalling applications requires the System password. User 1 may not read or write to files in other users’ home and settings directories.
/System<br />
/System/Utilities/Users<br />
/Users/User 1<br />
/Users/User 2<br />
/Users/User 3<br />
/Users/Shared/Programs
/Settings<br />
/Settings/User 1<br />
/Settings/User 2<br />
/Settings/User 3/System. This directory contains the kernel, drivers, resource files (f. ex. icons), preference panels, and so on. I also took a cue from Mac OS X by creating the specific /System/Utilities directory, where the operating system can store utilities such as the Activity Monitor, graphical Bluetooth tools, Network Utilty (graphical ping, whois, etc.), those sorts of things. What does and doesn’t go into that directory is fairly arbitrary, and is open for debate. Other binaries that usually reside in /bin on UNIX systems can also go into /System (say, something like /System/Binaries, since this is the 21st century – why use unclear acronyms).
/Programs, which carries the actual self-contained program directories (program bundles), and /Settings, where the program bundles can store their setting files in user-specific directories (f. ex. /Settings/User 1/Garden Designer – the name must be the same as the program bundle’s name). The idea is that these setting files are more or less human readable (preferably in a standard format, shared by all programs), so that advanced users can modify them by hand. An added benefit is that people can share settings files with one another, which might help in troubleshooting scenarios.
/Programs directory contains all the, well, programs. Unlike most other systems, the programs themselves will be distributed as program bundles, analogous to applications on RISC OS and Mac OS X/NEXTSTEP. This way, possible uncommon dependencies can be shipped within the program bundle itself, to prevent the idiotic situations where users of Linux are confronted with having to download and install a whole batch of gibberish that will only confuse them.
First of all, let’s say I’m having a discussion via email with Eugenia. Because each email is stored as a single file, I can easily set-up and save a live query with all emails from Eugenia, sorted with newest on top. If the discussion continues to develop, I can refine that query even further, by adding a query on the subject of the emails. All that without ever starting an email client. Remember, these searches are instant; a lot faster than Spotlight. Also, they are ‘live’ queries, which means that they are updated automatically and instantly. You can save those queries as if they were directories. Again, this requires no extra applications or whatsoever.
Another interesting scenario to use BeFS is when you are putting songs on your MP3 player. Want all music from Bruce Springsteen? Or all songs from the Devils & Dust album? All songs from the ‘rock’ genre? You can do that without ever touching a music player or other specialized applications.
You are only limited by your imagination.
And you really are only limited by your imagination. Combine the concept of live queries and attributes with program bundles, separate setting files on a per-user basis, and the possibilities are endless. Seriously.
Each program bundle can store the basic information in attributes, the stuff you’d expect: name, publisher, date of release, country of origin, names of the main contributors, website, license, and so on and so forth. The boring stuff, but handy when you forget where you got the application from. There is also the attribute that takes care of the ‘linking’ with the settings files, but how exactly this would work, I don’t know – I’m leaving this up to the programmers and coders.
I also want to introduce a concept alongside the version number. When we think of version numbers, we tend to think of the numbers dangling at the end of releases, which are usually fairly arbitrary and don’t really say anything at all. Those would become ‘external’ version numbers. What is new is the ‘internal’ version number. How programmers order these numbers is completely up to them, as long as the second release’s internal version number is higher than that of the first release. This is essential.
Take Garden Designer. You have Garden Designer 8.0 installed on your system, which happens to carry the internal version number of ‘348’. Now, say Garden Designer 8.1 is released – the idea with program bundles is that various releases are named the same. This new version should get an internal release number that is at least 348+1. This way, the system can compare the attribute for “internal version number” and see that the program bundle Garden Designer 8.0 carries an internal version number of 348, while Garden Designer 8.1 carries 349*.
From this, the system concludes that 349>348, and will update the program bundle accordingly. However, the fun doesn’t stop here.
Who says you would want to replace version 348? What if you’re unsure about version 349? You may have heard horror stories online, you may have had a bad update experience in the past, or you may be running a production system. Well, simply tell the system to allow the program bundles for version 348 and 349 to live side-by-side, so you can continue with 348, which you know works fine, and test 349 before ditching 348. Combined with the ability to sandbox program bundles (see the privilege section) this allows you to do extensive non-destructive testing on your system. It also enables you to easily switch back to an older version – a prompt would warn you the version you are about to install is older, but if the newer version doesn’t work as expected, you can ignore the dialog and replace the newer, broken version with the older, working one. Fairly hassle-free.
A few of you are probably worrying about the settings files in /Settings right about now. How would the settings files deal with multiple program bundles of the same program? The answer is straightforward: think attributes again – the settings files can carry the same attribute for “internal version number” as the program bundles themselves, so you can easily have multiple settings files living side-by-side peacefully. Using live queries, you can search for unused settings files (using the attribute that links them to the program bundles), and clean them up accordingly in case you want to remove old settings files that you thought you might need in the future when you deleted their applications.
Now we can finally come to the centralised updating issue. Some of you will have probably already done the math, but here it goes anyway: I envision a centralised repository of program bundles, all tagged – of course – with the proper internal version number attribute. A very simple application could check all your programs’ internal version numbers using a live query, and compare them to the internal version numbers of the program bundles in the repository. You could tell it to prompt you for each update with the replace/side-by-side option, or simply have it update everything.
The centralised updating tool being built on top of a set of live queries, it could be made as complicated or as simple as the programmer would want it to be. You can do simple queries like “search for outdated programs and update them”, but also more advanced and detailed queries such as “search for outdated programs from Microsoft and update them”, or “search for program bundles with multiple versions installed and list them”, or whatever other possibility that might be useful. And the beauty of it all? Live query support is built right into the file manager in BeOS, so you can do all this manually if you don’t like the available centralised update programs. Ultimate control for experts, hassle-free updating for inexperienced users, and anything in between.
If you like a centralised method of managing applications – use the centralised updater/installer tool. Don’t like it, and prefer a more manual approach? Don’t use the centralised tool. The choice is yours.
* Why not use dates instead of these numbers? Two reasons: date formats differ across the globe, and you have different timezones that might interfere with such a system – you’d need a method of working around such a problem, which would just introduce another complication to the system.
The beauty
The beauty of this system is that it combines the benefits of centralised management with the ease of the self-contained program. You can manage your programs by using the file manager and live queries, or you can use a centralised ‘bundle manager’ like Synaptic if you are into that sort of thing – or ditch live queries altogether, and just do as you do in OS X. You can install programs from an online repository like .deb and .rpm do, or simply download a single application online – or all at once.
Since all the tools are based on live queries, gone would be the days of one updating application ‘locking’ the package database, preventing you from installing another program (like in Linux). Since live queries are, well, live, they update automatically, so there’s no database to lock in the first place. Say you are using an update manager to update the programs on your machine. While the update is taking place, you browse around and encounter an interesting application you’d like to try. Instead of having to wait until the update manager is done and unlock the package database, you can simply install the application yourself, and the live queries used by the update manager automatically adds the program to its list.
On Windows, Linux, and Mac OS X, it’s quite difficult to run different versions of the same program side-by-side, simply because those systems have an all-or-nothing approach to updating programs; either you update and completely replace the old version, or you keep the old version and do not update. Very, very inflexible. My utopia allows for a lot more flexibility – but you’re not forced into using it. ‘Grandmother’ will never have to deal with any of the advanced functions and possibilities, while advanced users can go all-out and do whatever the hell they want, not limited by the user-friendliness that appeals so much to grandmother.
In my utopian system, you’re not forced into a specific way of thinking, like Linux, Windows, and Mac OS X do. You are free to maintain your programs in a way that you, the user, like. And let’s face it, who should be the one deciding what’s good for you?
//Whether we are talking the really old days (launching the Rambo game off a tape)// er … i thought the “really old days” was when I typed three dozens lines of Pascal into a PDP-11 in order to get a simple tax calculation to run …tapes? You’re high-tech!
Yeah, punch cards and paper tape baby! None of that high-tech magnetic media for me!
Edited 2008-05-05 21:14 UTC
Give him a break, he hasn’t even been alive for 3 decades yet (IIRC)
No, the “really old days” were when we had to use toggle switches to enter the code to boot the computer. And, if my memory serves me correctly, computer tapes predate Pascal
Isn’t this just a less elegant version of GoboLinux with added live (and I repeat — they really are live) queries?
Quite right. This does not seem quite that utopian – everything to do this is already there. The reason nobody has set up something quite like it, is that its not such a good idea after all.
Oh? You do know what GoboLinux is, right?
Care to elaborate? That’s four pages of detailed explanations and user scenarios you just read, and all you can counter it with is “not such a good idea”?
It doesn’t matter how long it is. A good idea is a good idea in two lines and in ten pages. Just because it can be detailed for hours doesn’t make any idea good. Note, with that I don’t say this particular idea is no good.
The only thing is, it’s hard to satisfy everybody’s taste. E.g. I don’t like the Mac-style of application installing, I prefer dpkg, and I don’t like the folder names with compulsory capitals, and so on. That doesn’t mean the concept itself is wrong.
‘Note, with that I don’t say this particular idea is no good. ‘
But you are missing the point of the above poster.
He critisized the guy that said Gobolinux is not a good idea, but that other guy did not give ANY REASON AT ALL.
Noone can act upon FUD
And btw, this suggestion here is different from Gobolinux in a few key areas. I am not saying it is good or bad here, but it is obviously that there are a few people who neither read this, nor know about Gobolinux – or opt to simply not give any reasons at all why something is “good” or “bad”.
It is pointless to debate about opinions without people giving any REASONS behind them.
I don’t think I misunderstood, but nevermind [I was just trying to point at what I frequently ran into in the past, that someone can actually make a case about an idea in a really long form, still the idea can be completely wrong, which in this particular case isn’t].
As about Gobo, I’d say the above ideas are far better than Gobo’s, since I always found Gobo to be a bit strained, and I actually found the trials I made with it irritating. It’s not a native approach, not a trial to make a better file/dir hierarchy and better package/app management, it feels more like a hack. A system following the above ideas could actually be something better, even if I don’t really like some aspects of it.
It looks somewhat similar, but Gobolinux is an attempt to achieve modularity within the typical GNU/Linux paradigm, where there’s no distinction between the base system and the applications, that is, any package can depend on other packages that may or may not be present, and it can fullfill the requirements of yet other packages.
On the other hand, Thom’s proposal assumes a base system, which makes it more like PC-BSD or Mac OS X. Once you have a system/applications distinction, package management is almost trivial. But then all distros would have to agree on one base system, which would either kill the well-known configurability of GNU/Linux (as in being able to have a very small installation) or be such a small system that lots of duplicated libraries would coexist, among other problems. The LSB guys have been trying to do just that for years, and I don’t know how much success the’ve had, but the fact is that third-party program packages are still distro-specific. TANSTAAFL :p
‘The LSB guys have been trying to do just that for years’
No they have not.
The LSB extends upon the FHS. How can they achieve AppDirs (or AppDir like solutions) when they extend the FHS?
Well, I meant that the LSB project tries to stablish a core system (including specific libraries), so that third party application packages can just depend on lsb-core. For instance, take this section:
http://refspecs.linux-foundation.org/LSB_3.2.0/LSB-Core-generic/LSB…
“Packages shall have a dependency that indicates which LSB modules are required. […] Packages shall not depend on other system-provided dependencies. They shall not depend on non-system-provided dependencies unless the package provider also makes available the LSB conforming packages needed to satisfy such dependencies. “
Yes, LSB tries to establish a core. And most of it is okay, but it still doesn’t go quite far enough. Personally, I think they need to get RPM out of LSB and replace it with something like (though not necessarily) Autopackage. It would make the LSB a far better standard, and would really help towards Linux on the Desktop too.
autopackage? meh…
i would rather see some kind of standardized dependencies file so that a converter program could read it and spit out a functional distro package based on the incoming archive.
the thought crossed my mind as well…
No, it’s an elaboration on MacOS X. The system bundle is presumed to provide a generous set of shared libraries and prolonged ABI stability — no small feat.
As with MacOS, any library excluded from the stable system bundle must be statically linked into all dependent program bundles, each of which may contain a different outdated version of the library.
The main differences from MacOS are the semantic filesystem and the bundle repository, along with the minor hierarchy change. It wouldn’t be difficult to prototype this system for OS X using Spotlight or perhaps Nepomuk.
I have one question for Thom: I assume the desktop environment would be part of the system bundle so that its libraries could be shared amongst program bundles. So where on the filesystem would the per-user settings for such system components live? For example, does my wallpaper belong in /Settings/butters/System or /System/Settings/butters?
“Applications in Mac OS X are generally not easy to remove at all, because they leave a trail of files around outside of /Applications that normal users rarely encounter. Over the course of time, this can amount to quite the mess.”
The only files that are lying around are:
– plist for preferences (like .dot files in ~/ in Linux)
– cache files
– sometimes a folder in “Application Support”
That’s all. You can easily search for the name of the program with Spotlight/Finder to remove those items, though it is NOT necessary because the only downside of those files lying on your disk would be waste MB… you have to run thousands of apps and deleting them to call it a mess.
“In addition, Mac OS X provides no way of updating applications in a central way, resulting in each application in OS X having its own updater application; hardly the user-friendly and consistent image Apple tries to adhere to.”
If an application is started it will inform you about a new version, quite simple. Most of these dialogs are pretty consistent to each other too. And there are applications or even widgets that can check all your applicatins for updates, though it isnt really necessary, because.. when u run it it gets updated / will inform you. It would be nice to see Apple implementing this mechanism that those update-applications use into their own software-updater. But as said.. not really necessary.
The big problem with the Mac is that the apps arent consistent. Not all auto update themselves. Just like in Linux some software has to be compiled. Still, it would be nice if the world could agree on one single way to notify and put out updates. RSS maybe? In a corporate setting, repos work well since its pretty easy to make packages of something that needed to be compiled. But for single users adding a lot of repos leads to package poisoning.
I so wish I had time to read this Thom. Like many of us, package management is the first and foremost reason we “advanced” Linux users will choose a distribution. Well, actually since I discovered Debian I’ve never looked back except for the occasional brush with Yum… and thinking that Red Hat has had APT as the example for YEARS, and Yum is the crap they came up with?!?!?
Anyway, care to make it a podcast or something that I can put on and tune in and out while I’m coding? Mostly kidding, that’s a lot of work… would be more efficient in the end if I just read it
Anyway, nice article, love the exclusives.
Edited 2008-05-05 22:38 UTC
‘Like many of us, package management is the first and foremost reason we “advanced” Linux users will choose a distribution.’
Not me. I guess I am not alone though. Personally, I simply refuse to use solutions that do not give me the _OPTION_ of having something like AppDirs.
AppDirs are simply a better way to manage applications than FHS, period. If I want to have something removed, i go kill the dir. I dont _want_ to DEPEND on a package manager in order to uninstall something again.
If you want to have different versions of the same programs then use Gentoo. Portage has a nice feature called SLOTs.
Hi Thom,
One of my many pet projects for Haiku has been a new “package format” for applications as well as a consistent updating mechanism and some sort of central application repository.
This might make a good initial design, since you make use of the live queries that we will have in Haiku. Overall it sounds pretty good.
I think the permissions system you describe could also be implemented.
But I think the file system layout will need to remain in the standard BeOS format we are inheriting. But I still think much of your design could still be used for that. For one thing I don’t see why you need a separate /Settings hierarchy when you could just have /Users/User 1/Settings or in Haiku /boot/home/user/config. Of course the multi-user aspects of Haiku are still in flux and probably won’t be sorted out until R2.
You’re right Ryan, this whole thing actually started out as a discussion in #haiku a long time ago. I was just musing aloud at how you could use the attributes and live queries in BFS to manage applications, and from there, this whole idea started to grow.
So it makes sense that my ideas fit Haiku so well.
The reason I chose for a separate hierarchy is because I want the /User/User 1 directory to be strictly a directory for the user’s documents, movies, photos, pr0n, and so on. I’m someone with a strong inclination towards order and cleanliness, so you can imagine why I’d like to not put settings files into the home directory.
And thanks for the compliments
. I’ve spent a lot of time on this proposal, and I believe I’m only scratching the surface of what attributes+live queries+program bundles can equate to. If you want to discuss this in more detail for whenever the package management plans for R1+1 come up, feel free to contact me, I’d love to participate in that discussion 
.
I don’t think that separating the user and its setup configuration files is a good idea: this means that when you want to backup your account, you need to save two directories not one..
As for the cleaness point of view as long as all the configuration files are in the setting directory, I don’t think that this is dirty.
That said, I’ve been convinced recently that purely hierarchical file is impossible to get ‘right’ so if the user documents and his configuration file are tagged by his login name (automatically) then the backup of the user’s data becomes far more easy..
I always see my User folder as “everything that makes this computer unique to me”. That includes my settings, so I can backup just one folder, and even move it to another computer and log in with everything there.
Locking would still be necessary because while queries are fast, installing is not.
Let’s say you use one query to install program P and start another query and uninstall library L (something P depends on) along with all older programs that need L.
There are two ways out of this:
a) Using a more fine grained locking along the lines of “install anything you like but make sure not to destroy L since P (currently being installed) will need it”. What, however, if you said install P from vendor V and while it was installing told the computer to remove all programs of vendor V. I’m almost sure no matter how smart the algorithm is there’ll still be situations where it has to say “Encountered Conflict C, do you want to do x or y?”
b) Making every program completely self contained.
This would have HUGE security implications.
A defect was found in a library that 100 of your applications use? Well, that’s to bad, you have to reinstall them all.
You can of course draw the line at some arbitrary point and say this lib is used ‘a lot’, so it’s shared.
In short, while your idea sounds good I believe it has tons of details, corner cases, and trade offs that still need to be sorted out.
Furthermore, it can only be realized embedded in a bigger ecosystem. To revisit the example with a defect in a lib, developers could tag their applications like “works with version x of this library” in a way that the program could automatically tell you “vulnerability found, switching to fixed version of lib L” or “vulnerability found, program not yet tested with safe version of L, do you want to upgrade and risk a crash or keep running this unsafe program”.
In addition, your system is still centralized.
There needs to be someone or some server to say “No, malwareGuy, you cannot call this ‘Paint 6’, there’s already ‘Paint 5’ and you didn’t write it”.
Somebody has to decide what goes on the server and what doesn’t.
I could go on and on but I think you get the point:
The idea is good but the devil is in the details.
The server?
Who says people can’t set up their own server for distributing program bundles? It wouldn’t be too hard, as long as the file system on the server preserves the attributes.
I also envision a set of command line tools that allow you to compare/update program bundles. Something like:
$ compare “/Programs/Garden Designer.bundle” “ftp.stuff.org/pub/bundles/Garden Designer.bundle”
Output:
$ /Programs/Garden Designer.bundle:438
$ http://ftp.stuff.org/pub/bundles/Garden Designer.bundle:439
Doesn’t every program need a unique identifier?
How do you make sure that no two programs can have the same identifier if everybody can set up a server and name/tag the apps as he pleases?
They’re not identifiers, they’re values stored as an attribute. Even if you have ten billion million attributes with value 345, if they belong to different files, that simply doesn’t matter.
Maybe I just misunderstand what you say but aren’t all these attributes together sort of an identifier?
So you could have two files with attributes
program=paint
vendor=ms
version=4.1
patch_level=127
…
and if they contained different binaries it would be a big problem. You could, of course, add mechanisms like checksumming the binaries and so on.
But then again you’d need a central server/group of servers that tell the user that a program with attributes x and y should have checksum z.
Am I missing something?
Or alternatively, the developer/maintainer or whoever does the bundling can simply use some sort of cryptographic key, which the installer uses to confirm the new bundle is indeed from the same source as the already installed bundle.
Basically, the problem you’re trying to solve is intrinsic to any form of installation, not only the idea posted in this article, so whatever solutions exist already can be used without any need for centralised administration.
If I recall correctly, zero-install used some crypto-signing for something similar.
Nice try, Thom – if you’re just reading it casually you almost miss the hand-waving about binaries that don’t belong in any particular ‘program bundle’, and about the issue of shared libraries, which is of course the big drawback of the OS X system that you *don’t* mention (because it persists in your vision). If you use shared libraries, you have a reliance on your vendor (situation with all current Linux distributions). If you don’t, you have security issues and ancient bugs that were fixed long ago cropping up all over the place (situation with Windows and OS X).
GNU/Linux as no reliance on vendor for anything.
The linux kernel as many version and modification of itself. So do the library , so do the x systems , so do the windows environment , etc …
SLS … Debian … Ubuntu …
SLS … Slackware … SLAX
Red Hat … Mandriva … PcLinuxOS
Xfree , X.org
KDE , Gnome , Xfce
ETC …
You got acces to source code , it’s Open Source developed , and it’s Free software.
You can fix it yourself , train someone to fix it or hire someone else to fix it for you.
Miss? It’s right there in the article:
Yes, it is. That’s the bit I’d characterize as hand-waving.
‘If you use shared libraries, you have a reliance on your vendor (situation with all current Linux distributions).’
That is true, and I am one that constantly critisizes the upstream vendors/maintainers as well, but you see – the situation on Mac is as bad as on Windows in that you are dependent on a company just as well.
So all these worlds more or less have similar problems. You depend on someone else.
Now, with similar problems already, I as a user would still like to choose AppDirs instead of FHS.
I dont want that others enforce the FHS upon me. But the big distributions have no inclination to change to AppDirs at all.
We should also have a /Users/Common directory for documents (and a /Settings/Common for settings) that all users on the system need access to. Of course the system needs to support user-controlled file permissions so User 1 can grant write privaleges to User 2 to a directory he owns but not allow User 3 to read it without being a System user and needing create a group.
What about file associations? I know this could be handled by the window manager but I’d like some central database of what applications are associated with what files, even on a user by user basis. On Computer 1 Firefox should be the default browser for all users but Computer 2 User 1 wants to use Firefox while User 2 likes Konqueror.
This new program management is the best idea I have ever heard for managing programs!
Why store user info in /Users/User1/ and settings in /Settings/User1?
Why wouldn’t you use /Users/User1/settings for settings? You want the home directory to be less portable? Less backup-able?
Real separability. For example, many people who test distros will want to share /home, but later find themselves with overlapping Settings that do not work with the other distros.
These days I maintain my files in /home/xiaokj/Link and make the link to some central repository (another partition). It would be nice if I need not do that, but rather link specific apps’ settings (like IM/Email that is really supposed to be persistant) instead of making sure the dozen other settings that should be separate (like Xauthority, KDE/Gnome settings…)
So it’s basically doing the same thing that so many people complain about with the rest of the file system. Scattering files all over the place.
If each program has an “internal” version number, then perhaps the corresponding /Settings/User/X directory be the version number instead of the program name? So if Garden Designer is v846, then the file is /Settings/User1/846. This way, you’d really be able to run parallel versions of apps.
But there’s another problem – there might be another program that is internally 846. We need a unique number. Wait, that already exists!; enter the GUID.
My point is, there are some really cool ideas in here, but ultimately, I think it needs a lot of refinement. Storing the settings in a separate space just means you’ve improperly used the home directory in the first place (see my above comment). Your permissions *will* be screwy when you put settings in one top level directory and user data in another. This is why there are hidden directories. Also, your arbitrary requirement of the settings directory sharing the name of the software package will need some sort of low-level monitor. It also means that when I release a new version of a package and choose to rename it with a version number, I can’t reliably find your user settings from any of the last several versions of my software, I have to try every possible combo or not let you migrate your settings from any version but the last one.
Frankly, I think the current OS X way is near perfect, with plenty of room for small improvements. I don’t think that, with disk space as it is, the system is “cluttered” by having lots of settings file – stored properly where settings should be stored! The idea is that it’s persistent, it’s there the next time you install the app. And I think Leopard’s Spotlight is plenty fast enough for 99% of users. Live queries are very cool, but very few people would use them in a mainstream OS, and existing technologies provide most of the end result.
And how user friendly is that? The hierarchy I devised is supposed to be ‘human readable’ – actually, that was one of its primary goals. File systems have been a mess since day one, and it simply needs to be fixed. OS X made tremendous strides in that regard, but in the end it’s just a virtual directory structure draped over a traditional UNIX/POSIX structure.
Hidden directories are evil. If a system needs to be secretive in order to not confuse a user, there’s a design error somewhere.
I addressed that issue with a typical utopian statement in the article:
internal versions can be easily confined to specific programs.
A more practical concern would be the fact that programs can change names. However, it can also be overcome with a specific file, in the program bundles, that state the previous names.
I think most of us have thought about such ideas in the past. The devil really is in the details as someone said above.
I tend to like the idea of ‘application bundles’ but then how do you handle shared libraries. You will end up doing the weird windows DLL handling.
For example you could have a ‘.lib’ file in each program directory that tells the loader which shared libraries to use for the application. The application bundle would include a version of the library that is ‘known’ to work.
shared libraries not belonging to a particular application can also be installed a generation /system/libs directory.
If a newer version of a ‘.lib’ is available, then somehow the system must decide if it wants to try that version instead of the version in the application bundle. This ‘somehow’ is undefined. Perhaps the system detects a new version, asks you if you want to try the upgraded version… and goes back if it fails. Maybe we leave it to some online repo… This somehow could get complicated fast.
——–
in terms of updating the system. I don’t really know why you’re complicating it with all this search and queries… Wouldn’t a simple file with the application bundle, pointing to some server location do. The system can check if there is an updated version and if so, prompt for you to download/install it.
One of the most frustrating thing when you live on a linux where you don’t have root access is to install programs. you basically need to go the “./configure –prefix=$HOME && make && make install” and do it yourself dependencies.
Not all programs require root privileges. Then, why not allow users to do “apt-get install whatever” and it goes to their directory until the program is superseded by a system wide installation?
Going along this idea would be a first step towards better program management.
Not all programs require root privileges. Then, why not allow users to do “apt-get install whatever” and it goes to their directory until the program is superseded by a system wide installation?
I have wondered about the same thing myself. But I guess it all boils down to the fact that people expect only two kinds of users to install software: system admins or users who own the computer. The truth is however that there are also people who don’t have the root password but might still wish to install something additional. Such users could f.ex. be your children.
So yeah, I basically like your idea. All the files and folders should however go under a single folder in the users’ home, like f.ex. /home/user/Applications and of course everything should be owned by the user and writable only by that user. Oh, and yes, the system admin should still be allowed to choose which users are allowed to install software this way. In corporate environments for example it’s often preferred that users are not allowed to install anything but instead ask the admin for that.
Most of the stuff kids are going to want to install will run in Wine, which handles installing crap applications inside a users home directory just fine.
I wish I was a kid 😉
The “ask the admin” approach usually takes some time and gives the admins a bad reputation of being not very responsive.
If not in an environment where installing an application may compromise security and should be prohibited, there is a clear benefit in letting the user do basic administration tasks (installing stuff). Debian-style package management makes this possible because installing something is designed to be a zero hassle operation. However, the neat thing about being superuser to install stuff is that it keeps the applications clean and working. A similar approach should be used by not giving the user actual write rights in his app folder, but using a setuid mechanism so that only the package manager can do that.
If programs are organized as AppDirs, and they are not allowed to rely on being installed in a specific location in the file system hierarchy, this may be achieved simply by unzipping the app wherever you prefer. No need for installers at all.
The trick with Thom’s attributes idea, is that you can still query installed applications even if no installer was used to put them there.
Unix security has a very simple way to do this. Just mount the home partition as noexec. There simply is no way you can stop the user from putting a binary in his home directory, but noexec makes sure that no user installed files can be executed.
I enjoyed reading the article very much and it seems like a good base for building a new packaging scheme. However, there are still some areas that need further work. Some assumptions of the proposed system were also not clear.
The first one is the operating system, is this meant for Windows, OS X (OS XI?), Linux, some other OS, or an entirely new system? For “legacy” systems, the biggest issue of all seems to be providing a migration path from existing software deployment methods to the new method.
The handling of shared libraries, and the necessity of separating user settings from user homes have been mentioned by others. Also, the write-up talks about a single host, but in practice, program and user directories are often shared among several hosts. It is not obvious that the system would work well in such scenarios. More detailed consideration may be needed about what is shared among which hosts and users. Depending on the OS environment, support for multiple architectures may need to be considered as well.
One of the biggest issues is the storing of settings. Supporting multiple versions of a program in parallel, sharing the same set of settings, is a pain in the ass for software vendors. I think that a lot of conflicts would arise from trying to run different versions of Word, Photoshop, or even GNOME2 with the same set of settings.
So if different versions have to share the same set of settings, many software will likely refuse to install without having prior versions removed first. In such case, if you have other software packages that depend on this kind of a package, you’re again stuck with dependency hell.
Furthermore, as mentioned in the article, sometimes the user or administrator does not trust a new version of a program. So even when the software supports shared settings, the user may not want to allow it to modify the settings used for the “trusted” version.
One solution may be to store settings for different versions separately, but have the installer generate the new settings automatically based on those for previous version.
Finally, the proposed system also does not seem to support installing programs without system privileges, which is normally possible for most Linux / OSX software and an increasing share of Windows software.
Good luck for working on this new system, and looking forward to hearing about the progress.
It’s not meant for any specific system, but you can see my involvement in Haiku and my love for BeOS shining through.
Except, they don’t. The whole idea is that the the directory /Settings/User 1/Garden Designer could contain multiple settings files for multiple versions – just as I explained in the article. They would be differentiated by their internal version numbers. The file manager could show them as follows:
/Settings/User 1/Garden Designer/settings.xml:438
/Settings/User 1/Garden Designer/settings.xml:439
The same goes for the programs:
/Programs/Garden Designer.bundle:438
/Programs/Garden Designer.bundle:439
The idea is that the system is clever enough to only display the internal version number when there are actually multiple versions installed. Since installing multiple versions is most likely an expert endeavour only, normal users will never encounter such internal version numbers attached to their program bundle files.
Installing Linux packages without system privileges? Some exotic Zero Install systems may be, but other, more conventional package systems all require system access (as far as I know).
Anyway, executable code should never be able to be installed without a system password, if you ask me. I’m quite strict in that, I know. Executable code is the basis for A LOT of attacks, so it simply shouldn’t be something just any user can dump on the system.
Edited 2008-05-06 10:31 UTC
Thanks for the clarifications. And sorry, I obviously somehow missed the paragraph about multiple setting files.
I still have some things to add…
Multiple versions
I don’t agree with having multiple versions being an unusual case that “normal users will never encounter”. In fact, I think that the ability to have multiple versions of the same software side-by-side might be one of the most sought-after features in program management.
Not being able to have multiple versions side-by-side is what keeps a huge number of users at businesses and many at home stick with older, often outdated software. Many people won’t upgrade unless they are absolutely certain that they can do the same things with the new version as they used to do with the previous. And with most existing systems not allowing to use multiple versions side-by-side, there is usually no way to make sure.
So I’m confident that users will install multiple versions when the program management system allows it. This is even more needed for systems with multiple users. In such case, the migration of all users may take longer, and so the time frame when multiple versions are installed side-by-side becomes longer. Note that company and school systems with large numbers of “normal” users fall into this category, where multiple versions would be available for extended periods of time.
Actually, in the utopistic program management system where multiple versions can exist side-by-side without conflicts, there should be no need for the user to explicitly uninstall previous versions. Old versions out of use could be removed automatically in a time-machine-like fashion. (In practice you should consider removing them sooner for security reasons.)
Installing with user privileges
Several other comments have mentioned this as well, so this also seems to be a sought-after feature. (This may also be related to the reasons above, e.g. some user wanting to use Firefox 3 on the company/school etc. system where Firefox 2 or maybe 1.5 is installed system-wide.)
I have to mention that when I said “installing” I was referring to software deployment in general, not one specific deployment framework. For example, I can download an archive (“bundle”) of Eclipse for Windows, Linux or Mac OS X. For each OS, I can extract it in my home folder (“install”) and run it from there. This does not require administrator privileges (“system password”). (For Linux, installing other applications in home folders may require tweaking with paths, but for most programs it is possible.)
Note that from that user’s point of view, after it has been installed, such an application can be used in the same way as a system-wide one, the only difference is that it is not available for other users.
With a clean framework as the one described in the article, it should not be difficult to add support for enabling this kind of installation for all bundles. (As suggested by others, they could be put under “/User/User 1/Programs” instead of “/Programs”, for example.) Doing so would reduce the amount of user actions required for such installation, allow automatized upgrading of all software a user has installed, etc. It does not make the system more vulnerable, compared to when the user has to do all these steps manually. Finally, it would also reduce sysadmin load as users who weren’t able to install by themselves would not need to ask the admin anymore.
but in the end it seems like it would basically be OSX + a centralized system, which isn’t exactly revolutionary. The whole thing with BFS live queries could be handled exactly the same with the centralized system using a database, you’re just moving the relation management from the db into the native filesystem.
…but my personal wish is for applications always to be fully self contained with any third party stuff they need contained within the program directory. Sure this would lead to bloat on disk size, but whats taking up more disk space on your hard drive? Those 10Gb of applications or the 100’s Gb of MP3s/AVIs?
Im a big game player and i like games which have save games IN the game directory (eg: under \save or \save\profile – for multiple profiles). Saving under My Documents a la Microsofts recommedations gets me really annoyed.
And dont even get me started on saving game/configuration information in the registry. The registry is a bad idea from the start. Whats wrong with .CFG text files?
While i accept the need to consider multi-user systems and privledges im sure self-contained apps could somehow work with this.
Anyway, nice article, even if some ideas need refinement.
The thing is if the application is really ‘fully self-contained’ that this increase not only the disk usage (which we don’t care I agree) but also the memory usage (which we do care about!) because a ‘shared’ library wouldn’t be shared anymore as each application would load the version stored in his directory, the OS wouldn’t know that these version are identical..
Currently the way I see to fix this is to have a ‘delta’ phase in the installation where all the ‘shared’ library contained in the application would be checked against the already installed one, if a shared library already exist in the same filesystem then it wouldn’t be installed but it would be hardlinked to the existing one.
Benefit: no disk or memory bloat and the application are still self-contained.
Inconvenient: an additionnal ‘delta’ phase
The big downside of the ‘self-containement’ part is that if a shared library must be upgraded then all the application which use it must be upgraded..
Had this too often with apps with shared components, eg some lib windows\system32. New application breaks old one, or old application breaks new one.
Perhaps this is sometimes caused by bad programming and/or packaging, but it does happen from time to time that two apps require exactly the same named lib that should reside in the exact same location or else it wont work, and those two versions of the lib are not compatible.
Hence my liking of self contained. And besides, sharing only bloats memory of those apps are memory resident/drivers/or simultaneously active. Not going to always be the case.
I wish Ubuntu would use a system like you described in this article. It’s much more ‘human’. I really have no idea where Ubuntu stores all it’s program data, and usually config files and log files etc are all stored in different places, for each different program.
It’s too confusing, and when i want to do something myself, instead of using Synaptic, i usually end up confused..
I really have no idea where Ubuntu stores all it’s program data,
1: Why do you need to know that?
2: Need aside, all that info is stored in the .deb. Kpackage (and Synaptic I think) have a way to show where it installs all the files in the package.
and usually config files and log files etc are all stored in different places
If by “different places” you mean various .directories in ~, then sure. What would you propose? All setting stored in one file (or hive, hello registry)? Or perhaps settings from different files presented in one unified interface (gconf might be what you are looking for)?
Or perhaps you want settings to be stored alongside the binaries in the self contained program folder, ala older Windows stuff? Thing is, that doesn’t even work in Windows anymore. Permissions and privileges means user settings go in the user’s folder. Are your settings in Application Data? Or Local Settings\Application Data? (or the registry?) By contrast, I find locating ~/.appName to be slightly easier (or ~/.kde/share/config/appName for that other DE).
I’ve said it before, but your complaints look a lot like what I am now going to dub WindowsPoweruserSyndrome. People who, instead of installing everything to the default location (c:\program files) discover that they can install games in e:\games and little apps to E:\myApps. It seems more organized, and you are in control! Linux then by contrast seems to take some of that control away when apt-get install puts things where it wants. It’s disorienting. “Where is the program? What do you mean they are all in /usr/bin? With the data in /usr/share and heaven knows where else?” Where is programX installed? Synaptic knows.
when i want to do something myself, instead of using Synaptic, i usually end up confused..
Compiling and installing a program for system wide use, and doing so outside the confines of the package manager can be a little disorganized, for sure. It’s easy enough to make install, but then uninstalling is a crapshoot. For this instance, I’d recommend getting used to checkinstall.
For the rest though, unless a program is not in the package database, it’s a lot easier to just let the package manager deal with it.
Linux is different. If one don’t want different, maybe one really does in fact want to be using Windows instead. It just seems to me to be a little lopsided to leave Windows for Linux to try to make Linux more like Windows.
Now I could obviously be reading way too much into your post. Maybe you don’t like Windows either. Maybe you don’t like how any OS does it. In that case, Linux, due to its open nature, is the perfect candidate for creating a system more to your liking, or hoping someone else does. Perhaps GoboLinux is your thing. But don’t expect every distro to do that.
Im confused…. is Linux now the choice for freedom or is that Windows?
I see Linux as the freedom lovers choice and i would like the choice of dumping my games in /home/loki/games/… which i do, as my wife has no need to Quake 3 anyway. If she wanted she could have her own install as well with her own mods.
Or if i want a common install why not drop it under /usr/games or whatever, or for that matter /games (eeek!!)… its my computer, its my OS, its my choice.
Package management certainly should be able to (and always presumed it did if you provide the right switches) allow you to dump where you want (providing you have the rights to the directory where you want to install).
Quite a few constructive comments already, thanks guys and lady – that’s exactly what I was aiming for. It’s one thing to come up with an idea, it’s another to be flexible and open enough to take in any criticism (valid or not) and deal with it. I’ll address the two repeating issues here, and will address any specific questions in replies to their respective comments.
First I’d like to point out to everyone that this is supposed to be the utopia of program management. Utopia kind of implies that it is impossible to achieve fully. Take any utopia, and you’ll see there’s massive overlooking of details, or simply not answering certain pressing questions as to ignore certain weaknesses (Earth in Star Trek, anyone?). So, certain things are most likely never fixable or addressable.
This is the most valid of criticisms, and also the hardest to properly address. A typical utopian solution is this: common libraries go into /System, and any special or uncommon libraries go into the program bundle they belong to. Nice utopia, of course, but this raises the question: what is common, and what isn’t? I can imagine the operating system vendor/programmers making that decision – they can provide a base set of libraries from the get go, and then listen to developer feedback as to which libraries they like and want in the basic set of libraries.
But you all have a very valid point: this is one of those things that requires balancing on a very thin line. For a company like Apple, something like this is easy to solve (Apple is god in their world), for Microsoft it’s a lot more difficult, and for the open source world it’s nigh on impossible.
Because that goes against my personal (it’s MY utopia, after all
) idea about what the home directory should contain: a user’s home directory is for files like photos, documents, pr0n – not for settings or config files. I always find it extremely clumsy to see stuff in my home directory that I would rarely manually touch anyway. I created the separate /Settings hierarchy for this purpose. Settings files != user docs, simply put.
Like I said, this is my utopia, and it might very well be that just about anyone else would like /Settings in their home directory – I don’t know. A few of you proposed something like /Users/User 1/Settings – I could probably live with that
.
Having had a personal conversation with Thom (more than a few times) and also having had thoughts about this sort of thing, I’m convinced that part of the problem with the article is Thom has unintentionally assumed people fully understand how BeOS/Haiku attributes work, and has left some things unconnected between what he wrote in this article and in his mind, compared to what’s going through the mind of the readers.
When used correctly, and with a copy utility that’s attribute-aware, the attributes of BeOS/Haiku make it feasible and practical to not get your panties all in a twist as to the exact human-readable directory structure, because when used correctly, you can think of a combination of attributes as an explicitly set directory path that can be set in any particular order. Indeed, attributes are stored in separate indexes which are a form of filesystem directory to start with, that you don’t typically see, at least… not without the native index listing tools.
Where am I going with this? Well, think of it like this: each added attribute makes it more readily found for any particular file attached to an application and/or user, and they aren’t mutually exclusive. Here is an example of how things could be tied between applications and users:
1. All applications have a human-friendly name: for all applications and their settings files, add the attribute to the files called “ApplicationName” and fill it accordingly.
2. All applications *should* have a version: for all applications and their settings files, add the attribute “ApplicationVersion” and fill it accordingly.
3. All applications have users, even if it is only the System: for all applications and their settings files, add the attribute “User” and fill it accordingly.
4. Someone above mentioned different hosts: for grins, we do exactly the same thing! Ditto for filling that accordingly.
5. Sometimes you want to experiment with different settings: explicitly have a separate date attribute for this purpose, and by default, use the latest dated files: this provides for versioning of settings at will. Ditto for filling that accordingly.
6. For external library dependencies, we do something similar: the library has its “ApplicationName” filled with itself, and lists all applications with their versions as their “users” and we thus get a nice symmetrical solution for handling all that the same way. As soon as there are no users of that particular library, it’s safe to blow it away. If you want to blow it away, by running the query, you can tell exactly which applications/libs also depend on it, and which users depend on those, and deal with it accordingly.
By doing a query along the lines of:
finding application name A
With latest version
With user name B
With Host name C
With latest version of settings D
With Libs defined by attributes E
You can explicitly choose to copy or execute any version of any application on the system, assuming you have permissions. Because permissions also exist on the attributes itself, you can hide details from the user where they don’t have the rights, and inform those running a query, “One or more dependencies exist for this” that they can’t change. The nice thing is because how attributes work, they effectively provide a symbolic link for each time you add one, to their respective information, but it’s location-independent: as long as the attribute indexes are built on each volume, the location of the file that has all these attributes (and it’d have to match for each one: because of all the differences in attributes, if you have two different users with otherwise identical attributes, they get two separate files) can be anywhere on the filesystem, so Thom can have his organizational method, and you can have yours, if you so desire: perhaps Thom hasn’t convinced you of the wisdom of having a Settings directory outside your home directory, so you have a Settings directory as a subdirectory of your home directory. Regardless of which one you do, you *still* need to have an attribute-aware copy/backup program, and it still requires the filesystem properly index and work with attributes.
An added note: for those libs which are badly abused for names where the names stay the same, but different actual versions end up getting put over them, each application should have a set of “Uses” libs files for each one in its directory that lists the MD5 checksum for each one, so the application (or the application loader) can detect that there’s been a change, and at least warn the user of that.
While all these attributes do indeed provide a heck of a lot of power to make things more manageable without an explicit package manager, here are the costs:
1. The user must always have a rather intelligent copy facility that’s aware of this information and how it is used.
2. Application developers need to use this internally: this is clearly not POSIX-compliant (for the current state of POSIX) and thus ported applications aren’t going to have this if they’re simply recompiles.
3. This requires fully indexed attribute storing filesystems: this isn’t the problem for the attributes, as Windows NT-based NTFS has attributes, though I don’t recall whether they’ve got their own indexes outside the standard file metadata.
4. Working with applications and files with all these attributes has a time overhead for all the disk seeks: once we move away from rotating media, this likely will be a non-issue, but it exists for now for most non-Flash-based filesystems.
5. I’ll be surprised if I didn’t forget something…
It’s called utopia for a reason … It can never exist in reality , the H factor always screw up things.
H = Humans.
—–
I will pass on commenting on it , I wrote a 500 page in 10 minute about all it’s flaws and unrealistic behaviors.
But then who am I to stop progress just upload it to a server so that we can download it and test it .. What’s that it’s just a theory ?
Also in Utopia there is no need to install , upgrade manually , uninstal software. Software are Free and gratis and you only pay for development or to buy a new computer and they come with 10 X 1 million terrabyte hard drive.
We don’t even really have a need or use computers.
First sorry for the poor english (its not my native language). And I am not shure how this ascii written directory “diagram works” on osnews forum… this is my first post here althou I have been reading osnews for years
/boot (kernel and etc that are needed for boot)
/system/ (Basic system files for working base system)
sbin (binaries that are always needed)
etc
man
lib
/programs/ (For system and all users, programs here)
nodep/programsname-version (must hav no dep! and respect home/settings saving! Otherwice free to custom at will)
defsettings/ (default settings that are copied to users settings when first run by user)
programsname-version/
bin/ (programs binary)
lib/ (programs personal libaries ONLY (no general other use))
defsettings/ (default settings that are copied to users settings when first run by user)
other/ (pictures, sounds etc related to the program)
deps/
strict/ (must have with strick version)
lib/
libname-versision/libfile-version (LINK or FILE see note 1a)
other/
programsname-version/ (LINK or copy of the entire program see note 1b)
musthave/ (must have)
lib/
libname-versision/libfile-version (LINK or FILE see note 2a)
other/
programsname-version/ (LINK or copy of the entire program see note 2b)
optionaly/ (that good to have(more functions), but not mandatory)
lib/
libname-versision/libfile-version (LINK or FILE see note 3a)
other/
programsname-version/ (LINK or copy of the entire program see note 3b)
User and Group directories
/home/username/
programs/ (for the user only own instable programs)
programsname-version LINK#
settings
programsname-version LINK# (settigs for the program that its linked for)
documents/
movies
music
pictures/family (users selected share) LINK¤
etc…
shared (default share) LINK¤
/home/groups/groupname (where thee groups files are stored and installed. See below)
Virtual links from users home and group directories!
/shared/groups/ (shared groups that can be made by the system and run by the groups privigles and rules see note 4)
wingames/ (exsample of “wingames” group that is shared by some users)
programs/sharesave/programsname-version (programs that have/save bin/settings in one directory)
newgames/programs/programsname-version LINK% (program that is shared, but has settings in settings folder)
settings/programsname-version LINK%
/shared/users/user1/sharenamefamily LINK#
/share (default share) LINK*
Priority wich gets run selected first (adjustable)
1) Users own programs/libs
2) users groups programs/libs
3) System programs/libs
Packet manager takes care of all programs excluding /programs/nodep and /shared/grops/programs(basicly just links)
/home/user/programs (if not othervice setup)
(note 1a/2b)
(if the system has it, its linked! otherwice its copied here in the install prosess or if the system changes/deletes the its copied here automaticly)
(note 2a/2b)
(if the system has it, its linked! Otherwice its copied here in the install prosess or if the system changes(question is made: “Do a copy here or relink?”). If Deleted then its copied automaticly here.
(LINK or FILE see note 3a/3b)
((if the system has it it linked! otherwice its asked what to do) and if changed only version numbering LINK is updated and if deleted System asks shall it be copied here or not?(default is no).
The idea of having so many program folder is to have real choise how software can be installed and still keep the system in order.
So what do you thing of the idea?
Hey Thom,
in some ways, your proposal looks similar to Haiku’s Installer RFC:
http://www.haiku-os.org/glass_elevator/rfc/installer
Did you have a look at that one? It’s not really cleaned up, though.
Bye,
Waldemar Kornewald
Perhaps you should check out Plan 9 and OpenVMS for more related ideas. VMS has some advanced FS characteristics inc. versioning and the ability to treat a file as a series of records, so you can choose to refer to a particular field or not within the FS. This includes copying parts of files over LAN’s, etc. Incidentally, you will notice MS has picked up quite a few ideas from VMS; search for both to read about their shared heritage, if you’re interested. I suppose MS will support versioning soon, since it’s been around for 20+ years!
Maybe they’ll just leapfrog it with WinFS (or whatever they’ll end up calling it).
Plan 9 gives each user their own virtualised FS: each user can mount (remote or local) resources as they see fit, creating their own personal FS namespace. This extends beyond files and folders to almost every network- or locally-accessible object. The Plan 9 site is down for me, but you can read a bit about it here: http://www.faqs.org/faqs/comp-os/plan9-faq/
In the medium-long term, I see more FS’ dropping the hierarchical `root -> folder 1 -> … -> folder n’ construction (it is only a design choice, after all, not a divine mandate!) and moving to a database-driven FS complete with rich attributes etc.
(Oh, and what about ZeroInstall? That achieves most of what you want, on a system of today. http://0install.net/ )
Thoughts?
Edited 2008-05-06 12:53 UTC
Overall some interesting ideas.
I like self contained bundles. Shared libraries are a mess that has never really been sorted out properly. I don’t think the security implications of having 15 copies of the same exploitable library in 15 different application bundles is all that big a deal.
I would like the program bundle specification and layout to be platform independent, and “fat” – for example a single bundle could contain Linux, OSX and Windows binaries. Yes, this will make for bigger bundles, but it will also make things simpler. Fundamentally people shouldn’t have to know what operating system they are running. They should be able to download a program and have it “just work”. Vendors would always have the option of creating single platform bundles if size is an issue.
I am not sure I get your file layout though. /Settings/UserX? How about /Users/UserX/Settings. And why the single global /Programs? Fundamentally any program should be installable by a single user without requiring any privilege escalation. We should have something like:
/Programs – Share programs
/Users – User home directories
—/User1
—–/Documents
—–/Programs – Programs installed only for this user
—–/Settings – Global and local program settings
/Settings – Global program default settings
For shared programs, /Settings would be populated at install time with a default settings file provided in the program bundle. When a user first runs the program the settings file for that bundle will be copied to their local /Users/UserX/Settings. This allows admins to provide global default settings. The bundle could also split it’s settings across multiple settings files, and mark some settings files as read only, so that they are never copied as user settings and are not editable by an unprivileged user.
As for program updates, I like the global repository idea, but I’d also like things to be a bit distributed. How about the program bundle provide a URL that always resolves to the latest version of the program bundle? This could be a central repository URL, or a developer’s website.
check out nixos http://nixos.org/ it presents a purely functional software and configuration management system, very interesting and different
How about making a “configuration” the user interface object the user interacts with, rather than a “program”? So you add, run, and remove a particular configuration of a program, rather than the program itself.
e.g. I add a “configuration” of Firefox (“Firefox 3 + loads of plugins”) to my desktop. It’s like an AppDir, except it contains the settings, plus a link to the program.
If I want to add another one (e.g. “Firefox 2 with no plugins” for, say, on-line banking) then I add one of those too. I now have two icons / menu entries in my GUI, one for each configuration.
Deleting a configuration loses its settings, which therefore no longer need to be hidden.
Programs themselves can be installed and garbage collected automatically. No need to bother the user about that. Shared libraries, dependency hell, multiple versions, automatic updates, non-root install, etc are easily solved for most software (http://0install.net, as others have mentioned already).