Ever since I started using computers, I've been baffled by the relative clumsiness of installing applications. Whether we are talking the really old days (launching the Rambo game off a tape), the '90s (running Keen or using installers in Windows 95), or the modern days (still those installers, but now also package management and self-contained applications); it's all relatively cumbersome, and they all have their downsides. I decided to put my money where my mouth is, and come up with my idealistic, utopian method of installing, running, updating, and uninstalling applications.
Post a Comment
Member since:
2005-09-13
Fans: 2
Member since:2005-07-06
Fans: 5
Yeah, punch cards and paper tape baby! None of that high-tech magnetic media for me!
Edited 2008-05-05 21:14 UTC
Member since:2006-01-26
Fans: 24
Yeah, punch cards and paper tape baby! None of that high-tech magnetic media for me! "
Give him a break, he hasn't even been alive for 3 decades yet (IIRC)
Member since:2005-10-12
Fans: 0
No, the "really old days" were when we had to use toggle switches to enter the code to boot the computer. And, if my memory serves me correctly, computer tapes predate Pascal
Member since:
2006-06-24
Fans: 0
Member since:2008-05-05
Fans: 0
Member since:2005-06-29
Fans: 20
Oh? You do know what GoboLinux is, right?
Care to elaborate? That's four pages of detailed explanations and user scenarios you just read, and all you can counter it with is "not such a good idea"?
Member since:2005-07-06
Fans: 1
It doesn't matter how long it is. A good idea is a good idea in two lines and in ten pages. Just because it can be detailed for hours doesn't make any idea good. Note, with that I don't say this particular idea is no good.
The only thing is, it's hard to satisfy everybody's taste. E.g. I don't like the Mac-style of application installing, I prefer dpkg, and I don't like the folder names with compulsory capitals, and so on. That doesn't mean the concept itself is wrong.
Member since:2008-04-04
Fans: 0
'Note, with that I don't say this particular idea is no good. '
But you are missing the point of the above poster.
He critisized the guy that said Gobolinux is not a good idea, but that other guy did not give ANY REASON AT ALL.
Noone can act upon FUD 
And btw, this suggestion here is different from Gobolinux in a few key areas. I am not saying it is good or bad here, but it is obviously that there are a few people who neither read this, nor know about Gobolinux - or opt to simply not give any reasons at all why something is "good" or "bad".
It is pointless to debate about opinions without people giving any REASONS behind them.
Member since:2006-09-27
Fans: 0
It looks somewhat similar, but Gobolinux is an attempt to achieve modularity within the typical GNU/Linux paradigm, where there's no distinction between the base system and the applications, that is, any package can depend on other packages that may or may not be present, and it can fullfill the requirements of yet other packages.
On the other hand, Thom's proposal assumes a base system, which makes it more like PC-BSD or Mac OS X. Once you have a system/applications distinction, package management is almost trivial. But then all distros would have to agree on one base system, which would either kill the well-known configurability of GNU/Linux (as in being able to have a very small installation) or be such a small system that lots of duplicated libraries would coexist, among other problems. The LSB guys have been trying to do just that for years, and I don't know how much success the've had, but the fact is that third-party program packages are still distro-specific. TANSTAAFL :p
Member since:2008-04-04
Fans: 0
Member since:2006-09-27
Fans: 0
Well, I meant that the LSB project tries to stablish a core system (including specific libraries), so that third party application packages can just depend on lsb-core. For instance, take this section:
http://refspecs.linux-foundation.org/LSB_3.2.0/LSB-Core-generic/LSB...
"Packages shall have a dependency that indicates which LSB modules are required. [...] Packages shall not depend on other system-provided dependencies. They shall not depend on non-system-provided dependencies unless the package provider also makes available the LSB conforming packages needed to satisfy such dependencies. "
Member since:2007-08-22
Fans: 0
http://refspecs.linux-foundation.org/LSB_3.2.0/LSB-Core-generic/LSB...
"Packages shall have a dependency that indicates which LSB modules are required. [...] Packages shall not depend on other system-provided dependencies. They shall not depend on non-system-provided dependencies unless the package provider also makes available the LSB conforming packages needed to satisfy such dependencies. "
Yes, LSB tries to establish a core. And most of it is okay, but it still doesn't go quite far enough. Personally, I think they need to get RPM out of LSB and replace it with something like (though not necessarily) Autopackage. It would make the LSB a far better standard, and would really help towards Linux on the Desktop too.
Member since:2005-07-06
Fans: 0
Member since:2005-07-08
Fans: 34
No, it's an elaboration on MacOS X. The system bundle is presumed to provide a generous set of shared libraries and prolonged ABI stability -- no small feat.
As with MacOS, any library excluded from the stable system bundle must be statically linked into all dependent program bundles, each of which may contain a different outdated version of the library.
The main differences from MacOS are the semantic filesystem and the bundle repository, along with the minor hierarchy change. It wouldn't be difficult to prototype this system for OS X using Spotlight or perhaps Nepomuk.
I have one question for Thom: I assume the desktop environment would be part of the system bundle so that its libraries could be shared amongst program bundles. So where on the filesystem would the per-user settings for such system components live? For example, does my wallpaper belong in /Settings/butters/System or /System/Settings/butters?
Member since:
2007-11-25
Fans: 0
"Applications in Mac OS X are generally not easy to remove at all, because they leave a trail of files around outside of /Applications that normal users rarely encounter. Over the course of time, this can amount to quite the mess."
The only files that are lying around are:
- plist for preferences (like .dot files in ~/ in Linux)
- cache files
- sometimes a folder in "Application Support"
That's all. You can easily search for the name of the program with Spotlight/Finder to remove those items, though it is NOT necessary because the only downside of those files lying on your disk would be waste MB... you have to run thousands of apps and deleting them to call it a mess.
"In addition, Mac OS X provides no way of updating applications in a central way, resulting in each application in OS X having its own updater application; hardly the user-friendly and consistent image Apple tries to adhere to."
If an application is started it will inform you about a new version, quite simple. Most of these dialogs are pretty consistent to each other too. And there are applications or even widgets that can check all your applicatins for updates, though it isnt really necessary, because.. when u run it it gets updated / will inform you. It would be nice to see Apple implementing this mechanism that those update-applications use into their own software-updater. But as said.. not really necessary.
Member since:2006-01-14
Fans: 1
The big problem with the Mac is that the apps arent consistent. Not all auto update themselves. Just like in Linux some software has to be compiled. Still, it would be nice if the world could agree on one single way to notify and put out updates. RSS maybe? In a corporate setting, repos work well since its pretty easy to make packages of something that needed to be compiled. But for single users adding a lot of repos leads to package poisoning.
Member since:
2006-03-03
Fans: 0
I so wish I had time to read this Thom. Like many of us, package management is the first and foremost reason we "advanced" Linux users will choose a distribution. Well, actually since I discovered Debian I've never looked back except for the occasional brush with Yum... and thinking that Red Hat has had APT as the example for YEARS, and Yum is the crap they came up with?!?!?
Anyway, care to make it a podcast or something that I can put on and tune in and out while I'm coding? Mostly kidding, that's a lot of work... would be more efficient in the end if I just read it 
Anyway, nice article, love the exclusives.
Edited 2008-05-05 22:38 UTC
Member since:2008-04-04
Fans: 0
'Like many of us, package management is the first and foremost reason we "advanced" Linux users will choose a distribution.'
Not me. I guess I am not alone though. Personally, I simply refuse to use solutions that do not give me the _OPTION_ of having something like AppDirs.
AppDirs are simply a better way to manage applications than FHS, period. If I want to have something removed, i go kill the dir. I dont _want_ to DEPEND on a package manager in order to uninstall something again.
Member since:
2008-02-26
Fans: 0
Member since:
2006-12-13
Fans: 2
Hi Thom,
One of my many pet projects for Haiku has been a new "package format" for applications as well as a consistent updating mechanism and some sort of central application repository.
This might make a good initial design, since you make use of the live queries that we will have in Haiku. Overall it sounds pretty good.
I think the permissions system you describe could also be implemented.
But I think the file system layout will need to remain in the standard BeOS format we are inheriting. But I still think much of your design could still be used for that. For one thing I don't see why you need a separate /Settings hierarchy when you could just have /Users/User 1/Settings or in Haiku /boot/home/user/config. Of course the multi-user aspects of Haiku are still in flux and probably won't be sorted out until R2.
Member since:2005-06-29
Fans: 20
You're right Ryan, this whole thing actually started out as a discussion in #haiku a long time ago. I was just musing aloud at how you could use the attributes and live queries in BFS to manage applications, and from there, this whole idea started to grow.
So it makes sense that my ideas fit Haiku so well.
The reason I chose for a separate hierarchy is because I want the /User/User 1 directory to be strictly a directory for the user's documents, movies, photos, pr0n, and so on. I'm someone with a strong inclination towards order and cleanliness, so you can imagine why I'd like to not put settings files into the home directory.
And thanks for the compliments
. I've spent a lot of time on this proposal, and I believe I'm only scratching the surface of what attributes+live queries+program bundles can equate to. If you want to discuss this in more detail for whenever the package management plans for R1+1 come up, feel free to contact me, I'd love to participate in that discussion .
Member since:2005-07-06
Fans: 1
I don't think that separating the user and its setup configuration files is a good idea: this means that when you want to backup your account, you need to save two directories not one..
As for the cleaness point of view as long as all the configuration files are in the setting directory, I don't think that this is dirty.
That said, I've been convinced recently that purely hierarchical file is impossible to get 'right' so if the user documents and his configuration file are tagged by his login name (automatically) then the backup of the user's data becomes far more easy..
Member since:2005-11-10
Fans: 14
Member since:
2006-07-30
Fans: 2
Locking would still be necessary because while queries are fast, installing is not.
Let's say you use one query to install program P and start another query and uninstall library L (something P depends on) along with all older programs that need L.
There are two ways out of this:
a) Using a more fine grained locking along the lines of "install anything you like but make sure not to destroy L since P (currently being installed) will need it". What, however, if you said install P from vendor V and while it was installing told the computer to remove all programs of vendor V. I'm almost sure no matter how smart the algorithm is there'll still be situations where it has to say "Encountered Conflict C, do you want to do x or y?"
b) Making every program completely self contained.
This would have HUGE security implications.
A defect was found in a library that 100 of your applications use? Well, that's to bad, you have to reinstall them all.
You can of course draw the line at some arbitrary point and say this lib is used 'a lot', so it's shared.
In short, while your idea sounds good I believe it has tons of details, corner cases, and trade offs that still need to be sorted out.
Furthermore, it can only be realized embedded in a bigger ecosystem. To revisit the example with a defect in a lib, developers could tag their applications like "works with version x of this library" in a way that the program could automatically tell you "vulnerability found, switching to fixed version of lib L" or "vulnerability found, program not yet tested with safe version of L, do you want to upgrade and risk a crash or keep running this unsafe program".
In addition, your system is still centralized.
There needs to be someone or some server to say "No, malwareGuy, you cannot call this 'Paint 6', there's already 'Paint 5' and you didn't write it".
Somebody has to decide what goes on the server and what doesn't.
I could go on and on but I think you get the point:
The idea is good but the devil is in the details.
Member since:2005-06-29
Fans: 20
The server?
Who says people can't set up their own server for distributing program bundles? It wouldn't be too hard, as long as the file system on the server preserves the attributes.
I also envision a set of command line tools that allow you to compare/update program bundles. Something like:
$ compare "/Programs/Garden Designer.bundle" "ftp.stuff.org/pub/bundles/Garden Designer.bundle"
Output:
$ /Programs/Garden Designer.bundle:438
$ ftp.stuff.org/pub/bundles/Garden Designer.bundle:439
Member since:2006-07-30
Fans: 2
Member since:2005-06-29
Fans: 20
How do you make sure that no two programs can have the same identifier if everybody can set up a server and name/tag the apps as he pleases?
They're not identifiers, they're values stored as an attribute. Even if you have ten billion million attributes with value 345, if they belong to different files, that simply doesn't matter.
Member since:2006-07-30
Fans: 2
Maybe I just misunderstand what you say but aren't all these attributes together sort of an identifier?
So you could have two files with attributes
program=paint
vendor=ms
version=4.1
patch_level=127
...
and if they contained different binaries it would be a big problem. You could, of course, add mechanisms like checksumming the binaries and so on.
But then again you'd need a central server/group of servers that tell the user that a program with attributes x and y should have checksum z.
Am I missing something?
Member since:
2005-07-06
Fans: 11
Nice try, Thom - if you're just reading it casually you almost miss the hand-waving about binaries that don't belong in any particular 'program bundle', and about the issue of shared libraries, which is of course the big drawback of the OS X system that you *don't* mention (because it persists in your vision). If you use shared libraries, you have a reliance on your vendor (situation with all current Linux distributions). If you don't, you have security issues and ancient bugs that were fixed long ago cropping up all over the place (situation with Windows and OS X).
Member since:2005-07-06
Fans: 8
GNU/Linux as no reliance on vendor for anything.
The linux kernel as many version and modification of itself. So do the library , so do the x systems , so do the windows environment , etc ...
SLS ... Debian ... Ubuntu ...
SLS ... Slackware ... SLAX
Red Hat ... Mandriva ... PcLinuxOS
Xfree , X.org
KDE , Gnome , Xfce
ETC ...
You got acces to source code , it's Open Source developed , and it's Free software.
You can fix it yourself , train someone to fix it or hire someone else to fix it for you.
Member since:2005-06-29
Fans: 20
Miss? It's right there in the article:
Member since:2005-07-06
Fans: 11
Member since:2008-04-04
Fans: 0
'If you use shared libraries, you have a reliance on your vendor (situation with all current Linux distributions).'
That is true, and I am one that constantly critisizes the upstream vendors/maintainers as well, but you see - the situation on Mac is as bad as on Windows in that you are dependent on a company just as well.
So all these worlds more or less have similar problems. You depend on someone else.
Now, with similar problems already, I as a user would still like to choose AppDirs instead of FHS.
I dont want that others enforce the FHS upon me. But the big distributions have no inclination to change to AppDirs at all.
Member since:
2006-11-01
Fans: 0
We should also have a /Users/Common directory for documents (and a /Settings/Common for settings) that all users on the system need access to. Of course the system needs to support user-controlled file permissions so User 1 can grant write privaleges to User 2 to a directory he owns but not allow User 3 to read it without being a System user and needing create a group.
What about file associations? I know this could be handled by the window manager but I'd like some central database of what applications are associated with what files, even on a user by user basis. On Computer 1 Firefox should be the default browser for all users but Computer 2 User 1 wants to use Firefox while User 2 likes Konqueror.
This new program management is the best idea I have ever heard for managing programs!
Member since:
2005-04-01
Fans: 16
Member since:2005-06-30
Fans: 0
Real separability. For example, many people who test distros will want to share /home, but later find themselves with overlapping Settings that do not work with the other distros.
These days I maintain my files in /home/xiaokj/Link and make the link to some central repository (another partition). It would be nice if I need not do that, but rather link specific apps' settings (like IM/Email that is really supposed to be persistant) instead of making sure the dozen other settings that should be separate (like Xauthority, KDE/Gnome settings...)
Member since:2006-01-02
Fans: 0
Member since:
2005-04-01
Fans: 16
If each program has an "internal" version number, then perhaps the corresponding /Settings/User/X directory be the version number instead of the program name? So if Garden Designer is v846, then the file is /Settings/User1/846. This way, you'd really be able to run parallel versions of apps.
But there's another problem - there might be another program that is internally 846. We need a unique number. Wait, that already exists!; enter the GUID.
My point is, there are some really cool ideas in here, but ultimately, I think it needs a lot of refinement. Storing the settings in a separate space just means you've improperly used the home directory in the first place (see my above comment). Your permissions *will* be screwy when you put settings in one top level directory and user data in another. This is why there are hidden directories. Also, your arbitrary requirement of the settings directory sharing the name of the software package will need some sort of low-level monitor. It also means that when I release a new version of a package and choose to rename it with a version number, I can't reliably find your user settings from any of the last several versions of my software, I have to try every possible combo or not let you migrate your settings from any version but the last one.
Frankly, I think the current OS X way is near perfect, with plenty of room for small improvements. I don't think that, with disk space as it is, the system is "cluttered" by having lots of settings file - stored properly where settings should be stored! The idea is that it's persistent, it's there the next time you install the app. And I think Leopard's Spotlight is plenty fast enough for 99% of users. Live queries are very cool, but very few people would use them in a mainstream OS, and existing technologies provide most of the end result.
Member since:2005-06-29
Fans: 20
But there's another problem - there might be another program that is internally 846. We need a unique number. Wait, that already exists!; enter the GUID.
And how user friendly is that? The hierarchy I devised is supposed to be 'human readable' - actually, that was one of its primary goals. File systems have been a mess since day one, and it simply needs to be fixed. OS X made tremendous strides in that regard, but in the end it's just a virtual directory structure draped over a traditional UNIX/POSIX structure.
Hidden directories are evil. If a system needs to be secretive in order to not confuse a user, there's a design error somewhere.
I addressed that issue with a typical utopian statement in the article:
Member since:2005-06-30
Fans: 0
Member since:
2006-01-10
Fans: 0
I think most of us have thought about such ideas in the past. The devil really is in the details as someone said above.
I tend to like the idea of 'application bundles' but then how do you handle shared libraries. You will end up doing the weird windows DLL handling.
For example you could have a '.lib' file in each program directory that tells the loader which shared libraries to use for the application. The application bundle would include a version of the library that is 'known' to work.
shared libraries not belonging to a particular application can also be installed a generation /system/libs directory.
If a newer version of a '.lib' is available, then somehow the system must decide if it wants to try that version instead of the version in the application bundle. This 'somehow' is undefined. Perhaps the system detects a new version, asks you if you want to try the upgraded version... and goes back if it fails. Maybe we leave it to some online repo... This somehow could get complicated fast.
--------
in terms of updating the system. I don't really know why you're complicating it with all this search and queries... Wouldn't a simple file with the application bundle, pointing to some server location do. The system can check if there is an updated version and if so, prompt for you to download/install it.
Member since:
2008-05-03
Fans: 0
One of the most frustrating thing when you live on a linux where you don't have root access is to install programs. you basically need to go the "./configure --prefix=$HOME && make && make install" and do it yourself dependencies.
Not all programs require root privileges. Then, why not allow users to do "apt-get install whatever" and it goes to their directory until the program is superseded by a system wide installation?
Going along this idea would be a first step towards better program management.
Member since:2006-02-15
Fans: 6
Not all programs require root privileges. Then, why not allow users to do "apt-get install whatever" and it goes to their directory until the program is superseded by a system wide installation?
I have wondered about the same thing myself. But I guess it all boils down to the fact that people expect only two kinds of users to install software: system admins or users who own the computer. The truth is however that there are also people who don't have the root password but might still wish to install something additional. Such users could f.ex. be your children.
So yeah, I basically like your idea. All the files and folders should however go under a single folder in the users' home, like f.ex. /home/user/Applications and of course everything should be owned by the user and writable only by that user. Oh, and yes, the system admin should still be allowed to choose which users are allowed to install software this way. In corporate environments for example it's often preferred that users are not allowed to install anything but instead ask the admin for that.
Member since:2006-01-11
Fans: 0
Member since:2008-05-03
Fans: 0
I wish I was a kid ;-)
The "ask the admin" approach usually takes some time and gives the admins a bad reputation of being not very responsive.
If not in an environment where installing an application may compromise security and should be prohibited, there is a clear benefit in letting the user do basic administration tasks (installing stuff). Debian-style package management makes this possible because installing something is designed to be a zero hassle operation. However, the neat thing about being superuser to install stuff is that it keeps the applications clean and working. A similar approach should be used by not giving the user actual write rights in his app folder, but using a setuid mechanism so that only the package manager can do that.
Member since:2005-07-11
Fans: 0
If programs are organized as AppDirs, and they are not allowed to rely on being installed in a specific location in the file system hierarchy, this may be achieved simply by unzipping the app wherever you prefer. No need for installers at all.
The trick with Thom's attributes idea, is that you can still query installed applications even if no installer was used to put them there.
Unix security has a very simple way to do this. Just mount the home partition as noexec. There simply is no way you can stop the user from putting a binary in his home directory, but noexec makes sure that no user installed files can be executed.
Member since:
2006-01-29
Fans: 0
I enjoyed reading the article very much and it seems like a good base for building a new packaging scheme. However, there are still some areas that need further work. Some assumptions of the proposed system were also not clear.
The first one is the operating system, is this meant for Windows, OS X (OS XI?), Linux, some other OS, or an entirely new system? For "legacy" systems, the biggest issue of all seems to be providing a migration path from existing software deployment methods to the new method.
The handling of shared libraries, and the necessity of separating user settings from user homes have been mentioned by others. Also, the write-up talks about a single host, but in practice, program and user directories are often shared among several hosts. It is not obvious that the system would work well in such scenarios. More detailed consideration may be needed about what is shared among which hosts and users. Depending on the OS environment, support for multiple architectures may need to be considered as well.
One of the biggest issues is the storing of settings. Supporting multiple versions of a program in parallel, sharing the same set of settings, is a pain in the ass for software vendors. I think that a lot of conflicts would arise from trying to run different versions of Word, Photoshop, or even GNOME2 with the same set of settings.
So if different versions have to share the same set of settings, many software will likely refuse to install without having prior versions removed first. In such case, if you have other software packages that depend on this kind of a package, you're again stuck with dependency hell.
Furthermore, as mentioned in the article, sometimes the user or administrator does not trust a new version of a program. So even when the software supports shared settings, the user may not want to allow it to modify the settings used for the "trusted" version.
One solution may be to store settings for different versions separately, but have the installer generate the new settings automatically based on those for previous version.
Finally, the proposed system also does not seem to support installing programs without system privileges, which is normally possible for most Linux / OSX software and an increasing share of Windows software.
Good luck for working on this new system, and looking forward to hearing about the progress.