OSNews

//Whether we are talking the really old days (launching the Rambo game off a tape)// er ... i thought the "really old days" was when I typed three dozens lines of Pascal into a PDP-11 in order to get a simple tax calculation to run ...tapes? You're high-tech!

Isn't this just a less elegant version of GoboLinux with added live (and I repeat -- they really are live) queries?

"Applications in Mac OS X are generally not easy to remove at all, because they leave a trail of files around outside of /Applications that normal users rarely encounter. Over the course of time, this can amount to quite the mess."

The only files that are lying around are:
- plist for preferences (like .dot files in ~/ in Linux)
- cache files
- sometimes a folder in "Application Support"
That's all. You can easily search for the name of the program with Spotlight/Finder to remove those items, though it is NOT necessary because the only downside of those files lying on your disk would be waste MB... you have to run thousands of apps and deleting them to call it a mess.


"In addition, Mac OS X provides no way of updating applications in a central way, resulting in each application in OS X having its own updater application; hardly the user-friendly and consistent image Apple tries to adhere to."

If an application is started it will inform you about a new version, quite simple. Most of these dialogs are pretty consistent to each other too. And there are applications or even widgets that can check all your applicatins for updates, though it isnt really necessary, because.. when u run it it gets updated / will inform you. It would be nice to see Apple implementing this mechanism that those update-applications use into their own software-updater. But as said.. not really necessary.

I so wish I had time to read this Thom. Like many of us, package management is the first and foremost reason we "advanced" Linux users will choose a distribution. Well, actually since I discovered Debian I've never looked back except for the occasional brush with Yum... and thinking that Red Hat has had APT as the example for YEARS, and Yum is the crap they came up with?!?!?

Anyway, care to make it a podcast or something that I can put on and tune in and out while I'm coding? Mostly kidding, that's a lot of work... would be more efficient in the end if I just read it Anyway, nice article, love the exclusives.

Edited 2008-05-05 22:38 UTC

If you want to have different versions of the same programs then use Gentoo. Portage has a nice feature called SLOTs.

Hi Thom,

One of my many pet projects for Haiku has been a new "package format" for applications as well as a consistent updating mechanism and some sort of central application repository.

This might make a good initial design, since you make use of the live queries that we will have in Haiku. Overall it sounds pretty good.

I think the permissions system you describe could also be implemented.

But I think the file system layout will need to remain in the standard BeOS format we are inheriting. But I still think much of your design could still be used for that. For one thing I don't see why you need a separate /Settings hierarchy when you could just have /Users/User 1/Settings or in Haiku /boot/home/user/config. Of course the multi-user aspects of Haiku are still in flux and probably won't be sorted out until R2.

Locking would still be necessary because while queries are fast, installing is not.
Let's say you use one query to install program P and start another query and uninstall library L (something P depends on) along with all older programs that need L.

There are two ways out of this:
a) Using a more fine grained locking along the lines of "install anything you like but make sure not to destroy L since P (currently being installed) will need it". What, however, if you said install P from vendor V and while it was installing told the computer to remove all programs of vendor V. I'm almost sure no matter how smart the algorithm is there'll still be situations where it has to say "Encountered Conflict C, do you want to do x or y?"

b) Making every program completely self contained.
This would have HUGE security implications.
A defect was found in a library that 100 of your applications use? Well, that's to bad, you have to reinstall them all.

You can of course draw the line at some arbitrary point and say this lib is used 'a lot', so it's shared.

In short, while your idea sounds good I believe it has tons of details, corner cases, and trade offs that still need to be sorted out.
Furthermore, it can only be realized embedded in a bigger ecosystem. To revisit the example with a defect in a lib, developers could tag their applications like "works with version x of this library" in a way that the program could automatically tell you "vulnerability found, switching to fixed version of lib L" or "vulnerability found, program not yet tested with safe version of L, do you want to upgrade and risk a crash or keep running this unsafe program".

In addition, your system is still centralized.
There needs to be someone or some server to say "No, malwareGuy, you cannot call this 'Paint 6', there's already 'Paint 5' and you didn't write it".

Somebody has to decide what goes on the server and what doesn't.

I could go on and on but I think you get the point:
The idea is good but the devil is in the details.

Nice try, Thom - if you're just reading it casually you almost miss the hand-waving about binaries that don't belong in any particular 'program bundle', and about the issue of shared libraries, which is of course the big drawback of the OS X system that you *don't* mention (because it persists in your vision). If you use shared libraries, you have a reliance on your vendor (situation with all current Linux distributions). If you don't, you have security issues and ancient bugs that were fixed long ago cropping up all over the place (situation with Windows and OS X).

We should also have a /Users/Common directory for documents (and a /Settings/Common for settings) that all users on the system need access to. Of course the system needs to support user-controlled file permissions so User 1 can grant write privaleges to User 2 to a directory he owns but not allow User 3 to read it without being a System user and needing create a group.

What about file associations? I know this could be handled by the window manager but I'd like some central database of what applications are associated with what files, even on a user by user basis. On Computer 1 Firefox should be the default browser for all users but Computer 2 User 1 wants to use Firefox while User 2 likes Konqueror.

This new program management is the best idea I have ever heard for managing programs!

Why store user info in /Users/User1/ and settings in /Settings/User1?

Why wouldn't you use /Users/User1/settings for settings? You want the home directory to be less portable? Less backup-able?

If each program has an "internal" version number, then perhaps the corresponding /Settings/User/X directory be the version number instead of the program name? So if Garden Designer is v846, then the file is /Settings/User1/846. This way, you'd really be able to run parallel versions of apps.

But there's another problem - there might be another program that is internally 846. We need a unique number. Wait, that already exists!; enter the GUID.

My point is, there are some really cool ideas in here, but ultimately, I think it needs a lot of refinement. Storing the settings in a separate space just means you've improperly used the home directory in the first place (see my above comment). Your permissions *will* be screwy when you put settings in one top level directory and user data in another. This is why there are hidden directories. Also, your arbitrary requirement of the settings directory sharing the name of the software package will need some sort of low-level monitor. It also means that when I release a new version of a package and choose to rename it with a version number, I can't reliably find your user settings from any of the last several versions of my software, I have to try every possible combo or not let you migrate your settings from any version but the last one.

Frankly, I think the current OS X way is near perfect, with plenty of room for small improvements. I don't think that, with disk space as it is, the system is "cluttered" by having lots of settings file - stored properly where settings should be stored! The idea is that it's persistent, it's there the next time you install the app. And I think Leopard's Spotlight is plenty fast enough for 99% of users. Live queries are very cool, but very few people would use them in a mainstream OS, and existing technologies provide most of the end result.

I think most of us have thought about such ideas in the past. The devil really is in the details as someone said above.

I tend to like the idea of 'application bundles' but then how do you handle shared libraries. You will end up doing the weird windows DLL handling.
For example you could have a '.lib' file in each program directory that tells the loader which shared libraries to use for the application. The application bundle would include a version of the library that is 'known' to work.

shared libraries not belonging to a particular application can also be installed a generation /system/libs directory.

If a newer version of a '.lib' is available, then somehow the system must decide if it wants to try that version instead of the version in the application bundle. This 'somehow' is undefined. Perhaps the system detects a new version, asks you if you want to try the upgraded version... and goes back if it fails. Maybe we leave it to some online repo... This somehow could get complicated fast.


--------
in terms of updating the system. I don't really know why you're complicating it with all this search and queries... Wouldn't a simple file with the application bundle, pointing to some server location do. The system can check if there is an updated version and if so, prompt for you to download/install it.

One of the most frustrating thing when you live on a linux where you don't have root access is to install programs. you basically need to go the "./configure --prefix=$HOME && make && make install" and do it yourself dependencies.

Not all programs require root privileges. Then, why not allow users to do "apt-get install whatever" and it goes to their directory until the program is superseded by a system wide installation?

Going along this idea would be a first step towards better program management.

I enjoyed reading the article very much and it seems like a good base for building a new packaging scheme. However, there are still some areas that need further work. Some assumptions of the proposed system were also not clear.

The first one is the operating system, is this meant for Windows, OS X (OS XI?), Linux, some other OS, or an entirely new system? For "legacy" systems, the biggest issue of all seems to be providing a migration path from existing software deployment methods to the new method.

The handling of shared libraries, and the necessity of separating user settings from user homes have been mentioned by others. Also, the write-up talks about a single host, but in practice, program and user directories are often shared among several hosts. It is not obvious that the system would work well in such scenarios. More detailed consideration may be needed about what is shared among which hosts and users. Depending on the OS environment, support for multiple architectures may need to be considered as well.

One of the biggest issues is the storing of settings. Supporting multiple versions of a program in parallel, sharing the same set of settings, is a pain in the ass for software vendors. I think that a lot of conflicts would arise from trying to run different versions of Word, Photoshop, or even GNOME2 with the same set of settings.
So if different versions have to share the same set of settings, many software will likely refuse to install without having prior versions removed first. In such case, if you have other software packages that depend on this kind of a package, you're again stuck with dependency hell.

Furthermore, as mentioned in the article, sometimes the user or administrator does not trust a new version of a program. So even when the software supports shared settings, the user may not want to allow it to modify the settings used for the "trusted" version.

One solution may be to store settings for different versions separately, but have the installer generate the new settings automatically based on those for previous version.

Finally, the proposed system also does not seem to support installing programs without system privileges, which is normally possible for most Linux / OSX software and an increasing share of Windows software.

Good luck for working on this new system, and looking forward to hearing about the progress.

but in the end it seems like it would basically be OSX + a centralized system, which isn't exactly revolutionary. The whole thing with BFS live queries could be handled exactly the same with the centralized system using a database, you're just moving the relation management from the db into the native filesystem.

...but my personal wish is for applications always to be fully self contained with any third party stuff they need contained within the program directory. Sure this would lead to bloat on disk size, but whats taking up more disk space on your hard drive? Those 10Gb of applications or the 100's Gb of MP3s/AVIs?

Im a big game player and i like games which have save games IN the game directory (eg: under \save or \save\profile - for multiple profiles). Saving under My Documents a la Microsofts recommedations gets me really annoyed.

And dont even get me started on saving game/configuration information in the registry. The registry is a bad idea from the start. Whats wrong with .CFG text files?

While i accept the need to consider multi-user systems and privledges im sure self-contained apps could somehow work with this.

Anyway, nice article, even if some ideas need refinement.

I wish Ubuntu would use a system like you described in this article. It's much more 'human'. I really have no idea where Ubuntu stores all it's program data, and usually config files and log files etc are all stored in different places, for each different program.

It's too confusing, and when i want to do something myself, instead of using Synaptic, i usually end up confused..