The desktop metaphor, used in practically every modern operating system, has been around for many years and has been very successful in making computer usage easy for even the most novice users. Whereas once a user had to type commands in and navigate through directories by entering every single character in a path name, the desktop graphical user interface metaphor made it possible to perform the same actions by simply pointing to and clicking the iconic views of the various files and folders.Despite the fact that this was a huge improvement and obviously an impressive and visually appealing solution in its time, most experienced users (and developers) currently think that there should be more productive ways to store information in and retrieve it from the computer.
As an example, let us see how unproductive a simple every-day task can be. Suppose we simply want to “print the image that Helen sent me a few days ago”. We would have to open the e-mail application, sort the messages by date to find the relevant message, save the attached image (very distracting, especially since we also have to manually select a location for the image to be saved), activate the file manager (or a photo editing application) navigate to the location where we saved the file (quite distracting too) and then open and print the file. If the attachment we are looking for is older, we may have to perform searches in the e-mail program and maybe also through the file system, if we think we have saved the attachment some time ago (all the time trying to remember the file name of the image). Now this procedure is a serious distraction from what we originally wanted to DO with the file!
No more folders – Enter categories
Better ways of locating our files are being designed. The trend now is to use file meta-data (information about the files and their usage) so that a user may be able to locate the files by attributes, such as the title of a music track, or by statistics of their usage. With the use of meta-data, instead of having to save the files on a single location, a user is free to assign a file to one or more categories. For example a picture may be simultaneously a member of the “Family Photos” the and the “Received by e-mail” categories. (This at first seems like having the same file in two folders, but the possibilities are more complex than with a hierarchy of folders, and cannot be simulated in current file systems except by using links, which would create a very complex hierarchy). The file manager would provide “category” views, similar to the folder views of current systems, but with the added advantage of being able to present the union or the junction of multiple categories of files at once.
With the support of a database, to store and index the extra (meta-data) information, one would be able to perform faster and more focused searches through the file system, even by using plain natural language commands. However, in a world where everyone is so accustomed to “pointing and clicking” the idea to search for a file by typing anything at all (even natural language commands) does not seem too attractive. Why not let the system use the meta-data in order to present more useful views of our files?
Let us see how we think for our files for a moment. What we know about them is scarcely the file’s name or the folder where we (once) put it. We mainly remember WHEN we used a file for the last time, and HOW this file appeared for the first time in our system. We know for instance that a particular file came as an e-mail attachment, or was downloaded from a web site, or copied from a CD or transferred from a digital camera; and we tend to also remember WHEN those operations happened. The computer should know too, by use of meta-data, and should be able to present the files in easily accessible views: i.e. views navigable by pointing and clicking.
Chronological file management
Imagine the file manager presenting you with a “Calendar View” of your files. The interface would resemble a lot the calendar views of Evolution or MS Outlook, for instance. Only, in this “calendar”, by clicking on a particular date, you would not see only the tasks and appointments of the day, but a view of all the files (documents, e-mails, downloaded files) that you processed that day. Similar views for weeks, months or even years are possible, but further categorisation and sorting of the files may be required for such long periods of time.
This approach could be viewed as the integration of many, currently separate applications such as PIM/calendar, e-mail client, web browser, download manager and file manager. However, behind the scenes, it is the specially added meta-data of the files that will make possible accessing them from originally separate applications. And this meta-data in most cases can be added automatically by the receiving applications (e.g. the e-mail client saves automatically the e-mail attachments to the disk and adds them to the “Received by e-mail” category).
Source oriented file management
The files in the calendar view may be categorized and grouped together with the usual methods (i.e. by file type or sorted by name), by manually assigning special category information to them or, even more importantly, automatically by the system according to their originating source. The e-mail attachments grouped together, further categorised by sender. The sites visited forming another group. The downloaded files in another group, possibly further categorised by site. The pictures from your digital camera in another group. The files copied from the network (or from removable media) in other groups and so on.
If we wanted to “print the image that Helen sent me a few days ago” in this system, all we had to do would be:
This approach also implies that there will be no need to manually save the files before using them. The files are automatically saved to appropriate positions and categorized, based on default rules, as they arrive. The user is then free to assign more specialised category information to the files if he/she thinks a file should be easily found by navigating specific category views.1. Select in the calendar the previous week.
2. Possibly scroll to view the group of files that were received as e-mail attachments. The image thumbnail would become visible.
3. Select and print the image
Jukebox searches
The use of file meta-data can also speed-up searches on off-line files -and these are not just the files on a temporarily unavailable network share.
Consider for instance the files stored on a CD. As soon as the CD is out of the drive, the computer has no clue about the files that were on the CD. Everyone knows how time consuming is to search through a CD collection inserting each CD in the drive. Instead, the system could keep a copy of the meta-data of CD files for each CD inserted, and be able to perform future searches through the collection off-line.
No trash can
Why delete files (and do it manually) from a 80GB hard disk when most of its space is free? The recycle bin is a rather outdated idea in a “garbage collecting” world. Files should never get deleted, either intentionally or by chance. They should be rated, according to their importance, so that the system will be able to propose appropriate actions when the disk is becoming full.
Files marked as important should probably always stay in the system and be regularly backed-up. Files used frequently should be marked automatically as important files. Files bearing no particular rating could be archived by default, and then removed from the system. Temporary files should be simply deleted.
There could be a rating equivalent to “garbage”, for items such as spam mail which should be really deleted, but this could actually be viewed as another form of the “Trash” folder found on current systems.
About the author
Athanassios Floros is an Electrical & Computer Engineer working in the Software Engineering sector since 1997. He is currently employed in one of the largest Communications Equipment and IT companies in Greece.
If you would like to see your thoughts or experiences with technology published, please consider writing an article for OSNews.
My way of dealing with the problem of Outlook folders being limited to storing email is with attachments. If I have a pdf relating to a project I’m working on I usually just email it to myself as an attachment with a good description. That way I don’t have to go find a folder somewhere with my non-email data. Pardon my lack of eloquence, earth shattering hangover…
‘data visualization’. And the calendar view is not the most expedient to use for it. Hyperbolic graphs, tree views, and others are more often used. A calendar view just doesn’t scale to hundreds of thousands of items, and no a search function doesn’t help it.
A good way to visualize categorized data over time is via a customized gantt chart, where Y becomes your categories (with drilldowns to subcategories, or month/year view) and X is time. You can size the chart items based on time spent using pieces of data. Thus by scrolling the X axis it is easiy to locate items you may have spent time working on in a category.
Ofcourse the proof is in the pudding and we implemented this for a movie/game/file satellite delivery system with literally thousands of pieces of content delivered each week with great success – i.e. one operator had no problems dealing with and interpreting this amount of data onscreen.
My 2c.
Not every computer out there has a 80 gig drive yet, especially portable machines. Even with larger disk drives, people find ways to fill them up (high quality video, flac audio 5.1, . While I’m all for the rating system that you propose, removing a way to manually delete files from the gui would be a bad move.
I’m really all for a category based filesystem and file manager. That would really help out on Unix type systems. No more having to worry about looking through several directories to find a program. All the ‘directories’ could be generated automatically. I’m not sure you could completely eliminate directories as actual objects that the user would interact with.
Say I create a directory called “Vacation” into this directory I place various seemingly unrelated things. Photos, Saved Webpages, Email, Letters, and other files. By dropping them into the directory, their metadata gets updated that they’re related to Vacations. Otherwise I would have to edit each files metadata individually. While the directory may not be directory in the traditional sense, it would still to the user behave exactly like one.
Isn’t this part of the revised filsystem model in Longhorn?
– Simon
Which is just logical vs. physical layout. And it is usually advantageous to have both. Physically you may want to drop all your new items in a ‘new photos’ directory. Logically you may want to separate them based on vacation and time, via their metadata tags. As the directory becomes unmanageable you may want to use the metadata tags to generate subdirectories. Whats the use here you say? For one some filesystems don’t deal well with thousands of files in one directory, and for other if you decide to take some of those photos and burn a cd for someone, they may not have the same logical view you do of the data, so you cater to the lowest common denominator and give them a cd with vacations split by category. Ofcourse this is one example and doesn’t prove the rule.
Isn’t this part of the revised filsystem model in Longhorn?
Sort of, yes. The difference being that Longhorn utilizes a special folder (“defaultstore” it’s called right now, as far as I can recall) where you have to drop the files in manually in order to use the metadata stuff. Then you can stack files, and organize them in much te same way Athanassios Floros proposes.
So it’s not the entire filesystem which is dropped.
Correct me if I’m wrong, though.
“As an example, let us see how unproductive a simple every-day task can be. Suppose we simply want to “print the image that Helen sent me a few days ago”. We would have to open the e-mail application, sort the messages by date to find the relevant message, save the attached image (very distracting, especially since we also have to manually select a location for the image to be saved), activate the file manager (or a photo editing application) navigate to the location where we saved the file (quite distracting too) and then open and print the file. ”
It isn’t that complicated.
The mails in each folder are already in chronological order (I have them set to “Most Recent First”). It is quite easy to spot a mail from a particular person. Double-click the mail to display it. Then click the Display button at the top of the window; this gives a list of the images attached, if any. Select the one you want. It comes up in a window. Select Print from the window’s menu.
For files other than emails, I always use a 2-column manager – Total Commander on Windows or DOpus4 on Amiga. These can be set at any moment to sort on various criteria, such as time, file size, alphabetical. Fast and efficient.
There are indeed two problems:
1. many files fit in several categories
2. the biggest category is “miscellaneous”.
<quote>All the ‘directories’ could be generated automatically. I’m not sure you could completely eliminate directories as actual objects that the user would interact with.
Say I create a directory called “Vacation” into this directory I place various seemingly unrelated things</quote>
That would be vdirectory, since the vdirectory SeasInMyEyes would also contain the pacific028.jpeg picture.
<quote>Isn’t this part of the revised filsystem model in Longhorn? </quote>
U mean gnome-storage?
“Consider for instance the files stored on a CD. As soon as the CD is out of the drive, the computer has no clue about the files that were on the CD. Everyone knows how time consuming is to search through a CD collection inserting each CD in the drive. Instead, the system could keep a copy of the meta-data of CD files for each CD inserted, and be able to perform future searches through the collection off-line.”
That would be very easy to do with the List command in AmigaOS. You can specify what data you want listed about each file. The end result would be a folder (or drawer) containing text files each of which is a full listing of all the files on a named CD. The Search command will find any string in that directory.
To make the listings, work out the exact command, list one CD, put in the next and hit the Up arrow in the Shell to repat the command. Hit return. Next disk. And so on.
I’m sure there are similar commands in Unix.
Everyone knows how time consuming is to search through a CD collection inserting each CD in the drive. Instead, the system could keep a copy of the meta-data of CD files for each CD inserted, and be able to perform future searches through the collection off-line.
No trash can
Why delete files (and do it manually) from a 80GB hard disk when most of its space is free?
Gee, it’s no wonder the author still has space on an 80GB drive, he’s still swapping CDs instead of ripping them and managing his collection on the hard disk. Sure, there are a handful of good uses for storing CD indexes, but that will, at best, only tell you which CD you have to search for (as opposed to swapping discs in and out of the drive until you find the data), rather than eliminate the need to search through a lot of discs.
The recycle bin is a rather outdated idea in a “garbage collecting” world. Files should never get deleted, either intentionally or by chance. They should be rated, according to their importance, so that the system will be able to propose appropriate actions when the disk is becoming full.
This is one of the most irritating ideas I’ve come across so far. How does the system know the importance of the data? Why is this rating more important than my desire to delete a specific file (remember, no intentional deletion)? The recycle bin is an outdated idea for me, but that is simply because I’ve set my recycle bin to never save any file I delete, and then removed it from my desktop. I have an 80GB drive in my system, and it’s far from empty (just the reverse, it’s nearly full), because I do rarely discard information. However, when I remove files, it’s far more often that it’s in the form of uninstalling programs (most often games) than removing data from the system, and no amount of rating files on the system is going to help (beyond the existing mechanism ranking how often I use a particular program, which still isn’t perfect).
The best of the ideas that have come about so far regarding new ways to represent the file system include one important factor: flexibility. In the end, no one system is going to work well for every user. Those that believe time-based organization would work well for them can do so, while those that work better with the existing directory systems can do so. The important part will be the flexibility to represent data in different ways without affecting the underlying storage of that data. Categorizing data without having to divide it into directories according to a particular scheme of categorization. I may be searching for a spreadsheet for a particular project, and in that case I’m fine whether I’ve chosen to organize my data by project or by document type. On the other hand, if I wanted to see other documents for the project, under the most-used systems today I’m screwed if I’ve organized my data in one manner, while on the other hand if I’m looking for similar spreadsheets for other projects I’m screwed if I’ve organized my data the other way.
Of course, in either case, time-based data is lost except in certain views, and even then the time-based data available is often very narrow. The email example, as already stated, is misleading, but there’s also the issue of whether the time and date a document was received is not only accurate in your mind, but is the real data you’re trying to sort by (after all, when you received it has nothing to do with when it was written or made effective).
The more information and options we get in regards to our data, the better, but in the long run we’re just generating new ways of looking at the same data, and while it may become easier for some people, the whole thing may be too complicated for new users, and much of the data will be inaccessable to even accomplished users who simply have no internalized concept of data in this fashion.
A file system/browser with the power of a relational database minus the complexity. From what I understand this is something that will be incorporated into Longhorn.
Imagine if the employees of some small business could all store their Purchase Order PDF files in a single directory “PODir” on a network share, and later go back and sort (query) them not just by the date-created or filename but by PO Number, Vendor ID, and Project ID. That is a powerful feature; yes they would have to enter these bits of info for each file, but its not much more work than typing up a filename.
This is being done already using multipart filenames or RDF files or an entire database to track the “sort-by” metadata, and custom programming to do the sorting. However, having this functionality built-into the filesystem would be cleaner and drastically simplify application design.
Is work of this type (as an answer to the Longhorn plans) happening in the linux world? Imagine if Longhorn comes out and suddenly the secretary is empowered to setup a PO tracking system that would have previously required a programmer and a week of custom programming. It may be rudimentary, but it gets the job done on-time and, for the cost of upgrading the five desktops in the office to Longhorn, under budget.
P.S. – throw the XAML saddle on that horse and you can really ride. Mozilla, Novell, Redhat, I hope you’re cooking up something across the corral.
I think, it doesn’t matter what hard drive I have, when I know that file is unneeded to me I don’t want to keep it on my hard drive. What is more, some files I don’t use for months, for example, my photos collection. But these files is important to me and I don’t want, that OS would decide to delete them because they are unused.
What more does one have to say?
Gnome Storage was exactly what I’m thinking of. You still have actual directories, but the directories (and sub directories) would essentially add metadata to the files, allowing you to search quickly, without having to actually go into the file and changing the metadata themselves. At least thats how I feel it should work. (As to whether thats how Gnome storage will work, I have no clue, but thats what I’d like to see)
Perhaps this would be a better example of behavior.
Dropping a song into ~/music/band/album would automatically change the tags to reflect that the song was by band, and from album, conversely, ripping music to the computer with tags of band/album would automatically generate ~/music/band/album (Sound Juicer already does the second half of that).
I’d like to see something like in MacOS X, the unix underbelly still exsists, but you get a applications directory frontend to your installed software. In / everything is divided up the normal way, in applications all files added by a program appear in one ‘directory’ dragging that directory to the trash would uninstall it.
hmm, it’s not bad to think about a new ideas but I just don’t seem to like it. The only good thing might be that I have ‘files’ (or so to speak objects) in various categories. However, I can already have _most_ of this functionality with symbolic links.
E.g. it’s difficult to say if I want my music videos in the /videos/music or the /music/videos folder. But with symbolic links that’s no problem, I just save them in e.g. /videos/music and make a symlink to that folder in /music/videos.
However, what I really can’t understand is the removal of the delete option. Only the word “automatical” removal sounds very bad. What if I don’t like rating my files, and don’t use an important file for some months. I definitely wouldn’t want my OS to even _consider_ deleting this important file to make room again.
Well and the last question:
What about the command line interface. I just cannot imagine currently how you could use this kind of filesystem as efficently as the current one, when working in a terminal when I can’t rely that an expression (or in the current case: a filename) is unique. So a: ‘rm flower’ would probably not be as unique as ‘rm ./flower.jpg’.
However, thanks for the editorial, I like hearing about new ideas, and yours definietly has its place.
I do
I don’t want my computer telling me whats right and wrong on my system drive.
Sounds like this article is for dummies..PC end user?
Oh wait, we have something called hummingbird for our clients.
Integrates their apps to most everything. Scanners, mail, file directories, etc…but it is a pain for us to setup.
And not cheap.
I think what the author is talking about is a system where, instead of deleting files, the user marks them as “unimportant” or “garbage” and then, when the system gets low on space, they get deleted. As he specifically says, the system would treat unmarked files as important by default.
– Simon
No trash can
Ok this guys use drugs or what?
What if I want to delete a file complete so no one can see it?
I can’t just rate it as not important because can still be used.
Delete will always be an option.
CD’s storing meta data about cd’s is actually a new add-on feature for BEOS. Check out OSnews from a few weeks ago.
If a file never gets deleted then what do you When you want to destroy your porn collection, so you can give the computer to your parents. Now those files are still around.
I have 160 of 180 gigs of Hard drive full. No I don’t store movies or music(only 6 gigs of music). Space may be cheap file sizes are growing as fast as hard drives.
Now for the bad news. If the OS does all this automatically, how hard would it be to create a virus that expands through the meta-data? Early Mac OS had this very probelm. It would hav eto store meta data about executables, thoguh the file may never excute the meta data must be read. Also you either append the files with the meta data, or put the meta data in a seperate file, that can be carried around or it becomes useless. If you can’t transfer it, then you can’t use it.
It just seems to me to be a major waste of space and resources.
This guy should work for Microsoft.
What’s so hard about naming a dir PHOTOS.
Put photos in PHOTOS.
I also disagree on the trash can idea.
imagine the you seem to have 5GB of free space (which would be so called unimportant garbage status files) and you want to copy a dvd-image to that space through a Gbit LAN. I don’t want to wait first until he deleted that 5GB of small files.
What the system should do in that case is open a window displaying ‘clutter’ files that could be deleted, and offer to burn the things to cd, also allowing you quick access to delete files that you don’t want anymore, or to have it ignore them by assigning a rating to them.
It shouldn’t wait very long either. If it waits till there are 1000 unimportant files laying around, people are very likely to burn them all, delete them all, and then wonder a week later what happened to certain files because they didn’t read the entries. So it should probably do this fairly regularly, somewhere in the 10-50 unimportant files accumulated range.
This would probably annoy quite a few people, but it would be less annoyance than people losing data because of a UI feature.
As an example, let us see how unproductive a simple every-day task can be. Suppose we simply want to “print the image that Helen sent me a few days ago”. We would have to open the e-mail application, sort the messages by date to find the relevant message, save the attached image (very distracting, especially since we also have to manually select a location for the image to be saved), activate the file manager (or a photo editing application) navigate to the location where we saved the file (quite distracting too) and then open and print the file. If the attachment we are looking for is older, we may have to perform searches in the e-mail program and maybe also through the file system, if we think we have saved the attachment some time ago (all the time trying to remember the file name of the image). Now this procedure is a serious distraction from what we originally wanted to DO with the file!
On some (older) Macintosh models you: open the e-mail application, sort the messages by date to find the relevant message, drag the attachment name/icon to the desktop printer icon, click OK, and there you are.
Shows you how some engineers/designers/system make life easier for user…
nda
Has anyone ever actually tried to rate their ENTIRE iTunes music library? Can you imagine how long it would be then to rate an entire filesystem: “uh, this little text files that was part of that project I did two years ago, is that a three-star or a two-star?” Even when doing it on the fly, supposing that you started from scratch on an OS allowing rating, how could you be sure that this “irrelevant” piece of data might not become more important later? I wouldn’t want to lose potentially valuable information after a month just because I forgot to rate that stupid file. (BTW, what’s with the craze about rating everything? iTunes, iPhoto, Amazon, movies, everybody want to reduce the universe to a five-star rating system).
Windows has a kind of similar system that compresses unused files after a certain period of time. Problem is, it might mangle your files, and some applications might choke on the compressed file. The point is: automatic management of your data is not always desirable.
Where I agree with this editorial is on the principle of developping original, useful views on one’s data. Somebody mentioned LiveQueries from BeOS: right on. MacOS(X) labels are another way of implementing some sort of view on your data, it would be nice to see the system exploit this a little more though.
Re. the idea of persistent storage: I don’t think it’s either a good idea. If ones wants something to go, it must go.
In my opinion a good system would help you SEE what’s used/not used, what’s the nature of the data one has, and allow you to dig better through your data with a combination of clever interfaces, categories, automatic + manual metadata, and GUIDE you into some sort of data management, not push you to throw your stuff away.
It sounds as if the author is groping towards hierarchical storage management with his ‘no delete’ pitch. HSM has been around for years at the enterprise level. This may be the time to bring it to the desktop.
Storage has always been a compromise between price, size, and speed. The fastest, but most expensive and smallest, memory is the set of CPU registers. Next is cache, then RAM, then hard disk, and so on down to optical or tape. Virtual memory has been around for decades, trading cheap, slow disk space for fast, expensive RAM. HSM does similar things at the file system level. If a file hasn’t been accessed for some time, move it from local hard disk to network disk, or from hard disk to optical disc. But make it look like it’s still local, and fetch it automatically when needed.
The author is not really proposing ‘no delete’. He gives several examples of things that should be deleted without question. But there is merit in the idea that there are files that should only be deleted after a holding period, and others which should be moved to offline storage rather than deleted. Current systems tend present the choice between ‘keep forever’ and ‘delete now’, forcing you to manually weed out the kept files later. It would be much better to be able to flag files upon creation for deletion after a certain interval or by a given date.
What’s so hard about naming a dir PHOTOS.
Put photos in PHOTOS.
But you need to create the illusion of the computer being smart and doing things for you. We are living in the 21st centrury for gods sake, we shouldn’t have to work for the computers the computers should work for us… well at least pretend that they do.
[/sarcasm]
Automatic stuff rarely works for everyone unless they have a really smart AI, which is quite impossible to introduce today. I mean how many people haven’t been frustrated with the “auto play” feature when you put in a cd. Now imagine a whole system working like that?
Sure you can learn some simple patterns but a human isn’t that easy to predict. The user should be the one in control, not the computer, the computer should only make it easier for the user to control it.
This guy is just rambeling on about ideas that has been in the air for over 20 years. The ideas has been there for a long time, however a complete and solid solution has yet to be seen. So quit talking and get on with it.
might be worth to have a look at
http://segusoland.sourceforge.net/screenshots.html
Um why aren’t your PO’s in a database??
The system we have at work is very simple, and does a lot of complex stuff. It was designed for netware 3.12, and dos with 286’s. It stores 5000 parts, 1000’s of vendors, 1000’s of clients, and 150,000 invoices since the system yet online in 1989.
It does everything from General Ledger, to Inventory Managment, to Accounting. Everything is in loadable modules, so that minimal system resources are used. I can access any section from any terminal, with the accounting sections closed off with passwords for security. We can look up any PO what we created, or any invoice that was created since we started. Not just in order by numbers, but also by vendors.
You just need the right software to begin with.
You don’t need a relational database to store meta data. If you store meta data, I would create a second file that stays with the first. Copying one the OS would copy the other. Then searches are performed using the second file only. Speed up the searches, and you can control items.
I wouldn’t want my file manager to locate files in such a way that extra data will linger around my computer. Secure deletion is hard enough, I can’t imange what you would do in trying to eliminate all the references t a file.
Maybe the concrete, actual ideas presented here are not so very smart, especially the idea of having no trash can. But it does make sense to try to develop *some* completely new ways to manage and organize files – instead of just having folders and files in them, plus some search function.
Think if Longhorn will have some such new and easy to understand file management scheme, and if Windows will still be the most used OS at that time. Most people may start to expect to see such features in other operating systems and environments too. Why not try to develop such new file management ideas already? Or do FOSS people want just to copy again in the future what MS developers have achieved to do? But what if MS has already patented that new file management technology? (ok, I think this from a Linux/FOSS point of view…
A bit out of topic but: This “source oriented file management” (or what ever you call it) – in a very general sense – is also something that the GNONE/Nautilus devs might consider too, as they seem to want to innovate so much. So something like this besides – or rather than – just the new spatial Nautilus thing that not so many people seem to like.
Anyway, just the plain old, trusted folders and files system does still seem to work for most people quite nicely, and maybe in the future too…:)
No deleting is leading to disk fragmentation which is seriously damaging speed.
Cookie (data on CD) coolecting. Most of the CD-s are inserted once and copied on disk. What about the data you pass by on Internet? E-mail?? Chat?? Problems with information storage definitelly aren’t geting and storing cookies. Main problem is information overload and what to delete when something is deleted. Again leading to damaging speed.
No system is smart enough to think for user. What and when to delete something. Mostly contracts and similiar documents aren’t touched for a long time.
Searching by using verbal descriptive sentences. Hello, Leisure Suit Larry. I loved guessing the right words in that game. Mostly I always knew what I want and all I had to do was just find how Sierra guys would say it. In this case another person, at least in corporate environment
Categories are maybe usefull for home user, but for working environment they are out of the question. There’s no people that would make same categories (or at least use the same descriptive language). Problem could be solved by data selective hierarchical design where categories (with subcategory option) would be stored on server and wouldn’t allow everybody to make his own but that poses another problem, who will take on this job, because this would just get to another network administration.
If problem would not be solved everything would look something like this: “Hello computer, show me the blonde babe with big boobs I saw last week”. You would probably get really good looking cow collection.
Most suitable implementation of this problem is probably Reiser4. All of the data is retaining standard structure and in the same time all data is residing in virtual descriptive structure. WinFS and Longhorn has almost the same paper solution. None of them is actualy leaving the standard structure, just adding another layer of data representation.
Rating is another missed point. What is valuable in corporate world for one and what for another person. What did author think, rating of people on network. Ok in this case managers trash is more important than some low level clerk, no matter how important it is to him. This guy would always be the first to loose data.
World will be turning for a long time before such specification will get usable even on scratch paper. To get it in use at working environment is completely different song.
Dropping a song into ~/music/band/album would automatically change the tags to reflect that the song was by band, and from album, conversely, ripping music to the computer with tags of band/album would automatically generate ~/music/band/album (Sound Juicer already does the second half of that).
Cool. I also keep my music in ~/music but with jazz/, metal/, pop/, whatever/ best-of subdirectories. Works fine for me as long as I don’t touch Rhythmbox, again. And it’s a the reason I probably touch Soundjuicer just once.
The metadata idea works only when you like other people organizing your home folder. I had that on Windows and it was one reason to leave it behind.
I recall suggesting a similar Concept to segusaLand to the BeOS OpenTracker Developers. IANAP, so I didn’t implement it, but the concept was as follows;
Since most Tracker Add-ons only work well on certain format Files, and these Executable Programs (the Tracker Add-ons) could be associated with a particular File Format (using the Built in Features of All BeOS Executables… a feature which exists in most modern Executable Formats), why not Hide Tracker Add-ons which don’t apply to a particular File.
I wouldn’t want to open a GoBe Productive Document with Army-Knife (an ID3 Tag Editor), nor would I want to see it in the Context Menu as a Tracker Add-on.
The same concept could be applied to segusoLand, but encapsulated in the File Manager, rather than implemented as a separate program.
[rant]I recommend all developers and GUI designers obtain a copy of FreeBe and install it on that old K6-233 you have lyin’ round. Progress in the BeOS Arena is currently Very Slow, but the concepts deigned by the People at Be had so much potential…[/rant]
Back on Topic;
This is a very Complex Concept. Most desktop users have just learned how to navigate a Filesystem. Introducing Automatic File Deletion/Backup and the requirement that a file be Rated, Classified or Labelled will take time.
Users aren’t going to adopt this concept immediately, and 3rd-Party software developers aren’t going to implement it’s features for several years after introduction.
Why do we need ‘completely new ways’ when just about everyone on earth understands ‘folders and files’.
It’s simple – everybody gets it.
Windows Explorer in XP is already such a mess I can’t use it – had to switch to 2xExplorer.
Try to open a directory with 15,000 or more files in Windows Explorer.
What’s with all the lame comments blasting this author? He’s totally right. “Standard” file management techniques are cumbersome and distracting. I should be able to assign data to multiple categories at once, create custom views, perform smart queries, access metadata, and also rate and/or flag files in order for various automatic actions to be performed.
Face it: current filesystems are braindead. Some previous commentator wrote: just put your photos in “Photos”. OK, I will. Gee, now I have 1100 files (or 10,000 files, or 50,000 files…) and they’re all named “DSCxxxxx.jpg” (insert number here). That’s really helpful. NOT!!! (And no, I’m not going to individually name several thousand photos manually!)
A program like iPhoto has to “fake it”. It automatically creates a basic date-based folder hierarchy and stores the files in the proper locations. But iPhoto itself doesn’t actually show you that hierarchy — it uses custom views (image-based) based on various criteria you select and it uses its own custom database in order to accomplish this.
But what if iPhoto’s database/file management was built-in to the OS? You could still use iPhoto the same as before, but other apps could easily have direct access to the same information in the same types of organizational structures. Any third-party could integrate their own “iPhoto”-type views and features into their own software. Right now, you can only do that in OS X by parsing iPhoto’s XML file and constructing your own “database” out of it. It works, but it’s a pain.
Every app has to reinvent the wheel if it wants to store data in a sophisticated manner. But that need not be so. BeOS, for instance, came with an e-mail viewer. It was called…the file manager (Tracker). Because every e-mail was a file with metadata attributes, you could view those e-mail files just like in a regular e-mail program. Double-clicking on the e-mail would show the e-mail’s contents in a separate viewer. It was, of course, an extremely basic system and most BeOS users used dedicated e-mail apps, but those apps could have used the file system for their data management (most didn’t, unfortunately).
The point is: what is the point of a filesystem? To store data. And clearly the best way to store data is in a database-style manner. Yet current filesystems don’t. They’re braindead. No beating around the bush. They just are.
I enjoyed reading this article, and I think that if more people would open their minds just a little bit, they’d see what the author is trying to get across. WE NEED A NEW FILESYSTEM!!!
Jared
I agree, 2xExplorer is by far the best file manager in windows, but I wouldn’t call Explorer a mess.
And when setting it up for a listview with small icons and no sidebar it’s quite fast.
The only problem (as with most M$ programs) with Explorer is that it just lacks features for power users.
[Files and folders…] It’s simple – everybody gets it.
Windows Explorer in XP is already such a mess I can’t use it
Is it just me, or did you just completely contradict yourself. If files are folders are great because they’re simple and everyone knows how to use them, then why has file management become so horrendous?
Jared
The point is: what is the point of a filesystem? To store data. And clearly the best way to store data is in a database-style manner
I wholeheartedly agree with everything you wrote. Opera Software (no, I don’t work for them) seems to have understood the problem with its mail client: each contact is an access point, attachments is another access point. This solves at least the problem exposed in the first part of the article (searching mail). I wonder if this can be extrapolated to other applications?
‘did you just completely contradict yourself’
no contradiction – Windows has already gotten away from the simple concept of files and folders.
Explorer tries to be a file manager and web browser and many more things at the same time.
There would be nothing wrong with a separate program that specialized in offering different views or organizations of the file system – that would be great, but not the basic file manager.
Google is going to do this and I bet they’ll get it right.
bottom line – don’t pimp the file mamager
Fragmentation is not an issue with any decent filesystem.
If you go back to Windows 98 and FAT32, then fragmentation is a problem. Not deleting files doesn’t cause it. If you start with a fresh filesystem and start writing files, they will be allocated contigious blocks.
Deleting files and reusing the space freed is what causes fragmentation in FAT filesystems.
Again, modern filesystems deal well with fragmentation, and legacy filesystems do better if you never delete…
I come from an AS400 background and that’s one thing that keeps those old boxes alive…it’s really easy to find manapulate any data on the machine from pretty much any program. Of course they’re really ugly EBDIC only machines…but bear with…
The key thing is that the OS, Applications, and Data are all kept in seperate filesystems that match their purpose. The core OS file system is very Unix-like and apps can sit in either system. But the user data area is very strict and confined…you don’t have many choices in you you want to organize your data. But all of the data is only 1 level deep so it’s really easy to modify anything you want! It allows you to use Database hooks directly on files. It also lets you focus on manapulating the data directly rather than continually developing new ways to “do it better”.
While I don’t think We need to copy AS400 [it is a bit old!] it’s got some great concepts that only something like Linux can pull off. First, we should have super clean seperation of OS, programs, settings, and user data…I think Knoppix is setting the way on this one because the live-cd format could “force” something like this and still keep your data portable to new OSes!
Second, I think that the “settings” and “user data” should be seperate partitions on hardware. It’s time to stop worrying about “sharing” and use hardware to the maximum. You’re allocated 4 hardware partitions on a drive, so this would be really easy to set up.
Third, implement the “query” & “filesystem” functions at the driver level! Make the “user data” devices look “flat” to the file system and use the “rawest” encoding possible…Take a driver like Reiser or JFS and put on “object oriented” & SQL hooks directly to the internal HDD structures… I’m looking at the “object databases” as examples of what I would want. Then you’d be able to handle the querying at the driver level as well as adding RDMS like functions for linking data types together… All of these things people talk about [file management, tracking, linking ] would be “stored procedures” on top of this new paridgm rather than 101 seperate beasties to get to dance together!
I keep seeing lots of effort into constantly re-inventing a “better mousetrap” when what we really need is to get a cat! OSS is the only group even capable of such a thing right now [well, OK, BeOS & Palms work much the same way!] It would be really cool to beat MS to their own roadmap! After all, they’ve given us 2 year’s notice!!!! The real thing needed in OSS is to get more USER input. The entire parigigm has to shift to make something like this useable…after all, nobody get’s it right the first time! If it’s to work there has to be a really good way to develop consenseus between machines on what “procedures” should be common and let the users make their own up then share them! That’s the real reason for the long-term Unix success…it’s a simple framework that was build up by consenseus, not dictated by a companies marketing department.
Take a look at Outlook’s Journal capabilities, if you’re interested in getting a partial implementation of the chronological-type thing you’re after.
Face it: current filesystems are braindead. Some previous commentator wrote: just put your photos in “Photos”.
OK, I will. Gee, now I have 1100 files (or 10,000 files, or 50,000 files…) and they’re all named “DSCxxxxx.jpg”
(insert number here). That’s really helpful. NOT!!! (And no, I’m not going to individually name several thousand
photos manually!)
But you are going to enter the meta-data for each file manually?
The only meta-data we will get from the camera is the technical info, and the date. And over time when GPS receivers becomes cheaper it will also be able to provide us with the location.
But you’d still have to enter any additional information yourself. There’s no way a database would solve it for you.
I think that meta-data in the filesystem is a great idea from a developers point of view. But from the users point of view there’s really no difference if the app is faking it or not as long as you only use specific apps for the task.
WinFS in that aspect is just as “fake” as iPhoto because it’s still just a layer on top of the filesystem.
Fragmentation is not an issue with any decent filesystem.
If you go back to Windows 98 and FAT32, then fragmentation is a problem. Not deleting files doesn’t cause it. If you start with a fresh filesystem and start writing files, they will be allocated contigious blocks.
Deleting files and reusing the space freed is what causes fragmentation in FAT filesystems.
Again, modern filesystems deal well with fragmentation, and legacy filesystems do better if you never delete…
Wrong. On any heavily used machine, fragmentation is a serious issue, regardless of which filesystem is being used. My current NTFS drives become quite heavily fragmented every several weeks, needing defragmentation about once per month.
I’d have to disagree with the author’s notion that “WHEN” is the primary attribute of a file. I know the author isn’t alone – David Gelernter was working on a similar filing arrangement, but I didn’t see any of his company’s test releases as practical.
I remember by “WHAT.” My WHEN’s are way screwed up and I have a very bad since of time. “A long time ago” could be years or weeks. The way smokers take smoke breaks, I know I’m far from alone here too. WHAT I need is what I look for.
The author’s problem is the same as other proposed solutions. They try to find a one-size fits all search & display solution. The way iTunes works – genre, song, year, composer, artist, etc is the way it should be. The search and organization method has to be extremely flexible. What if I wanted to make a playlist based on all the 5 star songs recently added within the last month to the collection from the 60’s on Elektra? The same flexibility has to stretch to the whole file system.
A playlist (directory), 5 star (importance), recently added (date sent/ received/ created/ modified), from the 60’s (subject), on Elektra (source). The WHEN factor is just one among many.
People may look at a file from very different perspectives. I may work on a project and build up a “shopping list.” I save this list after the project is complete so that others can reference what part #’s, manufacturers, suppliers, etc were used. The budget office, of course, will look at that list in a purely financial way. They may look at it as Dept X’s project from Q4. A worker in Contracting, as a list of suppliers – WHOs and WHATs. WHO can provide us these parts for Project X.
To simplify his task of opening a photo attachment from X, received during x using the file system, a photo editor, email, etc, the simplest solution is to erase the differences between applications. The task should be the focus instead of the application. Applications should be like plug-ins to a browser. I know developers lose a lot of branding opps, but they can still charge.
I know a lot of people will roll their eyes at this suggestion, but using the latest version of MS Works (full version) is a good preview of this functionality. Many of us who grew up in an application-centric environment have a hard time switching, but Works’task-based UI is extremely newb friendly. Almost all of the apps look the same and the main panes are task-organized.
The problem isn’t one of organization, it’s one of process and workflow.
A program like iPhoto has to “fake it”. It automatically creates a basic date-based folder hierarchy and stores the files in the proper locations.
No need. The EXIF tag contains all the information you mention. The filename is really irrelevant – you’re just pissed off because you’re sorting a listview on it. Maybe next time try enabling the Tabs such as description, dimensions, date picture taken, camera model, and so forth and sorting on those instead.
Now if you want to create your folders, tag files by sorting on different columns, and you’re done.
Sure there are more eloquent and flashy ways to display and sort these files, but explorer was designed to manage files and directories.
Don’t dump on the file manager for showing you a list of filenames *as you requested*. And don’t dump on it because you are trying to use a hammer as a chisel.
[i[Gee, now I have 1100 files (or 10,000 files, or 50,000 files…) and they’re all named “DSCxxxxx.jpg” (insert number here). That’s really helpful. NOT!!! (And no, I’m not going to individually name several thousand photos manually!)[/i]
This is really such an irrelevant detail. File systems were designed to be simple for a reason–so that they wouldn’t fail. It was expected smart software to manage files would be built ON TOP OF simplicity. We don’t need database support in a filesystem, as part of the architecture. But a “database-driven” file manager that runs on top of a simply-designed FS (that does IMPORTANT things, like making sure I don’t lose data).
As for this silly filenames all in a directory argument, I really find this ridiculous. First of all, have you ever managed a photo collection? I have managed mine for about 4 years. And I manage it the way that makes sense for ME to manage it, because my filesystem is simple. I have a directory called Photos. Below that, I have the year (2000, 2001, 2002, 2003, 2004). Then, within there I date a folder with the download month/day and a short description of what the download contained (i.e. 05.13 – Alex’s birthday). Within that directory I have those weird filenames you speak of, but it doesn’t matter because I usually just want a slideshow. I use no iPhoto or anything, just an image viewer and my space key. Has worked wonders for me.
The point is, that no amount of software smarts will name your files for you, or figure out what your photos are of. Take a picture of a beach, if you want to search for “photos of the beach” and your computer isn’t gonna figure out which pictures are of beaches without you telling it so.
WEAKNESSES: it’s true my filesystem management has a weakness–it precludes multiple views. For example, I have it sorted in terms of dates. If I wanted to see it sorted alphabetically by description, I’d have to do some digging. That’s where software like iPhoto comes in handy. But no file system is going to be smart enough to be an iPhoto replacement AND an iTunes replacement AND a whatever else replacement. Let the software be smart, let the filesystem be stupid (and redundant and fast). Ends to ends argument, sorta.
Try BeOs, Zeta (or maybe SkyOs later too ) and get impressed by the possibilities of the Be File System, created long time befor this discussion starts! So why should someone wait for Longhorn or something else ?
Maybe there is interest of porting the OpenBeOs Filesystem to the *nix world ( if it works ), which is already in a ready to use stage, to spread the advantages of this filesystem to many user.
The “no more folders” explained in this article was one the root of the BeOS project. At the beginning of BeOS, J.L. Gassée said “Users don’t need to know where files are stored”. It’s true. What J.L.G means is we don’t care where files (i.e. in which folder) are in the hard drive if we have powerfull functions to find what we need. The functions were, in BeOS, the Live Queries, giving possibilities to search files (e-mail, doc, mp3 and so on) on their meta-data.
As an old BeOS User i had only one folder (i.e. a black box) for my *all* personnal documents and i used LiveQueries to find needed documents. Fast and easy.
I don’t understand why, today, we have to store any document by name with extension. It’s obsolete, instead we should store document giving more informations (meta-data) about the document itself : keywords, title, and so on…
So basically what we need is a new file system… I was thinking about making an article to call for a really portable and free file system that could :
– handle meta-data
– support files biggers than 4GB
– read/write on Windows, Linux, OS X
– handle permissions
– handle symbolic links (really needed under Windows)
AFAIK none of this exists. And until it doesn’t I don’t see anything like this coming. Or not in a portable way.
What you described sounds exactly like BeFS/OpenBFS, except for the fact that it currently only runs on BeOS and SkyOS (and somewhat on linux). The problem is to get people to use it since it isn’t the native filesystem of Windows, OSX and Linux. And I assume that Windows will have issues booting on anything else than FAT32 or NTFS.
Companies would be afraid to write apps that uses it since they don’t know if the costumers are using it, and if there’s no apps that take advantage of it, why would people switch? the good old chicken and egg.
Windows users will have to settle for NTFS with WinFS on top of it because that’s where the app support will be. Linux developers will always have the problem of not knowing which filesystem the app is running on.
This new theorical file system could in fact be compatible with the usual tree-structured existing file systems:
The solution is to refer to file ‘properties’ by using the same syntax as for directories.
For example, an app that aks for the file
/usr/bin/vim
would be interpreted as a file that has “usr”, “bin”, and “vim” in 3 of any of its properties in the database-based filesystem.
It could be for example type=”bin”, title=”vim”, whatever=usr”.
This way we eliminate the need for using any ‘new’ system to access the files, and makes it all much more easy: if you want to convert the whole file system to the new database system, you only edit the properties of existing traditional ‘folders’, and that creates a new ‘property’ in the files, with the correspondent value.
It would make it easy and painless to migrate, with no need to hurry it or anything.
WEAKNESSES: it’s true my filesystem management has a weakness–it precludes multiple views. For example, I have it sorted in terms of dates. If I wanted to see it sorted alphabetically by description, I’d have to do some digging. That’s where software like iPhoto comes in handy. But no file system is going to be smart enough to be an iPhoto replacement AND an iTunes replacement AND a whatever else replacement. Let the software be smart, let the filesystem be stupid (and redundant and fast). Ends to ends argument, sorta.
You don’t have to make the filesystem smart for it to be able to replace applications like iPhoto and iTunes (in terms of file management/views) with a single application. Even the application itself doesn’t have to be all that intelligent. All you need is support for metadata (which exists on filesystems available for most OSs today), and the ability in the file manager to view the contents of multiple directories as if they were all in the same place (or sorted into virtual directories based on whatever conditions you wish). For instance, if you have a WinXP machine you can go into a folder full of MP3 files and, in detail view, sort by the fields in the ID3 tags. This is possible simply because WMP integration allowed the file manager to access metadata that is part of the file format. You can also add metadata to nearly any other file and sort accordingly. The only thing really missing is the virtual directory views and handling of multiple (real) directories, which is something that needs to be done in the file manager rather than the file system.
On the other hand, if it was built as an OS subsystem on top of the file system with a complete API to give developers access to the same functionality that’s available in the file manager, I’m sure many others could come up with interesting uses for the features; and it would still keep the file system relatively simple (though, again, metadata support in the file system is a requirement unless you’re going to maintain a seperate database to handle the metadata).
What I meant was that new applications that depend on the ability to store and retrive meta-data in the filesystem wouldn’t work if the user is using a filesystem without meta-data support.
For example, a modular framework I was building for BeOS a couple of years ago relied on the meta-data of the modules to work. If you would move the app to a FAT32 partition then it would simply not start because it wouldn’t find any modules. This is not a problem since BeFS is the native filesystem of the BeOS, but on systems like linux where the filesystem could be pretty much anyone it’s hard to make the app depend on those filesystem features.
“The task should be the focus instead of the application. Applications should be like plug-ins to a browser. I know developers lose a lot of branding opps, but they can still charge. ”
That’s what I was thinking for a long time. All we do now with computers is “managing files” and “running apps”, but in reality all we need to do is tasks, and nothing more. If the focus was tasks and they knew how to locate their documents, there was no need for file managers. Which all suck, by the way.
Not anymore, thanks to TAB completion in bash
this will not work in certain situations. all my digtal photos are damn important unless i decide to get rid of it because it is simply almost duplicate or terrible one.
I guess I’m having trouble understanding your position, there should be no file manager, there should be a file browser. Which is a file manager isn’t it? Unless, its limited to simply moving through directories and viewing theie contents. As for the file browser performing all tasks? Wouldn’t that result in a horrible interface and make it hard to work? This is one reason I don’t use KDE.
I’d rather have many small applications that let me perform one task well, than one meta-application charged with performing all tasks.
The problem with metadata, is that NOBODY wants to maintain metadata. It kills me to keep up the ID3 tags on my MP3 collection, because nobody does them right. How many of you add metadata descriptions to all of your photos? Your word processing documents? Hell, how many of you pro-metadata programmers grouse about commenting your code?
Metadata could be a great solution, but it won’t be. It takes just as much work (maybe more) to keep metadata up to date as it does to navigate a traditional file hierarchy.
Here is a Java based app that focuses on attributes:
http://www.udanex.com/dekk_index.html
http://www.udanex.com/install/start.html
“At the beginning of BeOS, J.L. Gassée said “Users don’t need to know where files are stored”. It’s true. What J.L.G means is we don’t care where files (i.e. in which folder) are in the hard drive if we have powerfull functions to find what we need.”
That may have been true then, but it is not true now. You do need to know whether a file is on a fixed internal drive or a removable drive or a memory stick or whatever.
Very well put. I agree entirely.
Ideally, file name extensions (.exe, .txt, .doc, .bmp, .gif etc.) should be part of the file’s meta-data, instead of part of the filename. Why? Separation of concerns. The file’s name and type are two different things, and should not be treated as if they were the same.
The problem is demonstrated by a user who renames a file “foo.gif” to “foo.jpg”. The file becomes unusable, despite the user’s doing something that intuitively makes sense.
Regards,
Kramii
“Why delete files (and do it manually) from a 80GB hard disk when most of its space is free? The recycle bin is a rather outdated idea in a “garbage collecting” world. Files should never get deleted, either intentionally or by chance. They should be rated, according to their importance, so that the system will be able to propose appropriate actions when the disk is becoming full.”
This is a terrible idea…all I can think is that the author is talking about a system that works well for *the author*.
As a sysadmin, I routinely run into cases where I want data deleted off my computer. Moving other people’s files, temporary storage while repairing or replacing someone’s computer, attachments or items that are just plain trash and need to be deleted…why would I *want* to keep that stuff?
Sometimes there are cases where you will need to get rid of stuff. That’s why Apple includes a “secure delete” of the trash now.
And there’s cases when you don’t know WHAT the user was thinking when they installed or downloaded some of the crap they get on their system. Why would I want to rate spyware as “unimportant” and leave it on the filesystem?? DELETE IT!
And what about virus infected files?
And why would the system have to halt to propose a “solution” to the user when it’s getting full? Unless it would “automatically” delete things as it grew…which would also mean that the hard disk would still have more entries to search for, and decreased drive speed as it’s deleting crap while at the same time trying to expand the temporary swap image for your new burn or network file copy. And on a multiuser system, how many ways would this proposal make the computer less efficient?
Silly silly silly idea…
I agree with various contributors that the author has many valid observations – in stark contrast to various other contributors who seem to have archaic attitudes towards content management. Use folders? Yes, it’s possible to organise certain kinds of information in a strict hierarchy, but have you ever tried to manage information which doesn’t conform so readily? When programming, do you insist on knowing the name of each object you create? “Oh, I must really know the name or memory address of every object – otherwise I’m not in full control!”
Some people have a point with respect to deletion – sometimes you want to delete things that you don’t care about any more, and sometimes you don’t want things being deleted without you knowing, but then the author did say that the system would *propose* deletion rather than just doing it. But the argument about storage being hierarchical is especially pertinent, and I can imagine that most users would be well served by such a scheme, just as long as they aren’t sent off repeatedly to find files on backup media.
One thing I would like to add to my perfect FS that would work on most OS would be to be able to have pseudo-files that could handle some action. The same way when you run a script with #!/bin/perl the OS knows what to do with it… For example a pseudo-file could be a link to an MP3 that could be read as a WAV PCM file (decoded). The OS would silently decode it.
Also about the question of chicken and egg. Such a filsesystem would be opt-in, ie when the OS can handle a feature (like meta-data) it does it. Otherwise it simply ignores it and just present the files as it knows…
/me dreaming…