“Depending on how much you care for writing new versions of old programs, Reiser4 could be either one of the greater boons to software engineers or one of the greater banes. Either way, programming on Linux will likely never be the same. It’s not too remote a possibility that other popular platforms will be influenced as well.” Read the article at Kuro5hin.
Reiser4 Rising
About The Author
Eugenia Loli
Ex-programmer, ex-editor in chief at OSNews.com, now a visual artist/filmmaker.
Follow me on Twitter @EugeniaLoli
It’s encouraging that people can find software solutions to security problems rather than abusive hardware solutions that result in lowering the value of a computer system to the users.
I think that there is more to the story regarding file systems that support a data model abstraction. I would like to see more articles of this nature, hopefully also aimed at beginners or else paying extra attention to examples.
This is likely to be the best opportunity to renew deeply our OSes, apps that changed so little in the past 20 years, especially under the poor Microsoft rules (you know, three letters extension and similar sh*t).
Go boys, go! and get’em!
Sorry, but my way of programming on Linux will not change a bit until the same functionality is available on at least OS X, Solaris, AIX, Free/Open/NetBSD and Irix, preferably backed by a formal specification. Portability is important, and there are way too many OS-specific APIs to track – if there is a significant increase in performance/usefullness, it might be justified to use them if they are available, but it usually isn’t.
Submitt stories from Kuro5hin? I might as well post the latest issue of the North Korean Daily times relating to the “great leaders” attitude to the west.
Just look at such crap as:
Why a New OS Matters (1.41 / 17) (#31)
by tofubar on Sat Aug 9th, 2003 at 08:42:22 PM EST
([email protected])
Because Linux is an archaic piece of shit.
And we actually have people consider that the site is quality?
Link to MS sites regarding comparisions between Windows and Linux, I have no problems, but to why link to this crap?
Do you know the “1.41” number means ? It’s moderation: 1.41 of 5. So you are proving that kuro5hin is crap just because of one random troll ? Should we deduce that osnews is crap just because of your own crap you just wrote ?
If you really feel so strongly, just read the piece itself to which the link refers and then ignore the comments beneath, so that your delicate sensibilities are not offended.
I never said the article was crap, I just can’t work out why someone would write an article for such a third rate site. Sure, OSNews.com has its own trolls but atleast they are shot out of the sky very quickly.
The article is VERY good quality and I would much rather see it here or maybe on the zdnet.com editorial they have.
Reiser4 is probably the first filesystem I’ve seen about which I can say, “that’s the way it should be done!” Too bad it’s not available for my beloved Mac OS X. HFS+ sucks.
Some mention of BeFS was made in this article. Just as a side note the BeFree project is porting BeFS to the Linux kernel so that Linux will use BeFS as its boot/root filesystem. This takes a lot of work to change things in the Linux VFS layer to support the things that BeFS has, such as live queries, etc. Once it’s done, Linux will become far more usable…
check out http://www.beunited.org for news.
An excellent article. “It’s not how hard you work, it’s how smart you work.”I have a lot of respect for Reiser. He has a pure intellect. File systems are know becoming intelligent applications. Gone are the days when file systems were just all about block sizes.
Regards,
Mystilleef
Will this file system actually be able to survive crashes reliably? I’d like to be able to play with Linux for a week or two without having to fix broken applications or reinstall it from scratch. My last installation of Mandrake Linux only lasted a few days before the Ext3 filesystem was so damaged by a crash that I had to start over.
I’ve been using Linux and ext3 for over a year and I have had only two crashes, but they have been related to the internet and my IP connection. Anyway Linux doesn’t really crash, nor does ext3 get corrupted or damaged, use MD5SUM on your ISO’s to make sure they are good.
A data model will allow you to have more portable applications, at least if there is a schema involved. The current and outdated file system cause data redundancy and cause applications to have a structural dependancy, but with a data model, there is a management system standing between your data and the application. Your application will now make an inquiry to the file system for the data rather than use a hard coded structure.
At least that’s some classis data base management theory. I’d like to know more about these new file systems because it seems like they are going to eventually replace the old file system.
Guessing from your IP, we’re in the same province. Pitty about you being stuck on “Vibe”, though
While, in my experience, Linux rarely crashes enough to hose the system completely*, Reiser4 should be much safer for your data. In addition to journaling metadata (which protects the integrity of the FS structure) it journals regular file data (which protects the file contents themselves). The big thing with Reiser4 is that it can do this without causing a large performance hit, which previous attempts at data journaling tended to do.
Is it funcional in the 2.4.x series?
Reiser4 looks like its goign to ROCK! I hpe that we’ll find it in the second round of distributions, April 2004 as well as the 2.6 kernel, KDE 3.2, GNOME 2.6 ?, GIMP 2, Galeon 2, Abiword 2, Koffice 1.3, KDevelop 3, Abiword 2, Python 2.3.x, Qt 3.3?, Umbrello, oh I better stop, i’m getting too excited
When will it be added to the 2.6 test kernel?
“Anyway Linux doesn’t really crash”
In general Linux is very stable, but it is possible to crash it. I’ve crashed Linux several times trying to get SWSUSP working, that’s what killed my last Linux installation. Also, where I live I often have power cuts that last longer than my UPS’ battery and I’m not always around to shut it down. Then there are family members who insist on hitting reset rather than waiting for it to shutdown…
“nor does ext3 get corrupted or damaged”
That might be your experience, but in my experience it often get’s damaged when the system isn’t shut down corectly. Sometimes it’s just an individual app that I was working on, other times I’ve had half the software stop working or the system fail to boot. Several times now it’s been faster to reinstall the OS from scratch than to fix all the damage.
“use MD5SUM on your ISO’s to make sure they are good.”
Is the ISOs were bad then surely the problems would have been there from the start, they wouldn’t have occured after the system crashed. If this had just happened once or on one particular computer I could assume that it was an isolated incident. But sooner or later I’ve suffered file system damage on every Linux installation I’ve used, three times now I’ve had it self destruct after a crash. It’s too much to be a coincidence and it stops me from trusting Linux enough to use it as my main OS.
I’ve been using GNU/Linux for years and I have found my systems to be very solid, even after unexpected power losses (running ext3 and ReiserFS 3 here). Whenever I encounter a system crash, I can usually trace it to some sort of hardware configuration problem, with binary-only Nvidia drivers being the worst offender.
It disturbs me that the *nix/*mux world sees everything as a file. With the power of PCs these days it would be nice to see them treat everything the way people see them…
Examples:
Applications
These should be single file entities, like a non-compressed zip file which internally holds all its dependencies like dll/so files and global settings.
User Settings
There should be a single file per user, application. Such as mozilla1.4.0-moocow.set. This would exist in a user’s settings folder or in a global setting folder.
Documents
There should be (and in some OSes, are) folder/s for documents. Linux, get on the bandwagon here. It would be nice to see better organization from all OSes here. Like having all my textual documents grouped above the image documents etc…
Hardware
For [put favorite profanity here]’s sake, hardware is not a file! Hardware is hardware. If you want to access it in a hierarchy, fine, but stop calling it a file. I like how BeOS hides hardware from the user for the most part. You just turn it on and the hardware works. There’s no remaining entries for since removed hardware and there’s no popup dialogs for hardware just connected. It’s like the OS somehow knows I don’t care about the hardware outside of the fact that I expect it to work. Other OSes could learn from this.
In closing, it would be nice if Linux and Unix would learn how to use their great power towards making PCs something for the average Joe/Jane without making us adapt to absurd concepts like folders are files.
CS classes should be mandatory. Those who don’t understand CS shouldn’t make stupid comments.
These should be single file entities, like a non-compressed zip file which internally holds all its dependencies like dll/so files and global settings.
>>>>>>>
You mean like an ELF binary, which does all of the above, sans the settings part?
There should be a single file per user, application. Such as mozilla1.4.0-moocow.set. This would exist in a user’s settings folder or in a global setting folder.
>>>>>>>>>
Do an ls -a in your home directory. Your suggestion is exactly what most applications do. Of course, it shouldn’t matter. The user shouldn’t know, or care, where the settings file is — he/she should use it thorugh the configuration interface in the app.
There should be (and in some OSes, are) folder/s for documents. Linux, get on the bandwagon here.
>>>>>>>>
What, praytell, do you think /home/<username> is for?
It would be nice to see better organization from all OSes here. Like having all my textual documents grouped above the image documents etc…
>>>>>>>>>
One of the things ReiserFS enables is a more database oriented filesystem organization.
For [put favorite profanity here]’s sake, hardware is not a file!
>>>>>>>>>
Its called a unifying metaphor. Its a lot easier to program when everything behaves like a file, than when everything has a different behavior. Unifying metaphors are what make complexity like that in current OSs managable. Windows sucks in this regard. The underlying NT kernel has this beautiful object abstraction, but the Win32 API has different calls for (say) mapping device memory than for mapping file memory!
I like how BeOS hides hardware from the user for the most part. You just turn it on and the hardware works. There’s no remaining entries for since removed hardware and there’s no popup dialogs for hardware just connected.
>>>>>>
That has zilch to do with the “devices are files” metaphor. If you take a look, BeOS handles devices just like Linux (using DevFS) handles devices. There is a /dev directory with device nodes. The difference is that Linux does less configuration automagically. OTOH, for the stuff that BeOS does support, Linux handles the same hardware without too much fuss.
In closing, it would be nice if Linux and Unix would learn how to use their great power towards making PCs something for the average Joe/Jane without making us adapt to absurd concepts like folders are files.
>>>>>>>>
Folders are files. There just files with entries that point to other folders. I know the “average Joe/Jane” doesn’t handle abstraction very well, but c’mon, what do you think a table of contents is? A piece of paper that points to other pieces of paper!
It disturbs me that the *nix/*mux world sees everything as a file. With the power of PCs these days it would be nice to see them treat everything the way people see them…
It disturbs me you think that way. All operating systems store information about hardware and software on the hard disk. It only makes sense for such information to be stored as files on the file system. Come to think of it, they are files. The concept is simple, open and easy to understand.
Applications
These should be single file entities, like a non-compressed zip file which internally holds all its dependencies like dll/so files and global settings.
Isn’t that the way Linux handles it? I don’t understand your proposition.
User Settings
There should be a single file per user, application. Such as mozilla1.4.0-moocow.set. This would exist in a user’s settings folder or in a global setting folder.
Dude, that’s exactly how Linux handles user settings.
Documents
There should be (and in some OSes, are) folder/s for documents. Linux, get on the bandwagon here. It would be nice to see better organization from all OSes here. Like having all my textual documents grouped above the image documents etc…
# mkdir documents videos pictures music downloads share
There you go. That’s the first thing *nix users do after they create a user account. The above command creates directories called documents, videos, pictures, music, downloads and share. It is just an example you can create more or less as you please. And if the above command is too difficult to use for Joe and Jane, then there are several graphical file managers that resembles the ones used in Windows, Mac or {enter your favorite operating system here}, that will magically create these directories via mouse clicks. It doesn’t get more organized than that.
Hardware
For [put favorite profanity here]’s sake, hardware is not a file! Hardware is hardware. If you want to access it in a hierarchy, fine, but stop calling it a file. I like how BeOS hides hardware from the user for the most part. You just turn it on and the hardware works. There’s no remaining entries for since removed hardware and there’s no popup dialogs for hardware just connected. It’s like the OS somehow knows I don’t care about the hardware outside of the fact that I expect it to work. Other OSes could learn from this.
On the hard disk, your hardwares are represented by nothing but files. You can call it all you want but they are files on the hard disk. It is great BeOS hides hardware from its users. Linux, on the other hand, hides nothing from its users. Linux users care about their hardware especially when many hardware vendors still refuse to write drivers for Linux. As the system administrator of my system, I’d like to know as much as I can about my system and it is a function of a good operating system to provide me with all the information I need, including my hardware.
In closing, it would be nice if Linux and Unix would learn how to use their great power towards making PCs something for the average Joe/Jane without making us adapt to absurd concepts like folders are files.
While I believe it should be goal of developers and coders to make there software applications, simple, straight forward and intuitive to use for everyone, it will be wonderful if Joe and Jane like the rest of did got an education on the operation computers. It is more than just an education in this day and age, it is a life long skill. We all sacrificed a lot to be computer literate, we were not born with the talent. It will also be wonderful if Joe and Jane began by accepting the fact that everything is Unix is treated as a file, for reasons we know understand.
Regards,
Mystilleef
That’s a good point. People need to be educated about computers, preferably in a professional setting. People view computers as a tool — fine. But at the very least, a computer is not a simple tool like a hammer, but a complex tool like an automated lathe or milling machine. Nobody operates such tools without at least basic instruction, and I don’t see any reason why people expect computers (which are an order of magnitude more complex than either of those tools) to take no instruction at all.
Looks like someone has no idea what a file system in the context of Reiser4 is.
Lemme help make the distinction: file tree != file system.
Your file tree is all those folders and file you see starting from / and branching to infinity. Your file system is the system used for reading and storing all those bits on your hard drive. Bitching about how most Linux distros don’t have a “My Documents” folder on a thread about Reiser4 shows that you’re really quite clueless.
Sorry.
-Erwos
Whoa there. Maybe if you took the time to understand him it would make for less hostile convesations here.
“These should be single file entities, like a non-compressed zip file which internally holds all its dependencies like dll/so files and global settings.”
You mean like an ELF binary, which does all of the above, sans the settings part?
I don’t think this is what he means. I think he’s talking about the app folder idea a la Next/Apple. A very nice concept as far as usability.
“There should be a single file per user, application. Such as mozilla1.4.0-moocow.set. This would exist in a user’s settings folder or in a global setting folder. ”
Do an ls -a in your home directory. Your suggestion is exactly what most applications do. Of course, it shouldn’t matter. The user shouldn’t know, or care, where the settings file is — he/she should use it thorugh the configuration interface in the app.
Sure, have you done this ls?? Have you mabye noticed .mozilla is a directory? Did you also notice some application specific stuff is under .gnome instead of in $HOME? Did you notice that some applications like evolution make directories that are not even . directories?
Again, moocow was talking about making settings easier to deal with. Why do you need to deal with them? When you change computers for instance, or do backups. Maybe you want to move your only your mozilla settings to another computer. It doesn’t help that there is no standard place to put them (other than in the home directory) and that they are all hidden on top.
“There should be (and in some OSes, are) folder/s for documents. Linux, get on the bandwagon here. ”
What, praytell, do you think /home/<username> is for?
Umm, configuration files apparently. And documents, and images, and coredumps, and you name it. Easy enough to fix though, really Gnome does an alright job, if it weren’t for those damn system files like evolution/ cluttering up $HOME.
“It would be nice to see better organization from all OSes here. Like having all my textual documents grouped above the image documents etc… ”
One of the things ReiserFS enables is a more database oriented filesystem organization.
Yes, sort of. But this is more of a desktop/GUI issue I think. The next version of Windows is attempting to address this and it will be very intersting to see how it turns out. I don’t think it should be integrated into the file system though. Good modular design would dictate that it should be a layer on top of the file system.
“For [put favorite profanity here]’s sake, hardware is not a file!”
Its called a unifying metaphor. Its a lot easier to program when everything behaves like a file, than when everything has a different behavior. Unifying metaphors are what make complexity like that in current OSs managable. Windows sucks in this regard. The underlying NT kernel has this beautiful object abstraction, but the Win32 API has different calls for (say) mapping device memory than for mapping file memory!
I agree with you on the rest of your comments, I don’t see anything wrong with the file metaphore for developers. Users should not be exposed to these things anyways. Devices should be accessed through GUIs in ways that make sense.
Moocow is coming from a very different perspective here, that of an end user. Please try to be more understanding toward him. Having a CS degree does not give you the right to be condesending. Besides, users can tell us developers a great deal. Sometimes we, as developers, prefer wierd wierd stuff, like vi, or tex.
Moocow made some good points, maybe a bit off topic though.
If the user with Administration privledge could use a Data Definition Lanugage (DDL) to set the constraints and relationships of the internal schema as metadata in the cataloge, than the administrator could create the “views” for other non administrative users. In addition the users would work with a conceptual schema that is layered but decentralized (data independance) from the underlying storage details.
———————-
conceptual schema (view)
———————-
internal schema
———————-
Searches use a Data Manipulation Language (DML) which allow you to declare the result, rather than the procedure of how to get the data (for example search C: drive recursively for myfile.txt, instead becomes SELECT * (all or part of the data) FROM myfile.txt; ). You don’t have to specify the “How” procedure but instead just the “What” to find, and not worry about how it is done or where to search. This could lead to much more sophisticated and faster search capabilities because physical data organization is abstracted away from your view. Note, that this could greatly improve program installation/update time and issues such as deinstallation/Clean Removal.
And than you would have a data dictionary which you would be able to reference for information about your data on the filesystem. For example, data files would allow you to perform different search capabilities on them than say music files through a Data Manipulation Language.
I also think that it’s likely that the web browser will be an interface into this file system.
When people talk about GUI versus command line, the best solution completely depends upon the user and the desired results. A command line interface can be technologically superior to a GUI in some curcumstances and it depends heavily upon the TYPE of user and the accessibility of the results. Sometimes even “depending on the type of user” a short-cut key(s) is the most technically superior solution.
Look like that file system like reiserfs, the one from Plan9 and WinFS are the way to go
We live pretty interesting periods I think. All these thing, one by one will evolve our habits with computer, and it’s not bad – I think those kind of change will go through the all OS 
For education… why a simple task like reading mail, writing text, launching a game has to be complex? On most computer with MacOS X or Windows, everyone can install a game or an apps(insert the cd, follow on screen instruction telling you to push next) without nearly any education! Teaching how to use his computer to my father took me 1h – including how to install simple peripheral(CG, adding an HD…).
And I think it would have take me less time with MacOS X or BeOS. So now, why would I put my father in front of Linux? I don’t think he’s going to like vim and xf86config
Nor the fact that to change his background he has to find the right option hidden in a bloody mess of silly things.
Yeah we know computer. Yeah we know how to modify config file by hand. But most user don’t, and don’t need to know. Basic education : usefull, must take 1 hour or less. Advanced education, usefull for us, programmers, took time. Everyone can use a hammer to throw a nail, only some can sculpt things with.
It disturbs me that the *nix/*mux world sees everything as a file. With the power of PCs these days it would be nice to see them treat everything the way people see them…
Well, I would like to see people learn how computers see this information. If you are working with a computer, wouldn’t it be more logical and efficient to talk to the computer in its own language than make it talk yours? Or did you let all those Ghz go to your head? Computers were designed to do math, use them efficiently and they will do a lot of it. Don’t, and well, you have a 2 Ghz email client.
Hey, maybe we can have it read the email for you.. but not if you expect it to understand words like email at an OS level.
Everything is a file works very well for me. Keep it simple, stupid.
when your system is scanned for all its files, updatedb or whatever, have it store symbolic links to certain filetypes that might be searched into a directory that can be used later to complete a very quick search by reading that directory. So it makes symlinks to *.mp3, *.ogg, *.ogm, *.asf, *.avi, *.mov, *.wmv, *.doc, *.xls, *.csv, *.html, *.exe or set executable, *.txt, etc. It would only take up a few thousand extra inodes, and usually we have plenty to spare.
the filesystem could automaticly build the symlinks itself whenever it wrote out a certain types of files. I would also like to see more use of SGI’s fam for desktop icons, etc.
It would only take up a few thousand extra inodes, and usually we have plenty to spare.
Usually we do. I have run out a couple times though and it’s frustrating because you generally want to remake the filesystem with more inodes. ReiserFS doesn’t use inodes though. Do a ‘df -h’ in linux and you’ll see it reports that it isn’t using any. For more information see http://namesys.com/
Whoa, so many people took what I said the wrong way. Let me articulate a little better then…
Single file entity apps – revisited
No not the same as elf binaries. More like jar files except the contents look like this…
/app.exe (or whatever the extension is for your OS)
/settings/global.set
/resources/
/resources/icon.ico
/resources/help.htm
/dependencies/abc.so
/dependencies/def.dll
The idea is to make the user see the application as a single file but inside this file are ALL the little files the application needs to run. You could then place all the application files in a single folder. No deleting of important dependency files, no need for dependency hell (and dll hell in the windows world), no need for rpm-styled install mechanisms, etc…
I see some people said I was getting off topic. True, sorry. I just got goosebumps from this directory = file thing and had a frappuccino so the caffeine is just hitting me.
It was good to see people realize the perspective I was coming from. Rayiner, you need to spend a couple years in Tech Supp. to gain a better grasp of how people want to use computers. Reiser4 is a step backwards in this regard. That’s what irks me. Who the hell are we developing for anyway, if not the end-user? Sitting in a cubicle developing doesn’t teach this unfortunately. You need to be within the customer’s reach for a long enough period that their opinion matters to you on a personal level.
For the more linux-savvy this is a great thing however, the performance will excel beyond other filesystems so I guess some celebrating is in order.
So in otherwords, you’re saying you want fat binaries like MacOS X does.
Who the hell are we developing for anyway, if not the end-user?
Well I write code because I enjoy creating with it. For the most part I write it for myself. Sure at work we have meetings and draw mock-ups and try to decide what will be the easiest for a user. On my own time I write it to do what I want. In my IRC client I left things out because I don’t use them. It isn’t (yet) fully RFC compliant but it does what I want.
If you or someone else wants to pay me code an interface that you (or they) like then I’ll do it but on my own time I’ll do it the way I think it should/needs to be done. As a side note Hans Reiser will add custom changes to ReiserFS (and probably other parts of linux) for about $5000/month ( http://www.namesys.com/support.html ).