Submitted by “In a recent article, Linux File Systems: Ready for the Future?, Henry Newman expands on what he feels are shortcomings in current GNU/Linux filesystems. Specifically, he believes current Linux filesystem technology cannot meet the demands that massive implementations of 100TB or larger require. He states he received some emotional responses trying to either refute his information or impugn his character, although those comments do not show on either of the article’s pages. This prompted me to get the real scoop on how Linux filesystem technology is trying to keep pace with the ever-growing need for storage space.”
1) is it not that Linux is running on more servers than desktops?
2) I take more issue with a company making hundreds of thousands of dollars per minute, all day, every day merely by shoveling numbers 24/7 than I do with an EXT3 check taking a week
Seriously, some developments just seem to go astray… not that I envy their money.
So this guy thinks that “at this minute” XFS is the bees knees ?
I too like XFS, for my /home partition at least. I tried installing Ubuntu with ext2 for /boot, xfs for /, /xfs for /home
The machine was sluggish.
so I tried again….
/boot was ext2
/ was JFS
/home was XFS
REALLY slow
reformatted…
/boot again was ext2
/ was ext3 noatime
/var was reiserfs with noatime and notail
/home was ext3 noatime
much much better
finally as a test
/boot is now ext2
/ is formatted in ext3 with noatime
/home is ext3 also with noatime
this seems to be the fast permutaion for Ubuntu 8.04, now it could be that Ubuntu has prioritised ext3 over other filesystems, only some further testing will show that conclusion.
How did you measure the performance of your system? Is it just the felt speed? I read a benchmark once which stated that JFS is the fastest Linux filesystem. Even faster than ext2 (and jfs *does* have a journal, ext2 don’t). I think this benchmark was linked here on osnews somewhere, but I don’t have the link at hand.
PS: I use jfs for /home and other data partitions and ext3 for / because fedora didn’t let me choose jfs for that.
I rarely trust anyones benchmarks. I prefer real world testing.
It seemed slower doing things like opening Open Office from cold, doing a backup over a network, just seemed slow…
So how can we verify that daft ‘test’ that you’ve done above and reproduce whatever it is that you’ve actually found? The noatime option you can add for just about any filesystem, so what you’ve got there means very little, if anything. All it does is eliminate the update of when a file was last accessed (sometimes necessary), which is a huge speed improvement, but I note that you haven’t done that for the other filesystems you’ve bitched about. relatime is probably a safer option, but squashing the atime problem on any filesystem is the biggest thing you can do to boost performance, even over file stripe aligning.
If you think you’re describing real world testing then I shudder. I really do.
Edited 2008-06-10 22:23 UTC
I agree with you, but if you read the original posting, I was saying this is my results under Ubuntu 8.04.
Personally I like XFS with noatime, and as this is my desktop machine, and only me uses it, I rarely need to know when a file was last accessed.
But, you are correct, the example I gave was just thrown together quickly and cannot be called realworld testing, as I never published any timings, I just quoted my opinion. These were based on the delay between clicking a program or a file to open, and the thing actually appearing.
So far, from what I have tried, it appears that Ubuntu 8.04 is optimised to use ext3 over all other filesystems.
As you noted ext2 lacks a journal. But, ext3 is the same ext2 filesystem with a journal (it’s an mke2fs formatting switch, -j). I have had the data cables connecting my drives to my RAID card come loose at least 3 times and been able to re-attach them and come back up with no [detectable] errors or side-effects. I was surprised by this since the cables came loose during heavy data transfer and not after a 5 to 30 second quiet period where a disk write could save my bacon.
I have reiserfs (noatime,user_xattr) on both / and /home and it’s very fast and haven’t had any issues in several years. I did once try ext2 and noticed a performance drop, but with ext3 I didn’t notice any difference.
Yes, I’m quite sure the perf would slay our expectations…
I have used ext2/3 (very long time ago),xfs,reiser (these more recently) on desktop and desktop, and it was a very rare occasion that I noticed speed differences between xfs and reiser. On servers I’ve always used (and still do) only xfs, and these days I use reiser (3) on my laptop, for me this turned out to be the bext solution.
But I have big trouble believing what the original article was about, since xfs can do all that he seem to miss as an fs feature, and it’s no secret that xfs is being used in many really large datacenters, and they seem to manage.
You have far too much time on your hands.
Too much time on my hands ?
Some people need to know which filesystem is the optimum, and if I am getting paid for messing around with them, then I say…
meh, someone is just jealous :p
How are you getting BETTER performance out of ext3 than XFS. Most benchmarks I’ve seen put XFS at just about the top of the heap (though JFS is also pretty good, as is Reiser3 if you’ve only got small files)
See my previous reply about using it in real world scenarios, not benchmarks.
Stats can always be manipulated to show anything
Using relatime for ext3, Ingo Molnar said,
“Getting rid of atime updates would give us more everyday Linux performance than all the pagecache speedups of the past 10 years, _combined”
I run Linux Mint Elyssa and relatime is default.
Oh really scientific. Well done there.
Good God. This just sounds like some prat on a Gentoo forum (not that I don’t like Gentoo, ran it for many years) who’s constantly running around like an idiot trying to find some esoteric performance advantage from a particular filesystem, whining when it doesn’t match up to their own little assessments and then whining like buggery to anyone who’ll listen when they have a power failure or a disk failure. Lo and behold, everyone has different answers on that one.
What the articles are talking about are extremely large storage arrays, and what filesystem you’d want to use if you had one. It is generally accepted that if you have something of even a few gigabytes, you’re storing files of a multi-gigabyte size and you can put something on a UPS then you choose XFS. If you don’t know why, try some Googling.
I’m the original author of the linked article and want to be sure this is clear (as stated a few times already): This is about massive file systems, consisting of single instances in the hundreds of terabytes. This isn’t your home computer, not even your SMB filesystem in most cases. There are MANY more features and abilities in XFS than you’ll find in most other filesystems available for Linux, therefore I dare say you’d need to tune XFS a bit more to get the better performance you expected.
Having said that, ext3 (and soon ext4) ReiserFS are good enough for desktop installs. Don’t move away if these are working well enough for you. Stuff like XFS is really best saved for enterprise-level installs since those admins must take time to mold it to their environment.
few questions:
1) formatting flags used
I bet that default
2) what flags used in fstab
noatime is not much really
3) sysctl parameters also allow tweaking fs
I would say that no matter which fs you are using, it is still not optimized.
I tend to use the XFS filesystem on my main machine, for both / and /home. My distro of choice (Zenwalk) uses it by default, and I never had any problems with it. No problems with it being slow or anything… in fact, it seems to mount faster than ext3.
For a laptop, I’m not sure what I would use. I hear that XFS can cause data corruption (but really, what FS can’t?) in case of power outages, and when on battery power a more resistant filesystem may be a better choice. On the other hand, the power has been cut from my desktop system numerous times with no noticeable corrupted files, so this may in fact be no problem at all.
Bottom line, I like XFS, and wouldn’t hesitate (too much
) to use it on more installations. JFS is also nice, but the one thing I like is xfs_fsr–XFS’s defragmentation tool. I never was too crazy about ext2/ext3; they felt relatively slow to me to both mount and use, and I don’t really like the fact that 5% is automatically reserved for the filesystem/root (even if it’s for good purposes–ie. defragmentation). Still looking forward to ext4’s new features, though.
One thing’s for sure: there’s no shortage of filesystems on Linux, and that’s a good thing.
The -m switch allows you to set whatever percentage you want. I normally use 1% since on any large partition that gives lots of space for moving stuff around. I rarely have files over 4 GB though, so you may want to use a percentage figure that takes your largest files into account.
Most comments here seems to mostly be regarding laptops and desktops. In such a situation, you rarely encounter the problems where ext3 is incredibly slow, or where XFS/reiserfs really excels. We’re talking larger installations here where huge filesystems and extreme amounts of files aren’t uncommon.
I work for a large company where it’s not uncommon for a user to have an excess of one million files just in their home directory. It’s structured in subdirs and all, but even so, imagine the sheer amount of files in a single directory.
Having a directory with a few hundred thousand files and directories in it, and each subdirectory having thousands upon thousands of files – try ext3 there.
mkdir testarea ; cd testarea
for i in $(seq 1 10000) ; do touch $i ; done
Now do some simple operations, like ls.. create a new file.. how about removing a file? Now imagine this is an NFS server, and the user enters a directory to list what’s there. The lookups to see who owns the files, what permissions they have. Imagine the user running an application, which reads files from such a directory, where every call to the filesystem takes that long.
I timed the above operation of just touching “just” ten thousand files on GNU/Linux with ext3 and Nexenta with ZFS (didn’t have access to XFS or reiserfs, but they are both very good with handling file/directory lookups), no tweaks, just straight from installation:
ext3
create files : 0m42.770s
remove single file : 0m0.074s
ls -l, single file : 0m0.295s
remove all files : 0m0.266s
zfs
create files : 4m37.307s
remove single file : 0m0.021s
ls -l, single file : 0m0.006s
remove all files : 0m0.276s
I was a bit surprised at the creation time for zfs, I must say. However, for other operations it’s faster, sometimes quite a bit faster. With ten, or a hundred times the amount of files I tested with today, perhaps it’s easier to see the problem – and this is just one aspect.
the status of filesystems are a tad weird..
reiserfs used to be the fastest.. not anymore.
(these numbers are rounded down to nearest minutes in the case of the few seconds ones)
ext3: not benchmarked due to obscene slowness(around jfs times, or worse)
extract 611k files, my maildir, in lots of dirs:
xfs: 8m
jfs: 1.5 hours
reiserfs: 8m
move all those 611k files into 1 dir:
xfs: 16m
jfs: 2 hours
reiserfs: 24m
find . | wc -l:
xfs: 4s
jfs: 15s
reiserfs: 27s
ls -l | wc -l:
xfs: 18s
jfs: 21s
reiserfs: 1m21s
rm -fr thedir:
xfs: 6m
jfs: stopped it after 30 min
reiserfs: 47m
ext3 is super extremely multi insanely slow with lots of files in 1 dir, it took like 20 minutes to just do ls in a dir with 200k files on ext3..
This also goes to show that reiserfs has regressed alot, but xfs also has improved.
in 2.6.5, reiserfs was alot faster than it is now, across the board, xfs was alot slower than now. reiserfs was faster than xfs is now.
Oh btw, total size of the 611k is 4.7gb