Recently hired by Ximian (now a subsidiary of Novell) in order to further improve the Linux kernel, Robert Love has interesting tasks ahead of himāintegration of all this low level work into the Linux desktop, specifically the GNOME Desktop and Developer Platform. The work is already coming to fruition as developer releases of “Project Utopia” (as it’s been dubbed) have already been released. So sit back and let’s see how Robert Love plans to make the Linux Desktop “Just Work”.
ArsTechnica Interviews Robert Love
About The Author
Eugenia Loli
Ex-programmer, ex-editor in chief at OSNews.com, now a visual artist/filmmaker.
Follow me on Twitter @EugeniaLoli
So we are yet to see the best of kernel-2.6. I didn’t even know there was another desktop scheduler in the works. So the best of Linux’ desktop interactivity is yet to come. Wow, I’m dancing reflexively.
Looks like there are some great ideas that are being fostered in the Novell think tank. Lets hope that these ideas keep flowing and hopefully we’ll see the end result in the Novell/SuSE Linux Desktop.
Regarding the 2.6 kernel, I am looking at get a new computer beginning of next year, most likely a dual Opteron. It will be interesting. By then KDE 4.0 development should be on the cards, GNOME will be progressing towards the big 3.0 and hopefully FreeBSD will be stablised.
What will be interesting is a comparision between the different operating systems running different desktops, also, the interesting part will be whether ISV’s pull finger and start producing those Linux applications.
If you go to:
http://primates.ximian.com/~rml/project_utopia/
As you can see he has Fedora rpm’s, but nothing for Suse? Ximian/Novel/Suse
Why Fedora?
Don’t take this the wrong way, I’m not shouting conspiracy, just curious why there arn’t suse rpms.
It is still at research and development level. In other words, they are still aren’t targeting customers with Project Utopia. It haven’t been integrated into any product, and I doubt that would happen any day now. What I’m saying is that it is pretty much to the developer to use the distribution he is comfortable with, and thus make RPMs of it.
Another thing is that SuSE have not released anything with the 2.6.x kernel. And I doubt that would happen until the final release of the version with the 2.6.x kernel (SuSE never have any beta programs, and unless Novel does something about it, I doubt they would do a beta program). In other words, RPMs for SuSE would be completely useless; unlike Fedora RPMs.
favoring I/O-bound processes implies favoring interactive processes (such as your text editor, mailer, web browser, and so on)
What a load of BS! Responsiveness in GUIs are not in any way bound by I/O performance. When my email client tries to open a window at the same time as two other programs are accessing the HD (e.g. dpkg and cp), guess which one will suffer? It’s because of moronic ideas like these by Mr Love that my xmms suffers from 1-2 second pauses every 2-3 seconds (even though I use a large I/O buffer, and have reniced xmms to -19) whenever there is more than one program accessing the HD. Add in some swapping and you get jerky mouse pointer movement (how on earth can a 850MHz computer be so busy that it can’t update the mouse pointer for 4 seconds?!?), 10-20 second redraws when switching virtual desktops, 10 second delays when trying to open right-click menus (0.1s being the max allowed, making 10s 10000% too much), etc. Using 2.6.1 my load is often 3-20 while cpu% is 2-8%. At the same time as windows are being redrawn in slow-motion (rendering them quite unusable) another program is happily copying files at 10MiB/s.
Now don’t get me wrong, the throughput is better than ever. It’s the responsiveness that is like that of windows 3.1. It is great for servers, but linux will not be ready for the desktop until the developers realize that desktops are not about throughput, but about responsiveness!
Linux needs a call which can be found on FreeBSD.
It is `rtprio’.
http://www.chedong.com/phpMan.php/man/rtprio/1
Basically, this call is `renice’ on steriods. If you use `rtprio’ on xmms, you will *never* hear a single stutter again. Period.
Try as you might to stall xmms (20 heavy disk copying sessions, a few greps of the entire filesystem, some busy network traffic etc.) your music will come first.
Those with FreeBSD give it a try:
`rtprio 16 xmms’
Now why doesn’t Linux have a way of doing this?!
As you can see he has Fedora rpm’s, but nothing for Suse? Ximian/Novel/Suse
Why Fedora?
Don’t take this the wrong way, I’m not shouting conspiracy, just curious why there arn’t suse rpms.
Seems like the opposite of a conspiracy to me.
Anyhow, SUSE is going to be the reference platform, soon. There is no secretive reason why I have not done it yet.
– rml
See http://tech9.net/rml/schedutils for the ‘chrt’ utility. It comes in Fedora and elsewhere.
Does the same thing I suspect.
– rml
Marcus Sundman is being inflammatory but is he raising valid points?
See http://tech9.net/rml/schedutils for the ‘chrt’ utility.
Thanks, I’ll check it out! *aptitude install schedutils*
This won’t work with all interactive programs, but at least I might be able to listen to music while waiting for redraws. š
In my experience he is maybe right about the I/O. Copying large files makes my GUI (Gnome menus and stuff) freeze. (And rendering antialiased text can make my mouse pointer jump around.) Not impressive.
Gentoo, 2.6.0-mm-something.
Athlon XP 20000+
On articles this long I usually like to print them out to read them, why does Ars Technica allow this ? If it’s a advertisement issue they could print the ads too and I wouldn’t care. But with their black background it’s a real nuisance…
rml;
I only said conspiracy cause I figured if I just left it at fedora, people here would start shouting about it. š
I was just curious, I have a box running Fedora that i test on, is there anything I should know about if I want to install these packages? Is it possible to use the .src.rpm’s on SUSE are the other deps that SUSE will need?
Great work btw!
Since SUSE is going to be the reference platform, and Novel baught SUSE and Ximian. Care to share what will become of the work SUSE has done for KDE and how much more intergration we will see with Gnome in SUSE?
I keep leaving out an “l”
š
There are two ways around that. You can use a web browser that lets you specify a custom style sheet for a page, or you can subscribe to Ars Technica to get access to their entire database of articles in PDF form.
Are you going to have a dir or do you have one under suse/people?
ftp://ftp.suse.com/pub/people
Marcus Sundman is being inflammatory but is he raising valid points?
No. He didn’t really say much, but I got two things:
Desktop performance is not I/O bound. I never said it was. I said that processes which spend most of their time in I/O wait are, by definition, interactive and that we should somehow favor those programs. E.g., we should run your text editor the instant you type into it, not a minute later. Right? Obvious?
Swapping is bad. Of course it is. We went a long way to improving page management in 2.6 and making it fairer, although swapping is never good.
Are you going to have a dir or do you have one under suse/people?
No.
Robert, the Guy obviously has never even had an operating systems class, and knows just enough to misunderstand the article to mean something else.
“What a load of BS! Responsiveness in GUIs are not in any way bound by I/O performance. When my email client tries to open a window at the same time as two other programs are accessing the HD (e.g. dpkg and cp), guess which one will suffer? It’s because of moronic ideas like these by Mr Love that my xmms suffers from 1-2 second pauses every 2-3 seconds (even though I use a large I/O buffer, and have reniced xmms to -19)”
Marcus- First off, I/O Bound does not just mean disk I/O. Second, your xmms issue is not what you think it is, as I can play mp3s fine on my 400mhz G3 pbook with kernel 2.6.1 fine and I’ve yet to have it skip under load (and trust me, with 128MB of RAM its swapping a lot). Make sure your sound driver doesn’t have issues, and that your X server isn’t -10.
Your GUI responsiveness DOES depend on you I/O stuff. If you move stuff around then with the 2.4 your mouse can stutter alot. On 2.6 it should not if you have preemptive kernel and using the antipacitory scheduler.
And the 2-3 second is realy weird, couse I never get those. DO not renice you xmms.
To get a usable desktop make sure you got DMA setup on you hardrives. check that like this:
$ hdparm -d /dev/hda/b/c/…
It should say: using_dma = 1 (on)
And make sure you have compiled you kernel preemtive support in: Processor type and features->Preemtive Kernel.
And compile and use the Anticipatory scheduler.
With Kernel 2.6 it is important that you dont nice up your X and other programs. Make sure X is not niced up.
The stuff with the GUI being slow… sounds like you have the wrong drivers perhaps loaded?
If you are using an Nvidia card try this:
$ cat /etc/X11/XF86Config | grep nv
If it says: Driver “nvidia”
Then you are using the right hardwareaccelerated driver from nvidia. If it says: Driver “nv”
Then you are using XFree’s not so fast one.
Trust me that your comp does not seem right setuped.
Try this:
$ hdparm -t /dev/hda
You should have a *ok* value for the speed. I get 45mb/sec with 2.6.1.
Good Luck.
Sounds like this Robert Love character is someone to be reckoned with. I’m looking forward to using the 2.6 kernel, it sounds like they did a number on that one. I’ll probably also read one of those books on Linux internals, but right now I’m reading Raymonds ‘The Art Of Unix Programming’. I noticed that Robert Love used some of the terminology such as glue layer, mechanism, policy, etc. He sounds like a Unix developer, he sounds like the real deal.
From the article:
Those of you who have tried the new 2.6 Linux kernels will undoubtedly have noticed how much more responsive the system feels under interactive use than earlier kernels.
I’m using right now the 2.6.0 avaliable on Debian sid repositories. I hardly feel any difference on responsiveness. What are you guys talking about? Where is the responsiveness everyone is talking about? Honestly.
Victor.
did you have the CC patches on the 2.4 kernel? also, Debian kernels are very vanilla and set to be lowest common denominator….I suggest downloading the source and configuring it yourself as some of the stuff like preemption might be turned off in the debian one.
Well, hopefully, you should not have to do this. The problem with giving apps realtime priority in that way are..
1 If they crash and take all the cpu time, you computer locks up as no other task can run. If you give other a few tasks even higher priority, then you are back in the same place you started, and effectively now have a round robin schedular.
2 Only root can give apps real time priority, so if there are any security holes in your media players, it’s a bit of a problem. Root should not be running entertainment apps.
The Linux audio community is working on ways around this too. At the moment, for good low latency performance, Jack and you apps like Ardour all have to be run as root.
Ie, you launch Jack with -R for realtime performance.
The way people are working around this is to use capabilities “A capability is a token used by a process to prove that it is allowed to do an operation on an object.” 2.6 has support for this, but I think getting it to work is a little messy right now. I just run all my sequencer stuff as root on Linux for the moment, like I have to do in windows and it works OK. It’s still not ideal though…
I don’t know if there is any definitive guide, but this mailing list entry may be of interest..
http://www.music.columbia.edu/pipermail/linux-audio-user/2003-Augus…
Ideally, the playback code and buffering in XMMS should be fixed to play nicer with ALSA and we would not have this problem. I use alsaplayer, which imho is better behaved here.
Doh! Re-read that man page and I think that root is needed to set the priority, but the app does not need to run as root. Could someone who knows more about BSD than me (not hard
clarify this please?
I don’t have much confidence in them achieving ‘utopia’ when a replacement to dnotify doesn’t even sound like its really on the table. For most users dnotify/fam is the cause of most annoyances when mounting media. Having something that mounts things for you nicely is useless if after using it you can’t unmount it.
> Ideally, the playback code and buffering in XMMS should be fixed to play nicer with ALSA and we would not have this problem. I use alsaplayer, which imho is better behaved here.
Yup, an audio player shouldn’t need to run as a real-time task in the normal case. It doesn’t take much CPU %.
I’m not sure how others haven’t noticed (someone was complaining), but XMMS used to skip a *lot* in 2.4. On 2.6.1 there’s no skipping even with a kernel compile.
I believe fam 2.7.0 has fixed the issue of not being able to unmount devices…
Can someone confirm?
I’m not going to evaluate or judge 2.6 until the major distributions roll it out in March.
did you have the CC patches on the 2.4 kernel? also, Debian kernels are very vanilla and set to be lowest common denominator….I suggest downloading the source and configuring it yourself as some of the stuff like preemption might be turned off in the debian one.
Taking a look at /boot/config-2.6.0-1-686 i find:
CONFIG_PREEMPT=y
So i suppose preemption is enabled.
I’m yet to see this “2.6 high responsiveness”…
Victor.
It’s there though dude, you can’t deny the source code. The scheduler is more responsive, and able to handle a heavier load. If you ran 10+ processors, it would scale better than 2.4. In addition threading carries less penalty, so your Java applications will run faster. I hear that Morton and Torvalds are still managing 300+ changes per release candidate, so in March the real testing will happen, when corporations upgrade to 2.6. A lot of work and a lot of attention is being shifted to the Linux desktop, so we can expect a lot of good things to happen. The 2.6 kernel is a large step toward hardware compatibility and Linux is also very versatile, it can be used for embedded development and with a preemtible kernel, that means that soft real time applications will be accomodated. So hold on to your horses, because Linux development is undergoing heavy R&D.
I wouldn’t mind if OSDL had more of an investment into XFree86. At least enough interest in it to document xlib.
the Guy obviously has never even had an operating systems class
I you by “the Guy” mean me then that is a lie.
First off, I/O Bound does not just mean disk I/O.
I know, but I don’t know enough about said subsystem in linux to know exactly what sucks.
Second, your xmms issue is not what you think it is, as I can play mp3s fine on my 400mhz G3 pbook with kernel 2.6.1 fine and I’ve yet to have it skip under load (and trust me, with 128MB of RAM its swapping a lot).
I have no idea what it is, I just know that everything dies when disk I/O gets higher. (10 MiB/s over ethernet is fine, but 8 MiB/s total HD access is not (assuming more than one program.)
Make sure your sound driver doesn’t have issues
It’s not just the sound that stutters. Pretty much everything suffer the same hiccups. That includes everything from the mouse pointer to gif anims in a browser window.
and that your X server isn’t -10.
It’s -8.
On 2.6 it should not if you have preemptive kernel and using the antipacitory scheduler.
I also think that it shouldn’t, but it does. š
make sure you got DMA setup on you hardrives
Of course I use DMA. I’m not that ignorant. š
make sure you have compiled you kernel preemtive support in
Of course.
And compile and use the Anticipatory scheduler
I don’t know how to use or not to use it, but I was under the impression that it’s the default I/O scheduler. (And of course I have “CONFIG_IOSCHED_AS=y”.)
If you are using an Nvidia card
Nope, I have ATI. Same result with “ati” as with “r128”. I haven’t tried the proprietary drivers, though.
$ hdparm -t /dev/hda
You should have a *ok* value for the speed. I get 45mb/sec with 2.6.1.
“/dev/hda: Timing buffered disk reads: 50 MB in 3.12 seconds = 16.01 MB/sec”
(Note that this is a 2.5″ HD on a laptop, though.)
I know, but I don’t know enough about said subsystem in linux to know exactly what sucks.
———–
You’re missing the fundemental point. An app that is I/O bound does not necessarily do a lot of disk I/O. An app that is I/O bound spends more time waiting for I/O than it does computing. Interactive tasks are by definition I/O bound, because they spend most of their time waiting for the user. When you favor I/O bound tasks, you favor interactive tasks. Look at it this way:
– Your e-mail client is blocked waiting for events from the keyboard.
– You type something in, which fills the keyboard buffer and unblocks the e-mail app.
– Now, if the kernel recognizes that the e-mail client is I/O bound, it will give it preference, and immediately schedule it to run.
– Your e-mail client processes the keyboard input, and blocks waiting for more.
Favoring I/O bound processes is precisely about favoring interactivity over throughput.
I have no idea what it is, I just know that everything dies when disk I/O gets higher. (10 MiB/s over ethernet is fine, but 8 MiB/s total HD access is not (assuming more than one program.)
——–
What kernel are you using? Ever since I started using the preemptive patches, I have yet to hear audio skip.
It’s not just the sound that stutters. Pretty much everything suffer the same hiccups. That includes everything from the mouse pointer to gif anims in a browser window.
———-
It really sounds like you have DMA disabled on your hard drive. These are pretty much the classic symptoms.
It’s -8.
———–
If its kernel 2.6.x, it should be 0. The new kernel does not require priority hacks for X. Making it -8 actually prevents the scheduler from balancing things correctly. When 2.6.x-based distros ship, they should have X run at 0 by default. Debian’s Debconf already detects if you have a 2.6 kernel and disables the XFree priority boost.
An app that is I/O bound does not necessarily do a lot of disk I/O. An app that is I/O bound spends more time waiting for I/O than it does computing.
I know, but that I/O doesn’t need much throughput, but low latency. I don’t care if it takes 5ms or 25ms for a program to react to my right mouse click. However, I do care about cp not hogging all resources so that it takes 10 seconds for the right click menu to be drawn. cp is very I/O bound, yet I couldn’t care less if it took it twice as long to copy the file as long as it won’t affect my email client drawing the right click menu.
– You type something in, which fills the keyboard buffer and unblocks the e-mail app.
– Now, if the kernel recognizes that the e-mail client is I/O bound, it will give it preference, and immediately schedule it to run.
So howcome everything goes slow motion when I run a few “cp Matrix.avi Matrix-backup.avi”-like processes?
What kernel are you using?/i>
2.6.1
[i]It really sounds like you have DMA disabled on your hard drive.
# hdparm -d 0 /dev/hda && hdparm -t /dev/hda
=> 3.77 MB/sec
# hdparm -d 1 /dev/hda && hdparm -t /dev/hda
=> 15.44 MB/sec
DMA is definitely switched on. Besides, there is no CPU load even when there is a lot of HD I/O, but in PIO mode there is. And, as I said, HD I/O throughput is much better in 2.6 than in 2.4.
The worst case of slow motion I get when I’m short on memory and running a large java program. After a while some parts of the java program have been paged out and then when it starts doing a complete garbage collection everything turns into ultra slow motion. I’ve had stop-the-world GC pauses of over an hour, while it is swapping non-stop and the system is completely unusable. Nice DoS “attack”, wouldn’t you say?
> /dev/hda: Timing buffered disk reads: 50 MB in 3.12 seconds = 16.01 MB/sec”
(Note that this is a 2.5″ HD on a laptop, though.)
This is very sucky. I assume you ran hdparm with all applications idle? If not, please rerun and post the results.
> don’t know how to use or not to use it, but I was under the impression that it’s the default I/O scheduler. (And of course I have “CONFIG_IOSCHED_AS=y”.)
Pass elevator=as in the kernel command line, if I remember correctly. Besides, isn’t this the default with 2.6? Check your dmesg, it should say something like ‘Using anticipatory IO scheduler’.
And as mentioned above, please let X run at nice 0. The new scheduler’s default behavior will DTRT.
Most of all, what DE do you use? XFCE is pretty zippy if you care to try it out.
> The worst case of slow motion I get when I’m short on memory and running a large java program.
You can change the JVM gc behavior, hope this helps:
http://java.sun.com/docs/hotspot/gc/
> So howcome everything goes slow motion when I run a few “cp Matrix.avi Matrix-backup.avi”-like processes?
That’s a transfer of about 750 MB or so, right? It’s likely that you’re running into swapping, and that’s bad with your terrible IO bandwidth. What does vmstat 1 say?
You may wanna try XFS instead of ext3.
What’s the spec on your box? Memory / bus bandwidth / CPU.
> > /dev/hda: Timing buffered disk reads: 50 MB in 3.12 seconds = 16.01 MB/sec”
> > (Note that this is a 2.5″ HD on a laptop, though.)
>
> This is very sucky.
I know. Laptop HDs usually suck.
You can change the JVM gc behavior
I know. I used to use -XX:+UseConcMarkSweepGC, but that is even worse when 2.6 starts using the swap.
a transfer of about 750 MB or so, right? It’s likely that you’re running into swapping
Are you saying that pages are swapped out to make space for large I/O buffers on linux? I thought only MS is that stupid.
You may wanna try XFS instead of ext3.
Sure, as soon as the people making HD tools (non-destructive partition resizers, recovery tools, etc.) starts supporting XFS.
What’s the spec on your box? Memory / bus bandwidth / CPU
256 MiB / don’t know / 850 MHz P3.
This is pretty much irrelevant, though. The performance is still at least 10000% of that of my old A500, yet the A500 is more responsive.
An app that is I/O bound spends more time waiting for I/O than it does computing. Interactive tasks are by definition I/O bound, because they spend most of their time waiting for the user.
Sigh.. Interactive applications don’t spend time waiting for I/O because of a slow I/O subsystem (unlike programs like cp), but because of a slow user. E.g. simply boosting the priority for all I/O bound programs is clearly not a solution. We want the email client to respond fast to user input no matter how many files are being copied at the same time.
It’s the latency that matters, not the throughput.
(By the way, I’m sorry for turning this discussion into a support forum. Perhaps we could continue the support and improvement suggestions someplace else, or even per email. Eugenia: it’d be nice if OSnews had threaded comments so that people could easily fold away threads of no interest.)
> Sure, as soon as the people making HD tools (non-destructive partition resizers, recovery tools, etc.) starts supporting XFS.
Well, you *could* make a backup
And the XFS developers could sure use more sheep^W testers.
> 256 MiB / don’t know / 850 MHz P3.
This is pretty much irrelevant, though. The performance is still at least 10000% of that of my old A500, yet the A500 is more responsive.
Umm sorry, I don’t know what an A500 is so can’t say what’s up. Your box is ok, apart from the IO b/w.
> Are you saying that pages are swapped out to make space for large I/O buffers on linux?
There is no separate buffer cache.
You’re probably better off posting a bug report to LKML, assuming you’re interested in some sort of resolution.
Under XP, moving a large file from one drive partition to another has a great impact on application launch time.
With all non-essential services stopped and only one other
program running, IE takes about 7 secs to launch and load the home page. While the file ( >500 megs) is being copied,
mouse responsiveness is normal but it takes 4 to 8 seconds longer for Taskbar menus to pop up, a little less time to highlight a menu option ( slow change of focus) and more than 30 seconds for IE to launch and load.
This is a Dell Optiplex GX150 ( Celeron 900mhz), 320 Meg PC133 SDRAM, 20G ATA-100 divided into 2 equal partitions with 1.5G free on the destination and 750 meg free on the source partition
> > Sure, as soon as the people making HD tools (non-destructive
> > partition resizers, recovery tools, etc.) starts supporting XFS.
>
> Well, you *could* make a backup
Actually I would very much like to have my non-OS directories on two of my computers synchronized. This would make it easy to work on either one, as well as provide a backup. However, I have yet to find a tool for making two-way sync’ing on live file systems (i.e. while there might be applications modifying files on both computers at the time you are sync’ing). I’ve been thinking of making such a tool, though, but I also have some other projects with higher priority so it’ll have to wait (and perhaps dnotify might be replaced by something nicer by then). 24h/day is way too little. š
Well, I couldn’t believe some of the things being said about the lack of responsiveness of a modern Linux distro so, on that same Dell Optiplex mentioned two posts up, I installed Mandrake 9.1 ( not the latest Mandrake) onto a drive containing Win XP. Linux is installed in the middle third of the drive, XP has the first 5 gigs, and there is a 10 gig NTFS partition at the end of the drive for downloads.
After installing the OS, I used Mandrake Update to get the most recent fixes for this version. I took all but one suggested update ( kdepim) for a total of 500 meg. Everything installed flawlessly – no reboot required.
Next, while using XMMS to play a list of mp3s that ran to about 40 min of music, I SIMULTANEOUSLY copied, from the 2 NTFS partitions, the entire Windowssystem32 folder to the var directory and a 450 meg iso to Mandrake’s root.
My XP system32 folder ( and subfolders) is about 600 meg and contains over 4500 files. Oh, I was also copying the entire RPMS3 folder from the third Mandrake 9.1 iso to my home dir – I have 3 screenshots, if anyone would like visual evidence. E-mail bannor99-at-H0tm4i1.c0m, if you care
to see them.
Oh, and I was surfing the ‘net as well.
Look, as a longtime Linux user, I’m well aware of the problems but it’s a hell of a lot better than what SOME would like you to believe.
But, look how far it’s come in a such a short time – how long did Windows take to reach a comparable state of usability? Never fear, the penguin will rule, one day.