Today I thought about how I can make my very own application to only link against those libraries that it really requires and not those it get provided by PKG-CONFIG.Problem-case:
When we use pkgconfig within our programs which of course makes sense, we also see that other libraries from other packages are being processed to our *_LIBS variables. So we end up in linking a lot of libraries to our application which we really do not need and which also causes (so I assume) a lot of overhead. After thinking for a while I looked some programs up by using:
readelf -d /usr/local/bin/gnome-terminal |grep NEEDED | wc -l
And saw that it was requiring over 52 libraries. I then recompiled gnome-terminal with
export CFLAGS = “-Os -s -Wl,–as-needed”
Pay attention to the –as-needed and the resulting binary required just 21 libraries afterwards instead of the 52 before because it only linked those really required to the final executable. I continued this with vte and saw that it required 29 libraries initially but after the –as-needed flag it went down to 7 libraries.
Now imagine this for the entire GNOME desktop or the entire KDE desktop and also package management systems which will reduce dependency tracking a lot. The problem with this method is that you surely don’t keep track of API compatibility but a good package management system will certainly make sure that the packages packed belong to a certain desktop version.
How much does this save/boost performance? Do you see any performance benefits from doing this? Are you able to measure anything?
I would really like to see a Gentooer do this and post there results, it may just be me hehe.
–Ashley
We need more tips and tricks like this. I’m definately going to experiment with this on my Gentoo box.
I’ve just had a quick go with Konsole, and it’s dropped from 40 libraries to 2.
That’s almost enough to be scary – although when you think about it, it didn’t really need libkdewallet…
I think it’s loading quicker, but it’s quite hard to tell since I haven’t done any serious tests. Plus there’s the question as to whether it was cached or not…
I’m a little hesitant because, well, I wonder if this is going to break something down the line. Can’t really see any reason for it to unless it drops a library that’s actually needed, but you never know.
Oh, and the author seems to have fallen into the common trap of -0s instead of -Os (zero vs. the letter O) 🙂
I am the author of this stuff and dealt with it the whole day, talking with some GNOMERS and of course some KDE people who gave me good hints and tipps as well. So basicly the –as-needed tip came from the KDE side.
The intend was to compile a tool that I liked to provide in binary for in a way that it only has the minimal needed libs compiled in it. Pkgconfig is of course a nice thing but it adds plenty of other libraries to the LIBS variable and makes your app keep references of these libs as well. So when you want to release binary only you need to be sure that everyone on his different distribution offers the same version of the libraries and this can easily reach more than 50 different types of libraries while your application’s code only require 2-3 libraries and the other libraries inherit from other libraries.
The benefits with -Wl,–as-needed is that it only links those libraries that are really required to your binary and reduces overhead in the code (I easily gained a couple of hundret of bytes with this, made the application lighter and doesn’t keep references to unneeded stuff). If it makes stuff truly noticable become faster is a matter of question and can only be valued if the entire OS+Tools+GUI (KDE/GNOME) is compiled with that.
When people talk about gnome-terminal depending on 50-60 libraries then this is mostly not correctly true because it shows the dependency of that program and references through all the libs it really requires. So what get shown is not necessarily what gets dlopen()’ed. Using the ‘readelf -d foobar’ command gives more feedback here (also thanks to the KDE guy for information). And you see that gnome-terminal is cluttered with tons of libraries and the information NEEDED. But NEEDED in this case is meant for ‘linked in’ and not TRULY needed. -Wl,–as-needed will change this behavior and links the REAL needed librararies to a binary and thus reduces the clutter within the file. A ldd gnome-terminal will surely throw out another huge dependency tree. But this also requires some knowledge how the file get handled if opened. What happens when the file gets loaded, will it wade through all the libraries linked towards it and open a reference ? Dunno.
By running profilers on some gnome apps I figured out that font-config took up most of the startup time. So I deleted most of my bitmap fonts (You have to leave “misc” on gentoo) and applications started up much more quickly.
I’ll try this tip here as well
Adding something I forgot and thanks for Arcangel mentioning it. It was a typo when writing this but here a line that I keep experimenting with at the moment. It optimizes in size, strips the junk, pipes the data instead writing it to harddisk, unrolls the loops and links against the stuff really needed.
export CFLAGS=”-Os -s -w -pipe -fomit-frame-pointer -funroll-loops -Wl,–as-needed”
http://www.ubuntuforums.org/showthread.php?t=17287
I am no ubuntu user and thus not participating to their forums. Though this stuff ain’t rocket science either but good to know that there are more people up to this. It will certainly help declutter a lot of distributions and declutter dependency a lot.
There is one caveat by this. There is no way to check whether the API of the other libraries on your system is still valid to what your application requires but on a good system this should be of no problem.
When loading an application that has been linked to bunch of unused libraries, all those libraries will be loaded first. Adding in memory usage (all these files get cached too) and startup times.
I hope Archlinux will adopt this soon.
I have given up hope that Gnome will ever be lightning fast. It doesn’t seems to be moving in that direction at all — more towards bloat ala KDE. If you have Fedora installed I would just pull up a shell and enter ‘yum install xfce’ and then login to an XFCE session. Then I would make either desktop icons or put application launchers on the XFCE4 taskbar for all your favorite Gnome utilities. Now you have speed and ease of use of Gnome.
The word “bloated” that was used with products from another large vendor of desktop OS:es can today only be applied to linux.
Today I tried to remove an –insignificant– app (I thought). So I typed (FC3), yum erase howl. Da, da, da… “do you want to remove this”, and “this” was the entire gnome system, openoffice, thunderbird, firefox and more. Well, I had to say no I guess to keep the system upp and running. Essentially the same happend when trying “yum erase evolution-data-server”.
Linux is really no fun today. I have 1GB RAM, 2GHz AMD64 and GNOME brings this machine to it’s knees (if it had any). After a few days, the swap use is at 1GB and things start to take time. We are two people using this machine, me and my wife. When I stoped using GNOME, swap use was down to a few hundred megs.
Theres a good reason Linux does not gain in the desktop space. It’s simply not a solution (yet). Look at firefox though. It has something to offer and it gets used. When linux (desktop) has something to offer, it too will be used.
/jarek
Jarek,
I have the same system as you do: 1GB RAM, 2GHz AMD64. I frequently have many programs open and never close them, and leave my computer on for weeks. Gnome is quite fast for me with fedora core 3 with all the updates. It *never* uses swap. This leads me to believe that there is something wrong with your setup. The worst memory consumer on my system is firefox, but this is getting better with every release since 1.0 (by this I mean the memory leakage problem).
Jarek: Sorry of your misfortune, but Gnome is probably not the problem in your case, or at least not alone. I have similar config for a year now, and I never been more happy. Not to start a flamewar but FC3 might be your problem. Some others distro give you more control over your system… hint, hint
I would like you to point out what is exactly bloated in KDE. No, 3 text editors does not make it. Cluttered toolbars don’t make it slow either.
If people what to know about
unused shared libs and what they do for performance,
the best source is dsohowto.pdf (google for it).
On a modern linux system you can find out
about unused shared libs by doing
ldd -u -r BINARY_OR_LIB
These unused libs affect mostly the startup time.
Most of them are not unused, they are used by one of the library dependencies, still
they increase the startup time.
This seems obvious. Why wouldn’t it be the default action of gcc to automatically NOT wastefully compile in code to load unneeded libraries?
What does GCC have to do with pkgconfig?
What does pkgconfig have to do with -Wl,–as-needed?
I have given up hope that Gnome will ever be lightning fast. It doesn’t seems to be moving in that direction at all — more towards bloat ala KDE.
Funny because despite all the claimed bloat in KDE it is lightning fast.
KDE is getting faster each release. And with GCC 4.0 (3.4 with backport) removing the loading time, it is already much faster then Gnome for me. Qt 4 is expected to have a 30% increase in performance, and to have separate libraries for ui and core functions, making it very fast.
So please stop saying that KDE is bloated: it is the fastest full featured desktop available, period. In my machine, it is way faster than XP or Gnome. (AMD 2600+ 512M Shuttle box). It is a lean and mean system, with very little duplicated code. Now deal with it.
Please do not confuse a colorful/heavy user interface with code bloat. KDE code is not bloated at all. And the user interface is improving as well. Sane defaults is the goal, KDE is not there yet, but is getting there. Compare KDE terminal (Konsole) x Gnome terminal, KEdit x GEdit, Nautilus x Konqueror, and you will see that the KDE programs are faster and uses less memory.
If you don’t believe it, try KDE 3.4 x Gnome in Ubuntu…
>> Funny because despite all the claimed bloat in KDE it is lightning fast.<<
Explain?
>Explain?
KDE code / classes avoid a lot of duplication, and are widely used inside KDE. KDE focused a lot in optimization in the last releases. And a GCC C++ weakness is being corrected by GCC 4.0: now you can export only the symbols you need, making the C++ bynaries much smaller and making them start much faster. No need for pre-link hacks, and the number of symbols is now more or less the same as in windows dlls. KDE gets a majopr boost to performance this way. But you do not have to believe me, try it yourself.
Okay, I tell ya what. I will try KDE again when Fedora 4 comes out and GCC 4.0 is shipped with it.
I used KDE that came with FC2 and I was suprised at how long it took for applications like Konqueror and Konsole to open. You have to keep in mind not all of us are running 64-bit 2 Gig. Athlon chips. I am still using my old Thunderbird 1.2 Gig. Athlon chip on a 512MB system. I honestly gave all three desktops a try: KDE, Gnome, and XFCE4 and in the end I switched to XFCE4 since it ran the fastest and ran Gnome utility applications well. KDE user interface design has always scared me. Everytime I open a KDE application up for the first time there are a billion icons and toolbars everywhere. I hate the fact that it forces the user to trim down all this UI bloat before it is usable.
[i]KDE is faster[i]
I agree, but speed is not the only reason I use my DE, GNOME may not be as fast as KDE but is aceptable and certainly not a showstopper for all the feautures GNOME offers to me, so despite GNOME is not as fast as KDE, for me is still the best free DE around.
>>But you do not have to believe me, try it yourself.<<
I did, after reading all the reviews raving about performance and a “50% cut in start-up time”. Here’s what I found:
Specs: Celeron 900 MHz (Coppermine), 128 MB RAM (p.o.s., I know, but not uncommon). Slackware 10.1, KDE 3.4 (from linuxpackages.net), and Dropline Gnome 2.8.3.
Start-up time (from the moment I type startx): 38 sec for KDE, 41 sec for Gnome. Opening the home directory with Nautilus: 9 sec, with Konqueror: 8 sec. In general, KDE/Qt apps do seem to load a bit faster than gtk apps do in Gnome. Trails when moving windows around are equally bad in both.
So, is KDE faster than Gnome? Yes.
Is it “lightning fast”? Hell no.
Is the difference in performance big enough to justify a switch for someone who likes the Gnome interface? I don’t think so.
Sure Gnome could use a speed-up, but KDE’s performance is way overhyped, IMO.
Sorry, but with 128 Mb RAM you can’t really expect anything with KDE. KDE needs LOTS of memory.
Anyway, I agree with Amadeo. KDE’s performance improves at an impressive rate.
I just upgraded my home computer (pentium 4 1.7 512 MB) from KDE 3.2 to 3.4 — compiled with the fvisibility stuff of GCC 3.4/4.0 — and it was like I did a CPU upgrade.
Kudos to the KDE team, now I can postpone buying a new computer for at least a year!
What takes more time — loading the shared libs into memory or unnecessary dependency tracking for apps?
If the former, why not load some unnecessary libs up front, and save time later; if the latter, perhaps “–as-needed” has a benefit.
“Sorry, but with 128 Mb RAM you can’t really expect anything with KDE. KDE needs LOTS of memory.
Anyway, I agree with Amadeo. KDE’s performance improves at an impressive rate.”
This is a widespread myth. It’s simply not true.
KDE is very efficient, for what it does, it definitely does not need LOTS of memory.
I have very good experience with running KDE 3.x on an old laptop with less than 128 MB RAM (128 MB, but 8-16 MB shared with the graphics card). And it’s perfectly usable and quite fast. Definitely much faster than Windows XP on the same machine, as Windows XP, just like GNOME and Mac OS X need much more memory than KDE. Yet, people still spread myths that KDE is memory-hungry. It is not.
Just load the default KDE desktop in a plain simple distro like Slackware or Arch and watch the system statistics. The memory consumption is fairly low (cache doesn’t count). When in doubt, compare with GNOME.
And as you use various KDE programs, the memory usage doesn’t increase that much thanks to the code/library reuse. If you use some “lightweight” window manager and run severeal different apps in it (like Firefox, OO.o etc.), you’ll actually need more RAM than you would if you used KDE-only environment.
Watch out, -Wl,–as-needed can and will break somethings.
OpenJade breaks with it.
But I have noticed better performence on a few apps that ive rebuilt with that added to my CFLAGS.
Be aware that –as-neede requires you to have installed binutils >= 2.15
Today I’ll test it on my Gentoo Linux box.
last time i tried xfce it was pointless and bloated. go with any WM of your choice and rox. rox is the fastest, most responsive file manager i have ever used. yeah, it doesnt have a built in cd-burner, and it doesnt double as your web browser, but that is the point with this thing. simple fast file management.
http://rox.sourceforge.net/phpwiki/
I have recently played around with Gnome 2.10 on my test system. First thing I noticed was a speed increase. It also used a lot less ram. In 2.8.3 it used about 400+ megs of my 512 ram. Mind you about half was cahced. Now under 2.10 it only used 96megs of ram, with some cached. It was really quite amazing. XFCE4 used the same amount as a full blown 2.10 Gnome setup.
I marked you as abusive as this is totally off topic. This doesn’t mention rox at all, yet you feel the need to interject it. This story was about pkg-config and speeding up gnome. Not how blazing fast and lightweigh Rox is. Yes I agree it is lightweight, but it’s NOT on topic.
Sorry, just sick of the “hey why not just drop the topic and go with my favorite choice” interjections. And i will continue doing this on every thread i read…
Claims about this or that being faster need to be proved with numbers and comparing different DE’s on the same same system like orfanota did.
No they don’t actually. This is not a scientific investigation and posting numbers which are unverifiable since they cannot be repeated (We don’t have his hardware), and also since it has not been tested on a broad range of machines eliminating potential machine specific issues. Lastly I can make numbers up easily.
My anecdotal evidence of using KDE and Gnome of several machines including a PPC machine leads me to my conclusion that Gnome is sluggish but acceptable on powerful machines and KDE is lightening fast on relatively powerful machine.
I will admit though that I have not tested KDE on any machine with less than 256 megs of RAM.
This is gonna make a tiny speedup in startup speed, if anything. Opening Gnome apps from KDE may see the biggest speed increase.
Yea, I have about 195MB of memory in use; with Gnome having run for about 12 hours. And I’ve got firefox, gaim, and xchat open with a rediculously high quality background.
Still seems a bit high to me; but Gnome always has been. Gnome’s footprint has gone way down from 2.2 to 2.8 as well; I noticed when I upgraded my laptop from RHEL3 to RHEL4 that suddenly there was a lot less swapping going on!
People criticize gnome for it’s imperfections; they’re rarely based on more than a grain of truth IME. Gnome is a very nice stable desktop; although they did ruin the menus in 2.10
.
XP’s gui can be tweaked to be fine in 96mb or less of ram. I know, I have pc’s running it with 64mb. I’m sure any DE can be setup to run nicely in little ammounts of ram. XPs GUI slowness comes from crappy defaults.. 400ms MenuShowDelay? wtf? just edit the registry and itll become lightning fast
There appears to be some problems that will come from this doing it on every program. Interesting discussion
http://thread.gmane.org/gmane.linux.redhat.fedora.testers/24698
I didn’t make those numbers up. They are the averages of four startups of each DE, each after a fresh reboot, and all four were very close. Any hardware-specific issues would probably have affected both DEs. Why don’t you post some of your numbers instead of suggesting that I faked mine and then presenting us with your “anecdotal evidence”.
How many times do I need to reiterate the “GNOME/GTK+ is slow” is a troll?
I’m trying this out on my gentoo box (AMD64 3200/512MB DDR), and I seem to have gained at LEAST 5 seconds on Firefox and Evolution startup times.
RAM usage seems to be dropping as well.
“How many times do I need to reiterate the “GNOME/GTK+ is slow” is a troll?”
When so many people, independently, say it’s slow, when every discussion about GTK+/GNOME is flooded with these posts about GTK+, when so many people experience it – yes, they do, on their own computers (don’t say they don’t experience it – this is their experience, not yours)… I suggest it could perhaps even be true then.
you’re running Gentoo on a 3200+ / 512MB and you’ve got five seconds to *lose* from Firefox startup? Dude, you have problems. Firefox doesn’t take five seconds to start on my 2500+ HTPC. What hard disk do you have in that thing?
Damm dude, on my AMD 3000(32bit) 512MB ram, I can get firefox to load in about 1 sec. And thunderbird.
user 0m1.423s
sys 0m0.113s
Thunderbird.
I’ve recompiled 3/4 of my system with the new flag (Gentoo). Very impressive results. The GNOME desktop with no applications running takes up about 61megs. The desktop and applications feel and run smoother, lighter and faster.
There are some caveats. I had some problems with gtk-engines not finding the GTK-1.2 libraries and as such failing to compile. An sgml util package also failed to compile because of something to do with openjade.
Outside those two issues, everything compiled flawlessly including core system libraries like glibc, gcc et al. Most gnome packages and libraries gave warnings about ld inconsitencies, but nothing grave and nothing is borked yet.
If you are a Gentoo user, it’s worth testing it out. Just add the flag to your LDFLAGS in /etc/make.conf and recompile gnome. They don’t call us ricers for nothing.
Umh, something is wrong with your setup.
I use a Thinkpad T40 (PIV 1.5 GHz and 512 MB RAM), with FC3 and KDE 3.4 and no swap at all, even with DB2 and Websphere Developer Studio running!
Regards,
—
Nildevice.
-Wl,–as-needed breaks imlib2_loaders in Gentoo, and xine-lib rc8. I’m trying to compile E17 from CVS using portage. Maybe some site could do a wiki on -Wl,–as-needed and list all these packages that break. xine-lib 1.0 works with -Wl,–as-needed, but for imlib2_loaders-1.2.0-20050220 I had to disable it.
Since noone sofar posted numbers – a tiny overview is available here:
http://article.gmane.org/gmane.comp.t2.devel/681
Which is in the T2 SDE (http://www.t2-project.org) mailing list archive.
I don’t use gnome at all, but I did a full recompile of KDE 3.4 with this yesterday, including external apps like amarok, kvirc and k3b.
CFLAGS=”-Os -march=pentium4 -Wl,–as-needed”
CXXFLAGS=”-Os -march=pentium4 -Wl,–as-needed”
LDFLAGS=”-Wl,–as-needed”
(OK, I probably didn’t need to specify this again with an LDFLAGS variable but that’s what I did to make sure everything got it)
I did not use the -s switch for fear of breaking complex libraries, but I do use make install-strip, which does the default –strip-all for binaries and –strip-unneeded for libraries I think. I always use -Os -march=pentium4 for KDE (whole system is built with –march=pentium4), so the only change was the linker option.
Did it cut down on the linking? A little (I can’t really quantify this either for reasons stated by others, because I just compared ldd results). It also changed the order in which things are referenced. This would have probably helped immensely with KDE if we had this a few years ago. It used to be horrible, the way they linked everything with everything but they’ve really cleaned up their act that way. If I recall correctly, I think this really started to improve around KDE 2.2.2.
Did this –as-needed improve performance (startup or otherwise) perceptibly? Not really. KDE has been taught to be efficient bloatware, as others have pointed out (congrats to the KDE team), and it’s subjective anyways.
Did this break anything in KDE for me? Yes indeed. Nothing major, but so far I’ve noted that kstars and kig won’t run (they segfault). I don’t use them, I just open all the apps looking for breakage after a new build. Also, on exiting KDE the kicker is crashing. It wasn’t doing that before. (not a big deal, it doesn’t crash while KDE is running)
This system is a Pentium4 2GHz machine with 1 gig of RAM. The software is my own custom LFS system. (based on LFS 5.x but completely Linux 2.6 based and using modern versions of everything, and NPTL)
I’m not going to restore the backup I made of my previous KDE 3.4 build before I deleted it, but I don’t think I’ll use -Wl,–as-needed for KDE again.
Well, just for the archives I thought I’d best come back and say that none of the breakage in KDE apps was caused by the -Wl,–as-needed. I just hadn’t noticed them in KDE 3.4. Duh on me, I thought I had tried those apps. The kstars and kig issue was the sse2 turned on by –march=pentium4 (using -mno-sse2 for the kdeedu package fixed it). The kicker crash messages on the screen after exiting also still occur occasionally (but less frequently on this rebuild). Harmless anyways.
It’s been an interesting discussion here, and I just had to try that with KDE. It didn’t do much good, but also didn’t do any harm.