“In January we published a review of the AMD Shanghai Opteron CPUs on Linux– when we looked at four of the Opteron 2384 models. The performance of these 45nm quad-core workstation/server processors were great when compared to the earlier AMD Barcelona processors on Ubuntu Linux, but how is their performance when running Sun’s OpenSolaris operating system? Up for viewing today are dual AMD Shanghai benchmarks when running OpenSolaris 2008.11, Ubuntu 8.10, and a daily build of the forthcoming Ubuntu 9.04 release.”
I’m genuinely surprised at how poorly OpenSolaris did in their tests. It won 1/19 tests and that was java-based. Also, these seem to be somewhat realistic tests unlike the last set of benchmarks posted (how many clicks does it take to install?).
yeah, definitely surprised. I just tested the liveCD and it “felt” a bit sluggish. But I thought it was just the liveCD. Any idea why?! maybe old GCC?
Slowlaris indeed
(*no bashing intended!*)
> Any idea why?! maybe old GCC?
Some of this might very well be a GCC issue.
It would be interesting to see these tests redone using Sun’s compiler instead of GCC (which can be installed using ss-dev on OpenSolaris).
GCC 3.4 is fairly ancient that’s for sure. I know there is a plan to integrate 4.3 into OpenSolaris, I know I came across a package on opensolaris.org that was pre-built by someone that gave you this.
GCC is defintely an issue.
Definitely GCC. Not only it’s old, it’s also not ‘native’ to Solaris. GCC is developed and optimized for Linux. (Open)Solaris is built with Sun Studio. These benchmarks are always interresting, too bad this one is kind of spoiled by choosing incorrect compiler.
How is the compiler optimized to the OS? Can you give more details on that?
I would expect the standard library to be highly optimized, but not the compiler (optimizing only on a lower, hardware level).
Or does GCC on Solaris still use the GNU C Library in contrary to Sun’s?
Check explanation here:
http://www.opensolaris.org/jive/thread.jspa?threadID=91438&tstart=0
Thank you for the link but it doesn’t change the fact that the GCC included with OpenSolaris needs to be upgraded immediately – and it should have been done years ago. This has been a known problem in Solaris and yet no one can be bothered sorting it out.
Why hasn’t Sun got its act together and updated its GCC bundled with Solaris? its been a known issue for year – it isn’t as though it something that has cropped up recently. They wonder why people keep going back to Linux from Solaris – the issues are right in their face, too bad they ignore then in favour of blaming the boogyman.
> Thank you for the link but it doesn’t change
> the fact that the GCC included with OpenSolaris
> needs to be upgraded immediately – and it should
> have been done years ago. This has been a known
> problem in Solaris and yet no one can be bothered
> sorting it out.
Well, arguably you shouldn’t be using GCC on OpenSolaris anyway. You should be using the Sun Studio compilers, which are also free and can be installed from the package repository.
> Why hasn’t Sun got its act together and updated
> its GCC bundled with Solaris?
Again… Because you should be using the Sun Studio compilers with OpenSolaris. Not GCC.
It’s not a question of Sun not having their act together here. It’s a question of users who switch from Linux not grokking that there are better compilers out there than GCC that they should be using instead when they switch to a different OS.
Edited 2009-02-10 15:16 UTC
Because the Sun compiler isn’t compatible with everything out there; XChat will not compile with the Sun compile unless you muck around with a setting inside one of the source code files – IIRC relating to the NTLM proxy authentication. That is one example – I’m sure there are many more out there too.
People would use something other than GCC once their compiler is actually compatible with the source code out there. Yes, the code should be compliant out of the tar ball but at the same time to expect that every project should change their code so that works out of the box with the Sun Studio compiler is just ludicrous. The code is there, for better or worse, its up to Sun to get their compiler to compile code that would compile with gcc. If the code isn’t compiling, all the elongated intellectual wankery regarding the superiority of Studio Compiler will be thrown out the window in frustration when someone can’t get their opensource application to compile.
The aim should be that people can grab a random application with source code (that compiles on Linux and *BSD) – once it can compile out of the box and with their compilers, then they can say there has been progress. Want proof of the cl*sterf*ck that is Studio compiler, check out the patches upon patches for JDS, pkgbuild’s opensource repository, and some things that just can’t be built using Studio Compiler – then maybe you’ll change your tune.
Edited 2009-02-10 15:46 UTC
My guess is they are much more familiar with Linux and did the comparison based on default installations. It’s the old argument of whether each OS should be optimized by experts for benchmarking, or whether each OS should ship optimized by default.
There is a good argument for each method depending on who your intended customer is. For Desktop use, the OS should definitely be shipped pre-optimized as much as possible. For experienced UNIX server admins, they often like to optimize and customize things themselves so they know exactly how their system is set up.
I wonder what the result would be if Linux used GCC v3.x vs Linux GCC v4.x? I bet that GCC v3.x would loose big. Therefore that bench is not really fair.
I remember the old benchmark there, OpenSolaris vs FreeBSD vs Linux, and OpenSolaris used 32 bit binaries vs Linux 64 bit binaries! Someone compiled the same binaries on OpenSolaris, 32bit and 64bit – and his benchmarks doubled going from 32bit to 64bit! That means that the OpenSolaris benches should also have been doubled, easily winning.
I think the phoronix tester is quite dumb. Seriously.
Someone posted evidence that Linux is way faster than Solaris; an article migrated 800MHz SPARC to 2.8 dual CPU intel Xeon Linux and concluded that Linux is faster. Quite dumb. I wonder, why are there so many dumb people in this world? Seriously. I mean it? How in earth can you compare 800MHz CPU vs 2.8GHz CPU? If you do that, you are really really stupid.
Not to start a flame war, but it seems clear that the people who ran this benchmark didn’t know what the hell they were doing:
(A) They ran it on OpenSolaris using an outdated GCC.
(B) They never considered that the native Sun compiler might be faster.
(C) They never bothered to check whether Ubuntu GCC was producing 64 bit binaries by default, and that was not happening on OpenSolaris.
Ultimately, this comes down to a test that was biased in favor of Linux to begin with. And the test is totally meaningless because of that.
Edited 2009-02-10 00:56 UTC
The phoronix test suite is still under development.
One crucial way to make it more useful is to include scalability tests.
OS-A is faster than OS-B on 1 cpu core for a test. But will OS-B maintain performance as number of cpu cores increases while OS-A degrades?
Solaris is known to be slow for many microbenchmarks. That’s not why peopel buy it. Its scalable to large hardware and workloads. Better tests would reveal the nature of this scalability.
http://queue.acm.org/detail.cfm?id=1095419
They should start contributing to OpenSolaris. How about Bluetooth!!! 😉
OpenSolaris may be slow on some benchmarks. But as a long time Linux user who recently decided to give OpenSolaris a try, I can honestly say that it “feels” at least as fast as Linux to me when it comes to every day tasks on my desktop where I have been using it for the past month. In some cases it even “feels” faster. Whether it actually is or not is not something I can attest to at this point.
I will say that when it comes to multitasking, it “feels” more capable than Linux though. I can do multiple fairly CPU intensive tasks in the background (like compiling software), and still do things on the desktop foreground that “feel” more responsive than Linux (video seems to run smoother, etc). This is likely where Solaris really shines, given its extensive background on massively-multiuser systems. It is extremely capable at juggling multiple tasks. Because of this, I think these tests ultimately fail to capture the strengths of Solaris where it feels, at least in my mind, like it beats out Linux.
It would be interesting if there were some possible to way to benchmark “actual user experience speed”, especially when performing multiple complex tasks at the same time”, but unfortunately, as far as I know, there is not.
For full disclaimer purposes, part of the reason I decided to try OpenSolaris is cause I got bored with Linux. Linux has become too mature, and too mainstream, and there’s almost no room for the average developer to contribute to it anymore. OpenSolaris is much more interesting when it comes to that. There’s encumbered code that needs to be rewritten so it can be open source, tons of packages that need to be ported, etc.
So even if it is a little slower than Linux… Well, I think it is just more fun these days
Edited 2009-02-10 03:47 UTC
How on earth can you call an OS without a stable kernel API, decent backwards binary compatibility, scattered into multiple incompatible distributions mature?
has a gcc4.
The benchmarks that were selected for this comparison are completely inappropriate for the proposed question. Almost all of the benchmarks are highly compute-intensive. For such programs, the operating system effectively hands off control to the program, interupts it only very very briefly, and then resumes control when it is finished. In other words, the operating system does virtually nothing. It is mentioned above that the compiler might affect things, but this is indeed a dramatic understatement. These benchmarks are effectively only testing the compiler. Running these programs compiled using the same compiler across different platforms would produce differences of only a couple percent (from scheduling overhead, context switch times, memory management routines, cpu affinities, etc). The only potentially valid benchmarks are the SQLite and File Encryption benchmarks, and even these require greater scrutiny.
I am not a big opensolaris fan myself, but using such programs to benchmark an operating system is exceedingly foolish.