Performance of Java on FBSD vs. others...

[Nikos Ntarmos]

Hi again.

On Mon, Dec 18, 2006 at 09:19:06AM -0800, Nick Johnson wrote:
> On Mon, 18 Dec 2006, Achilleas Mantzios wrote:
> 
> > Raising such an important issue as the current thread, i think
> > deserves something more than complete silence.
> 
> The silence is probably because a lot of us have jobs and lives to
> lead and can't spend our every waking moment focused on Java
> performance on FreeBSD.

You were right on that one. Sorry for keeping this radio silence;
real-life workload seem to be catching up with me, without me
necessarily catching up with it... :( I've put this issue in the closet
for the time being, but will surely revisit it some time after the
current state of alert is over.

Well, I haven't laid completely dormant wrt this issue; I went many times
over my code, profiled various aspects of it, run low-level benchmarks,
played (a bit too much) with the garbage collectors and other aspects of
the JVM, and so on. FWIW it seems that -Xrs (plus a number of less
significant, impact-wise, cryptic -XX:blah flags) is the winner here,
although I can't understand the reason why; I mean, -Xrs did the biggest
difference, but why is it that important (not) to intercept SIGHUP,
SIGINT, and SIGTERM? As I said earlier, I'm so willing to pursuit this
thing, although that will surely have to wait for at least another
month. I do tend to agree though that perhaps it would be wiser to take
this over a -STABLE branch, at least so as to dodge such criticism as
"current is noisy" et al.

> I think I'd be more comfortable seeing the results of some
> *comprehensive* benchmarks, rather than just one case that seems to
> perform better on one platform than another.  If you put some thought
> into it, I wager you could always find some test case that works well
> on one architecture and poorly on another.

You are also right on this one too. My test case is too complicated,
touching upon many parts of the system (e.g. threads, network i/o,
swapping, etc.), to lead to any definite conclusions. However, the seer
performance difference is too large to comprehend.

I still believe, though, that using my code to compare the performance
of JDK accross various operating systems on the same hardware is an
"apples vs oranges" case only to the extent that the OSs differ. I mean,
what I see as an end user is a quite large performance gap. I shouldn't
care if that is due to the inner workings of FreeBSD vs Linux vs Win32,
or due to the internals of the beast that is JVM and HotSpot, or because
of different allocation/garbage collection algorithms, or due to a
combination of all of the above plus more. If the defaults make JVM slow
for me (and others) on FreeBSD then it's the defaults that have to
change, or at least be documented for people to know. That was my point
to start with and I still stand firm on it.

> There are so many problems with the "benchmark" presented that I don't
> believe it can be used as the basis for troubleshooting performance
> problems.  We need to see a benchmark that does comprehensive testing
> of performance for a variety of Java functions on the same hardware
> with the same internal configuration.  Maybe SPECjbb2005 or
> CaffeineMark 3.0?

Now that's a really good idea. I'll certainly have a look at
CaffeineMark as soon as time constraints allow. I'd be more than
interested to see some results from someone running -STABLE and some
other OS on the same hardware, as a reference point. SPECjbb might be
harder to get my hands on, but I'll surely give it an (academic) shot.

The conclusions from my (limited) research on this issue so far are that
(i) the native freebsd jdk is slower -- at least for my workload -- than
the linux jdk on freebsd and the corresponding jdk on linux and win32
(although -server -Xrs somewhat bridges the performance gap -- sorry, no
numbers to report at the moment), but (ii) the native freebsd jdk is
_much_ more stable -- at least for my workload -- than the linux jdk on
freebsd (at the moment I'm waiting for a medium workload to complete;
it's been running on native freebsd jdk for the last 15 hours and will
probably need a couple more; the linux jdk on freebsd dies with an
internal error after the first hour or so of high load).

\n\n