High performance computing on FreeBSD

[O. Hartmann]

Dear Sirs.

FreeBSd is now since 1996 my companion in scientific computing and 
related server systems and also my favorite operating system for every 
network stuff, firewalls and desktop systems I ever used.

Now going ahaed with 64Bit, FreeBSD 6.X has been canceled for desktop 
systems due to the lack of a working JAVA in native 64Bit and especially 
a working native 64 Bit OpenOffice environment.

Nevertheless, the experience of our group and especially of mine with 
several flavours of Linux, used at our computer center and its network 
performance and stability in comparison to FreeBSD's over the same time 
period let me tend to ask for a FreeBSD based high performance computer 
cluster more than such one founded on a Linux distribution. But there 
are some open issues and those need to be discussed deeper.

First targets SMP/Node performance. I was very curious about SCHED_ULE 
when introduced in FreeBSD 5.X and was said to deliver a performance 
boost on SMP boxes. I'm still waiting for that to come true, every SMP 
scaling benchmark that has been taken in our computer center said Linux 
has the better SMP performance (on the same Opteron hardware, but I do 
not have specific details about that, sorry).
Next point is the intercommunication of nodes. Infiniband or with 
special Hypertransport coupplings nodes will be able to communicate very 
fast. GBit LAN will be the least option, so the question is whether 
plans for or ready solutions for the node connections are underway.
The last question refers to Fortran. Well, most of our scientists still 
work with Fortran77 or Fortran90/95 and it is hard to bring them towards 
C/C++, so the existence of good Fotran compiler will be essential. GCC 
4.1/4.2 isn't standard in FreeBSD 6.X but many of other FreeBSD users 
told me they use the port's gcc 4.X very successful. But I feel better 
when the new GNU compiler collection will be the standard for FreeBSD. 
This may sound weird for some of yours, but I like the ease of upgrading 
software in FreeBSD which has reached a very, very high standard over 
the past 10 years (and it isn't comparable to jarsh  weirdness I 
experienced with Linux, Solaris or Windows). So, utilizing standard 
ports and the base compiler collection gives a very stable and high 
quality platform - in my opinion.

All right, this above mentioned fundamentals should be the basis for a 
small cluster system for numerical research.
I still looking for benchmark tests, pro and contra regarding BSD/Linux 
(except the existence of better compiler software for Linux) and the 
state of development of high performance node interconnect and 
designated driver software.

Target hardware will be a four or six node Opteron/Athlon64 platform 
with dual socket/dual core chips, with 4 or 8 GB local RAM and 200 GB 
local SATA disk drives, but main disk array will be RAID system attached 
via GBit LAN or, if possible, faster. The big question will remain in 
how the nodes should be interconnected and what kind of OS will be able 
to handle a specific interconnect (HTX/Infiniband).

In the case my questions are to unspecific or naiv, please excuse that.

Oliver