Request for Cluster Recommendations

[Andy Sporner]

Hi Michael,

>For the most part, my codes have been data
>parallel and have invlolved broadcasting
>parameters and merging results at the end of
>distributed serial computations.  They have
>involved many evaluations of small matrices
>and data sets and I get better speedups with
>simultaneous serial executions.  Much of the
>work I'm looking towards would make use
>of a grid approach in the style of seti at home
>or the factoring projects.
>  
>
Seti at home is very nice indeed.  I am a participant as
well (Captain Blank)

>I would expect bursts of communication
>separated by periods of computation;
>Overall, communication wouldn't be 
>so much of a bottleneck, but I'd like it
>to be fast when it does occur.  If Andy
>Sporner's 30% figure holds up, I think
>local disks as buffers would allow the
>network access to be smeared out.  I've got
>40% in mind as an upper performance limit
>for ethernet due to collisions, but I can't back
>that up.  Local buffers seem to allow for
>scheduling comm. so as to avoid collisions.
>

With a switch you wouldn't normally have enough collisions
to worry about, unless you approach the bandwidth of the media.

When I spoke of the 30% I was refering to a extended burst.  We
benchmark our firewall loadbalancer(nitro) with 4 firewalls and
a traffic generation farm for entire weekends for reliability
testing,  this is where I observed this value.  The Firewall machines
are 2.8 Ghz Athlon servers.

>
>Starting out with 100Mbit may make sense as
>a cheaper/simpler startup - no NIC
>purchases.  After running some simulation
>and benchmarking with the apps, upgrading to Gigabit wouldn't be an undue burden.
>

That's not a bad idea, but many of the 1U servers (not to mention
the better motherboards (such as ASUS) are with GB as standard).
The switch is a little more expensive, but not that drastic.

>Some initial attempts at estimating
>communication & compute demands would
>be in order. Aside from the fiber options, is the cabling the same for 100Mbit and Gbit?
>
Using Cat6 -- yes, but we have also used cat 5 with some success.
If you start wth CAT-6 you will always win.

Good luck!



Andy