Performance Problems.. Server hardware smoked by $500 box?

[Mark Terribile]

John Straiton writes:

> I'm pretty confused right now with trying to
> determine the nature of a performance problem ...
> on one of my servers. ... in pulling up websites
> from the machine, my silly POS development
> box has nearly double performance ...

There's lots of tricky stuff that can be going wrong.
I spent some time in my last two jobs (anybody got
a new one in NJ?) on speeding up stuff like this
and the first thing I try to do is put some kind of
steady-state load on the boxen and monitor each box
involved with  systat 1 -vmstat  .  There's one hell
of a lot of information there, and interactions are
sometimes hard to see.  If the CPU is fully occupied,
it could be the network stack (which will NOT show
up at interrupt level) and that can depend on what
interface chipset you're using as well.  Or it could
be ... well, get the data first.  If you'd like to
send me a few sample screens, I'll try to make
suggestions on what to check next.  You want to have
a series from each of the three configurations you're
using.  And being able to _watch_ what's happening
on  systat  is worth a whole lot of non-sequenced
snapshots.

Are you running firewall software on the production
machine?  I don't know how the FreeBSD version will
affect performance, but it can't help.  How about
the reports from  top ?  What do they say?  What's
soaking up the processor?

Running on a 1GHz PIII two years ago, I was able to
get a web proxy (not squid!) to serve 1500+ requests
per second, with about 200 MBit/sec of ethernet
traffic
(inbound and out).  (The product never made it into
full-scale production, largely due to financial
problems in the large, well-known corporation.)
So the problem isn't horsepower, but something not
using it well.

Can you try running the back end box on a simple
disk without the RAID in the way?  I don't recall
all the properties of RAID 5 right now, but in general
RAID trades disk transactions away to get disk
throughput.  In your application, you probably need
transactions more than throughput.

Dumb question: have you tried swapping cables/ports
on the ethernet connections?  Does one link support
jumbo frames and the other not?  How about network
buffers: have you got enough configured, and how
many are tied up at a time?

Performance is often a negative art: find the worst
roadblock and remove it, then the next worst after
that, and so forth.

                                   Mark Terribile


__________________________________
Do you Yahoo!?
Yahoo! SiteBuilder - Free, easy-to-use web site design software
http://sitebuilder.yahoo.com