Network performance in a dual CPU system

[Marcos Bedinelli]

Hi,


On 10-Feb-06, at 13:06, Chuck Swiger wrote:

> Marcos Bedinelli wrote:
> [ ... ]
>> mull [~]$vmstat -i
>> interrupt                          total       rate
>> irq1: atkbd0                        3466          0
>> irq6: fdc0                            10          0
>> irq13: npx0                            1          0
>> irq14: ata0                           47          0
>> irq21: fxp1                     20462527          8
>> irq28: bge0                   3511765157       1444
>> irq29: bge1                   3633124373       1494
>> irq30: aac0                      1842472          0
>> cpu0: timer                    566751007        233
>> Total                         7733949060       3181
>
> Interesting, what do you have HZ ("sysctl kern.clockrate") set to?  
> Does setting
> it to somewhere around 500, 1000, or 2000 help?  You're definitely 
> going to want
> to increase HZ if you enable polling mode...


mull [~]$sysctl kern.clockrate
kern.clockrate: { hz = 1000, tick = 1000, profhz = 666, stathz = 133 }




>> mull [~]$netstat -m
>> 644/646/1290 mbufs in use (current/cache/total)
>> 643/407/1050/17088 mbuf clusters in use (current/cache/total/max)
>> 0/5/4528 sfbufs in use (current/peak/max)
>> 1447K/975K/2422K bytes allocated to network (current/cache/total)
>> 0 requests for sfbufs denied
>> 0 requests for sfbufs delayed
>> 0 requests for I/O initiated by sendfile
>> 0 calls to protocol drain routines
>
> You might try increasing the # of kern.ipc.nmbclusters, say by a 
> factor of 2.
> This may not help much, seems like you're bottlenecking servicing the 
> bge
> interrupts.
>
> I assume you've read "man tuning" and do not have something like 
> WITNESS enabled
> in your kernel?  :-)


WITNESS is not part of my current KERNCONF file.

To be honest, I didn't read the whole "man tuning" document because 
there are many parts that don't apply to this situation. However, the 
"CPU, MEMORY, DISK, NETWORK" section is quite clear about the 
following:

"If your system runs out of CPU (idle times are perpetually 0%) then 
you need to consider upgrading the CPU or moving to an SMP motherboard 
(multiple CPU's), or perhaps you need to revisit the programs that are 
causing the load and try to optimize them."

That's basically the problem I am experiencing: memory is fine, swap is 
fine, disk access is fine, CPU utilization is way high...

The machine is in production and needs to have its performance improved 
asap. Consequently, we are fine with the idea of spending some $ with a 
second processor, provided that someone can tell me whether such matter 
can be solved using this approach. What we would like to avoid is 
spending $ with a second CPU that ultimately won't do any good for us.

Cheers!

--
Marcos