packet drop with intel gigabit / marwell gigabit

[OxY]

CPU utilization is 0% if apache is not running and 10-20%, when running and
serving 30-40 concurrent downloads (traffic is 3-4MB/s on fxp0 interface)
i measured the network performance with 'iperf' util, started the server on 
my box
and benchmarked with a client on the other gigabit machine.
it showed 0% packet drop, when apache was not running and 4-7%, when 
running..
then i checked how it behave when i shut down apache and init a local file 
copy from one
(not system!) disk to other (not system disk either). packet drop was 5-10%, 
due to the higher load.
so i think interrupts or just the load takes the network performance, but i 
have no clue how to fix it.
is it possible that the 2000+ xp amd is just weak to serve such a traffic? 
(em0 traffic's maximum is 18-23MB/s)
i think it might be around 30MB/s without packet drop.

I did FTP measurement, because what i want is to copy files with high speed 
from the
other gigabit machine. However FTP needs resources (CPU, I/O, etc), but 
iperf not!
iperf shows 20% CPU utilization when apache not running and when there's no 
packet drop.

ps: Now apache says: 14 requests currently being processed
traffic is 1MB/s on fxp0, and em0 benchmark with iperf says (64k udp window 
size):

[  3]  0.0-10.0 sec    235 MBytes    197 Mbits/sec
[  3] Sent 167375 datagrams
[  3] Server Report:
[  3]  0.0-10.0 sec    229 MBytes    192 Mbits/sec  0.066 ms 4115/167375 
(2.5%)

the other gigabit machine is OK, because i have 0% packet drop, when my 
machine is totally idle.

----- Original Message ----- 
From: "Jin Guojun [VFFS]" <g_jin at lbl.gov>
To: "OxY" <oxy at field.hu>; "Chuck Swiger" <cswiger at mac.com>
Cc: <freebsd-performance at freebsd.org>
Sent: Sunday, March 19, 2006 1:49 AM
Subject: Re: packet drop with intel gigabit / marwell gigabit


> It is still not clear how you did measurement.
> Did FTP show such % drop? or Did you measure it by other tools?
> How did you measured incoming traffic?
>
> http://field.hu/netstat.txt shows 0 tcp packet drop.
>
> Anyway, the first thing first is to have CPU utilization when you see 
> packet drop.
> This can be get from running "top" or "vmstat 1". As well as run
> netstat -i -p tcp | grep -i drop
> If CPU utilization is approaching 100%, either the traffic is no 2 MBps,
> or some process is taking CPU time. For this reason, "top" is a better
> tool to use. At this point, if you run netstat command multiple times,
> you would see drop counter increasing.
> Once you find out what process takes CPU time, then further tuning can be
> determined.
>
> If CPU utilization is well below 70-80%, then you need to use tcpdump and
> tcptrace to visualize what cause packet drop, then perform a solution.
>
> Jin
>
> ----- Original Message -----
> From: "OxY" <oxy at field.hu>
> To: "Chuck Swiger" <cswiger at mac.com>
> Cc: <freebsd-performance at freebsd.org>
> Sent: Saturday, March 18, 2006 2:23 PM
> Subject: Re: packet drop with intel gigabit / marwell gigabit
>
>
>> currently i use HZ=2000
>> here's the output of netstat -i, -s, and vmstat -i :
>> (currently i am uploading on the gigabit with ftp, 3 threads)
>>
>> Field root# vmstat -i
>> interrupt                          total       rate
>> irq0: clk                       27503959       1993
>> irq1: atkbd0                           1          0
>> irq3: fxp0                             2          0
>> irq7:                                146          0
>> stray irq7                           146          0
>> irq8: rtc                        1765569        127
>> irq10: atapci1                   2807786        203
>> irq11: atapci0                    475039         34
>> irq13: npx0                            1          0
>> irq14: ata0                           99          0
>> Total                           32552748       2359
>>
>> Field root# netstat -i
>> Name    Mtu Network       Address              Ipkts Ierrs    Opkts Oerrs 
>> Coll
>> fxp0   1500 <Link#1>      00:a0:c9:8d:79:68 13163545     0 21899372     1 
>> 0
>> fxp0   1500 195.38.96.64/ field                  141     - 6     -     -
>> em0    1500 <Link#2>      00:0e:0c:a2:ac:42 68644181     4 66793904     0 
>> 0
>> em0    1500 195.38.96.64/ field             211255811     - -     -
>> lo0   16384 <Link#3>                        129622061     0 129622061 0 0
>>
>> netstat -s is here:
>> http://field.hu/netstat.txt
>>
>> ----- Original Message ----- 
>> From: "Chuck Swiger" <cswiger at mac.com>
>> To: "OxY" <oxy at field.hu>
>> Cc: <freebsd-performance at freebsd.org>
>> Sent: Saturday, March 18, 2006 1:37 PM
>> Subject: Re: packet drop with intel gigabit / marwell gigabit
>>
>>
>>> OxY wrote:
>>>> yeah, i googled these settings, but i put them back to default then!
>>>> i measured iperf performance, and it showed that the packet drop is
>>>> depending on the system load..
>>>
>>> If you are using the normal interrupt-driven configuration, you should 
>>> look at
>>> netstat -i, -s, and vmstat -i.  If you're turning on device polling, you 
>>> ought
>>> to retry your testing at higher HZ (try 2000 or 5000):
>>>
>>>   echo 'kern.hz="2000"' >> /boot/loader.conf
>>>
>>> -- 
>>> -Chuck