packet drop with intel gigabit / marwell gigabit
oxy at field.hu
Sun Mar 19 08:59:46 UTC 2006
CPU utilization is 0% if apache is not running and 10-20%, when running and
serving 30-40 concurrent downloads (traffic is 3-4MB/s on fxp0 interface)
i measured the network performance with 'iperf' util, started the server on
and benchmarked with a client on the other gigabit machine.
it showed 0% packet drop, when apache was not running and 4-7%, when
then i checked how it behave when i shut down apache and init a local file
copy from one
(not system!) disk to other (not system disk either). packet drop was 5-10%,
due to the higher load.
so i think interrupts or just the load takes the network performance, but i
have no clue how to fix it.
is it possible that the 2000+ xp amd is just weak to serve such a traffic?
(em0 traffic's maximum is 18-23MB/s)
i think it might be around 30MB/s without packet drop.
I did FTP measurement, because what i want is to copy files with high speed
other gigabit machine. However FTP needs resources (CPU, I/O, etc), but
iperf shows 20% CPU utilization when apache not running and when there's no
ps: Now apache says: 14 requests currently being processed
traffic is 1MB/s on fxp0, and em0 benchmark with iperf says (64k udp window
[ 3] 0.0-10.0 sec 235 MBytes 197 Mbits/sec
[ 3] Sent 167375 datagrams
[ 3] Server Report:
[ 3] 0.0-10.0 sec 229 MBytes 192 Mbits/sec 0.066 ms 4115/167375
the other gigabit machine is OK, because i have 0% packet drop, when my
machine is totally idle.
----- Original Message -----
From: "Jin Guojun [VFFS]" <g_jin at lbl.gov>
To: "OxY" <oxy at field.hu>; "Chuck Swiger" <cswiger at mac.com>
Cc: <freebsd-performance at freebsd.org>
Sent: Sunday, March 19, 2006 1:49 AM
Subject: Re: packet drop with intel gigabit / marwell gigabit
> It is still not clear how you did measurement.
> Did FTP show such % drop? or Did you measure it by other tools?
> How did you measured incoming traffic?
> http://field.hu/netstat.txt shows 0 tcp packet drop.
> Anyway, the first thing first is to have CPU utilization when you see
> packet drop.
> This can be get from running "top" or "vmstat 1". As well as run
> netstat -i -p tcp | grep -i drop
> If CPU utilization is approaching 100%, either the traffic is no 2 MBps,
> or some process is taking CPU time. For this reason, "top" is a better
> tool to use. At this point, if you run netstat command multiple times,
> you would see drop counter increasing.
> Once you find out what process takes CPU time, then further tuning can be
> If CPU utilization is well below 70-80%, then you need to use tcpdump and
> tcptrace to visualize what cause packet drop, then perform a solution.
> ----- Original Message -----
> From: "OxY" <oxy at field.hu>
> To: "Chuck Swiger" <cswiger at mac.com>
> Cc: <freebsd-performance at freebsd.org>
> Sent: Saturday, March 18, 2006 2:23 PM
> Subject: Re: packet drop with intel gigabit / marwell gigabit
>> currently i use HZ=2000
>> here's the output of netstat -i, -s, and vmstat -i :
>> (currently i am uploading on the gigabit with ftp, 3 threads)
>> Field root# vmstat -i
>> interrupt total rate
>> irq0: clk 27503959 1993
>> irq1: atkbd0 1 0
>> irq3: fxp0 2 0
>> irq7: 146 0
>> stray irq7 146 0
>> irq8: rtc 1765569 127
>> irq10: atapci1 2807786 203
>> irq11: atapci0 475039 34
>> irq13: npx0 1 0
>> irq14: ata0 99 0
>> Total 32552748 2359
>> Field root# netstat -i
>> Name Mtu Network Address Ipkts Ierrs Opkts Oerrs
>> fxp0 1500 <Link#1> 00:a0:c9:8d:79:68 13163545 0 21899372 1
>> fxp0 1500 22.214.171.124/ field 141 - 6 - -
>> em0 1500 <Link#2> 00:0e:0c:a2:ac:42 68644181 4 66793904 0
>> em0 1500 126.96.36.199/ field 211255811 - - -
>> lo0 16384 <Link#3> 129622061 0 129622061 0 0
>> netstat -s is here:
>> ----- Original Message -----
>> From: "Chuck Swiger" <cswiger at mac.com>
>> To: "OxY" <oxy at field.hu>
>> Cc: <freebsd-performance at freebsd.org>
>> Sent: Saturday, March 18, 2006 1:37 PM
>> Subject: Re: packet drop with intel gigabit / marwell gigabit
>>> OxY wrote:
>>>> yeah, i googled these settings, but i put them back to default then!
>>>> i measured iperf performance, and it showed that the packet drop is
>>>> depending on the system load..
>>> If you are using the normal interrupt-driven configuration, you should
>>> look at
>>> netstat -i, -s, and vmstat -i. If you're turning on device polling, you
>>> to retry your testing at higher HZ (try 2000 or 5000):
>>> echo 'kern.hz="2000"' >> /boot/loader.conf
More information about the freebsd-performance