Comparison of FreeBSD/Linux TCP Throughput performance

[Adrian Chadd]

uhm:

kristy# netperf -H 192.168.10.2 -p 22113 -l 10
TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.10.2
(192.168.10.2) port 0 AF_INET
Recv   Send    Send
Socket Socket  Message  Elapsed
Size   Size    Size     Time     Throughput
bytes  bytes   bytes    secs.    10^6bits/sec

  8192  65536  65536    10.00     862.48

1 megabyte socket buffers threw an error. I'll see why later.

Now, as for why 64k socket buffers gave a slower result than 8k socket
buffers... ah. If I change the sending end to use 64k socket buffers:

TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.10.2
(192.168.10.2) port 0 AF_INET
Recv   Send    Send
Socket Socket  Message  Elapsed
Size   Size    Size     Time     Throughput
bytes  bytes   bytes    secs.    10^6bits/sec

 65536  65536  65536    10.00     916.23



Adrian

2009/10/19 Brent Jones <brent at servuhome.net>:
> On Sun, Oct 18, 2009 at 7:55 AM, Adrian Chadd <adrian at freebsd.org> wrote:
>> FYI, I installed netperf on my local p4-D test boxes that I use for
>> other testing.
>>
>> 128 byte send/receive buffers on the client side:
>>
>> kristy# netperf -H 192.168.10.2 -p 22113 -l 10
>> TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.10.2
>> (192.168.10.2) port 0 AF_INET
>> Recv   Send    Send
>> Socket Socket  Message  Elapsed
>> Size   Size    Size     Time     Throughput
>> bytes  bytes   bytes    secs.    10^6bits/sec
>>
>>  8192    128    128    10.00     426.17
>>
>> 1kbyte send/receive buffers:
>>
>> kristy# netperf -H 192.168.10.2 -p 22113 -l 10
>> TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.10.2
>> (192.168.10.2) port 0 AF_INET
>> Recv   Send    Send
>> Socket Socket  Message  Elapsed
>> Size   Size    Size     Time     Throughput
>> bytes  bytes   bytes    secs.    10^6bits/sec
>>
>>  8192   1024   1024    10.00     903.39
>>
>> 8kbyte send/receive buffers:
>>
>> kristy# netperf -H 192.168.10.2 -p 22113 -l 10
>> TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.10.2
>> (192.168.10.2) port 0 AF_INET
>> Recv   Send    Send
>> Socket Socket  Message  Elapsed
>> Size   Size    Size     Time     Throughput
>> bytes  bytes   bytes    secs.    10^6bits/sec
>>
>>  8192   8192   8192    10.00     913.71
>>
>> Both boxes are 7.2-REL amd64 boxes on 3.4GHz Pentium-D CPUs using some
>> onboard flavour of the intel e1000 NIC:
>>
>>    device     = '82573E Intel Corporation 82573E Gigabit Ethernet
>> Controller (Copper)'
>>
>> They are connected via a Cisco 3750G L3 switch. In fact, the traffic
>> is routed, rather than switched.
>>
>> My /etc/sysctl.conf:
>>
>> net.inet.icmp.icmplim=0
>> net.inet.icmp.icmplim_output=0
>> net.inet.tcp.msl=3000
>> net.inet.tcp.sendspace=8192
>> net.inet.tcp.recvspace=8192
>> kern.maxfilesperproc=65536
>> kern.maxfiles=262144
>> kern.ipc.maxsockets=32768
>> kern.ipc.somaxconn=1024
>> kern.ipc.nmbclusters=131072
>> net.inet.ip.fw.enable=0
>> kern.ipc.somaxconn=10240
>>
>> 2c,
>>
>>
>> Adrian
>>
>> 2009/10/15 Hongtao Yin <htyin at huawei.com>:
>>> Hi,
>>>
>>>
>
> Can you try with 64K and up tp 1MB buffers?
>
> I see ~1Gbit speeds with my FreeBSD boxes using Broadcom NIC's and
> cheap Netgear switches.
> I'm not sure how the original tester got such poor numbers, when my
> setup is relatively low end, and sustaining Gbit speeds is no major
> feat.
>
>
>
> --
> Brent Jones
> brent at servuhome.net
>