Re[5]: Assymetric NIC performance problem

[Konstantin Kuzvesov]

On Mon,  Oct 7, 2013 at 16:57 -07:00, Kevin Oberman <rkoberman at gmail.com> wrote:
>On Mon, Oct 7, 2013 at 8:44 AM, Konstantin Kuzvesov  < kuzvesov at list.ru > wrote:
>>
>>I've got a FreeBSD file server running Samba, file upload speeds are okay, but downloads are too slow.
>>First, I decided it is Samba's fault, but then I ran iperf tests...
>>
>>On a windows machine with gigabit NIC:
>>Z:\iperf>iperf -c 192.168.0.1
>>------------------------------------------------------------
>>Client connecting to 192.168.0.1, TCP port 5001
>>TCP window size: 64.0 KByte (default)
>>------------------------------------------------------------
>>[  3] local 192.168.0.2 port 1064 connected with 192.168.0.1 port 5001
>>[ ID] Interval       Transfer     Bandwidth
>>[  3]  0.0-10.2 sec  12.4 MBytes  10.2 Mbits/sec
>>
>>Z:\iperf>iperf -s
>>------------------------------------------------------------
>>Server listening on TCP port 5001
>>TCP window size: 64.0 KByte (default)
>>------------------------------------------------------------
>>[  4] local 192.168.0.2 port 5001 connected with 192.168.0.1 port 41220
>>[ ID] Interval       Transfer     Bandwidth
>>[  4]  0.0-10.0 sec   716 MBytes   600 Mbits/sec
>>
>>And on a another with FastEthernet NIC:
>>C:\iperf>iperf.exe -c 192.168.0.1
>>------------------------------------------------------------
>>Client connecting to 192.168.0.1, TCP port 5001
>>TCP window size: 64.0 KByte (default)
>>------------------------------------------------------------
>>[  3] local 192.168.0.5 port 4756 connected with 192.168.0.1 port 5001
>>[ ID] Interval       Transfer     Bandwidth
>>[  3]  0.0-10.1 sec  11.4 MBytes  9.42 Mbits/sec
>>
>>C:\iperf>iperf.exe -s
>>------------------------------------------------------------
>>Server listening on TCP port 5001
>>TCP window size: 64.0 KByte (default)
>>------------------------------------------------------------
>>[  4] local 192.168.0.5 port 5001 connected with 192.168.0.1 port 18558
>>[ ID] Interval       Transfer     Bandwidth
>>[  4]  0.0-10.0 sec   106 MBytes  88.5 Mbits/sec
>>
>>Both tests show server's NIC performance degradation to around 10Mbit/s when traffic goes from server to client. And everything works fine in other direction.
>>
>>I verified the cables and hub by simply connecting server and a test machine with a new short patch cord. I tried to change server's NIC from D-Link DGE-528T to Planet ENW-9604. And it became even worse -
>> using Planet NIC
>> speeds slowed down to around 500Mbit/s to server and the same 10Mbit/s to client. I tried to change NIC's media to 100BaseTX, it didn't help too. What else should I try to fix the problem? Maybe my system requires is just misconfigured?
>>
>>System configuration:
>>OS: FreeBSD 9.2-release
>>Kernel: generic
>>Firewall: none
>>/boot/loader.conf - load zfs modules only
>>/etc/sysctl.conf - empty
>>NIC: D-Link DGE-528T / Planet ENW-9604
>>
>>--
>>Konstantin Kuzvesov
> 
>Output from ifconfig would probably be helpful, but I'd also 
suggest that you capture packets (or, at least headers) for at least the
 start of the transfer and look at them with wireshark or some similar 
tool. 
>
>wireshark can tell you quite a bit and, if you feed the 
capture into tcptrace, you can really see what is happening. (Note, 
wireshark provides indications of problems that can make sense to people
 not conversant with the gory details of TCP, though some issues may not 
be obvious. tcptrace output can be utterly cryptic if you don't 
understand a lot of the details of TCP and congestion control.
>
>Both
 wireshark and tcptrace are in ports and are best installed on a 
workstation. The tcpdump output can be used as input to both. ("tcpdump 
-pw FILE -i INTERFACE host ADDRESS" can do the job. Then copy the 
capture to the right place for analysis. But start with configuration 
and counters for the interface (netstat -i).
>-- 
>R. Kevin Oberman, Network Engineer
>E-mail:  rkoberman at gmail.com

Sorry, I didn't know that UDP bandwidth must be specified manually, otherwise it defaults to 1.0Mbit/s.
New UDP tests show that there must be some TCP misconfiguration:

#iperf -s -u
------------------------------------------------------------
Server listening on UDP port 5001
Receiving 1470 byte datagrams
UDP buffer size: 41.1 KByte (default)
------------------------------------------------------------
[  3] local 192.168.0.1 port 5001 connected with 192.168.0.2 port 1039
[ ID] Interval       Transfer     Bandwidth        Jitter   Lost/Total Datagrams
[  3]  0.0-10.0 sec   626 MBytes   525 Mbits/sec   0.062 ms 13181/459995 (2.9%)
[  3]  0.0-10.0 sec  1 datagrams received out-of-order

#iperf -c 192.168.0.2 -u -b 1000M
------------------------------------------------------------
Client connecting to 192.168.0.2, UDP port 5001
Sending 1470 byte datagrams
UDP buffer size: 9.00 KByte (default)
------------------------------------------------------------
[  3] local 192.168.0.1 port 15347 connected with 192.168.0.2 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   882 MBytes   740 Mbits/sec
[  3] Sent 632673 datagrams
[  3] WARNING: did not receive ack of last datagram after 10 tries.

The only problem wireshark showed to me is wrong IP header checksums in packets originated from server.

-- 
Konstantin Kuzvesov