vge traffic problem

[David Ehrmann]

Pyun YongHyeon wrote:
>> I ran the test, again, and had tcpdump capture the packets (on the host 
>> with the vge interface).
>>
>> tcpdump -i vge0 -w dump.cap -K -s 0 host 10.0.1.2
>>
>> When I opened it up in Wireshark, it's reporting that the outgoing UDP 
>> checksums are incorrect; they're always 0x1ae3.  That said, maybe the 
>>     
>
> Because bpf(4) see the frame before controller inserts checksum
> value it always reports incorrect checksum. It's normal for TX
> checksum offload capable controller.
> In order to verify the hardware assisted checksum you should check
> that on receiver side. If the checksum was invalid, receiver
> dropped them. See "netstat -s | grep bad" on receiver host.
>
>   
I did this one between two similar FreeBSD systems (8.0 and 8.0 RC1, I 
think).  There wasn't an error in netstat -s on the receiver, but I ran 
tcpdump, anyway.  The checksums were good, but iperf ended with "read 
failed: connection refused."  On the non-vge box, once the vge box is 
done sending, it sends four identical UDP packets to the vge host, I 
think to ask it for sending statistics.  Instead of statistics, it got 
back an ICMP packet saying "port unreachable."  This happens with a 
freebsd client, but not a linux client (I didn't confirm the ICMP 
message with tcpdump, though).  All iperf versions are the same (2.0.4), 
but the linux one doesn't have any threading (which shouldn't be an 
issue when I'm not using that option).
>> checksums are done in hardware AFTER tcpdump sees them.
>>
>> I set net.inet.udp.checksum to 0.  The bad checksums are gone, but I 
>> still see dropped packets.
>>
>>     
>
> This doesn't disable TX checksum offload of driver. Instead use
> #ifconfig vge0 -txcsum
>
>   
I'm still losing packets, but now it's interesting.  The other FreeBSD 
machine (a VMware VM on a fairly fast machine with another gigabit 
connection) loses just a few, now, and sometimes none.  The linux box 
(200mhz, runs OpenWrt, 100mbps ethernet) still loses more than 1% of 
packets.  I cut iperf back to 100kbps.  The results were much better (0 
packets lost), but this did happen one time:

[  3] WARNING: did not receive ack of last datagram after 10 tries.

Maybe this is related to "port unreachable," and maybe to my nfs 
problem.  The drops seem to be related to iperf not being able to do 
much processing at 200 mhz (which is a little surprising; I was only 
asking for 1Mbps).
>> It's on the motherboard, probably wired directly to a PCI-E connection, 
>> but yes, it is a VT6130.
>>     
>
> Show me "pciconf -lcv" output.
>   
vge0 at pci0:3:0:0:        class=0x020000 card=0x01101106 chip=0x31191106 
rev=0x82 hdr=0x00
    vendor     = 'VIA Technologies Inc'
    device     = ''Velocity' Gigabit Ethernet Controllers 
(VT6120/VT6121/VT6122)'
    class      = network
    subclass   = ethernet
    cap 01[50] = powerspec 3  supports D0 D1 D2 D3  current D0
    cap 10[90] = PCI-Express 1 endpoint max data 128(128) link x1(x1)
    cap 05[c0] = MSI supports 1 message, 64 bit, vector masks

It might not be vge, after all, and might just be its speed...but that 
shouldn't cause a tcp nfs issue.