rsync corrupted MAC
Larry Rosenman
ler at lerctr.org
Tue Oct 11 00:43:08 UTC 2011
On Mon, 10 Oct 2011, Jeremy Chadwick wrote:
> On Mon, Oct 10, 2011 at 04:15:25PM -0500, Larry Rosenman wrote:
>> On 10/10/2011 3:57 PM, Louis Mamakos wrote:
>>> On Oct 10, 2011, at 2:38 PM, Larry Rosenman wrote:
>>>
>>>> On 10/10/2011 10:47 AM, John Baldwin wrote:
>>>>> On Sunday, October 09, 2011 5:06:26 pm Larry Rosenman wrote:
>>>>>> Any ideas on which side or what might be broke here?
>>>>>>
>>>>>> ler/MAIL-ARCHIVE/2008/12/INBOX
>>>>>> Corrupted MAC on input.
>>>>>> Disconnecting: Packet corrupt
>>>>>> rsync: connection unexpectedly closed (33845045 bytes received so far)
>>>>> [receiver]
>>>>>> rsync error: error in rsync protocol data stream (code 12) at io.c(605)
>>>>> [receiver=3.0.9]
>>>>>> rsync: connection unexpectedly closed (1450 bytes received so far)
>>>>> [generator]
>>>>>> rsync error: unexplained error (code 255) at io.c(605) [generator=3.0.9]
>>>>> I've had somewhat similar issues (ssh getting corruption in its data stream)
>>>>> when a NIC in my netbook was corrupting packet data when it ran at 1G (it
>>>>> worked fine at 10/100). Pyun eventually fixed the issue by applying enough
>>>>> workarounds (it was likely a hardware bug in the NIC's chipset). However, it
>>>>> wasn't easy to debug unfortunately. :(
>>>>>
>>>> Any ideas on where to start?
>>>>
>>>> from the 8.2 box (tbh.lerctr.org in the script):
>>>>
>>>> 8.2->PIX->Provider->Internet->Motorola SBG6580 (Time-Warner)->Trendnet TEG-160WS Gig switch->9.0 box (borg.lerctr.org).
>>>>
>>>> So, where do I start?
>>> I'd turn off IP / TCP / UDP checksum offloading on your NIC if it supports it, and see if you are getting network layer checksum errors. If the IP checksum is wrong, then it happened on the last hops between the NIC and memory or across the previous network hop.
>>>
>>>
>>>
>> Good idea, but, it didn't show ANY errors on EITHER side (both are
>> em nics).
>>
>> Next?
>> $ ifconfig em0
>> em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
>> options=2098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>
>> ether 00:30:48:2e:99:ba
>> inet 192.147.25.65 netmask 0xffffff00 broadcast 192.147.25.255
>> inet6 fe80::230:48ff:fe2e:99ba%em0 prefixlen 64 scopeid 0x1
>> inet 192.147.25.45 netmask 0xffffff00 broadcast 192.147.25.255
>> inet 192.147.25.11 netmask 0xffffff00 broadcast 192.147.25.255
>> nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
>> media: Ethernet autoselect (100baseTX <full-duplex>)
>> status: active
>> $
>> $ uname -a
>> FreeBSD thebighonker.lerctr.org 8.2-STABLE FreeBSD 8.2-STABLE #45:
>> Sat Oct 8 10:57:43 CDT 2011
>> root at thebighonker.lerctr.org:/usr/obj/usr/src/sys/THEBIGHONKER
>> amd64
>> $
>>
>>
>>
>> $ ifconfig em0
>> em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
>> options=2088<VLAN_MTU,VLAN_HWCSUM,WOL_MAGIC>
>> ether 00:30:48:8e:9f:f3
>> inet 192.168.200.4 netmask 0xffffff00 broadcast 192.168.200.255
>> inet6 fe80::230:48ff:fe8e:9ff3%em0 prefixlen 64 scopeid 0x1
>> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
>> media: Ethernet autoselect (1000baseT <full-duplex>)
>> status: active
>> $ uname -a
>> FreeBSD borg.lerctr.org 9.0-BETA3 FreeBSD 9.0-BETA3 #1: Sun Oct 9
>> 10:03:42 CDT 2011
>> root at borg.lerctr.org:/usr/obj/usr/src/sys/BORG-DTRACE amd64
>> $
>
> Can you please provide output from the following commands executed on
> the machine showing the problem? The above commands show nothing
> useful, other than the fact that one machine is at 100/full and the
> other is at 1000/full (I don't know your network setup). Commands:
>
> * netstat -inbd -I em0
> * sysctl -a dev.em.0
> * Issue command "sysctl dev.em.0.debug=1", then type "dmesg" and
> provide all of the new output you will see at the bottom that
> pertains to the NIC
>
> If you Google this problem, you will find that the majority of the time
> it's caused by NIC drivers acting oddly.
>
> Also, I believe the em(4) driver in 9.x is slightly different than on
> 8.x, so I'm CC'ing Jack Vogel here.
>
>
from 9.0:
Name Mtu Network Address Ipkts Ierrs Idrop Ibytes Opkts Oerrs Obytes Coll Drop
em0 1500 <Link#1> 00:30:48:8e:9f:f3 69776975 0 0 59660392277 52592789 0 104743924118 0 0
em0 1500 192.168.200.0 192.168.200.4 69759773 - - 58681934612 96397272 - 104003761109 - -
em0 1500 fe80::230:48f fe80::230:48ff:fe 0 - - 0 3 - 248 - -
dev.em.0.%desc: Intel(R) PRO/1000 Network Connection 7.2.3
dev.em.0.%driver: em
dev.em.0.%location: slot=0 function=0
dev.em.0.%pnpinfo: vendor=0x8086 device=0x1096 subvendor=0x15d9 subdevice=0x0000 class=0x020000
dev.em.0.%parent: pci6
dev.em.0.nvm: -1
dev.em.0.debug: -1
dev.em.0.rx_int_delay: 0
dev.em.0.tx_int_delay: 66
dev.em.0.rx_abs_int_delay: 66
dev.em.0.tx_abs_int_delay: 66
dev.em.0.rx_processing_limit: 100
dev.em.0.flow_control: 3
dev.em.0.eee_control: 0
dev.em.0.link_irq: 0
dev.em.0.mbuf_alloc_fail: 0
dev.em.0.cluster_alloc_fail: 0
dev.em.0.dropped: 0
dev.em.0.tx_dma_fail: 21755
dev.em.0.rx_overruns: 0
dev.em.0.watchdog_timeouts: 0
dev.em.0.device_control: 1851969
dev.em.0.rx_control: 67141634
dev.em.0.fc_high_water: 30720
dev.em.0.fc_low_water: 29220
dev.em.0.queue0.txd_head: 136
dev.em.0.queue0.txd_tail: 136
dev.em.0.queue0.tx_irq: 0
dev.em.0.queue0.no_desc_avail: 0
dev.em.0.queue0.rxd_head: 2
dev.em.0.queue0.rxd_tail: 1
dev.em.0.queue0.rx_irq: 0
dev.em.0.mac_stats.excess_coll: 0
dev.em.0.mac_stats.single_coll: 0
dev.em.0.mac_stats.multiple_coll: 0
dev.em.0.mac_stats.late_coll: 0
dev.em.0.mac_stats.collision_count: 0
dev.em.0.mac_stats.symbol_errors: 0
dev.em.0.mac_stats.sequence_errors: 0
dev.em.0.mac_stats.defer_count: 0
dev.em.0.mac_stats.missed_packets: 0
dev.em.0.mac_stats.recv_no_buff: 0
dev.em.0.mac_stats.recv_undersize: 0
dev.em.0.mac_stats.recv_fragmented: 0
dev.em.0.mac_stats.recv_oversize: 0
dev.em.0.mac_stats.recv_jabber: 0
dev.em.0.mac_stats.recv_errs: 0
dev.em.0.mac_stats.crc_errs: 0
dev.em.0.mac_stats.alignment_errs: 0
dev.em.0.mac_stats.coll_ext_errs: 0
dev.em.0.mac_stats.xon_recvd: 0
dev.em.0.mac_stats.xon_txd: 0
dev.em.0.mac_stats.xoff_recvd: 0
dev.em.0.mac_stats.xoff_txd: 0
dev.em.0.mac_stats.total_pkts_recvd: 69774324
dev.em.0.mac_stats.good_pkts_recvd: 69774324
dev.em.0.mac_stats.bcast_pkts_recvd: 28156
dev.em.0.mac_stats.mcast_pkts_recvd: 1758
dev.em.0.mac_stats.rx_frames_64: 54177
dev.em.0.mac_stats.rx_frames_65_127: 30157358
dev.em.0.mac_stats.rx_frames_128_255: 1092948
dev.em.0.mac_stats.rx_frames_256_511: 125295
dev.em.0.mac_stats.rx_frames_512_1023: 128081
dev.em.0.mac_stats.rx_frames_1024_1522: 38216465
dev.em.0.mac_stats.good_octets_recvd: 59938624047
dev.em.0.mac_stats.good_octets_txd: 106613836902
dev.em.0.mac_stats.total_pkts_txd: 96250538
dev.em.0.mac_stats.good_pkts_txd: 96250538
dev.em.0.mac_stats.bcast_pkts_txd: 2989
dev.em.0.mac_stats.mcast_pkts_txd: 0
dev.em.0.mac_stats.tx_frames_64: 7551
dev.em.0.mac_stats.tx_frames_65_127: 26727682
dev.em.0.mac_stats.tx_frames_128_255: 227574
dev.em.0.mac_stats.tx_frames_256_511: 167383
dev.em.0.mac_stats.tx_frames_512_1023: 302141
dev.em.0.mac_stats.tx_frames_1024_1522: 68818207
dev.em.0.mac_stats.tso_txd: 17244234
dev.em.0.mac_stats.tso_ctx_fail: 0
dev.em.0.interrupts.asserts: 65945396
dev.em.0.interrupts.rx_pkt_timer: 8917
dev.em.0.interrupts.rx_abs_timer: 0
dev.em.0.interrupts.tx_pkt_timer: 1461
dev.em.0.interrupts.tx_abs_timer: 1951
dev.em.0.interrupts.tx_queue_empty: 0
dev.em.0.interrupts.tx_queue_min_thresh: 0
dev.em.0.interrupts.rx_desc_min_thresh: 0
dev.em.0.interrupts.rx_overrun: 0
Interface is RUNNING and INACTIVE
em0: hw tdh = 221, hw tdt = 221
em0: hw rdh = 467, hw rdt = 466
em0: Tx Queue Status = 0
em0: TX descriptors avail = 1024
em0: Tx Descriptors avail failure = 0
em0: RX discarded packets = 0
em0: RX Next to Check = 467
em0: RX Next to Refresh = 466
$
from 8.2:
Name Mtu Network Address Ipkts Ierrs Idrop Ibytes Opkts Oerrs Obytes Coll Drop
em0 1500 <Link#1> 00:30:48:2e:99:ba 276150 0 0 42614583 285398 0 207023352 0 0
em0 1500 192.147.25.0/ 192.147.25.65 318062 - - 48331105 285363 - 203026287 - -
em0 1500 fe80::230:48f fe80::230:48ff:fe 0 - - 0 1 - 96 - -
em0 1500 192.147.25.0/ 192.147.25.45 25071 - - 1782211 0 - 0 - -
em0 1500 192.147.25.0/ 192.147.25.11 38433 - - 2742827 0 - 0 - -
dev.em.0.%desc: Intel(R) PRO/1000 Legacy Network Connection 1.0.3
dev.em.0.%driver: em
dev.em.0.%location: slot=2 function=0
dev.em.0.%pnpinfo: vendor=0x8086 device=0x1079 subvendor=0x15d9 subdevice=0x117a class=0x020000
dev.em.0.%parent: pci3
dev.em.0.nvm: -1
dev.em.0.rx_int_delay: 0
dev.em.0.tx_int_delay: 66
dev.em.0.rx_abs_int_delay: 66
dev.em.0.tx_abs_int_delay: 66
dev.em.0.rx_processing_limit: 100
dev.em.0.flow_control: 3
dev.em.0.mbuf_alloc_fail: 0
dev.em.0.cluster_alloc_fail: 0
dev.em.0.dropped: 0
dev.em.0.tx_dma_fail: 0
dev.em.0.tx_desc_fail1: 0
dev.em.0.tx_desc_fail2: 0
dev.em.0.rx_overruns: 0
dev.em.0.watchdog_timeouts: 0
dev.em.0.device_control: 1089471041
dev.em.0.rx_control: 32770
dev.em.0.fc_high_water: 47104
dev.em.0.fc_low_water: 45604
dev.em.0.fifo_workaround: 0
dev.em.0.fifo_reset: 0
dev.em.0.txd_head: 73
dev.em.0.txd_tail: 75
dev.em.0.rxd_head: 130
dev.em.0.rxd_tail: 129
dev.em.0.mac_stats.excess_coll: 0
dev.em.0.mac_stats.single_coll: 0
dev.em.0.mac_stats.multiple_coll: 0
dev.em.0.mac_stats.late_coll: 0
dev.em.0.mac_stats.collision_count: 0
dev.em.0.mac_stats.symbol_errors: 0
dev.em.0.mac_stats.sequence_errors: 0
dev.em.0.mac_stats.defer_count: 0
dev.em.0.mac_stats.missed_packets: 0
dev.em.0.mac_stats.recv_no_buff: 0
dev.em.0.mac_stats.recv_undersize: 0
dev.em.0.mac_stats.recv_fragmented: 0
dev.em.0.mac_stats.recv_oversize: 0
dev.em.0.mac_stats.recv_jabber: 0
dev.em.0.mac_stats.recv_errs: 0
dev.em.0.mac_stats.crc_errs: 0
dev.em.0.mac_stats.alignment_errs: 0
dev.em.0.mac_stats.coll_ext_errs: 0
dev.em.0.mac_stats.xon_recvd: 0
dev.em.0.mac_stats.xon_txd: 0
dev.em.0.mac_stats.xoff_recvd: 0
dev.em.0.mac_stats.xoff_txd: 0
dev.em.0.mac_stats.total_pkts_recvd: 276318
dev.em.0.mac_stats.good_pkts_recvd: 276318
dev.em.0.mac_stats.bcast_pkts_recvd: 8
dev.em.0.mac_stats.mcast_pkts_recvd: 0
dev.em.0.mac_stats.rx_frames_64: 9012
dev.em.0.mac_stats.rx_frames_65_127: 205540
dev.em.0.mac_stats.rx_frames_128_255: 44078
dev.em.0.mac_stats.rx_frames_256_511: 3166
dev.em.0.mac_stats.rx_frames_512_1023: 3934
dev.em.0.mac_stats.rx_frames_1024_1522: 10588
dev.em.0.mac_stats.good_octets_recvd: 43761022
dev.em.0.mac_stats.good_octets_txd: 208238998
dev.em.0.mac_stats.total_pkts_txd: 285534
dev.em.0.mac_stats.good_pkts_txd: 285534
dev.em.0.mac_stats.bcast_pkts_txd: 22
dev.em.0.mac_stats.mcast_pkts_txd: 3
dev.em.0.mac_stats.tx_frames_64: 6172
dev.em.0.mac_stats.tx_frames_65_127: 75983
dev.em.0.mac_stats.tx_frames_128_255: 53030
dev.em.0.mac_stats.tx_frames_256_511: 23216
dev.em.0.mac_stats.tx_frames_512_1023: 1472
dev.em.0.mac_stats.tx_frames_1024_1522: 125661
dev.em.0.mac_stats.tso_txd: 0
dev.em.0.mac_stats.tso_ctx_fail: 0
$ sudo sysctl dev.em.0.debug=1
sysctl: unknown oid 'dev.em.0.debug'
$
--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 512-248-2683 E-Mail: ler at lerctr.org
US Mail: 430 Valona Loop, Round Rock, TX 78681-3893
More information about the freebsd-stable
mailing list