Problems with bge (possibly related to r208993)

[Artem Kim]

On Tuesday 15 June 2010 21:50:03 you wrote:
. . .
> > nas2 # netstat-ndI bge1
> > Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll Drop
> > bge1 1500 <Link#3> 00:1 b: 78: a3: 3c: 01 418543876 1972918 0 446063237 0
> > 0 0 bge1 1500 XX.XX.6.12 XX.XX.6.133 890,306 - - 1,076,833 - - -
> 
> Ok, I see very large number of Ierrs here. When you send some packets
> from other hosts to nas2(bge1), do you see Ierrs counter is
> increasing?
. . .
 
> It seems RX does not work at all. Because you have zero Drop(from
> netstat) I think you didn't hit mbuf resource shortage situation.
> Ierr counter is increased whenever controller drops frames due to
> receiving errors(e.g. CRC). Given that you have no cabling issue,
> it could be caused by speed/duplex mismatches between bge1 and link
> partner. Does the link partner also agrees on resolved speed/duplex
> of bge1?

I had some negotiation problems. But the problems were observed on the other 
NIC - bge0. bge0 is connected to the dlink-3627 and bge1 is not always setup 
speed/duplex mode correctly. Usually this is solved by link0 setting. Flag 
link0 I set for bge1 and bge0. Flag link0 used quite a long time (years).

bge1 and bge0 have link0, when I got the problem on NAS2 first time. Then I 
reset link0 and reboot NAS2. After some time I got the same problem again 
(current state). However, I do not see any obvious problems with  bge0 <-> AT-
x900.

current state of the bge0 link partner:

awplus>show int port1.0.12
Interface port1.0.12
  Scope: both
  Link is UP, administrative state is UP
  Thrash-limiting
    Status Not Detected, Action learn-disable, Timeout 1(s)
  Hardware is Ethernet, address is 0000.cd29.6e09
  index 5012 metric 1 mru 1522
  current duplex full, current speed 1000, polarity auto
  configured duplex auto, configured speed auto
  <UP,BROADCAST,RUNNING,MULTICAST>
  VRF Binding: Not bound
  SNMP link-status traps: Disabled
    input packets 136255660241, bytes 119549292157319, dropped 0, multicast 
packets 5482013
    output packets 122988526534, bytes 121030195520423, multicast packets 
532582 broadcast packets 2198512

awplus>show int port1.0.12 status
Port       Name               Status           Vlan Duplex   Speed Type
port1.0.12                    connected          55 a-full  a-1000 1000BASE-T

awplus>sh mac address-table |i port1.0.12
55   port1.0.12   001b.78a3.3c01   forward   dynamic


nas2# ifconfig bge1
bge1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8009b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LINKSTATE>
        ether 00:1b:78:a3:3c:01
        inet XX.XX.6.133 netmask 0xffffffc0 broadcast XX.XX.6.191
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active


I tried to do ping -i .01 XX.XX.6.133  from other host:
nas2# netstat -hI bge1 1
            input         (bge1)           output
   packets  errs idrops      bytes    packets  errs      bytes colls
         0     0     0          0          0     0          0     0
         0     1     0          0          0     0          0     0
         0     0     0          0          0     0          0     0
         0     0     0          0          0     0          0     0
ping-> 0    33     0          0          0     0          0     0
         0    94     0          0          0     0          0     0
         0    93     0          0          0     0          0     0
         0    94     0          0          0     0          0     0
         0    94     0          0          0     0          0     0
         0    83     0          0          0     0          0     0
         0     0     0          0          0     0          0     0


ping -i .01 XX.XX.6.129 from NAS2 (XX.XX.6.129 have static arp-entry):

nas2# netstat -hI bge1 1
            input         (bge1)           output
   packets  errs idrops      bytes    packets  errs      bytes colls
         0     1     0          0          0     0          0     0
         0     1     0          0          0     0          0     0
         0     0     0          0          0     0          0     0
ping->   0     0     0          0          0     0          0     0
         0    40     0          0         62     0       5.9K     0
         0    93     0          0         89     0       8.5K     0
         0    91     0          0         89     0       8.5K     0
         0    91     0          0         88     0       8.4K     0
         0    91     0          0         89     0       8.5K     0
         0    92     0          0         88     0       8.4K     0
         0    93     0          0         88     0       8.4K     0
         0    92     0          0         89     0       8.5K     0
         0     0     0          0         85     0       8.1K     0
         0    87     0          0          0     0          0     0


ping -i .01 XX.XX.6.133 from other host:

before:
nas2# netstat -ndI bge1

Name    Mtu Network       Address              Ipkts Ierrs Idrop    Opkts 
Oerrs  Coll Drop
bge1   1500 <Link#3>      00:1b:78:a3:3c:01 418543876 2042520     0 446111781     
0     0    0
bge1   1500 XX.XX.6.12 XX.XX.6.133      890306     -     -  1125376     -     
-    -

after:
Name    Mtu Network       Address              Ipkts Ierrs Idrop    Opkts 
Oerrs  Coll Drop
bge1   1500 <Link#3>      00:1b:78:a3:3c:01 418543876 2043445     0 446111781     
0     0    0
bge1   1500 XX.XX.6.12 XX.XX.6.133      890306     -     -  1125376     -     
-    -