Unreliable bge link state events (and associated dhclient problems)

[Brooks Davis]

On Wed, Jul 27, 2005 at 10:48:04PM -0700, Ben Jencks wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On my Thinkpad T43p, the bge card seems to be unreliable in giving link
> state events. This causes dhclient to behave incorrectly (at least my
> understanding of correctly, given what I've read on -current, -questions
> and -stable).

Thanks for the detailed writeup.  It should give us some places to start
looking.

> I'm starting with the interface up and fully configured through
> dhclient.
> 
> brj at wagner$ ifconfig bge0
> bge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
>         options=1a<TXCSUM,VLAN_MTU,VLAN_HWTAGGING>
>         inet6 fe80::211:25ff:feb2:bc6c%bge0 prefixlen 64 scopeid 0x1 
>         inet6 2001:5c0:8316:0:211:25ff:feb2:bc6c prefixlen 64 autoconf 
>         inet 10.11.55.6 netmask 0xfffffff8 broadcast 10.11.55.7
>         ether 00:11:25:b2:bc:6c
>         media: Ethernet autoselect (1000baseTX <full-duplex>)
>         status: active
> 
> Now I unplug the cable.
> dmesg does not report any change. When I ask ifconfig, I get
> 
> brj at wagner$ ifconfig bge0
> bge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
>         options=1a<TXCSUM,VLAN_MTU,VLAN_HWTAGGING>
>         inet6 fe80::211:25ff:feb2:bc6c%bge0 prefixlen 64 scopeid 0x1 
>         inet6 2001:5c0:8316:0:211:25ff:feb2:bc6c prefixlen 64 autoconf 
>         inet 10.11.55.6 netmask 0xfffffff8 broadcast 10.11.55.7
>         ether 00:11:25:b2:bc:6c
>         media: Ethernet autoselect (none)
>         status: no carrier
> 
> Also, upon querying ifconfig, dmesg reports:
> bge0: link state changed to DOWN
> 
> dhclient stays running, and the IP address is still configured, even
> after the kernel notices the link state change.

The ifconfig polling triggering updates seem to be a common issue.  I've
also seen it reported with fxp(4).  IIRC, we also need this fixed for
CARP.  The fact that dhclient stays running is odd and troubling.  The
dmesg output is generated by the same function that triggers the routing
message so something weird is going on there.

> When I then plug the cable back in, it requires another ifconfig poll to
> get the kernel to notice.
> 
> If I manually stop dhclient after unplugging, bge0 is taken down.
> bge0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
>         options=1a<TXCSUM,VLAN_MTU,VLAN_HWTAGGING>
>         inet6 fe80::211:25ff:feb2:bc6c%bge0 prefixlen 64 scopeid 0x1 
>         inet6 2001:5c0:8316:0:211:25ff:feb2:bc6c prefixlen 64 autoconf 
>         inet 10.11.55.6 netmask 0xfffffff8 broadcast 10.11.55.7
>         ether 00:11:25:b2:bc:6c
>         media: Ethernet autoselect (none)
>         status: no carrier
> 
> (The address is still configured after dhclient exits, is this right?)

Maybe.  It inhibits moving to another interface on the same network, but
prevents some connections from dying prematurely.  If you're stopping
dhclient with /etc/rc.d/dhclient, it's done by "ifconfig <ifn> down" so
that's expected.  I'm no longer sure it's the right thing to do though.

> My understanding was that downed interfaces don't report
> events. However, when I plug the cable in when it's down, it is
> immediately noticed, plus dhclient is started properly.

This seems to be wrong assuming we want up and down to be administrative
states.  I suspect we won't complete the handling of that in time for
6.0 given the number of drivers in the tree, but I think it's the right
end point.

> If, after unplugging and stopping dhclient, I manually ifconfig bge0 up,
> then plug in again, I still get the correct behavior (event is
> immediately noticed, and dhclient is started).
> 
> I also tried without bge0 configured in rc.conf. I found that the only
> event that reliably appears when it should, rather than on an ifconfig
> poll, is the UP event after ifconfig bge0 down; ifconfig bge0 up. Once I
> got a down event properly, but I couldn't reproduce it.
> 
> So, the problem seems to be that link state events are not generated
> properly after the first UP event after ifconfig bge0 up. Also, dhclient
> doesn't seem to respond to the DOWN events generated by an ifconfig
> poll.
>
> bge0 at pci2:0:0:  class=0x020000 card=0x05771014 chip=0x167d14e4 rev=0x11 hdr=0x00
>     vendor   = 'Broadcom Corporation'
>     device   = 'BCM5750A1M NetXtreme Gigabit Ethernet PCI Express'
>     class    = network
>     subclass = ethernet
> 
> 
> Sorry for the long post, but I figured better too much info than too
> little.

Thanks again for the writeup.  I'll have to see what I can see with one
of my bge equipped machines.

-- Brooks

-- 
Any statement of the form "X is the one, true Y" is FALSE.
PGP fingerprint 655D 519C 26A7 82E7 2529  9BF0 5D8E 8BE9 F238 1AD4
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20050727/5f8998f7/attachment.bin