bce packet loss

[David Christensen]

> > There won't be any indication in the driver since flow control
> > is managed in hardware.  You'd need a wire capture to see that
> > bce(4) has stopped sending frames in response to receiving an
> > XOFF flow control frame or started sending frames in response
> > to receiving an XON flow control frame.
> 
> Ah.  I was hoping for something in the ifconfig output.  I'll see if
> tcpdump and wireshark can tell me anything about this host.
> 

Flow control frames are consumed by the MAC and aren't passed
to the OS stack so you won't see them with a host based packet
capture application like tcpdump.  You'd need something like
a JDSU Xgig protocol analyzer to see them on the wire.

> One the one host (w/bce) I just set to full auto, the switch claims to
> have negotiated 1000FD w/flow control (this specifically shows as
> "auto+enabled" on the switch side).
> 
> I see that the "sysctl dev.bce.1" tree has some info, and I can see that
> the NIC is receiving flow control frames:
> 
> dev.bce.1.stat_XonPauseFramesReceived: 16638
> dev.bce.1.stat_XoffPauseFramesReceived: 17239
> 
> These lines are a bit puzzling though:
> 
> dev.bce.1.stat_FlowControlDone: 0
> dev.bce.1.stat_XoffStateEntered: 0

Auto-negotiation occurs between PHYs and then the link comes
up.  When link is up bce(4) will configure the MAC based on
the auto-neg results returned by brgphy(4) (the switch should
follow a similar procedure on its side).  Flow control occurs
at the MAC level.  It looks like the MAC is not configured to 
support flow control as negotiated by the PHY during auto-neg,
resulting in flow control frames being ignored.

The driver will set this in the MAC based on PHY results:
http://fxr.watson.org/fxr/source/dev/bce/if_bce.c#L2046
Either the PHY results are coming back incorrectly or it
may be firmware on bce(4) is disabling the behavior even
though bce(4) correctly enables it.

Dave