Continuing problem with RPI USB-based network adapters

[Hans Petter Selasky]

On 06/13/17 16:45, Karl Denninger wrote:
> A good long while back I tried to run down an apparent problem with
> ue-based network drivers that seemed to be linked to having more than
> one interface instance attached to a physical interface -- such as using
> "ue0" for the "base" link and "ue0.2" for VLAN 2 on the same physical wire.
> 
> The symptoms would be that the interface would "flap" every 10 or 20
> minutes; it would go down an up without apparent cause.
> 
> I can now quite-reliably report that it's not linked to VLANs; it also
> appears to show up *ANY* time there are multiple instances of an
> Ethernet interface up on the "ue" driver, irrespective of whether it's
> multiple instances on one interface (e.g. the VLAN example) OR multiple
> instances on multiple physical interfaces (e.g. ue0, ue1 on a plugged-in
> USB ethernet instance, etc.)
> 
> I have _*41 days*_ of uptime at present on a single-instance device with
> ZERO flaps.  But on a device with three instances, one with two physical
> interfaces in which one has a VLAN and base, the other just a base
> interface, it happens every few minutes.  If I "down" the VLAN interface
> /it still happens./
> 
> Jun 13 09:04:53 IPGw kernel: ue0.3: link state changed to UP
> Jun 13 09:25:46 IPGw kernel: ue0: link state changed to DOWN
> Jun 13 09:25:46 IPGw kernel: ue0.3: link state changed to DOWN
> Jun 13 09:25:46 IPGw kernel: ue0: link state changed to UP
> Jun 13 09:25:46 IPGw kernel: ue0.3: link state changed to UP
> Jun 13 09:37:50 IPGw kernel: ue0: link state changed to DOWN
> Jun 13 09:37:50 IPGw kernel: ue0.3: link state changed to DOWN
> Jun 13 09:37:50 IPGw kernel: ue0: link state changed to UP
> Jun 13 09:37:50 IPGw kernel: ue0.3: link state changed to UP
> 
> If there are logging entries that I can enable to try to find the cause
> of this it would be great -- this particular device is a RPI3 running
> -HEAD, but the issue traces back to at least 11.0 on the RPI2, where I
> saw it repeatedly close to a year ago, and there has apparently been no
> resolution.
> 
> This looks PR-worthy but without some sort of trace on the REASON for
> the flap it's not so useful, thus the question as to whether I can dig
> up a logging option that will inform as to *why* the interface was
> marked down.  It is NOT the switch port that the unit is plugged into OR
> the physical RPI3 hardware; I have swapped both the RPI3 board and
> switch port but the problem still exists and the other unit with one
> interface in service and NO flaps over 41 days of uptime is plugged into
> the same physical network switch.

Hi Karl,

The link state changed events come from the PHY on the devices you have 
shown. This is handled by the so-called MII bus code in FreeBSD.

sys/dev/mii/smscphy.c

> static void
> smscphy_status(struct mii_softc *sc)
> {
>         struct mii_data *mii;
>         uint32_t bmcr, bmsr, status;
> 
>         mii = sc->mii_pdata;
>         mii->mii_media_status = IFM_AVALID;
>         mii->mii_media_active = IFM_ETHER;
> 
>         bmsr = PHY_READ(sc, MII_BMSR) | PHY_READ(sc, MII_BMSR);
>         if ((bmsr & BMSR_LINK) != 0)
>                 mii->mii_media_status |= IFM_ACTIVE;


I see the code is reading the MII_BMSR status register multiple times. 
Maybe this is to cover up some kind of bug.

I suggest as a first step, change the two places where MII_BMSR is read 
to read it 4 times instead of two. Does it make any difference.

Also I would suggest that you can add some prints to inspect the 
individual values of each MII_BMSR read, to see what is going on.

CC'ed Benno and Oleksandr in case they have any suggestion.

--HPS