Ubiquiti EdgeRouter Lite works multi-user with -CURRENT.

[Joe Holden]

Juli Mallett wrote:
> On Sat, May 25, 2013 at 7:35 AM, Milan Obuch <freebsd-mips at dino.sk> wrote:
> 
>> On Thu, 23 May 2013 10:56:04 -0700, Juli Mallett <jmallett at FreeBSD.org>
>> wrote:
>>
>> [ snip ]
>>
>>> In this case, I'd say that the most useful thing to instrument is
>>> likely cvm_do_timer in sys/mips/cavium/octe/ethernet.c.  I'd suggest
>>> first changing:
>>>
>>> 138 if (priv->need_link_update) {
>>> 139 updated++;
>>> 140 taskqueue_enqueue(cvm_oct_link_taskq, &priv->link_task);
>>> 141 }
>>>
>>> To be just
>>>
>>> taskqueue_enqueue(cvm_oct_link_taskq, &priv->link_task);
>>>
>> It took me some time to respond for some reasons, anyway I built a
>> kernel with this change and there is something clearly wrong. There is
>> basically no change in behavior, if booted with gigabit connection, it
>> works with gigabit speed but not with 100 megabit and vice versa. Just
>> some cometic change - once per two seconds or so I keep getting on
>> console following
>>
>> octe0: Link down
>> octe1: Link down
>> octe2: Link down
>>
>> It looks like some phy register is not read correctly or not correct
>> phy register is read and acted upon. Just a guess, of course.
>> Interesting thing also not yet mentioned here - ifconfig octeN output
>> show correct speed in its media: line.
>>
>> Also, when changing cables, I got following printed on console:
>>
>> Port 0 receive error code 13, packet dropped
>>
>> No idea it if shows something, for now.
>>
>>> If that helps, then I think there's two things that need to be done:
>>>
>>> First, we shouldn't use slow-polling to do that anyway, we should just
>>> call taskqueue_enqueue whenever we set need_link_update to 1 if it was
>>> not 1 already.  This is mostly a structural-cosmetic kind of thing.
>>>
>>> Second, cvm_oct_rgmii_poll in ethernet-rgmii.c is probably wrong in
>>> some way.  Instrumenting it to print out the various register states
>>> and see if a human can see link status changes even if it can't may be
>>> helpful.  Alternately, you may want/need to look at UBNT's patches for
>>> the ERLite.  I may have missed some change it requires (if so, please
>>> let me know rather than just committing the changes; I've tried to
>>> isolate and keep minimal vendor-specific changes since we support so
>>> many vendors; not every vendor shares my concern, and a lot of them
>>> make the code too complex, or other things we don't want.)  It could
>>> be just a matter of not using/programming/undermining RGMII in the way
>>> the hardware wants, or it could be that we need a PHY driver or
>>> something else.
>>>
>>> Actually, looking at how the hardware is set up, it looks like we
>>> should be using a PHY driver already, so it may be that none of that
>>> is helpful (since the internal link status management is only used if
>>> we don't have a PHY) and the real fix needs to happen in atphy.c.  Or
>>> that for this hardware we should just use the internal link status
>>> management rather than using the PHY.
>>>
>> In dmesg, following is relevant to ethernet, I think:
>>
>> octebus0: <Cavium Octeon Ethernet pseudo-bus> on ciu0
>> Interface 0 has 3 ports (RGMII)
>> Warning: Enabling IPD when IPD already enabled.
>> Warning: Enabling PKO when PKO already enabled.
>> octe0: <Cavium Octeon RGMII Ethernet> on octebus0
>> miibus0: <MII bus> on octe0
>> atphy0: <Atheros F1 10/100/1000 PHY> PHY 7 on miibus0
>> atphy0: OUI 0x00c82e, model 0x0007, rev. 2
>> atphy0:  none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX,
>> 1000baseSX-FDX, 1000baseT-FDX, 1000baseT-FDX-master, auto octe0: bpf
>> attached octe0: Ethernet address: dc:9f:db:29:a3:92
>> octe1: <Cavium Octeon RGMII Ethernet> on octebus0
>> miibus1: <MII bus> on octe1
>> atphy1: <Atheros F1 10/100/1000 PHY> PHY 6 on miibus1
>> atphy1: OUI 0x00c82e, model 0x0007, rev. 2
>> atphy1:  none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX,
>> 1000baseSX-FDX, 1000baseT-FDX, 1000baseT-FDX-master, auto octe1: bpf
>> attached octe1: Ethernet address: dc:9f:db:29:a3:93
>> octe2: <Cavium Octeon RGMII Ethernet> on octebus0
>> miibus2: <MII bus> on octe2
>> atphy2: <Atheros F1 10/100/1000 PHY> PHY 5 on miibus2
>> atphy2: OUI 0x00c82e, model 0x0007, rev. 2
>> atphy2:  none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX,
>> 1000baseSX-FDX, 1000baseT-FDX, 1000baseT-FDX-master, auto octe2: bpf
>> attached octe2: Ethernet address: dc:9f:db:29:a3:94
>>
>> so maybe something should be done in atphy driver as someone mentioned
>> later in this thread or between phy driver and some cavium code... I am
>> going to try to undestand the code in cavium/octe directory, but any
>> hint would be welcome for sure.
>>
> 
> Ok, well, I have a slightly-radical hint :)
> 
> In cvm_oct_mdio_setup_device, set phy_id to -1 instead of to
> cvmx_helper_board_get_mii_address(...).  It's a silly thought, but I think
> there's a chance it might work.  This would be using the internal link
> state management rather than using a PHY.  (This isn't the right change to
> make to effect that, but it's worth a shot.)
> 
> Thanks,
> Juli.
Just gave that a shot - results in unknown link type and no connectivity