rpi4 network boot

[Matthew Grooms]

On 2/7/2021 10:12 PM, Mike Karels wrote:
> On 7 Feb 2021, at 19:24, Matthew Grooms wrote:
>
>> On 2/7/2021 2:00 PM, Klaus Küchemann wrote:
>>> Am 07.02.2021 um 20:19 schrieb Torfinn Ingolfsen 
>>> <torfinn.ingolfsen at getmail.no>:
>>>> I believe he is asking you to verify that the fstab file for the 
>>>> _client_ (aka the diskless machine your'e trying to boot) is 
>>>> correct. That file exists as
>>>> ${NFSROOTDIR}/etc/fstab  on the boot server.
>>>>
>>>> HTH
>>>> -- 
>>>> Torfinn Ingolfsen <torfinn.ingolfsen at getmail.no>
>>>> _______________________________________________
>>>> freebsd-arm at freebsd.org mailing list
>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-arm
>>>> To unsubscribe, send any mail to "freebsd-arm-unsubscribe at freebsd.org“
>>> Yes, the fstab-setup is important…... but then I realized Matthew`s 
>>> boot environment :
>>> He has u-boot sitting on the pxe-Server, whereas the eeprom was 
>>> flashed for completely diskless netboot
>>> as a requirement .
>>> With that setup in fact we will end up in n infinite loop of an 
>>> 'genet0: gen_intr2‘ -console-output,
>>> regardless of fstab or whatever configuration elsewhere I tried.
>> Thanks for testing and re-producing that. Corroborating test results 
>> were also posted by others in the FreeBSD forum thread I referenced 
>> earlier.
>>> I have now refreshed the eeprom to it’s default(where diskless 
>>> pxe-boot is disabled )
>>> And again successfully net-booted (14_current in this case) :
>>>
>>> https://dmesgd.nycbug.org/index.cgi?do=view&id=5917
>>>
>>> Since netboot from u-boot on the uSD requiresdonly 1 command by hand 
>>> (load net0:/boot/kernel/kernel
>>> )(+ command :’boot’) and there’s no problem with the genet-driver I 
>>> guess that
>>> It’s an issue of the pxe-flashed eeprom where we don’t yet know 
>>> exactly what happens.
>>
>> I have a slightly different take: The network device appears to work 
>> well initially when used by the eeprom and after the hand-off to 
>> u-boot. The FreeBSD genet driver generally works well unless the 
>> device is first used by eeprom/u-boot. I suspect that u-boot is 
>> performing some task that leaves the device in a state that is 
>> incompatible what FreeBSD expects and/or can work with. Perhaps there 
>> is a way to force some a full device reset of genet either before 
>> u-boot hands things off to FreeBSD or right after FreeBSD loads the 
>> genet driver? If so, maybe that would be worth trying, even as a 
>> hacky test, to see if it's the network device state that's wonky.
>
> That’s a reasonable theory, but I don’t actually know how to reset the 
> chip
> (I’ve never seen documentation for it).  I’m also not confident that the
> driver would fully initialize the chip from a reset.  However, there is
> probably more to the problem.  Klaus told me that the irq values are
> different when using PXE, which means the OS is getting different config
> information as well.  The irqs print in the main auto configuration line
> for genet0.  But it’s possible that something as simple as calling the
> same interrupt routine from both interrupts might work.
>
Hi Mike,

Thanks for the feedback and suggestion. I tried modifying the driver to 
use the same handler for both interrupts ...

Index: if_genet.c
===================================================================
--- if_genet.c  (revision 369225)
+++ if_genet.c  (working copy)
@@ -291,7 +291,7 @@
         }

         error = bus_setup_intr(dev, sc->res[_RES_IRQ2],
-           INTR_TYPE_NET | INTR_MPSAFE, NULL, gen_intr2, sc, &sc->ih2);
+           INTR_TYPE_NET | INTR_MPSAFE, NULL, gen_intr, sc, &sc->ih2);
         if (error != 0) {
                 device_printf(dev, "cannot setup interrupt handler2\n");
                 goto fail;

And probably screwed that up, but the gen_intr2 message doesn't get 
spewed to the console any longer. Instead, it just seems to lock up in 
the same place. Having a quick glance a the Linux driver, it would 
appear that they use two interrupt routines to two service different 
sets of Rx/Tx queues, priority and default ...

/* bcmgenet_isr1: handle Rx and Tx priority queues */

https://github.com/torvalds/linux/blob/master/drivers/net/ethernet/broadcom/genet/bcmgenet.c#L3077

/* bcmgenet_isr0: handle Rx and Tx default queues + other stuff */

https://github.com/torvalds/linux/blob/master/drivers/net/ethernet/broadcom/genet/bcmgenet.c#L3125

Perhaps the eeprom or u-boot drivers are doing something similar before 
handing off to FreeBSD? If so, could it be that the second interrupt is 
being called to handle additional queue things that FreeBSD isn't aware 
of? Forgive me for writing things I don't entirely comprehend :(

Thanks,

-Matthew