Re: Saving environment variables in u-boot

On Thu, Dec 16, 2021 at 11:00:46PM -0800, Mark Millard wrote:
> On 2021-Dec-16, at 17:36, bob prohaska <fbsd@www.zefox.net> wrote:
> 
> > 
> > Yes, the microSD contains a dd of 
> > FreeBSD-13.0-RELEASE-arm64-aarch64-RPI.img
> > The DOS partition is writeable, AFAIK.
> 
> Hmm. That means there is also a UFS root with
> a kernel and world on the microsd card. Both
> the microsd card and the USB drive contain
> contain such, as well as each having an
> /etc/fstab (and so on).
> 
> Having multiple such makes for a mess for
> controlling which is used (and knowing which
> is used). What is the first stage that you
> made sure the USB drive was in use and how
> did you make sure?
> 

I installed a microSD containing only the FAT slice from 
FreeBSD-13.0-RELEASE-arm64-aarch64-RPI.img, the FreeBSD
slice being deleted using gpart. There is a timeout file.

A hands-off warm boot reports
resetting system ... 

U-Boot 2020.10 (Apr 09 2021 - 03:55:54 +0000)

DRAM:  948 MiB
RPI 3 Model B (0xa02082)
MMC:   mmc@7e300000: 0
Loading Environment from FAT... In:    serial
Out:   vidconsole
Err:   vidconsole
Net:   No ethernet found.
starting USB...
Bus usb@7e980000: USB DWC2
scanning bus usb@7e980000 for devices... 5 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found
Hit any key to stop autoboot:  0 
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
Found EFI removable media binary efi/boot/bootaa64.efi
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
Scanning disk mmc@7e300000.blk...
Found 2 disks
No EFI system partition
BootOrder not defined
EFI boot manager: Cannot load any image
1259212 bytes read in 125 ms (9.6 MiB/s)
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
Booting /efi\boot\bootaa64.efi
[whitespace deleted]
Consoles: EFI console  
    Reading loader env vars from /efi/freebsd/loader.env
Setting currdev to disk0p1:
FreeBSD/arm64 EFI loader, Revision 1.1

   Command line arguments: loader.efi
   Image base: 0x39e03000
   EFI version: 2.80
   EFI Firmware: Das U-Boot (rev 8224.4096)
   Console: comconsole (0)
   Load Path: /efi\boot\bootaa64.efi
   Load Device: /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(2)/SD(0)/HD(1,0x01,0,0x81f,0x18fa8)
Trying ESP: /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(2)/SD(0)/HD(1,0x01,0,0x81f,0x18fa8)
Setting currdev to disk0p1:
ERROR: cannot open /boot/lua/loader.lua: no such file or directory.


Type '?' for a list of commands, 'help' for more detailed help.
OK 

AIUI, disk0p1 is the microSD, which has no /boot/lua/loader.lua
That makes sense. If u-boot finds the USB mass storage device
then running 
run bootcmd_usb0 starts the system successfully.

It does appear that the FAT partition on the USB disk is involved.
If I hide the contents of da0s1 by placing them in a subdirectory
the boot fails, even if u-boot has found the USB disk:

Hit any key to stop autoboot:  0 
U-Boot>   usb reset
resetting USB...
Bus usb@7e980000: USB DWC2
scanning bus usb@7e980000 for devices... 6 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found

[Note that the six devices include the disk, it can be seen in usb tree.
For some reason it isn't recognized as a mass storage device.]

U-Boot> editenv usb_pgood_delay
edit: 2
U-Boot> usb reset
resetting USB...
Bus usb@7e980000: USB DWC2
scanning bus usb@7e980000 for devices... 6 USB Device(s) found
       scanning usb for storage devices... 1 Storage Device(s) found
U-Boot> run bootcmd_usb0

Device 0: Vendor: SABRENT  Rev: 1214 Prod: 
            Type: Hard Disk
            Capacity: 953869.7 MB = 931.5 GB (1953525168 x 512)
... is now current device
Scanning usb 0:1...
U-Boot> 
and there it stops. Looks like both FAT partitions have a role to play.

An earlier experiment tried booting the microSD card in a USB adapter,
That worked correctly with nothing in the microSD slot, so the internal
"boot from USB" feature might work if it could be slowed down sufficiently. 
> 
> The order of activity is:
> 
> MSDOSFS: (all on one of: microsd card vs. usb drive)
> RPi* firmware (I do not report the file-level sequencing)
> U-Boot
> FreeBSD loader (which uses information from U-Boot)
> 
> UFS: (both: together on one of: microsd card vs. usb drive)
> FreeBSD kernel
> FreeBSD world
> 
> (I do not specify which MSDOSFS is used when
> there are multiple viable ones in separate places.
> Similarly for UFS.)
>
From the experiment today it seems both are required.
The first discovers the USB disk, the second uses that
information to proceed further. 
 
> 
> (It is technically possible to have kernel vs.
> world in separate UFS partitions, possibly on
> separate media. I've an example context that I
> deal with that way to avoid a U-boot limitation
> for the device: the kernel can find the world
> where the U-Boot/FreeBSD-Loader can not. (The
> FreeBSD loader depends on what USB can find: it
> does not look elsewhere.) I only mention this
> in case I need to reference it in the future,
> avoiding a surprise in such a case. Otherwise
> ignore the complication.)
> 

Are you referring to a case where the root is on
microSD but /usr and friends is on a USB device? 


> You might move (not copy) the MSDOSFS material
> between the microsd card and the usb drive during
> experiments, avoiding having duplications. There
> is the possibility of instead renaming things so
> files are not found on a partition, for example.
> A similar point goes for UFS materials: avoid
> having multiple viable-for-booting UFS file
> systems present.
> 
> As stands, things seem too hard to track for me
> to be of much help. Please make things obvious for
> what was in use by making the material uniquely
> placed (for the form that can be used).
> 
I believe the experiment above (deleting UFS on microSD, 
hiding the DOS files on USB) has been an equivalent 
configuration. It looks like control passes in some way 
between the DOS slices, though how is unclear.



> Separate issues/questions:
> 
> Do you have the file "timeout" in the MSDOSFS, in
> addition to config.txt and the like? If not, I
> recommend including it.
> 
Yes, it's been tried on both microSD and USB devices. 
Seems like the only one that can matter is on microSD.


> What other RPi* firmware controls for having
> various deliberate RPi* firmware delays do you
> have set up?
>
 
The Raspberry Pi config.txt description lists several delay commands
that can be placed in config.txt. None seem related to USB discovery,
might any come into play early enough to be useful? They're named
bootcode_delay
boot_delay
boot_delay_ms

I tried them casually and didn't notice much change. Are they worth
revisiting more carefully? 

> It is not so much that these would be sufficient,
> but they do establish some context before U-Boot
> is even active. It could be important to
> understand that context. (Unsure at this point.)
> 
> 
> >> But that was for the u-boot-rpi4 or u-boot-rpi-arm64 ports.
> >> (They also later mentioned using "usb_pgood_delay=2000\0"
> >> instead, a figure they found in a bunch of configrations.)
> >> 
> > The Pi3 in question is capable of booting from solid-state USB
> > storage without any microSD card, but fails to detect a mechanical
> > disk. Which is the appropriate u-boot-rpi3 port to tamper with? I 
> > tried sysutils/u-boot-rpi3 as an upgrade but that simply froze. 
> > The u-boot from FreeBSD-13.0-RELEASE-arm64-aarch64-RPI.img finds
> > the USB mechanical disk,  but erratically. 
> 
> FreeBSD-13.0-RELEASE-arm64-aarch64-RPI.img uses u-boot-rpi-arm64
> as I remember: it is set up for both the RPI3 and RPI4. Given that
> it works some, that would be the U-Boot port I would experiment
> with if I was doing the experiments.
> 
> >> So something somewhat analogous might help if you are willing
> >> to build and use your own u-boot port variant.
> > 
> > Obviously, that's a fraught enterprise at my skill level....
> > I'm still somewhat hazy on the actual boot sequence when
> > chaining from microSD to USB. 
> 
> Having the extra text in the U-Boot build does not really
> depend on the chaining order question.
> 
> But, to repeat (sticking to the simpler context),
> the order is:
> 
> MSDOSFS: (all on one of: microsd card vs. usb drive)
> RPi* firmware (I do not report the file-level sequencing)
> U-Boot
> FreeBSD loader (which uses information from U-Boot)
> 
> UFS: (both: together on one of: microsd card vs. usb drive)
> FreeBSD kernel
> FreeBSD world
> 
> 
> > Indeed, it's unclear how or if u-boot plays a role in starting 
> > RasPiOS. The term u-boot isn't found on the Raspberry Pi doc 
> > website, and the Pi isn't mentioned in the Denx manuals. Those
> > discoveries surprised me.
> 
> RaspiOS and RaspiOS64 do not use U-boot. The RPi*'s can boot some
> forms of a linux kernel directly and that is what RaspiOS and
> RaspiOS64 are doing. 

That clears up a major misunderstanding, thank you!

> That is why the config.txt type of content
> makes no mention of u-boot or of any kernel= assignment in RaspiOS
> and RaspiOS64. By contrast, config.txt for FreeBSD lists kernel=
> and an explicit *u-boot*.bin as the value.

Without thinking about it very carefully I tried to use the
"bootcode.bin-only" method for booting an older Pi2 from a
usb hard disk, as described in
https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#special-bootcode-bin-only-boot-mode
It worked on the Pi2 but failed on the Pi3, causing an immediate hang.
Might that avenue be worth pursuing? In hindsight, that it worked on a
Pi2 is quite surprising.  

With my thanks!

bob prohaska