Re: Saving environment variables in u-boot
Date: Sat, 18 Dec 2021 03:12:32 UTC
On 2021-Dec-17, at 19:02, Mark Millard via freebsd-arm <freebsd-arm@freebsd.org> wrote: > On 2021-Dec-17, at 16:59, bob prohaska <fbsd@www.zefox.net> wrote: > >> On Thu, Dec 16, 2021 at 11:00:46PM -0800, Mark Millard wrote: >>> On 2021-Dec-16, at 17:36, bob prohaska <fbsd@www.zefox.net> wrote: >>> >>>> >>>> Yes, the microSD contains a dd of >>>> FreeBSD-13.0-RELEASE-arm64-aarch64-RPI.img >>>> The DOS partition is writeable, AFAIK. >>> >>> Hmm. That means there is also a UFS root with >>> a kernel and world on the microsd card. Both >>> the microsd card and the USB drive contain >>> contain such, as well as each having an >>> /etc/fstab (and so on). >>> >>> Having multiple such makes for a mess for >>> controlling which is used (and knowing which >>> is used). What is the first stage that you >>> made sure the USB drive was in use and how >>> did you make sure? >>> >> >> I installed a microSD containing only the FAT slice from >> FreeBSD-13.0-RELEASE-arm64-aarch64-RPI.img, the FreeBSD >> slice being deleted using gpart. There is a timeout file. > > Ultimately, you may need to publish a full capture of the > debug output that goes to the serial console when such > debug output has been fully enabled. This could end up > being for each of several variations tried. > > Did you still have the MSDOSFS material on the USB in a > viable form as well? > > https://www.raspberrypi.com/documentation/computers/config_txt.html > reports that there are boot options that can go in config.txt : > > bootcode_delay > boot_delay > boot_delay_ms > > Quoting . . . > > QUOTE > bootcode_delay > > The bootcode_delay command delays for a given number of seconds in bootcode.bin before loading start.elf: the default value is 0. > END QUOTE > > QUOTE > boot_delay > > The boot_delay command instructs to wait for a given number of seconds in start.elf before loading the kernel: the default value is 1. The total delay in milliseconds is calculated as (1000 x boot_delay) + boot_delay_ms. This can be useful if your SD card needs a while to get ready before Linux is able to boot from it. > > boot_delay_ms > > The boot_delay_ms command means wait for a given number of milliseconds in start.elf, together with boot_delay, before loading the kernel. The default value is 0. > END QUOTE > > (So boot_delay_ms is only for smaller scale adjustments and can > be ignored here.) > > Here "loading the kernel" means what config.txt was told was > the kernel: some *u-boot*.bin for the FreeBSD context. > > > So one experiment is to have only bootcode.bin and a config.txt > on the microsd card's MSDOSFS, specifying a significantly > large value for bootcode_delay . The hope would be to be able > to get the start*.elf file and the rest from the USB drive. > > If that does not work, then the next is to have the RPi* firmware > only on the microsd card's MSDOSFS and have config.txt also > specify a significantly large value for boot_delay . This > first variation would not have a *u-boot*.bin or the FreeBSD > loader on the microsd card's MSDOSFS but only on the USB > drive's MSDOSFS. > > If that does not work, then the next variation moves just the > *u-boot*.bin file to the microsd card. > > (If that fails then controlling U-Boot's delays can be involved. > For now I only list the path of leaving U-Boot's delays as they > are: one short branch of the exploration.) > > If that does not work, then the next variation moves just the > FreeBSD loader file to the microsd card. For the above one I missed a word, making things unclear. So, instead: If that does not work, then the next variation ALSO moves just the FreeBSD loader file to the microsd card. ( *u-boot*.bin stays on the microsd card from the prior step. ) > With that I'll stop listing things to test, pending results on > what I've listed. > >> A hands-off warm boot reports >> resetting system ... >> >> U-Boot 2020.10 (Apr 09 2021 - 03:55:54 +0000) >> >> DRAM: 948 MiB >> RPI 3 Model B (0xa02082) >> MMC: mmc@7e300000: 0 >> Loading Environment from FAT... In: serial >> Out: vidconsole >> Err: vidconsole >> Net: No ethernet found. >> starting USB... >> Bus usb@7e980000: USB DWC2 >> scanning bus usb@7e980000 for devices... 5 USB Device(s) found >> scanning usb for storage devices... 0 Storage Device(s) found > > I would need to also see the RPi* firmware's prior > debug output to be able to interpret this. I can not > tell for sure which device's U-Boot copy is operating > for the above. I'd guess that is the the micrsd card's > MSDOSFS one. > >> Hit any key to stop autoboot: 0 >> switch to partitions #0, OK >> mmc0 is current device >> Scanning mmc 0:1... > > This looks to be getting a copy of the FreeBSD loader > from the microsd card's MSDOSFS. > > The FreeBSD loader will only see the same Storage Devices > that U-Boot did: it gets the information from U-Boot. > >> Found EFI removable media binary efi/boot/bootaa64.efi > > There was a FreeBSD loader in the microsd card's > MSDOSFS. > >> libfdt fdt_check_header(): FDT_ERR_BADMAGIC >> Scanning disk mmc@7e300000.blk... >> Found 2 disks >> No EFI system partition >> BootOrder not defined >> EFI boot manager: Cannot load any image >> 1259212 bytes read in 125 ms (9.6 MiB/s) >> libfdt fdt_check_header(): FDT_ERR_BADMAGIC >> Booting /efi\boot\bootaa64.efi > > That FreeBSD loader is what it is using, not one > from the USB drive. > >> [whitespace deleted] >> Consoles: EFI console >> Reading loader env vars from /efi/freebsd/loader.env >> Setting currdev to disk0p1: > > "disk0p1" is U-Boot terminology for a drive, despite > this being the FreeBSD loader's output. The FreeBSD > loader uses the U-Boot information, not FreeBSD's > information. > >> FreeBSD/arm64 EFI loader, Revision 1.1 >> >> Command line arguments: loader.efi >> Image base: 0x39e03000 >> EFI version: 2.80 >> EFI Firmware: Das U-Boot (rev 8224.4096) >> Console: comconsole (0) >> Load Path: /efi\boot\bootaa64.efi >> Load Device: /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(2)/SD(0)/HD(1,0x01,0,0x81f,0x18fa8) >> Trying ESP: /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(2)/SD(0)/HD(1,0x01,0,0x81f,0x18fa8) > > This is reporting where the FreeBSD loader was found. > It indicates that disk0p1 was the microsd card's > MSDOSFS. > >> Setting currdev to disk0p1: > > The microsd card's MSDOSFS again. > >> ERROR: cannot open /boot/lua/loader.lua: no such file or directory. > > There is no FreeBSD directory-tree there to find things > like /boot/lua/loader.lua in. > > Currently we are trying to have the /boot/lua/loader.lua come > from the USB drive's UFS file system, not the microsd card. > >> Type '?' for a list of commands, 'help' for more detailed help. >> OK >> >> AIUI, disk0p1 is the microSD, which has no /boot/lua/loader.lua >> That makes sense. If u-boot finds the USB mass storage device >> then running >> run bootcmd_usb0 starts the system successfully. >> >> It does appear that the FAT partition on the USB disk is involved. > > I see no evidence of that above. (Below is a different context.) > >> If I hide the contents of da0s1 by placing them in a subdirectory >> the boot fails, even if u-boot has found the USB disk: >> >> Hit any key to stop autoboot: 0 >> U-Boot> usb reset >> resetting USB... >> Bus usb@7e980000: USB DWC2 >> scanning bus usb@7e980000 for devices... 6 USB Device(s) found >> scanning usb for storage devices... 0 Storage Device(s) found >> >> [Note that the six devices include the disk, it can be seen in usb tree. >> For some reason it isn't recognized as a mass storage device.] >> >> U-Boot> editenv usb_pgood_delay >> edit: 2 >> U-Boot> usb reset >> resetting USB... >> Bus usb@7e980000: USB DWC2 >> scanning bus usb@7e980000 for devices... 6 USB Device(s) found >> scanning usb for storage devices... 1 Storage Device(s) found >> U-Boot> run bootcmd_usb0 >> >> Device 0: Vendor: SABRENT Rev: 1214 Prod: >> Type: Hard Disk >> Capacity: 953869.7 MB = 931.5 GB (1953525168 x 512) >> ... is now current device >> Scanning usb 0:1... >> U-Boot> >> and there it stops. Looks like both FAT partitions have a role to play. > > That is not clear to me. But I'd rather be lookning into > earlier stages at this point, so that "1 Storage Device(s) found" > hopefully ends up being automatic. > > If we get that to be automatic, then what is happening here > becomes the next area to investigate. But the investigation > is not far enough along to justify investigating this now. > >> An earlier experiment tried booting the microSD card in a USB adapter, >> That worked correctly with nothing in the microSD slot, so the internal >> "boot from USB" feature might work if it could be slowed down sufficiently. > > Agreed, that the first thing is to get it to start up > the disk and wait for it. There may be more at issue > after that. > >>> The order of activity is: >>> >>> MSDOSFS: (all on one of: microsd card vs. usb drive) >>> RPi* firmware (I do not report the file-level sequencing) >>> U-Boot >>> FreeBSD loader (which uses information from U-Boot) >>> >>> UFS: (both: together on one of: microsd card vs. usb drive) >>> FreeBSD kernel >>> FreeBSD world >>> >>> (I do not specify which MSDOSFS is used when >>> there are multiple viable ones in separate places. >>> Similarly for UFS.) >>> >> From the experiment today it seems both are required. > > I do not get such an interpretation from your experiment. > Until "1 Storage Device(s) found" happens automatically > the context is not nicely comparable for the later stages. > >> The first discovers the USB disk, the second uses that >> information to proceed further. >> >>> >>> (It is technically possible to have kernel vs. >>> world in separate UFS partitions, possibly on >>> separate media. I've an example context that I >>> deal with that way to avoid a U-boot limitation >>> for the device: the kernel can find the world >>> where the U-Boot/FreeBSD-Loader can not. (The >>> FreeBSD loader depends on what USB can find: it >>> does not look elsewhere.) I only mention this >>> in case I need to reference it in the future, >>> avoiding a surprise in such a case. Otherwise >>> ignore the complication.) >>> >> >> Are you referring to a case where the root is on >> microSD but /usr and friends is on a USB device? > > Again: I suggest ignoring the details of this for > now. It would just confuse things. But if you > insist: > > On the Rock64 I have a e.MMC in use instead of a > microsd card, and in the end the world is on a > USB3 SSD. But the U-Boot involved can not access > the USB3 --os neither can the FreeBSD loader. > The FreeBSD kernel is the first stage that can > access USB3. > > The e.MMC has the MSDOSFS. It has an partial > copy of what would normally be some of the USB3 > drive's content. (I named the mount point > microsd_ufs before I switched to the e.MMC .) > It has only (much hidden in ". . ."s). > > # find /microsd_ufs -print | sort | less > /microsd_ufs > /microsd_ufs/.snap > /microsd_ufs/COPYRIGHT > /microsd_ufs/boot > /microsd_ufs/boot/beastie.4th > /microsd_ufs/boot/boot1.efi > /microsd_ufs/boot/brand-fbsd.4th > /microsd_ufs/boot/brand.4th > /microsd_ufs/boot/check-password.4th > /microsd_ufs/boot/color.4th > /microsd_ufs/boot/copy_boot_to_microsd.sh > /microsd_ufs/boot/defaults > /microsd_ufs/boot/defaults/loader.conf > /microsd_ufs/boot/delay.4th > /microsd_ufs/boot/dtb > . . . > /microsd_ufs/boot/kernel > /microsd_ufs/boot/kernel/accf_data.ko > /microsd_ufs/boot/kernel/accf_dns.ko > . . . > /microsd_ufs/boot/kernel/kernel > . . . > /microsd_ufs/boot/loader.4th > /microsd_ufs/boot/loader.conf > /microsd_ufs/boot/loader.conf.d > /microsd_ufs/boot/loader.efi > /microsd_ufs/boot/loader.help > /microsd_ufs/boot/loader.rc > /microsd_ufs/boot/loader_4th.efi > /microsd_ufs/boot/loader_lua.efi > /microsd_ufs/boot/loader_simp.efi > /microsd_ufs/boot/logo-beastie.4th > /microsd_ufs/boot/logo-beastiebw.4th > /microsd_ufs/boot/logo-fbsdbw.4th > /microsd_ufs/boot/logo-orb.4th > /microsd_ufs/boot/logo-orbbw.4th > /microsd_ufs/boot/lua > /microsd_ufs/boot/lua/cli.lua > /microsd_ufs/boot/lua/color.lua > /microsd_ufs/boot/lua/config.lua > /microsd_ufs/boot/lua/core.lua > /microsd_ufs/boot/lua/drawer.lua > /microsd_ufs/boot/lua/gfx-beastie.lua > /microsd_ufs/boot/lua/gfx-beastiebw.lua > /microsd_ufs/boot/lua/gfx-fbsdbw.lua > /microsd_ufs/boot/lua/gfx-orb.lua > /microsd_ufs/boot/lua/gfx-orbbw.lua > /microsd_ufs/boot/lua/hook.lua > /microsd_ufs/boot/lua/loader.lua > /microsd_ufs/boot/lua/menu.lua > /microsd_ufs/boot/lua/password.lua > /microsd_ufs/boot/lua/screen.lua > /microsd_ufs/boot/menu-commands.4th > /microsd_ufs/boot/menu.4th > /microsd_ufs/boot/menu.rc > /microsd_ufs/boot/menusets.4th > /microsd_ufs/boot/modules > /microsd_ufs/boot/screen.4th > /microsd_ufs/boot/shortcuts.4th > /microsd_ufs/boot/support.4th > /microsd_ufs/boot/uboot > /microsd_ufs/boot/version.4th > /microsd_ufs/boot/zfs > /microsd_ufs/etc > /microsd_ufs/etc/hostid > /microsd_ufs/lost+found > > Almost no world content. /microsd_ufs/boot/loader.conf > has (in part): > > vfs.root.mountfrom="ufs:/dev/gpt/Rock64root" > kern.cam.boot_delay=10000 > vfs.mountroot.timeout=10 > vfs.root_mount_always_wait=1 > > That vfs.root.mountfrom notation redirects to the > USB3 drive for world but the /boot/loader.conf and > /boot/kernel/kernel and /etc/hostid were first > loaded from the e.MMC media, not the USB drive. > > As I have it, the USB3 drive also has a copy of > the same /boot/... and such materials and load > module activity is from the USB3 copy. I have to > keep the two places tracking so nothing ends up > mismatching. > >>> You might move (not copy) the MSDOSFS material >>> between the microsd card and the usb drive during >>> experiments, avoiding having duplications. There >>> is the possibility of instead renaming things so >>> files are not found on a partition, for example. >>> A similar point goes for UFS materials: avoid >>> having multiple viable-for-booting UFS file >>> systems present. >>> >>> As stands, things seem too hard to track for me >>> to be of much help. Please make things obvious for >>> what was in use by making the material uniquely >>> placed (for the form that can be used). >>> >> I believe the experiment above (deleting UFS on microSD, >> hiding the DOS files on USB) has been an equivalent >> configuration. It looks like control passes in some way >> between the DOS slices, though how is unclear. > > I do not agree to that specific interpretation of the > sequence you did (on the evidence currently available). > Until "1 Storage Device(s) found" is automatic this > later-stage material is too late to yet be relevant > or to have an appropriate context. > > I'm staying focused on getting the "1 Storage Device(s) > found" to be automatic. Absent that you are likely stuck > with doing something similar to be Rock64 e.MMC way of > working --where /boot/loader.conf and /boot/kernel/kernel > and /etc/hostid for early activity is from a UFS > partition on the microsd card. > >>> Separate issues/questions: >>> >>> Do you have the file "timeout" in the MSDOSFS, in >>> addition to config.txt and the like? If not, I >>> recommend including it. >>> >> Yes, it's been tried on both microSD and USB devices. >> Seems like the only one that can matter is on microSD. >> >> >>> What other RPi* firmware controls for having >>> various deliberate RPi* firmware delays do you >>> have set up? >>> >> >> The Raspberry Pi config.txt description lists several delay commands >> that can be placed in config.txt. None seem related to USB discovery, >> might any come into play early enough to be useful? They're named >> bootcode_delay >> boot_delay >> boot_delay_ms >> >> I tried them casually and didn't notice much change. Are they worth >> revisiting more carefully? > > See my earlier notes. > >>> It is not so much that these would be sufficient, >>> but they do establish some context before U-Boot >>> is even active. It could be important to >>> understand that context. (Unsure at this point.) >>> >>> >>>>> But that was for the u-boot-rpi4 or u-boot-rpi-arm64 ports. >>>>> (They also later mentioned using "usb_pgood_delay=2000\0" >>>>> instead, a figure they found in a bunch of configrations.) >>>>> >>>> The Pi3 in question is capable of booting from solid-state USB >>>> storage without any microSD card, but fails to detect a mechanical >>>> disk. Which is the appropriate u-boot-rpi3 port to tamper with? I >>>> tried sysutils/u-boot-rpi3 as an upgrade but that simply froze. >>>> The u-boot from FreeBSD-13.0-RELEASE-arm64-aarch64-RPI.img finds >>>> the USB mechanical disk, but erratically. >>> >>> FreeBSD-13.0-RELEASE-arm64-aarch64-RPI.img uses u-boot-rpi-arm64 >>> as I remember: it is set up for both the RPI3 and RPI4. Given that >>> it works some, that would be the U-Boot port I would experiment >>> with if I was doing the experiments. >>> >>>>> So something somewhat analogous might help if you are willing >>>>> to build and use your own u-boot port variant. >>>> >>>> Obviously, that's a fraught enterprise at my skill level.... >>>> I'm still somewhat hazy on the actual boot sequence when >>>> chaining from microSD to USB. >>> >>> Having the extra text in the U-Boot build does not really >>> depend on the chaining order question. >>> >>> But, to repeat (sticking to the simpler context), >>> the order is: >>> >>> MSDOSFS: (all on one of: microsd card vs. usb drive) >>> RPi* firmware (I do not report the file-level sequencing) >>> U-Boot >>> FreeBSD loader (which uses information from U-Boot) >>> >>> UFS: (both: together on one of: microsd card vs. usb drive) >>> FreeBSD kernel >>> FreeBSD world >>> >>> >>>> Indeed, it's unclear how or if u-boot plays a role in starting >>>> RasPiOS. The term u-boot isn't found on the Raspberry Pi doc >>>> website, and the Pi isn't mentioned in the Denx manuals. Those >>>> discoveries surprised me. >>> >>> RaspiOS and RaspiOS64 do not use U-boot. The RPi*'s can boot some >>> forms of a linux kernel directly and that is what RaspiOS and >>> RaspiOS64 are doing. >> >> That clears up a major misunderstanding, thank you! >> >>> That is why the config.txt type of content >>> makes no mention of u-boot or of any kernel= assignment in RaspiOS >>> and RaspiOS64. By contrast, config.txt for FreeBSD lists kernel= >>> and an explicit *u-boot*.bin as the value. >> >> Without thinking about it very carefully I tried to use the >> "bootcode.bin-only" method for booting an older Pi2 from a >> usb hard disk, as described in >> https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#special-bootcode-bin-only-boot-mode >> It worked on the Pi2 but failed on the Pi3, causing an immediate hang. > > This gets back into the use of a config.txt with a > bootcode_delay assignment also being in the MSDOSFS > on the microsd card and the file timeout also being > in the MSDOSFS on the microsd card. Only if those > delays together can lead to the USB drive being > accessible will it get any farther. > > I'd suggest such experiments. The vintage of bootcode.bin > matters as I understand. > >> Might that avenue be worth pursuing? In hindsight, that it worked on a >> Pi2 is quite surprising. >> > === Mark Millard marklmi at yahoo.com