From nobody Sat Dec 18 03:12:32 2021 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 7BBD618E439C for ; Sat, 18 Dec 2021 03:12:42 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic314-21.consmr.mail.gq1.yahoo.com (sonic314-21.consmr.mail.gq1.yahoo.com [98.137.69.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4JG9rn3G1Tz3Gx2 for ; Sat, 18 Dec 2021 03:12:41 +0000 (UTC) (envelope-from marklmi@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1639797159; bh=bzBAzesE7Ry4a0Ss/z6ApCFglA4fjlhYrIkVRBgIWV4=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=SxJuD2pZBvS9c4SgjJxucl5qv5rkA4YvHCX6kjzqDQE0qSYUFvxeIQy5kFvaQvegJ7q7WcpoxxHAnJX/Szqo4Dr0dn5qa+2iOvW4g39COkiWJpHo129rfgugU8lcHHkfJDANutKv/RO/DsT0ewLf1+5XK53ca1QqoDPxDxj6zLZUwdhgzBy/m4VE8cc0RPIRn/QO4/PS1TjxKhIwM3h9rJKu+SWiMN3yG16yshuq/pnfQBagjw5rzlzSHApxERq9QrT7Mx0BWRB6ZYG1BU9A1hqrGC6TrZeK2ul2LCGYvJqZMAp7cB7ZtUJvPdREzdmYJ7SaJ3fr8ERs0z9RvPPv9g== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1639797159; bh=H+Uwh+/ePf2zmH0ikN1LxA2XCZWYMU6cQC4ofUH2OCs=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=cvnLoAyieatRdRbywBI6QzFdzfAQr4ZeJOHERaJtI0iW07i6UONCcn7dRz4B3Lp51ZtRo+wTv0SwEZQrAp/drdJLv7m2fQAlRq3HG+KunjdUBDieGbikxWVjKDkSQkvrPcYNVP1Pgq/Sb2DG8vxE9CwoRr6afN6QYWjr3705yT76puhdPgSMwmy29QmZCMhIjN/Q/MA/Xjlt+FEZilWQFaTFiLOAekrXRFrqvHJzzTW7oTRpUfc0G3aEOSr5NbWR1BSVnkWhd4vc8PMlY7JKJl0XsFMpagNOiAGCN+q+oNyOsPDuikoeSfH1Fm1BTTYnfeHi+xQqF5T0EAGSHD/jHg== X-YMail-OSG: u_i41.gVM1mK4yoq9Amo7vhphDJCn5492lbFMCDKWgTKMsL4otet3JoPpKWrFS4 DWZVI.R9HY2v9YrwVrmq1Dv46n9dr8Y_SLLy5QlzuafQx.ZAAreRDaT9F040DPvGwBduIjRmbctr _Fm04nUgLS98POtX3wVXgkbt8_XoAY6X.2YcCprB76YDxrmRebNQ3MsFrQaZexLkv.dH883zj5aJ ntfh0wP0qWiI12M0q2cMTpdeMdrsqavSLQsklSpz_xb8YP9RIEqfN6hBbMhfNwLeTa_JWwBJhH3G Id4pmyNBNCEJK4TTBU1y3qqGy6k619eQeppLS17xlgazw2g2_yo1BGpy8I4mw96Zok4PlDlXGcr4 VMZO8OlZoHEi9mL78asXFxsjnrYV.RmppFogwcYz7tRfjCSBemiwNWjKCPtrNVQxQDBZK8xZKWW6 9JBBVzaj.hmTHgXYzena0naFJjE_pdHty6DNZUjYM.VPWa2oTHH8NIR8tXZCpaD7gh5Q3TmspbfR R.4o4vX4ABQYWySiuwclR9R54yk3nrN3mPw8lmHHxCo2ygo7.7UNoxLjOGdUcoqsiJ6OBO4CndBV KLU8zDwzVg4mwm0BzFofAWSz4psMcS555JeOzqyyMstO.RBg7su54HIdEiIh.n455uzHu8Vrzbff Sc_iCOcat.0hjMWgF5Cee1ryOuimQh.7EUWWq4naTizT8J6rYVK8eqtwjIy2M7xFRAIyu14Oa4yD 8ueSvGv4XnH10qiWZjNN8kPOAfYZ1hhXR66tGKMb31MiIejKZA25M7V.0CMf4suSTv0WYtXLy2j4 kMB_d2DWIORTewhEPNGANId_ObfayVHXRXnZMOzB2hR1C7ZdK94WB8igCz2rpKUeFSK8GKPLihwg Sn5RjHZl_jy28YECLvpWto6uwN0TpKtsg0sipn8VcduUGozTluxUn.VMYBMvtnnSv2i9CbP0rzhH 9jc_7wXo3EvcUvlLmR2cJsfNWMpMCCn6dQMAfu7NK9rrheybGM4ET8M_vJqfmedN8HzgYYpoXBjL gYHzFMcQAVrYtVY6OWmXOM7TNPT9cQsIAjtWtfxdE8qCx.qciFjkwPpLsijCgyrlqX8KThgthZjT AD6hohn_CKJzdCD_OKrWXtHiABNG5RTW1dMVl_oemt4EPQS4KrmUA1IZqrzfOdwS2vpBxioypeYW 3.9wmIJfmpTPShdOx8ZjJw3vnGkdK81OL9rFTYCGiLBT5pHMK61cSLfCPuSLwwlUMAvshYopdSD0 oLnk4ZHrGUaMErz3ik559FZPEMD5XD6HvuhUXh7tog2rYsSWFhyZkD5zSaAhSevvRzYt_eRotGDJ kqq.psv2gMmDidGXk3k8ty3ZRzGNb6s8jn6Tu.J2AStJEzknvvMFUdBtAgiXlO0lBv3U6jqHUdat 7nSuzhT9cOy_qhaqdyCRIHjeelnzSBnXtkK619oQssyoe_5V_spqx6Ona8jVcwLAS0XV9T658Daj CI3IAxuaRY3DY95l9qPzPgixONkVQo5u8TPncw9Z_HG4bIpLhVMLB4FbaQuSk63kaB.SsJA6QPo1 G1ZLJibeNKLarM_56F1GR2F.rrYUsrHNaWdTYBO8UOzHi31Wuv0HbRYIZEvIouKMrkqQwU2wHpYI X3ZPZJTaK8VD40PHelLQnMcIb90yL0G6K3cexO8tJTmnM.3R_QxQ_EMGtuBQmMWJ38S8ckuEKCBf nztCxB7jJk1VX.ib7MXHz5XBHay2mOk.SFoaJN3s95Yx5Z9SzJk8MrLLj9ubXWlBsxVxtIunB7Rc R3QGiMMWwUBN1aFlwYJANfvlqm4R8vpfbysAi8_uvaRbGZpD90DoVBXaxZBWCa8EYyi5H1wfxAqR sIKl1HJYBCEUyfm9uDbysxe1IhdgKiweD5MviaW0.VsF_6YwsXrfs6Ny2ys6vMgMqpdY6PTXn6zL f_B92SOCoCziP10Di5KN.QPvCLy66RY9e3JZDGJO5AtUGo0a0pIBABt3ZetoVslNFzg2GMzOG1Pd gAG1EhMbQTzPXKewhKilMTcWo1pCuwR1L9LMRM6pOjMZ3JNeVotv3Es8hd1BRaXfM4z_Hntd_fZ4 Zm9mjQXbteyzsrniPlerrvUrBrAahbv6pTX9wWCf0DtMNuu5M5_O7J4ShQIttHAVbyIEm.hPn7lm 1lPhSWEiAb6l2x_DbTZykmTsPjDnmYPKDFuNNHZ7ib_bo4fkQ69iGn8uZyL_kchOzlr3RAJehOjK vDIZv2deajGKMBXV_YKNhXSW.F.3i.7ibtvGLZTH2TuXXMywDTawbmTTB.sxYSipNuwShhHuGLSp yMln6wGBTBZ0fPOA- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic314.consmr.mail.gq1.yahoo.com with HTTP; Sat, 18 Dec 2021 03:12:39 +0000 Received: by kubenode550.mail-prod1.omega.bf1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID fd659f7710d8a12e8e7f9c4dbd108a9d; Sat, 18 Dec 2021 03:12:35 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Subject: Re: Saving environment variables in u-boot In-Reply-To: <5C44D0E6-2FF1-4EEB-B21A-83333D6FCF46@yahoo.com> Date: Fri, 17 Dec 2021 19:12:32 -0800 Cc: Free BSD Content-Transfer-Encoding: quoted-printable Message-Id: <9D416106-660F-40BB-98D2-1354B53D2FEF@yahoo.com> References: <20211216180704.GA4173@www.zefox.net> <214132DD-A095-4349-BB81-B79CB8CF6B0C@yahoo.com> <20211217013613.GA4452@www.zefox.net> <20211218005946.GA7670@www.zefox.net> <5C44D0E6-2FF1-4EEB-B21A-83333D6FCF46@yahoo.com> To: bob prohaska X-Mailer: Apple Mail (2.3654.120.0.1.13) X-Rspamd-Queue-Id: 4JG9rn3G1Tz3Gx2 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=SxJuD2pZ; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.84 as permitted sender) smtp.mailfrom=marklmi@yahoo.com X-Spamd-Result: default: False [-3.18 / 15.00]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; NEURAL_HAM_LONG(-1.00)[-1.000]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.84:from]; NEURAL_HAM_SHORT(-0.68)[-0.680]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_COUNT_TWO(0.00)[2]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.84:from] Reply-To: marklmi@yahoo.com From: Mark Millard via freebsd-arm X-Original-From: Mark Millard X-ThisMailContainsUnwantedMimeParts: N On 2021-Dec-17, at 19:02, Mark Millard via freebsd-arm = wrote: > On 2021-Dec-17, at 16:59, bob prohaska wrote: >=20 >> On Thu, Dec 16, 2021 at 11:00:46PM -0800, Mark Millard wrote: >>> On 2021-Dec-16, at 17:36, bob prohaska wrote: >>>=20 >>>>=20 >>>> Yes, the microSD contains a dd of=20 >>>> FreeBSD-13.0-RELEASE-arm64-aarch64-RPI.img >>>> The DOS partition is writeable, AFAIK. >>>=20 >>> Hmm. That means there is also a UFS root with >>> a kernel and world on the microsd card. Both >>> the microsd card and the USB drive contain >>> contain such, as well as each having an >>> /etc/fstab (and so on). >>>=20 >>> Having multiple such makes for a mess for >>> controlling which is used (and knowing which >>> is used). What is the first stage that you >>> made sure the USB drive was in use and how >>> did you make sure? >>>=20 >>=20 >> I installed a microSD containing only the FAT slice from=20 >> FreeBSD-13.0-RELEASE-arm64-aarch64-RPI.img, the FreeBSD >> slice being deleted using gpart. There is a timeout file. >=20 > Ultimately, you may need to publish a full capture of the > debug output that goes to the serial console when such > debug output has been fully enabled. This could end up > being for each of several variations tried. >=20 > Did you still have the MSDOSFS material on the USB in a > viable form as well? >=20 > https://www.raspberrypi.com/documentation/computers/config_txt.html > reports that there are boot options that can go in config.txt : >=20 > bootcode_delay > boot_delay > boot_delay_ms >=20 > Quoting . . . >=20 > QUOTE > bootcode_delay >=20 > The bootcode_delay command delays for a given number of seconds in = bootcode.bin before loading start.elf: the default value is 0. > END QUOTE >=20 > QUOTE > boot_delay >=20 > The boot_delay command instructs to wait for a given number of seconds = in start.elf before loading the kernel: the default value is 1. The = total delay in milliseconds is calculated as (1000 x boot_delay) + = boot_delay_ms. This can be useful if your SD card needs a while to get = ready before Linux is able to boot from it. >=20 > boot_delay_ms >=20 > The boot_delay_ms command means wait for a given number of = milliseconds in start.elf, together with boot_delay, before loading the = kernel. The default value is 0. > END QUOTE >=20 > (So boot_delay_ms is only for smaller scale adjustments and can > be ignored here.) >=20 > Here "loading the kernel" means what config.txt was told was > the kernel: some *u-boot*.bin for the FreeBSD context. >=20 >=20 > So one experiment is to have only bootcode.bin and a config.txt > on the microsd card's MSDOSFS, specifying a significantly > large value for bootcode_delay . The hope would be to be able > to get the start*.elf file and the rest from the USB drive. >=20 > If that does not work, then the next is to have the RPi* firmware > only on the microsd card's MSDOSFS and have config.txt also > specify a significantly large value for boot_delay . This > first variation would not have a *u-boot*.bin or the FreeBSD > loader on the microsd card's MSDOSFS but only on the USB > drive's MSDOSFS. >=20 > If that does not work, then the next variation moves just the > *u-boot*.bin file to the microsd card. >=20 > (If that fails then controlling U-Boot's delays can be involved. > For now I only list the path of leaving U-Boot's delays as they > are: one short branch of the exploration.) >=20 > If that does not work, then the next variation moves just the > FreeBSD loader file to the microsd card. For the above one I missed a word, making things unclear. So, instead: If that does not work, then the next variation ALSO moves just the FreeBSD loader file to the microsd card. ( *u-boot*.bin stays on the microsd card from the prior step. ) > With that I'll stop listing things to test, pending results on > what I've listed. >=20 >> A hands-off warm boot reports >> resetting system ...=20 >>=20 >> U-Boot 2020.10 (Apr 09 2021 - 03:55:54 +0000) >>=20 >> DRAM: 948 MiB >> RPI 3 Model B (0xa02082) >> MMC: mmc@7e300000: 0 >> Loading Environment from FAT... In: serial >> Out: vidconsole >> Err: vidconsole >> Net: No ethernet found. >> starting USB... >> Bus usb@7e980000: USB DWC2 >> scanning bus usb@7e980000 for devices... 5 USB Device(s) found >> scanning usb for storage devices... 0 Storage Device(s) found >=20 > I would need to also see the RPi* firmware's prior > debug output to be able to interpret this. I can not > tell for sure which device's U-Boot copy is operating > for the above. I'd guess that is the the micrsd card's > MSDOSFS one. >=20 >> Hit any key to stop autoboot: 0=20 >> switch to partitions #0, OK >> mmc0 is current device >> Scanning mmc 0:1... >=20 > This looks to be getting a copy of the FreeBSD loader > from the microsd card's MSDOSFS. >=20 > The FreeBSD loader will only see the same Storage Devices > that U-Boot did: it gets the information from U-Boot. >=20 >> Found EFI removable media binary efi/boot/bootaa64.efi >=20 > There was a FreeBSD loader in the microsd card's > MSDOSFS. >=20 >> libfdt fdt_check_header(): FDT_ERR_BADMAGIC >> Scanning disk mmc@7e300000.blk... >> Found 2 disks >> No EFI system partition >> BootOrder not defined >> EFI boot manager: Cannot load any image >> 1259212 bytes read in 125 ms (9.6 MiB/s) >> libfdt fdt_check_header(): FDT_ERR_BADMAGIC >> Booting /efi\boot\bootaa64.efi >=20 > That FreeBSD loader is what it is using, not one > from the USB drive. >=20 >> [whitespace deleted] >> Consoles: EFI console =20 >> Reading loader env vars from /efi/freebsd/loader.env >> Setting currdev to disk0p1: >=20 > "disk0p1" is U-Boot terminology for a drive, despite > this being the FreeBSD loader's output. The FreeBSD > loader uses the U-Boot information, not FreeBSD's > information. >=20 >> FreeBSD/arm64 EFI loader, Revision 1.1 >>=20 >> Command line arguments: loader.efi >> Image base: 0x39e03000 >> EFI version: 2.80 >> EFI Firmware: Das U-Boot (rev 8224.4096) >> Console: comconsole (0) >> Load Path: /efi\boot\bootaa64.efi >> Load Device: = /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(2)/SD(0)/HD(1,0x01,0,0x81f= ,0x18fa8) >> Trying ESP: = /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(2)/SD(0)/HD(1,0x01,0,0x81f= ,0x18fa8) >=20 > This is reporting where the FreeBSD loader was found. > It indicates that disk0p1 was the microsd card's > MSDOSFS. >=20 >> Setting currdev to disk0p1: >=20 > The microsd card's MSDOSFS again. >=20 >> ERROR: cannot open /boot/lua/loader.lua: no such file or directory. >=20 > There is no FreeBSD directory-tree there to find things > like /boot/lua/loader.lua in. >=20 > Currently we are trying to have the /boot/lua/loader.lua come > from the USB drive's UFS file system, not the microsd card. >=20 >> Type '?' for a list of commands, 'help' for more detailed help. >> OK=20 >>=20 >> AIUI, disk0p1 is the microSD, which has no /boot/lua/loader.lua >> That makes sense. If u-boot finds the USB mass storage device >> then running=20 >> run bootcmd_usb0 starts the system successfully. >>=20 >> It does appear that the FAT partition on the USB disk is involved. >=20 > I see no evidence of that above. (Below is a different context.) >=20 >> If I hide the contents of da0s1 by placing them in a subdirectory >> the boot fails, even if u-boot has found the USB disk: >>=20 >> Hit any key to stop autoboot: 0=20 >> U-Boot> usb reset >> resetting USB... >> Bus usb@7e980000: USB DWC2 >> scanning bus usb@7e980000 for devices... 6 USB Device(s) found >> scanning usb for storage devices... 0 Storage Device(s) found >>=20 >> [Note that the six devices include the disk, it can be seen in usb = tree. >> For some reason it isn't recognized as a mass storage device.] >>=20 >> U-Boot> editenv usb_pgood_delay >> edit: 2 >> U-Boot> usb reset >> resetting USB... >> Bus usb@7e980000: USB DWC2 >> scanning bus usb@7e980000 for devices... 6 USB Device(s) found >> scanning usb for storage devices... 1 Storage Device(s) found >> U-Boot> run bootcmd_usb0 >>=20 >> Device 0: Vendor: SABRENT Rev: 1214 Prod:=20 >> Type: Hard Disk >> Capacity: 953869.7 MB =3D 931.5 GB (1953525168 x 512) >> ... is now current device >> Scanning usb 0:1... >> U-Boot>=20 >> and there it stops. Looks like both FAT partitions have a role to = play. >=20 > That is not clear to me. But I'd rather be lookning into > earlier stages at this point, so that "1 Storage Device(s) found" > hopefully ends up being automatic. >=20 > If we get that to be automatic, then what is happening here > becomes the next area to investigate. But the investigation > is not far enough along to justify investigating this now. >=20 >> An earlier experiment tried booting the microSD card in a USB = adapter, >> That worked correctly with nothing in the microSD slot, so the = internal >> "boot from USB" feature might work if it could be slowed down = sufficiently.=20 >=20 > Agreed, that the first thing is to get it to start up > the disk and wait for it. There may be more at issue > after that. >=20 >>> The order of activity is: >>>=20 >>> MSDOSFS: (all on one of: microsd card vs. usb drive) >>> RPi* firmware (I do not report the file-level sequencing) >>> U-Boot >>> FreeBSD loader (which uses information from U-Boot) >>>=20 >>> UFS: (both: together on one of: microsd card vs. usb drive) >>> FreeBSD kernel >>> FreeBSD world >>>=20 >>> (I do not specify which MSDOSFS is used when >>> there are multiple viable ones in separate places. >>> Similarly for UFS.) >>>=20 >> =46rom the experiment today it seems both are required. >=20 > I do not get such an interpretation from your experiment. > Until "1 Storage Device(s) found" happens automatically > the context is not nicely comparable for the later stages. >=20 >> The first discovers the USB disk, the second uses that >> information to proceed further.=20 >>=20 >>>=20 >>> (It is technically possible to have kernel vs. >>> world in separate UFS partitions, possibly on >>> separate media. I've an example context that I >>> deal with that way to avoid a U-boot limitation >>> for the device: the kernel can find the world >>> where the U-Boot/FreeBSD-Loader can not. (The >>> FreeBSD loader depends on what USB can find: it >>> does not look elsewhere.) I only mention this >>> in case I need to reference it in the future, >>> avoiding a surprise in such a case. Otherwise >>> ignore the complication.) >>>=20 >>=20 >> Are you referring to a case where the root is on >> microSD but /usr and friends is on a USB device?=20 >=20 > Again: I suggest ignoring the details of this for > now. It would just confuse things. But if you > insist: >=20 > On the Rock64 I have a e.MMC in use instead of a > microsd card, and in the end the world is on a > USB3 SSD. But the U-Boot involved can not access > the USB3 --os neither can the FreeBSD loader. > The FreeBSD kernel is the first stage that can > access USB3. >=20 > The e.MMC has the MSDOSFS. It has an partial > copy of what would normally be some of the USB3 > drive's content. (I named the mount point > microsd_ufs before I switched to the e.MMC .) > It has only (much hidden in ". . ."s). >=20 > # find /microsd_ufs -print | sort | less > /microsd_ufs > /microsd_ufs/.snap > /microsd_ufs/COPYRIGHT > /microsd_ufs/boot > /microsd_ufs/boot/beastie.4th > /microsd_ufs/boot/boot1.efi > /microsd_ufs/boot/brand-fbsd.4th > /microsd_ufs/boot/brand.4th > /microsd_ufs/boot/check-password.4th > /microsd_ufs/boot/color.4th > /microsd_ufs/boot/copy_boot_to_microsd.sh > /microsd_ufs/boot/defaults > /microsd_ufs/boot/defaults/loader.conf > /microsd_ufs/boot/delay.4th > /microsd_ufs/boot/dtb > . . . > /microsd_ufs/boot/kernel > /microsd_ufs/boot/kernel/accf_data.ko > /microsd_ufs/boot/kernel/accf_dns.ko > . . . > /microsd_ufs/boot/kernel/kernel > . . . > /microsd_ufs/boot/loader.4th > /microsd_ufs/boot/loader.conf > /microsd_ufs/boot/loader.conf.d > /microsd_ufs/boot/loader.efi > /microsd_ufs/boot/loader.help > /microsd_ufs/boot/loader.rc > /microsd_ufs/boot/loader_4th.efi > /microsd_ufs/boot/loader_lua.efi > /microsd_ufs/boot/loader_simp.efi > /microsd_ufs/boot/logo-beastie.4th > /microsd_ufs/boot/logo-beastiebw.4th > /microsd_ufs/boot/logo-fbsdbw.4th > /microsd_ufs/boot/logo-orb.4th > /microsd_ufs/boot/logo-orbbw.4th > /microsd_ufs/boot/lua > /microsd_ufs/boot/lua/cli.lua > /microsd_ufs/boot/lua/color.lua > /microsd_ufs/boot/lua/config.lua > /microsd_ufs/boot/lua/core.lua > /microsd_ufs/boot/lua/drawer.lua > /microsd_ufs/boot/lua/gfx-beastie.lua > /microsd_ufs/boot/lua/gfx-beastiebw.lua > /microsd_ufs/boot/lua/gfx-fbsdbw.lua > /microsd_ufs/boot/lua/gfx-orb.lua > /microsd_ufs/boot/lua/gfx-orbbw.lua > /microsd_ufs/boot/lua/hook.lua > /microsd_ufs/boot/lua/loader.lua > /microsd_ufs/boot/lua/menu.lua > /microsd_ufs/boot/lua/password.lua > /microsd_ufs/boot/lua/screen.lua > /microsd_ufs/boot/menu-commands.4th > /microsd_ufs/boot/menu.4th > /microsd_ufs/boot/menu.rc > /microsd_ufs/boot/menusets.4th > /microsd_ufs/boot/modules > /microsd_ufs/boot/screen.4th > /microsd_ufs/boot/shortcuts.4th > /microsd_ufs/boot/support.4th > /microsd_ufs/boot/uboot > /microsd_ufs/boot/version.4th > /microsd_ufs/boot/zfs > /microsd_ufs/etc > /microsd_ufs/etc/hostid > /microsd_ufs/lost+found >=20 > Almost no world content. /microsd_ufs/boot/loader.conf > has (in part): >=20 > vfs.root.mountfrom=3D"ufs:/dev/gpt/Rock64root" > kern.cam.boot_delay=3D10000 > vfs.mountroot.timeout=3D10 > vfs.root_mount_always_wait=3D1 >=20 > That vfs.root.mountfrom notation redirects to the > USB3 drive for world but the /boot/loader.conf and > /boot/kernel/kernel and /etc/hostid were first > loaded from the e.MMC media, not the USB drive. >=20 > As I have it, the USB3 drive also has a copy of > the same /boot/... and such materials and load > module activity is from the USB3 copy. I have to > keep the two places tracking so nothing ends up > mismatching. >=20 >>> You might move (not copy) the MSDOSFS material >>> between the microsd card and the usb drive during >>> experiments, avoiding having duplications. There >>> is the possibility of instead renaming things so >>> files are not found on a partition, for example. >>> A similar point goes for UFS materials: avoid >>> having multiple viable-for-booting UFS file >>> systems present. >>>=20 >>> As stands, things seem too hard to track for me >>> to be of much help. Please make things obvious for >>> what was in use by making the material uniquely >>> placed (for the form that can be used). >>>=20 >> I believe the experiment above (deleting UFS on microSD,=20 >> hiding the DOS files on USB) has been an equivalent=20 >> configuration. It looks like control passes in some way=20 >> between the DOS slices, though how is unclear. >=20 > I do not agree to that specific interpretation of the > sequence you did (on the evidence currently available). > Until "1 Storage Device(s) found" is automatic this > later-stage material is too late to yet be relevant > or to have an appropriate context. >=20 > I'm staying focused on getting the "1 Storage Device(s) > found" to be automatic. Absent that you are likely stuck > with doing something similar to be Rock64 e.MMC way of > working --where /boot/loader.conf and /boot/kernel/kernel > and /etc/hostid for early activity is from a UFS > partition on the microsd card. >=20 >>> Separate issues/questions: >>>=20 >>> Do you have the file "timeout" in the MSDOSFS, in >>> addition to config.txt and the like? If not, I >>> recommend including it. >>>=20 >> Yes, it's been tried on both microSD and USB devices.=20 >> Seems like the only one that can matter is on microSD. >>=20 >>=20 >>> What other RPi* firmware controls for having >>> various deliberate RPi* firmware delays do you >>> have set up? >>>=20 >>=20 >> The Raspberry Pi config.txt description lists several delay commands >> that can be placed in config.txt. None seem related to USB discovery, >> might any come into play early enough to be useful? They're named >> bootcode_delay >> boot_delay >> boot_delay_ms >>=20 >> I tried them casually and didn't notice much change. Are they worth >> revisiting more carefully?=20 >=20 > See my earlier notes. >=20 >>> It is not so much that these would be sufficient, >>> but they do establish some context before U-Boot >>> is even active. It could be important to >>> understand that context. (Unsure at this point.) >>>=20 >>>=20 >>>>> But that was for the u-boot-rpi4 or u-boot-rpi-arm64 ports. >>>>> (They also later mentioned using "usb_pgood_delay=3D2000\0" >>>>> instead, a figure they found in a bunch of configrations.) >>>>>=20 >>>> The Pi3 in question is capable of booting from solid-state USB >>>> storage without any microSD card, but fails to detect a mechanical >>>> disk. Which is the appropriate u-boot-rpi3 port to tamper with? I=20= >>>> tried sysutils/u-boot-rpi3 as an upgrade but that simply froze.=20 >>>> The u-boot from FreeBSD-13.0-RELEASE-arm64-aarch64-RPI.img finds >>>> the USB mechanical disk, but erratically.=20 >>>=20 >>> FreeBSD-13.0-RELEASE-arm64-aarch64-RPI.img uses u-boot-rpi-arm64 >>> as I remember: it is set up for both the RPI3 and RPI4. Given that >>> it works some, that would be the U-Boot port I would experiment >>> with if I was doing the experiments. >>>=20 >>>>> So something somewhat analogous might help if you are willing >>>>> to build and use your own u-boot port variant. >>>>=20 >>>> Obviously, that's a fraught enterprise at my skill level.... >>>> I'm still somewhat hazy on the actual boot sequence when >>>> chaining from microSD to USB.=20 >>>=20 >>> Having the extra text in the U-Boot build does not really >>> depend on the chaining order question. >>>=20 >>> But, to repeat (sticking to the simpler context), >>> the order is: >>>=20 >>> MSDOSFS: (all on one of: microsd card vs. usb drive) >>> RPi* firmware (I do not report the file-level sequencing) >>> U-Boot >>> FreeBSD loader (which uses information from U-Boot) >>>=20 >>> UFS: (both: together on one of: microsd card vs. usb drive) >>> FreeBSD kernel >>> FreeBSD world >>>=20 >>>=20 >>>> Indeed, it's unclear how or if u-boot plays a role in starting=20 >>>> RasPiOS. The term u-boot isn't found on the Raspberry Pi doc=20 >>>> website, and the Pi isn't mentioned in the Denx manuals. Those >>>> discoveries surprised me. >>>=20 >>> RaspiOS and RaspiOS64 do not use U-boot. The RPi*'s can boot some >>> forms of a linux kernel directly and that is what RaspiOS and >>> RaspiOS64 are doing.=20 >>=20 >> That clears up a major misunderstanding, thank you! >>=20 >>> That is why the config.txt type of content >>> makes no mention of u-boot or of any kernel=3D assignment in RaspiOS >>> and RaspiOS64. By contrast, config.txt for FreeBSD lists kernel=3D >>> and an explicit *u-boot*.bin as the value. >>=20 >> Without thinking about it very carefully I tried to use the >> "bootcode.bin-only" method for booting an older Pi2 from a >> usb hard disk, as described in >> = https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#spec= ial-bootcode-bin-only-boot-mode >> It worked on the Pi2 but failed on the Pi3, causing an immediate = hang. >=20 > This gets back into the use of a config.txt with a > bootcode_delay assignment also being in the MSDOSFS > on the microsd card and the file timeout also being > in the MSDOSFS on the microsd card. Only if those > delays together can lead to the USB drive being > accessible will it get any farther. >=20 > I'd suggest such experiments. The vintage of bootcode.bin > matters as I understand. >=20 >> Might that avenue be worth pursuing? In hindsight, that it worked on = a >> Pi2 is quite surprising. =20 >>=20 >=20 =3D=3D=3D Mark Millard marklmi at yahoo.com