Re: Booting rock64 from USB SSD
- In reply to: Mark Millard : "Re: Booting rock64 from USB SSD"
- Go to: [ bottom of page ] [ top of archives ] [ this month ]
Date: Fri, 01 Apr 2022 22:39:29 UTC
W dniu 1.04.2022 o 22:52, Mark Millard pisze:
> On 2022-Apr-1, at 06:51, Łukasz Moskała <lm@lukaszmoskala.pl> wrote:
>
>> Hi everyone,
>>
>> I want to boot my rock64 from SSD (because all sd cards I could find are crap).
>>
>> I followed those instructions to flash u-boot to SPI: https://github.com/ayufan-rock64/linux-build/blob/master/recipes/flash-spi.md
>>
>> I was then able to boot FreeBSD from SSD (dd FreeBSD-13.0-RELEASE-arm64-aarch64-ROCK64.img to SSD, just like it would be to SD card), but ethernet wasn't working. Actually, it could receive packets but no outgoing packets were sent. Not even ARP packets - switch didn't detect any device on that port, but link was UP
>>
>> I assumed it was something to do with u-boot, so I erased u-boot from flash, then DD FreeBSD-13.0-RELEASE-arm64-aarch64-ROCK64.img to SD card, and it worked without problems.
>>
>> Do I understand correctly that this means that FreeBSD on rock64 uses custom u-boot? Can I somehow flash it to SPI?
>>
>> Possible workaround that I can think of would be to put /boot on sd card, install u-boot to sd card, then put / on SSD.
>
> My memory is that USB2 booting could be more normal, in that the
> U-Boot from ports could access USB2 storage but not USB3 storage,
> at least when I set this up long ago. I wanted USB3 use, thus the
> below details.
>
> I have U-Boot on the /media/ device below. As I remember, it could not
> deal with the USB3 port, which is the one I wanted to use. The
> FreeBSD kernel was the first stage that could deal with the USB3 port
> back when I set up the Rock64 that I have access to.
>
> I've not had to change the basic structure since I set it up. There
> could be aspects that could be different these days and I'd not know.
> Think of any comments as potentially being old information.
>
> Note: mmcsd0 here is actually an eMMC device instead of the microsd
> card slot contents, leaving the microsd card slot free for other uses.
> But I'd used a microsd card this way before using the eMMC device
> (as I remember anyway).
>
> # gpart show -p
> => 63 244277185 mmcsd0 MBR (116G)
> 63 32705 - free - (16M)
> 32768 102312 mmcsd0s1 fat32lba [active] (50M)
> 135080 28760 - free - (14M)
> 163840 241172480 mmcsd0s2 freebsd (115G)
> 241336320 2940928 - free - (1.4G)
>
> => 0 241172480 mmcsd0s2 BSD (115G)
> 0 230686720 mmcsd0s2a freebsd-ufs (110G)
> 230686720 10485760 - free - (5.0G)
>
> => 40 1953525088 da0 GPT (932G)
> 40 532480 da0p1 efi (260M)
> 532520 2008 - free - (1.0M)
> 534528 7340032 da0p2 freebsd-swap (3.5G)
> 7874560 1048576 - free - (512M)
> 8923136 23068672 da0p3 freebsd-swap (11G)
> 31991808 2097152 - free - (1.0G)
> 34088960 33554432 da0p4 freebsd-swap (16G)
> 67643392 1740636160 da0p5 freebsd-ufs (830G)
> 1808279552 4194304 da0p6 freebsd-swap (2.0G)
> 1812473856 141051272 - free - (67G)
>
> For reference, from "gpart show -pl" :
>
> 67643392 1740636160 da0p5 Rock64root (830G)
>
> (The USB drive can boot other systems as well, with
> widely varying amounts of RAM. Thus the efi partition
> and the odd set of freebsd-swap partitions.)
>
> So, /media/ below is mmcsd0s1's fat32lba and /mnt/ is
> mmcsd0s2a's freebsd-ufs. da0 is the USB3 SSD media,
> which I do not give other details of here. (I manually
> mounted these for this note.)
>
> # ls -Tld /media/*/*/* /mnt/* /mnt/etc/*
> -r-xr-xr-x 1 root wheel 1243772 Jan 28 12:33:00 2022 /media/EFI/BOOT/bootaa64.efi
> -r-xr-xr-x 1 root wheel 50618 Jan 28 12:32:28 2022 /media/dtb/rockchip/rk3328-rock64.dtb
> -r--r--r-- 1 root wheel 6170 Feb 1 04:48:34 2020 /mnt/COPYRIGHT
> drwxr-xr-x 23 root wheel 1536 Jan 28 15:26:41 2022 /mnt/boot
> drwxr-xr-x 2 root wheel 512 Apr 26 14:39:22 2020 /mnt/etc
> -rw-r--r-- 1 root wheel 37 Dec 31 16:00:18 2009 /mnt/etc/hostid
> drwx------ 2 root wheel 33280 Nov 27 09:46:08 2019 /mnt/lost+found
>
> # ls -Tld /mnt/boot/dtb/overlays/rk3328-*
> -r--r--r-- 1 root wheel 238 Jan 28 12:32:28 2022 /mnt/boot/dtb/overlays/rk3328-analog-sound.dtbo
> -r--r--r-- 1 root wheel 1281 Jan 28 12:32:28 2022 /mnt/boot/dtb/overlays/rk3328-dwc3.dtbo
>
> (I make no use of rk3328-analog-sound.dtbo .)
>
> /mnt/boot/loader.conf has, in part,
>
> # A msdosfs /MNTPNT/dtb/rockchip/rk3328-rock64.dtb
> # copy of the ufs /boot/dtb/rockchip/rk3328-rock64.dtb
> # uses the intended DTB in u-boot and the kernel --and
> # avoids needing to tell the kernel where to find a
> # copy . . .
> #rk3328_rock64_load="YES"
> #rk3328_rock64_type="dtb"
> #rk3328_rock64_name="/boot/dtb/rockchip/rk3328-rock64.dtb"
> #
> # rk3328 USB3-related:
> fdt_overlays="rk3328-dwc3.dtbo"
> # ucom is not automatically being loaded when umodem is loaded at boot.
> ucom_load="YES"
> umodem_load="YES"
> #
> vfs.root.mountfrom="ufs:/dev/gpt/Rock64root"
> kern.cam.boot_delay=10000
> vfs.mountroot.timeout=10
> vfs.root_mount_always_wait=1
>
> ===
> Mark Millard
> marklmi at yahoo.com
>
Hi Mark,
With some idea from what you did, I managed to do this:
=================== BEGIN COMMAND LIST ============================
sudo pkg install u-boot-rock64
truncate -s 512M rock64-sd.img
truncate -s 5G rock64-disk.img
sudo mdconfig -a -t vnode -f
FreeBSD-13.0-RELEASE-arm64-aarch64-ROCK64.img #returned md0
sudo mdconfig -a -t vnode -f rock64-sd.img #returned md1
sudo mdconfig -a -t vnode -f rock64-disk.img #returned md2
sudo gpart create -s gpt md1
sudo gpart add -t efi -b 32768 -s 50M -i 1 md1
sudo gpart add -t freebsd-ufs -i 2 md1
sudo newfs -O 2 -L bootfs /dev/md1p2
sudo dd if=/dev/md0p1 of=/dev/md1p1 bs=1M
mkdir mp_old mp_new
sudo mount -o ro /dev/md0p2 mp_old
sudo mount -o rw /dev/md1p2 mp_new
sudo rsync -rxav mp_old/boot/ mp_new/boot/
echo 'fdt_overlays="rk3328-dwc3.dtbo"' | sudo tee -a mp_new/boot/loader.conf
echo 'vfs.root.mountfrom="ufs:/dev/ufs/rootfs"' | sudo tee -a
mp_new/boot/loader.conf
echo 'kern.cam.boot_delay=10000' | sudo tee -a mp_new/boot/loader.conf
echo 'vfs.mountroot.timeout=10' | sudo tee -a mp_new/boot/loader.conf
echo 'vfs.root_mount_always_wait=1' | sudo tee -a mp_new/boot/loader.conf
sudo gpart create -s gpt md2
sudo gpart add -t freebsd-ufs -a 4096 -i 1 md2
sudo newfs -O 2 -t -L rootfs /dev/md2p1
mkdir mp_newroot
sudo mount -o rw /dev/md2p1 mp_newroot
sudo rsync -rxav --exclude=boot mp_old/ mp_newroot/
sudo rm mp_newroot/etc/fstab
sudo mkdir mp_newroot/media/sd
echo '/dev/ufs/rootfs / ufs rw 1 1' | sudo tee -a mp_newroot/etc/fstab
echo '/dev/ufs/bootfs /media/sd ufs rw,noatime 1 2' | sudo tee -a
mp_newroot/etc/fstab
echo '/dev/msdosfs/EFI /media/sd/boot/efi msdosfs rw,noatime 0 0' | sudo
tee -a mp_newroot/etc/fstab
echo 'tmpfs /tmp tmpfs rw,mode=1777,size=50m 0 0' | sudo tee -a
mp_newroot/etc/fstab
cd mp_newroot
sudo ln -sf media/sd/boot boot
cd ..
sudo umount mp_newroot
sudo umount mp_new
sudo umount mp_old
sudo dd if=/usr/local/share/u-boot/u-boot-rock64/idbloader.img
of=/dev/md1 seek=64 bs=512 conv=sync
sudo dd if=/usr/local/share/u-boot/u-boot-rock64/u-boot.itb of=/dev/md1
seek=16384 bs=512 conv=sync
sudo mdconfig -d -u 2
sudo mdconfig -d -u 1
sudo mdconfig -d -u 0
=================== END COMMAND LIST ============================
(my apologies if some lines wrap improperly)
then, DD rock64-sd.img to sdcard and rock64-disk.img to disk.
It would be possible to do that directly on disk and sdcard itself, but
in my case, machine on which I ran these commands doesn't have sd card
reader.
Anyway, it successfully booted with SSD plugged into USB3 port, despite
those messages:
Root mount waiting for: usbus0 usbus2 usbus3 usbus4 CAM
usb_alloc_device: set address 2 failed (USB_ERR_IOERROR, ignored)
usbd_setup_device_desc: getting device descriptor at addr 2 failed,
USB_ERR_IOERROR
after which da0 was detected and boot continued. Network is working, and
everything seems okay (uptime is 6 minutes as of moment I'm writing
this). I'm using official pine64 usb-sata adapter, but I suppose it
doesn't really matter. I don't have any other adapters to check. I
copied 2.5GiB file to it and it seems stable.
I got a lot of "devmatch: Can't read linker hints file."
That was fixed by kldxref -R /media/sd/boot
root@generic:~ # df -h
Filesystem Size Used Avail Capacity Mounted on
/dev/ufs/rootfs 108G 4.2G 95G 4% /
devfs 1.0K 1.0K 0B 100% /dev
/dev/ufs/bootfs 432M 82M 316M 21% /media/sd
/dev/msdosfs/EFI 50M 2.7M 47M 5% /media/sd/boot/efi
tmpfs 50M 4.0K 50M 0% /tmp
root@generic:~ # mount
/dev/ufs/rootfs on / (ufs, local)
devfs on /dev (devfs)
/dev/ufs/bootfs on /media/sd (ufs, local, noatime)
/dev/msdosfs/EFI on /media/sd/boot/efi (msdosfs, local, noatime)
tmpfs on /tmp (tmpfs, local)
root@generic:~ # gpart show -p
=> 40 1048496 mmcsd0 GPT (7.4G) [CORRUPT]
40 32728 - free - (16M)
32768 102400 mmcsd0p1 efi (50M)
135168 913368 mmcsd0p2 freebsd-ufs (446M)
=> 40 234441568 da0 GPT (112G)
40 4056 - free - (2.0M)
4096 234437512 da0p1 freebsd-ufs (112G)
Thanks for your help, and I hope that whatever I made here will be
usefull to someone.
--
Łukasz Moskała