FYI: RPi4 (8 GiByte) USB3 vs. head -r363123: still a no-go for booting a USB3 / in my experiments
Mark Millard
marklmi at yahoo.com
Sun Jul 19 02:18:43 UTC 2020
> On 2020-Jul-18, at 14:37, Robert Crowston <crowston at protonmail.com> wrote:
>
> I believe this differential (https://reviews.freebsd.org/D25261) would resolve it, but I haven't got around to addressing the comments there yet.
>
> -- RHC.
Yep, a -r363123 kernel with that also it booted / from
the USB3 SSD. (The kernel came from the mmcsd0.)
Notes for this u-boot-rpi4 based context:
Unlike for uefi v1.17, genet0 and mmcsd0 show up.
There is a big asymmetry for sending vs. receiving
over genet0 (192.168.1.126 here):
# iperf3 -c 192.168.1.120 --get-server-output -B 192.168.1.126
Connecting to host 192.168.1.120, port 5201
[ 5] local 192.168.1.126 port 56649 connected to 192.168.1.120 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 22.7 MBytes 191 Mbits/sec 85 38.4 KBytes
[ 5] 1.00-2.00 sec 23.4 MBytes 196 Mbits/sec 146 25.7 KBytes
[ 5] 2.00-3.00 sec 23.2 MBytes 195 Mbits/sec 226 27.1 KBytes
[ 5] 3.00-4.00 sec 23.3 MBytes 195 Mbits/sec 229 95.0 KBytes
[ 5] 4.00-5.00 sec 23.6 MBytes 198 Mbits/sec 228 83.7 KBytes
[ 5] 5.00-6.00 sec 23.7 MBytes 199 Mbits/sec 209 7.13 KBytes
[ 5] 6.00-7.00 sec 23.7 MBytes 199 Mbits/sec 207 49.8 KBytes
[ 5] 7.00-8.00 sec 23.6 MBytes 198 Mbits/sec 217 1.43 KBytes
[ 5] 8.00-9.00 sec 23.6 MBytes 198 Mbits/sec 224 71.3 KBytes
[ 5] 9.00-10.00 sec 23.5 MBytes 197 Mbits/sec 242 8.55 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 234 MBytes 197 Mbits/sec 2013 sender
[ 5] 0.00-10.27 sec 234 MBytes 191 Mbits/sec receiver
Server output:
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.1.126, port 57882
[ 5] local 192.168.1.120 port 5201 connected to 192.168.1.126 port 56649
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 16.4 MBytes 138 Mbits/sec
[ 5] 1.00-2.00 sec 23.4 MBytes 196 Mbits/sec
[ 5] 2.00-3.00 sec 23.2 MBytes 195 Mbits/sec
[ 5] 3.00-4.00 sec 23.1 MBytes 194 Mbits/sec
[ 5] 4.00-5.00 sec 23.6 MBytes 198 Mbits/sec
[ 5] 5.00-6.00 sec 23.7 MBytes 199 Mbits/sec
[ 5] 6.00-7.00 sec 23.7 MBytes 199 Mbits/sec
[ 5] 7.00-8.00 sec 23.6 MBytes 198 Mbits/sec
[ 5] 8.00-9.00 sec 23.6 MBytes 198 Mbits/sec
[ 5] 9.00-10.00 sec 23.5 MBytes 197 Mbits/sec
[ 5] 10.00-10.27 sec 6.30 MBytes 197 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.27 sec 234 MBytes 191 Mbits/sec receiver
iperf Done.
Rock64orRPi4# iperf3 -R -c 192.168.1.120 --get-server-output -B 192.168.1.126
Connecting to host 192.168.1.120, port 5201
Reverse mode, remote host 192.168.1.120 is sending
[ 5] local 192.168.1.126 port 25404 connected to 192.168.1.120 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 111 MBytes 933 Mbits/sec
[ 5] 1.00-2.00 sec 111 MBytes 933 Mbits/sec
[ 5] 2.00-3.00 sec 111 MBytes 933 Mbits/sec
[ 5] 3.00-4.00 sec 111 MBytes 933 Mbits/sec
[ 5] 4.00-5.00 sec 111 MBytes 933 Mbits/sec
[ 5] 5.00-6.00 sec 111 MBytes 933 Mbits/sec
[ 5] 6.00-7.00 sec 111 MBytes 933 Mbits/sec
[ 5] 7.00-8.00 sec 111 MBytes 933 Mbits/sec
[ 5] 8.00-9.00 sec 111 MBytes 933 Mbits/sec
[ 5] 9.00-10.00 sec 111 MBytes 933 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.26 sec 1.09 GBytes 910 Mbits/sec 923 sender
[ 5] 0.00-10.00 sec 1.09 GBytes 933 Mbits/sec receiver
Server output:
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.1.126, port 41483
[ 5] local 192.168.1.120 port 5201 connected to 192.168.1.126 port 25404
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 83.7 MBytes 702 Mbits/sec 62 1.61 MBytes
[ 5] 1.00-2.00 sec 111 MBytes 932 Mbits/sec 92 1.61 MBytes
[ 5] 2.00-3.00 sec 111 MBytes 934 Mbits/sec 93 175 KBytes
[ 5] 3.00-4.00 sec 111 MBytes 933 Mbits/sec 95 88.4 KBytes
[ 5] 4.00-5.00 sec 111 MBytes 933 Mbits/sec 94 268 KBytes
[ 5] 5.00-6.00 sec 111 MBytes 932 Mbits/sec 89 291 KBytes
[ 5] 6.00-7.00 sec 111 MBytes 933 Mbits/sec 92 4.28 KBytes
[ 5] 7.00-8.00 sec 111 MBytes 933 Mbits/sec 92 1.27 MBytes
[ 5] 8.00-9.00 sec 111 MBytes 933 Mbits/sec 94 1.61 MBytes
[ 5] 9.00-10.00 sec 111 MBytes 933 Mbits/sec 96 1.42 MBytes
[ 5] 10.00-10.26 sec 29.3 MBytes 931 Mbits/sec 24 1.61 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.26 sec 1.09 GBytes 910 Mbits/sec 923 sender
iperf Done.
My attempt to duplicate an about 11 GiByte .tar file failed:
# cp -aRx /usr/obj/clang-armv7-on-aarch64.tar /usr/obj/mmjnk.tar
g_vfs_done():gpt/Rock64root[READ(offset=274911461376, length=32768)]error = 5
c/usr/obj/clang-armv7-on-UFS /dev/gpt/Rock64root (/) cylinder checksum failed: cg 270, cgp: 0x0 != bp: 0xa023cccf
aarch64.tar: Input/output error
UFS /dev/gpt/Rock64root (/) cylinder checksum failed: cg 270, cgp: 0x0 != bp: 0xa023cccf
pid 1123 (ntpd), jid 0, uid 0: exited on signal 6 (core dumped)
pid 1240 (login), jid 0, uid 0: exited on signal 11 (core dumped)
Same media as used for Rock64 experiments. I've had no
evidence of problems there.
However uefi based booting the RPi4 also has problems
unless uefi is set to force 3 GiBytes of RAM (at most).
The problem has usually been visible as the diff
after the copy showing 4 KiByte or slightly less
having differences according to diff or cmp or the
like. There is evidence of the content and where
changing without updates to the files, suggesting
garbage read data.
Again, same media for 3GiByte RAM or on the Rock64:
no evidence of problems for such operations.
My initial guess is that the handling of the RPi4
DMA limitations is not yet correct overall,
apparently for both uefi and u-boot style booting.
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Wednesday, 15 July 2020 10:09, Mark Millard via freebsd-arm <freebsd-arm at freebsd.org> wrote:
>
>> I did the following experiment mostly just to observe
>> the current status for sysutils/u-boot-rpi4 based
>> booting of the RPi4 (with rather modern RPi4 firmware
>> in use). (I normally use uefi/acpi instead of u-boot,
>> uefi now at v1.17 . I was hoping to see if u-boot based
>> also had a bug that uefi contexts have.)
>>
>> With the kernel on the microsd card (and earlier stage
>> materials), boot -v reported (before mounting / from
>> USB3 became relevant):
>>
>> pci1: <PCI bus> on pcib1
>> pcib1: allocated bus range (1-1) for rid 0 of pci1
>> pci1: domain=0, physical bus=1
>> found-> vendor=0x1106, dev=0x3483, revid=0x01
>>
>> domain=0, bus=1, slot=0, func=0
>> class=0c-03-30, hdrtype=0x00, mfdev=0
>> cmdreg=0x0000, statreg=0x0010, cachelnsz=0 (dwords)
>> lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
>> intpin=a, irq=0
>> powerspec 3 supports D0 D3 current D0
>> MSI supports 4 messages, 64 bit
>> map[10]: type Memory, range 64, base 0, size 12, memory disabled
>>
>>
>> pcib1: slot 0 INTA is routed to irq 82
>> xhci0: <XHCI (generic) USB 3.0 controller> irq 82 at device 0.0 on pci1
>> pcib1: allocated memory range (0xf8000000-0xf8000fff) for rid 10 of xhci0
>> xhci0: Lazy allocation of 0x1000 bytes rid 0x10 type 3 at 0xf8000000
>> xhci0: 32 bytes context size, 64-bit DMA
>> xhci0: attempting to allocate 1 MSI vectors (4 supported)
>> xhci0: using IRQ 83 for MSI
>> xhci0: MSI enabled
>> xhci0: Controller reset timeout.
>> xhci0: XHCI halt/start/probe failed err=18
>> xhci0: Controller reset timeout.
>> device_attach: xhci0 attach returned 6
>> . . .
>> simplebus2: xhci at 7e9c0000 mem 0x7e9c0000-0x7eabffff irq 78 disabled compat generic-xhci (no driver attached)
>>
>> So the USB3 ends up unavailable.
>>
>> / would have been from a USB3 SSD if things had worked.
>>
===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
More information about the freebsd-arm
mailing list