UEFI/ACPI 1.17 based RPi4B boot: -mcpu=cortex-a72 built kernel fails root fs USB3 SSD mount; -mcpu=cortex-a53 built kernel works

Sun Aug 9 06:52:09 UTC 2020

I tried to switch from having one boot media used for
both the RPi4B and Rock64 to having separate media
with the RPi4B tuned to the cortex-a72. Thus I
discovered the following. Hopefully the variability
vs. cortex-a53 tuned kernels points to something
that can be fixed in the kernel.

When I install and try to boot a kernel built
using -mcpu=cortexA72 I get the following
problem while it it trying to mount the root
files system from the USB drive:

. . .
Release APs...Trying to mount root from ufs:/dev/gpt/RPi4Broot []...
done
. . .
CPU  1: ARM Cortex-A72 r0p3 affinity:  1
CPU  2: ARM Cortex-A72 r0p3 affinity:  2
CPU  3: ARM Cortex-A72 r0p3 affinity:  3
ugen0.1: <Generic XHCI root HUB> at usbus0
ugen1.1: <DWCOTG OTG Root HUB> at usbus1
uhub0 on usbus0
uhub1 on usbus1
uhub1: <DWCOTG OTG Root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
uhub0: <Generic XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
uhub1: 1 port with 1 removable, self powered
uhub0: 5 ports with 4 removable, self powered
Root mount waiting for: usbus0 CAM
xhci0: Resetting controller
usb_alloc_device: device init 2 failed (USB_ERR_TIMEOUT, ignored)
ugen0.2: <Unknown > at usbus0 (disconnected)
uhub_reattach_port: could not allocate new device
Root mount waiting for: usbus0 CAM
usb_alloc_device: device init 2 failed (USB_ERR_TIMEOUT, ignored)
ugen0.2: <Unknown > at usbus0 (disconnected)
uhub_reattach_port: could not allocate new device
uhub0: at usbus0, port 1, addr 1 (disconnected)
uhub0: detached
uhub0 on usbus0
uhub0: <Generic XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
uhub0: 5 ports with 4 removable, self powered
Root mount waiting for: CAM
xhci0: Resetting controller
usb_alloc_device: device init 2 failed (USB_ERR_TIMEOUT, ignored)
ugen0.2: <Unknown > at usbus0 (disconnected)
uhub_reattach_port: could not allocate new device
Root mount waiting for: CAM
usb_alloc_device: device init 2 failed (USB_ERR_TIMEOUT, ignored)
ugen0.2: <Unknown > at usbus0 (disconnected)
uhub_reattach_port: could not allocate new device
uhub0: at usbus0, port 1, addr 1 (disconnected)
uhub0: detached
uhub0 on usbus0
uhub0: <Generic XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
uhub0: 5 ports with 4 removable, self powered
Root mount waiting for: CAM
xhci0: Resetting controller
usb_alloc_device: device init 2 failed (USB_ERR_TIMEOUT, ignored)
ugen0.2: <Unknown > at usbus0 (disconnected)
uhub_reattach_port: could not allocate new device
Root mount waiting for: CAM
usb_alloc_device: device init 2 failed (USB_ERR_TIMEOUT, ignored)
Root mount waiting for:ugen0.2: <Unknown > at usbus0 (disconnected)
 CAM
uhub_reattach_port: could not allocate new device
uhub0: at usbus0, port 1, addr 1 (disconnected)
uhub0: detached
uhub0 on usbus0
uhub0: <Generic XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
uhub0: 5 ports with 4 removable, self powered
Root mount waiting for: CAM
xhci0: Resetting controller
usb_alloc_device: device init 2 failed (USB_ERR_TIMEOUT, ignored)
ugen0.2: <Unknown > at usbus0 (disconnected)
uhub_reattach_port: could not allocate new device
Root mount waiting for: CAM
Mounting from ufs:/dev/gpt/RPi4Broot failed with error 2; retrying for 10 more seconds
usb_alloc_device: device init 2 failed (USB_ERR_TIMEOUT, ignored)
ugen0.2: <Unknown > at usbus0 (disconnected)
uhub_reattach_port: could not allocate new device
uhub0: at usbus0, port 1, addr 1 (disconnected)
uhub0: detached
uhub0 on usbus0
uhub0: <Generic XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
uhub0: 5 ports with 4 removable, self powered
xhci0: Resetting controller
usb_alloc_device: device init 2 failed (USB_ERR_TIMEOUT, ignored)
ugen0.2: <Unknown > at usbus0 (disconnected)
uhub_reattach_port: could not allocate new device
usb_alloc_device: device init 2 failed (USB_ERR_TIMEOUT, ignored)
ugen0.2: <Unknown > at usbus0 (disconnected)
uhub_reattach_port: could not allocate new device
uhub0: at usbus0, port 1, addr 1 (disconnected)
uhub0: detached
uhub0 on usbus0
uhub0: <Generic XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
uhub0: 5 ports with 4 removable, self powered
xhci0: Resetting controller
usb_alloc_device: device init 2 failed (USB_ERR_TIMEOUT, ignored)
ugen0.2: <Unknown > at usbus0 (disconnected)
uhub_reattach_port: could not allocate new device
usb_alloc_device: device init 2 failed (USB_ERR_TIMEOUT, ignored)
ugen0.2: <Unknown > at usbus0 (disconnected)
uhub_reattach_port: could not allocate new device
uhub0: at usbus0, port 1, addr 1 (disconnected)
uhub0: detached
uhub0 on usbus0
uhub0: <Generic XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
uhub0: 5 ports with 4 removable, self powered
xhci0: Resetting controller
usb_alloc_device: device init 2 failed (USB_ERR_TIMEOUT, ignored)
ugen0.2: <Unknown > at usbus0 (disconnected)
uhub_reattach_port: could not allocate new device
usb_alloc_device: device init 2 failed (USB_ERR_TIMEOUT, ignored)
ugen0.2: <Unknown > at usbus0 (disconnected)
uhub_reattach_port: could not allocate new device
uhub0: at usbus0, port 1, addr 1 (disconnected)
uhub0: detached
uhub0 on usbus0
uhub0: <Generic XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
uhub0: 5 ports with 4 removable, self powered
xhci0: Resetting controller
usb_alloc_device: device init 2 failed (USB_ERR_TIMEOUT, ignored)
ugen0.2: <Unknown > at usbus0 (disconnected)
uhub_reattach_port: could not allocate new device
Mounting from ufs:/dev/gpt/RPi4Broot failed with error 2.

Loader variables:
  vfs.root.mountfrom=ufs:/dev/gpt/RPi4Broot

Manual root filesystem specification:
  <fstype>:<device> [options]
      Mount <device> using filesystem <fstype>
      and with the specified (optional) option list.

    eg. ufs:/dev/da0s1a
        zfs:zroot/ROOT/default
        cd9660:/dev/cd0 ro
          (which is equivalent to: mount -t cd9660 -o ro /dev/cd0 /)

  ?               List valid disk boot devices
  .               Yield 1 second (for background tasks)
  <empty line>    Abort manual input

mountroot> usb_alloc_device: device init 2 failed (USB_ERR_TIMEOUT, ignored)
ugen0.2: <Unknown > at usbus0 (disconnected)
uhub_reattach_port: could not allocate new device
uhub0: at usbus0, port 1, addr 1 (disconnected)
uhub0: detached
uhub0 on usbus0
uhub0: <Generic XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
uhub0: 5 ports with 4 removable, self powered
. . .

(It keeps going.)

By contrast, installing a kernel built using -mcpu=cortex-a53
leads to boots that work fine. (This is how the common media
was set up for booting either the Rock64 or RPi4B: tuned for
the Rock64. So all my prior RPi4B activity was this way.)

Note: The following is the same media as above, just with
the kernel replaced.

. . .
Trying to mount root from ufs:/dev/gpt/RPi4Broot []...
. . .
CPU  1: ARM Cortex-A72 r0p3 affinity:  1
CPU  2: ARM Cortex-A72 r0p3 affinity:  2
CPU  3: ARM Cortex-A72 r0p3 affinity:  3
ugen0.1: <Generic XHCI root HUB> at usbus0
ugen1.1: <DWCOTG OTG Root HUB> at usbus1
uhub0 on usbus0
uhub1 on usbus1
uhub0: <Generic XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
uhub1: <DWCOTG OTG Root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
uhub1: 1 port with 1 removable, self powered
uhub0: 5 ports with 4 removable, self powered
Root mount waiting for: usbus0 CAM
xhci0: Resetting controller
usb_alloc_device: device init 2 failed (USB_ERR_TIMEOUT, ignored)
ugen0.2: <Unknown > at usbus0 (disconnected)
uhub_reattach_port: could not allocate new device
Root mount waiting for: usbus0 CAM
usb_alloc_device: device init 2 failed (USB_ERR_TIMEOUT, ignored)
ugen0.2: <Unknown > at usbus0 (disconnected)
uhub_reattach_port: could not allocate new device
uhub0: at usbus0, port 1, addr 1 (disconnected)
uhub0: detached
uhub0 on usbus0
uhub0: <Generic XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
uhub0: 5 ports with 4 removable, self powered
Root mount waiting for: CAM
ugen0.2: <vendor 0x2109 USB2.0 Hub> at usbus0
uhub2 on uhub0
uhub2: <vendor 0x2109 USB2.0 Hub, class 9/0, rev 2.10/4.21, addr 1> on usbus0
uhub2: 4 ports with 4 removable, self powered
Root mount waiting for: CAM
Root mount waiting for: CAM
ugen0.3: <OWC Envoy Pro mini> at usbus0
umass0 on uhub0
umass0: <OWC Envoy Pro mini, class 0/0, rev 3.00/1.00, addr 2> on usbus0
umass0:  SCSI over Bulk-Only; quirks = 0x0100
umass0:0:0: Attached to scbus0
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
da0 at umass-sim0 bus 0 scbus0 target 0 lun 0
da0: <OWC Envoy Pro mini 0> Fixed Direct Access SPC-4 SCSI device
da0: Serial Number 000000000011
da0: 400.000MB/s transfers
da0: 228936MB (468862128 512 byte sectors)
da0: quirks=0x2<NO_6_BYTE>
Setting hostuuid: a4f7fbeb-f668-11de-b280-ebb65474e619.
Setting hostid: 0xcd8e9e25.
Starting file system checks:
/dev/gpt/RPi4Broot: FILE SYSTEM CLEAN; SKIPPING CHECKS
. . .

Notes:

Both cortex-a72 and -a53 are ARMv8.0-A.
Cortex-a72 does have out of order execution
(5 wide dispatch), while cortex-a53 does not.
Branch prediction is also different, as is
the "exec." port count (8 vs. 2). And so on.
So code tuned for cortex-a72 likely has a
different order to try to take advantage of
such points and the execution order need not
match the program order for cortex-a53
tuning.

But I do not know if that is the important
distinction for the kernel's failure to mount
the USB3 SSD for kernels targeting cortex-a72.
Both being ARMv8.0-A should make other sorts
of distinctions fairly minimal, or so I'd
expect.

Both cortex-a?? contexts have been using:

over_voltage=6
arm_freq=2000

(I've been using the pair of setting from my
first use of the UEFI/ACPI software. I've
also compared to a 2G Hz MACCHIATObin Double
Shot.)

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)