[Bug 250377] loader fails to detect zpool & reports ZFS: i/o error - all block copies unavailable since 12.0-12.2

[bugzilla-noreply at freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=250377

            Bug ID: 250377
           Summary: loader fails to detect zpool & reports ZFS: i/o error
                    - all block copies unavailable since 12.0-12.2
           Product: Base System
           Version: 12.1-RELEASE
          Hardware: amd64
                OS: Any
            Status: New
          Severity: Affects Some People
          Priority: ---
         Component: kern
          Assignee: bugs at FreeBSD.org
          Reporter: dch at freebsd.org

FreeBSD 12.x amd64 (efi) loaders seems to behave differently than 11.x
under xhyve OSX based hypervisor, where 11.x used to work.

Using xhyve[1], 12.x amd64 all fails *after successful install* on reboot at:

/boot/kernel/kernel text=0x16bdcc4 data=0x140 data=0x75fe80 ZFS: i/o error -
all block copies unavailable

elf64_loadimage: read failed
can't load file '/boot/kernel/kernel': input/output error

Checking the resulting disk image by mounting on FreeBSD via mdconfig & zpool
scrub etc seems to be fine. Remounting it from the install CD is also fine.

I have used xhyve on OSX to distribute FreeBSD VMs to developers for a few
years now, and it's worked just fine on 10.x and 11.x for a simple reliable
boot, including 11.4R.

I re-tested these with 12.0,12.1 and now 12.2 RC2 and they all fail with the
same error, but only after install completed successfully, which I don't yet
understand.

Finally, updating kernel from a working 11.4R -> 12.1R breaks with the same
error on first reboot. Suggesting the newer loader itself is the issue.

This seems a bit of an odd situation.

What might be different between the install phase, and resulting loader
detection after install?

The error appears to be unaffected by block size, and is present since 12.0 at
least.

Rolling back xhyve patches to the original ~2017-2018 version also shows no
variation, so this appears to be something different in FreeBSD 12.x and not
the hypervisor itself.

[1]: https://github.com/machyve/xhyve "bhyve for OSX"
[2]: https://github.com/machyve/xhyve/issues/111 

The same image, whether produced from release ISOs on FreeBSD or on xhyve,
fails when launched from xhyve.

how to reproduce (requires a mac with homebrew at least for the final step)

- create a 5G sparse disk image
- start FreeBSD and do install iso dance
- move the dis image after install to OSX
- test via following invocation of xhyve

#!/bin/sh
UUID="-U deaddead-dead-dead-dead-deaddeaddead"

CD="/downloads/images/FreeBSD-12.1-RELEASE-amd64-disc1.iso"
IMG="xhyve-12.1R.img"
USERBOOT="$(brew --prefix xhyve)/share/xhyve/test/userboot.so"
BOOTVOLUME=$CD

KERNELENV=""

MEM="-m 8G"
SMP="-c 2"
PCI_DEV="-s 0:0,hostbridge -s 31,lpc"
NET="-s 2:0,virtio-net"
IMG_CD="-s 3:0,ahci-cd,$CD"
IMG_HDD="-s 4:0,virtio-blk,$IMG"
LPC_DEV="-l com1,stdio"
ACPI="-A"

sudo xhyve $ACPI $MEM $SMP \
    $PCI_DEV $LPC_DEV \
    $NET \
    $IMG_CD $IMG_HDD \
    $UUID -f fbsd,$USERBOOT,$BOOTVOLUME,"$KERNELENV"

-- 
You are receiving this mail because:
You are the assignee for the bug.