[Bug 295311] bhyve: Disk errors on a Debian VM when using ahci-hd, failure to boot with nvme

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=295311

            Bug ID: 295311
           Summary: bhyve: Disk errors on a Debian VM when using ahci-hd,
                    failure to boot with nvme
           Product: Base System
           Version: 15.1-RELEASE
          Hardware: amd64
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: bhyve
          Assignee: virtualization@FreeBSD.org
          Reporter: 9vlc@proton.me
 Attachment #270762 text/plain
         mime type:

Created attachment 270762
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=270762&action=edit
VM boot log file

After updating my host from 15.0-RELEASE-p4 to 15.1-BETA2 my Debian 13 VM with
storage as a separate NVMe drive has stopped booting:
BdsDxe: failed to load Boot0001 "UEFI bhyve-NVMe NVME-1-0" from
PciRoot(0x0)/Pci(0x1,0x0)/NVMe(0x1,01-00-77-15-20-FC-9C-58): Not Found
BdsDxe: No bootable option or device was found.
BdsDxe: Press any key to enter the Boot Manager Menu.

Switching the disk device in bhyve from "nvme" to "ahci-hd" results in it
getting to GRUB
and attempting to boot, but throwing a ton of disk errors like:
[    2.240679]          res 41/04:00:00:00:00/00:00:00:00:00/a0 Emask 0x1
(device error)
[    2.241182] ata1.00: status: { DRDY ERR }
[    2.241317] ata1.00: error: { ABRT }
[    2.252197] ata1.00: configured for UDMA/133
[    2.262639] ata1.00: device reported invalid CHS sector 0
[    2.262828] ata1: EH complete
[    2.291791] ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
[    2.292094] ata1.00: irq_stat 0x40000001
[    2.292272] ata1.00: failed command: FLUSH CACHE EXT
[    2.292487] ata1.00: cmd ea/00:00:00:00:00/00:00:00:00:00/a0 tag 10
[    2.292487]          res 41/04:00:00:00:00/00:00:00:00:00/a0 Emask 0x1
(device error)
[    2.293073] ata1.00: status: { DRDY ERR }
[    2.293220] ata1.00: error: { ABRT }

The VM does boot, but takes quite some time to do so and I absolutely do not
trust
it with my data in the current state.

Here is the bhyve commandline I used:
bhyve -DSHAw \
        -c 2 -m 4g \
        -l bootrom,/usr/local/share/edk2-bhyve/BHYVE_UEFI.fd \
        -s 0,hostbridge -s 31,lpc \
        -l com1,/dev/nmdm0A \
        \
        -s 1:0,nvme,/dev/nda0 \
        \
        -s 2:0,virtio-net,tap0,mac=REDACTED \
        \
        -s 5:0,fbuf,tcp=0.0.0.0:5900,wait \
        -s 5:1,xhci,tablet \
        \
        $vm_name

After some more testing, it seems like this only happens when I use the
physical NVMe as the VM's storage.
Doing passthrough on the NVMe does result in improvement: The VM boots although
with some complaining from bhyve:
(~/Desktop/linb) % sudo sh bhyve.sh 
fbuf frame buffer base: 0x3188f4600000 [sz 33554432]
pci_passthru: unmap_pptdev_mmio failed: No such file or directory
pci_passthru: unmap_pptdev_mmio failed: No such file or directory
pci_passthru: unmap_pptdev_mmio failed: No such file or directory
pci_passthru: unmap_pptdev_mmio failed: No such file or directory
pci_passthru: unmap_pptdev_mmio failed: No such file or directory
pci_passthru: unmap_pptdev_mmio failed: No such file or directory
pci_passthru: unmap_pptdev_mmio failed: No such file or directory
pci_passthru: unmap_pptdev_mmio failed: No such file or directory
pci_passthru: unmap_pptdev_mmio failed: No such file or directory
pci_passthru: unmap_pptdev_mmio failed: No such file or directory
pci_passthru: unmap_pptdev_mmio failed: No such file or directory
pci_passthru: unmap_pptdev_mmio failed: No such file or directory
pci_passthru: unmap_pptdev_mmio failed: No such file or directory
pci_passthru: unmap_pptdev_mmio failed: No such file or directory

Providing its copy as a file to the VM works fine. The OS installed on the NVMe
boots on bare metal.

The NVMe is a 512GB Apacer from the 2280 line, I would like to know if a
similar issue appears when 
nvme0@pci0:1:0:0:       class=0x010802 rev=0x01 hdr=0x00 vendor=0x1987
device=0x5013 subvendor=0x1987 subdevice=0x5013
    vendor     = 'Phison Electronics Corporation'
    device     = 'PS5013-E13 PCIe3 NVMe Controller (DRAM-less)'
    class      = mass storage
    subclass   = NVM

I have attached a full boot log of the VM loading to the rescue mode with the
emulated storage as "ahci-hd".
I really don't know if this is reproducible outside of my setup.

-- 
You are receiving this mail because:
You are the assignee for the bug.