bhyve vtblk error

[Andriy Gapon]

The following happened with an illumos VM in bhyve.
Usually I reboot that VM by first powering it off using illumos 'poweroff'
command.  But yesterday I tried to do that using 'reboot' command.  Apparently
in that case illumos performs a warm reboot where a new kernel instance is
executed without the full hardware reset and firmware boot.

So, I got the following error in the guest during the reboot:
Block device: blkdev at 0,0, blkdev0
blkdev0 is /pci at 0,0/pci1af4,2 at 4/blkdev at 0,0
/pci at 0,0/pci1af4,2 at 4/blkdev at 0,0 (blkdev0) online
WARNING: blkdev0: Invalid media block size (0)
WARNING: blkdev0: Invalid media block size (0)
NOTICE: Cannot read the pool label from '/pci at 0,0/pci1af4,2 at 4/blkdev at 0,0:a'
NOTICE: spa_import_rootpool: error 5
Cannot mount root on /pci at 0,0/pci1af4,2 at 4/blkdev at 0,0:a fstype zfs

At the same time bhyve printed the following message:
vtblk: read from CFGVEC: bad size 4


P.S.
There is another problem that perhaps is worth some attention as well.
The guest paniced after it failed to import the root pool.  bhyve got stuck in a
weird way after that:
vm exit[2]
vm exit[1]
vm exit[3]
        reason          SVM
        rip             0xfffffffffb8440aa
        inst_length     0
        exitcode        0x7f
        exitinfo1       0
        exitinfo2       0
        reason          SVM
        rip             0xfffffffffb8440aa
        inst_length     0
        exitcode        0x7f
        exitinfo1       0
        exitinfo2       0
        reason          SVM
        rip             0xfffffffffb8440aa
        inst_length     0
        exitcode        0x7f
        exitinfo1       0
        exitinfo2       0
^T
load: 0.98  cmd: bhyve 87140 [suspended] 20861.13r 643.47u 41127.33s 0% 3727592k

As you can see the process somehow became suspended and I was not able to
recover it (SIGKILL didn't help) and had to reboot the host.

-- 
Andriy Gapon