Booting r298488 as Xen Dom0 may break ZFS pool?

[Marcin Cieslak]

On Mon, 25 Apr 2016, Roger Pau Monné wrote:

> > I have managed to make zpool boot again by doing voodoo
> > similar to this one:
> > 
> > [root at rescue ~]# zpool import -R /mnt zroot
> > [root at rescue ~]# mount -t devfs devfs /mnt/dev
> > [root at rescue ~]# chroot /mnt /bin/tcsh
> > 
> > (... Running make install in /usr/src/sys/boot ...)
> > 
> > root at rescue:/ # gpart bootcode -p /boot/gptzfsboot -i 1 ada0
> > partcode written to ada0p1
> > root at rescue:/ # gpart bootcode -p /boot/gptzfsboot -i 1 ada1
> > partcode written to ada1p1
> > root at rescue:/ # exit
> > [root at rescue ~]# umount /mnt/dev
> > [root at rescue ~]# zpool export zroot
> > [root at rescue ~]# reboot
> > 
> > Why zpool metadata get corrupted?
> 
> TBH, I have no idea. Can you also paste the log of the system (Xen + 
> FreeBSD) when it fails to boot? If that's not possible, can you at least 
> paste the output of `xl dmesg` when booted correctly under Xen?

I am using a VGA-emulating console attached by my friendly hosting
provider (Hetzner) and it does not give me the terminal-like output,
only VGA. From the quickly scrolling Xen kernel messages
I am getting to the stage where it is "scrubbing" and hands over
to the kernel.

> What operations did you perform when the system booted correctly using 
> FreeBSD/Xen?

Nothing in particular. It seems it succeeds every other time.

> Does the disk get corrupted even if the system fails to boot? AFAICT, it 
> seems like it's only the bootcode that gets corrupted, is that right?

I am not sure what really gets corrupted - once I noticed that
just rewriting bootcode (without "make install") didn't help (at least once).

Maybe this is some leftover from the 10.3 -> 11 upgrade? It's hard
to install 11 clean on the rented server.... But no idea how would *that*
affect booting.

I also have two drives on the controller, maybe something is not in sync there?

Marcin