kern/166566: [zfs] zfs split renders 2 disk (MBR based) mirror unbootable

[Andriy Gapon]

The following reply was made to PR kern/166566; it has been noted by GNATS.

From: Andriy Gapon <avg at FreeBSD.org>
To: hartzell at alerce.com, bug-followup at FreeBSD.org
Cc:  
Subject: Re: kern/166566: [zfs] zfs split renders 2 disk (MBR based) mirror
 unbootable
Date: Tue, 26 Jun 2012 13:47:50 +0300

 [restoring bug-followup]
 
 on 25/06/2012 19:48 George Hartzell said the following:
 > Here are two images (more than one screen full) of the lsdev -v output
 > from the loader that *actually loads the system* (when it's
 > working...).
 
 I have a theory of what's going on.
 
 I believe that after zpool split the following items get updated with new
 information:
 - vdev label on the disk that remains in the main pool (ada3 + zroot)
 - vdev label on the disk that goes to the new pool (ada1 + zsplitroot)
 - zpool.cache file in the main/active/remaining pool (zroot)
 
 The following item still has outdated information:
 - zpool.cache file in the new pool (zsplitroot)
 
 This happens because the new pool gets the contents of the original pool at
 split start time (before any new ids are generated).  The file can not be
 updated automatically because the new pool remains "un-imported" (exported)
 after the split.  If it is desired that the zsplitroot's zpool.cache is updated
 it has to be done manually - by importing the pool, etc.
 
 I believe that what you see is a result of you always booting in such a way that
 the zfs boot code and zfsloader find zsplitroot pool before zroot pool.  This is
 confirmed by the screenshot which shows that zsplitroot is listed before zroot.
 Because of that the stale zpool.cache file is used and as a result the ZFS code
 in kernel can not find disks/pools based on the stale IDs.
 
 I think that you have to change the boot order using BIOS, so that you boot from
 ada3 disk.  You should verify at the loader prompt that that is indeed the case
 and zroot is found first and is used as a boot pool.
 
 If your BIOS either doesn't allow to change boot order, or lies about it or
 doesn't change bios disk numbering such that a boot disk is the first drive
 (disk0 / "BIOS drive C"), then I recommend thatyou set 'currdev' loader variable
 to point to zroot pool.  Depending on your zfsloader version it should be done
 in one of the following ways:
 set currdev=zfs:zroot:
 set currdev=zfs1
 You can examine default value of the variable (with 'show' command) to see which
 scheme should be used.
 
 Please test this.
 
 -- 
 Andriy Gapon