kern/166566: [zfs] zfs split renders 2 disk (MBR based) mirror unbootable

George Hartzell hartzell at alerce.com
Tue Jun 26 21:40:12 UTC 2012


The following reply was made to PR kern/166566; it has been noted by GNATS.

From: George Hartzell <hartzell at alerce.com>
To: Andriy Gapon <avg at FreeBSD.org>
Cc: hartzell at alerce.com,
    bug-followup at FreeBSD.org
Subject: Re: kern/166566: [zfs] zfs split renders 2 disk (MBR based) mirror
 unbootable
Date: Tue, 26 Jun 2012 14:38:17 -0700

 Andriy Gapon writes:
  > 
  > [restoring bug-followup]
  > 
  > on 25/06/2012 19:48 George Hartzell said the following:
  > > Here are two images (more than one screen full) of the lsdev -v output
  > > from the loader that *actually loads the system* (when it's
  > > working...).
  > 
  > I have a theory of what's going on.
  > 
  > I believe that after zpool split the following items get updated with new
  > information:
  > - vdev label on the disk that remains in the main pool (ada3 + zroot)
  > - vdev label on the disk that goes to the new pool (ada1 + zsplitroot)
  > - zpool.cache file in the main/active/remaining pool (zroot)
  > 
  > The following item still has outdated information:
  > - zpool.cache file in the new pool (zsplitroot)
  > 
  > This happens because the new pool gets the contents of the original pool at
  > split start time (before any new ids are generated).  The file can not be
  > updated automatically because the new pool remains "un-imported" (exported)
  > after the split.  If it is desired that the zsplitroot's zpool.cache is updated
  > it has to be done manually - by importing the pool, etc.
  > 
  > I believe that what you see is a result of you always booting in such a way that
  > the zfs boot code and zfsloader find zsplitroot pool before zroot pool.  This is
  > confirmed by the screenshot which shows that zsplitroot is listed before zroot.
  > Because of that the stale zpool.cache file is used and as a result the ZFS code
  > in kernel can not find disks/pools based on the stale IDs.
  > 
  > I think that you have to change the boot order using BIOS, so that you boot from
  > ada3 disk.  You should verify at the loader prompt that that is indeed the case
  > and zroot is found first and is used as a boot pool.
  > 
  > If your BIOS either doesn't allow to change boot order, or lies about it or
  > doesn't change bios disk numbering such that a boot disk is the first drive
  > (disk0 / "BIOS drive C"), then I recommend thatyou set 'currdev' loader variable
  > to point to zroot pool.  Depending on your zfsloader version it should be done
  > in one of the following ways:
  > set currdev=zfs:zroot:
  > set currdev=zfs1
  > You can examine default value of the variable (with 'show' command) to see which
  > scheme should be used.
  > 
  > Please test this.
 
 We're very close.
 
 First thing, I discovered that I was wrong about its being able to
 boot from either disk.  I don't think I ever misspoke to you (don't
 see it in the bug report, can't find it in our personal emails) but I
 certainly had it in my head that I had tried booting from both disks
 and neither worked.  It turns out that one will boot but the other
 will not.
 
 Some background (for the bug trail).  This is a mac pro with four
 internal SATA disks.  When you power on with the option key held down
 you're presented with a graphic view of things that you can boot from.
 In my configuration I have two disks labeled "dinky 1" and "dinky 2"
 (Mac disks in a Mac OS X software raid) and two labeled "Windows" (the
 boot stuff considers anything with an MBR to be Windows, sigh...).
 Macs play fast and loose with device numbering/naming and I have no
 way to tell the Windows devices apart.  For the rest of this
 discussion I'll just refer to the four drives as A, B, C, and D (dinky
 1, dinky 2, Windows, Windows) in left to right order.  
 
 Out of habit, I tend to boot into BSD on drive C (two hits of the
 right arrow key).
 
 While I was playing with setting currdev in the loader I realized that
 I could boot from disk D but not from disk C (no matter how I set
 currdev).  
 
 It turns out that when I boot from drive C and do an lsdev -v at the
 loader prompt I get
 
    ...
    zsplitroot  ada1s1a
    zroot       ada3s1a
 
 but/and when I boot from drive D and do an lsdev -v at the loader
 prompt I get
 
    ...
    zroot       ada3s1a
    zsplitroot  ada1s1a
 
 Notice that the order is different (confirming your observation).
 
 When I boot off of C using vfs.zfs.debug=1 I get messages about
 mismatched GUID's and failure to open the device it's looking for.
 When I boot off of D things are fine.  This is consistent with your
 idea that there is incorrect information in the zpool.cache file on
 the filesystem in the zsplitroot pool.
 
 currdev does not seem to have any effect, it looks like something else
 is being used to find the initial zfs pool(s).
 
 I'm not sure what there is to do to make the situation better.  It's
 (probably) not the usual use case that zpool splilt is being used to
 split the pool that contains the filesystem that contains the
 zpool.cache file, so it would be an awfully special case to do
 something special to handle it.
 
 It seems like "the right thing" to do would be for the user (me) to do
 the zpool split with the -R option then copy the correct zpool.cache
 file into the split-off pool's root filesystem.  I'll repair my
 currently broken mirror and give that a try.
 
 Thanks for all the help!
 
 g.


More information about the freebsd-fs mailing list