ZFS booting from MBR Question

[Paul Kraus]

I have an older system that did not want to boot from ZFS/GPT, so I installed it ZFS/MBR and it boots and runs fine, with one small exception. It is configured as a 4-way mirror (for read performance) but when the system boots it is clear that all of the parts of the boot loader are only installed on ONE of the four drives. I can watch it try to boot from each of the four drives and only succeeded from one of them. This is running 10.1 and I installed via bsdinstall and the ZFS-on-root option changing partition type from the ZFS Configuration screen. The error echoes to the console and confers not being able to find a zfs label, but zdb -l shows all 4 copies of the label intact on both part ions with zpools on them (bootpool and rootpool, yes, I use root pool instead of the FreeBSD default because I started with ZFS under Solaris in 2006 or so).

I have not been able to find any documentation for how to write all the components of the boot loader for ZFS/MBR manually. Does someone here know what bsdinstall is doing under the covers ?

I am assuming the issue is the phase 1 boot loader on ada<n>s1 partition, but am not sure how to write that correctly, and I am afraid of overwriting the one good one I have.

Relevant info below

root at m2:/dev # gpart show ada0
=>       63  976773105  ada0  MBR  (466G)
         63  976773105     1  freebsd  [active]  (466G)

root at m2:/dev # gpart show ada0s1
=>        0  976773105  ada0s1  BSD  (466G)
          0    4194304       1  freebsd-zfs  (2.0G)
    4194304    4194304       2  freebsd-swap  (2.0G)
    8388608  968384497       4  freebsd-zfs  (462G)

root at m2:/dev # zpool status
  pool: bootpool
 state: ONLINE
  scan: scrub repaired 0 in 0h1m with 0 errors on Wed Jun 10 03:07:15 2015
config:

	NAME         STATE     READ WRITE CKSUM
	bootpool     ONLINE       0     0     0
	  mirror-0   ONLINE       0     0     0
	    ada0s1a  ONLINE       0     0     0
	    ada1s1a  ONLINE       0     0     0
	    ada2s1a  ONLINE       0     0     0
	    ada3s1a  ONLINE       0     0     0

errors: No known data errors

  pool: rootpool
 state: ONLINE
  scan: scrub repaired 0 in 1h53m with 0 errors on Wed Jun 10 05:00:12 2015
config:

	NAME         STATE     READ WRITE CKSUM
	rootpool     ONLINE       0     0     0
	  mirror-0   ONLINE       0     0     0
	    ada0s1d  ONLINE       0     0     0
	    ada1s1d  ONLINE       0     0     0
	    ada2s1d  ONLINE       0     0     0
	    ada3s1d  ONLINE       0     0     0

errors: No known data errors
root at m2:/dev # zfs list
NAME                     USED  AVAIL  REFER  MOUNTPOINT
bootpool                 502M  1.43G   501M  /bootpool
rootpool                 209G   237G    19K  none
rootpool/ROOT           6.33G   237G    19K  none
rootpool/ROOT/20150603     1K   237G  3.52G  /
rootpool/ROOT/default   6.33G   237G   817M  /
rootpool/software       3.63G   237G  3.63G  /software
rootpool/tmp             104K   237G   104K  /tmp
rootpool/usr            4.29G   237G    19K  /usr
rootpool/usr/home         28K   237G    19K  /usr/home
rootpool/usr/ports      3.94G   237G  3.00G  /usr/ports
rootpool/usr/src         359M   237G   359M  /usr/src
rootpool/var             194G   237G    19K  /var
rootpool/var/crash        19K   237G    19K  /var/crash
rootpool/var/log        42.4M   237G   400K  /var/log
rootpool/var/mail        194G   237G   190G  /var/mail
rootpool/var/spool      1.62M   237G   798K  /var/spool
rootpool/var/tmp          19K   237G    19K  /var/tmp
root at m2:/dev # 


--
Paul Kraus
paul at kraus-haus.org