Mounting from zfs failed with error 22 with gmirror

[Ronald Klop]

On Sat, 14 Sep 2013 21:53:02 +0200, Daniel Gerzo <danger at freebsd.org>  
wrote:

> Hello list,
>
> I have come across this thing and I don't have an idea what to do next.
>
> I have this partition setup:
>
> [root at rescue ~]# gpart show
> =>        34  3907029101  ada0  GPT  (1.8T)
>            34           6        - free -  (3.0k)
>            40        1024     1  freebsd-boot  (512k)
>          1064    83886080     2  freebsd-swap  (40G)
>      83887144  3823141984     3  freebsd-zfs  (1.8T)
>    3907029128           7        - free -  (3.5k)
>
> =>        34  3907029101  ada1  GPT  (1.8T)
>            34           6        - free -  (3.0k)
>            40        1024     1  freebsd-boot  (512k)
>          1064    83886080     2  freebsd-swap  (40G)
>      83887144  3823141984     3  freebsd-zfs  (1.8T)
>    3907029128           7        - free -  (3.5k)
>
> [root at rescue ~]# gpart show -l
> =>        34  3907029101  ada0  GPT  (1.8T)
>            34           6        - free -  (3.0k)
>            40        1024     1  boot0  (512k)
>          1064    83886080     2  swap0  (40G)
>      83887144  3823141984     3  sys0  (1.8T)
>    3907029128           7        - free -  (3.5k)
>
> =>        34  3907029101  ada1  GPT  (1.8T)
>            34           6        - free -  (3.0k)
>            40        1024     1  boot1  (512k)
>          1064    83886080     2  swap1  (40G)
>      83887144  3823141984     3  sys1  (1.8T)
>    3907029128           7        - free -  (3.5k)
> [root at rescue ~]# zpool import -f -o altroot=/mnt -o  
> cachefile=/boot/zfs/zpool.cache sys
> [root at rescue ~]# zpool status
>    pool: sys
>   state: ONLINE
>    scan: none requested
> config:
>
>      NAME          STATE     READ WRITE CKSUM
>      sys           ONLINE       0     0     0
>        mirror-0    ONLINE       0     0     0
>          gpt/sys0  ONLINE       0     0     0
>          gpt/sys1  ONLINE       0     0     0
>
> errors: No known data errors
> [root at rescue ~]# zdb
> sys:
>      version: 28
>      name: 'sys'
>      state: 0
>      txg: 13622
>      pool_guid: 13749191682008517984
>      hostid: 966392425
>      hostname: 'rescue'
>      vdev_children: 1
>      vdev_tree:
>          type: 'root'
>          id: 0
>          guid: 13749191682008517984
>          children[0]:
>              type: 'mirror'
>              id: 0
>              guid: 10821644781744913225
>              metaslab_array: 30
>              metaslab_shift: 34
>              ashift: 12
>              asize: 1957443928064
>              is_log: 0
>              create_txg: 4
>              children[0]:
>                  type: 'disk'
>                  id: 0
>                  guid: 12516881521540558071
>                  path: '/dev/gpt/sys0'
>                  phys_path: '/dev/gpt/sys0'
>                  whole_disk: 1
>                  create_txg: 4
>              children[1]:
>                  type: 'disk'
>                  id: 1
>                  guid: 187152467666907385
>                  path: '/dev/gpt/sys1'
>                  phys_path: '/dev/gpt/sys1'
>                  whole_disk: 1
>                  create_txg: 4
> [root at rescue ~]# zpool get bootfs sys
> NAME  PROPERTY  VALUE             SOURCE
> sys   bootfs    sys/default/root  local
> [root at rescue ~]# gmirror status
>         Name    Status  Components
> mirror/swap  COMPLETE  ada1p2 (ACTIVE)
>                         ada0p2 (ACTIVE)
>
> The problem is that while I do not load geom_mirror from loader.conf,  
> the machine boots fine, however as soon as I enable gmirror in  
> loader.conf the machine doesn't boot and errors with
>
> /Trying to mount root from zfs:sys/default/root [].../
>
> /Mounting from zfs:sys/default/root failed with error 22.
> /
>
>
> and it hangs in the prompt asking me to enter device to mount root from.
>
> I found only this  
> http://lists.freebsd.org/pipermail/freebsd-current/2012-November/037910.html  
> email where avg@ mentions that it might be a bug in his code, but no  
> further followups. However that is almost a year ago and I got trapped  
> by this on 9.2-RC4.
>
> Could anyone possibly give me some hints? (Please keep in in cc: as I am  
> not subscribed to fs@)
>
> Thank you in advance!
>
> Kind regards,
> Daniel

Is it possible you used these disks with gmirror in the past and you did  
not correctly clean the meta-data from the disks when you started using  
zfs?
So gmirror now detects meta-data, starts handling the disks in some way  
and zfs does not get all the sectors of the disk it is expecting?

Ronald.