ZFS 'mount error 5' hits production server

[Michael Schmiedgen]

On 12/04/2014 00:11, Steven Hartland wrote:
> Have you been able to boot from rescue / mfsroot cd?

I got remote console via KVM and booted kernel.old
from loader prompt. Everythins is fine with old 10.0
kernel. Finished upgrading by installing 10.1 userland
did not help either.

Thanks,
   Michael

> On 03/12/2014 20:11, Michael Schmiedgen wrote:
>> Hi list,
>>
>> today I upgraded one of our servers from 10.0 to 10.1 with
>> freebsd-update. GENERIC kernel, ZFS 2-way mirror, GPT, nothing
>> special. After booting kernel, when trying to mount the
>> root file system I get: 'cannot mount, error 5' or something.
>>
>> Strangely I got this error a few weeks ago at home, running
>> CURRENT with custom kernel:
>> https://lists.freebsd.org/pipermail/freebsd-current/2014-October/052900.html
>>
>>
>> I created the pools and datasets manually via console. Has
>> anything changed? With legacy mountpoint, or that one needs a
>> boot dataset, or something?
>>
>> Thanks,
>>   Michael
>>
>>
>> Configuration is:
>>
>> gpart show:
>>
>> =>        34  5860533101  ada0  GPT  (2.7T)
>>           34         256     1  freebsd-boot  (128K)
>>          290           6        - free -  (3.0K)
>>          296     8388608     2  freebsd-swap  (4.0G)
>>      8388904  5662310400     3  freebsd-zfs  (2.6T)
>>   5670699304   189833831        - free -  (91G)
>>
>> =>        34  5860533101  ada1  GPT  (2.7T)
>>           34         256     1  freebsd-boot  (128K)
>>          290           6        - free -  (3.0K)
>>          296     8388608     2  freebsd-swap  (4.0G)
>>      8388904  5662310400     3  freebsd-zfs  (2.6T)
>>   5670699304   189833831        - free -  (91G)
>>
>>
>> zpool status:
>>
>> NAME  PROPERTY                       VALUE      SOURCE
>> tank  size                           2.62T      -
>> tank  capacity                       3%         -
>> tank  altroot                        -          default
>> tank  health                         ONLINE     -
>> tank  guid                           XXX        default
>> tank  version                        -          default
>> tank  bootfs                         tank       local
>> tank  delegation                     on         default
>> tank  autoreplace                    off        default
>> tank  cachefile                      -          default
>> tank  failmode                       wait       default
>> tank  listsnapshots                  off        default
>> tank  autoexpand                     off        default
>> tank  dedupditto                     0          default
>> tank  dedupratio                     1.00x      -
>> tank  free                           2.53T      -
>> tank  allocated                      99.0G      -
>> tank  readonly                       off        -
>> tank  comment                        -          default
>> tank  expandsize                     0          -
>> tank  freeing                        0          default
>> tank  fragmentation                  0%         default
>> tank  leaked                         0          default
>> tank  feature at async_destroy          enabled    local
>> tank  feature at empty_bpobj            active     local
>> tank  feature at lz4_compress           enabled    local
>> tank  feature at multi_vdev_crash_dump  enabled    local
>> tank  feature at spacemap_histogram     disabled   local
>> tank  feature at enabled_txg            disabled   local
>> tank  feature at hole_birth             disabled   local
>> tank  feature at extensible_dataset     disabled   local
>> tank  feature at embedded_data          disabled   local
>> tank  feature at bookmarks              disabled   local
>> tank  feature at filesystem_limits      disabled   local
>>
>>
>> zfs get all tank:
>>
>> NAME  PROPERTY              VALUE       SOURCE
>> tank  type                  filesystem  -
>> tank  creation              Aug 2012    -
>> tank  used                  98.9G       -
>> tank  available             2.49T       -
>> tank  referenced            8.34G       -
>> tank  compressratio         1.00x       -
>> tank  mounted               yes         -
>> tank  quota                 none        default
>> tank  reservation           none        default
>> tank  recordsize            128K        default
>> tank  mountpoint            legacy      local
>> tank  sharenfs              off         default
>> tank  checksum              on          default
>> tank  compression           off         default
>> tank  atime                 off         local
>> tank  devices               on          default
>> tank  exec                  on          default
>> tank  setuid                on          default
>> tank  readonly              off         default
>> tank  jailed                off         default
>> tank  snapdir               hidden      default
>> tank  aclmode               discard     default
>> tank  aclinherit            restricted  default
>> tank  canmount              on          default
>> tank  xattr                 off         temporary
>> tank  copies                1           default
>> tank  version               5           -
>> tank  utf8only              off         -
>> tank  normalization         none        -
>> tank  casesensitivity       sensitive   -
>> tank  vscan                 off         default
>> tank  nbmand                off         default
>> tank  sharesmb              off         default
>> tank  refquota              none        default
>> tank  refreservation        none        default
>> tank  primarycache          all         default
>> tank  secondarycache        all         default
>> tank  usedbysnapshots       4.53G       -
>> tank  usedbydataset         8.34G       -
>> tank  usedbychildren        86.1G       -
>> tank  usedbyrefreservation  0           -
>> tank  logbias               latency     default
>> tank  dedup                 off         default
>> tank  mlslabel                          -
>> tank  sync                  standard    default
>> tank  refcompressratio      1.00x       -
>> tank  written               3.10G       -
>> tank  logicalused           91.7G       -
>> tank  logicalreferenced     6.71G       -
>> tank  volmode               default     default
>> tank  filesystem_limit      none        default
>> tank  snapshot_limit        none        default
>> tank  filesystem_count      none        default
>> tank  snapshot_count        none        default
>> tank  redundant_metadata    all         default