ZFS bug: zpool expandz says 16.0E, clearly wrong

Marie Helene Kvello-Aune marieheleneka at gmail.com
Tue Jan 26 16:17:50 UTC 2016


I've stumbled across a curiosity with my zpool. The command 'zpool list'
states that the EXPANDZ property/value is 16.0E. This is clearly incorrect.
:)

The pool consist of a single RaidZ2 vdev of 6 drives, and two cache drives.
No log device. Executing 'zpool list -v' shows that each member of the
RaidZ2 has an 'EXPANDZ' value of '-', as expected. But the RaidZ2 itself,
and the pool, has a EXPANDZ value of 16.0E.

Each member of the RaidZ2 vdev is a GELI encrypted partition.

# uname -a
FreeBSD saturn 10.3-PRERELEASE FreeBSD 10.3-PRERELEASE #1 r294289: Mon Jan
18 21:22:41 CET 2016     root at saturn:/usr/obj/usr/src/sys/GENERIC  amd64
(It was built from 10-STABLE)

Once I discovered this curiosity I started a scrub, which is scheduled to
finish in about 7 hours from now. It seems to be done scanning the zpool
metadata and haven't repaired anything yet.

Output of zpool list, zpool list -v, zpool history and zfs list are
attached.

Anyone have any idea how to figure out what has happened here?

Regards,
Marie Helene Kvello-Aune
marieheleneka at gmail.com
-------------- next part --------------
# zpool list storage
NAME      SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
storage  21.6T  4.07T  17.6T     16.0E    11%    18%  1.00x  ONLINE  -

# zpool list -v storage
NAME               SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
storage           21.6T  4.07T  17.6T     16.0E    11%    18%  1.00x  ONLINE  -
  raidz2          21.6T  4.07T  17.6T     16.0E    11%    18%
    gpt/Bay1.eli      -      -      -         -      -      -
    gpt/Bay2.eli      -      -      -         -      -      -
    gpt/Bay3.eli      -      -      -         -      -      -
    gpt/Bay4.eli      -      -      -         -      -      -
    gpt/Bay5.eli      -      -      -         -      -      -
    gpt/Bay6.eli      -      -      -         -      -      -
cache                 -      -      -         -      -      -
  gpt/cache0.eli  8.00G  7.98G  15.4M         -     0%    99%
  gpt/cache1.eli  8.00G  7.98G  16.3M         -     0%    99%


# zpool history storage
History for 'storage':
2015-12-26.16:14:43 zpool create storage raidz2 /dev/gpt/Bay1 /dev/gpt/Bay2 /dev/gpt/Bay3 /dev/gpt/Bay4 /dev/gpt/Bay5 /dev/gpt/Bay6 cache /dev/gpt/cache0 /dev/gpt/cache1
2015-12-26.16:30:40 zfs create storage/jails
2015-12-26.16:30:47 zfs create storage/iSCSI
2015-12-26.16:30:51 zfs create storage/CIFS
2015-12-26.16:31:04 zfs create storage/CIFS/bilder
2015-12-30.00:17:23 zpool offline storage gpt/Bay1
2015-12-30.00:20:39 zpool replace storage gpt/Bay1 gpt/Bay1.eli
2015-12-30.00:21:00 zpool offline storage gpt/Bay2
2015-12-30.00:21:05 zpool offline storage gpt/Bay3
2015-12-30.00:21:28 zpool replace storage gpt/Bay2 gpt/Bay2.eli
2015-12-30.00:21:34 zpool replace storage gpt/Bay3 gpt/Bay3.eli
2015-12-30.00:21:46 zpool offline storage gpt/Bay4
2015-12-30.00:21:52 zpool offline storage gpt/Bay5
2015-12-30.00:22:11 zpool replace storage gpt/Bay4 gpt/Bay4.eli
2015-12-30.00:22:16 zpool replace storage gpt/Bay5 gpt/Bay5.eli
2015-12-30.00:22:29 zpool offline storage gpt/Bay6
2015-12-30.00:22:34 zpool replace storage gpt/Bay6 gpt/Bay6.eli
2015-12-30.00:24:00 zpool remove storage gpt/cache0
2015-12-30.00:24:05 zpool remove storage gpt/cache1
2015-12-30.00:25:51 zpool export storage
2015-12-30.00:26:03 zpool import storage
2015-12-30.00:26:08 zpool add storage cache gpt/cache0.eli
2015-12-30.00:26:14 zpool add storage cache gpt/cache1.eli
2015-12-30.00:26:57 zpool remove storage gpt/cache0.eli
2015-12-30.00:27:02 zpool remove storage gpt/cache1.eli
2015-12-30.00:27:39 zpool add storage cache gpt/cache0.eli
2015-12-30.00:27:44 zpool add storage cache gpt/cache1.eli
2015-12-30.00:31:18 zpool scrub storage
2015-12-30.00:31:49 zpool scrub storage
2015-12-30.00:36:01 zpool remove storage gpt/cache0.eli
2015-12-30.00:36:06 zpool remove storage gpt/cache1.eli
2015-12-30.00:37:05 zpool add storage cache gpt/cache0.eli gpt/cache1.eli
2015-12-30.23:20:53 zfs create storage/jails/samba
2015-12-30.23:22:59 zfs set compress=lz4 storage/jails
2015-12-30.23:23:02 zfs destroy storage/jails/samba
2015-12-30.23:23:17 zfs create -o quota=2g storage/jails/samba
2015-12-30.23:27:38 zfs set mountpoint=/usr/jails storage/jails
2015-12-30.23:28:43 zfs set mountpoint=/usr/jails/samba/media storage/CIFS
2015-12-30.23:31:55 zfs set jailed=on storage/CIFS
2015-12-30.23:45:34 zfs set mountpoint=/media storage/CIFS
2015-12-31.00:12:06 zfs set quota=1t storage/CIFS/bilder
2016-01-01.23:30:24 zfs create -o mountpoint=none -o volmode=dev storage/bhyve
2016-01-01.23:30:43 zfs set compress=lz4 storage/bhyve
2016-01-01.23:34:17 zfs create -V 32g -b 4k storage/bhyve/test
2016-01-01.23:38:02 zfs create -o compress=gzip-9 storage/ISO
2016-01-01.23:57:42 zfs set vm:cpu=1 storage/bhyve
2016-01-01.23:57:50 zfs set vm:memory=512 storage/bhyve
2016-01-01.23:58:33 zfs create storage/bhyve/small
2016-01-01.23:58:54 zfs rename storage/bhyve/test storage/bhyve/small/test
2016-01-01.23:59:13 zfs inherit vm:cpu storage/bhyve
2016-01-01.23:59:18 zfs inherit vm:memory storage/bhyve
2016-01-01.23:59:25 zfs set vm:memory=512 storage/bhyve/small
2016-01-01.23:59:32 zfs set vm:cpu=1 storage/bhyve/small
2016-01-02.00:00:43 zfs set vm:interface:0=tap0 storage/bhyve/small/test
2016-01-02.00:29:10 zfs create -V 16g -b 4k storage/bhyve/small/test2
2016-01-02.00:29:34 zfs set vm:enabled=NO storage/bhyve
2016-01-02.00:34:32 zfs set vm:enabled=YES storage/bhyve/small/test
2016-01-02.00:34:37 zfs set vm:enabled=YES storage/bhyve/small/test2
2016-01-02.00:34:49 zfs create -V 16g -b 4k storage/bhyve/small/test3
2016-01-02.00:39:07 zfs set vm:interfaces=1 storage/bhyve/small/test
2016-01-02.00:39:30 zfs inherit vm:interface:0 storage/bhyve/small/test
2016-01-02.00:39:40 zfs set vm:interfaces=tap0 tap1 storage/bhyve/small/test
2016-01-02.02:09:43 zfs rename storage/bhyve/small/test storage/bhyve/small/test1
2016-01-03.06:39:45 zfs create storage/jails/zabbix
2016-01-03.06:40:02 zfs set quota=8g storage/jails/zabbix
2016-01-03.06:40:53 zfs set quota=1t storage/jails
2016-01-03.06:40:58 zfs set quota=128g storage/jails
2016-01-18.22:05:42 zfs set recordsize=1m storage/CIFS
2016-01-18.22:34:16 zfs set compress=lz4 storage/CIFS/bilder
2016-01-18.22:34:33 zfs set compress=off storage/CIFS/bilder
2016-01-19.14:49:45 zfs create -o quota=256g -o compress=gzip-9 storage/CIFS/downloads
2016-01-19.14:54:13 zfs set jailed=off storage/CIFS
2016-01-19.15:00:04 zfs set mountpoint=/mnt/storage storage
2016-01-19.15:00:15 zfs inherit mountpoint storage/CIFS
2016-01-21.09:20:53 zfs create -o quota=4g storage/jails/ansible
2016-01-21.09:23:43 zfs rename storage/jails/ansible storage/jails/ansible-master
2016-01-21.09:26:46 zfs create storage/jaildata
2016-01-21.09:26:51 zfs create storage/jaildata/ansible-master
2016-01-21.09:28:03 zfs set quota=1g storage/jails/ansible-master
2016-01-21.09:28:08 zfs set quota=4g storage/jails/ansible-master
2016-01-21.09:28:17 zfs set quota=4g storage/jaildata/ansible-master
2016-01-21.10:24:10 zfs create -o quota=8g storage/jails/gitlab
2016-01-21.10:24:23 zfs create -o quota=256g storage/jaildata/gitlab
2016-01-21.10:29:00 zfs create -o recordsize=8k storage/jaildata/gitlab/sql
2016-01-21.10:29:16 zfs create -o recordsize=1m storage/jaildata/gitlab/data
2016-01-21.12:21:54 zfs rename -f storage/jails/gitlab storage/jails/gogs
2016-01-21.12:22:09 zfs rename -f storage/jaildata/gitlab storage/jaildata/gogs
2016-01-24.23:53:14 zfs destroy -r storage/iSCSI/marie
2016-01-25.00:03:04 zfs destroy -r storage/iSCSI/marie
2016-01-25.00:03:19 zfs destroy -r storage/iSCSI/marie
2016-01-25.00:33:05 zfs destroy -r storage/iSCSI/marie
2016-01-25.00:33:17 zfs destroy -r storage/iSCSI/marie
2016-01-25.19:48:23 zfs set volmode=dev storage/iSCSI/marie/2
2016-01-25.19:48:45 zfs rename storage/iSCSI/marie/2 storage/iSCSI/marie/2_rn
2016-01-25.19:48:52 zfs rename storage/iSCSI/marie/2_rn storage/iSCSI/marie/2
2016-01-26.04:27:45 zfs recv -uFv storage/iSCSI/marie
2016-01-26.16:55:35 zfs set volmode=dev storage/iSCSI
2016-01-26.16:56:09 zpool scrub storage

# zfs list -r storage
NAME                              USED  AVAIL  REFER  MOUNTPOINT
storage                          2.78T  11.2T  1.22M  /mnt/storage
storage/CIFS                      305G  11.2T   192K  /mnt/storage/CIFS
storage/CIFS/bilder               292G   732G   292G  /mnt/storage/CIFS/bilder
storage/CIFS/downloads           13.1G   243G  13.1G  /mnt/storage/CIFS/downloads
storage/ISO                       474M  11.2T   474M  /mnt/storage/ISO
storage/bhyve                    68.0G  11.2T   192K  none
storage/bhyve/small              68.0G  11.2T   192K  none
storage/bhyve/small/test1        34.0G  11.2T   128K  -
storage/bhyve/small/test2        17.0G  11.2T   128K  -
storage/bhyve/small/test3        17.0G  11.2T   128K  -
storage/iSCSI                    2.41T  11.2T   192K  /mnt/storage/iSCSI
storage/iSCSI/marie              2.41T  11.2T   192K  /mnt/storage/iSCSI/marie
storage/iSCSI/marie/1            2.09T  11.2T  2.09T  -
storage/iSCSI/marie/2             332G  11.2T   331G  -
storage/iSCSI/marie/3            34.8M  11.2T  34.6M  -
storage/jaildata                  508M  11.2T   192K  /mnt/storage/jaildata
storage/jaildata/ansible-master   192K  4.00G   192K  /mnt/storage/jaildata/ansible-master
storage/jaildata/gogs             507M   256G   192K  /mnt/storage/jaildata/gogs
storage/jaildata/gogs/data        283M   256G   283M  /mnt/storage/jaildata/gogs/data
storage/jaildata/gogs/sql         225M   256G   225M  /mnt/storage/jaildata/gogs/sql
storage/jails                    2.55G   125G   208K  /usr/jails
storage/jails/ansible-master      472M  3.54G   472M  /usr/jails/ansible-master
storage/jails/gogs                750M  7.27G   750M  /usr/jails/gogs
storage/jails/samba               635M  1.38G   635M  /usr/jails/samba
storage/jails/zabbix              758M  7.26G   758M  /usr/jails/zabbix


More information about the freebsd-fs mailing list