Releasing deleted blocks from a sparse ZVOL used by bhyve guest

Paul Esson paul.esson at
Mon May 14 15:33:53 UTC 2018

Hi Folks,

I'm attempting to use a sparse ZVOL from a FreeBSD 11.1-RELEASE host as a data disk for a bhyve FreeBSD 11.1-RELEASE guest that in turn uses this device as the source for a ZFS filesystem.  When I delete data from the filesystem on the guest this is reflected in its used space, but not in the ZVOL on the host.  I've read suggestions that the use of a block driver that supports TRIM or SCSi unmap is crucial such as virtio-scsi, but from what I can tell my vm-bhyve implementation is using virtio - certainly the disk1_type parameter in the configuration file has a value of "virtio-blk".

Can anyone confirm this is indeed the issue and whether it is possible to switch to the relevant driver?  Or am I barking up the wrong tree altogether?

Host ZFS Filesystem

NAME       PROPERTY              VALUE                   SOURCE
dc1-hn-01  type                  filesystem              -
dc1-hn-01  creation              Mon Apr 23 14:35 2018   -
dc1-hn-01  used                  36.7G                   -
dc1-hn-01  available             78.2T                   -
dc1-hn-01  referenced            201K                    -
dc1-hn-01  compressratio         1.00x                   -
dc1-hn-01  mounted               yes                     -
dc1-hn-01  quota                 none                    default
dc1-hn-01  reservation           none                    default
dc1-hn-01  recordsize            128K                    default
dc1-hn-01  mountpoint            /export/data/dc1-hn-01  local
dc1-hn-01  sharenfs              off                     default
dc1-hn-01  checksum              on                      default
dc1-hn-01  compression           off                     default
dc1-hn-01  atime                 on                      default
dc1-hn-01  devices               on                      default
dc1-hn-01  exec                  on                      default
dc1-hn-01  setuid                on                      default
dc1-hn-01  readonly              off                     default
dc1-hn-01  jailed                off                     default
dc1-hn-01  snapdir               hidden                  default
dc1-hn-01  aclmode               discard                 default
dc1-hn-01  aclinherit            restricted              default
dc1-hn-01  canmount              on                      default
dc1-hn-01  xattr                 off                     temporary
dc1-hn-01  copies                1                       default
dc1-hn-01  version               5                       -
dc1-hn-01  utf8only              off                     -
dc1-hn-01  normalization         none                    -
dc1-hn-01  casesensitivity       sensitive               -
dc1-hn-01  vscan                 off                     default
dc1-hn-01  nbmand                off                     default
dc1-hn-01  sharesmb              off                     default
dc1-hn-01  refquota              none                    default
dc1-hn-01  refreservation        none                    default
dc1-hn-01  primarycache          all                     default
dc1-hn-01  secondarycache        all                     default
dc1-hn-01  usedbysnapshots       0                       -
dc1-hn-01  usedbydataset         201K                    -
dc1-hn-01  usedbychildren        36.7G                   -
dc1-hn-01  usedbyrefreservation  0                       -
dc1-hn-01  logbias               latency                 default
dc1-hn-01  dedup                 off                     default
dc1-hn-01  mlslabel                                      -
dc1-hn-01  sync                  standard                default
dc1-hn-01  refcompressratio      1.00x                   -
dc1-hn-01  written               201K                    -
dc1-hn-01  logicalused           4.12G                   -
dc1-hn-01  logicalreferenced     36.5K                   -
dc1-hn-01  volmode               default                 default
dc1-hn-01  filesystem_limit      none                    default
dc1-hn-01  snapshot_limit        none                    default
dc1-hn-01  filesystem_count      none                    default
dc1-hn-01  snapshot_count        none                    default
dc1-hn-01  redundant_metadata    all                     default

Host ZFS volume - used by vm-bhyve guest

NAME                               PROPERTY              VALUE                  SOURCE
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  type                  volume                 -
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  creation              Wed May  2 14:06 2018  -
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  used                  4.70G                  -
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  available             78.2T                  -
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  referenced            4.70G                  -
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  compressratio         1.00x                  -
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  reservation           none                   default
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  volsize               28T                    local
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  volblocksize          8K                     -
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  checksum              on                     default
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  compression           off                    default
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  readonly              off                    default
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  copies                1                      default
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  refreservation        none                   default
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  primarycache          all                    default
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  secondarycache        all                    default
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  usedbysnapshots       0                      -
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  usedbydataset         4.70G                  -
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  usedbychildren        0                      -
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  usedbyrefreservation  0                      -
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  logbias               latency                default
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  dedup                 off                    default
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  mlslabel                                     -
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  sync                  standard               default
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  refcompressratio      1.00x                  -
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  written               4.70G                  -
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  logicalused           2.06G                  -
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  logicalreferenced     2.06G                  -
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  volmode               dev                    local
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  snapshot_limit        none                   default
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  snapshot_count        none                   default
dc1-hn-01/vm/dc1-olbp-sn-11/disk1  redundant_metadata    all                    default

Guest ZFS Filesystem

NAME       PROPERTY              VALUE                   SOURCE
dc1-sn-11  type                  filesystem              -
dc1-sn-11  creation              Wed May  2 15:49 2018   -
dc1-sn-11  used                  752K                    -
dc1-sn-11  available             26.9T                   -
dc1-sn-11  referenced            176K                    -
dc1-sn-11  compressratio         1.00x                   -
dc1-sn-11  mounted               yes                     -
dc1-sn-11  quota                 none                    default
dc1-sn-11  reservation           none                    default
dc1-sn-11  recordsize            128K                    default
dc1-sn-11  mountpoint            /export/data/dc1-sn-11  local
dc1-sn-11  sharenfs              off                     default
dc1-sn-11  checksum              on                      default
dc1-sn-11  compression           off                     default
dc1-sn-11  atime                 on                      default
dc1-sn-11  devices               on                      default
dc1-sn-11  exec                  on                      default
dc1-sn-11  setuid                on                      default
dc1-sn-11  readonly              off                     default
dc1-sn-11  jailed                off                     default
dc1-sn-11  snapdir               hidden                  default
dc1-sn-11  aclmode               discard                 default
dc1-sn-11  aclinherit            restricted              default
dc1-sn-11  canmount              on                      default
dc1-sn-11  xattr                 off                     temporary
dc1-sn-11  copies                1                       default
dc1-sn-11  version               5                       -
dc1-sn-11  utf8only              off                     -
dc1-sn-11  normalization         none                    -
dc1-sn-11  casesensitivity       sensitive               -
dc1-sn-11  vscan                 off                     default
dc1-sn-11  nbmand                off                     default
dc1-sn-11  sharesmb              off                     default
dc1-sn-11  refquota              none                    default
dc1-sn-11  refreservation        none                    default
dc1-sn-11  primarycache          all                     default
dc1-sn-11  secondarycache        all                     default
dc1-sn-11  usedbysnapshots       0                       -
dc1-sn-11  usedbydataset         176K                    -
dc1-sn-11  usedbychildren        576K                    -
dc1-sn-11  usedbyrefreservation  0                       -
dc1-sn-11  logbias               latency                 default
dc1-sn-11  dedup                 off                     default
dc1-sn-11  mlslabel                                      -
dc1-sn-11  sync                  standard                default
dc1-sn-11  refcompressratio      1.00x                   -
dc1-sn-11  written               176K                    -
dc1-sn-11  logicalused           153K                    -
dc1-sn-11  logicalreferenced     55K                     -
dc1-sn-11  volmode               default                 default
dc1-sn-11  filesystem_limit      none                    default
dc1-sn-11  snapshot_limit        none                    default
dc1-sn-11  filesystem_count      none                    default
dc1-sn-11  snapshot_count        none                    default
dc1-sn-11  redundant_metadata    all                     default


Paul Esson  |  Redstor Limited
t  +44 (0)118 951 5235  |   m  +44 (0)776 690 6514
e  paul.esson at<mailto:paul.esson at><>
[cid:image001.jpg at 01D3EB9E.9D818E70]<>

More information about the freebsd-virtualization mailing list