ZFS: Mount partition from ZVOL with volmode=dev

[Paul Chakravarti]

>On 26/12/2014 23:54, Paul Chakravarti wrote:
>> Hello,
>>
>> I am using a ZVOL configured with 'volmode=dev' as the virtio disk device for
>> a bhyve instance (which works fine) but was trying to workout whether there
>> was any way of mounting the underlying partitions on the host system - the
>> partitions don’t show up under /dev/zvol as separate devices with
>> 'volmode=dev' so was wondering is there is any other way of getting at these
>> other than mounting in a bhyve instance?
>>
>> Thanks, Paul

> I cant reproduce this on HEAD r276067
>
> zfs create -V 8192 -o volmode=dev tank/tvol
> root at head:src> ls -l /dev/zvol/tank/
> total 0
> crw-r-----  1 root  operator  0x85 Dec 27 00:08 tvol
>
>      Regards
>      Steve

Hi,

Sorry - I should have been clearer. The zvol shows up on the host
system but the partitions aren’t exposed to geom.

On the host system:

    # zfs create -V10G -o volmode=dev tank/vps/vm0

    # zfs list -o name,used,volmode tank/vps/vm0
    NAME           USED  VOLMODE
    tank/vps/vm0  10.3G      dev

    # ls -l /dev/zvol/tank/vps/
    total 0
    crw-r-----  1 root  operator  0x7b Dec 27 13:30 vm0

The zvol mounted on the bhyve guest:

    # bhyveload -c /dev/nmdm0A -m 512M  -d /dev/zvol/tank/vps/vm0 vm0
    # bhyve -c 2 -m 512M -A -H -P -s 0:0,hostbridge -s 1:0,virtio-net,tap0 -s 2:0,lpc -s 3:0,virtio-blk,/dev/zvol/tank/vps/vm0 -l com1,/dev/nmdm0A vm0

On the bhyve guest this shows up as a geom device:

    root at vm0:~ # geom disk list
    Geom name: vtbd0
    Providers:
    1. Name: vtbd0
       Mediasize: 10737418240 (10G)
       Sectorsize: 512
       Mode: r2w2e3
       descr: (null)
       ident: BHYVE-747A-2A76-FAC
       fwsectors: 0
       fwheads: 0

And is partitioned in the guest as follows:

    root at vm0:~ # gpart show -p
    =>      34  20971453    vtbd0  GPT  (10G)
            34      1024  vtbd0p1  freebsd-boot  (512K)
          1058  19919872  vtbd0p2  freebsd-ufs  (9.5G)
      19920930   1048576  vtbd0p3  freebsd-swap  (512M)
      20969506      1981           - free -  (991K)

What I am trying to work out is whether there is any way I can mount the guest 
UFS partition on the host with volmode=guest - with volmode=default (ie. geom
when vfs.zfs.vol.mode=1) the device does show up as a geom provider and you can
just mount the partition from /dev/vol directly. The ZFS man page suggests that
you can’t but given that you can clearly mount from within a VM was wondering
is there is any way round this on the host (I am trying to clone a disk device
to run multiple bhyve instances but want to mount and modify some of the 
rc.conf parameters before passing to bhyve)

     volmode=default | geom | dev | none
         This property specifies how volumes should be exposed to the OS.
         Setting it to geom exposes volumes as geom(4) providers, providing
         maximal functionality.  Setting it to dev exposes volumes only as
         cdev device in devfs.  Such volumes can be accessed only as raw disk
         device files, i.e. they can not be partitioned, mounted, participate
         in RAIDs, etc, but they are faster, and in some use scenarios with
         untrusted consumer, such as NAS or VM storage, can be more safe.
         Volumes with property set to none are not exposed outside ZFS, but
         can be snapshoted, cloned, replicated, etc, that can be suitable for
         backup purposes.  Value default means that volumes exposition is con-
         trolled by system-wide sysctl/tunable vfs.zfs.vol.mode, where geom,
         dev and none are encoded as 1, 2 and 3 respectively.  The default
         values is geom.  This property can be changed any time, but so far it
         is processed only during volume creation and pool import.

Using volmode=default (geom - vfs.zfs.vol.mode=1) causes the installer to fail
when you try to create a UFS filesystem under bhyve - it is possible to get
round this by creating the partitions manually but my preference would be to
use volmode=dev.

Paul