Re: Setting up ZFS L2ARC on a zvol
- Reply: julio_a_meroh.net: "Re: Setting up ZFS L2ARC on a zvol"
- In reply to: julio_a_meroh.net: "Setting up ZFS L2ARC on a zvol"
- Go to: [ bottom of page ] [ top of archives ] [ this month ]
Date: Thu, 27 Oct 2022 01:46:59 UTC
On 10/26/22 06:32, julio@meroh.net wrote: > Hello, > > I'm setting up a new machine in which I have an NVMe drive and a bunch of hard disks. The hard disks are going to be a ZFS pool and the NVMe drive is going to be the root file system + the L2ARC for the pool. > > Now... I'm considering using ZFS as well for the root file system (as a separate single-drive pool) in order to simplify admin operations: I want to host some VMs on the NVMe for speed, and using zvols will be very helpful as I don't have to come up with the partition sizes upfront. > > And here comes the question: can the L2ARC of the hard disk pool be backed by a zvol on the NVMe pool (again, so that I don't have to use fixed-size partitions)? > > I gave a simple try to this setup and it's not working, so I'm wondering if this is just not a good idea and thus is unsupported, or if there is a bug: > > root@think:~ # zfs create -V 16G -o primarycache=none zroot/l2arg > root@think:~ # zpool add scratch cache zvol/zroot/l2arc > cannot add to 'scratch': no such pool or dataset > root@think:~ # Oct 26 05:45:28 think ZFS[3677]: vdev problem, zpool=scratch path=/dev/zvol/zroot/l2arc type=ereport.fs.zfs.vdev.open_failed > root@think:~ # ls /dev/zvol/zroot/l2arc > /dev/zvol/zroot/l2arc > > Thanks! Testing on a VirtualBox VM, it looks like you cannot use a ZFS volume as a cache device for another ZFS pool: 2022-10-26 17:47:11 toor@vf1 ~ # freebsd-version ; uname -a 12.3-RELEASE-p7 FreeBSD vf1.tracy.holgerdanske.com 12.3-RELEASE-p6 FreeBSD 12.3-RELEASE-p6 GENERIC amd64 2022-10-26 17:47:32 toor@vf1 ~ # camcontrol devlist <VBOX HARDDISK 1.0> at scbus0 target 0 lun 0 (pass0,ada0) <VBOX HARDDISK 1.0> at scbus0 target 1 lun 0 (pass1,ada1) <VBOX HARDDISK 1.0> at scbus1 target 0 lun 0 (pass2,ada2) 2022-10-26 18:02:58 toor@vf1 ~ # zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT vf1_zroot 3.75G 2.67G 1.08G - - 43% 71% 1.00x ONLINE - vf1zpool1 960M 3.15M 957M - - 8% 0% 1.00x ONLINE - vf1zpool2 960M 2.31M 958M - - 14% 0% 1.00x ONLINE - 2022-10-26 17:52:26 toor@vf1 ~ # zfs create -V 128M vf1_zroot/myvol 2022-10-26 17:55:06 toor@vf1 ~ # zfs list -d 1 vf1_zroot NAME USED AVAIL REFER MOUNTPOINT vf1_zroot 2.80G 840M 88K /vf1_zroot vf1_zroot/ROOT 2.65G 840M 88K none vf1_zroot/myvol 134M 974M 56K - vf1_zroot/tmp 644K 840M 644K /tmp vf1_zroot/usr 16.3M 840M 88K /usr vf1_zroot/var 1.51M 840M 88K /var 2022-10-26 17:58:04 toor@vf1 ~ # find /dev -name myvol /dev/zvol/vf1_zroot/myvol 2022-10-26 17:58:14 toor@vf1 ~ # ll /dev/zvol/vf1_zroot/myvol crw-r----- 1 root operator 0x67 2022/10/26 17:55:06 /dev/zvol/vf1_zroot/myvol 2022-10-26 17:58:45 toor@vf1 ~ # zpool add vf1zpool1 cache zvol/vf1_zroot/myvol cannot add to 'vf1zpool1': no such pool or dataset 2022-10-26 17:59:15 toor@vf1 ~ # zpool add vf1zpool1 cache /dev/zvol/vf1_zroot/myvol cannot add to 'vf1zpool1': no such pool or dataset Assuming you move root to another device and can repurpose the NVMe, perhaps gvinum(8) would meet your needs. David