From nobody Thu Apr 21 20:27:45 2022 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 9BA6D1A0EAF9 for ; Thu, 21 Apr 2022 20:27:56 +0000 (UTC) (envelope-from mikej@paymentallianceintl.com) Received: from us-smtp-delivery-197.mimecast.com (us-smtp-delivery-197.mimecast.com [170.10.133.197]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.mimecast.com", Issuer "DigiCert TLS RSA SHA256 2020 CA1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Kkpy36ZNzz3hNx for ; Thu, 21 Apr 2022 20:27:55 +0000 (UTC) (envelope-from mikej@paymentallianceintl.com) Received: from MAIL-HUB.pai.local (175.158.26.216.gopai.com [216.26.158.175]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id us-mta-110-0h67UqB1M56WaGdQEJ7xng-1; Thu, 21 Apr 2022 16:27:46 -0400 X-MC-Unique: 0h67UqB1M56WaGdQEJ7xng-1 Received: from MAIL-HUB.pai.local (10.10.0.250) by MAIL-HUB.pai.local (10.10.0.250) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Thu, 21 Apr 2022 16:27:45 -0400 Received: from MAIL-HUB.pai.local ([fe80::a02e:93c2:c16a:6af8]) by MAIL-HUB.pai.local ([fe80::a02e:93c2:c16a:6af8%15]) with mapi id 15.00.1497.033; Thu, 21 Apr 2022 16:27:45 -0400 From: Michael Jung To: freebsd-fs Subject: ZFS reservations for type=volume Thread-Topic: ZFS reservations for type=volume Thread-Index: AdhVu5cQcIQitTxvR6aGtLet26Jbwg== Date: Thu, 21 Apr 2022 20:27:45 +0000 Message-ID: <94c10113c3964b1bbd3f4ddf934c23cd@MAIL-HUB.pai.local> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [10.250.0.59] x-c2processedorg: 474f336e-f930-49ec-9717-e3226b5b6e6e List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: paymentallianceintl.com Content-Language: en-US Content-Type: multipart/alternative; boundary="_000_94c10113c3964b1bbd3f4ddf934c23cdMAILHUBpailocal_" X-Rspamd-Queue-Id: 4Kkpy36ZNzz3hNx X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=paymentallianceintl.com; spf=pass (mx1.freebsd.org: domain of mikej@paymentallianceintl.com designates 170.10.133.197 as permitted sender) smtp.mailfrom=mikej@paymentallianceintl.com X-Spamd-Result: default: False [-2.42 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998]; HAS_XOIP(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:170.10.133.0/24]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; NEURAL_SPAM_SHORT(0.48)[0.477]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[4]; TO_DN_ALL(0.00)[]; DMARC_POLICY_ALLOW(-0.50)[paymentallianceintl.com,none]; MLMMJ_DEST(0.00)[freebsd-fs]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:30031, ipnet:170.10.132.0/23, country:US]; RCVD_TLS_LAST(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[170.10.133.197:from] X-ThisMailContainsUnwantedMimeParts: N --_000_94c10113c3964b1bbd3f4ddf934c23cdMAILHUBpailocal_ Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I have a zfs block dataset raid-5400-1/esxi-store1 that I share as an iscsi= target and that works great. I have set a reservation on that block device equal to its size so that is not sparse and thus while= I could over provision guests on the provided LUN, the storage presented as the LUN capacity will always be available. At lea= st this is what I want to achieve. What I find strange is that the reservation does not seem to be applied to = the ZFS pool 'raid-5400-1'. Do you really need to set your maximum reservation at the pool level, and then apply reservati= ons to all datasets on that volume? And if so I would assume you could never set reservations for datasets totaling more = than what was reserved for the pool 'raid-5400-1'. I could build out a test environment and figure out constraints but I'd rea= lly like to know the "how it is supposed to work" not the "how I find it to work". Thanks in advance. FreeBSD 14.0-CURRENT #4 main-n253875-8e72f458c6d: (this is a raidz2 pool - not my draid pool) root@draid:/usr/src/contrib/bearssl # zfs list -o name,type,used,avail,rati= o,compression,reservation,refreservation,volsize raid-5400-1 NAME TYPE USED AVAIL RATIO COMPRESS RESERV REFRESERV = VOLSIZE raid-5400-1 filesystem 18.0T 2.08T 1.36x on none = none - <- no reservation @pool root@draid:/usr/src/contrib/bearssl # root@draid:/usr/src/contrib/bearssl # zfs list -o name,type,used,avail,rati= o,compression,reservation,refreservation,volsize raid-5400-1/esxi-store1 NAME TYPE USED AVAIL RATIO COMPRESS RESERV = REFRESERV VOLSIZE raid-5400-1/esxi-store1 volume 16.9T 18.5T 1.78x zstd 15.6= T 16.9T 15.6T <- reservation @dataset root@draid:/usr/src/contrib/bearssl # zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP = HEALTH ALTROOT ccache 9.50G 9.10G 406M - - 88% 95% 1.00x = ONLINE - raid-5400-1 24.6T 1.52T 23.1T - - 2% 6% 1.00x = ONLINE - <- Free does not reflect reservation @pool tank 18.5T 605G 17.9T - - 0% 3% 1.00x = ONLINE - zfsroot 103G 33.3G 69.7G - - 31% 32% 1.00x = ONLINE - root@draid:/usr/src/contrib/bearssl # CONFIDENTIALITY NOTE: This message is intended only for the use of the individual or entity to whom it is addressed and may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, please notify us by telephone at (502) 212-4000 or notify us at: PAI, Dept. 99, 2101 High Wickham Place, Suite 101, Louisville, KY 40245 Disclaimer The information contained in this communication from the sender is confiden= tial. It is intended solely for use by the recipient and others authorized = to receive it. If you are not the recipient, you are hereby notified that a= ny disclosure, copying, distribution or taking action in relation of the co= ntents of this information is strictly prohibited and may be unlawful. This email has been scanned for viruses and malware, and may have been auto= matically archived by Mimecast, a leader in email security and cyber resili= ence. Mimecast integrates email defenses with brand protection, security aw= areness training, web security, compliance and other essential capabilities= . Mimecast helps protect large and small organizations from malicious activ= ity, human error and technology failure; and to lead the movement toward bu= ilding a more resilient world. To find out more, visit our website. --_000_94c10113c3964b1bbd3f4ddf934c23cdMAILHUBpailocal_ Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <= /head>

I have a zfs block dataset raid-5400-1/esxi-store1 t= hat I share as an iscsi target and that works great.  I have set a res= ervation

on that block device equal to its size so that is no= t sparse and thus while I could over provision guests on the provided LUN,<= o:p>

the storage presented as the LUN capacity will alway= s be available.  At least this is what I want to achieve.

 

What I find strange is that the reservation does not= seem to be applied to the ZFS pool ‘raid-5400-1’.  Do you= really need

to set your maximum reservation at the pool level, a= nd then apply reservations to all datasets on that volume?  And if so<= o:p>

I would assume you could never set reservations for = datasets totaling more than what was reserved for the pool ‘raid-5400= -1’.

 

I could build out a test environment and figure out = constraints but I’d really like to know the “how it is supposed= to work”

not the “how I find it to work”.

 

Thanks in advance.

 

FreeBSD 14.0-CURRENT #4 main-n253875-8e72f458c6d:

 

(this is a raidz2 pool – not my draid pool)

 

 

root@draid:/usr/src/contrib/bearssl # zfs list -o na= me,type,used,avail,ratio,compression,reservation,refreservation,volsize rai= d-5400-1

NAME        = TYPE   USED  AVAIL  RATIO  COMPRESS  &n= bsp;     RESERV  REFRESERV  VOLSIZE

raid-5400-1  filesystem  18.0T  2.08T=   1.36x  on          = ;      none     &nbs= p; none        -    =            <- no reser= vation @pool

root@draid:/usr/src/contrib/bearssl #

 

root@draid:/usr/src/contrib/bearssl # zfs list -o na= me,type,used,avail,ratio,compression,reservation,refreservation,volsize rai= d-5400-1/esxi-store1

NAME        =              TY= PE   USED  AVAIL  RATIO  COMPRESS   = ;     RESERV  REFRESERV  VOLSIZE

raid-5400-1/esxi-store1  volume  16.9T&nbs= p; 18.5T  1.78x  zstd       &n= bsp;     15.6T      16.9T = ;   15.6T        <- reserva= tion @dataset

 

root@draid:/usr/src/contrib/bearssl # zpool list

NAME        =   SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ =   FRAG    CAP  DEDUP    HEALTH = ; ALTROOT

ccache       9.50G&nbs= p; 9.10G   406M        - =         -    88% &nb= sp;  95%  1.00x    ONLINE  -

raid-5400-1  24.6T  1.52T  23.1T = ;       -      =    -     2%     6%  = 1.00x    ONLINE  -      &= nbsp;           &nbs= p;  <- Free does not reflect reservation @pool

tank        = 18.5T   605G  17.9T      &nbs= p; -         -   &nb= sp; 0%     3%  1.00x    ONLINE = ; -

zfsroot       103G&nbs= p; 33.3G  69.7G        -  = ;       -    31%   &= nbsp;32%  1.00x    ONLINE  -

root@draid:/usr/src/contrib/bearssl #

 

 

CONFIDENTIALITY NOTE: This message is intended only for the use
of the individual or entity to whom it is addressed and may
contain information that is privileged, confidential, and
exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby
notified that any dissemination, distribution or copying
of this communication is strictly prohibited. If you have
received this transmission in error, please notify us by
telephone at (502) 212-4000 or notify us at: PAI, Dept. 99,
2101 High Wickham Place, Suite 101, Louisville, KY 40245





<= b>Disclaimer

The information contained in this communication from the sender i= s confidential. It is intended solely for use by the recipient and others a= uthorized to receive it. If you are not the recipient, you are hereby notif= ied that any disclosure, copying, distribution or taking action in relation= of the contents of this information is strictly prohibited and may be unla= wful.

This email has been scanned for viruses and malware, and may h= ave been automatically archived by Mimecast, a leader in email security and= cyber resilience. Mimecast integrates email defenses with brand protection= , security awareness training, web security, compliance and other essential= capabilities. Mimecast helps protect large and small organizations from ma= licious activity, human error and technology failure; and to lead the movem= ent toward building a more resilient world. To find out more, visit our web= site.

--_000_94c10113c3964b1bbd3f4ddf934c23cdMAILHUBpailocal_--