From nobody Wed Apr 06 18:10:20 2022 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id BD9D11A37675; Wed, 6 Apr 2022 18:10:55 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4KYXct5Ypzz3MfF; Wed, 6 Apr 2022 18:10:54 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221] (may be forged)) by hz.grosbein.net (8.16.1/8.16.1) with ESMTPS id 236IAp5j058598 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 6 Apr 2022 18:10:52 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: egoitz@ramattack.net Received: from [10.58.0.11] (dadvw [10.58.0.11] (may be forged)) by eg.sd.rdtc.ru (8.16.1/8.16.1) with ESMTPS id 236IAPF0092951 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Thu, 7 Apr 2022 01:10:50 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: {* 05.00 *}Re: Desperate with 870 QVO and ZFS To: egoitz@ramattack.net References: <4e98275152e23141eae40dbe7ba5571f@ramattack.net> <665236B1-8F61-4B0E-BD9B-7B501B8BD617@ultra-secure.de> <0ef282aee34b441f1991334e2edbcaec@ramattack.net> <28e11d7ec0ac5dbea45f9f271fc28f06@ramattack.net> <7aa95cb4bf1fd38b3fce93bc26826042@ramattack.net> Cc: freebsd-fs@freebsd.org, freebsd-hackers@freebsd.org, Freebsd performance From: Eugene Grosbein Message-ID: Date: Thu, 7 Apr 2022 01:10:20 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 In-Reply-To: <7aa95cb4bf1fd38b3fce93bc26826042@ramattack.net> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,SHORTCIRCUIT autolearn=disabled version=3.4.2 X-Spam-Report: * -0.0 SHORTCIRCUIT No description available. * -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 4KYXct5Ypzz3MfF X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=fail (mx1.freebsd.org: domain of eugen@grosbein.net does not designate 2a01:4f8:c2c:26d8::2 as permitted sender) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-2.10 / 15.00]; ARC_NA(0.00)[]; R_SPF_FAIL(1.00)[-all]; FREEFALL_USER(0.00)[eugen]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MLMMJ_DEST(0.00)[freebsd-performance,freebsd-hackers,freebsd-fs]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/32, country:DE]; RCVD_TLS_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-ThisMailContainsUnwantedMimeParts: N 06.04.2022 23:51, egoitz@ramattack.net wrote: > About your recommendations... Eugene, if some of them wouldn't be working as expected, > could we revert some or all of them Yes, it all can be reverted. Just write down original sysctl values if you are going to change it. > 1) Make sure the pool has enough free space because ZFS can became crawling slow otherwise. > > *This is just an example... but you can see all similarly....* > > *zpool list* > *NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT* > *zroot 448G 2.27G 446G - - 1% 0% 1.00x ONLINE -* > *mail_dataset 58.2T 19.4T 38.8T - - 32% 33% 1.00x ONLINE -* It's all right. > 2) Increase recordsize upto 1MB for file systems located in the pool > so ZFS is allowed to use bigger request sizes for read/write operations > > *We have the default... so 128K...* It will not hurt increasing it upto 1MB. > 5) If you have good power supply and stable (non-crashing) OS, try increasing > sysctl vfs.zfs.txg.timeout from defaule 5sec, but do not be extreme (f.e. upto 10sec). > Maybe it will increase amount of long writes and decrease amount of short writes, that is good. > > *Well I have sync in disabled in the datasets... do you still think it's good to change it? Yes, try it. Disabling sync makes sense if you have lots of fsync() operations but other small writes are not affected unless you raise vfs.zfs.txg.timeout > *What about the vfs.zfs.dirty_data_max and the vfs.zfs.dirty_data_max_max, would you increase them from 4GB it's set now?.* Never tried that and cannot tell.