From nobody Sat Aug 28 02:03:05 2021 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 91DE6178476B for ; Sat, 28 Aug 2021 02:03:32 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vtr.rulingia.com (vtr.rulingia.com [IPv6:2001:19f0:5801:ebe:5400:1ff:fe53:30fd]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA512 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "vtr.rulingia.com", Issuer "R3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GxKcg0vBpz4TQV; Sat, 28 Aug 2021 02:03:30 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from server.rulingia.com (2001-44b8-31fc-0d00-11e8-f292-ae6b-537a.static.ipv6.internode.on.net [IPv6:2001:44b8:31fc:d00:11e8:f292:ae6b:537a]) by vtr.rulingia.com (8.16.1/8.15.2) with ESMTPS id 17S23BX5084633 (version=TLSv1.3 cipher=AEAD-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 28 Aug 2021 12:03:17 +1000 (AEST) (envelope-from peter@rulingia.com) DKIM-Filter: OpenDKIM Filter v2.10.3 vtr.rulingia.com 17S23BX5084633 X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.16.1/8.16.1) with ESMTPS id 17S235TP090583 (version=TLSv1.3 cipher=AEAD-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 28 Aug 2021 12:03:06 +1000 (AEST) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.16.1/8.16.1/Submit) id 17S235vQ090582; Sat, 28 Aug 2021 12:03:05 +1000 (AEST) (envelope-from peter) Date: Sat, 28 Aug 2021 12:03:05 +1000 From: Peter Jeremy To: Alan Somers Cc: George Michaelson , Ben RUBSON , freebsd-fs Subject: Re: ZFS on high-latency devices Message-ID: References: <023225AD-2A97-47C5-9FE4-3ABF1BFD66F1@gmx.com> List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="R5DOk3Sae8nJZlGU" Content-Disposition: inline In-Reply-To: X-PGP-Key: http://www.rulingia.com/keys/peter.pgp X-Rspamd-Queue-Id: 4GxKcg0vBpz4TQV X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=quarantine) header.from=rulingia.com; spf=pass (mx1.freebsd.org: domain of peter@rulingia.com designates 2001:19f0:5801:ebe:5400:1ff:fe53:30fd as permitted sender) smtp.mailfrom=peter@rulingia.com X-Spamd-Result: default: False [-5.90 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FREEFALL_USER(0.00)[peter]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MID_RHS_MATCH_FROMTLD(0.00)[]; DMARC_POLICY_ALLOW(-0.50)[rulingia.com,quarantine]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:20473, ipnet:2001:19f0:5800::/38, country:US]; FREEMAIL_CC(0.00)[algebras.org,gmx.com,freebsd.org]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N --R5DOk3Sae8nJZlGU Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2021-Aug-22 17:48:13 -0600, Alan Somers wrote: >mbuffer is not going to help the OP. I agree that mbuffer won't help. I already use something equivalent to remove the read latency on the send side. >And if I understand correctly, he's >connecting over a WAN, not a LAN. ZFS will never achieve decent >performance in such a setup. It's designed as a local file system, and >assumes it can quickly read metadata off of the disks at any time. Yes. But, at least with a relatively empty destination, zfs actually does almost no reads whilst doing a recv. As far as I can tell, the problem is that zfs does a complete flush of all data and metadata at snapshot boundaries. This is painful even with local filesystems (it typically takes >1s to recv an empty snapshot with local disks). >The >OP's best option is to go with "a": encrypt each dataset and send them with >"zfs send --raw". I don't know why he thinks that it would be "very >difficult". It's quite easy, if he doesn't care about old snapshots. Jus= t: I agree that "zfs send --raw" is the best solution to network RTT and I agree that migrating to ZFS native encryption is quite easy if you don't care about any ZFS features. However, I do care about old snapshots - migrating to ZFS native encryption is a non-starter if it involves throwing away all my old snapshots and clones. I have also been working on migrating to native encryption. I know how to migrate snapshots and think there a way to migrate clones (but I need to validate it). The remaining definite blocker is working out how to migrate the pool root filesystem (including snapshots). --=20 Peter Jeremy --R5DOk3Sae8nJZlGU Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEE7rKYbDBnHnTmXCJ+FqWXoOSiCzQFAmEpmVNfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEVF QjI5ODZDMzA2NzFFNzRFNjVDMjI3RTE2QTU5N0EwRTRBMjBCMzQACgkQFqWXoOSi CzRr7hAAmb3w4XGMgHuPi97RlWZcYahErnCaYB+ROGs2pJLRUVJQyTDGxBQqVVJ8 AEiNYtMwdCqtj9WI42klYuJQB5tO4IhByKulcDs8oqzeUSJW2IVeqRiMfqxJMyuk 74XHBLiDCZyy6+U4d/Bh6YIWQZpXsZXNUFJ+GcKy/1Y0ciyPAjeugRXzxpj5+Wg3 PaXC2zOnlipWSbHI9966JsKn/2yIXIsPQadfIhBy0KsdlXGdRq2mGVkgmaXpLBJD nxdN+fY/6ighFRuc4orh2RY4HOIpRPnWGYpG4MBAjTB2CskrgpSRxms2vYsoOaYq hJOe6t5BjWehh+cC3aq86Z9M15ceaogIk+D9D6okOOsOUBJfCfDUoNmlzAhZAZpp YEfp1tiToPKvBtccWAsIs8qoxp9aaChuwl6qxpELIg6e/U/FdpWV2R4xJYuzUIW5 CW8ArFLm28yRYSNvfNiylJQrwK9dZSUSWTGD4mUlU4Nt3/Sil5nAsvdZ1i530lNZ PUWuxlqO47sa2bUamvP/Tua6WOxhF0boNMaX6d4iPuf/X+S8yhOBbl/uvYAFD7Di ZKKbdA8yjzf46mLVeZy0UFytkf61InDDlkGIiXaQA+RNCqu4Tm8mGI6vEh0C3EMt SwcmG+ooxWJTfdIF1lTDHcGrHho8zklbBgQJWWD0e+NNagEMcHg= =hLr9 -----END PGP SIGNATURE----- --R5DOk3Sae8nJZlGU--