From nobody Thu Jun 09 19:20:04 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 6FCF9836DFA; Thu, 9 Jun 2022 19:20:04 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LJv782jpKz3C7S; Thu, 9 Jun 2022 19:20:04 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1654802404; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Y38sNFlULZcovIJrr2tbkovGY8XrYimQb0bLfWAS7A0=; b=bqzT1GaxXyx+j/+ZTCCtXAKqcW1LdBmOQVJahwLzqbb5MNhuhbi7r0pka65jgk0fbIX4fS HHhOY/dceRdfWlTTp2oVmNbaVTEdmXNdathKnxmwkI0BRN75V36mdokhvNTz1VqpPGktXK VtQHe534YzHMgu5xlqM7/TuGthJpacM/jMQytX0KVQMJPBjPZjozZSBdWPaceyQq+/GECx 3B0SIl8nkgoF+aGpL3oybDr2Ja825TLSyAr6hwGSlMHl/3lwaLeiOfqUnniAZvuV28k6IF HanEUNVsTlSBX7dvoKY+qbviki9wzG+7ghn2MuqhKYungMuDYgeBvIS0sB2ZJQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3361718E7; Thu, 9 Jun 2022 19:20:04 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 259JK4vU017166; Thu, 9 Jun 2022 19:20:04 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 259JK4o0017163; Thu, 9 Jun 2022 19:20:04 GMT (envelope-from git) Date: Thu, 9 Jun 2022 19:20:04 GMT Message-Id: <202206091920.259JK4o0017163@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Toomas Soome Subject: git: e50e40684aa6 - main - loader: add support for gzip compression List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: tsoome X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: e50e40684aa61526327be713de512e0f9934477f Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1654802404; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Y38sNFlULZcovIJrr2tbkovGY8XrYimQb0bLfWAS7A0=; b=ZZzMfYOMfV4KKeiWqH0kGOzoopSK4uza4cjP+Z8dQwLulWSexbNyC+8JEj1ZwAAejRglns FQtsQUV7jrOtuKGa5CiDuw1/j10QTdIL0W2g10zGP8H7vAey+S0Yny5M+G8OQjs6onpqTC haHwDGqOlXhv26557WIWC4W2C/IY24Ulov8wvyTQrQM1EFFfHLzG2wBGq7CN1LgTT9ApJP vgPxiYgdK01tVCp79X0XqQgeCqJzeXgVp3TmvKETRvxxMrYKmcNG/pPbtBkr2ZvNXDKT6Z BeoomlPLX0zDr2yBbJBX6eyW2pVfJ0enJ2wjj+B7fnEjd5+MB180it4Q7aMcyA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1654802404; a=rsa-sha256; cv=none; b=giCocV1xiPw5SKd/jszTGg/p2fISoItqmIxvwNqynhA2joPdvWGo+JW1SAUyyOozXE2UkD DMZ4NMgfuX7fOiW0XJSWV/zui/PKlVmvpwHdGo6/ci53qzVxSvqHR/NeQVmTpx7gAbkN11 OC58JjDBMN52W25YxwHT2cvL3GzOcKi32XLgom42YaWUra7+8v/z5AclzSj8XlOBNb31m9 U8TkycqD2TTKke7q5XCc5hoDCNNeHDmMLu4wFxU4G7yzPayKc0S6hjUzEajLQFO3UvcMSr tLDZSVtQUVVlt2ytYgKT7oaJZPDXkqMysCMR9UJFFd3af3uNM420v0FrG0TJjA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by tsoome: URL: https://cgit.FreeBSD.org/src/commit/?id=e50e40684aa61526327be713de512e0f9934477f commit e50e40684aa61526327be713de512e0f9934477f Author: Toomas Soome AuthorDate: 2022-05-25 18:53:57 +0000 Commit: Toomas Soome CommitDate: 2022-06-09 17:54:30 +0000 loader: add support for gzip compression As we do have zlib code in loader, we should also support gzip compression in zfs. PR: 153173 Submitted by: Mikhail Zakharov Reviewed by: imp, markj, delphij Differential Revision: https://reviews.freebsd.org/D35320 MFC after: 1 month --- sys/cddl/boot/zfs/README | 1 + sys/cddl/boot/zfs/gzip.c | 96 +++++++++++++++++++++++++++++++++++++++++++++ sys/cddl/boot/zfs/zfssubr.c | 19 ++++----- 3 files changed, 107 insertions(+), 9 deletions(-) diff --git a/sys/cddl/boot/zfs/README b/sys/cddl/boot/zfs/README index 5f16d5cddf26..0c2006096c99 100644 --- a/sys/cddl/boot/zfs/README +++ b/sys/cddl/boot/zfs/README @@ -5,6 +5,7 @@ are used by the ZFS bootstrap: fletcher.c checksum support sha256.c checksum support + gzip.c compression support lz4.c compression support lzjb.c compression support blkptr.c ZFS embedded-data block pointers support diff --git a/sys/cddl/boot/zfs/gzip.c b/sys/cddl/boot/zfs/gzip.c new file mode 100644 index 000000000000..35b5a3241dc0 --- /dev/null +++ b/sys/cddl/boot/zfs/gzip.c @@ -0,0 +1,96 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright 2007 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +/* + * Portions Copyright 2022 Mikhail Zakharov + */ + +#include +#include + +static void * +zfs_zcalloc(void *opaque __unused, uint_t items, uint_t size) +{ + + return (calloc(items, size)); +} + +static void +zfs_zcfree(void *opaque __unused, void *ptr) +{ + free(ptr); +} + +/* + * Uncompress the buffer 'src' into the buffer 'dst'. The caller must store + * the expected decompressed data size externally so it can be passed in. + * The resulting decompressed size is then returned through dstlen. This + * function return Z_OK on success, or another error code on failure. + */ +static inline int +z_uncompress(void *dst, size_t *dstlen, const void *src, size_t srclen) +{ + z_stream zs; + int err; + + bzero(&zs, sizeof (zs)); + zs.next_in = (unsigned char *)src; + zs.avail_in = srclen; + zs.next_out = dst; + zs.avail_out = *dstlen; + zs.zalloc = zfs_zcalloc; + zs.zfree = zfs_zcfree; + + /* + * Call inflateInit2() specifying a window size of DEF_WBITS + * with the 6th bit set to indicate that the compression format + * type (zlib or gzip) should be automatically detected. + */ + if ((err = inflateInit2(&zs, DEF_WBITS | 0x20)) != Z_OK) + return (err); + + if ((err = inflate(&zs, Z_FINISH)) != Z_STREAM_END) { + (void) inflateEnd(&zs); + return (err == Z_OK ? Z_BUF_ERROR : err); + } + + *dstlen = zs.total_out; + return (inflateEnd(&zs)); +} + +static int +gzip_decompress(void *s_start, void *d_start, size_t s_len, size_t d_len, + int n __unused) +{ + size_t dstlen = d_len; + + ASSERT(d_len >= s_len); + + if (z_uncompress(d_start, &dstlen, s_start, s_len) != Z_OK) + return (-1); + + return (0); +} diff --git a/sys/cddl/boot/zfs/zfssubr.c b/sys/cddl/boot/zfs/zfssubr.c index 2603a69d6f9f..2e309fc10be4 100644 --- a/sys/cddl/boot/zfs/zfssubr.c +++ b/sys/cddl/boot/zfs/zfssubr.c @@ -163,6 +163,7 @@ typedef struct zio_compress_info { #include "lzjb.c" #include "zle.c" +#include "gzip.c" /* * Compression vectors. @@ -173,15 +174,15 @@ static zio_compress_info_t zio_compress_table[ZIO_COMPRESS_FUNCTIONS] = { {NULL, NULL, 0, "uncompressed"}, {NULL, lzjb_decompress, 0, "lzjb"}, {NULL, NULL, 0, "empty"}, - {NULL, NULL, 1, "gzip-1"}, - {NULL, NULL, 2, "gzip-2"}, - {NULL, NULL, 3, "gzip-3"}, - {NULL, NULL, 4, "gzip-4"}, - {NULL, NULL, 5, "gzip-5"}, - {NULL, NULL, 6, "gzip-6"}, - {NULL, NULL, 7, "gzip-7"}, - {NULL, NULL, 8, "gzip-8"}, - {NULL, NULL, 9, "gzip-9"}, + {NULL, gzip_decompress, 1, "gzip-1"}, + {NULL, gzip_decompress, 2, "gzip-2"}, + {NULL, gzip_decompress, 3, "gzip-3"}, + {NULL, gzip_decompress, 4, "gzip-4"}, + {NULL, gzip_decompress, 5, "gzip-5"}, + {NULL, gzip_decompress, 6, "gzip-6"}, + {NULL, gzip_decompress, 7, "gzip-7"}, + {NULL, gzip_decompress, 8, "gzip-8"}, + {NULL, gzip_decompress, 9, "gzip-9"}, {NULL, zle_decompress, 64, "zle"}, {NULL, lz4_decompress, 0, "lz4"}, #ifdef HAS_ZSTD_ZFS