From nobody Wed Nov 12 21:31:28 2025 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4d6Gmw5f4mz6GdSf; Wed, 12 Nov 2025 21:31:28 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R12" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4d6Gmw57tjz49N5; Wed, 12 Nov 2025 21:31:28 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1762983088; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=aLZRCfGuFx3AAj4LSXC1uzOtJYT2r4xDY3xaFWHW+3I=; b=pJPXrCRF4wiy3YfBJgd3pxI1QNp/a5MVKNg8cIBn8Mi9OBQ6viQ6LTgJEw78+a61p6ljaA d+g0eUFvumQW5FdR5Zb7CoCYdHnc1mnm3IjoOjzrBQxXAhUgwqpCjgPVLGtFwDTE0w6hal 8waI5ARVPkPjbTdxpxId1UH6d7yV7N3Sp7oxeNYoyPsP/vjPLk3bcRNjtoFSwkuPD/l40f 4TK7mGNiMOmbkl8OTXkhgaiH+QRNcqQaArMSxW7iv7ksUf2OIgvaIqmR63GpcAiOGQ9Unv +xN4F/BaY1CS8mf2Xh+R+/lLVCnkQ+EQwpAmgGbYX8wVP298wjBYcz1cczQZGA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1762983088; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=aLZRCfGuFx3AAj4LSXC1uzOtJYT2r4xDY3xaFWHW+3I=; b=fNa3QdMVSIMmOstebdJHe5CyPlwN9uu3sLf1bNvt7/FcViyacJ5O3gMX/IFh5gbO7ZLm1r o1eqi2JXXen4CybRC3ilJ7XzX4zu7rb31lZv5RQcIxh16vIf/jAVm2N9kbIe1o2Fhtgisy M34N8M8rxMDm7SmOujiQSVMGovZmSyv9OuxpejwuJfWj3nobG0DBfbhFKiBPQnRO7sG2Rp hzI6PzUn0P/W59cE4YxhoalbcNBkYrKfeyv2Zh/9SHYJkoFKMyptm3awwcsmB98uW+Gb0d IElbh2c9MbBfQEtJuAdryqhRUQ3WvJ019EKyKuVxYAHuAuGXq5oDObjqHiKZ0A== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1762983088; a=rsa-sha256; cv=none; b=qJbJ0B9ultD7tz0xmdi5WQnqvdsK5Vj4NV3SGAjgUporSbYEvv9/gTMWx2nN/jj0Nl2Lib F2IbvC3xF8+w28OD22q0ayS1bF+Ko6+OSdhXliwkB1rSG6y1qnZ+waev9WNOnnTtq5PURp VcOT6EuzNslV62cR/iUkNKunDsV5VHLIvuCaawZDOUIKJV4v5mWAr8kokGJ61AlCv38aYi RSlVuANIOPpZEPI6PJGAJXt/iYyp243p++Mt8xI+TwH8Tks9Ny8qQbyeVTqTLNNL09JxFS Q3X4qJqcKcerc2KLz9BLBW0xLU4afqaSHRvJU9dqaOhWavICBq6YsTWWweUJGA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4d6Gmw4bjnzs3K; Wed, 12 Nov 2025 21:31:28 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 5ACLVSOh016548; Wed, 12 Nov 2025 21:31:28 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 5ACLVSoI016545; Wed, 12 Nov 2025 21:31:28 GMT (envelope-from git) Date: Wed, 12 Nov 2025 21:31:28 GMT Message-Id: <202511122131.5ACLVSoI016545@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Dag-Erling =?utf-8?Q?Sm=C3=B8rgrav?= Subject: git: a34a57d4b4eb - main - tarfs: Fix support for large files List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: des X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: a34a57d4b4eba88dfa5541d7d77b63b01c1a1a9a Auto-Submitted: auto-generated The branch main has been updated by des: URL: https://cgit.FreeBSD.org/src/commit/?id=a34a57d4b4eba88dfa5541d7d77b63b01c1a1a9a commit a34a57d4b4eba88dfa5541d7d77b63b01c1a1a9a Author: Dag-Erling Smørgrav AuthorDate: 2025-11-12 21:23:49 +0000 Commit: Dag-Erling Smørgrav CommitDate: 2025-11-12 21:24:43 +0000 tarfs: Fix support for large files * When fast-forwarding through a zstd frame, we incorrectly used the min() inline function instead of the MIN() macro. The function truncates the result to unsigned int, resulting in a decompression error when trying to seek more than 4 GB into the frame. * POSIX states that a size extended header record overrides the size field in the header if present, and that one must be included if the size of the file exceeds 8 GB (the size field maxes out at 64 GB). * Reduce repetition in the exthdr parser by deduplicating the syntax error handler. MFC after: 1 week Sponsored by: Klara, Inc. Fixes: 69d94f4c7608 ("Add tarfs, a filesystem backed by tarballs.") Reviewed by: allanjude Differential Revision: https://reviews.freebsd.org/D53718 --- sys/fs/tarfs/tarfs_io.c | 2 +- sys/fs/tarfs/tarfs_vfsops.c | 48 +++++++++++++++++++--------------------- tests/sys/fs/tarfs/tarfs_test.sh | 27 ++++++++++++++++++++++ 3 files changed, 51 insertions(+), 26 deletions(-) diff --git a/sys/fs/tarfs/tarfs_io.c b/sys/fs/tarfs/tarfs_io.c index a3d8df62d7df..e250c5cbce5a 100644 --- a/sys/fs/tarfs/tarfs_io.c +++ b/sys/fs/tarfs/tarfs_io.c @@ -444,7 +444,7 @@ tarfs_zread_zstd(struct tarfs_zio *zio, struct uio *uiop) } if (zio->opos < off) { /* to be discarded */ - zob.size = min(off - zio->opos, len); + zob.size = MIN(off - zio->opos, len); zob.pos = 0; } else { zob.size = len; diff --git a/sys/fs/tarfs/tarfs_vfsops.c b/sys/fs/tarfs/tarfs_vfsops.c index 4cc70e4d5781..cae780cb71e5 100644 --- a/sys/fs/tarfs/tarfs_vfsops.c +++ b/sys/fs/tarfs/tarfs_vfsops.c @@ -441,7 +441,7 @@ tarfs_alloc_one(struct tarfs_mount *tmp, size_t *blknump) int endmarker = 0; char *namep, *sep; struct tarfs_node *parent, *tnp, *other; - size_t namelen = 0, linklen = 0, realsize = 0, sz; + size_t namelen = 0, linklen = 0, realsize = 0, extsize = 0, sz; ssize_t res; dev_t rdev; gid_t gid; @@ -588,10 +588,7 @@ again: char *eol, *key, *value, *sep; size_t len = strtoul(line, &sep, 10); if (len == 0 || sep == line || *sep != ' ') { - TARFS_DPF(ALLOC, "%s: exthdr syntax error\n", - __func__); - error = EINVAL; - goto bad; + goto syntax; } if ((uintptr_t)line + len < (uintptr_t)line || line + len > exthdr + sz) { @@ -606,16 +603,18 @@ again: key = sep + 1; sep = strchr(key, '='); if (sep == NULL) { - TARFS_DPF(ALLOC, "%s: exthdr syntax error\n", - __func__); - error = EINVAL; - goto bad; + goto syntax; } *sep = '\0'; value = sep + 1; TARFS_DPF(ALLOC, "%s: exthdr %s=%s\n", __func__, key, value); - if (strcmp(key, "path") == 0) { + if (strcmp(key, "size") == 0) { + extsize = strtol(value, &sep, 10); + if (sep != eol) { + goto syntax; + } + } else if (strcmp(key, "path") == 0) { name = value; namelen = eol - value; } else if (strcmp(key, "linkpath") == 0) { @@ -625,47 +624,42 @@ again: sparse = true; major = strtol(value, &sep, 10); if (sep != eol) { - printf("exthdr syntax error\n"); - error = EINVAL; - goto bad; + goto syntax; } } else if (strcmp(key, "GNU.sparse.minor") == 0) { sparse = true; minor = strtol(value, &sep, 10); if (sep != eol) { - printf("exthdr syntax error\n"); - error = EINVAL; - goto bad; + goto syntax; } } else if (strcmp(key, "GNU.sparse.name") == 0) { sparse = true; name = value; namelen = eol - value; if (namelen == 0) { - printf("exthdr syntax error\n"); - error = EINVAL; - goto bad; + goto syntax; } } else if (strcmp(key, "GNU.sparse.realsize") == 0) { sparse = true; realsize = strtoul(value, &sep, 10); if (sep != eol) { - printf("exthdr syntax error\n"); - error = EINVAL; - goto bad; + goto syntax; } } else if (strcmp(key, "SCHILY.fflags") == 0) { flags |= tarfs_strtofflags(value, &sep); if (sep != eol) { - printf("exthdr syntax error\n"); - error = EINVAL; - goto bad; + goto syntax; } } } goto again; } + /* do we have a size from an exthdr? */ + if (extsize > 0) { + sz = extsize; + } + /* sparse file consistency checks */ if (sparse) { TARFS_DPF(ALLOC, "%s: %s: sparse %ld.%ld (%zu bytes)\n", __func__, @@ -832,6 +826,10 @@ skip: sbuf_delete(namebuf); } return (0); +syntax: + TARFS_DPF(ALLOC, "%s: exthdr syntax error\n", __func__); + error = EINVAL; + goto bad; eof: TARFS_DPF(IO, "%s: premature end of file\n", __func__); error = EIO; diff --git a/tests/sys/fs/tarfs/tarfs_test.sh b/tests/sys/fs/tarfs/tarfs_test.sh index d4de71271985..505bfc5325f0 100644 --- a/tests/sys/fs/tarfs/tarfs_test.sh +++ b/tests/sys/fs/tarfs/tarfs_test.sh @@ -396,6 +396,32 @@ tarfs_git_archive_cleanup() { tarfs_cleanup } +atf_test_case tarfs_large cleanup +tarfs_large_head() { + atf_set "descr" "Test support for large files" + atf_set "require.user" "root" + atf_set "require.kmods" "tarfs" + atf_set "timeout" "600" +} +tarfs_large_body() { + tarfs_setup + local tarball="${PWD}/tarfs_test.tar.zst" + local exp off + for exp in 31 32 33 34 35 36 ; do + for off in 1 0 ; do + local size=$(((1<