From nobody Wed Feb 09 23:52:51 2022 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id C7FC619C2270; Wed, 9 Feb 2022 23:52:51 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JvGsH4vcsz4hDL; Wed, 9 Feb 2022 23:52:51 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1644450771; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ald3EsRzz7whHCj2dyIzU9TU5e2mg7rtgF8/NDUoXGw=; b=uz6bqavTUu0c5FUxkqJIry5AULPKl+UUNtSC7A+wCAIz42BxFjMBUTUBbtWI+Q4H6VPdhO JEr6JS7W7UBWV9YWxO8JvTFiWW0SVVc18kH6pM/Zvv4Y/ylaOF1Xz2mwvy6X/JQK8BJ+0j +jHUi3vo5+Xwzs4UfpUzT1SjcKVx+/3PHzfpLHHaQTwOeL0u7x68+yXslV/lv6dvkvrOt/ /nUefll67eClKhrfBMFwYcjaqFIFF08f+b04IUZlXw8cJCGYaeiFxbD8XGtaG4srenFMCg 48X5kvcAWL+dg9exfECWmpLRGneBU9ikMv03Y8NblqB+oYBBqe/KWTvD6rgpnQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 881A6168CF; Wed, 9 Feb 2022 23:52:51 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 219Nqpo8043108; Wed, 9 Feb 2022 23:52:51 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 219Nqprv043107; Wed, 9 Feb 2022 23:52:51 GMT (envelope-from git) Date: Wed, 9 Feb 2022 23:52:51 GMT Message-Id: <202202092352.219Nqprv043107@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Martin Matuska Subject: git: 833a452e9f08 - main - libarchive: import changes from upstream List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mm X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 833a452e9f082a7982a31c21f0da437dbbe0a39d Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1644450771; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ald3EsRzz7whHCj2dyIzU9TU5e2mg7rtgF8/NDUoXGw=; b=AG7UXacYF9DnJLGGXRghfFN2pZqrJjRy4UklCqtagX5zrWk/GHk60jQrENOMy35XjlZKlF c9vQQVhAH8B/9Ke5i9kcGWzrCr9DRqVku9qBnGtXhgV9nlsVfhkpTVvKeBHxO5SqWXwjH9 hgAN0bJGPffOuRnQkkL48UTtwsN8p9fy8HrLU6whXsrTGuk0CF/i1zqBowjWWg6XX1J5S6 U8afr2kN6XdnHJbMxQ6xwbsgDTMZMsug/6OfyqY8KD+i992gCx5pBX5Oy3nMFrW4si1PzV 9gplTGEmlGBMMY5iz6sezpRU9K/DMq8XikeO/pCt2QRBOyvq94oxafyrByYmjg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1644450771; a=rsa-sha256; cv=none; b=WMiuuCimge36Ozn+mjgtdtu7ki1GYykbEz9eHlGGrljJSiCIRPRFM2yCPt/cvhfMcUzKgH FLSnc+a55sF5m6UTcKhVVLHpxuE4ofPkl4tM+WEuyPt2LP8og5uYr3IrkxZxlDvfcsWo28 svxzmWgcCU53p+vq+cYS70Jc3Wr2nzhFPWw8r3vXSV6n5KNcRAk+1MNxV3k6gIgIyCvMlj sN4hNWLaFSPBYZS6VR9xds5ecE9cq3IAzf6sop9tsZYzP9RBCDQhRiVt/uMTDcLi+oEvNB +Xy3spy6++NldNIFYXgFYm8oWU14I30sg4RwR1aSfVYHAcU+KR4TuBuS/O4q9Q== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by mm: URL: https://cgit.FreeBSD.org/src/commit/?id=833a452e9f082a7982a31c21f0da437dbbe0a39d commit 833a452e9f082a7982a31c21f0da437dbbe0a39d Merge: 17a56f3fabdf 84631082f67b Author: Martin Matuska AuthorDate: 2022-02-09 23:35:42 +0000 Commit: Martin Matuska CommitDate: 2022-02-09 23:35:42 +0000 libarchive: import changes from upstream Libarchive 3.6.0 New features: PR #1614: tar: new option "--no-read-sparse" PR #1503: RAR reader: filter support PR #1585: RAR5 reader: self-extracting archive support New features (not used in FreeBSD base): PR #1567: tar: threads support for zstd (#1567) PR #1518: ZIP reader: zstd decompression support Security Fixes: PR #1491, #1492, #1493, CVE-2021-36976: fix invalid memory access and out of bounds read in RAR5 reader PR #1566, #1618, CVE-2021-31566: extended fix for following symlinks when processing the fixup list Other notable bugfixes and improvements: PR #1620: tar: respect "--ignore-zeros" in c, r and u modes PR #1625: reduced size of application binaries MFC after: 2 weeks Relnotes: yes contrib/libarchive/.editorconfig | 18 + contrib/libarchive/.github/workflows/cifuzz.yml | 24 + contrib/libarchive/NEWS | 4 + .../build/autoconf/m4_ax_compile_check_sizeof.m4 | 115 + contrib/libarchive/libarchive/archive.h | 8 +- contrib/libarchive/libarchive/archive_blake2.h | 4 +- .../libarchive/libarchive/archive_blake2_impl.h | 2 +- .../libarchive/libarchive/archive_blake2s_ref.c | 1 + .../libarchive/libarchive/archive_blake2sp_ref.c | 1 + contrib/libarchive/libarchive/archive_cryptor.c | 8 - contrib/libarchive/libarchive/archive_entry.h | 4 +- contrib/libarchive/libarchive/archive_getdate.c | 2 +- contrib/libarchive/libarchive/archive_pack_dev.c | 3 +- contrib/libarchive/libarchive/archive_platform.h | 30 + contrib/libarchive/libarchive/archive_private.h | 5 +- contrib/libarchive/libarchive/archive_read.c | 150 +- .../libarchive/archive_read_append_filter.c | 4 +- contrib/libarchive/libarchive/archive_read_disk.3 | 80 +- .../libarchive/archive_read_disk_entry_from_file.c | 8 +- .../libarchive/archive_read_disk_posix.c | 28 +- .../libarchive/libarchive/archive_read_private.h | 53 +- .../libarchive/archive_read_set_options.c | 32 +- .../libarchive/archive_read_support_filter_bzip2.c | 36 +- .../archive_read_support_filter_compress.c | 43 +- .../libarchive/archive_read_support_filter_grzip.c | 23 +- .../libarchive/archive_read_support_filter_gzip.c | 35 +- .../libarchive/archive_read_support_filter_lrzip.c | 24 +- .../libarchive/archive_read_support_filter_lz4.c | 36 +- .../libarchive/archive_read_support_filter_lzop.c | 29 +- .../archive_read_support_filter_program.c | 63 +- .../libarchive/archive_read_support_filter_rpm.c | 32 +- .../libarchive/archive_read_support_filter_uu.c | 32 +- .../libarchive/archive_read_support_filter_xz.c | 75 +- .../libarchive/archive_read_support_filter_zstd.c | 29 +- .../libarchive/archive_read_support_format_mtree.c | 117 +- .../libarchive/archive_read_support_format_rar.c | 806 +- .../libarchive/archive_read_support_format_rar5.c | 247 +- .../libarchive/archive_read_support_format_tar.c | 16 +- .../libarchive/archive_read_support_format_zip.c | 159 +- contrib/libarchive/libarchive/archive_string.c | 2 +- contrib/libarchive/libarchive/archive_write.c | 36 +- .../libarchive/archive_write_add_filter_xz.c | 4 +- .../libarchive/archive_write_add_filter_zstd.c | 42 +- contrib/libarchive/libarchive/archive_write_disk.3 | 8 +- .../libarchive/archive_write_disk_posix.c | 33 +- .../libarchive/archive_write_set_format_cpio.c | 1 + .../archive_write_set_format_cpio_binary.c | 32 +- .../libarchive/archive_write_set_format_iso9660.c | 2 + .../libarchive/archive_write_set_format_pax.c | 10 +- .../libarchive/archive_write_set_format_zip.c | 14 +- contrib/libarchive/libarchive/libarchive.3 | 40 +- .../test/test_archive_write_add_filter_by_name.c | 12 +- .../libarchive/test/test_compat_tar_directory.c | 77 + .../test/test_compat_tar_directory_1.tar.uu | 50 + contrib/libarchive/libarchive/test/test_fuzz.c | 5 + .../libarchive/test/test_read_data_large.c | 4 +- .../libarchive/libarchive/test/test_read_extract.c | 3 +- .../libarchive/test/test_read_format_rar5.c | 76 + ...format_rar5_bad_window_sz_in_mltarc_file.rar.uu | 7 + ...at_rar5_decode_number_out_of_bounds_read.rar.uu | 10 + .../test/test_read_format_rar5_sfx.exe.uu | 7048 +++++ ...d_format_rar5_window_buf_and_size_desync.rar.uu | 11 + .../libarchive/test/test_read_format_rar_filter.c | 57 + .../test/test_read_format_rar_filter.rar.uu | 26650 +++++++++++++++++++ .../libarchive/test/test_read_format_zip.c | 124 + .../test/test_read_format_zip_zstd.zipx.uu | 18 + .../test/test_read_format_zip_zstd_multi.zipx.uu | 94 + .../libarchive/libarchive/test/test_read_large.c | 4 +- .../libarchive/test/test_read_pax_truncated.c | 4 +- .../libarchive/test/test_read_truncated.c | 3 +- .../libarchive/test/test_read_truncated_filter.c | 22 +- .../libarchive/libarchive/test/test_sparse_basic.c | 37 +- .../libarchive/test/test_write_disk_secure746.c | 4 - .../libarchive/test/test_write_filter_zstd.c | 4 + .../libarchive/test/test_write_format_7zip_large.c | 6 +- .../test/test_write_format_zip_entry_size_unset.c | 321 + contrib/libarchive/tar/bsdtar.1 | 21 + contrib/libarchive/tar/bsdtar.c | 61 +- contrib/libarchive/tar/bsdtar.h | 4 + contrib/libarchive/tar/cmdline.c | 2 + contrib/libarchive/tar/creation_set.c | 2 +- contrib/libarchive/tar/read.c | 7 +- .../libarchive/tar/test/test_option_ignore_zeros.c | 147 + contrib/libarchive/tar/write.c | 4 + contrib/libarchive/test_utils/test_common.h | 2 + contrib/libarchive/test_utils/test_main.c | 104 +- contrib/libarchive/test_utils/test_utils.c | 151 +- contrib/libarchive/test_utils/test_utils.h | 11 +- lib/libarchive/tests/Makefile | 13 + usr.bin/bsdcat/tests/Makefile | 2 + usr.bin/cpio/tests/Makefile | 2 + usr.bin/tar/tests/Makefile | 3 + 92 files changed, 36971 insertions(+), 826 deletions(-) diff --cc contrib/libarchive/.editorconfig index 000000000000,2b797a039e97..2b797a039e97 mode 000000,100644..100644 --- a/contrib/libarchive/.editorconfig +++ b/contrib/libarchive/.editorconfig diff --cc contrib/libarchive/.github/workflows/cifuzz.yml index 000000000000,e5c60e4b0de2..e5c60e4b0de2 mode 000000,100644..100644 --- a/contrib/libarchive/.github/workflows/cifuzz.yml +++ b/contrib/libarchive/.github/workflows/cifuzz.yml diff --cc contrib/libarchive/build/autoconf/m4_ax_compile_check_sizeof.m4 index 000000000000,f834df6346c9..f834df6346c9 mode 000000,100644..100644 --- a/contrib/libarchive/build/autoconf/m4_ax_compile_check_sizeof.m4 +++ b/contrib/libarchive/build/autoconf/m4_ax_compile_check_sizeof.m4 diff --cc contrib/libarchive/libarchive/test/test_compat_tar_directory.c index 000000000000,5750db3628e3..5750db3628e3 mode 000000,100644..100644 --- a/contrib/libarchive/libarchive/test/test_compat_tar_directory.c +++ b/contrib/libarchive/libarchive/test/test_compat_tar_directory.c diff --cc contrib/libarchive/libarchive/test/test_compat_tar_directory_1.tar.uu index 000000000000,3e1ead9b5172..3e1ead9b5172 mode 000000,100644..100644 --- a/contrib/libarchive/libarchive/test/test_compat_tar_directory_1.tar.uu +++ b/contrib/libarchive/libarchive/test/test_compat_tar_directory_1.tar.uu diff --cc contrib/libarchive/libarchive/test/test_fuzz.c index 7ca0fd1dad26,000000000000..ba3a553db987 mode 100644,000000..100644 --- a/contrib/libarchive/libarchive/test/test_fuzz.c +++ b/contrib/libarchive/libarchive/test/test_fuzz.c @@@ -1,640 -1,0 +1,645 @@@ +/*- + * Copyright (c) 2003-2007 Tim Kientzle + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR(S) ``AS IS'' AND ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. + * IN NO EVENT SHALL THE AUTHOR(S) BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ +#include "test.h" +__FBSDID("$FreeBSD$"); + +/* + * This was inspired by an ISO fuzz tester written by Michal Zalewski + * and posted to the "vulnwatch" mailing list on March 17, 2005: + * http://seclists.org/vulnwatch/2005/q1/0088.html + * + * This test simply reads each archive image into memory, pokes + * random values into it and runs it through libarchive. It tries + * to damage about 1% of each file and repeats the exercise 100 times + * with each file. + * + * Unlike most other tests, this test does not verify libarchive's + * responses other than to ensure that libarchive doesn't crash. + * + * Due to the deliberately random nature of this test, it may be hard + * to reproduce failures. Because this test deliberately attempts to + * induce crashes, there's little that can be done in the way of + * post-failure diagnostics. + */ + +/* Because this works for any archive, we can just re-use the archives + * developed for other tests. */ +struct files { + int uncompress; /* If 1, decompress the file before fuzzing. */ + const char **names; +}; + +static void +test_fuzz(const struct files *filesets) +{ + const void *blk; + size_t blk_size; + int64_t blk_offset; + int n; + const char *skip_fuzz_tests; + + skip_fuzz_tests = getenv("SKIP_TEST_FUZZ"); + if (skip_fuzz_tests != NULL) { + skipping("Skipping fuzz tests due to SKIP_TEST_FUZZ " + "environment variable"); + return; + } + + for (n = 0; filesets[n].names != NULL; ++n) { + const size_t buffsize = 30000000; + struct archive_entry *ae; + struct archive *a; + char *rawimage = NULL, *image = NULL, *tmp = NULL; + size_t size = 0, oldsize = 0; + int i, q; + + extract_reference_files(filesets[n].names); + if (filesets[n].uncompress) { + int r; + /* Use format_raw to decompress the data. */ + assert((a = archive_read_new()) != NULL); + assertEqualIntA(a, ARCHIVE_OK, + archive_read_support_filter_all(a)); + assertEqualIntA(a, ARCHIVE_OK, + archive_read_support_format_raw(a)); + r = archive_read_open_filenames(a, filesets[n].names, 16384); + if (r != ARCHIVE_OK) { + archive_read_free(a); + if (filesets[n].names[0] == NULL || filesets[n].names[1] == NULL) { + skipping("Cannot uncompress fileset"); + } else { + skipping("Cannot uncompress %s", filesets[n].names[0]); + } + continue; + } + assertEqualIntA(a, ARCHIVE_OK, + archive_read_next_header(a, &ae)); + rawimage = malloc(buffsize); + size = archive_read_data(a, rawimage, buffsize); + assertEqualIntA(a, ARCHIVE_EOF, + archive_read_next_header(a, &ae)); + assertEqualInt(ARCHIVE_OK, + archive_read_free(a)); + assert(size > 0); + if (filesets[n].names[0] == NULL || filesets[n].names[1] == NULL) { + failure("Internal buffer is not big enough for " + "uncompressed test files"); + } else { + failure("Internal buffer is not big enough for " + "uncompressed test file: %s", filesets[n].names[0]); + } + if (!assert(size < buffsize)) { + free(rawimage); + rawimage = NULL; + continue; + } + } else { + for (i = 0; filesets[n].names[i] != NULL; ++i) + { + char *newraw; + tmp = slurpfile(&size, "%s", + filesets[n].names[i]); + newraw = realloc(rawimage, oldsize + size); + if (!assert(newraw != NULL)) + { + free(rawimage); + rawimage = NULL; + free(tmp); + continue; + } + rawimage = newraw; + memcpy(rawimage + oldsize, tmp, size); + oldsize += size; + size = oldsize; + free(tmp); + } + } + if (size == 0) { + free(rawimage); + rawimage = NULL; + continue; + } + image = malloc(size); + assert(image != NULL); + if (image == NULL) { + free(rawimage); + rawimage = NULL; + return; + } + + assert(rawimage != NULL); + + srand((unsigned)time(NULL)); + + for (i = 0; i < 1000; ++i) { + FILE *f; + int j, numbytes, trycnt; + + /* Fuzz < 1% of the bytes in the archive. */ + memcpy(image, rawimage, size); + q = (int)size / 100; + if (q < 4) + q = 4; + numbytes = (int)(rand() % q); + for (j = 0; j < numbytes; ++j) + image[rand() % size] = (char)rand(); + + /* Save the messed-up image to a file. + * If we crash, that file will be useful. */ + for (trycnt = 0; trycnt < 3; trycnt++) { + f = fopen("after.test.failure.send.this.file." + "to.libarchive.maintainers.with.system.details", "wb"); + if (f != NULL) + break; +#if defined(_WIN32) && !defined(__CYGWIN__) + /* + * Sometimes previous close operation does not completely + * end at this time. So we should take a wait while + * the operation running. + */ + Sleep(100); +#endif + } + assert(f != NULL); + assertEqualInt((size_t)size, fwrite(image, 1, (size_t)size, f)); + fclose(f); + + // Try to read all headers and bodies. + assert((a = archive_read_new()) != NULL); + assertEqualIntA(a, ARCHIVE_OK, + archive_read_support_filter_all(a)); + assertEqualIntA(a, ARCHIVE_OK, + archive_read_support_format_all(a)); + + if (0 == archive_read_open_memory(a, image, size)) { + while(0 == archive_read_next_header(a, &ae)) { + while (0 == archive_read_data_block(a, + &blk, &blk_size, &blk_offset)) + continue; + } + archive_read_close(a); + } + archive_read_free(a); + + // Just list headers, skip bodies. + assert((a = archive_read_new()) != NULL); + assertEqualIntA(a, ARCHIVE_OK, + archive_read_support_filter_all(a)); + assertEqualIntA(a, ARCHIVE_OK, + archive_read_support_format_all(a)); + + if (0 == archive_read_open_memory(a, image, size)) { + while(0 == archive_read_next_header(a, &ae)) { + } + archive_read_close(a); + } + archive_read_free(a); + } + free(image); + free(rawimage); + } +} + +DEFINE_TEST(test_fuzz_ar) +{ + static const char *fileset1[] = { + "test_read_format_ar.ar", + NULL + }; + static const struct files filesets[] = { + {0, fileset1}, + {1, NULL} + }; + test_fuzz(filesets); +} + +DEFINE_TEST(test_fuzz_cab) +{ + static const char *fileset1[] = { + "test_fuzz.cab", + NULL + }; + static const struct files filesets[] = { + {0, fileset1}, + {1, NULL} + }; + test_fuzz(filesets); +} + +DEFINE_TEST(test_fuzz_cpio) +{ + static const char *fileset1[] = { + "test_read_format_cpio_bin_be.cpio", + NULL + }; + static const char *fileset2[] = { + "test_read_format_cpio_bin_le.cpio", + NULL + }; + static const char *fileset3[] = { + /* Test RPM unwrapper */ + "test_read_format_cpio_svr4_gzip_rpm.rpm", + NULL + }; + static const struct files filesets[] = { + {0, fileset1}, + {0, fileset2}, + {0, fileset3}, + {1, NULL} + }; + test_fuzz(filesets); +} + +DEFINE_TEST(test_fuzz_iso9660) +{ + static const char *fileset1[] = { + "test_fuzz_1.iso.Z", + NULL + }; + static const struct files filesets[] = { + {0, fileset1}, /* Exercise compress decompressor. */ + {1, fileset1}, + {1, NULL} + }; + test_fuzz(filesets); +} + +DEFINE_TEST(test_fuzz_lzh) +{ + static const char *fileset1[] = { + "test_fuzz.lzh", + NULL + }; + static const struct files filesets[] = { + {0, fileset1}, + {1, NULL} + }; + test_fuzz(filesets); +} + +DEFINE_TEST(test_fuzz_mtree) +{ + static const char *fileset1[] = { + "test_read_format_mtree.mtree", + NULL + }; + static const struct files filesets[] = { + {0, fileset1}, + {1, NULL} + }; + test_fuzz(filesets); +} + +DEFINE_TEST(test_fuzz_rar) +{ + static const char *fileset1[] = { + /* Uncompressed RAR test */ + "test_read_format_rar.rar", + NULL + }; + static const char *fileset2[] = { + /* RAR file with binary data */ + "test_read_format_rar_binary_data.rar", + NULL + }; + static const char *fileset3[] = { + /* Best Compressed RAR test */ + "test_read_format_rar_compress_best.rar", + NULL + }; + static const char *fileset4[] = { + /* Normal Compressed RAR test */ + "test_read_format_rar_compress_normal.rar", + NULL + }; + static const char *fileset5[] = { + /* Normal Compressed Multi LZSS blocks RAR test */ + "test_read_format_rar_multi_lzss_blocks.rar", + NULL + }; + static const char *fileset6[] = { + /* RAR with no EOF header */ + "test_read_format_rar_noeof.rar", + NULL + }; + static const char *fileset7[] = { + /* Best Compressed RAR file with both PPMd and LZSS blocks */ + "test_read_format_rar_ppmd_lzss_conversion.rar", + NULL + }; + static const char *fileset8[] = { + /* RAR with subblocks */ + "test_read_format_rar_subblock.rar", + NULL + }; + static const char *fileset9[] = { + /* RAR with Unicode filenames */ + "test_read_format_rar_unicode.rar", + NULL + }; + static const char *fileset10[] = { + "test_read_format_rar_multivolume.part0001.rar", + "test_read_format_rar_multivolume.part0002.rar", + "test_read_format_rar_multivolume.part0003.rar", + "test_read_format_rar_multivolume.part0004.rar", + NULL + }; + static const struct files filesets[] = { + {0, fileset1}, + {0, fileset2}, + {0, fileset3}, + {0, fileset4}, + {0, fileset5}, + {0, fileset6}, + {0, fileset7}, + {0, fileset8}, + {0, fileset9}, + {0, fileset10}, + {1, NULL} + }; + test_fuzz(filesets); +} + +DEFINE_TEST(test_fuzz_tar) +{ + static const char *fileset1[] = { + "test_compat_bzip2_1.tbz", + NULL + }; + static const char *fileset2[] = { + "test_compat_gtar_1.tar", + NULL + }; + static const char *fileset3[] = { + "test_compat_gzip_1.tgz", + NULL + }; + static const char *fileset4[] = { + "test_compat_gzip_2.tgz", + NULL + }; + static const char *fileset5[] = { + "test_compat_tar_hardlink_1.tar", + NULL + }; + static const char *fileset6[] = { + "test_compat_xz_1.txz", + NULL + }; + static const char *fileset7[] = { + "test_read_format_gtar_sparse_1_17_posix10_modified.tar", + NULL + }; + static const char *fileset8[] = { + "test_read_format_tar_empty_filename.tar", + NULL + }; +#if HAVE_LIBLZO2 && HAVE_LZO_LZO1X_H && HAVE_LZO_LZOCONF_H + static const char *fileset9[] = { + "test_compat_lzop_1.tar.lzo", + NULL + }; +#endif +#if HAVE_ZSTD_H && HAVE_LIBZSTD + static const char *fileset10[] = { + "test_compat_zstd_1.tar.zst", + NULL + }; +#endif ++ static const char *fileset11[] = { ++ "test_compat_tar_directory_1.tar", ++ NULL ++ }; + static const struct files filesets[] = { + {0, fileset1}, /* Exercise bzip2 decompressor. */ + {1, fileset1}, + {0, fileset2}, + {0, fileset3}, /* Exercise gzip decompressor. */ + {0, fileset4}, /* Exercise gzip decompressor. */ + {0, fileset5}, + {0, fileset6}, /* Exercise xz decompressor. */ + {0, fileset7}, + {0, fileset8}, +#if HAVE_LIBLZO2 && HAVE_LZO_LZO1X_H && HAVE_LZO_LZOCONF_H + {0, fileset9}, /* Exercise lzo decompressor. */ +#endif +#if HAVE_ZSTD_H && HAVE_LIBZSTD + {0, fileset10}, /* Exercise zstd decompressor. */ +#endif ++ {0, fileset11}, + {1, NULL} + }; + test_fuzz(filesets); +} + +DEFINE_TEST(test_fuzz_zip) +{ + static const char *fileset1[] = { + "test_compat_zip_1.zip", + NULL + }; + static const char *fileset2[] = { + "test_compat_zip_2.zip", + NULL + }; + static const char *fileset3[] = { + "test_compat_zip_3.zip", + NULL + }; + static const char *fileset4[] = { + "test_compat_zip_4.zip", + NULL + }; + static const char *fileset5[] = { + "test_compat_zip_5.zip", + NULL + }; + static const char *fileset6[] = { + "test_compat_zip_6.zip", + NULL + }; + static const char *fileset7[] = { + "test_read_format_zip.zip", + NULL + }; + static const char *fileset8[] = { + "test_read_format_zip_comment_stored_1.zip", + NULL + }; + static const char *fileset9[] = { + "test_read_format_zip_comment_stored_2.zip", + NULL + }; + static const char *fileset10[] = { + "test_read_format_zip_encryption_data.zip", + NULL + }; + static const char *fileset11[] = { + "test_read_format_zip_encryption_header.zip", + NULL + }; + static const char *fileset12[] = { + "test_read_format_zip_encryption_partially.zip", + NULL + }; + static const char *fileset13[] = { + "test_read_format_zip_filename_cp866.zip", + NULL + }; + static const char *fileset14[] = { + "test_read_format_zip_filename_cp932.zip", + NULL + }; + static const char *fileset15[] = { + "test_read_format_zip_filename_koi8r.zip", + NULL + }; + static const char *fileset16[] = { + "test_read_format_zip_filename_utf8_jp.zip", + NULL + }; + static const char *fileset17[] = { + "test_read_format_zip_filename_utf8_ru.zip", + NULL + }; + static const char *fileset18[] = { + "test_read_format_zip_filename_utf8_ru2.zip", + NULL + }; + static const char *fileset19[] = { + "test_read_format_zip_length_at_end.zip", + NULL + }; + static const char *fileset20[] = { + "test_read_format_zip_mac_metadata.zip", + NULL + }; + static const char *fileset21[] = { + "test_read_format_zip_malformed1.zip", + NULL + }; + static const char *fileset22[] = { + "test_read_format_zip_msdos.zip", + NULL + }; + static const char *fileset23[] = { + "test_read_format_zip_nested.zip", + NULL + }; + static const char *fileset24[] = { + "test_read_format_zip_nofiletype.zip", + NULL + }; + static const char *fileset25[] = { + "test_read_format_zip_padded1.zip", + NULL + }; + static const char *fileset26[] = { + "test_read_format_zip_padded2.zip", + NULL + }; + static const char *fileset27[] = { + "test_read_format_zip_padded3.zip", + NULL + }; + static const char *fileset28[] = { + "test_read_format_zip_symlink.zip", + NULL + }; + static const char *fileset29[] = { + "test_read_format_zip_traditional_encryption_data.zip", + NULL + }; + static const char *fileset30[] = { + "test_read_format_zip_ux.zip", + NULL + }; + static const char *fileset31[] = { + "test_read_format_zip_winzip_aes128.zip", + NULL + }; + static const char *fileset32[] = { + "test_read_format_zip_winzip_aes256.zip", + NULL + }; + static const char *fileset33[] = { + "test_read_format_zip_winzip_aes256_large.zip", + NULL + }; + static const char *fileset34[] = { + "test_read_format_zip_winzip_aes256_stored.zip", + NULL + }; + static const char *fileset35[] = { + "test_read_format_zip_zip64a.zip", + NULL + }; + static const char *fileset36[] = { + "test_read_format_zip_zip64b.zip", + NULL + }; + + static const struct files filesets[] = { + {0, fileset1}, + {0, fileset2}, + {0, fileset3}, + {0, fileset4}, + {0, fileset5}, + {0, fileset6}, + {0, fileset7}, + {0, fileset8}, + {0, fileset9}, + {0, fileset10}, + {0, fileset11}, + {0, fileset12}, + {0, fileset13}, + {0, fileset14}, + {0, fileset15}, + {0, fileset16}, + {0, fileset17}, + {0, fileset18}, + {0, fileset19}, + {0, fileset20}, + {0, fileset21}, + {0, fileset22}, + {0, fileset23}, + {0, fileset24}, + {0, fileset25}, + {0, fileset26}, + {0, fileset27}, + {0, fileset28}, + {0, fileset29}, + {0, fileset30}, + {0, fileset31}, + {0, fileset32}, + {0, fileset33}, + {0, fileset34}, + {0, fileset35}, + {0, fileset36}, + {1, NULL} + }; + test_fuzz(filesets); +} + diff --cc contrib/libarchive/libarchive/test/test_read_format_rar5_bad_window_sz_in_mltarc_file.rar.uu index 000000000000,7684bc199bf9..7684bc199bf9 mode 000000,100644..100644 --- a/contrib/libarchive/libarchive/test/test_read_format_rar5_bad_window_sz_in_mltarc_file.rar.uu +++ b/contrib/libarchive/libarchive/test/test_read_format_rar5_bad_window_sz_in_mltarc_file.rar.uu diff --cc contrib/libarchive/libarchive/test/test_read_format_rar5_decode_number_out_of_bounds_read.rar.uu index 000000000000,34d8ce3f74d7..34d8ce3f74d7 mode 000000,100644..100644 --- a/contrib/libarchive/libarchive/test/test_read_format_rar5_decode_number_out_of_bounds_read.rar.uu +++ b/contrib/libarchive/libarchive/test/test_read_format_rar5_decode_number_out_of_bounds_read.rar.uu diff --cc contrib/libarchive/libarchive/test/test_read_format_rar5_sfx.exe.uu index 000000000000,fdf83af1a238..fdf83af1a238 mode 000000,100644..100644 --- a/contrib/libarchive/libarchive/test/test_read_format_rar5_sfx.exe.uu +++ b/contrib/libarchive/libarchive/test/test_read_format_rar5_sfx.exe.uu diff --cc contrib/libarchive/libarchive/test/test_read_format_rar5_window_buf_and_size_desync.rar.uu index 000000000000,9e7d20ff62ef..9e7d20ff62ef mode 000000,100644..100644 --- a/contrib/libarchive/libarchive/test/test_read_format_rar5_window_buf_and_size_desync.rar.uu +++ b/contrib/libarchive/libarchive/test/test_read_format_rar5_window_buf_and_size_desync.rar.uu diff --cc contrib/libarchive/libarchive/test/test_read_format_rar_filter.c index 000000000000,c939c6eefff5..c939c6eefff5 mode 000000,100644..100644 --- a/contrib/libarchive/libarchive/test/test_read_format_rar_filter.c +++ b/contrib/libarchive/libarchive/test/test_read_format_rar_filter.c diff --cc contrib/libarchive/libarchive/test/test_read_format_rar_filter.rar.uu index 000000000000,9b98918f1f61..9b98918f1f61 mode 000000,100644..100644 --- a/contrib/libarchive/libarchive/test/test_read_format_rar_filter.rar.uu +++ b/contrib/libarchive/libarchive/test/test_read_format_rar_filter.rar.uu diff --cc contrib/libarchive/libarchive/test/test_read_format_zip_zstd.zipx.uu index 000000000000,58a295844019..58a295844019 mode 000000,100644..100644 --- a/contrib/libarchive/libarchive/test/test_read_format_zip_zstd.zipx.uu +++ b/contrib/libarchive/libarchive/test/test_read_format_zip_zstd.zipx.uu diff --cc contrib/libarchive/libarchive/test/test_read_format_zip_zstd_multi.zipx.uu index 000000000000,502024e1c126..502024e1c126 mode 000000,100644..100644 --- a/contrib/libarchive/libarchive/test/test_read_format_zip_zstd_multi.zipx.uu +++ b/contrib/libarchive/libarchive/test/test_read_format_zip_zstd_multi.zipx.uu diff --cc contrib/libarchive/libarchive/test/test_write_format_zip_entry_size_unset.c index 000000000000,13b5f1b2db1a..13b5f1b2db1a mode 000000,100644..100644 --- a/contrib/libarchive/libarchive/test/test_write_format_zip_entry_size_unset.c +++ b/contrib/libarchive/libarchive/test/test_write_format_zip_entry_size_unset.c diff --cc contrib/libarchive/tar/read.c index bf7d55b81cb9,000000000000..2a907518d7cb mode 100644,000000..100644 --- a/contrib/libarchive/tar/read.c +++ b/contrib/libarchive/tar/read.c @@@ -1,420 -1,0 +1,419 @@@ +/*- + * Copyright (c) 2003-2007 Tim Kientzle + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR(S) ``AS IS'' AND ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. + * IN NO EVENT SHALL THE AUTHOR(S) BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include "bsdtar_platform.h" +__FBSDID("$FreeBSD$"); + +#ifdef HAVE_SYS_TYPES_H +#include +#endif +#ifdef HAVE_SYS_PARAM_H +#include +#endif +#ifdef HAVE_SYS_STAT_H +#include +#endif + +#ifdef HAVE_ERRNO_H +#include +#endif + +#ifdef HAVE_FCNTL_H +#include +#endif + +#ifdef HAVE_GRP_H +#include +#endif + +#ifdef HAVE_IO_H +#include +#endif + +#ifdef HAVE_LIMITS_H +#include +#endif +#ifdef HAVE_PWD_H +#include +#endif +#ifdef HAVE_STDINT_H +#include +#endif +#include +#ifdef HAVE_STDLIB_H +#include +#endif +#ifdef HAVE_STRING_H +#include +#endif +#ifdef HAVE_TIME_H +#include +#endif +#ifdef HAVE_UNISTD_H +#include +#endif + +#include "bsdtar.h" +#include "err.h" + +struct progress_data { + struct bsdtar *bsdtar; + struct archive *archive; + struct archive_entry *entry; +}; + +static void read_archive(struct bsdtar *bsdtar, char mode, struct archive *); +static int unmatched_inclusions_warn(struct archive *matching, const char *); + + +void +tar_mode_t(struct bsdtar *bsdtar) +{ + read_archive(bsdtar, 't', NULL); + if (unmatched_inclusions_warn(bsdtar->matching, + "Not found in archive") != 0) + bsdtar->return_value = 1; +} + +void +tar_mode_x(struct bsdtar *bsdtar) +{ + struct archive *writer; + + writer = archive_write_disk_new(); + if (writer == NULL) + lafe_errc(1, ENOMEM, "Cannot allocate disk writer object"); + if ((bsdtar->flags & OPTFLAG_NUMERIC_OWNER) == 0) + archive_write_disk_set_standard_lookup(writer); + archive_write_disk_set_options(writer, bsdtar->extract_flags); + + read_archive(bsdtar, 'x', writer); + + if (unmatched_inclusions_warn(bsdtar->matching, + "Not found in archive") != 0) + bsdtar->return_value = 1; + archive_write_free(writer); +} + +static void +progress_func(void *cookie) +{ + struct progress_data *progress_data = (struct progress_data *)cookie; + struct bsdtar *bsdtar = progress_data->bsdtar; + struct archive *a = progress_data->archive; + struct archive_entry *entry = progress_data->entry; + uint64_t comp, uncomp; + int compression; + *** 2388 LINES SKIPPED ***