tar/libarchive gzip problem [was: portsnap corrupted]
Tim Kientzle
tim at kientzle.com
Thu Jan 1 03:09:08 UTC 2009
I just committed a fix for this (r186670). Please let
me know if you see any other problems.
Tim
Tim Kientzle wrote:
> I think I know what this is. I tried recently to add
> support for concatenated gzip files. This involves looking
> ahead for another GZip header after the end of the compressed
> data. The current version screws this up and ends up trying
> to match the file CRC as a Gzip header. About 1 in 256 files
> will match the first byte, which triggers the subsequent meltdown.
> This also explains why neither of us saw it in testing.
> (I knew the code didn't actually work for concatenated gzip
> files but didn't realize it would break decode of some
> regular non-concatenated files.)
>
> The attached patch simply disables this additional
> header check, which should fix the immediate problem.
> Please try it and let me know.
>
> Apologies,
>
> Tim
>
> Simon L. Nielsen wrote:
>
>> Hey Tim,
>>
>> I think one of the recent changes to tar or libarchive broke gzip
>> handling in some cases. See more below.
>>
>> [portsnap extract fails with gzip error]
>>
>> I'm not sure why I didn't run into it in my tests, but I think the
>> problem is in tar / libarchive's handling of gzip files. Taking one
>> random "broken" file [1] it fails with tar's build in decompression,
>> but works using external zcat.
>>
>> [1]
>> http://portsnap1.freebsd.org/f/19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz
>>
>> http://people.freebsd.org/~simon/tmp/19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz
>>
>>
>>
>> Broken system:
>>
>> [simon at eddie:/tmp] tar --version
>> bsdtar 2.5.903a - libarchive 2.5.903a
>> [simon at eddie:/tmp] uname -a
>> FreeBSD eddie.nitro.dk 8.0-CURRENT FreeBSD 8.0-CURRENT #1: Tue Dec 30
>> 22:28:33 CET 2008
>> simon at eddie.nitro.dk:/FreeBSD/obj/FreeBSD/system-CURRENT/sys/EDDIE i386
>> [simon at eddie:/tmp] tar tvf
>> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz >
>> /dev/null
>> tar: Error opening archive: Invalid GZip header (saw 99 at offset 1)
>> [simon at eddie:/tmp] zcat
>> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz |
>> tar tf - > /dev/null [simon at eddie:/tmp] zcat
>> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz |
>> sha256 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d
>>
>>
>> OK system:
>>
>> [simon at benji:/tmp] tar --version
>> bsdtar 2.5.5 - libarchive 2.5.5
>> [simon at benji:/tmp] uname -a
>> FreeBSD benji.s 7.1-RC2 FreeBSD 7.1-RC2 #0: Tue Dec 23 15:18:30 UTC
>> 2008 root at logan.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386
>> [simon at benji:/tmp] tar tvf
>> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz >
>> /dev/null [simon at benji:/tmp] zcat
>> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz |
>> sha256 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d
>>
>>
>> Another OK system:
>>
>> [simon at ref8-amd64:files] tar --version
>> bsdtar 2.5.5 - libarchive 2.5.5
>> [simon at ref8-amd64:files] uname -a
>> FreeBSD ref8-amd64.freebsd.org 8.0-CURRENT FreeBSD 8.0-CURRENT #2
>> r184542:185402: Fri Nov 28 19:14:40 UTC 2008
>> peter at ref8-amd64.freebsd.org:/scratch/src/sys/amd64/compile/REF8-AMD64
>> amd64
>> [simon at ref8-amd64:files] tar tvf
>> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz >
>> /dev/null
>> [simon at ref8-amd64:files] zcat
>> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz |
>> sha256
>> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d
>>
>
> ------------------------------------------------------------------------
>
> Index: archive_read_support_compression_gzip.c
> ===================================================================
> --- archive_read_support_compression_gzip.c (revision 185679)
> +++ archive_read_support_compression_gzip.c (working copy)
> @@ -428,8 +428,7 @@
> "Failed to clean up gzip decompressor");
> return (ARCHIVE_FATAL);
> }
> - /* Restart header parser with the next block. */
> - state->header_state = state->header_done = 0;
> + state->eof = 1;
> /* FALL THROUGH */
> case Z_OK: /* Decompressor made some progress. */
> /* If we filled our buffer, update stats and return. */
More information about the freebsd-current
mailing list