tar/libarchive gzip problem [was: portsnap corrupted]

Tim Kientzle kientzle at freebsd.org
Wed Dec 31 23:34:28 UTC 2008


I think I know what this is.  I tried recently to add
support for concatenated gzip files.  This involves looking
ahead for another GZip header after the end of the compressed
data.  The current version screws this up and ends up trying
to match the file CRC as a Gzip header.  About 1 in 256 files
will match the first byte, which triggers the subsequent meltdown.
This also explains why neither of us saw it in testing.
(I knew the code didn't actually work for concatenated gzip
files but didn't realize it would break decode of some
regular non-concatenated files.)

The attached patch simply disables this additional
header check, which should fix the immediate problem.
Please try it and let me know.

Apologies,

Tim

Simon L. Nielsen wrote:
> Hey Tim,
> 
> I think one of the recent changes to tar or libarchive broke gzip
> handling in some cases.  See more below.
> 
> [portsnap extract fails with gzip error]
> 
> I'm not sure why I didn't run into it in my tests, but I think the
> problem is in tar / libarchive's handling of gzip files.  Taking one
> random "broken" file [1] it fails with tar's build in decompression,
> but works using external zcat.
> 
> [1] http://portsnap1.freebsd.org/f/19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz
> http://people.freebsd.org/~simon/tmp/19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz
> 
> 
> Broken system:
> 
> [simon at eddie:/tmp] tar --version
> bsdtar 2.5.903a - libarchive 2.5.903a
> [simon at eddie:/tmp] uname -a
> FreeBSD eddie.nitro.dk 8.0-CURRENT FreeBSD 8.0-CURRENT #1: Tue Dec 30 22:28:33 CET 2008     simon at eddie.nitro.dk:/FreeBSD/obj/FreeBSD/system-CURRENT/sys/EDDIE  i386
> [simon at eddie:/tmp] tar tvf 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz > /dev/null
> tar: Error opening archive: Invalid GZip header (saw 99 at offset 1)
> [simon at eddie:/tmp] zcat 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz | tar tf - > /dev/null 
> [simon at eddie:/tmp] zcat 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz | sha256 
> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d
> 
> 
> OK system:
> 
> [simon at benji:/tmp] tar --version
> bsdtar 2.5.5 - libarchive 2.5.5
> [simon at benji:/tmp] uname -a
> FreeBSD benji.s 7.1-RC2 FreeBSD 7.1-RC2 #0: Tue Dec 23 15:18:30 UTC 2008     root at logan.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  i386
> [simon at benji:/tmp] tar tvf 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz > /dev/null 
> [simon at benji:/tmp] zcat 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz | sha256 
> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d
> 
> 
> Another OK system:
> 
> [simon at ref8-amd64:files] tar --version
> bsdtar 2.5.5 - libarchive 2.5.5
> [simon at ref8-amd64:files] uname -a
> FreeBSD ref8-amd64.freebsd.org 8.0-CURRENT FreeBSD 8.0-CURRENT #2 r184542:185402: Fri Nov 28 19:14:40 UTC 2008     peter at ref8-amd64.freebsd.org:/scratch/src/sys/amd64/compile/REF8-AMD64  amd64
> [simon at ref8-amd64:files] tar tvf 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz > /dev/null
> [simon at ref8-amd64:files] zcat 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz | sha256
> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: gzip_decompression.patch
Type: text/x-patch
Size: 606 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-current/attachments/20081231/cb7a06bd/gzip_decompression.bin


More information about the freebsd-current mailing list