tar/libarchive gzip problem [was: portsnap corrupted]
Simon L. Nielsen
simon at FreeBSD.org
Thu Jan 1 09:51:54 UTC 2009
On 2008.12.31 18:31:17 -0800, Tim Kientzle wrote:
> I just committed a fix for this (r186670). Please let
> me know if you see any other problems.
This seems to fix it, thanks!
Max, could you see if you have portsnap problems with the latest
libarchive?
> Tim Kientzle wrote:
> > I think I know what this is. I tried recently to add
> > support for concatenated gzip files. This involves looking
> > ahead for another GZip header after the end of the compressed
> > data. The current version screws this up and ends up trying
> > to match the file CRC as a Gzip header. About 1 in 256 files
> > will match the first byte, which triggers the subsequent meltdown.
> > This also explains why neither of us saw it in testing.
> > (I knew the code didn't actually work for concatenated gzip
> > files but didn't realize it would break decode of some
> > regular non-concatenated files.)
> >
> > The attached patch simply disables this additional
> > header check, which should fix the immediate problem.
> > Please try it and let me know.
> >
> > Apologies,
> >
> > Tim
> >
> > Simon L. Nielsen wrote:
> >
> >> Hey Tim,
> >>
> >> I think one of the recent changes to tar or libarchive broke gzip
> >> handling in some cases. See more below.
> >>
> >> [portsnap extract fails with gzip error]
> >>
> >> I'm not sure why I didn't run into it in my tests, but I think the
> >> problem is in tar / libarchive's handling of gzip files. Taking one
> >> random "broken" file [1] it fails with tar's build in decompression,
> >> but works using external zcat.
> >>
> >> [1]
> >> http://portsnap1.freebsd.org/f/19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz
> >>
> >> http://people.freebsd.org/~simon/tmp/19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz
> >>
> >>
> >>
> >> Broken system:
> >>
> >> [simon at eddie:/tmp] tar --version
> >> bsdtar 2.5.903a - libarchive 2.5.903a
> >> [simon at eddie:/tmp] uname -a
> >> FreeBSD eddie.nitro.dk 8.0-CURRENT FreeBSD 8.0-CURRENT #1: Tue Dec 30
> >> 22:28:33 CET 2008
> >> simon at eddie.nitro.dk:/FreeBSD/obj/FreeBSD/system-CURRENT/sys/EDDIE i386
> >> [simon at eddie:/tmp] tar tvf
> >> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz >
> >> /dev/null
> >> tar: Error opening archive: Invalid GZip header (saw 99 at offset 1)
> >> [simon at eddie:/tmp] zcat
> >> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz |
> >> tar tf - > /dev/null [simon at eddie:/tmp] zcat
> >> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz |
> >> sha256 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d
> >>
> >>
> >> OK system:
> >>
> >> [simon at benji:/tmp] tar --version
> >> bsdtar 2.5.5 - libarchive 2.5.5
> >> [simon at benji:/tmp] uname -a
> >> FreeBSD benji.s 7.1-RC2 FreeBSD 7.1-RC2 #0: Tue Dec 23 15:18:30 UTC
> >> 2008 root at logan.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386
> >> [simon at benji:/tmp] tar tvf
> >> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz >
> >> /dev/null [simon at benji:/tmp] zcat
> >> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz |
> >> sha256 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d
> >>
> >>
> >> Another OK system:
> >>
> >> [simon at ref8-amd64:files] tar --version
> >> bsdtar 2.5.5 - libarchive 2.5.5
> >> [simon at ref8-amd64:files] uname -a
> >> FreeBSD ref8-amd64.freebsd.org 8.0-CURRENT FreeBSD 8.0-CURRENT #2
> >> r184542:185402: Fri Nov 28 19:14:40 UTC 2008
> >> peter at ref8-amd64.freebsd.org:/scratch/src/sys/amd64/compile/REF8-AMD64
> >> amd64
> >> [simon at ref8-amd64:files] tar tvf
> >> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz >
> >> /dev/null
> >> [simon at ref8-amd64:files] zcat
> >> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz |
> >> sha256
> >> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d
> >>
> >
> > ------------------------------------------------------------------------
> >
> > Index: archive_read_support_compression_gzip.c
> > ===================================================================
> > --- archive_read_support_compression_gzip.c (revision 185679)
> > +++ archive_read_support_compression_gzip.c (working copy)
> > @@ -428,8 +428,7 @@
> > "Failed to clean up gzip decompressor");
> > return (ARCHIVE_FATAL);
> > }
> > - /* Restart header parser with the next block. */
> > - state->header_state = state->header_done = 0;
> > + state->eof = 1;
> > /* FALL THROUGH */
> > case Z_OK: /* Decompressor made some progress. */
> > /* If we filled our buffer, update stats and return. */
>
--
Simon L. Nielsen
More information about the freebsd-current
mailing list