Issues w/ gzip for > 2GB files?
tjr at freebsd.org
Fri Apr 23 18:55:50 PDT 2004
[Context lost -- I was not subscribed to the list at the time.]
I can reproduce this, but the error that I get is slightly different:
zcat: bigfile.gz: invalid compressed data--length error
This is a bug in gzip. By checking the code, "length error" means that the
file did not expand to its original size *modulo 2^32* (see RFC 1952
section 2.3.1 "ISIZE"). gzip does not explicitly compute the remainder
mod. 2^32; instead it computes it implicitly by storing the size in an
"unsigned long", which it presumes is 32 bits wide. But since "long" is
64-bit on amd64, the modulo is not performed, so the extracted file size !=
(original file size mod 2^32) for files larger than 4 GB.
The problem you encountered may another instances of this same basic
incorrect assumption. Try this patch and let me know how things go.
--- gnu/usr.bin/gzip/gzip.h.orig Sat Apr 24 11:51:16 2004
+++ gnu/usr.bin/gzip/gzip.h Sat Apr 24 11:40:43 2004
@@ -41,9 +41,11 @@
#define local static
-typedef unsigned char uch;
-typedef unsigned short ush;
-typedef unsigned long ulg;
+typedef uint8_t uch;
+typedef uint16_t ush;
+typedef uint32_t ulg;
/* Return codes from gzip */
#define OK 0
In any case, the bug ought to be reported to the gzip developers.
More information about the freebsd-amd64