Issues w/ gzip for > 2GB files?

[Tim Robbins]

[Context lost -- I was not subscribed to the list at the time.]

I can reproduce this, but the error that I get is slightly different:

zcat: bigfile.gz: invalid compressed data--length error

This is a bug in gzip. By checking the code, "length error" means that the
file did not expand to its original size *modulo 2^32* (see RFC 1952
section 2.3.1 "ISIZE"). gzip does not explicitly compute the remainder
mod. 2^32; instead it computes it implicitly by storing the size in an
"unsigned long", which it presumes is 32 bits wide. But since "long" is
64-bit on amd64, the modulo is not performed, so the extracted file size !=
(original file size mod 2^32) for files larger than 4 GB.

The problem you encountered may another instances of this same basic
incorrect assumption. Try this patch and let me know how things go.

--- gnu/usr.bin/gzip/gzip.h.orig	Sat Apr 24 11:51:16 2004
+++ gnu/usr.bin/gzip/gzip.h	Sat Apr 24 11:40:43 2004
@@ -41,9 +41,11 @@
 
 #define local static
 
-typedef unsigned char  uch;
-typedef unsigned short ush;
-typedef unsigned long  ulg;
+#include <stdint.h>
+
+typedef uint8_t   uch;
+typedef uint16_t  ush;
+typedef uint32_t  ulg;
 
 /* Return codes from gzip */
 #define OK      0


In any case, the bug ought to be reported to the gzip developers.


Tim