speed of bzip2 versus gzip

[James Long]

On Fri, Jul 20, 2007 at 05:50:20PM -0700, Chuck Swiger wrote:
> On Jul 20, 2007, at 5:37 PM, Norberto Meijome wrote:
>>> Is it normal for bzip2 to be significantly slower than gzip?
>>> If not, where can I look for things that might be causing
>>> "bzip2 --fast" to take 50-60 times longer to compress a
>>> (sendmail log) file than gzip?
>> 
>> i never measured it to see if it is 50-60 times slower, but yes, gzip 
>> blows
>> bzip2 out of the water on speed. I wanted to use bzip2 to compress 
>> multi-GB
>> weblog files, but gzip beat it my miles, and bzip2 wasn't THAT much better 
>> @
>> compressing it to make it worth it.
> 
> Thanks for the feedback, Norberto.
> 
> Of course, it all depends on what your priorities are, too-- if what you 
> want is a final tarball which is being mirrored and downloaded frequently, 
> then your goal is to obtain the absolute best compression, and how much CPU 
> --best takes isn't important.
>
> Comparing the default (-5 compression?) of gzip to bzip2 would probably be 
> more reasonable if you care about reasonably timely compression.

If I read the man page correctly, bzip2 defaults to --best, which is why
I compared gzip to bzip2 --fast.  With the 1.5G sendmail log, bzip2 --fast 
compresses to just under 10M in about 55 minutes, give or take.  bzip2
--best compresses 1.5G to 1.8M, but takes about 2.25 hours.  gzip
compresses almost as well (with 3% or so) as --fast, but does it in 1 
minute instead of 55 on a dual P-III 1.4GHz (but of course, using only
one CPU).

Jim