"tar -c|gzip" faster than "tar -cz"?!?

Sergey Babkin babkin at verizon.net
Fri Oct 13 18:14:03 PDT 2006


>From: Oliver Fromme <olli at lurza.secnetix.de>

> > > The tar|gzip command uses 18% less CPU and is 10% faster. It 
> > > is clear the HDD is the bottleneck.
> > 
> > Now it's clear to me :)
> > 
> > This makes sense if tar is single-threaded: there's only one thread of
> > execution, and it can either be waiting on the disk, or compressing
> > data.  With two processes, gzip can compress while tar blocks on disk
> > IO.
>
>No.  During my tests there was no physical disk I/O (the
>disk LED was *OFF* all the time).  So tar certainly wasn't
>blocking on disk I/O.
>
>The difference in CPU time (and wall clock time) seems
>simply to be caused by different compression code.  gzip
>is noticeably more efficient than libz, at least on the
>OS/processor combination where I tested it (Athlon64 with
>FreeBSD/i386 6.2-PRERELEASE).

Any chance that gzip uses a different version of libz?
Or maybe the buffer size is different? Yet another
possibility could be if tar calls zlib with the SYNC
(or is that FLUSH? something like that) flag on each 
chunk, this would kill both the performance and the 
compression rate. Then again, the default compression
level may be different (but it should be making the
speed higher if the ratio falls lower).

-SB


More information about the freebsd-hackers mailing list