"tar -c|gzip" faster than "tar -cz"?!?

[Pieter de Goeje]

On Thursday 12 October 2006 17:24, Oliver Fromme wrote:
> Interestingly, in both tests the compressed size of the
> "gzip" case was slightly larger than the "tar cz" case.
> That's the opposite of what I got in my very first test
> (when archiving the root file system).

I also found that the difference in size is insignificant.

> I'm not concerned about the difference in compression
> sizes, because it's in the sub-percent range.  But I'm
> more concerned about the CPU times ("user" times).
> It makes quite a clear difference in all of my tests.

I can confirm, however in my opinion the difference isn't really significant. 
Maybe a different compiler or gcc with better optimization settings could 
produce executables that are equally fast.

First test on an AMD Athlon64 with 1GB memory running -STABLE and 
~3GB /usr/ports dir.

Warmup caches with tar -cf /dev/null /usr/ports.

tar -czf /dev/null /usr/ports
      622.64 real       244.37 user         9.36 sys

tar -cf - /usr/ports | gzip > /dev/null 
      565.15 real       195.65 user        12.38 sys

The tar|gzip command uses 18% less CPU and is 10% faster. It is clear the HDD 
is the bottleneck.

Second test on an AMD Sempron with 512MB memory and 240MB directory also 
running -STABLE.
Warmup caches with tar -cf /dev/null /usr/ports/distfiles/gnome2

tar -czf - /usr/ports/distfiles/gnome2 > /dev/null
33.71 real        31.56 user         1.04 sys
32.98 real        31.00 user         0.96 sys

tar -cf - /usr/ports/distfiles/gnome2 | gzip > /dev/null
29.09 real        26.65 user         1.52 sys
29.18 real        26.62 user         1.62 sys

The tar|gzip command uses 15% less CPU and is 12% faster. Very little disk I/O 
occured during this test.

-- Pieter