Compression benchmarks 2016
Some time has passed since the last compression benchmarks and new contenders entered the race, so let's do another round of benchmarks, shall we?
MacBook Pro 2009
This laptop ships with an Intel Core2 Duo P8700 processor, so these tests may take a while:
$ tar -cf test.tar /usr/share/
$ ls -goh test.tar
-rw-r--r-- 1 384M Oct 6 08:00 test.tar
$ time for i in {1..10}; do ~/bin/compress-test.sh test.tar | tee results_${i}.out; done
[...]
real 2046m5.142s
user 222m1.302s
sys 3m30.933s
So, 10 rounds of compressing and decompressing this tarball took 34 hours to complete. The results break down to:
$ for o in 9c 1c dc; do
for p in gzip pigz bzip2 pbzip2 xz lzma zstd pzstd brotli; do
awk "/"$p"\/"$o"/ {sum+=\$3} END {print \"$p/$o\t\", sum/10}" results_*.out
done | sort -nk2; echo
done
pzstd/9c 19.7
zstd/9c 53.4
brotli/9c 234.5
pigz/9c 746.4
pbzip2/9c 764.6
gzip/9c 775.2
lzma/9c 1180.2
bzip2/9c 1563.9
xz/9c 3825
pzstd/1c 2.4
brotli/1c 4.7
zstd/1c 6.1
pigz/1c 6.2
gzip/1c 10.4
pbzip2/1c 752
xz/1c 778.7
lzma/1c 779.5
bzip2/1c 1532.3
pzstd/dc 0.8
zstd/dc 1.8
gzip/dc 2.4
pigz/dc 2.4
brotli/dc 2.9
pbzip2/dc 9.1
lzma/dc 10.2
xz/dc 10.8
bzip2/dc 748
Thinkpad E431
This machine comes with an i7-3632QM CPU and our test tarball is somewhat bigger:
$ tar -cf test.tar /usr/share/locale/ /usr/share/games/quake3/
$ ls -goh test.tar
-rw------- 1 978M Oct 8 22:38 test.tar
$ time for i in {1..10}; do ~/bin/compress-test.sh test.tar | tee results_${i}.out; done
[...]
real 420m39.764s
user 529m13.192s
sys 3m46.148s
After 7 hours, the results are in:
$ for o in 9c 1c dc; do
for p in gzip pigz bzip2 pbzip2 xz lzma zstd pzstd brotli; do
awk "/"$p"\/"$o"/ {sum+=\$3} END {print \"$p/$o\t\", sum/10}" results_*.out
done | sort -nk2; echo
done
pzstd/9c 17.4
pigz/9c 17.5
pbzip2/9c 31.5
zstd/9c 70.4
gzip/9c 84.4
bzip2/9c 145.3
brotli/9c 260
xz/9c 612.4
lzma/9c 622.4
pzstd/1c 3.3
pigz/1c 7.2
brotli/1c 8
zstd/1c 10.2
pbzip2/1c 26
gzip/1c 27.8
bzip2/1c 141.6
lzma/1c 181.5
xz/1c 185.2
pzstd/dc 0.6
zstd/dc 2.1
brotli/dc 4.8
pigz/dc 5
gzip/dc 8
pbzip2/dc 8.8
xz/dc 36.5
lzma/dc 40.2
bzip2/dc 53.3
PowerBook G4
This (older) machine is still running 24/7, so let's see which compressor we should use in the future:$ tar -cf test.tar /usr/share/doc/gcc-4.9-base/ /usr/share/perl5 $ ls -goh test.tar -rw-r--r-- 1 41M Oct 15 02:53 test.tar $ PROGRAMS="gzip bzip2 xz lzma brotli zstd" \ ~/bin/compress-test.sh -n 10 -f test.tar | tee ~/r.log $ ~/bin/compress-test.sh -r ~/r.log ### Fastest compressor: ### zstd/1c: 1.90 seconds / 63.300% smaller ### brotli/1c: 2.20 seconds / 57.900% smaller ### gzip/1c: 4.80 seconds / 58.800% smaller ### zstd/9c: 11.30 seconds / 66.000% smaller ### gzip/9c: 19.00 seconds / 62.500% smaller ### bzip2/1c: 36.90 seconds / 63.800% smaller ### lzma/1c: 37.80 seconds / 65.700% smaller ### xz/1c: 40.20 seconds / 66.000% smaller ### brotli/9c: 60.50 seconds / 66.800% smaller ### bzip2/9c: 63.00 seconds / 66.000% smaller ### xz/9c: 111.90 seconds / 68.000% smaller ### lzma/9c: 115.90 seconds / 67.700% smaller ### Smallest size: ### zstd/9c: 11.30 seconds / 66.000% smaller ### zstd/1c: 1.90 seconds / 63.300% smaller ### xz/9c: 111.90 seconds / 68.000% smaller ### xz/1c: 40.20 seconds / 66.000% smaller ### lzma/9c: 115.90 seconds / 67.700% smaller ### lzma/1c: 37.80 seconds / 65.700% smaller ### gzip/9c: 19.00 seconds / 62.500% smaller ### gzip/1c: 4.80 seconds / 58.800% smaller ### bzip2/9c: 63.00 seconds / 66.000% smaller ### bzip2/1c: 36.90 seconds / 63.800% smaller ### brotli/9c: 60.50 seconds / 66.800% smaller ### brotli/1c: 2.20 seconds / 57.900% smaller ### Fastest decompressor: ### zstd/dc: .80 seconds ### brotli/dc: 1.20 seconds ### gzip/dc: 1.20 seconds ### xz/dc: 1.70 seconds ### lzma/dc: 3.20 seconds ### bzip2/dc: 7.20 seconds