GitXplorerGitXplorer
f

go-gunzip-bench

public
3 stars
0 forks
0 issues

Commits

List of commits on branch master.
Unverified
671ce2c09f85b7c430d528a20c209ebc97053ff6

Redo benchmarks for golang 1.11 beta2

fflx42 committed 6 years ago
Unverified
f1aa8711883cda019b3a7559f0ffeef572f502e1

Add vendoring with golang/dep

fflx42 committed 6 years ago
Unverified
a1ed53f774c8f921c97c626aa701d58b84eb6e92

Add pgzip to the benchmark and update all results

fflx42 committed 6 years ago
Unverified
10a8a237d5b1c115017807ba3a72d3c2db743221

Document the issue filed against github.com/golang/go

fflx42 committed 7 years ago
Unverified
4bdaa48c99c602eaf77ba5da15d63cf75e2a8e68

Initial commit

fflx42 committed 7 years ago

README

The README file for this repository.

go-gunzip-bench

Benchmark multiple ways of decompressing a gzip file in golang.

Published as a support for https://github.com/golang/go/issues/23154

Test configuration

$ grep -m 1 'model name' /proc/cpuinfo
model name      : Intel(R) Core(TM) i7-5930K CPU @ 3.50GHz

$ uname -a
Linux name 4.15.0-29-generic #31-Ubuntu SMP Tue Jul 17 15:39:52 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

$ go version
go version go1.11beta2 linux/amd64

Preparation

$ go build -ldflags="-s -w"

$ mkdir -p tmp

$ sudo mount -t tmpfs tmpfs tmp/

$ wget -nv https://developer.download.nvidia.com/compute/redist/cudnn/v7.0.5/cudnn-9.0-linux-x64-v7.tgz -O tmp/cudnn.tgz
2017-12-15 18:03:22 URL:https://developer.download.nvidia.com/compute/redist/cudnn/v7.0.5/cudnn-9.0-linux-x64-v7.tgz [348817823/348817823] -> "tmp/cudnn.tgz" [1]

# Note: we use `runtime.GOMAXPROCS(1)`
$ echo performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
performance

Don't use "compress/gzip", pipe to gunzip(1)

$ ./go-gunzip-bench 0 tmp/cudnn.tgz
tmp/cudnn.tgz:  4.915795273s

36b429f6f780ab46d6dfd5888918968cd5882ef6b6f4cbd97d596a2da211a4c7  tmp/cudnn.tar

Chain two readers, low memory usage, most idiomatic solution

$ ./go-gunzip-bench 1 tmp/cudnn.tgz
tmp/cudnn.tgz:  6.892494625s

36b429f6f780ab46d6dfd5888918968cd5882ef6b6f4cbd97d596a2da211a4c7  tmp/cudnn.tar

Read the whole file in-memory, stream decompress/write to output file.

$ ./go-gunzip-bench 2 tmp/cudnn.tgz
tmp/cudnn.tgz:  6.903480461s

36b429f6f780ab46d6dfd5888918968cd5882ef6b6f4cbd97d596a2da211a4c7  tmp/cudnn.tar

Read the whole file in-memory, and decompress the whole file in-memory.

$ ./go-gunzip-bench 3 tmp/cudnn.tgz
tmp/cudnn.tgz:  7.735190816s

36b429f6f780ab46d6dfd5888918968cd5882ef6b6f4cbd97d596a2da211a4c7  tmp/cudnn.tar

Method 1 but using cgzip, a golang wrapper for zlib (using cgo).

$ ./go-gunzip-bench 4 tmp/cudnn.tgz
tmp/cudnn.tgz:  3.426593085s

36b429f6f780ab46d6dfd5888918968cd5882ef6b6f4cbd97d596a2da211a4c7  tmp/cudnn.tar

Method 1 but using pgzip.

$ ./go-gunzip-bench 5 tmp/cudnn.tgz
tmp/cudnn.tgz:  6.922467413s

36b429f6f780ab46d6dfd5888918968cd5882ef6b6f4cbd97d596a2da211a4c7  tmp/cudnn.tar

gunzip(1) for read, decompress, write.

$ /usr/bin/time gunzip --keep --force tmp/cudnn.tgz && sha256sum tmp/cudnn.tar
4.73user 0.14system 0:04.88elapsed 100%CPU (0avgtext+0avgdata 1824maxresident)k
0inputs+0outputs (0major+156minor)pagefaults 0swaps
36b429f6f780ab46d6dfd5888918968cd5882ef6b6f4cbd97d596a2da211a4c7  tmp/cudnn.tar

Cleanup

$ sudo umount tmp/

$ rm go-gunzip-bench