flat assembler
Message board for the users of flat assembler.

Index > Heap > Parallel Bzip2 and Parallel Gzip (multi-core aware)

Author
Thread Post new topic Reply to topic
rugxulo



Joined: 09 Aug 2005
Posts: 2341
Location: Usono (aka, USA)
rugxulo
Okay, all you multi-core folks (not me!) should try these out. I'm curious to know how well they work.

Parallel Bzip2

Parallel Gzip
Post 20 Jan 2007, 22:01
View user's profile Send private message Visit poster's website Reply with quote
rugxulo



Joined: 09 Aug 2005
Posts: 2341
Location: Usono (aka, USA)
rugxulo
Quote:


Newsgroups: comp.compression
From: "Mark Adler"
Date: 28 Jan 2007 18:56:00 -0800
Subject: Re: Parallel gzip


I have updated my parallel implementation of gzip (called pigz) to
version 1.1. I rewrote the parallelism, and it now makes much better
use of my four Xeon cores, around 97%. You can get it here:

http://zlib.net/pigz.c

It needs to be compiled with zlib 1.2.3.
Post 05 Feb 2007, 19:42
View user's profile Send private message Visit poster's website Reply with quote
f0dder



Joined: 19 Feb 2004
Posts: 3170
Location: Denmark
f0dder
Near-linear speedup? That's pretty frigging cool, if it doesn't affect the compression ratio negatively (but bzip2 works on chunks, doesn't it? At least that'd explain near-linear speedup Smile ).
Post 05 Feb 2007, 23:25
View user's profile Send private message Visit poster's website Reply with quote
rugxulo



Joined: 09 Aug 2005
Posts: 2341
Location: Usono (aka, USA)
rugxulo
Bzip2 was recently updated to 1.0.4 (Dec. 20, 2006) although I don't think it was anything really significant.

Quote:

1.0.4 (20 Dec 06)
~~~~~~~~~~~~~~~~~
Fixes some minor bugs since the last version, 1.0.3.

* Fix file permissions race problem (CAN-2005-0953).

* Avoid possible segfault in BZ2_bzclose. From Coverity's NetBSD
scan.

* 'const'/prototype cleanups in the C code.

* Change default install location to /usr/local, and handle multiple
'make install's without error.

* Sanitise file names more carefully in bzgrep. Fixes CAN-2005-0758
to the extent that applies to bzgrep.

* Use 'mktemp' rather than 'tempfile' in bzdiff.

* Tighten up a couple of assertions in blocksort.c following automated
analysis.

* Fix minor doc/comment bugs.
Post 06 Feb 2007, 20:58
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum


Copyright © 1999-2020, Tomasz Grysztar. Also on YouTube, Twitter.

Website powered by rwasa.