Skip to main content

gzip vs. pigz vs. bzip2 vs. pbzip2

Shortly after the last benchmark, I came across pigz (parallel gzip) and a bigger (real-world) task to complete:

$ time gzip -c file.tar > file.tar
real     41m52.636s
user     33m58.392s
sys       2m26.903s

$ time pigz -c file.tar > file.tar.pigz
real     18m34.894s
user     54m07.784s
sys       3m47.910s

$ time bzip2 -c file.tar > file.tar.bz2
real    838m47.771s
user    830m48.621s
sys       2m18.429s

$ time pbzip2 -c file.tar > file.tar.pbz2
real     58m06.466s
user   1748m17.785s
sys       4m49.537s

$ ls -lhgo
-rw-r--r--   1  15G Jun 24 02:03 file.tar
-rw-r--r--   1 598M Jun 24 22:10 file.tar.gz
-rw-r--r--   1 600M Jun 24 21:02 file.tar.pigz
-rw-r--r--   1 304M Jun 25 12:44 file.tar.bz2
-rw-r--r--   1 306M Jun 25 13:42 file.tar.pbz2
Hardware: Sun SPARC Enterprise T5120, 1.2GHz 8-Core SPARC V9, 4GB RAM

autossh & expect

Hah! Don't try this at home work, kids:

$ cat otp.txt 

$ cat autossh.exp
#!/usr/bin/expect -f
set ipaddress [lindex $argv 0]
set password [lindex $argv 1]
set command [lindex $argv 2]
set timeout 30
spawn ssh -L 1234:localhost:80 -N $ipaddress
expect {
        "yes/no"    { send "yes\r"; exp_continue }
        "password:" { send "$password\r" }
expect ""
# send "$command \r"
# send "exit\r"
expect eof

$ cat otp.txt | while read p; do ./autossh.exp $p; done
Yes, public key authentication would be the way to go. But when used in combination with a (known) one-time password list, the above does the trick. No, I'm not actually using this and neither should you!

Fedora 15

Ready for yet another rant? If not, please skip this one. Fedora 15 has been released in late May 2011 so I upgraded my Alpha installation, let's see what we've got:

  • Sound does work now on this MacBookPro, if the speakers are unmuted:
    $ grep amixer /etc/rc.local 
    /usr/bin/amixer -c 0 set 'Surround Speaker' 115 unmute
    /usr/bin/amixer -c 0 set 'Front Speaker' 115 unmute
    /usr/bin/amixer -c 0 set 'Surround Speaker Playback Volum' 115 unmute

  • abrt is still unusable - how are people supposed to report bugs when the bugreporting tool is not working?

  • Since Empathy won't support OTR, trying to install Pidgin seemed the logical alternative. Not so on Fedora 15:
    $ yum install pidgin-otr
    ---> Package pidgin-otr.x86_64 0:3.2.0-4.fc15 will be installed
    --> Processing Dependency: pidgin >= 2.0.0 for package: pidgin-otr-3.2.0-4.fc15.x86_64
    --> Processing Dependency: libotr >= 3.2.0 for package: pidgin-otr-3.2.0-4.fc15.x86_64
    --> Processing Dependency: for package: 
    --> Running transaction check
    ---> Package libotr.x86_64 0:3.2.0-6.fc15 will be installed
    ---> Package pidgin.x86_64 0:2.7.11-2.fc15 will be installed
    --> Processing Dependency: libpurple(x86-64) = 2.7.11-2.fc15 for package:
    --> Finished Dependency Resolution
    Error: Package: pidgin-2.7.11-2.fc15.x86_64 (fedora)
               Requires: libpurple(x86-64) = 2.7.11-2.fc15
               Installed: libpurple-2.8.0-1.fc15.x86_64 (@updates-testing)
                   libpurple(x86-64) = 2.8.0-1.fc15
               Available: libpurple-2.7.11-2.fc15.x86_64 (fedora)
                   libpurple(x86-64) = 2.7.11-2.fc15
     You could try using --skip-broken to work around the problem
     You could try running: rpm -Va --nofiles --nodigest
    (Bugreport pending)

  • gvfs is killing me. It needlessly consumes resouces (also: try running du -sh * in your home directory when you accidently browsed some larger SMB share) and one cannot really remove it w/o removing all the depending packages too. I was able to remove gvfs-smb and gvfs-gphoto2, let's see if this helps.

  • Talking about resources, this is on a freshly installed Fedora 15:
    $ mount | awk '{print $5}' | sort | uniq -c | sort -n
          1 binfmt_misc
          1 debugfs
          1 devpts
          1 devtmpfs
          1 fusectl
          1 hugetlbfs
          1 mqueue
          1 proc
          1 rpc_pipefs
          1 securityfs
          1 selinuxfs
          1 sysfs
          1 vfat
          3 ext4
          5 autofs
          7 tmpfs
         11 cgroup
    Yes, that's eleven cgroup filesystems! And with all the other devices mounted all over the place (/dev/sda4 mounted in 3 different places? debugfs, what for? hugetlbfs, on a desktop machine - really?) the output of mount(8) just isn't useful any more.

  • Another thing: wasn't there a 20SecondStartup project? Oh, it's 100% completed? Hm, my system must be too slow then, it takes over a minute until the login screen appears. Pressing ESC during startup reveals a lot of daemons starting up. Let's help ourselves:
    for i in cups dnsmasq ebtables fcoe iscsi iscsid livesys livesys-late lldpad \
                lvm2-monitor mdmonitor sandbox; do echo $i && chkconfig "$i" off

  • Oh, we're such a professional distribution - let's not allow our users to be insulted.

  • Ever since YUM had been introduced, I found it very slow compared to RPM or APT. And even today, it still is:
    $ time apt-cache search . | wc -l
    real    0m1.548s
    user    0m0.892s
    sys     0m0.124s
    $ time yum --cacheonly --debuglevel 3 list available | grep time:
    Config time: 0.210
    pkgsack time: 1.460
    rpmdb time: 0.000
    real    0m22.277s
    user    0m17.956s
    sys     0m2.957s
    Note: there are 28031 packages on this Debian/5.0 machine (PowerPC G4) and 23929 packages on that Fedora 15 (MacBookPro) machine - and yum(8), even w/o updating its caches is still ~20 times slower. Other operations are also much slower and bring the system to its knees. Upgrading from Fedora 15 -alpha to -final while watching a movie? Good luck... It's pathetic watching this system thrashing while yum is "working" :-\

  • That's it for now. Missing bug reports are pending, stay tuned.

    Why do I still put up with Fedora then? I'm still in need of a desktop operating system for this MacBookPro (which is an unfriendly beast to open source software, considering their hardware choices). And Fedora felt like the most mature desktop OS once, but maybe this has changed somehow....

The specified service has been marked for deletion

While trying to install eventlog-to-syslog on WindowsXP, one has to register evtsys.exe as a service ("Eventlog to Syslog"). Playing around with that a bit, this happened when trying to uninstall the service:

> sc stop evtsys
> sc delete evtsys
The specified service has been marked for deletion.
Why yes indeed, it had been marked for deletion - but the services.msc window was still open and the service could not be uninstalled properly the first time (sigh....). Closing the services.msc window might help; sometimes removing all the leftovers in the registry does the trick. But sometimes only a reboot will resolve this - as usual :-\

be_get_uuid: failed to get uuid property from BE root dataset user properties.

So, there's this old OpenSolaris 111b installation, desperately needing an upgrade. But pkg(5) (sic!) fails to complete:

# BE_PRINT_ERR=true pkg image-update -v
Retrieving catalog ''... 
Loading catalog cache ... 
Refreshing catalog 
Refreshing catalog 1/1 
Creating Plan 
be_get_uuid: failed to get uuid property from BE root dataset user properties.
pkg: image-update cannot be done on live image
This has been partly covered in #7877 and #8313 and apparently been fixed but on this very system a newer version of SUNWbeadm cannot be installed - for the very reason that pkg image-update is not working. Installing just SUNWbeadm was not possible either, because it'd clash with max_version in pkg:/entire.

So, where to go from here? Installing the whole OS from scratch would be the easiest choice, I suppose. But also the least interesting. Let's download a newer copy of beadm (or grab the scripts from a more current system), compile Python2.6 and try again, shall we? Stay tuned....

git: fatal: invalid reference: foo

Today, git barfed with:

$ git bisect start
fatal: invalid reference: b
Huh? What is "reference b", what is git trying to tell me here? Pondering on this a bit more I remembered that I once created a branch named "b". But that was quite some time ago and git branch -a did not show the branch either. And bit reset --hard did not help. Grrr. There was a patch floating around dealing with this issue, but AFAICT it "only" made the error message more verbose. So I felt lucky and did:
$ egrep -rl 'b$' .git | grep -v objects
And indeed, there were those old BISECT_* files, which must have been from an old git bisect run. Removing the .git/BISECT_* did the trick and now we're in the middle of yet another bisect - oh jolly :-\

Ischariot Pasadelski

Ich weiss zwar nicht, wer oder was hinter steckt, ich benutze diese Seite auch nicht. Interessant fand ich nur, dass sie als Beispieltext (!) unter "Autor, Titel und Beschreibung der Galerie" einen ganz besonderen Kuenstler nennen:

  [Author: Ischariot Pasadelski]
  [Gallery Title: Fluktuation 8]
  [Gallery Description: Die Arbeiten eines polnischen Action-Künstlers]

fluktuation 8

Removing (some) newlines

I wanted to consolidate certain lines in a file:

$ cat file.txt
 - foo (11)
    - bar 11

 - foo (15)
    - bar 15
The strings "aaa", "foo" and "bar" were kinda static, but not the numbers. I wanted something like this instead:
aaa013 - foo (11)    - bar 11
aaa015 - foo (15)    - bar 15
Sure, I can use tr(1) to remove all newlines:
$ tr -d '\n' < file.txt
aaa013 - foo (11)    - bar 11aaa015 - foo (15)    - bar 15
...but that's not exactly what I wanted. The following trick did help though:
$ sed 's/^aaa.*/&:/;s/foo.*/&:/' file.txt | perl -p -e "s/:\n//"
aaa013: - foo (11):    - bar 11

aaa015: - foo (15):    - bar 15
Since some strings were somewhat static, I added a colon (":") to the end of those strings. That way the Perl snippet had something to look for when replacing newlines ("\n", with a preceding colon) with nothing.