Skip to main content

Fedora 20, revisited

Back in November when I installed Fedora 20 beta on this MacBook Pro I considered this as some kind of experiment on how long it would take me to get used to "Linux on the desktop".

Coming from MacOS 10.7 (Lion), it took me a few days until I got used to different keyboard shortcuts (Command-Tab vs. ALT-Tab to switch between applications (or Windows) Command-O vs. Enter to open files, Command-{ vs. ALT-number to switch between tabs, etc.) and to setup all these little things that are working out-of-the-box on MacOS but need manual tinkering on Linux. A few things were listed in this earlier posting already, so I won't repeat them here. But in the course of 3 months of usage, more stuff came up and I wanted to share this with the outside world:

  • WiFi, this never-ending story. And not really a fault in Linux or Fedora (since it cannot be solved on a technical level but has to be solved on a legal level), but an annoyance nevertheless. This MacBook Pro has a Broadcom BCM4322 WiFi chip and needs a firmware blob to function properly. The whole setup is easy enough, but still annoying that one has to do this manually.

  • I noticed that the keyboard backlight is gone and the keys on this MacBook Pro (F5, F6) were not doing anything to change that. I set up xbindkeys to enable and adjust the keyboard backlight. Automatic adjustment is still not possible but I didn't care for that.

  • No Twitter clients: sometimes I'd like to use Twitter on the desktop (no, not the awful web frontend) but because Twitter Inc. changed its API in 2013, many clients had to fix their codebase to reflect those changes. Fedora offeres quite a few clients, but all were in a non-working or non-usable state:
  • I use VLC to watch movies because Totem (now called "Videos", sigh...) won't play files with non-free codecs and installing gstreamer plugins did not help. But VLC won't inhibt the screensaver while watching movies. Major annoyance on a desktop system!

  • After all those years, power management on Linux with pm-suspend still has issues: pm-suspend won't work when /proc is mounted with hidepid=2 - and I do use this option. The bug is still open, not sure if this one is on anyone's focus.

  • Hibernating to an encrypted swap partition is still not possible (although the bug says it was fixed in Fedora 13), so hibernation was not an option here. But suspending resp. waking up from suspend was indeed the biggest problem on this machine. Often enough, a blue Fedora logo is displayed after waking up and there's no way to login. Switching to a text-console was possible though - but logging in here was accompanied by a strange System is booting up. See pam_nologin(8) message. One could go to init 3 and back to init 5 again but then all applications had to be restarted, not a pretty thing to do on a desktop system. Eventually this got tracked down and a fix has been released but only 2 months after initial release - quite annoying for such a basic usecase.

  • Oh, and there's still this problem with sound: I know, I've covered this already in my earlier posting but since it's such a drag, let me repeat this: sound is unusable on a MacBook Pro running Fedora Linux. Overlooking the mute/unmute issues, getting the microphone to work was even harder. After booting I always ended up messing around with pavucontrol for 10 to 20 minutes, unmuting every control, toggling and sliding all the knobs and bars I could find and sometimes, if the moon was full and the gods were benevolent, I could get the microphone to work. Until the next reboot and the cycle begins again. Looks like things haven't improved since Fedora 11 (April 2009) when it comes to sound, which is kind of a big issue for a desktop system. (See also: why-alsa-sucks.png)

That's quite a list and all those little things (and the things I already forgot about) piled up and consumed a large amount of time to debug instead of doing actual work. The sound problem and the Fedora-logo-after-suspend issue were the biggest issues for me and when the latter problem finally got fixed, some other update sneaked in and now this MacoBook's display won't come back on after wakeup - so now the whole thing has to be debugged again - this was the point when I decided to go back to MacOS, I just could not take it anymore. Yes, there's a PowerBook G4 in the closet here, working as a small file server and happily running Debian/GNU Linux for years now but maybe Apple hardware just isn't the right choice when it comes to running Linux on the desktop. Maybe I shall try again in a few years, when Fedora 30 or so comes out and I'll take another look.

df(1) in MacOS 10.9

I haven't used "Mavericks" that much yet, but one thing that caught my eye early on was a change in df(1):

$ df -h /
Filesystem   Size   Used  Avail Capacity iused    ifree %iused  Mounted on
/dev/disk1  118Gi   14Gi  104Gi    12% 3626080 27340805   12%   /
Wtf? Why are they displaying inode allocation by default? Well, in their manpage they state:
 -i      Include statistics on the number of free inodes. This option is now the default
         to conform to Version 3 of the Single UNIX Specification (``SUSv3'')
         Use -P to suppress this output.
Ah, SUSv3. That was released in 2002 - of course MacOS 10.9 has to catch up on portability here, sure1 :-)

Let's look at SUSv3 (aka "IEEE Std 1003.1-2001") then:
   The df utility shall write the amount of available space [XSI] > and file slots <
And this XSI stands for "Extension":
The functionality described is an XSI extension. Functionality marked XSI is also an
extension to the ISO C standard. Application writers may confidently make use of an
extension on all systems supporting the X/Open System Interfaces Extension.
So, it's not so much a conformity issue but a mere choice to include the ouput of inode usage in the newest iteration of MacOS X. While it's still not clear why they changed it (and left -t a no-op), let's look at other Unix versions:
$ uname -srv; df -h /
SunOS 5.10 Generic_144500-19
Filesystem             size   used  avail capacity  Mounted on
/dev/md/dsk/d1         7.9G   3.0G   4.8G    39%    /

$ uname -srv; df -k /
AIX 3 5
Filesystem    1024-blocks      Free %Used    Iused %Iused Mounted on
/dev/hd4           262144    175244   34%     3919    10% /
To restore the old behaviour in MacOS, use -P:
$ df -Ph /
Filesystem   Size   Used  Avail Capacity  Mounted on
/dev/disk1  118Gi   14Gi  104Gi    12%    /

1Overlooking the fact that their manpage still reads "May 8, 1995" at the end...

Sorted by pmem

On this SLES11sp2 system, ps won't sort by "percent of physical memory":

$ ps -eo vsz,rss,pmem,pid --sort -rss | head -4
597624 515344  3.0 3201
395936 108752  0.6 15819
2254136 59280  0.3 39188

$ ps -eo vsz,rss,pmem,pid --sort -pmem | head -4
 10528   840  0.0     1
     0     0  0.0     2
     0     0  0.0     3
Turns out SLESsp2 (released 29 Feb 2012) still ships with procps version 3.2.7, even though this had been fixed back in 2010:
$ rpm -q --queryformat "%{NAME} %{VERSION} %{URL}\n" $(rpm -qf `which ps`) 
procps 3.2.7 

Fedora 20

Shortly after Mavericks came out, I installed it on this MacBook Pro of mine. But for some reason FileVault could not be activated and a reinstall was due. So what, I thought - if I have to reinstall anyway, why not try Fedora again?

Installation of the latest Fedora 20 beta release went just fine and I'm now writing this in front of an XFCE desktop and all in all it works better than expected. That is, I've only had one annoying SELinux alert (caused by the Flash plugin anyway) so far and even sound seems to work out of the box - it looks like we've finally arrived in the 21 century, hm? :-)

A few kinks remain though:

  • While sound is working, it's sometimes kinda distorted. I noticed that it's happening in Skype (still at version more often, but there's nothing to debug on and I don't know how to reproduce this reliably.

  • Speaking of sound, the xfce4-mixer application has a weird feature: when I press the "mute" button on my Apple keyboard, three controls are muted in xfce-mixer: Master, Headphone and Speaker. When I rightclick on the mixer applet and select "Mute", the same happens. But when I click "Mute" again (to unmute), only the Master control is unmuted, both Headphone and Speaker stay muted and have to be unmuted manually. Or, I can press "mute" on the keyboard again to unmute both of them. Oh, someone already filed a bug for this.

  • This builtin Nvidia chip is horrible and nouveau has a hard time coping with that shitty piece of crap. And it seems to get worse, I had two Xorg crashes today because of this. This is hardly acceptable and might be a reason to go back to MacOS.

A few other things had to be taken care of as well:
  • Ecryptfs was pretty straighforward to set up, much unlike earlier releases. As a requirement, the user needs to be in the "ecryptfs" group:
    # usermod -a -G ecryptfs bobby
    # id bobby
    uid=1000(bobby) gid=1000(bobby) groups=1000(bobby),100(users),990(ecryptfs)
    $ ecryptfs-migrate-home -u bobby
    INFO:  Checking disk space, this may take a few moments.  Please be patient.
    INFO:  Checking for open files in /home/bobby
    Enter your login passphrase [bobby]: 
    With that in place, ecryptfs should be enabled for this particular user on the next login:
    $ grep -r ecryptfs /etc/pam.d/
    /etc/pam.d/postlogin-ac:auth        optional unwrap
    /etc/pam.d/postlogin-ac:password    optional unwrap
    /etc/pam.d/postlogin-ac:session     optional unwrap

  • Speaking of crypto, let's enable encrypted swap too:
    $ cat /proc/swaps 
    Filename                                Type            Size    Used    Priority
    /dev/sda5                               partition       8191996 0       -1
    $ sudo swapoff /dev/sda5
    $ grep swap /etc/crypttab /etc/fstab
    /etc/crypttab:swap /dev/sda5 /dev/urandom swap,cipher=twofish-xts-essiv:sha256,size=256,hash=sha512
    /etc/fstab:/dev/mapper/swap       swap  swap    defaults        0 0
    I couldn't get systemd to restart the appropriate service for this, but on the next reboot, encypted swap was enabled.

  • Switching from MacOS to Linux also made the Apple key stop working. xev tells me that its keycode is 133 (Super_L) resp. 134 (Super_R), but I haven't found out yet how to map Super_L + Tab to "application switching" instead of "windows switching".

  • I still want those non-free multimedia codecs, so we have to install 3rd party repositories. I'm using RPM Fusion and it contains all I need.

  • What else? Oh, Picasa! Well, Google dropped Picasa for Linux last year, but its WINE "implementation" often needed quite a few tricks anyway to run on recent Linux distributions, so no surprise here. A good alternative (for my needs) is Shotwell. It creates an internal dabase in $HOME/.cache/shotwell but won't mess with the original image material, nice. I haven't imported the whole 20k pictures yet so I can't really tell how its performance will be.

  • AirFoil is not working, as they don't offer a Linux client, but this whole story warrants for a another blog post.

  • Yum is still soooo slow:
    # time apt-get update
    real    0m17.721s
    user    0m8.880s
    sys     0m1.760s
    # apt-cache dumpavail  | grep -c ^Package
    # yes N | /usr/bin/time -p yum update
    real 58.66
    user 3.21
    sys 0.50
    # yum list available | wc -l
    Note: the apt-get command was run on a 750MHz PowerPC machine while yum was run on a 2.53GHz Intel Core2 Duo system - go figure. Also, the available packages list of the Fedora machine has sometimes two versions for a single package: a i686 version and a x86_64 version, so it's really only about 33000 packages in total. And it's way too verbose - who cares about ~50 "repomd.xml does not match metalink" errors if there are still mirrors available? -- fixed with yum-3.4.3-120.fc20

I haven't missed any other application yet and so far it's working OK. It's not all that smooth like a well polished MacOS X, but it's good enough to get some work done and I don't have to put up with these Apple shenanigans any more - now I have to deal with Linux fuckups ;-)

ETXTBSY: Text file busy

On a recent Freakshow episode, the question arose what would happen if a running executable were to be changed while still running. As it turns out, the behaviour is different, depending on the platform and even possibly the filesystem.*)

On a Solaris 10 (x86) system, the following happened:

$ gcc hallo.c -o hallo.exe
$ gcc hallo_capital.c -o hallo_capital.exe
$ ./hallo.exe
Hallo Welt!
Hallo Welt!
Hallo Welt!
Now, on a different terminal we overwrite (but don't unlink) it:
$ cat hallo_capital.exe > hallo.exe
On the first terminal again, "Welt" changes to "WELT":
Hallo Welt!
Hallo WELT!
Hallo WELT!
Hallo WELT!
When I changed even more (e.g. using printf instead of puts), the program would stop or even crash.

So, what do other platforms do? On Linux 3.12.0-rc5 we're not able to overwrite the running executable and ETXTBSY is returned:
$ cat hallo_capital.exe > hallo.exe
bash: hallo.exe: Text file busy
Or, via strace:
open("hallo_capital.exe", O_RDONLY|O_LARGEFILE) = 3
dup2(3, 0)                              = 0
close(3)                                = 0
_llseek(0, 0, [0], SEEK_CUR)            = 0
open("hallo.exe", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = -1 ETXTBSY (Text file busy)
The same happens on a FreeBSD 9.1 system.

Interestingly, the running executable can be overwritten on Darwin (so much for the BSD part in MacOS), but the output does not change, i.e. the running binary is not modified.

*) [Citation needed]

VMware is full of shit

I wanted to download VMware Fusion (VMware Workstation for Macs) so I went to the VMware site and clicked on "Download Free Trial" and got an instant download dialog. Once the 511 MB file was on my disk I thought "Hm, before I install that thing, let's see its checksum is valid!".

Looking for the SHA1 checksum I ended up at this place, but what's that? Two versions of the same product? Let's see: apparently the version I downloaded was called "VMware Fusion 6 with VMware Tools and a 12-month complimentary subscription to McAfee VirusScan Plus" (with said 511 MB in size) and the other version is "VMware Fusion 6 Only" with only 234 MB in size. Are you kidding me?

I know, other companies bundle their programs with pesty add-on programs and sneaky toolbars as well, but holy shit! A more than doubled download size to include a crappy virus scanner as your default download option? You must be out of your mind!

Network traffic per port

A few weeks ago we got a request from the application team to find out which application uses the most network traffic at the moment. Of course, iftop or NetHogs would have been most helpful here, but this wasn't installed on this SLES 11 box. We couldn't convince sar(1) either to output how much traffic there is on a certain network port.

Hm, there was tcpdump (v3.9.8) installed on that box, maybe there is some packet size information in there that we could use? Let's see:

$ sudo /usr/sbin/tcpdump -np -c 1 2>/dev/null
03:09:45.731126 IP > P 931760268:931760449(181) ack 1045182458 win 501 
And there it was! According to its man page, that number in parentheses was the packet size in bytes.

Let's try this. Since each of our applications on that host is listening on a TCP socket, gather all the network ports we care for:
 $ netstat -ntl | awk '{print $4}' | awk -F: '/^[0-9]/ {print $2}' | sort -n > listening
Run tcpdump for a certain amount of time:
 $ ( sudo /usr/sbin/tcpdump -np > dump.pcap & ) && sleep 10 && sudo pkill tcpdump
Now, for every listening port, sum up the packet size for each packet found in the tcpdump output:
 $ for p in `cat listening`; do
    printf "PORT: $p     KB/s: "
    egrep "\."$p"[\ :]" dump.pcap | \
       awk '/\([0-9]*\)/ {print $7}' | \
       sed 's/.*(//;s/)//' | awk '{sum+=$1} END {print sum/10/1024}'
    done | sort -nk4 | tail -5
 PORT: 4673     KB/s: 0
 PORT: 4673     KB/s: 0
 PORT:   22     KB/s: 0.003
 PORT: 1512     KB/s: 5.371
 PORT: 1522     KB/s: 301.004
We matched for outbound and inbound packets here with that "PORT[\ :]" expression - adjust as needed.

So, there we have it: the application communicating on port 1552 transferred at a rate of 301 KB/s at the time we measured. Use netstat -p or lsof -i to find out the corresponding application for this network port.

Note: on another system a newer version of tcpdump (v4.3.0) was installed and the output changed considerably:
 $ sudo /usr/sbin/tcpdump -nnp -c 1 2>/dev/null
 18:47:19.598570 IP > Flags [P.], seq 2487026730:2487027316, ack \
  651238704, win 1040, options [nop,nop,TS val 321092974 ecr 2893804], length 586
Here, the size of the packet is at the end of the line and we can simplify our search routine a bit:
 $ for p in `cat listening`; do
    printf "PORT: $p     KB/s: "
    egrep "\."$p"[\ :]" dump.pcap | \
       awk '/length [1-9]/ {sum+=$NF} END {print sum/10/1024}'
And for completeness' sake, here's a (slower) version without grep:
 $ for p in `cat listening`; do 
    awk "/\."$p":.*length [0-9]*$/ {sum+=\$NF} END {print \"PORT: $p  KB/s:  \" sum/10/1024}" dump.pcap
Update: another (and maybe more elegant) solution is to create temporary iptables rules for all interesting ports and then use its builtin traffic counters (though formatting is kinda weird):
 $ for p in `cat listening`; do iptables -A INPUT -p tcp --dport $p; done
 $ iptables -n -L INPUT -v | awk '/pkts/ || /dpt:/ {print $1,$2,$NF}' | sort -nk1 | tail
 0 0 dpt:8114
 0 0 dpt:8115
 0 0 dpt:8120
 0 0 dpt:9999
 pkts bytes destination
 17      1031 dpt:4143
 25     10284 dpt:8119
 139    31000 dpt:8116
 690     121K dpt:8123
 15302    23M dpt:1234
 $ for p in `cat listening`; do iptables -D INPUT -p tcp --dport $p; done

A kingdom for a new laptop!

So, this MacBook Pro from 2009 is still running and serves me well. Upgraded to 8GB RAM and equipped with a 128GB SSD drive, can't complain too much. But sometimes it's really slow, especially the CPU has a hard time keeping up with serving a couple of virtual machines at once. So, maybe it's time for a hardware refresh.

I've been scanning the websites of the usual suspects for some time now but still haven't had any luck. I tend to come back to the same sites but for some reason they never have the right machine that suits my requirements. So, what are my requirements?

  • Fast (i.e. 4th Generation Intel i7)
  • at least 16GB RAM, 128GB SSD
  • preferably Intel HD Graphics, as its Linux support is superb
  • Bluetooth and a webcam with reasonable resolution would be nice too. Bonus points if the power cord is able to handle european power outlets too (as the MagSafe does)
  • Linux should run just fine
  • Not too big, at most 14". Otherwise I can just buy a desktop machine.
  • Good looking. This will be a tough one.
Btw, what's wrong with the current MacBook Pro models? Well, the fastest one comes pretty close: 2.9GHz Intel i7 (albeit "only" in its 3rd Generation, "Ivy Bridge"), Intel HD Graphics 4000, can be ordered with an SSD hard drive, aluminium chassis. But takes only 8GB of memory - and they cannot be upgraded wrt to system memory or storage. While I can live with an 128GB SSD for some years, 8 GB of RAM is just not enough. Still, these MacBook Pro models score enough points in my book with their webcam and its design. But they're just awful when it comes to running Linux, there's so much b0rkage and their WiFi card most probably will need special firmware - I can't deal with this shit any more.

Let's have a look at the most promising contenders, shall we?
  • A ThinkPadTM would be cool. Of course! Built from Lenovo now but still a brand name to remember. Checked out their "online shop"...and almost turned around. Who designs such crap? The only reasonable filter was the CPU type, the rest of the filters were about "weight" (5 to 6 lbs...are you kidding? Who would select "heavy, please"?) or operating system (again: why should I care? I can install any OS on it anyway!). Well, the best thing they had was a ThinkPad X1 Carbon Ultrabook, but "only" with a 3rd generation Intel i7 CPU and also 8GB memory. I guess one could upgrade to more RAM later on. But their product descriptions are just unusable? How is "ThinkPad X1 Carbon Ultrabook" vs "ThinkPad X1 Carbon Ultrabook with Professional Operating System" helpful? Or even descriptive? In smallprint one can see that the latter has a faster CPU and more memory. And all their models list as a feature "Integrated Battery" - really, in a laptop? They really need to get their act together when it comes to web shops.

  • Next up: DELL. All the business people have it, so it should be OK, right? Pro: they have already laptops with the 4th generation Intel i7 CPU and offer "12GB & up" on memory, yay! And one can also check "Intel" as a video card. Could we have a winner here? But when all those options are checked, only two laptops over 14" are left (15.6in/3.0 GHz and 17.3in/1.8 GHz). Both (somewhat) good looking, but too large for my taste. And both are delivered with spinning rust - step forward, one step back?

  • What else? HP? Hahahaha...excuse me :-) Yes, they may have had a good time selling printers back then but they suck at designing personal computers. Have you seen their laptops? I have: a monstrosity of cheap plastic entangled with mediocre hardware, just awful. Their bestTM device under 15" is a HP Split 13t-m000 x2 PC (whatever that means. Who thinks of these names?) which comes with an 1.5GHz Intel i5 CPU and 8GB memory. No, thanks.

  • There's also Toshiba. Hm, really? Well, let's give it a try. Their web shop presents me with "shop by family" (how would I know?) or "show by category": value/performance/thin&light - yes, everything please! Finally found it, at the bottom of the page, "See all laptops". Checked Intel i7 (only 3rd gen), max 14" and Intel graphics, webcam and SSD and had still 7 products to choose from - not bad! Added 8GB RAM and "more than 6 hours battery life" to it - and we're down to one result: a Toshiba Portege Z930-BT9300 Ultrabook (again, who thinks of these names? How is a customer supposed to remember this?). And it still can be thoroughly customized, I'm kinda impressed. But also wary of Toshiba, what good are they when it comes to laptops?

I'd have wished for a ThinkPad to win this race but with these few options available I don't see this happening. And I don't see a Toshiba laptop to serve me just yet. Luckily my MacBook Pro is still holding up, so let's hope it survives another year...and then the hunt begins again :-\

cat: foo: input file is output file

I needed to grow a text file and thus decided to do this:

$ ls -a > foo
$ cat foo >> foo
cat: foo: input file is output file

$ cat --version | head -1
cat (GNU coreutils) 8.13
Hm? Why does cat know that stdout is redirected to itself? And even if it does know, why should it care? On FreeBSD 9.1 (and on MacOS X), stdout is fed ad infintum to itself:
$ ls -a > foo
$ cat foo >> foo
$ ls -lh foo
-rw-------  1 alice  users  -   16M Jun 24 04:22 foo
On Solaris 10, cat behaves similar to GNU/coreutils
$ cat foo >> foo
cat: input/output files 'foo' identical

$ pkginfo -l SUNWcsu | grep VER
   VERSION:  11.10.0,REV=2005.
The workaround is to use a pipe:
$ ls -lgo foo
-rw-------+  1     100 Jun 24 06:36 foo
$ cat foo | cat >> foo
$ ls -lgo foo
-rw-------+  1     200 Jun 24 06:36 foo