Skip to main content

GRUB: error: file '/grub/i386-pc/normal.mod' not found

This has happend quite a while ago, but it was weird enough and unexpected that I still wanted to share it here. After a scheduled reboot, this machine greeted me with:

error: file '/grub/i386-pc/normal.mod' not found.
grub rescue>


So, apparently GRUB has some kind of trouble. What happened here? After some looking around in this (very minimalistic) rescue shell it became clear that almost everything below /boot/grub had been removed. Maybe by some br0ken update script, I never really found out what may have caused this.

But the interwebs were helpful as usual and the trick was now to find all the missing files elsewhere. This being a Debian system, the Grub files were also available in /usr/lib/grub/i386-pc/ (yes, even on an amd64 system). Without being able to boot into another rescue system and re-install Grub, Still in the grub rescue shell, we now have to load every needed module from that location:

> insmod /usr/lib/grub/i386-pc/normal.mod
> insmod /usr/lib/grub/i386-pc/ext2.mod
> insmod /usr/lib/grub/i386-pc/linux.mod
> [...]


We may need more modules than that, and some modules may even depend on other modules, so this took some trial and error to figure out. But in the end a basic Grub configuration could be submitted:

> set root (hd0,msdos2)
> linux (hd0,msdos2)/vmlinuz root=/dev/sda2
> initrd (hd0,msdos2)/initrd.img


If this fails, more modules may need to be loaded.

So, if this really boots, make sure to install Grub again :-)

asciinema & asciicast2gif

Sometimes a help text to convey how to properly use a command, or a help text exists but people are not reading it. In these cases a short video might do the trick. asciinema is quite popular and allows to record terminal sessions and replay them later on. But to embed these recordings in a web page another tool was needed: asciicast2gif.

asciinema was easy to install as its available in many distributions:

sudo dnf install asciinema


Record sessions with rec, play sessions with play, easy enough.

asciicast2gif wasn't available in our distribution and needed to be installed manually:

npm install asciicast2gif
sudo dnf install ImageMagick gifsicle


The first few tries failed with a horrid error message, so let's see what asciicast2gif really does:

$ asciicast2gif /tmp/test.cast /tmp/test.gif
[...]
convert -loop 0 -delay 20  /tmp/tmp.UccOho2kuG/0.png \
                -delay 420 /tmp/tmp.UccOho2kuG/1.png \
                -delay 86  /tmp/tmp.UccOho2kuG/2.png \
          [...] -layers Optimize gif:- | gifsicle -k 64 -O2 -Okeep-empty --lossy=80 -o /tmp/test.gif


And looking in dmesg revealed that convert was running out of memory while trying to convert a ~30 second screen cast recording! After granting the machine a bit more memory, the command completed and we now had a working GIF to embed in our web page:

$ asciicast2gif -t tango test.cast test.gif
==> Loading test.cast...
==> Spawning PhantomJS renderer...
==> Generating frame screenshots...
==> Combining 137 screenshots into GIF file...
gifsicle: warning: huge GIF, conserving memory (processing may take a while)
==> Done.

$ mv test.gif ~/www/
$ cat ~/www/test.md
[...]
[video](test.gif)


Note: limiting the memory usage of the node process did not seem to work, as can be seen in top:

Tasks: 117 total,   2 running, 115 sleeping,   0 stopped,   0 zombie
%Cpu0  :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu1  :  2.0 us,  0.0 sy,  0.0 ni, 74.0 id, 24.0 wa,  0.0 hi,  0.0 si,  0.0 st
GiB Mem :      7.8 total,      1.7 free,      6.0 used,      0.1 buff/cache
GiB Swap:      0.1 total,      0.0 free,      0.1 used.      1.6 avail Mem

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
   1971 dummy     20   0 5996.9m   5.8g   1.5m R 103.0  75.0   1:05.03 convert -loop 0 -delay 13 [...]
   1958 dummy     20   0  635.3m  30.7m   0.2m S   0.0   0.4   0:05.03 node --max-old-space-size=512 [...]


The actual gif quality varies depending on what's shown on the terminal. Movie like sequences look a bit crappy though, but that's not what we wanted to do in the first place, hm? :-) test.gif


error message

gifsicle:<stdin>: empty file

    at checkExecSyncError (child_process.js:630:11)
    at Object.execSync (child_process.js:666:15)
    at Dp (/home/dummy/node_modules/asciicast2gif/main.js:708:246)
    at /home/dummy/node_modules/asciicast2gif/main.js:713:178
    at Function.b [as h] (/home/dummy/node_modules/asciicast2gif/main.js:709:287)
    at ep (/home/dummy/node_modules/asciicast2gif/main.js:697:48)
    at /home/dummy/node_modules/asciicast2gif/main.js:697:193
    at /home/dummy/node_modules/asciicast2gif/main.js:689:264
    at Immediate.Po (/home/dummy/node_modules/asciicast2gif/main.js:685:331)
    at processImmediate (internal/timers.js:456:21) {
  status: 1,
  signal: null,
  output: [
    null,
    Buffer(0) [Uint8Array] [],
    Buffer(29) [Uint8Array] [
      103, 105, 102, 115, 105,  99, 108,
      101,  58,  60, 115, 116, 100, 105,
      110,  62,  58,  32, 101, 109, 112,
      116, 121,  32, 102, 105, 108, 101,
       10
    ]
  ],
  pid: 2836,   
  stdout: Buffer(0) [Uint8Array] [],
  stderr: Buffer(29) [Uint8Array] [
    103, 105, 102, 115, 105,  99, 108,
    101,  58,  60, 115, 116, 100, 105,
    110,  62,  58,  32, 101, 109, 112,
    116, 121,  32, 102, 105, 108, 101,
     10
  ]
}

Nikola

Nikola

This post is long overdue. After this domain moved to a different provider, I decided that Serendipity served me well enough for a long time but I no longer had the need for a dynamic site and wanted to switch to a static site instead. Nikola appeared to be the best fit for my needs and also was able to convert the S9Y blog into static pages.

Export the S9Y blog via its RSS feed:

$ wget "http://nerdbynature.local/s9y/rss.php?version=2.0&all=1" -O index.rss
$ ls -hgo index.rss
-rw-r----- 1 1.2M Aug  4 02:30 index.rss


Install Nikola via venv:

sudo apt-get install python3-venv
python3 -m venv opt/nikola
cd opt/nikola/
source bin/activate
bin/python -m pip install -U pip setuptools wheel
bin/python -m pip install -U "Nikola[extras]"


We don't even need the exported .rss file from before, we can import on-the-fly:

$ nikola plugin -i import_feed
$ nikola import_feed --output-folder=www/s9y \
    --url="http://nerdbynature.local/s9y/rss.php?version=2.0&all=1"


With all that in place, we're amost ready to go. Let's install a theme, build our site and serve it:

$ cd www/s9y/
$ nikola theme -i lanyon
$ nikola build && nikola serve


Be sure to tweak conf.py as needed (see below).

That was all fine and dandy, but all the (newly imported) posts were named a bit funny:

$ ls posts/
s9y20061006debianunstable-and-apache22.html
s9y20061119benchmarks-2619-rc5-git4.html
s9y20061205testing-mm-playing-around-with-xfs.html
[...]


While new posts could be created with NEW_POST_DATE_PATH=true set, I really wanted my old posts to be accessible via posts/%year/%month/%day/%title. Some ugly lines of shell later, this did the trick:

set -e
for p in $(ls posts/s9y*html); do
    DATE="$(date +%Y/%m/%d -d $(echo "$p" | sed 's|posts/s9y||' | cut -c-8))"
    NAME="$(echo $p | cut -c18- | sed 's/\.html$//')"
    mkdir -p            posts/${DATE}
    mv ${p}             posts/${DATE}/${NAME}.html
    mv ${p%%.html}.meta posts/${DATE}/${NAME}.meta
    sed "s|slug: .*|slug: ${NAME}|" -i posts/${DATE}/${NAME}.meta
    echo
done


After some cosmetic changes the (imported) blog was ready to be deployed into the webserver's document root:

$ nikola build && nikola deploy


And that's it! Goodbye S9Y, hello Nikola :-D

conf.py

$ cat conf.py
[...]
BLOG_TITLE = "nerdbynature"
SITE_URL = "https://nerdbynature.de/s9y/"
BASE_URL = "https://nerdbynature.de/s9y/"
BLOG_EMAIL = "admin@example.com"
BLOG_DESCRIPTION = "Who took the 'we' out of weblog?"

NAVIGATION_LINKS = {
     DEFAULT_LANG: (
       ("/archive.html", "Archives"),
       ("/categories/index.html", "Tags"),
       ("/rss.xml", "RSS feed"),
       ("/feed.atom", "Atom feed"),
       ("/imprint.html", "Imprint"),
     ),
}

THEME    = "lanyon"
TIMEZONE = "SystemV/PST8PDT"
DATE_FANCINESS = 0
GENERATE_RSS   = True
GENERATE_ATOM  = True
FEED_LENGTH    = 10000
USE_CDN        = False

POSTS = (
       ("posts/*.html", "posts", "post.tmpl"),
       ("posts/*.md",   "posts", "post.tmpl"),
)

COMPILERS = {
        "rest": ('.txt', '.rst'),
        "markdown": ('.md', '.mdown', '.markdown', '.wp'),
        "html": ('.html', '.htm')
}

DEPLOY_COMMANDS = {
    'default': [
       "rsync -av --delete output/ admin@webserver:/var/www/",
#      "tar -czf /tmp/s9y.tar.gz output/"
    ]
}

Fedora: where is bigint.pm?

Recently something like this happened:

$ perl -Mbigint -e 'print 1->is_zero()."\n"'
Can't locate bigint.pm in @INC (you may need to install the bigint module)

OK, but which package will provide bigint? (not to be confused with Math::BigInt!)

Debian has apt-file:

$ apt-file search bigint.pm
perl-modules-5.28: /usr/share/perl/5.28.1/bigint.pm

Arch Linux has Pacman:
$ pacman -F bigint.pm
core/perl 5.28.1-1 (base) [installed: 5.30.1-1]
    usr/share/perl5/core_perl/bigint.pm

openSUSE has zypper but its search function isn't returning much. However, bigint.pm is provided by their standard perl package:
$ rpm -qf `locate bigint`
perl-5.30.1-3.2.x86_64

And Fedora has dnf, but whatprovides doesn't return anything and search only returns slightly unrelated results:
$ dnf search bigint
texlive-bigints-doc.noarch
perl-Math-BigInt-GMP.x86_64
perl-Math-BigInt-FastCalc.x86_64
texlive-bigints.noarch
perl-Math-BigInt.noarch
php-pear-math-biginteger.noarch

But none of those actually provided bigint.pm. Thankfully a comment in RHBZ#1286363 provided the key command on how to install the correct Perl module:
$ sudo dnf install 'perl(bigint)'

With that in place, the missing bigint.pm would be installed and the command above executes just fine. Of course, this works for other pragmas just as well:
$ dnf install 'perl(threads)'
Package perl-threads-1:2.22-439.fc31.x86_64 is already installed.

SELinux is preventing dnsmasq from using the dac_override capability.

While trying to set log-facility=/var/log/dnsmasq.log in dnsmasq.conf resulted in an SELinux splat:

SELinux is preventing dnsmasq from using the dac_override capability.
[...]
Raw Audit Messages
type=AVC msg=audit(1583125188.633:22508): avc:  denied  { dac_override } for  pid=1501431 comm="dnsmasq" capability=1  scontext=system_u:system_r:dnsmasq_t:s0 tcontext=system_u:system_r:dnsmasq_t:s0 tclass=capability permissive=0

Hash: dnsmasq,dnsmasq_t,dnsmasq_t,capability,dac_override
This had been reported before (in 2018), but for /var/lib/dnsmasq/dnsmasq.leases, this time it was about /var/log/dnsmasq.log and we had everything in place:
$  ls -lZ /var/log/dnsmasq.log 
-rw-r-----. 1 dnsmasq root system_u:object_r:dnsmasq_var_log_t:s0 79783 \
            Mar  1 20:59 /var/log/dnsmasq.log
Before granting dac_override to dnsmasq, we found this all explained in another blog post:
[...] The simple thing to do from an SELinux point of view would be to add the allow rule

allow dovecot_t self:capability dac_override;

But from a security proint of view, this is lousy.  The much better solution would be to 'relax' the permissions on the socket by adding group read/write.
And indeed, this helped as expected:
$ chmod -c g+w /var/log/dnsmasq.log
mode of '/var/log/dnsmasq.log' changed from 0640 (rw-r-----) to 0660 (rw-rw----)
Now dnsmasq would start and is able to log to /var/log/dnsmasq.log.

Resize a NetBSD root disk

A NetBSD DomU (Xen) needed more disk space for its root disk. While this may not be worth mentioning with the help of cfdisk or GNU/parted in Linux land, I haven't done this yet on a NetBSD system. Hubert describes this in part on his blog, but doesn't really enlarge the partition but adds and configures another partition to the disk. So, let's describe the whole process, including the resize_ffs part.

First, we need to resize the actual device of course. We're using LVM for our PV domain:

$ lvresize --size +4G vg0/netbsd-disk0
After starting the DomU, we can see the new disk size:
netbsd: xbd0: 4096 MB, 512 bytes/sect x 8388608 sectors
netbsd: xbd0: 8192 MB, 512 bytes/sect x 16777216 sectors
These sector numbers will be important in the next step, editing the disklabel:
$ disklabel xbd0
# /dev/rxbd0:
type: unknown
disk: disk0
label:
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 16
sectors/cylinder: 1008
cylinders: 8322
total sectors: 8388608
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0           # microseconds
track-to-track seek: 0  # microseconds
drivedata: 0

16 partitions:
#        size    offset     fstype [fsize bsize cpg/sgs]
 a:   7863408         0     4.2BSD   2048 16384     0
 b:    525200   7863408       swap                   
 c:   8388608         0     unused      0     0
 d:   8388608         0     unused      0     0
We need to do a few things now:
  • Adjusting the total sectors
  • Adjusting the d partition, the full disk on x86.
  • adjusting the c partition, the NetBSD part of the disk. As all partitions will belong to this NetBSD installation, its size will be equal to the d partition.
  • Adjust the offset of the swap partition.
  • And finally adjust the size of our root partition, a in our case.
Let's do all that in one go:
$ disklabel -e xbd0                         
# /dev/rxbd0:
type: unknown
disk: disk0
label: 
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 16
sectors/cylinder: 1008
cylinders: 8322
total sectors: 16777216
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0           # microseconds
track-to-track seek: 0  # microseconds
drivedata: 0 

16 partitions:
#        size    offset     fstype [fsize bsize cpg/sgs]
 a:  16252016         0     4.2BSD   2048 16384     0
 b:    525200  16252016       swap                   
 c:  16777216         0     unused      0     0
 d:  16777216         0     unused      0     0
With 16777216 as a (new) total sector count, and a swap size of (unchanged) 525200 sectors, this leaves 16252016 sectors for the root disk.

In Linux, the partition table would need to be initialized and I decided to reboot the VM to make this happen. Only afterwards I learned about disklabel -i -r: "Read the on-disk label for sd0, edit it using the built-in interactive editor and reinstall in-core as well as on-disk".

Now that the disklabel has been adjusted, we still need to resize the file system:
$ resize_ffs -p -v /dev/rxbd0a
It's required to manually run fsck on file system before you can resize it

 Did you run fsck on your disk (Yes/No) ? Yes
Growing fs from 1965852 blocks to 4063004 blocks.

$ df -h /
Filesystem         Size       Used      Avail %Cap Mounted on
/dev/xbd0a         3.6G       3.4G       8.5M  99% /
Hm, still nothing. Rebooting the VM once more, but now fsck was unhappy:
Starting root file system check:
/dev/rxbd0a: BAD SUPER BLOCK: VALUES IN SUPER BLOCK DISAGREE WITH THOSE IN FIRST ALTERNATE

/dev/rxbd0a: UNEXPECTED INCONSISTENCY; RUN fsck_ffs MANUALLY.
Automatic file system check failed; help!
ERROR: ABORTING BOOT (sending SIGTERM to parent)!
[1]   Terminated              (stty status "^T...
Enter pathname of shell or RETURN for /bin/sh:

# fsck_ffs -f /dev/rxbd0a
** /dev/rxbd0a
BAD SUPER BLOCK: VALUES IN SUPER BLOCK DISAGREE WITH THOSE IN FIRST ALTERNATE
** File system is already clean
** Last Mounted on /
** Root file system
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames
** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
** Phase 5 - Check Cyl groups
ALTERNATE SUPERBLK(S) ARE INCORRECT
SALVAGE? [yn] y

SUMMARY INFORMATION BAD
SALVAGE? [yn] y

BLK(S) MISSING IN BIT MAPS
SALVAGE? [yn] y

158187 files, 1806197 used, 99638 free (654 frags, 12373 blocks, 0.0% fragmentation)

***** FILE SYSTEM WAS MODIFIED *****
While we're in this rescue shell, let's try resize_ffs once more:
# resize_ffs -p -v /dev/rxbd0a
It's required to manually run fsck on file system before you can resize it

 Did you run fsck on your disk (Yes/No) ? Yes
Growing fs from 1965852 blocks to 4063004 blocks.
Another reboot later and now the system is able to see its new disk space:
$ df -h /
Filesystem         Size       Used      Avail %Cap Mounted on
/dev/xbd0a         7.5G       3.4G       3.7G  48% /

From autofs to systemd.automount

The venerable autofs mechanism to automatically mount and unmount network shares still works with today's systems but lately I noticed that NFS and CIFS shares would hang when I unplug my laptop from the local network and connect at another site (e.g. work, or a random coffee shop) where the usual network shares are not reachable. More and more processes will hang (and waiting for the network resource to re-appear) and eventually the machine will be almost unusable and only a reboot may help.

Of course one could configure a VPN to make these resources available all the time, but I don't really need these network shares and I'm already running a VPN when I'm out and about, so this would be unnecessary and overly complicated. With the reign of systemd it is now possible to have systemd handle automounting via the systemd.automount unit, so let's see if it handles these situations better.

autofs

While several tutorials on how to implement this already exist, let's recap first how autofs works. The main configuration file is /etc/auto.master, containing nothing more than:
+dir:/etc/auto.master.d
+auto.master
In /etc/auto.master.d the real map files are referenced:
$ cat /etc/auto.master.d/local.autofs 
/mnt/smb /etc/auto.cifs
/mnt/nfs /etc/auto.nfs
These map files will contain the share definitions:
# auto.cifs
win0  -fstype=cifs,vers=3.0,fsc,guest,rw,nodev,nosuid,noexec,fsc ://smb/win0
win1  -fstype=cifs,vers=3.0,fsc,guest,ro,nodev,nosuid,noexec,fsc ://smb/win1

# auto.nfs
data0  -fstype=nfs,rw,nodev,nosuid,noexec,bg,intr,sec=sys,acl,fsc nfs:/mnt/data0
data1  -fstype=nfs,ro,nodev,nosuid,noexec,bg,intr,sec=sys,acl,fsc nfs:/mnt/data1
Once autofs.service is reloaded, the shares should be accessible.

systemd.automount

But let's dismantle all that and now turn to systemd.automount. For each (network) share we will need a .mount and also a .automount unit file:
$ cat /usr/local/etc/mnt-nfs-data0.mount 
[Unit]
Description=NFS data0

[Mount]
What=nfs:/mnt/data0
Where=/mnt/nfs/data0
Type=nfs4
Options=rw,nodev,nosuid,noexec,bg,intr,sec=sys,acl,fsc

[Install]
WantedBy=multi-user.target
$ cat /usr/local/etc/mnt-nfs-data0.automount 
[Unit]
Description=Automount NFS data0

[Automount]
Where=/mnt/nfs/data0

[Install]
WantedBy=multi-user.target
Link both unit files to /etc/systemd/system, repeat for each network share as needed:
sudo ln -s /usr/local/etc/mnt-nfs-data0.mount     /etc/systemd/system/
sudo ln -s /usr/local/etc/mnt-nfs-data0.automount /etc/systemd/system/
The .mount unit files only need to be linked; the .automount files need to be enabled and started:
sudo systemctl enable mnt-nfs-data0.automount
sudo systemctl start  mnt-nfs-data0.automount
With that, the share should be accessible:
$ mount | grep -m1 mnt/nfs
systemd-1 on /mnt/nfs/data0 type autofs (rw,relatime,fd=48,pgrp=1,timeout=0[...]
This configuration has now been running on my laptop for a few months and it feels like it behaves better when these network resources go away and the machine isn't locking up any more. Yay \o/

conditional name resolving with dnsmasq

For some reason I needed to install a lightweight DNS forwarder on my local machine. A host file would not be sufficient, I really needed some kind of local DNS machinery that allows for specific queries to be answered by certain DNS servers. But looking more closely, there were already two DNS servers running on that machine:

$ sudo netstat -lnpu | grep :53\ 
udp   127.0.0.53:53     0.0.0.0:*  3845/systemd-resolved
udp   192.168.122.1:53  0.0.0.0:*  3189/dnsmasq  
The first one is systemd-resolved, that mainly seems to care about which resolv.conf to use and provides only some basic configuration parameters, not sufficient to what's needed here.

The second one is from libvirtd, running on the default address of the virtual virbr0 interface. And indeed, some parameters could be adjusted and the following actually worked:
$ sudo virsh net-edit --network default
[...]
  <dns>
    <forwarder domain='example.com' addr='1.2.3.4'>
    <forwarder domain='foobar.net'  addr='2.2.2.2'/>
    <forwarder addr='5.5.5.5'/>
  </dns>

$ sudo virsh net-destroy --network default
$ sudo virsh net-start   --network default
This would forward queries for example.com to 1.2.3.4 and all queries not listed here to 5.5.5.5. That was kind of what was needed, but these rules needed to be updated from time to time and editing XML stanzas for DNS entries felt somewhat unnatural. Also, with that setup I would depend on libvirt to always be installed and in working condition. If, for some reason, the libvirt setups breaks and its dnsmasq instance doesn't come up, the system would have no DNS services. But since dnsmasq was installed anyway, let's just use that.

After disabling systemd-resolved, the DNS part of libvirt's dnsmasq instance needed to be disabled too:
  <dns enable='no'/>
With that, port 53 was free to use and a new dnsmasq instance could be spawned.
$ cat /etc/dnsmasq.d/local.conf
interface=lo
listen-address=127.0.0.1
log-queries
server=5.5.5.5

server=/example.com/1.2.3.4
server=/foobar.net/2.2.2.2
All unspecified queries will go to 5.5.5.5. We could also omit that and with the absence of a no-resolv directive, dnsmasq will forward all unspecified queries to a name server specified in /etc/resolv.conf. That way we can have distinct (private) name servers for certain domains, and a stable fallback for everything else. Neat :-)

iSCSI fun

Long time no blog post, I know. Maybe this has to do with the fact that all these posts have always been mental notes to myself first, and bits of lessons learned for the everybody else second and most of my mental notes end up in a wiki installation of mine, which gets updated way more often than this blog :-\

Anyway, the other day I was trying to build Android for a Sony phone but I didn't have enough disk space on my laptop to do so. Checking out all the sources and the build takes almost 200 GB of space - but luckily I had an (encrypted) external disk available that I could use just for this. Once plugged in and decrypted I realized that I really wanted the build environment to match the proposed requirements as closely as possible. Running a Fedora 28 desktop, let's use a Ubuntu 18.04 virtual machine for the actual build:

$ vboxmanage createhd disk --filename /opt/vm/generic/disk2.vdi --size 204800 --variant Fixed
$ vboxmanage storageattach ubuntu0 --storagectl SATA --device 0 --port 2 --type hdd --medium /opt/vm/generic/disk2.vdi
Note that we use a fixed disk image to prevent some nasty I/O errors within that virtual machine. With all that in place, we could start the machine and start to build AOSP. With regards to the aforementioned nasty I/O errors, I really must give credit to btrfs in this scenario with its built-in data checksums. I know, ZFS has this feature for ages but as this still isn't available upstream, btrfs is the next best thing.

So, while this was all fine and dandy, I still had this disk enclosure attached to my laptop and thus I couldn't move around with my laptop as I usually do. Talk about #firstworldproblems! :-) So, why not attach the enclosure to my "server" instead and see if I could somehow access the enclosure over WiFi? For some reason the first thing that pops into my head was NBD and while I had an network block device setup going in the past, it wasn't much fun and would fail too often. So let's use iSCSI instead.

Once the disk enclosure was attached to the "server", the block devices needed to be passed on to the Xen DomU that was supposed to present them as an iSCSI target:
$ lsblk /dev/sd[de]
NAME MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sdd    8:48   0 931.5G  0 disk 
sde    8:64   0 931.5G  0 disk 
 
$ xl block-attach virt2 'format=raw, vdev=xvdd, access=rw, target=/dev/sdd'
$ xl block-attach virt2 'format=raw, vdev=xvde, access=rw, target=/dev/sde'
In the virtual machine, the tgt needed to be installed and configured:
$ cat /etc/tgt/conf.d/md0.conf 
default-driver iscsi

# RAID-0
<target iqn.example.local:virt2-sdd>
    backing-store dev/sdd
    initiator-address 10.0.0.15
</target>
With that in place, we can go back to our laptop again and attach the disk:
$ iscsiadm -m discovery -t sendtargets -p virt2
10.0.0.24:3260,1 iqn.example.local:virt2-sdd

$ iscsiadm -m node --targetname "iqn.example.local:virt2-sdd" --portal 10.0.0.24:3260 --login

$ lsblk /dev/sdb 
NAME MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sdb    8:16   0  1.8T  0 disk
This worked :-)

Now we can decrypt the disk (sdb) again and make it available to our VirtualBox VM. It may not be the fastest setup (WiFi being the bottleneck), but it worked and now I can move around with my laptop again, with the disk enclosure sitting somewhere else. Yay! :-)

How to copy a DVD

The other day my neighbor came over and asked me if it was possible to make a copy of a DVD, and then burn the copy to a blank DVD, so that it can be played in a DVD player. I rarely use DVDs and my current computer doesn't even have a DVD drive anymore, so I used an older MacBook Pro for this task.

Looking back, the easiest way to do this would be to get a copy of the movie from DVD somewhere else and then find out how to make a playable DVD out of it, but why not go the whole way and find out about the extraction part as well?

DVD Ripping

The proper term here seems to be Ripping and in the past I sometimes used the wonderful HandBrake to do just that. Handbrake can then convert the ripped copy to other formats, but we're not quite there yet.

Usually it was sufficient to play a DVD once with VLC which would then use libdvdcss to store the CSS key in ~/.dvdcss, in turn allowing HandBrake to decrypt the same, which is essential for the rip to complete. And while this worked beforeTM, this time the resulting video was all distorted and felt like watching an old, mangled VHS tape, so something wasn't right.

The internet was full of similar reports and suggestions too, the main theme being "Just install libdvdcss to the correct location for HandBrake to find and it should just work". Well, instead of just relying on VLC to do the decryption once, I did install libdvdcss via Homebrew, hoping that Handbrake will be able to find it:

$ otool -L /usr/local/lib/libdvdcss.dylib
/usr/local/lib/libdvdcss.dylib:
        /usr/local/opt/libdvdcss/lib/libdvdcss.2.dylib (compatibility version 5.0.0, current version 5.0.0)
        /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1259.20.0)
        /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)

And lo and behold, Handbrake did find the library, but then would crash reliably when trying to open the DVD. Bummer. And only the closing comment in the same bug report shed some light on this, suggesting that 1) libdvdcss is not recommended anyway and 2) another tool named "MakeMKV" would help here.
And off we are, still trying to complete the DVD ripping part, when we could have just downloaded the movie from somewhere else :-) Luckily MakeMKV really did the trick and was easy enough to use, and offers a 30 60 day trial version, which is just fine for this one-off experiment.

DVD Creation

MakeMKV produced a ~3 GB file (MPEG-2 Video, AC3 Audio) which now needed to be converted into the DVD-Video format and finally burned onto a blank DVD-R.

I had Burn installed on this Mac, and while it is able to burn DVD-Video, it wouldn't understand the .mkv container format. The interwebs are full of recommendations for something called "iSkysoft DVD Creator", which is offered from so many shady looking websites, and under so many different alternate names that it's hard to not suspect something sinister. At least on first sight, the image does not present itself as malware, so maybe it's safe enough to try? After removing my tin foil hat and installing, this DVD creator was indeed able to parse the .mkv file and burn a DVD-Video disc in the correct format. But, as I was using the Trial Version, the whole movie was overlayed with a huge visible watermark. Lacking official documentation regarding this fact, I should have suspected for this to happen. Hm, so...what else is out there?

Digging through the Homebrew-Cask database I found DVDStyler, which should be up to the task as well. And it's released as Open Source software, cool beans!

The DVDStyler interface felt a bit awkward, but never look a gift horse in the mouth (I can't believe that this is a real proverb in the English language!) and a few mouse clicks and a coffee later, a DVD-Video copy was produced. Yay!

Next time I must remember to direct my neighbor to the next video-on-demand platform instead of ever fumbling with DVD copies again :-)

Privatkopie