More memory with ZRAM

Some of my machines have only little memory and I looked for a better way to utilize what little memory there is in the system. Without being able to increase the physical memory available, there are basically 3 options here:

  • Disable unused applications
  • Reduce the memory footprint of running applications
  • Cheat :-)
After exhausting the first two options, I remembered some memory management mechanisms for the Linux kernel that were introduced a while ago but I've never used them so far:

KSM

KSM (Kernel Samepage Merging) is a memory-saving de-duplication feature but it's only really useful for applications using the madvise(2) system call and is often used when hosting virtual machines, e.g. KVM. I'm not running KVM on this machine but activated it anyway - but no application seems to use madvise(2) and it didn't help anything regarding memory usage.

zswap

There's zswap, a lightweight compressed cache for swap pages. But instead of a real swap device, the compression takes place in a dynamically allocated memory pool. To enable it, the system must be booted with zswap.enabled=1 but I didn't want to reboot my system just yet, so I skipped that option for now.

Update: I've enabled zswap in the same VM from the zram test below and ran the same test - but the results are rather irritating:
$ awk '{print $NF}' /proc/cmdline 
zswap.enabled=1

$ i=0; while true; do
   [ $i -gt 2000 -a `expr $i % 50` = 0 ] && printf "$i  "
   bash -c "sleep 10000 &"; i=$((i+1))
done
2050  2100  2150  2200  2250  2300  ^C

$ free -m
              total    used  free  shared  buff/cache  available
Mem:            241    192       3       0     46        7
Swap:           127    127       0

$ pgrep -c sleep
2333

$ grep -r . /sys/kernel/debug/zswap/
/sys/kernel/debug/zswap/stored_pages:24754
/sys/kernel/debug/zswap/pool_total_size:50696192
/sys/kernel/debug/zswap/duplicate_entry:0
/sys/kernel/debug/zswap/written_back_pages:32762
/sys/kernel/debug/zswap/reject_compress_poor:456
/sys/kernel/debug/zswap/reject_kmemcache_fail:0
/sys/kernel/debug/zswap/reject_alloc_fail:0
/sys/kernel/debug/zswap/reject_reclaim_fail:0
/sys/kernel/debug/zswap/pool_limit_hit:16253
We max out at ~2300 instances of bash & sleep which is even less than when running without any compression...?

zram

zram has been around for a while now and looked like the most promising contender. On a machine with 1GB RAM, I'd allocate 75% for our compressed swap device:
$ modprobe zram
$ echo 768M > /sys/block/zram0/disksize
$ mkswap /dev/zram0
$ swapon -p2 /dev/zram0
The machine is quite busy and it doesn't take long until it starts swapping to our new swap device1):
$ grep . /sys/block/zram0/{num_{reads,writes},{compr,orig}_data_size}
/sys/block/zram0/num_reads:1953540
/sys/block/zram0/num_writes:2333028
/sys/block/zram0/orig_data_size:671141888
/sys/block/zram0/compr_data_size:283552915
The compression ration is quite good, we're using only 42% of our precious real memory. I wanted to do some tests though to see if this can be measured in some kind of micro benchmark. In a 256MB Fedora Linux VM, we started GNU/bash along with /bin/sleep over and over again, let see how far we got:
$ i=0; while true; do
   [ $i -gt 2400 -a `expr $i % 50` = 0 ] && printf "$i  "
   bash -c "sleep 10000 &"; i=$((i+1))
done
2450  2500  2550  2600  2650  2700 ^C

$ pgrep -c sleep
2711

$ free -m
              total    used  free  shared  buff/cache  available
Mem:            241    192       3       0     45        5
Swap:           127    127       0
All memory is used up and starting any more programs is almost impossible now. This was repeatable, it always stopped around ~2700 instances and then came to a grinding halt. Let's try again with ZRAM:
$ pkill sleep
$ modprobe zram && echo 128M > /sys/block/zram0/disksize && mkswap /dev/zram0 && swapon -p2 /dev/zram0
$ i=0; while true; do
   [ $i -gt 2500 -a `expr $i % 100` = 0 ] && printf "$i  "
   bash -c "sleep 10000 &"; i=$((i+1))
done
2600  2700  2800  2900  3000  3100  3200 ^C

$ pgrep -c sleep
3201

$ free -m
              total    used  free  shared  buff/cache  available
Mem:            241    186       2       0     52        6
Swap:           255    209      46
With ZRAM enabled, it maxes out at ~3100, and makes it up to 3200 if we wait a bit longer (although we still seem to have 46MB free swap available). Again, this is also repeatable. And since we're only starting the same program over and over again, our compression ratio is even better1):
$ grep . /sys/block/zram0/{num_{reads,writes},{compr,orig}_data_size}
/sys/block/zram0/num_reads:55002
/sys/block/zram0/num_writes:70129
/sys/block/zram0/orig_data_size:128086016
/sys/block/zram0/compr_data_size:34523998
Btw, did someone say DriveSpace? :-)

1) Note: these sysfs entries will be deprecated in future kernel versions,