Enable AUFS support for docker on SoYouStart (OVH)

30/10/2014 Eric Gazoni How to chrishoefliger

I'm using a SoYouStart server from OVH for a project where I need to deploy Docker containers.

I kept having errors of the like:

Error response from daemon: Cannot start container f5fa26a2b00e154dd851cfbf4459d03e996dea7b3fc18d9c51dc5b24bb132e32: fork/exec /var/lib/docker/init/dockerinit-1.0.1: invalid argument
After checking on Docker's GitHub, it seems it is related to devicemapper driver used by default by Docker when AUFS is not present in the kernel. How do I know I'm not using AUFS already ?
$ docker info
Containers: 8
Images: 24
Storage Driver: devicemapper
 Pool Name: docker-9:1-133602-pool
 Data file: /var/lib/docker/devicemapper/devicemapper/data
 Metadata file: /var/lib/docker/devicemapper/devicemapper/metadata
 Data Space Used: 1207.9 Mb
 Data Space Total: 102400.0 Mb
 Metadata Space Used: 2.2 Mb
 Metadata Space Total: 2048.0 Mb
Execution Driver: native-0.2
Kernel Version: 3.10.23-xxxx-std-ipv6-64
The two important bits of information here :
Storage Driver: devicemapper
and
Kernel Version: 3.10.23-xxxx-std-ipv6-64
this is the regular OVH home-made kernel, proposed on all their servers by default. The problem is that I need to get the linux-image-extra package installed in order to benefit from AUFS, but it won't load with the current kernel. I just need to load the stock Ubuntu server kernel and extras
$ apt-cache search linux-image-extra
linux-image-extra-3.13.0-24-generic - Linux kernel extra modules for version 3.13.0 on 64 bit x86 SMP
linux-image-extra-virtual - Transitional package.
linux-image-extra-3.13.0-27-generic - Linux kernel extra modules for version 3.13.0 on 64 bit x86 SMP
linux-image-extra-3.13.0-29-generic - Linux kernel extra modules for version 3.13.0 on 64 bit x86 SMP
linux-image-extra-3.13.0-30-generic - Linux kernel extra modules for version 3.13.0 on 64 bit x86 SMP
linux-image-extra-3.13.0-32-generic - Linux kernel extra modules for version 3.13.0 on 64 bit x86 SMP
linux-image-extra-3.13.0-33-generic - Linux kernel extra modules for version 3.13.0 on 64 bit x86 SMP
linux-image-extra-3.13.0-34-generic - Linux kernel extra modules for version 3.13.0 on 64 bit x86 SMP
linux-image-extra-3.13.0-35-generic - Linux kernel extra modules for version 3.13.0 on 64 bit x86 SMP
linux-image-extra-3.13.0-36-generic - Linux kernel extra modules for version 3.13.0 on 64 bit x86 SMP
linux-image-extra-3.13.0-37-generic - Linux kernel extra modules for version 3.13.0 on 64 bit x86 SMP
linux-image-extra-3.13.0-39-generic - Linux kernel extra modules for version 3.13.0 on 64 bit x86 SMP
$ apt-get install linux-image-extra-3.13.0-39-generic
It seems there are extra packages (containing AUFS) for kernel from 3.13 up until revision 39, so we take that one, and install the linux-image-extra- package, as it will fetch the base kernel automatically, because it depends on it. Is that all ? No. This will only install the newest kernel, we still need to make it load at boot time. In the base server image used by OVH, the home-made kernel is placed very low in the grub selection list, so it will get priority over our stock kernel. I was stuck for some time on that before I found the solution: tell grub to pick our newly installed kernel by default.
$ update-grub
Generating grub configuration file ...
Found linux image: /boot/bzImage-3.10.23-xxxx-std-ipv6-64
Found linux image: /boot/vmlinuz-3.13.0-39-generic
Found initrd image: /boot/initrd.img-3.13.0-39-generic
Found linux image: /boot/vmlinuz-3.13.0-37-generic
Found initrd image: /boot/initrd.img-3.13.0-37-generic
 No volume groups found
done
$ fgrep menuentry /boot/grub/grub.cfg
if [ x"${feature_menuentry_id}" = xy ]; then
 menuentry_id_option="--id"
 menuentry_id_option=""
export menuentry_id_option
menuentry "Ubuntu 14.04 LTS, OVH kernel 3.10.23-xxxx-std-ipv6-64" {
menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-62de63de-78ea-4741-8c93-2c05a6899ef6' {
submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-62de63de-78ea-4741-8c93-2c05a6899ef6' {
 menuentry 'Ubuntu, with Linux 3.13.0-39-generic' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-3.13.0-39-generic-advanced-62de63de-78ea-4741-8c93-2c05a6899ef6' {
 menuentry 'Ubuntu, with Linux 3.13.0-39-generic (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-3.13.0-39-generic-recovery-62de63de-78ea-4741-8c93-2c05a6899ef6' {
 menuentry 'Ubuntu, with Linux 3.13.0-37-generic' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-3.13.0-37-generic-advanced-62de63de-78ea-4741-8c93-2c05a6899ef6' {
 menuentry 'Ubuntu, with Linux 3.13.0-37-generic (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-3.13.0-37-generic-recovery-62de63de-78ea-4741-8c93-2c05a6899ef6' {
This is the configuration menu for GRUB. Now open your eyes: you'll notice that, on my server, I have the default OVH kernel listed as a "menuentry", while my newly installed kernel is in a "submenu" menuentry line. This is very important, because in most guides, people just assume it's going to be listed as a main entry, not in a sub-menu, and you need to take this into account when setting the default grub value:
$ vim /etc/default/grub
[... add the following line at the bottom of the file ...]
GRUB_DEFAULT="Advanced options for Ubuntu>Ubuntu, with Linux 3.13.0-39-generic"
$ reboot now
Yes, you have to mention GRUB to look in the sub-menu (the pattern is "submenu_name>submenu_entry", no space before and after the > sign), otherwise it will look in the main entries, find nothing, and boot on the first item of the list ... the OVH kernel, which is not what we want. On a side note, I read here that you could also move the grub entry for the OVH kernel, but this solution just hides the OVH main menuentry from GRUB, so there will be no "first item on the list" (GRUB does not look into submenus), thus the server will not boot anymore and you will have to reboot on the rescue kernel to fix things out. After the reboot, Docker is using AUFS:
$ uname -r
3.13.0-37-generic
$ docker info
Containers: 0
Images: 0
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Dirs: 0
Execution Driver: native-0.2
Kernel Version: 3.13.0-37-generic
WARNING: No swap limit support
Eric Gazoni

Eric is a software engineer, founder of Adimian, splitting his time between making his customers lives better, filling the paperworks and doing puzzles with his twin boys.