Vnx-rootfsubuntu

How to create a KVM Ubuntu root filesystem for VNX

Basic installation

Follow this procedure to create a KVM Ubuntu based root filesystem for VNX. The procedure has been tested with Ubuntu 9.10, 10.04, 10.10, 11.04, 12.04, 13.04, 13.10, 14.04, 14.10, 15.04, 15.10 and 16.04.

• Create the filesystem disk image:

qemu-img create -f qcow2 vnx_rootfs_kvm_ubuntu.qcow2 20G

• Get Ubuntu installation CD. For example:

wget ftp://ftp.rediris.es/mirror/ubuntu-releases/16.04/ubuntu-16.04-server-i386.iso
cp ubuntu-16.04-server-i386.iso /almacen/iso

Note: use 'server' or 'desktop' CD versions depending on the system you want to create.

• Create the virtual machine with:

vnx --create-rootfs vnx_rootfs_kvm_ubuntu.qcow2 --install-media /almacen/iso/ubuntu-16.04-server-i386.iso --mem 512M

Note: add "--arch x86_64" option for 64 bits virtual machines

• Follow Ubuntu installation menus to install a basic system with ssh server.
• Configure a serial console on ttyS0 (skip this step for 15.04 or later releases):

cd /etc/init
cp tty1.conf ttyS0.conf
sed -i -e 's/tty1/ttyS0/' ttyS0.conf

• Activate startup traces on serial console by editting /etc/default/grub file and setting the GRUB_CMDLINE_LINUX_DEFAULT variable to "console=ttyS0". Also change the boot menu timeout to 0 (sometimes virtual machines get stopped on the boot menu when starting on high loaded systems):

GRUB_CMDLINE_LINUX_DEFAULT="console=ttyS0"
GRUB_TIMEOUT=0
GRUB_RECORDFAIL_TIMEOUT=1

• Make grub process the previous changes:

update-grub

• Add a timeout to systemd-networkd-wait-online service to avoid long waits at startup. Edit /lib/systemd/system/systemd-networkd-wait-online.service and change the following line:

ExecStart=/lib/systemd/systemd-networkd-wait-online --timeout 20

• Finally, delete the net udev rules file and halt the system:

rm /etc/udev/rules.d/70-persistent-net.rules
halt -p

Configuration

• Restart the system with the following command:

vnx --modify-rootfs vnx_rootfs_kvm_ubuntu.qcow2 --update-aced --mem 512M

Note: add "--arch x86_64" option for 64 bits virtual machines Note: ignore the errors "timeout waiting for response on VM socket". 768M are needed if you are installing a root filesystem with desktop interface

• Access the system through the text console to easy the copy-paste of commands:

virsh console vnx_rootfs_kvm_ubuntu.qcow2

• Access the console and sudo root:

sudo su

• Update the system

apt-get update
apt-get dist-upgrade

• Install XML::DOM perl package and ACPI daemon:

apt-get install libxml-libxml-perl libnetaddr-ip-perl acpid

• For 17.10 or newer install ifupdown

apt-get install ifupdown

• Install VNX autoconfiguration daemon:

mount /dev/sdb /mnt/
perl /mnt/vnxaced-lf/install_vnxaced

Change 'sdb' by 'vdb' in case virtio drivers are being used.

• Edit /etc/network/interfaces file and comment all lines related to eth0, eth1, etc interfaces. Leave only the loopback (lo) interface.
• For Ubuntu 22.04, uninstall cloud-init:

apt remove --purge cloud-init cloud-initramfs-copymods cloud-initramfs-dyn-netconf cloud-guest-utils netplan.io

• Reduce networkd-wait timeout. Edit the systemd-networkd-wait-online.service:

sudo vim /etc/systemd/system/network-online.target.wants/systemd-networkd-wait-online.service

And in the [Service] block, add the following line:

TimeoutStartSec=5sec

• Optional: install graphical user interface.
• Minimal:

# recommended option
sudo apt-get install lubuntu-desktop

# old recipe not tested in later versions
sudo apt-get install xorg gnome-core gksu gdm gnome-system-tools gnome-nettool firefox-gnome-support

• Complete:

sudo apt-get install ubuntu-desktop

Note: to avoid nautilus being launched any time you remotely execute a command on the virtual machine using VNX (which interferes with the normal execution of commands), you should disable the start of programs when media insertion takes place. Go to "System settings->System->Details->Removable Media" and deselect the checkbox "Never prompt or start programs on media insertion".

• Optional: install other services:
• Apache server:

sudo apt-get install apache2
update-rc.d -f apache2 remove      # to avoid automatic start in old versions
systemctl disable apache2.service  # to avoid automatic start in new versions

• Other tools

sudo apt-get install traceroute
sudo apt-get install xterm   # needed to have the 'resize' tool to resize consoles 

• Create a file /etc/vnx_rootfs_version to store version number and informacion about modification:

VER=v0.25
OS=Ubuntu 16.04 32 bits
DESC=Basic Ubuntu 16.04 root filesystem without GUI

• Zero the image empty space to allow reducing the size of the image:

dd if=/dev/zero of=/mytempfile
rm -f /mytempfile

• Stop the machine with vnx_halt:

sudo vnx_halt

• Reduce the size of the image:

mv vnx_rootfs_kvm_ubuntu.qcow2 vnx_rootfs_kvm_ubuntu.qcow2.bak
qemu-img convert -O qcow2 vnx_rootfs_kvm_ubuntu.qcow2.bak vnx_rootfs_kvm_ubuntu.qcow2

If everything went well, your root filesystem will be ready to be used with VNX. You can make a simple test using the simple_ubuntu.xml scenario distributed with VNX.

Installing additional software

To install additional software or to modify your root file system, you just have to:

• Start a virtual machine from it:

vnx --modify-rootfs vnx_rootfs_kvm_ubuntu.qcow2

• Check network connectivity. Maybe you have to activate the network interface by hand:

dhclient eth0

Note: use "ip link show" to know which network interface to use.

• Do the modifications you want.
• Finally, halt the system using:

vnx_halt

Examples

• dhcp server and relay:
• Install dhcp3 packages:

apt-get install dhcp3-server dhcp3-relay

• Disable autostart (optional):

update-rc.d -f isc-dhcp-server remove
update-rc.d -f isc-dhcp-relay remove

Updating VNXACED

You can automatically update the VNXACE daemon with the following command:

vnx --modify-rootfs vnx_rootfs_kvm_ubuntu.qcow2 --update-aced -y

If VNXACE daemon is not updated automatically, you can do it manually by accessing the virtual machine console and type:

mount /dev/sdb /mnt/
perl /mnt/vnxaced-lf/install_vnxaced

Known problems

• Sometimes after restarting, the virtual machines stop at showing the grub menu and do not start until you manually choose one option. To avoid it, just follow the instructions here: http://www.linuxquestions.org/questions/linux-server-73/how-to-disable-grub-2-menu-even-after-server-crash-796562/. Beware that the changes you make to grub.cfg file are lost after executing "update-grub" command.
• In Ubuntu 12.04 Desktop, graphical commands execution does not work. Command execution fails with "ERROR: no user logged on display :0.0" (see /var/log/vnxaced.log). If you just open a "terminal" window, commands work correctly (does not work if you open other applications; only when you start a terminal...).
• Each time a cdrom is mounted (for example, whenever a command is executed on the virtual machine) the following error appears in the console:

Jul 27 22:33:31 vnx kernel: [ 4384.875886] ata1.01: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6
Jul 27 22:33:31 vnx kernel: [ 4385.291374] ata1.01: BMDMA stat 0x5
Jul 27 22:33:31 vnx kernel: [ 4385.493411] sr 0:0:1:0: [sr0] CDB: Read(10): 28 00 00 00 00 18 00 00 01 00
Jul 27 22:33:31 vnx kernel: [ 4385.493460] ata1.01: cmd a0/01:00:00:00:08/00:00:00:00:00/b0 tag 0 dma 2048 in
Jul 27 22:33:31 vnx kernel: [ 4385.493461]          res 01/60:00:00:00:08/00:00:00:00:00/b0 Emask 0x3 (HSM violation)
Jul 27 22:33:31 vnx kernel: [ 4386.263553] ata1.01: status: { ERR }

Despite of the error trace, the commands are executed correctly. This error does not appear on Ubuntu 9.10 filesystems.