Xen-test-debian

Xen HOWTOS for Debian

Installing Xen basic packages in Debian

First of all, update the APT repository

 apt-get update

Xen Hypervisor

Install the Xen hypervisor. Note that the version may have changed: search for the current one using 'apt-cache search xen-hypervisor'.

 apt-get install xen-hypervisor-3.0.3-1-i386

In the case you get "CPU0 panic" errors when booting this hypervisor, try with the PAE version instead:

 apt-get install xen-hypervisor-3.0.3-1-i386-pae

Install the Xen kernel

 apt-get install linux-image-xen-686

Install Xen tools

 apt-get install xen-tools

Install the libc6-xen

I'm not sure what it's for or even if this installation could be omitted :) In some distributions (Ubutun) it is automatically installed as dependence of the previously introduced packages.

 apt-get install libc6-xen

Tunning Xen

Grub boot configuration

Kernel boot configuration should be upgraded when installing linux-image-xen-686 package. However, this automatic upgrade doesn't seem to work properly, so manual editing of /boot/grub/menu.lst is required (Debian has shown this behaviour; however, other distributions, like Ubuntu, seems not being suffering it).

Considering the linux-image-xen-686 installed /boot/vmlinuz-2.6.18-4-xen-686 (exact version may change, so check it), you should configure the following booting option in your /boot/grub/menu.lst file:

  title           Debian GNU/Linux, kernel 2.6.18-4-xen-686
  root            (hd0,0)
  kernel          /xen-3.0.3-1-i386-pae.gz
  module          /vmlinuz-2.6.18-4-xen-686 root=/dev/sda2 ro
  module          /initrd.img-2.6.18-4-xen-686
  savedefault
  boot

Configure networking

I needed to edit /etc/xen/xend-config.sxp and configure the network-script option, in order virtual machines get directly bridged to the external interface (eth1 in my case, change it appropriately):

  (network-script 'network-bridge netdev=eth1')

Configure loop

If you want to run many virtual machines simultaneously (more than 8) you have to use max_loop option for loop module (http://lists.xensource.com/archives/html/xen-users/2006-06/msg01150.html). For example:

   rmmod loop
   modprobe loop max_loop=64

You can also configure as default option for modprobe, editing /etc/modprobe.d/options and adding the following line. If the file doesn't exist, create it. (Thanks to Ramon Casellas for the tip :)

  options loop max_loop=64

Creating a virtual machine

Use the following configuration file (let's say test.cfg):

   kernel  = '/boot/vmlinuz-2.6.18-4-xen-686'
   ramdisk = '/boot/initrd.img-2.6.18-4-xen-686'
   memory  = '50'
   root    = '/dev/sda1 ro'
   disk    = [ "file:/usr/share/vnuml/filesystems/xen_fs,sda1,w" ]
   name    = 'occ0-v'
   vif     = [ '' ]
   on_poweroff = 'destroy'
   on_reboot   = 'restart'
   on_crash    = 'restart'

Comments:

• You can use the VNUML filesystems as xen_fs. It is highly recommendable to install kernel modules before. The commands are (exact version may change, so check it):

  mount /usr/share/vnuml/filesystems/xen_fs /mnt/loop -o loop
  cd /mnt/loop/lib/modules
  cp -va /lib/modules/2.6.18-4-xen-686 .
  cd /
  umount /mnt/loop

Warning:: differently from the usual configuration with VNUML, this filesystem doesn't work in COW mode, all modification performed during virtual machine operation will be stored in the xen_fs. In order to avoid changes in the root_fs file, I recommend you to use a copy ('cp root_fs xen_fs') and not to use directly the one you are using for your COW-ed based VNUML scenarios.

• The vif line configures a network interface in the virtual machine (named 'eth0'). I haven't already checked, but it is supposed that additional vif lines will add additional interfaces (eth1, eth2, etc.
• This configuration uses 50MB of memory
• This configuration does not use swap memory

In order to boot the machine (note that you don't have to specify the complete pathname of test.cfg if it is located in a default directory; it seems that xm looks in /etc/xen by default).

 xm create test.cfg -c

It also can take a while to boot the virtual machine. The '-c' is to get the booting output and loging in the same console where xm is invoked (this is very useful to know when the virtual machine has been booted). To come back to the host enviroment just press Ctrl+].

Uselful management operations

To list active domains:

 xm list

To connect to the virtual machine console:

 xm console <domU>

To finish with the virtual machine:

 xm destroy <domU>

To get performance info:

 xm top

Troubleshooting

• If you have problems related with memory when booting virtual machines, you may find useful the dom0_mem kernel switch. Edit your GRUB configuration so your kernel line looks like this:

 kernel /boot/xen-3.0.3-1-i386.gz dom0_mem=512M

changing 512 for the desired memory in your case, of course. Note that if you use a dom0_mem value too close to your actual physical RAM memory you will get a panic error when booting the host: as a hint dom0_mem<= phy_mem-32M (assuming the Xen hypervisor consumes less than 32MB).

• If you run Xen in a VMware virtual machine using NAT, you should use an special MAC prefix for your Xen virtual machine: 00:0C:29. Edit the /etc/xen/test.cfg file and edit the vif line so looks (for example):

 vif  = [ 'mac=00:0C:29:11:22:33' ]

the problem is quite similar to the one described in the following post in the vnuml-users list: https://lists.dit.upm.es/pipermail/vnuml-users/2005-September/000204.html

• If you get the following error when creating a new virtual machine: Error: I need 65536 KiB, but dom0_min_mem is 200704 and shrinking to 200704 KiB would leave only 43236 KiB free. This happens because of the sum of memory assigned to virtual machines is exceeding the limit reserve to dom0. To reduce that limit, edit the dom0-min-mem parameter in /etc/xen/xend-config.sxp file. For example, in order to use a 96MB limit:

  (dom0-min-mem 96)

You need to restart xend in order changes in /etc/xen/xend-config.sxp take effect:

  /etc/init.d/xend restart

Open issues

The strange messages that appear in the virtual machine consoles:

 4gb seg fixup, process syslogd (pid 1107), cs:ip 73:b7efc5f0
 4gb seg fixup, process syslogd (pid 1107), cs:ip 73:b7efc570
 4gb seg fixup, process klogd (pid 1113), cs:ip 73:b7e2e5f0
 4gb seg fixup, process klogd (pid 1113), cs:ip 73:b7eafd10
 printk: 38 messages suppressed.
 4gb seg fixup, process named (pid 1128), cs:ip 73:b7bc5c1c

The problem is described in the Xen FAQ. It is not only a "aesthetic" issue; it impacts performance.

In http://lists.xensource.com/archives/html/xen-users/2006-11/msg00056.html is suggested:

  echo 'hwcap 0 nosegneg' > /etc/ld.so.conf.d/libc6-xen.conf

However, it didn't solve the problem (at least in my case using Debian :) The following didn't work already:

http://wiki.xensource.com/xenwiki/DebianTlsLibcDiversion http://lists.xensource.com/archives/html/xen-users/2007-03/msg00424.html

Uselful references

• http://www.debian-administration.org/articles/396
• http://wiki.xensource.com/xenwiki/XenNetworking