From VNX
Jump to: navigation, search

Distributed VNX (EDIV) Installation Guide

This document describes how to create a cluster of VNX hosts to be able to deploy VNX virtual scenarios distributed over them.

Figure 1: EDIV cluster


To be able to deploy VNX scenarios using EDIV distributed capabilities you need a cluster made of:

  • Cluster hosts. Two or more systems with a modern Linux distribution (Ubuntu 10.04 or newer recommended) and a processor with virtualization support (VT-x for Intel or AMD-V for AMD). Two Ethernet network interfaces are recommended, to be able to separate the cluster management traffic from the virtual machines interconnection traffic. Cluster hosts do not need to be homogeneous: cluster configuration and segmentation algorithms will take into account the characteristics of each host.
  • Most present systems are susceptible to be used in a cluster; the complexity of the scenarios a cluster is able to manage will of course depend on the resources (CPU and memory) of the hosts.

  • Cluster controller: a system with a modern Linux distribution (Ubuntu 10.04 or newer recommended). This system will run the software to control the distributed deployment of scenarios. It does not have any relevant requirements and the controller function can be integrated in one of the hosts.

Hardware installation

The only specific hardware requirement is the connection of the second network interface card (eth1 in the figure) of each host to the switch that will carry inter-virtual-machine traffic. The host ports in the switch have to be configured in "tagged" mode and a range of VLANs has to be assigned in the switch for VNX traffic (this range will be later configured in the cluster config file).

Once the switch configuration is ready, it is recommended to validate it by manually configuring one or more VLAN subinterfaces in each host and test connectivity among them. For example, to test connectivity through VLAN 100:

host1# vconfig add eth1 100
host1# ifconfig eth1.100
host2# vconfig add eth1 100
host2# ifconfig eth1.100
host2# ping

Software installation

Follow these steps to install EDIV/VNX over the cluster:

  • VNX Installation. Install VNX on the cluster controller and on every cluster host following the standard VNX installation procedure
    • Check that VNX is working correctly on every host by starting some of the VNX examples distributed.
    • Download the root filesystems needed and copy them to the controller and to every host
  • SSH Access to hosts. Cluster controller executes commands on hosts using SSH. Therefore, SSH access without password request is needed from the cluster controller to every host. You can configure it manually following available recipes (just google "SSH without password") or with the help of handy "ssh-copy-id" command:
    • From the controller, just execute:
    • for host in host1 host2 host3; do ssh-copy-id root@$host; done

      substituting "host1 host2 host3" by the list of hostnames or IP addresses of your cluster hosts.

  • Database Installation. The cluster controller uses a database to store persistent information. As the requirements are very low, any database that can be used through perl DBI module can be used. SQlite is the recommended one; mysql has been also tested succesfully.
    • Install database package and, optionally, install an application to manage the database (only needed in case of problems):
    • apt-get install sqlite 
      apt-get install sqliteman


      apt-get install mysql-server
      apt-get install mysql-query-browser
      apt-get install phpmyadmin
  • Cluster configuration. In the controller, edit VNX configuration file (/etc/vnx.conf) and ADD the cluster configuration sections. You can find an example configuration in /usr/share/vnx/etc/vnx.cluster.conf.sample:
    • Add example cluster configuration sections to your config file:
    • cat /usr/share/vnx/etc/vnx.cluster.conf.sample >> /etc/vnx.conf
    • Change them according to your data:
      • Database configuration: configure the parameters for database access
        • If you use SQlite (recommended) configure:
        • type = sqlite
          file = /var/run/vnx/cluster.db  # File where the database will be stored
        • If you use mysql configure:
        • type = mysql
          name = ediv          # Name of the database
          host = localhost     # Host where the database ser is
          port = 3306          # Port of the database server
          user = mydbuser      # User with permissions to create and access the db
          pass = mydbpasswd    # User password
      • VLAN configuration: configure the range of VLANs used to interconnect virtual machines deployed to different hosts:
      • [vlan]
        first = 1000      # First VLAN id to use
        last = 1039       # Last VLAN id to use
      • Cluster configuration: configure the names of the cluster hosts, the default segmentation algorithm and the management interfaces addresses. The host names defined here can be freely choosen by the user. They are the names used internally by EDIV to identify each host. The host names or IP addresses of each host are defined later in a specific section for each host.
      • #
        # Cluster hosts list
        host = host1 
        host = host2
        # segmentation algorithms available (see /usr/share/vnx/lib/seg-alg/): 
        #    RoundRobin, WeightedRoundRobin, DynamicWeightedRoundRobin 
        default_segmentation = RoundRobin
        # management network interfaces addresses
        mgmt_network =
        mgmt_network_mask = 16
      • Hosts configuration: configure the characteristics of each cluster host
      • #
        # Host properties
        #    Definition of each hosts parameters:
        #        - host_name (string): host IP name or IP address
        #        - mem (integer):      host RAM memory in Megabytes 
        #    - cpu (integer):          percentage speed relative to the 
        #                              fastest host of the cluster
        #    - max_vms (integer):      maximum virtual machines allowed to be
        #                              virtualized in a host (0 = unlimited).
        #    - if_name (string):       name of the network interface used to
        #                              interconnect virtual machines deployed
        #                              in different hosts
        #    Syntax example:
        #    [h1]
        #    host_name = host1.domain.org
        #    mem = 2048
        #    cpu = 100
        #    max_vms = 0
        #    if_name = eth0
        #    [h2]
        #    host_name = host2.domain.org
        #    mem = 1024
        #    cpu = 85
        #    max_vms = 8
        #    if_name = eth0
        host_name = host1.domain.org
        mem = 8192
        cpu = 200
        max_vms = 0
        if_name = eth1
        host_name = host2.domain.org
        mem = 4096
        cpu = 100
        max_vms = 0
        if_name = eth1
  • Database creation. Create the database with:
  • ediv --create-db

After installation is finished, you can check the cluster status with "--check-cluster" option. For example:

# ediv --check-cluster

Distributed Virtual Networks over LinuX (EDIV) -- http://www.dit.upm.es/vnx - vnx@dit.upm.es
Version: 2.0b.2189 (built on 29/05/2012_15:24)
  TMP dir=/tmp
  VNX dir=/root/.vnx
Using configuration file:     /etc/vnx.conf
-- Checking host1 availability...ACTIVE
-- Checking host2 availability...chopito is active
Hosts in cluster:
    pasito                   (active) vnx_ver=2.0b.2179,29/05/2012_02:39
    chopito                  (active) vnx_ver=2.0b.2179,29/05/2012_02:39

Using segmentation algorithm: RoundRobin
---- mode: check-cluster
---- Cluster status checked
---- Succesfully finished ----