Virtual Machines can be executed on any execution site with WMware or Xen (via libvirt). To do this, Condor must be informed of some details of the VM installation.
What follows is not a comprehensive list of the VM Universe options; rather, it is intended to serve as a starting point for those users interested in getting VM Universe up and running quickly. For further, more comprehensive coverage of the configuration options please refer to section 3.3.27.
Begin by installing the virtualization package according to the vendor's instructions. We have successfully used both VMware Server and Xen. If you are considering running on a Windows system, you will also need to install a Perl distribution; for this we have used ActivePerl successfully.
If you are considering Xen, then there are four things that must exist on a system to fully support it. First, a Xen kernel must be running on the execute machine. This running Xen kernel acts as Dom0, in Xen terminology, under which all VMs are started, called DomUs Xen terminology. Second, either the virsh or xm utilities must be available, and their companion libvirtd and Xend services must be running. Third, a reasonably recent version of the mkisofs utility must be available, for creation of CD-ROM disk images. Fourth, the pygrub program must be available, for execution of VMs whose disks contain the kernel they will run.
Condor's configuration file, condor_ config, includes several of the VM configuration options. Some options are required, while others are optional. Here we only discus those that are required.
First, you are required to specify the type of VM that is installed. For instance, the following tells Condor we are using VMware:
VM_TYPE = vmware
You are also required to specify the location of condor_ vm-gaph, as well as its configuration file. A basic condor_ vm-gaph configuration is shipped with Condor and can be copied to reside alongside condor_ config. For a Windows installation, these options may look like this:
VM_GAHP_SERVER = $(SBIN)/condor_vm-gahp.exe VM_GAHP_CONFIG = $(RELEASE_DIR)/condor_vmgahp_config.vmware
The final required configuration setting is the location where the condor_ vm-gaph should write its logs. By default, this is set to /dev/null on UNIX and Linux and NUL on Windows; however, if logging is required, it can be set to a specific path. The following will work on both Windows and UNIX/Linux systems, provided the VMGahpLogs directory exists:
VM_GAHP_LOG = $(LOG)/VMGahpLogs/VMGahpLog.$(USERNAME)
The next set of options that need to be set, belong in the condor_ vm-gahp's configuration file. Again, you must specify the kind of virtual machine software that is installed on the host:
VM_TYPE = vmware
You must also tell the condor_ vm-gaph which version of the software you are using:
VM_VERSION = server1.0.4
While required, this option does not alter the behavior of the condor_ vm-gaph. Instead, it is added to the ClassAd for the machine, so it can be matched against. This way, if future releases of VMware/Xen support new features that are desirable for your job, you can match on this string.
If you are using VMware you also need to set the location of the Perl executable. In most cases, however, the default value should suffice:
VMWARE_PERL = perl
This, of course, assumes the Perl executable is in the path. If this is not the case, then a full path to the Perl executable will be required.
The final required option is the location of the VMware control script. On Windows the following is valid:
VMWARE_PERL = C:\condor\bin\condor_vm_vmware.pl
On UNIX/Linux installations the path must be set to reflect the location of your installation.
Unlike its partner, VMware, Xen configurations must set which local users are eligible to execute the condor_ vm-gaph:
ALLOW_USERS = condor
This is a requirement because Xen requires root privileges and therefore condor_ vm-gaph is installed with setuid-root. The defaults is initially set to condor as is illustrated above; however, there may be reasons to add other users to this list.
Next you must set which program controls the Xen hypervisor. In most cases you will not need a full path, as /usr/sbin is in the root user's path; however, if you are not running the vm-gahp as root, then you will need to specify the complete path to the control program. For instance:
XEN_CONTROLLER = /usr/sbin/xm
Thirdly, the location of the control script must be set. For example:
XEN_SCRIPT = /usr/local/condor/sbin/condor_vm_xen.sh
The last required option not included in the default Xen configuration is XEN_DEFAULT_KERNEL : this is the kernel image that will be used in cases where the user does not specify one explicitly in their job submission. In most cases, this is can be the default kernel from which the system was booted. For instance, the following was used on a Fedora Core installation:
XEN_DEFAULT_KERNEL = /boot/vmlinuz-2.6.18-1.2798.fc6xen
There is one final option worth mentioning: XEN_DEFAULT_INITRD . It's not a required option, but if you do decide to use it, there are a few things that you should be careful with. Unlike the kernel image above, this image cannot be the stock one used to boot the system. The reason for this is that Xen requires several device drivers in DomUs: xennet and xenblk. This can be easily fixed by creating a new initrd using mkinitrd and loading the drivers into it.
Once the configuration options have been set, restart the condor_ startd daemon on that host. For example:
> condor_restart -startd leovinus
The condor_ startd daemon takes a few moments to exercise the VM capabilities of the condor_ vm-gahp, query its properties, and then advertise the machine to the pool as VM-capable. If the set up succeeded, then condor_ status will tell you the host is now VM-capable by printing the VM type and the version number:
> condor_status -vm leovinus
After a suitable amount of time, if this command does not give any output, then the condor_ vm-gahp is having difficulty executing the VM software. The exact cause of the problem depends on the details of the VM, the local installation, and a variety of other factors. We can offer only limited advice on these matters:
For Xen, the VM Universe is only available when root starts Condor. This is a restriction currently imposed because root priveleges are required to create a VM on top of a Xen kernel. Specifically, root is needed to properly use the virsh or xm utilities that control creation and management of Xen guest virtual machines. This restriction may be lifted in future versions depending on features provided by the underlying tools, virsh and xm, or upon Condor's direct support of Qemu VMs that do not require network access.