Virtual Machines can be executed on any execution site with VMware or Xen (via libvirt). To do this, Condor must be informed of some details of the VM installation.
What follows is not a comprehensive list of the VM Universe options; rather, it is intended to serve as a starting point for those users interested in getting VM Universe up and running quickly. For further, more comprehensive coverage of the configuration options please refer to section 3.3.28.
Begin by installing the virtualization package according to the vendor's instructions. We have successfully used both VMware Server and Xen. If you are considering running on a Windows system, you will also need to install a Perl distribution; for this we have used ActivePerl successfully.
If you are considering Xen, then there are four things that must exist on a system to fully support it. First, a Xen kernel must be running on the execute machine. This running Xen kernel acts as Dom0, in Xen terminology, under which all VMs are started, called DomUs Xen terminology. Second, the virsh utility must be available, and its companion libvirtd and Xend services must be running. Third, a reasonably recent version of the mkisofs utility must be available, for creation of CD-ROM disk images. Fourth, the pygrub program must be available, for execution of VMs whose disks contain the kernel they will run.
There are a number of configuration parameters related to virtual machines. Some options are required, while others are optional. Here we only discuss those that are required.
First, you are required to specify the type of VM that is installed. For instance, the following tells Condor we are using VMware:
VM_TYPE = vmware
You are also required to specify the location of condor_vm-gahp and its log file. On a Windows installation, these options would look like this:
VM_GAHP_SERVER = $(SBIN)/condor_vm-gahp.exe VM_GAHP_LOG = $(LOG)/VMGahpLog
You must also provide a version string for the Virtual Machine software you are using:
VM_VERSION = server1.0.4
While required, this option does not alter the behavior of Condor. Instead, it is added to the ClassAd for the machine, so it can be matched against. This way, if future releases of VMware/Xen support new features that are desirable for your job, you can match on this string.
If you are using VMware you also need to set the location of the Perl executable. In most cases, however, the default value should suffice:
VMWARE_PERL = perl
This, of course, assumes the Perl executable is in the path. If this is not the case, then a full path to the Perl executable will be required.
The final required option is the location of the VMware control script. It's located in Condor's sbin directory:
VMWARE_PERL = $(SBIN)/condor_vm_vmware.pl
Xen configurations must set the location of the control script Condor uses to interact with Xen:
XEN_SCRIPT = $(SBIN)/condor_vm_xen.sh
The other required option not included in the default Xen configuration is XEN_DEFAULT_KERNEL : this is the kernel image that will be used in cases where the user does not specify one explicitly in their job submission. In most cases, this is can be the default kernel from which the system was booted. For instance, the following was used on a Fedora Core installation:
XEN_DEFAULT_KERNEL = /boot/vmlinuz-2.6.18-1.2798.fc6xen
There is one final option worth mentioning: XEN_DEFAULT_INITRD . It's not a required option, but if you do decide to use it, there are a few things that you should be careful with. Unlike the kernel image above, this image cannot be the stock one used to boot the system. The reason for this is that Xen requires several device drivers in DomUs: xennet and xenblk. This can be easily fixed by creating a new initrd using mkinitrd and loading the drivers into it.
Once the configuration options have been set, restart the condor_startd daemon on that host. For example:
> condor_restart -startd leovinus
The condor_startd daemon takes a few moments to exercise the VM capabilities of the condor_vm-gahp, query its properties, and then advertise the machine to the pool as VM-capable. If the set up succeeded, then condor_status will tell you the host is now VM-capable by printing the VM type and the version number:
> condor_status -vm leovinus
After a suitable amount of time, if this command does not give any output, then the condor_vm-gahp is having difficulty executing the VM software. The exact cause of the problem depends on the details of the VM, the local installation, and a variety of other factors. We can offer only limited advice on these matters:
For Xen, the VM Universe is only available when root starts Condor. This is a restriction currently imposed because root privileges are required to create a VM on top of a Xen kernel. Specifically, root is needed to properly use the virsh utility that controls creation and management of Xen guest virtual machines. This restriction may be lifted in future versions depending on features provided by the underlying tools, virsh, or upon Condor's direct support of Qemu VMs that do not require network access.