I have been playing around with condor since last week. It is available through apt-get so it was somehow straightforward to experiment with it. As it turns out the .deb package installed version 7.2.4 which is unfortunate because the support for transferring directories was only added in version 7.5.4 (with further releases fixing a slew of bugs related to that feature).

This post is not about condor (see my condor notes). It is about two days spent upgrading my development environment to get to the point where I can write code that uses that condor feature. The following paragraphs shows typical decisions and consequences that are an intrinsic part of software management.

The question is how do I get a version of condor above 7.5.4 installed on my system. The latest stable version of condor is, as of today, 7.6.6 so the closer the better. I am not picky. As long as I can get the required feature installed quickly and move on with my actual work the better. We are still figuring out how to leverage condor in our cloud infrastructure. Being picky will come in due time.

Starting an upgrade process

I had used ubuntu 10.04 (natty) thus far. Since it is almost two years old at this point, my first idea was to update to ubuntu 11.10 (oneiric), the 12.04 LTS still being in beta.

Ubuntu 11.10 provides linux kernel 3.0.0. Almost as expected the vmware tools did not compile with the new kernel headers. That is a problem because my development setup involves mounting my home directory on the OSX host read-only in the guest virtual machine, then ssh into that machine from Terminal.app. This way I can use reliable copy/paste shortcuts on the terminal session and write code with emacs on OSX. I used VMware Fusion 3.1 until then. Patching vmware tools wasn't beyond the realm of possibilities but I figured for less than fifty dollars, I could save me a lot of time and trouble down the line by just upgrading to VMware Fusion 4.1. I did. On the way I also installed OSX updates to avoid breaking my setup later on when another tool would require those updates. It is called an update but at 1.4Gb, it could as well be a complete new system. Anyway after a couple hours I was now running OSX 10.7.3 and VMware Fusion 4.1. Ubuntu 11.10 installed perfectly and the VMware tools worked without a glitch. None-the-less the condor version provided by oneiric was still 7.2.4.

In the meantime I also discovered that Fusion 4.1 moved some of the configuration files around. I first found out the mac addresses for my vms and then fixed their ip addresses in the dhcp configuration.

$ find /Library/Virtual\ Machines -name '*.vmx' -exec grep -H 'ethernet0.generatedAddress ' {} \;
$ diff prev /Library/Preferences/VMware\ Fusion/vmnet8/dhcpd.conf
+host armada {
+    hardware ethernet 00:0c:29:a6:1b:41;
+    fixed-address 192.168.144.10;
+    option host-name "armada";
+}

Finally I restarted the network interface by running:

$ /Applications/VMware\ Fusion.app/Contents/Library/vmnet-cli --stop
$ /Applications/VMware\ Fusion.app/Contents/Library/vmnet-cli --start

Going down the rat hole

At that point I could have decide to compile the latest condor sources but instead I figured I would see what version of condor is packaged in Fedora. So I went on to install Fedora 16 on VMware Fusion. After the usual modifications to be able to ssh into the Fedora virtual machine,

$ systemctl start sshd.service
$ systemctl enable sshd.service
$ diff -u prev /etc/sysconfig/iptables
 -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
+-A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
 -A INPUT -p icmp -j ACCEPT
$ iptables-restore < /etc/sysconfig/iptables

I was back to trying to install the vmware tools and try to get vmhgfs to work. That were things got ugly. Fedora 16 iso ships with linux kernel 3.1.0 but a first *yum update* will bring kernel 3.2.7 thus *yum install kernel-devel* will bring sources for kernel 3.2.7 in /usr/src/kernels. The choices are thus either to compile VMware Tools for kernel 3.1.0 and/or 3.2.7. The following patch managed to do that but unfortunately vmhgfs would load but mounting the host filesystem would fail (see grep vmware-tools /var/log/messages).

$ yum groupinstall "Development Tools"
$ yum install kernel-devel
$ yum install kernel-devel-3.1.0
$ tar zxvf /media/VMware\ Tools/VMwareTools-8.8.1-528969.tar.gz
$ cd vmware-tools-distrib
$ tar xvf lib/modules/source/vmhgfs.tar
# fix error: assignment of read-only member 'i_nlink'
$ diff -u prev vmhgfs-only/fsutil.c
- inode->i_nlink = 1;
+ set_nlink(inode, 1);
# fix error: expected ')' before numeric constant
$ diff -u prev vmhgfs-only/tcp.c
+ #include <linux/moduleparam.h>
$ tar cvf lib/modules/source/vmhgfs.tar vmhgfs-only
$ ./vmware-install.pl

As it turns out, there is an open source alternative to the vmware tools called open-vm-tools. Of course it comes as package in Ubuntu but does not seem to be part of Fedora repo. I downloaded open-vm-tools-8.6.0-425873.tar.gz (stable) and open-vm-tools-2011.12.20-562307.tar.gz (devel). The first one errored out while compiling with linux kernel 3.2.7 sources. The second one compiled fine and even managed to work. I was able to mount my host home directory on a Fedora 16 vm with:

# Make sure we are running the 3.2.7 kernel.
$ uname -a
$ yum groupinstall "Development Tools"
$ yum install kernel-devel
$ yum install glib2-devel pam-devel libX11-devel libXext-devel libXinerama-devel libXi-devel libXrender-devel libXrandr-devel libXtst-devel libXScrnSaver-devel uriparser-devel libpng-devel gtk2-devel gtkmm24-devel libdnet-devel gcc-c++ libicu-devel
# Not sure the following one help but I did install it.
$ yum install dkms
$ cd open-vm-tools-2011.12.20-562307
$ ./configure
$ make
$ make install
$ mkdir -p /mnt/hgfs
$ /usr/local/bin/vmtoolsd &
# verify modules are loaded and running
$ lsmod | grep vm
$ ps -e | grep vm

I finally had Fedora 16 / VMware Fusion 4.1 / OSX 10.7.3 providing the same features I had working with my previous setup two days earlier. Where was I? Oh yeah, condor...

$ yum install condor
$ condor_version
7.7.3

Conclusion

VMware Fusion works almost seamlessly with Ubuntu. That is not the case with Fedora. Unfortunately condor is a lot better integrated with Fedora than Ubuntu.

VirtualBox is now an Oracle product. Since Oracle is also famous for "stealing" Redhat, there might be a chance a VirtualBox / Fedora combination works out of the box. I don't know. Maybe I will try when I am forced to chase down another upgrade path.