DealingWithiPXE

Problem

New hypervisors come with iPXE installed instead of gPXE. This is not a problem in itself since iPXE brings a lot of new interesting features compared to gPXE. But the virtio iPXE ROM that's provided by default doesn't behave as it should with VMs that have several NICs : it only tries to boot from the first found NIC and then it stops if the boot failed, instead of trying the other NICs. For us, it's a big problem since many VMs have two NICs : the first one is the public, and the second one is the private, and that's the one that will be used to pxe-boot.

Solution

Instead of downgrading to gPXE (the past pathetic workaround), there's a better way that consists in customizing the rom by embedding an iPXE script in it. The generic method to embed scripts is described here, and iPXE scripting is explained here. We needed a script that tries all the NICs, and we have found this cool iPXE menu on GitHubGist (it does a bit more than just trying all the NICs in turn, its default behaviour...).

Here is the detailed recipe of the solution that we applied on our hypervisors :

1. Git clone the iPXE code :

git clone git://git.ipxe.org/ipxe.git

2. Get into the ipxe git dir and download the script in it :

cd ipxe
wget ...

3. Build iPXE :

cd src && make EMBED=../nic-menu.ipxe && cd ..

Note : During our first attempt, we've got an error saying that lzma.h was missing. It was fixed by installing the package xz-devel.

4. Replace the old rom by the new one :

4.1 The new virtio rom with the embedded script in ipxe/src/bin/1af41000.rom 4.2 Copy this file to the hypervisor :

scp ipxe/src/bin/1af41000.rom hypervisor:/usr/share/qemu-kvm/1af41000_new.rom

4.3 Change a symlink so that the new rom is used (on the hypervisor) :

cd /usr/share/qemu-kvm
rm pxe-virtio.rom
ln -s 1af41000_new.rom pxe-virtio.rom