Passthrough

As soon as you start a serious Xen project, you realize that the container in which your dom-Us reside are too small. You want to use computer periphery. This is no problem as long as the device you are thinking of can be abstracted, like your network card, but if you are actually planning on using some physical device, it gets more complicated. The Xen hypervisor has total control over your computer, but as for most hardware the hypervisor doesn’t know (and also shouldn’t know, because this would only make the hypervisor more vulnerable) how to use a specific device, the control over  the device is passed onto the dom-0. This is where you will have to get it.

For some (!) USB devices there is USB 1.1 speed passthrough , if your guest is a HVM guest. For all of your guests, there is an option called PVUSB, but this one requires special drivers in the dom-U and the dom-0. These drivers work for the old Xenlinux kernels, but as of now they are not rewritten for the vanilla kernel. This makes USB passthrough useless for a lot of people, but you need to keep in mind that a lot of Xen developement is done for professional environments and USB is more of a desktop thing.

If you are disappointed, let me tell you that there is a simple solution that works for most cases. If you can’t pass a single USB port/device, why not passing an entire PCI(e) card to your guest?

PCI(e) passthrough is a simple and powerful thing, as longs as your hardware has support for VT-d or IOMMU. If not, it gets a little bit more complicated, dangerous and limited . My advice for you is to bite the bullet and get new hardware. If you don’t have the right hardware, PCI(e) passthrough is limited to paravirtualized guests. I spent a lot of time getting such passthrough to work, only to realize that my piece hardware doesn’t work well with Linux (a common thing, even after two decades of Linux).

Please note, if you are planning on passing a graphics card, you need VGA passthrough too, as graphic cards have to support a lot of legacy stuff. VGA passthrough is not available for paravirtualized guests.

One last warning, Xens passthrough and (live-)migration features are mutually exclusive, for obvious resons.

If you want to pass a PCI(e) device to a dom-U, you first have to seize it from the dom-0. This means unbinding the driver currently used for the device and then binding the pciback driver. For this task, I extended the script found in the Xenwiki.

vim /etc/init.d/pciback

#!/bin/sh

### BEGIN INIT INFO

# Provides: pciback

# Required-Start: $local_fs

# Required-Stop: $local_fs

# Should-Start:

# Should-Stop:

# X-Start-Before: xen

# X-Stop-After:

# Default-Start: 2 3 4 5

# Default-Stop:

### END INIT INFO

. /lib/lsb/init-functions

scriptname=/etc/init.d/pciback

configfile=’/etc/xen/pciback.conf’

unbind()

{

# Unbind a PCI function from its driver as necessary

[ ! -e /sys/bus/pci/devices/$1/driver/unbind ] || \

echo -n $1 > /sys/bus/pci/devices/$1/driver/unbind

}

bind()

{

# Add a new slot to the PCI Backend’s list

echo -n $1 > /sys/bus/pci/drivers/pciback/new_slot

# Now that the backend is watching for the slot, bind to it

echo -n $1 > /sys/bus/pci/drivers/pciback/bind

}

case “$1″ in

start)

cat $configfile |while read line;do

echo “$line” | grep “^#” >/dev/null 2>&1 && continue

unbind $line

bind $line

done

;;

stop|status|restart|force-reload)

# As we don’t know which driver was bound before, there is not much we can do here

;;

*)

echo “Usage: $scriptname start” >&2

exit 3

;;

esac

Same here for Debian Squeeze xend instead of xen

# X-Start-Before: xend

Make the script executable

chmod 744 /etc/init.d/pciback

let insserv find a place for your script

insserv pciback

you should also create a config file

vim /etc/xen/pciback.conf

#this file contains the list of pci(e) devices, the pciback drive sould bind to at system startup.

# please provide the devices in long BDF notation.

# example:

# 0000:08:00.0

# NO EMPTY LINES!

#

The pciback driver might not be available, so load the necessary kernel module

modprobe xen_pciback

And make sure it gets loaded at dom-0 start

echo "xen_pciback" >> /etc/modules

if you have forgotten what echo and >> do, this simply adds a new line with xen_pciback, but if you don’t want to use cho and >> you can open the file with vim.

If you want to pass a device, you have to find out its address in BDF notation and add the a new line in our config file at /etc/xen/pciback.conf.

If you haven’t installed the pciutils already, do so now.

aptitude install pciutils

this will allow you to use lspci

lspci

find your device and note the number in front of the description. XX:XX.X should be the format, which is short for 0000:XX:XX.X

add the long address to the pciback config file

echo "0000:XX:XX.X"  >> /etc/xen/pciback.conf

or use vim, if you don’t feel comfortable using echo and the redirections. But I think if you made it this far, this shouldn’t shock you anymore.

Now let’s check if the script works! To see which driver is currently in use, type:

lspci -v -s XX:XX.X

the last line should read like this

Kernel driver in use: foo

now start the script, or even better restart your server.

/etc/init.d/pciback start

now read again:

lspci -v -s XX:XX.X

Kernel driver in use: pciback

If the pciback driver isn’t loaded, there is no point in continuing, find your mistake first.

If all went well the PCI device should now be ready for passthrough, so let’s ask xm!

xm pci-list-assignable-devices

When the device shows up, you can assign it to a guest, this can happen dynamically or better statically by adding an extra line to the dom-U config file. Use the short BDF notation here.

echo "pci = [ 'XX:XX.X' ] " >> /etc/xen/test.cfg

The only thing left, is starting your guest.