Installing a Xeon Phi coprocessor on Debian Squeeze

Written by erik - 29 july 2013

Today I installed a Xeon Phi coprocessor in a Debian box. The software is only supported for Red Hat systems so I was a little bit afraid. Fortunately it was easy. I took most of my information from the readme files of mpssd and from that guy. Thanks to him because he did most of the job.

Ok, let's get started. Hardware wise, you need a 16x PCI-express 3.0 slot and you need the BIOS to be configured to be compatible with devices with more than 4GB of ressource. There was a NVIDIA K20 with 6GB of memory plugged on the machine I used; but still the BIOS complained. A simple BIOS configuration allowed the machine boot. (Check the BIOS configuration under PCI-express.)

From here I did everything as root. Get the MPSS archive from Intel's website. (Mine was called mpss_gold_update_3-2.1.6720-15-rhel-6.3.tar). Untar it with tar xvf mpss_gold_update_*.tar. You'll get a bunch of RPM files. Install alien if you do not already have it with aptitude install alien. Convert all the RPM files except the -kmod- one into DEB files with for file in `ls *.rpm | grep -v kmod` ; do alien --scripts $file; done. Install them all with for file in `ls *.deb | grep -v kmod`;do dpkg --install $file; done.

Up to now, we installed all the files you need beside the kernel module. The kernel module that is provided is only for Red Hat system (with a kernel 2.6.32). My machine had a kernel 2.6.39, so I needed to do a little bit more work. Fortunately, the source for the kernel are available in the src/ directory. Convert that RPM file into a tgz with alien: alien -t intel-mic-kmod-*.src.rpm. When you uncompress it with tar zxvf intel-mic-kmod-*tgz you get a .spec file you do not care about and a tar.bz2 file that contain the kernel module source code. Uncompress it with tar jxvf intel-mic-kmod-*.tar.bz2. Compile it with make and install it with make install. You should now have a kernel module in /lib/modules/`uname -a`/extra/mic.ko. Somehow, the installation script does not run depmod to declare the module so you need to run it manually with depmod.

To use a Xeon Phi you also need a service running on the host. The files are already installed, but there is no startup script for the service. Go to /etc/init.d. We will use the skeleton script as a base. cp skeleton mpssd. You need to edit that file and perform the following modifications. Change the 3rd line to Provides: mpssd. Change line 22 to DESC="MIC service deamon". Change line 23 to NAME=mpssd. Comment line 25, you don't need arguments. At the beginning of function do_start, add the following line modprobe mic; [ -d "/var/lock/subsys" ] || mkdir /var/lock/subsys. The first part makes sure the mic kernel module is loaded. That should not be necessary because a udev rule was added earlier, but somehow I needed it. The second part ensures that the directory used by mpssd to store its lock file is properly created. Save and quit. Add the proper permission to the startup script with chmod +x mpssd. And at that point the service can be started with service mpssd start. Starting and stopping the service takes time (about a minute).

Now you need to be able to use the card. Typically one logs on the card using ssh. The card automatically takes the private IP address but it is not routable automatically from linux. So you need to add a network interface in /etc/network/interfaces:

iface mic0 inet static

Load the interface with ifup mic0. To log on the card, you will need to ssh to the card using key-based autentification. If you have local users to the machine (I do not think that NIS based ones are supported with that technique), you should make sure that you have an ssh key on the ones that want to use the MIC card. If not, just get them to run ssh-keygen. After that, you need to load the ssh keys on the MIC card but mpssd need to be off to do that. Just running service stop mpssd; micctrl --resetconfig; service start mpssd should do the trick. I am not sure at which point, but an entry to /etc/hosts was made. So ssh mic0 should just work; otherwise ssh

From there you are pretty much all set as you have a working installation of your Xeon Phi. You will certainly need ICC to do anything useful. Some people mount the host filesystem on the device, which can be pretty useful. I won't cover these because they are not really Debian specific and other people will certainly explain these better than me. NFS mounting is explained in MPSS readme file but I haven't tested it.

Happy hacking

EDIT on 2014/01/28: it seems that to use the offload mode, one need to have proper permission on /dev/mic/*. You can edit the udev rule in /etc/udev/rules.d/50-udev-scif.rules.

Classified in : Homepage, geek, programming, en - Tags : xeonphi, linux - 7 comments

Demenagement en Caroline du Nord

Written by erik - 12 april 2013

Bonjour a vous,

J'ai ete bien occupe ces derniers mois par mon travail et mes candidatures. Apres avoir envoye des dossiers a plus de 40 universites americaines et passe de nombreux entretiens, je suis arrive a un arrangement final. Je rejoindrai University of North Carolina at Charlotte dans le departement de Computer Science en tant qu'Assistant Professor. Je prendrais mon poste autour du 15 Aout et demenagerai de Columbus un peu avant.

Si vous passez dans le coin, faites moi signe!

PS: peut etre que je devrais renommer ce blog "Prof Saule in the US" :)

Classified in : Homepage, 3617mylife, fr - Tags : none - 2 comments

Rss feed of the articles




Last articles

Last comments