A Ubuntu MPI Cluster (PART 1 - server setup)

Distributive computing has been around for a long time and there are many different ways of doing it. Today i will be looking at the Message Passing Interface (MPI) standard and will be using the MPICH2 sources provided from http://www.mcs.anl.gov/research/projects/mpich2/

On such a cluster you can run any application that has been designed for MPI, for some benchmarks i ran John The Ripper. (you can get the MPI version here: http://www.bindshell.net/tools/johntheripper/john-1.7.3.1-all-2-mpi8.tar.gz)

Results of the benchmark

No MPI:

Benchmarking: Traditional DES [128/128 BS SSE2]... DONE Many salts: 1986K c/s real, 1982K c/s virtual

MPI (4 cores over 2 machines)

Benchmarking: Traditional DES [128/128 BS SSE2]... DONE Many salts: 5151K c/s real, 5151K c/s virtual

As you can see its about a 3 fold improvement.

I have decided to go with Ubuntu for the base system as I'm most familiar with this distro.

Configure the Head Node:

Download the necessary packages

sudo apt-get install nfs-kernel-server openssh-server build-essential

Create the share for NFS

sudo mkdir /mirror

sudo mkdir /mirror/mpiu

run the following to add "/mirror *(rw,sync)" to /etc/exports

echo /mirror *(rw,sync) >> /etc/exports

restart NFS

sudo /etc/init.d/nfs-kernel-server restart

Create a user for the cluster

sudo useradd -d /mirror/mpiu

sudo passwd mpiu

su mpiu

sudo chown mpiu /mirror

Now we setup SSH

ssh-keygen -t dsa

*Don't set a passphrase

Add the key to Authoirzed keys

cd ~/.ssh

cat id_dsa.pub >> authorized_keys

Download MPICH2

wget http://www.mcs.anl.gov/research/projects/mpich2/downloads/tarballs/1.0.7/mpich2-1.0.7.tar.gz

mkdir /mirror/mpich2

tar -xvf mpich2-1.0.7.tar.gz

cd mpich2-1.0.7

./configure --prefix=/mirror/mpich2

make

sudo make install

Now add the following lines to /mirror/mpiu/.bashrc

export PATH=/mirror/mpich2/bin:$PATH

export PATH

LD_LIBRARY_PATH="/mirror/mpich2/lib:$LD_LIBRARY_PATH"

export LD_LIBRARY_PATH

Now edit /etc/environment and add the following

/mirror/mpich2/bin

Add the hostnames of the machines which are in the cluster into ~/mpd.hosts (currently there will be none if this is your server)

And then run the following commands

 echo secretword=password >> ~/.mpd.conf
sudo chmod 600 ~/.mpd.conf

And there you go, a MPI server running on ubuntu 8.04

A Ubuntu MPI Cluster (PART 2 - client setup) (coming soon!)