PelicanHPC GNU Linux
Jan. 2012. v2.7 is available. This is a fairly minor update.
Octave 3.6.0, dynare 3.4.2, and pelicanhpc.pdf
is on the desktop. Other than these changes, the rest is just a sync
with Debian.
Dec. 2011: a note on building images: if you want to build a 64
bit image, you need to do it from a host that has 64 bit Linux
installed. You can build a 32 bit image on either 32 or 64 bit hosts.
If your host is 32 bit, change the ARCHITECTURE and KERNEL entries
under ### OPTIONS FOR IMAGE ### to the appropriate values (they are
commented out for your image building ease).
Nov. 2011: An academic paper style discussion of PelicanHPC is in the document pelicanhpc.pdf. If any reasonably reputable journal would like to publish this, please let me know.
I ran across a couple of videos about PelicanHPC. This first one is a screencast, using the newly released v2.4, and shows how to set up a virtual cluster using Virtualbox. This second one shows the setup of a real cluster, using the slightly older v2.2, but all of the information is relevant. Thanks to both!
22 Feb. 2011. The tutorial is available in Russian. Thanks to Ivan Bogatryev.
Linux Pro Magazine article on PelicanHPC (June 2009) issue is here
14 September 2009. There is a fast new download mirror, thanks to the Medical Informatics department of Heilbronn University and Heidelberg University. Thanks to Simon Streicher for setting this up!
30 June 2009. I finally managed to make a decent screencast that shows how to use PelicanHPC. It's 34MB, though. Check it here if you have the curiosity and bandwidth.
29 Sept 2008. If you were looking for ParallelKnoppix and you arrived here, you're in the right place. PelicanHPC is the continuation of the ParallelKnoppix project. The PK web page is no longer available, and re-directs here. ISO images for the last version of PK are achived here on the download page, but you're encouraged to use PelicanHPC instead.
PelicanHPC is an iso-hybrid (CD or USB) image that let's you set up a high performance computing cluster in a few minutes. A Pelican cluster allows you to do parallel computing using MPI. You can run Pelican on a single multiple core machine to use all cores to solve a problem, or you can network multiple computers together to make a cluster. The frontend node (either a real computer or a virtual machine) boots from the image. The compute nodes boot by PXE, using the frontend node as the server. All of the nodes of the cluster get their filesystems from the same image, so it is guaranteed that all nodes run the the same software. Packages can be added to all nodes using apt-get, thanks to aufs. The bootable image is created by running a single script, which takes advantage of the Debian Live infrastructure. It is very easy to create a custom version with new packages by adding package names to the script and then running it. PelicanHPC is creates a fairly bare-bones cluster, with few applications other than those relevant to the author's research and teaching. To use PelicanHPC for serious work in other disciplines, it will be necessary to create a customized version. The tutorial explains how to do this. It is not very difficult. Two examples of specialized distros that add to the PelicanHPC base are MOLA and birgHPC. These are very good examples of how it is intended for the make_pelican script to be used.
Pelican is created using tools from the Debian Live project. To make your own version you only need live-build (also deboostrap and rsync) and the make_pelican script, which is provided below. All of this is also available on the PelicanHPC images, so you can build a custom version while running PelicanHPC.
Contains the latest stable version of the OpenMPI implementation of MPI.
Available for 64 bit CPUs. An older version is available for 32 bit CPUs.
Contains extensive example programs using GNU Octave. Also has the Linpack HPL benchmark.
Pelican releases and all testing is done using the stable (Squeeze) version of Debian GNU Linux as the base.
Here are some screenshots (click to enlarge):
I'd appreciate it if you would try to get a new release using bittorrent, if possible. Check for torrents here. When the torrent gets stale, use the normal download page or a mirror (see below). The make_pelican script and pre-built ISO images are always available on the normal download page. To use the script to make your own version, you need to install the live_helper package. Get the most recent version.
Germany:
http://download.mi.hs-heilbronn.de/pelicanhpc/
Provided by the Medical
Informatics
department
of
Heilbronn
University
and
Heidelberg
University.
http://download.semmel.org/PK/
username
"parallel"
password
"knoppix"
The main documentation is the tutorial, (also in Russian) which is a series of screenshots with comments. For those of you who fly by the seat of your pants, boot a computer using the CD, and follow the instructions that come up on the screen.
An academic paper style discussion
is in the document pelicanhpc.pdf.
If
any
reputable
journal
would
like
to
publish
this,
please
get in touch.
Warning: this is somewhat out of date. Screencast video that shows how to set up and use PelicanHPC. Warning - this is about 34MB.
User Forum: the main place to ask questions
How to use ksysguard as a cluster monitor. (Thanks to A. Balsa)
Article in Linux Pro Magazine from June 2009.
Please
let
me
know
if
you
use
this
software
to
produce
published
academic
work.
Some other distros for setting up a HPC cluster are mentioned here. They may be better choices for certain tasks.
MOLA is a distro for virtual screening of small molecules. It was made by modifying make_pelican. Here's a tutorial that shows how it works.
birgHPC is a distro for bioinformatics built using an adapted version of make_pelican. It adds MPICH2, as well as a suite of tools for bioinformatics including GROMACS.
KestrelHPC is a system for setting up a cluster of netbooted clients, using a frontend machine that runs Debian or Ubuntu GNU/Linux. It is available as a .deb package. Now at its second version.
ABC Gnu/Linux is a distro for setting up a HPC cluster that works in both live and installed modes. I have tried it in live mode on a virtual cluster, and it works nicely, in my experience. It has the ganglia monitor, which is nice for seeing the details of the operation of the cluser. The frontend node automatically begins to operate as a dhcp server, using eth0, so be careful not to interfere with existing networks if you boot it up.
Cluster by
Night is another live CD approach to setting up a cluster for
MPI work. It works within an existing network, which is not the case
with PelicanHPC. It seems to be a very nice solution when it is not
possible to set up a private network.
This part is here to help convince
people that funding this project is worthwhile. Check
at
StatCounter (then click on Recent Visitor Map) to see an
updated version of this map:
(c)
2011 Michael Creel