Raspberry Pi cluster

The Raspberry Pi Cluster project aims to test various aspects of clustered computing using a number of Raspberry Pi computers. Part of this follows the work of Professor Simon Cox of Southampton University in his work creating the Raspberry Pi Supercomputer.

Objectives of the Raspberry Pi Cluster

The objectives of the project are to address some of the more complicated aspects of clustered computing and bring them to a wider audience.

In a world where much of the data processing and storage is done in "the cloud" it is easy to forget these are just many computers connected together. This project aims to give an insight into how the cloud works and some of the particular problems it faces.

The Software behind the cluster

To build our cluster and illustrate some of the complexities of distributed computing I am going to write the software for the cluster. The language I writing it in is python for the wide range of libraries and the fact it is supported by default on the Raspberry Pi and many other linux operating systems.

As I create the software I will blog each stage and release the full source code on github. A full list of the current tutorials are available on this site.

Why choose a Raspberry Pi for the cluster

Before this project began I reviewed a number of development boards available to the public to try and build the cluster from.

One of the primary requirements for the cluster is that I wanted them to be physical machines instead of running many virtual machines to simulate one. The reason is that although many data centers run virtual machines which can fallover to any available physical machine it creates a layer of redundancy. I explicitly want to discuss various systems to cope with nodes going offline rather than let a smart piece of software manage it.

For my cluster I wanted to have, at a minimum, 5 nodes. This then set a limit to what I could purchase for the initial project. This meant that buying 5 intel galileo's would be potentially out of the question for the initial project as they are relatively expensive.

I choose the Raspberry Pi for it being relatively cheap, having good software support and a variety of output pins to use for both standard a computer output (HDMI) and an IoT sensor network (GPIO pins).

Current state of the Raspberry Pi Cluster

Currently my cluster is comprised of 7 nodes although two are currently not in the node balancer.

Their versions and any specific role is listed below.