Raspberry Pi cluster Tutorials
In building the cluster I have documented each step and written them up as tutorials. These explain why and how I am building the software portion of the cluster.
Tutorial 01 - Logging Liveness
In the first tutorial I talked about setting up a simple python script to log data to a file every 10 seconds.
This will form the basis of logging what each node in the cluster is doing. One of the requirements is that the logger should work for multi-threaded processes. This is why I introduce the standard python library "logging".
The full details for Tutorial 01 - Logging Liveness is available on my blog
Tutorial 02 - Packaging common functionality
This second post builds on the first by packaging up the logging code we created last time.
As I am planning on running the cluster with a number of scripts for the server and client I need to be able to share the code. This will remove duplication and means we have standard ways of doing common operations. We will start by creating the common package structure to hold our code.
The full details for Tutorial 02 - Packaging common functionality is available on my blog
Tutorial 03 - Basic node communication between two nodes
In this tutorial I build a very basic one way communication method to send data from the slave to a master.
The basis of any cluster is being able to collaborate and communicate between nodes. This initial implementation of this will create a connection between a master and slave node and send data from slave to master. The master will print out any messages recieved from the slave. The slave will send a message every 10 seconds with a message.
The full details for Tutorial 03 - Basic node communication between two nodes is available on my blog
Tutorial 04 - Configuration Files with ConfigParser
Here I am using moving some of the configuration data from being hardcoded into the scripts into configuration files.
As I deploy the scripts to the nodes I don't want to have to change the scripts for each node. Instead I am going to use configuration files to store data specific to the node such as the name of the node. By using a specific configuration parsing module we can easily keep track of variables that are required to be stored.
The full details for Tutorial 04 - Configuration Files with ConfigParser is available on my blog
Tutorial 05 - Sending data with JSON
In this tutorial I am using JSON to send messages between the nodes in a machine readable format.
To make it easy to extend across various scripts I am packaging the functionality in a DataPackager module. This will allow me to modify how the cluster talks in the future as the implementation will be abstracted into this module.
The full details for Tutorial 05 - Sending data with JSON is available on my blog
Tutorial 06 - Sending Slave Details to the Master
This tutorial focuses on modifying the original data sending functions to use a more generic format.
This will allow the master to read what messages are being received and determine what action it needs to take for each message type.
Once this is done there will be a versatile system for receiving messages from a node and processing them.
In the future tutorials I will look into creating a two-way dialogue between the master and slave.
The full details for Tutorial 06 - Sending Slave Details to the Master is available on my blog