Raspberry Pi Cluster Node – 08 Slave Helper Functions

This post builds on my previous posts in the Raspberry Pi Cluster series by adding a number of slave helper functions.  This update will begin the process of fully automating the slaves.

Preparing the Slaves for Automation

Before the slaves are ready to be fully automated there are a number of commands I want it to be able to perform when needed. Some of these are obvious  such as being able to be rebooted and shut down.

The others are a little more complex, such as being able to update both the code it is running and operating system.

Slave Commands

Below I will describe some of the commands that I have written for the slave and how I am going to use them. All of these functions are going to be stored in the NodeTasks.py file.

Reboot

def reboot():
    subprocess.call(["sudo", "reboot"])

The reboot command is going to be used for whenever the node needs to be rebooted. Typically this will be called once it has been updated to fully ensure it is running the latest software.

Once the node has been rebooted it is expected to come back online and talk to the master. This will be handled in a future post.

Shutdown

def shutdown():
    subprocess.call(["sudo", "shutdown", "-h"])

The shutdown command is going to be used when I want to take the nodes offline for work. This will be rarely used but will be important for maintenance.

Once called a node should fully shut down so that its power can be removed. The node will be manually booted once maintenance is finished. Again it is assumed that it will automatically connect to the mater once booted.

Update Node

def update_node():
    node_update_log_1 = subprocess.check_output(["sudo", "apt-get", "update"])
    node_update_log_2 = subprocess.check_output(["sudo", "apt-get", "upgrade"])
    return [node_update_log_1, node_update_log_2]

For now it is assumed all nodes are running apt based package systems. This means that to update them we can run apt-get update, followed by apt-get upgrade. 

The output of these two commands are returned from this function. This was not done for the previous two functions because rebooting and shutting down will not return any data, as the node will have turned off.

Get Node Time

def get_node_time():
    return time.time()

It is important to make sure all nodes have the correct time. This is to ensure that the cluster is all running the same time. When messages are encrypted the time will become more important. This reasoning for this will be discussed in future blogposts.

Get Node Codebase Revision

def get_node_codebase_revision():
    #Note: This is not nesscarily the revision of the code currently running
    current_rev = subprocess.check_output(["git", "rev-parse", "HEAD"])
    return current_rev.rstrip("\r\n")

To start with it is assumed that the nodes will synchronise code using a shared git repository. This means its important to know what revision each node is running.

This command will call git to determine the current revision of the code. This however will only determine the current revision in the directory which may be different from the revision that is running.

This problem will be resolved and discussed in detail in a future blogpost.

Get Node Codebase Revision Time

def get_node_codebase_revision_time():
    node_rev = get_node_codebase_revision()
    rev_time = subprocess.check_output(["git", "show", "-s", "--format=%ct", node_rev])
    return int(rev_time.rstrip("\r\n"))

Since git does not have sequential revision numbers like SVN or Perforce you cannot quickly know whether one revision is ahead of another. However by using git show you can obtain the time of the commit.

This will be used by the master to determine which node has the later codebase, and update other nodes not on this version. This uses the previously defined function get_node_codebase_revision to get the current codebase revision.

Update Node Codebase

def update_node_codebase():
    update_log = subprocess.check_output(["git", "pull"])
    return update_log

The final command is to update the node codebase using git.  As mentioned earlier it is assumed that all nodes are deployed from a git repository.

By using the command git pull the code deployed on the node will be updated from the master git repository. It is assumed here that the node will be configured so no password is needed to update the git repository.

Therefore to update the nodes we should just be able to run git pull and the node will successfully be updated. This will be used in conjunction with rebooting the node to update it to the latest version.

Summary of newly added functions

Now we have a number of functions available to the Nodes to able to control them. At this stage they are still not ready to run master defined scripts but these will form the basis of the automation.

The next tutorial will focus on converting the master script to accept any number of slave nodes.

The full code is available on Github, any comments or questions can be raised there as issues or posted below.