Raspberry Pi Cluster Node – 07 Sending data to the Slave

This post builds on my previous posts in the Raspberry Pi Cluster series by adding the ability to receive data from the master. In this update, I will be adding a way for the slave to request data and have it returned by the master.

Moving machine details into its own file

The first thing that I am going to do is move the machine details currently in the slave, to a separate file. In the future, this will allow obtaining more information about the node. However, I am moving it into a separate file for now so the Slave and Master can access the data.

For now, my machine file will include the following function and be accessible to both slave and master:

import psutil
import platform
import multiprocessing
import socket


def get_base_machine_info():
    return {
        'hostname': socket.gethostname(),
        'cpu_percent_used': psutil.cpu_percent(1),
        'ram': psutil.virtual_memory().total,
        'cpu': platform.processor(),
        'cpu_cores': multiprocessing.cpu_count()
    }

Configuring the Master to respond to information requests

In the master message handling while loop I am going to add a new message type to be handled. The master will listen to any messages with the type info and return any information the slave requests. The payload will define what type of information it is looking for and return it. The following segment of code is the new elif statement used for info type messages.

elif message['type'] == 'info':
    logger.info("Slave wants to know my info about " + message['payload'])
    if message['payload'] == 'computer_details':
        clientsocket.send(create_payload(get_base_machine_info(), "master_info"))
    else:
        clientsocket.send(create_payload("unknown", "bad_message"))

Here I am checking if the message type is info and logging a message that the slave is requesting information about the specific payload. Each payload will require different handling and more types will be added in the future. For now I have added a single type computer_details to match the message type the slave sends the master.

This calls the get_base_machine_info() function we earlier abstracted into a function, imported from the MachineInfo file.

If the slave requests information about an unknown type a bad_message payload is created and returned to the slave. Going forward this will be a standard payload type that will be handled differently.

Once the master has sent the requested data to the slave it continues to listen to messages and act on them.

Configuring the slave to request information from the Master

I have decided that as part of the initial hello to the master the slave will send its machine details, and request the same from the master. This is also refactored a little to move the piece of code handling the machine details into the above MachineInfo file. Below is the new handshake for the slave as it joins the cluster.

logger.info("Sending an initial hello to master")
sock.send(create_payload(get_base_machine_info(), 'computer_details'))
sock.send(create_payload("computer_details", "info"))

message = get_message(sock)
logger.info("We have information about the master " + json.dumps(message['payload']))

Once we have sent our machine info we request the machine info of the master. This is again performed using create_payload with the type info and payload computer_details.

Once we have sent the message asking for the master’s details we then use get_message to retrieve the reply from the master. This is used identically to how the master receives the slave’s messages and uses the same underlying shared code.

Summary

Now we have a structure which lets the master and slave communicate by sending and requesting information.

In the next post I will look at adding a few more payloads to let the master control the slave further. These will form the basis of the master requesting the slave to perform computation.

The full code is available on Github, any comments or questions can be raised there as issues or posted below.