![\"\"](\"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/09\/single_master_many_slave.png\")
<\/a>As discussed in the previous post on Distributed Computing on the Raspberry Pi Cluster<\/a> there will be many slaves and a single master in the first stage.<\/p>\nThe image reproduced opposite shows the process each slave will go through. They will repeatedly ask for work, then perform the action given to them.<\/p>\n
If there is no work then they will idle for a period of time and then ask the master again for work.<\/p>\n
This cycle will form the basis of running distributed work on the cluster. Any live node will be able to come online and request work. Once the work is done it will report back to the master with the result.<\/p>\n
Then the loop will continue again requesting more work.<\/p>\n
\u00a0What we are going to use<\/h2>\n
This first script will create the event loop to check if there is work to do and add logging to keep track of what the node is doing.<\/p>\n
I am going to use the standard python logger to keep track of the work the node is doing. I will be configuring this to log data to both the console and a file. This will mean that I can view what the process is doing by observing the running console and ensure this data is saved to a file.<\/p>\n
Another really useful feature that the python logger has is that it is inherently thread safe. For the moment we are only using a single thread so this won’t matter. However in the future I plan to move towards making this applicable multi-threaded and having logging that handles this will be very useful.<\/p>\n
Setting up the python logger<\/h2>\n
The python logging module can be simply imported using\u00a0 To start with I am going to set up a formatter\u00a0for the logs.<\/p>\n This formatter will be used to define the format of the logs produced. This includes all the important information including the time, thread, level and the message logged. As discussed above while the current program is not multi-threaded this will be in the future so it will be helpful to keep track of which thread is writing to the log.<\/p>\n Now that I have a formatter\u00a0I can start to get the base logger object. This can be obtained using the below code.<\/p>\n Here I am getting the base logger and setting the level of messages that will be logged. By setting it to To actually save print out or save the log I need to attach handlers to the log. The most basic type of handler is the console handler. This works by printing all the information out to the console. I create one using the following code.<\/p>\n Once the handler has been created with In addition to logging to the console I want to keep a store of all messages in case I need to review what has been occurring. I am able to do this with the following code by creating a file handler.<\/p>\n Here as I am creating a logger to save to a file I supply a filename Now we have created our logger and various handlers to display the data we are ready to use it.<\/p>\n The logger object has a number of methods to allow us to log a message and data. The basic log function is<\/p>\n Here you log a specific message at a specific error level. This error level is an integer representing the severity of the message. The message along with any additional arguments will be used by your formatter to display and store the message. With log you have to specify a specific error level, the basic logging levels, as taken from the python site<\/a> are:<\/p>\n This means that a logger set to a default level of These work identically to Now I have the logger set up I can create a really basic Raspberry Pi Cluster worker. For this example it won’t perform any jobs but this structure will be used in the future.<\/p>\n I will define a function that will eventually be used to find work to perform. In this case I will just use my logger to log that there is no work to be done.<\/p>\nimport logging<\/code>.\u00a0Once we have imported this we have access to all the logging features needed for our Raspberry Pi Cluster.<\/p>\nlogFormatter = logging.Formatter(\r\n "%(asctime)s [%(threadName)-12.12s] [%(levelname)-5.5s] %(message)s")<\/pre>\n
logger = logging.getLogger()\r\nlogger.setLevel(logging.DEBUG)<\/pre>\n
DEBUG<\/code>\u00a0I ensure that all messages will be logged. This base level set is inherited by all child loggers and output handlers.<\/p>\nconsoleHandler = logging.StreamHandler()\r\nconsoleHandler.setFormatter(logFormatter)\r\nconsoleHandler.setLevel(logging.INFO)\r\nlogger.addHandler(consoleHandler)<\/pre>\n
logging.StreamHandler()<\/code>\u00a0I attach the formatter I created earlier. For the console output I have also set the logging level to INFO<\/code>. This means only messages of INFO<\/code> and higher will be printed to the console. The final but most important line attaches the handler to the logger we created earlier. This will receive the messages sent to the logger and will print them to the console.<\/p>\nfileHandler = logging.FileHandler("mainlogger.log")\r\nfileHandler.setFormatter(logFormatter)\r\nlogger.addHandler(fileHandler)<\/pre>\nmainlogger.log<\/code>. This will be the file that is written to each time the logger runs. By default each time this code runs the file will have log data appended to it. This means if the script is stopped and rerun old log data won’t be lost. Again like above we use our custom log formatter to format our logs in a specific way.<\/p>\nUsing the Python Logger<\/h2>\n
logger.log(level, message, args)<\/pre>\n
\n\n
\n \nLevel<\/th>\n Numeric value<\/th>\n<\/tr>\n<\/thead>\n \n CRITICAL<\/span><\/code><\/td>\n50<\/td>\n<\/tr>\n \n ERROR<\/span><\/code><\/td>\n40<\/td>\n<\/tr>\n \n WARNING<\/span><\/code><\/td>\n30<\/td>\n<\/tr>\n \n INFO<\/span><\/code><\/td>\n20<\/td>\n<\/tr>\n \n DEBUG<\/span><\/code><\/td>\n10<\/td>\n<\/tr>\n \n NOTSET<\/span><\/code><\/td>\n0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n DEBUG<\/code>\u00a0will log any messages set at level 10 and above. So that you don’t have to remember these values you can use the helper methods<\/p>\nlogger.debug(message, args)\r\nlogger.info(message, args)\r\nlogger.warning(message, args)\r\nlogger.error(message, args)\r\nlogger.critical(message, args)<\/pre>\n
logger.log()<\/code>\u00a0except they have preset the log level to the respective integer (based on the above table). For ease of use I will be using the\u00a0specific logger functions defined above.<\/p>\nCreating the basis of a Raspberry Pi Cluster worker<\/h2>\n
def find_work_to_do():\r\n logger.info("Node slave still alive")\r\n logger.info("Looking for work to run")\r\n return None<\/pre>\n