{"id":2179,"date":"2019-05-08T13:00:59","date_gmt":"2019-05-08T12:00:59","guid":{"rendered":"https:\/\/chewett.co.uk\/blog\/?p=2179"},"modified":"2020-09-26T22:31:07","modified_gmt":"2020-09-26T21:31:07","slug":"raspberry-pi-cluster-node-15-a-more-complex-webserver","status":"publish","type":"post","link":"https:\/\/chewett.co.uk\/blog\/2179\/raspberry-pi-cluster-node-15-a-more-complex-webserver\/","title":{"rendered":"Raspberry Pi Cluster Node \u2013 15 A more complex webserver"},"content":{"rendered":"\n
\"\"<\/figure>\n\n\n\n

This tutorial focuses on improving the webserver to display information about the slaves connected to the master using python Bottle.<\/p>\n\n\n\n\n\n\n\n

Refactoring the Master Script<\/h2>\n\n\n\n

To start with the changes, I am going to focus on the master script. This is going to move the changes into a new class.<\/p>\n\n\n

\nclass RpiMaster:\n\n     def __init__(self, socket_bind_ip, socket_port):\n        self.socket_bind_ip = socket_bind_ip\n        self.socket_port = socket_port\n        self.connected_clients = {}\n<\/pre><\/div>\n\n\n
This class will hold some information about the connected clients, in addition to the bind IP address and port number.<\/p>\n\n\n\n
The main method in this class will be start()<\/code> which will be called to begin setting up the sockets and allow slaves to connect.<\/p>\n\n\n
\n def start(self):\n        logger.info("Starting script...")\n\n        listening_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)\n        listening_socket.bind((self.socket_bind_ip, self.socket_port))\n\n        listening_socket.listen(10)  # listen to 10 connects\n        while True:\n            (clientsocket, address) = listening_socket.accept()\n            logger.info("Got client at {address}".format(address=address))\n\n            rpi_client = RpiClusterClient(self, clientsocket, address)\n            self.connected_clients[rpi_client.uuid] = rpi_client\n            rpi_client.start()\n<\/pre><\/div>\n\n\n
This takes much of the code from the old basic_master<\/code> file and wraps it in the class.  Some changes here are pulling a UUID created from the RpiClusterClient<\/code> and storing this in an dictionary for easy access.<\/p>\n\n\n\n
By combing this data into an object some helper methods can be created to help access the data and remove it  when a client disconnects.<\/p>\n\n\n
\n def remove_client(self, rpi_client):\n        del self.connected_clients[rpi_client.uuid]\n\ndef get_slave_details(self):\n    slave_details = {}\n    for uuid in self.connected_clients:\n        slave_details[uuid] = {\n            "uuid": uuid,\n            "address": str(self.connected_clients[uuid].address[0]) + ":" + str(self.connected_clients[uuid].address[1]),\n\n         }\n\n     return slave_details\n<\/pre><\/div>\n\n\n
Here the two helper functions are added to this class. The first remove_client<\/code> removes the client from the connected_clients<\/code> dictionary. This is called once the client has disconnected.<\/p>\n\n\n\n
The second is used to get some basic information about all currently connected clients. This will be used by the slaves to request information about the other connected clients.<\/p>\n\n\n\n
Changing the RpiClusterClient to assign UUIDs<\/h2>\n\n\n\n
Instead of using a random int that may be liable to collisions the RpiClusterClient<\/code> is going to be changed to assign UUID’s to each connecting slave.<\/p>\n\n\n\n
This changes the constructor to the following code.<\/p>\n\n\n
\n    def __init__(self, master, clientsocket, address):\n        threading.Thread.__init__(self)\n        self.uuid = uuid.uuid4().hex\n        self.master = master\n        self.clientsocket = clientsocket\n        self.address = address\n        self.node_specifications = None\n<\/pre><\/div>\n\n\n
In addition to randomly creating a uuid, this accepts the RpiMaster object so it can get information from the master.<\/p>\n\n\n\n
The node_specifications<\/code> variable is set to None<\/code> and will be filled in once the slave connects. This then changes the code that receives the computer details to instead of printing it out, also save it. <\/p>\n\n\n
\nelif message['type'] == 'computer_details':\n    self.node_specifications = message['payload']\n    logger.info("Received Computer specifications: " + json.dumps(self.node_specifications))\n<\/pre><\/div>\n\n\n
In the body of the message parsing code I have also added two more pieces of information the slave can request.<\/p>\n\n\n
\nelif message['payload'] == 'uuid':\n    send_message(self.clientsocket, create_payload(self.uuid, "uuid"))\nelif message['payload'] == 'slave_details':\n    slave_details = self.master.get_slave_details()\n    send_message(self.clientsocket, create_payload(slave_details, "slave_details"))\n<\/pre><\/div>\n\n\n
Here the slave can relieve its currently assigned UUID and the details of all slaves connected to the master. The slave will be changed so that once it has connected it will request the UUID to save.<\/p>\n\n\n\n
The slave details call will be used by the webserver to receive all information about the slaves connected.<\/p>\n\n\n\n
The final change to the RpiClusterClient is to remove itself from the master once it disconnects. This uses the newly created remove_client<\/code> method.<\/p>\n\n\n
\nexcept DisconnectionException as e:\n    logger.info("Got disconnection exception with message: " + e.message)\n    logger.info("Shutting down slave connection handler")\n    self.master.remove_client(self)\n<\/pre><\/div>\n\n\n
By doing this the list of connected slaves will always be up to date.<\/p>\n\n\n\n
Changes to the RpiBasicSlaveThread<\/h2>\n\n\n\n
The only change needed to the RpiBasicSlaveThread class is to change its random ID to the UUID the master assigns.<\/p>\n\n\n\n
The constructor changes from creating the client_number to instead creating a placeholder for the UUID. This will later be requested from the master in the perform_action<\/code> method.<\/p>\n\n\n\n
This makes the start of the perform action method the following.<\/p>\n\n\n
\nlogger.info("Sending an initial hello to master")\nsend_message(self.sock, create_payload("uuid", "info"))\nmessage = get_message(self.sock)\nself.uuid = message['payload']\nlogger.info("My assigned UUID is " + self.uuid)\n<\/pre><\/div>\n\n\n
Once connected, the first call of the basic slave will be to request the UUID. This is then stored in the slave thread.<\/p>\n\n\n\n
Changing the RpiWebserverSlaveThread to retrieve slave details<\/h2>\n\n\n\n
Now we are able to request slave details from the master we need to change the RpiWebserverSlaveThread<\/code> to request these details. This changes the class to hold the following information.<\/p>\n\n\n
\nclass RpiWebserverSlaveThread(RpiBasicSlaveThread):\n\n    current_master_details = None\n    current_slave_details = None\n    webserver_data_updated = None\n\n    def perform_action(self):\n        logger.info("Now sending a keepalive to the master")\n        send_message(self.sock, create_payload("I am still alive, client: {num}".format(num=self.uuid)))\n        send_message(self.sock, create_payload("computer_details", "info"))\n        master_details = get_message(self.sock)\n        send_message(self.sock, create_payload("slave_details", "info"))\n        slave_details = get_message(self.sock)\n\n        RpiWebserverSlaveThread.current_master_details = master_details['payload']\n        RpiWebserverSlaveThread.current_slave_details = slave_details['payload']\n        RpiWebserverSlaveThread.webserver_data_updated = datetime.datetime.now()\n        time.sleep(5)\n\n<\/pre><\/div>\n\n\n
This will request both the master and slave details every 5 seconds and store it as a class variable. This information is then accessible in the Bottle webserver and can be served by the webserver.<\/p>\n\n\n\n
Changing the Bottle Webserver to serve slave details<\/h2>\n\n\n\n
The final changes needed are to change the Bottle webserver so that it serves the slave details. This involves changing the route as below.<\/p>\n\n\n
\n@route('\/')\ndef index():\n    return template("templates\/ClusterHomepage.html",\n                    masterinfo=json.dumps(RpiWebserverSlaveThread.current_master_details, indent=4, sort_keys=True),\n                    slaveinfo=json.dumps(RpiWebserverSlaveThread.current_slave_details, indent=4, sort_keys=True)\n                    )\n<\/pre><\/div>\n\n\n
Once the route has changed the final tweak that needs to be made is to add this information to the html file.<\/p>\n\n\n
\n<h2>Master Information<\/h2>\n<pre>{{masterinfo}}<\/pre>\n\n<h2>Slave Information<\/h2>\n<pre>{{slaveinfo}}<\/pre>\n<\/pre><\/div>\n\n\n
Summary of improving the Webserver information<\/h2>\n\n\n\n
In this tutorial the cluster is improved so that each slave is assigned a unique UUID. In addition the python Bottle webserver now displays information about the slaves.<\/p>\n\n\n\n
In the future this will be used to view the cluster as it processes tasks.<\/p>\n\n\n\n
Now the basics of the cluster is in place we can look at starting to define jobs that each node can run.<\/p>\n\n\n\n
The full code is available on Github<\/a>, any comments or questions can be raised there as issues or posted below.  <\/p>\n","protected":false},"excerpt":{"rendered":"
This tutorial focuses on improving the webserver to display information about the slaves connected to the master using python Bottle.<\/p>\n","protected":false},"author":1,"featured_media":2191,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"This week I talked about making a more complex webserver for the #RaspberryPi #Cluster project #DistributedComputing","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[122],"tags":[102,184,185,37],"class_list":["post-2179","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-raspberry-pi-cluster","tag-distributed-computing","tag-python","tag-rasbian","tag-raspberry-pi"],"wppr_data":{"cwp_meta_box_check":"No"},"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/05\/rpi_cluster_15_more_complex_webserver.jpg?fit=800%2C800&ssl=1","jetpack_shortlink":"https:\/\/wp.me\/p2toWX-z9","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":2090,"url":"https:\/\/chewett.co.uk\/blog\/2090\/raspberry-pi-cluster-node-13-abstracting-slave-code\/","url_meta":{"origin":2179,"position":0},"title":"Raspberry Pi Cluster Node \u2013 13 Abstracting Slave Code","author":"Chewett","date":"March 23, 2019","format":false,"excerpt":"This post builds on my previous posts in the Raspberry Pi Cluster series by abstracting the slave code so it is ready for more complex slaves. Why I am abstracting the Slave code As the system becomes more complex there will be a number of slaves performing different tasks. Currently\u2026","rel":"","context":"In "Raspberry Pi Cluster"","block_context":{"text":"Raspberry Pi Cluster","link":"https:\/\/chewett.co.uk\/blog\/category\/raspberry-pi-cluster\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/03\/rpi_cluster_13_abstracting_slave_code.jpg?fit=800%2C800&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/03\/rpi_cluster_13_abstracting_slave_code.jpg?fit=800%2C800&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/03\/rpi_cluster_13_abstracting_slave_code.jpg?fit=800%2C800&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/03\/rpi_cluster_13_abstracting_slave_code.jpg?fit=800%2C800&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":1001,"url":"https:\/\/chewett.co.uk\/blog\/1001\/raspberry-pi-cluster-node-04-configuration-files-configparser\/","url_meta":{"origin":2179,"position":1},"title":"Raspberry Pi Cluster Node \u2013 04 Configuration Files with ConfigParser","author":"Chewett","date":"February 10, 2018","format":false,"excerpt":"This post builds on the\u00a0third step to create a Raspberry Pi Cluster Node\u00a0to store our configuration settings in a config file. Here we move all the configuration settings in our script into a useful .cfg file using the python ConfigParser. Why use Configuration Files? When developing a system there will\u2026","rel":"","context":"In "Raspberry Pi Cluster"","block_context":{"text":"Raspberry Pi Cluster","link":"https:\/\/chewett.co.uk\/blog\/category\/raspberry-pi-cluster\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2018\/02\/rpi_cluster_04_config_parser.jpg?fit=800%2C800&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2018\/02\/rpi_cluster_04_config_parser.jpg?fit=800%2C800&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2018\/02\/rpi_cluster_04_config_parser.jpg?fit=800%2C800&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2018\/02\/rpi_cluster_04_config_parser.jpg?fit=800%2C800&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":2127,"url":"https:\/\/chewett.co.uk\/blog\/2127\/raspberry-pi-cluster-node-14-a-simple-webserver\/","url_meta":{"origin":2179,"position":2},"title":"Raspberry Pi Cluster Node \u2013 14 A simple webserver","author":"Chewett","date":"April 17, 2019","format":false,"excerpt":"This tutorial focuses on creating a simple webserver that displays the status of the master using python Bottle. What will the webserver be used for? To interact with the cluster I am planning on making a small set of webpages. Initially these will just display information about the cluster but\u2026","rel":"","context":"In "Raspberry Pi Cluster"","block_context":{"text":"Raspberry Pi Cluster","link":"https:\/\/chewett.co.uk\/blog\/category\/raspberry-pi-cluster\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/04\/rpi_cluster_14_a_simple_webserver.jpg?fit=800%2C800&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/04\/rpi_cluster_14_a_simple_webserver.jpg?fit=800%2C800&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/04\/rpi_cluster_14_a_simple_webserver.jpg?fit=800%2C800&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/04\/rpi_cluster_14_a_simple_webserver.jpg?fit=800%2C800&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":1964,"url":"https:\/\/chewett.co.uk\/blog\/1964\/raspberry-pi-cluster-node-11-automatic-slave-reconnection\/","url_meta":{"origin":2179,"position":3},"title":"Raspberry Pi Cluster Node \u2013 11 Automatic Slave Reconnection","author":"Chewett","date":"January 30, 2019","format":false,"excerpt":"This post builds on\u00a0my previous posts in the Raspberry Pi Cluster series\u00a0by modifying the slave to automatically reconnect to the master when the connection is lost. Making the Slave automatically rejoin the master There are a lot of situations where the slave may fail to communicate with the master. These\u2026","rel":"","context":"In "Raspberry Pi Cluster"","block_context":{"text":"Raspberry Pi Cluster","link":"https:\/\/chewett.co.uk\/blog\/category\/raspberry-pi-cluster\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/01\/rpi_cluster_11_automaic_slave_reconnection.jpg?fit=800%2C800&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/01\/rpi_cluster_11_automaic_slave_reconnection.jpg?fit=800%2C800&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/01\/rpi_cluster_11_automaic_slave_reconnection.jpg?fit=800%2C800&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/01\/rpi_cluster_11_automaic_slave_reconnection.jpg?fit=800%2C800&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":2002,"url":"https:\/\/chewett.co.uk\/blog\/2002\/raspberry-pi-cluster-node-12-automatic-start-with-rc-local\/","url_meta":{"origin":2179,"position":4},"title":"Raspberry Pi Cluster Node \u2013 12 Automatic Start with rc.local","author":"Chewett","date":"March 6, 2019","format":false,"excerpt":"This post builds on\u00a0my previous posts in the Raspberry Pi Cluster series by creating a script which automatically starts the slave and master with rc.local. What is the rc.local file? The rc.local file is used on many Linux operating systems to schedule operations to run after services are started on\u2026","rel":"","context":"In "Raspberry Pi Cluster"","block_context":{"text":"Raspberry Pi Cluster","link":"https:\/\/chewett.co.uk\/blog\/category\/raspberry-pi-cluster\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/02\/rpi_cluster_12_automatic_start.jpg?fit=800%2C800&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/02\/rpi_cluster_12_automatic_start.jpg?fit=800%2C800&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/02\/rpi_cluster_12_automatic_start.jpg?fit=800%2C800&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/02\/rpi_cluster_12_automatic_start.jpg?fit=800%2C800&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":881,"url":"https:\/\/chewett.co.uk\/blog\/881\/raspberry-pi-cluster-node-02-packaging-common-functionality\/","url_meta":{"origin":2179,"position":5},"title":"Raspberry Pi Cluster Node \u2013 02 Packaging common functionality","author":"Chewett","date":"November 29, 2017","format":false,"excerpt":"This post builds on the first step to create a Raspberry Pi Cluster node to package the common functionality that will be shared between master and clients. Packaging common functionality With the Raspberry Pi Cluster project there will be a number of things all nodes will do. This can lead\u2026","rel":"","context":"In "Raspberry Pi Cluster"","block_context":{"text":"Raspberry Pi Cluster","link":"https:\/\/chewett.co.uk\/blog\/category\/raspberry-pi-cluster\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/11\/rpi_cluster_02_packaging.jpg?fit=800%2C800&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/11\/rpi_cluster_02_packaging.jpg?fit=800%2C800&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/11\/rpi_cluster_02_packaging.jpg?fit=800%2C800&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/11\/rpi_cluster_02_packaging.jpg?fit=800%2C800&ssl=1&resize=700%2C400 2x"},"classes":[]}],"_links":{"self":[{"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/posts\/2179","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/comments?post=2179"}],"version-history":[{"count":7,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/posts\/2179\/revisions"}],"predecessor-version":[{"id":2664,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/posts\/2179\/revisions\/2664"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/media\/2191"}],"wp:attachment":[{"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/media?parent=2179"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/categories?post=2179"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/tags?post=2179"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}