{"id":738,"date":"2017-10-14T13:00:50","date_gmt":"2017-10-14T12:00:50","guid":{"rendered":"http:\/\/chewett.co.uk\/blog\/?p=738"},"modified":"2017-10-13T22:23:44","modified_gmt":"2017-10-13T21:23:44","slug":"raspberry-pi-cluster-github-repository","status":"publish","type":"post","link":"https:\/\/chewett.co.uk\/blog\/738\/raspberry-pi-cluster-github-repository\/","title":{"rendered":"Raspberry Pi Cluster Github Repository"},"content":{"rendered":"<p>This post is a summary of the plan for the Raspberry Pi Cluster project.<\/p>\n<p><!--more--><\/p>\n<h2>Progress so far<\/h2>\n<p>So far I have been creating a variety of scripts to see what you I can easily do with the cluster project. At the moment these scripts are spread across the nodes and not easily accessible.<\/p>\n<p>Current examples running on the cluster<\/p>\n<ul>\n<li>All nodes connected via sockets<\/li>\n<li>Reporting node statistics to master (cpu, ram, swap)<\/li>\n<li>Shared distributed filesystem<\/li>\n<li>Multi node\u00a0file download and organisation<\/li>\n<li>Distributed md5 hash generation<\/li>\n<li>Web UI management interface<\/li>\n<\/ul>\n<p>These have all been written as proof of concept scripts to ensure they work on the cluster. Now I have found a number of projects the cluster can easily perform I am writing them up.<\/p>\n<h2>The repository and plans<\/h2>\n<p>Using a GitHub\u00a0repository I will be describing steps to build up the cluster. This will go from a single node logging that it is alive to many nodes running many processes. Using the various examples I have written already I will describe how to build it using code examples.<\/p>\n<p>The Github Project for this is <a href=\"https:\/\/github.com\/chewett\/RaspberryPiCluster\" target=\"_blank\" rel=\"noopener\">https:\/\/github.com\/chewett\/RaspberryPiCluster<\/a><\/p>\n<p>Each blog post will link to the GitHub project with full code available. The blog posts will explain what we are doing and what it will be used for. Each subproject will build in the previous to build up a full codebase for our cluster.<\/p>\n<p>Making all of the information opensource will hopefully let others play with clustered computing on a small scale.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This post is a summary of the plan for the Raspberry Pi Cluster project.<\/p>\n","protected":false},"author":1,"featured_media":739,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[122],"tags":[76,101],"class_list":["post-738","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-raspberry-pi-cluster","tag-github","tag-raspberry-pi-cluster"],"wppr_data":{"cwp_meta_box_check":"No"},"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/10\/github_raspberry_pi_start.jpg?fit=800%2C800&ssl=1","jetpack_shortlink":"https:\/\/wp.me\/p2toWX-bU","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":1839,"url":"https:\/\/chewett.co.uk\/blog\/1839\/raspberry-pi-cluster-node-08-slave-helper-functions\/","url_meta":{"origin":738,"position":0},"title":"Raspberry Pi Cluster Node &#8211; 08 Slave Helper Functions","author":"Chewett","date":"December 19, 2018","format":false,"excerpt":"This post builds on\u00a0my previous posts in the Raspberry Pi Cluster series\u00a0by adding a number of slave helper functions.\u00a0 This update will begin the process of fully automating the slaves. Preparing the Slaves for Automation Before the slaves are ready to be fully automated there are a number of commands\u2026","rel":"","context":"In &quot;Raspberry Pi Cluster&quot;","block_context":{"text":"Raspberry Pi Cluster","link":"https:\/\/chewett.co.uk\/blog\/category\/raspberry-pi-cluster\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2018\/12\/rpi_cluster_08_slave_functions.jpg?fit=800%2C800&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2018\/12\/rpi_cluster_08_slave_functions.jpg?fit=800%2C800&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2018\/12\/rpi_cluster_08_slave_functions.jpg?fit=800%2C800&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2018\/12\/rpi_cluster_08_slave_functions.jpg?fit=800%2C800&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":901,"url":"https:\/\/chewett.co.uk\/blog\/901\/raspberry-pi-cluster-node-03-basic-node-communication-two-nodes\/","url_meta":{"origin":738,"position":1},"title":"Raspberry Pi Cluster Node \u2013 03 Basic node communication between two nodes","author":"Chewett","date":"December 9, 2017","format":false,"excerpt":"This post builds on the second step to create a Raspberry Pi Cluster Node to create a hello world application across two different scripts. Here we will create a socket connection between the two and use it to send a message to the master. Designing the client-master Raspberry Pi Nodes\u2026","rel":"","context":"In &quot;Raspberry Pi Cluster&quot;","block_context":{"text":"Raspberry Pi Cluster","link":"https:\/\/chewett.co.uk\/blog\/category\/raspberry-pi-cluster\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/12\/rpi_cluster_03_basic_communication.jpg?fit=800%2C800&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/12\/rpi_cluster_03_basic_communication.jpg?fit=800%2C800&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/12\/rpi_cluster_03_basic_communication.jpg?fit=800%2C800&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/12\/rpi_cluster_03_basic_communication.jpg?fit=800%2C800&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":1054,"url":"https:\/\/chewett.co.uk\/blog\/1054\/raspberry-pi-tutorial-website-online-now\/","url_meta":{"origin":738,"position":2},"title":"Raspberry Pi Tutorial Website online now!","author":"Chewett","date":"March 10, 2018","format":false,"excerpt":"This post talks about the new page on my website with linking to all my Raspberry Pi Cluster tutorials. Raspberry Pi Cluster Tutorial Webpage Now I have a couple Raspberry Pi Cluster tutorials I decided to link to all of them on my website. This will form the basis of\u2026","rel":"","context":"In &quot;Raspberry Pi Cluster&quot;","block_context":{"text":"Raspberry Pi Cluster","link":"https:\/\/chewett.co.uk\/blog\/category\/raspberry-pi-cluster\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2018\/03\/rpi_tutorial_website_online.jpg?fit=800%2C800&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2018\/03\/rpi_tutorial_website_online.jpg?fit=800%2C800&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2018\/03\/rpi_tutorial_website_online.jpg?fit=800%2C800&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2018\/03\/rpi_tutorial_website_online.jpg?fit=800%2C800&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":658,"url":"https:\/\/chewett.co.uk\/blog\/658\/distributed-computing-raspberry-pi-cluster\/","url_meta":{"origin":738,"position":3},"title":"Distributed computing on the Raspberry Pi Cluster","author":"Chewett","date":"September 23, 2017","format":false,"excerpt":"This post describes the planned setup for distributed computing on the Raspberry Pi Cluster. A basic distributed system A very basic distributed system will have one Raspberry Pi asking as a master machine. This will coordinate work among all of the slaves. Eventually, we will move away from having a\u2026","rel":"","context":"In &quot;Raspberry Pi Cluster&quot;","block_context":{"text":"Raspberry Pi Cluster","link":"https:\/\/chewett.co.uk\/blog\/category\/raspberry-pi-cluster\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/09\/distributed_computing_design.jpg?fit=800%2C800&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/09\/distributed_computing_design.jpg?fit=800%2C800&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/09\/distributed_computing_design.jpg?fit=800%2C800&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/09\/distributed_computing_design.jpg?fit=800%2C800&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":2780,"url":"https:\/\/chewett.co.uk\/blog\/2780\/raspberry-pi-cluster-node-18-raspberry-pi-temperature-monitoring\/","url_meta":{"origin":738,"position":4},"title":"Raspberry Pi Cluster Node \u2013 18 Raspberry Pi Temperature Monitoring","author":"Chewett","date":"February 20, 2021","format":false,"excerpt":"This post builds on\u00a0my previous posts in the Raspberry Pi Cluster series\u00a0by starting to log temperature with the RaspberryPiVcgencmd Python module. Installing RaspberryPiVcgencmd RaspberryPiVcgencmd is a small python module aimed to control vcgencmd and allow programmatic access to it. This can be installed with the following command. python3 -m pip\u2026","rel":"","context":"In &quot;Raspberry Pi Cluster&quot;","block_context":{"text":"Raspberry Pi Cluster","link":"https:\/\/chewett.co.uk\/blog\/category\/raspberry-pi-cluster\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2021\/02\/raspi_cluster_18_cputemperature_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2021\/02\/raspi_cluster_18_cputemperature_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2021\/02\/raspi_cluster_18_cputemperature_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2021\/02\/raspi_cluster_18_cputemperature_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2021\/02\/raspi_cluster_18_cputemperature_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":2002,"url":"https:\/\/chewett.co.uk\/blog\/2002\/raspberry-pi-cluster-node-12-automatic-start-with-rc-local\/","url_meta":{"origin":738,"position":5},"title":"Raspberry Pi Cluster Node \u2013 12 Automatic Start with rc.local","author":"Chewett","date":"March 6, 2019","format":false,"excerpt":"This post builds on\u00a0my previous posts in the Raspberry Pi Cluster series by creating a script which automatically starts the slave and master with rc.local. What is the rc.local file? The rc.local file is used on many Linux operating systems to schedule operations to run after services are started on\u2026","rel":"","context":"In &quot;Raspberry Pi Cluster&quot;","block_context":{"text":"Raspberry Pi Cluster","link":"https:\/\/chewett.co.uk\/blog\/category\/raspberry-pi-cluster\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/02\/rpi_cluster_12_automatic_start.jpg?fit=800%2C800&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/02\/rpi_cluster_12_automatic_start.jpg?fit=800%2C800&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/02\/rpi_cluster_12_automatic_start.jpg?fit=800%2C800&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/02\/rpi_cluster_12_automatic_start.jpg?fit=800%2C800&ssl=1&resize=700%2C400 2x"},"classes":[]}],"_links":{"self":[{"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/posts\/738","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/comments?post=738"}],"version-history":[{"count":1,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/posts\/738\/revisions"}],"predecessor-version":[{"id":740,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/posts\/738\/revisions\/740"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/media\/739"}],"wp:attachment":[{"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/media?parent=738"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/categories?post=738"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/tags?post=738"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}