{"id":232,"date":"2016-10-16T21:53:37","date_gmt":"2016-10-16T20:53:37","guid":{"rendered":"http:\/\/chewett.co.uk\/?p=232"},"modified":"2017-05-22T00:24:35","modified_gmt":"2017-05-21T23:24:35","slug":"building-raspberry-pi-cluster","status":"publish","type":"post","link":"https:\/\/chewett.co.uk\/blog\/232\/building-raspberry-pi-cluster\/","title":{"rendered":"Building a Raspberry Pi cluster"},"content":{"rendered":"<p><a href=\"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2016\/10\/DSC_0525.jpg\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" data-attachment-id=\"254\" data-permalink=\"https:\/\/chewett.co.uk\/blog\/232\/building-raspberry-pi-cluster\/dsc_0525\/\" data-orig-file=\"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2016\/10\/DSC_0525.jpg?fit=600%2C338&amp;ssl=1\" data-orig-size=\"600,338\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"dsc_0525\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2016\/10\/DSC_0525.jpg?fit=300%2C169&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2016\/10\/DSC_0525.jpg?fit=600%2C338&amp;ssl=1\" class=\"wp-image-254 alignleft\" src=\"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2016\/10\/DSC_0525.jpg?resize=335%2C189\" alt=\"dsc_0525\" width=\"335\" height=\"189\" srcset=\"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2016\/10\/DSC_0525.jpg?w=600&amp;ssl=1 600w, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2016\/10\/DSC_0525.jpg?resize=300%2C169&amp;ssl=1 300w, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2016\/10\/DSC_0525.jpg?resize=50%2C28&amp;ssl=1 50w\" sizes=\"auto, (max-width: 335px) 100vw, 335px\" \/><\/a>Today&#8217;s blog marks the official start of my Raspberry Pi cluster project. Here I will be documenting the process of assembling,\u00a0building, and running a small cluster.\u00a0This blog will include both code snippets, \u00a0hardware diagrams and the results of various projects using the cluster.<\/p>\n<h1>Aims of the project<\/h1>\n<p>The\u00a0Raspberry Pi cluster will be looking into distributed computing from both a hardware and software perspective. Some of the smaller projects planned include:<\/p>\n<ul>\n<li>Youtube playlist syncing and download<\/li>\n<li>Dynamic task management and distribution<\/li>\n<li>MD5 hash calculators<\/li>\n<li>Realtime Reddit thread analysis<\/li>\n<\/ul>\n<h2>But why a\u00a0Raspberry Pi when it lacks power\/ram\/etc<\/h2>\n<p>Many of the tasks I am planning for the raspberry pi can be run faster and more easily on a single computer. My (5 year old) laptop will have more RAM and processing power than the initial cluster. If I was going for pure power I would run these tasks on a GPU. However the cluster is to experiment\u00a0with\u00a0the process of writing and working with a distributed environment.<\/p>\n<p>Part of the reason why I have chosen a raspberry pi is that it is readily available, relatively cheap, and there is a lot of \u00a0support for it to run Linux. I am planning on running <a href=\"https:\/\/www.raspberrypi.org\/blog\/introducing-pixel\/\">Raspbian Jessie PIXEL<\/a> to begin with with the possibility of switching to full on Debian or Fedora later.<\/p>\n<h2>First steps to a Raspberry Pi Cluster<\/h2>\n<p>Already I have one Raspberry Pi 1 Model B Rev 2 which I am going to start developing software for while I collect the other parts. During this time I will\u00a0search around for a number of raspberry Pi&#8217;s.\u00a0My preference will be\u00a0for\u00a0the older versions so that I may purchase more of them cheaply.<\/p>\n<p>Powering them is going to be an investigative point as\u00a0I dont want to power them using\u00a0multiple Raspberry Pi cables as these are costly\u00a0in larger numbers and will occupy lots of power sockets.<\/p>\n<p>As I am expecting to buy a couple different Raspberry Pi versions I am going to look for Micro SD cards which come with adapters. This is because Raspberry Pi 1 B models take a SD card\u00a0and later models take a Micro SD card.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Today&#8217;s blog marks the official start of my Raspberry Pi cluster project. Here I will be documenting the process of assembling,\u00a0building, and running a small cluster.\u00a0This blog will include both code snippets, \u00a0hardware diagrams and the results of various projects using the cluster. Aims of the project The\u00a0Raspberry Pi cluster will be looking into distributed [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[122],"tags":[100,102,37,101],"class_list":["post-232","post","type-post","status-publish","format-standard","hentry","category-raspberry-pi-cluster","tag-cluster","tag-distributed-computing","tag-raspberry-pi","tag-raspberry-pi-cluster"],"wppr_data":{"cwp_meta_box_check":"No"},"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p2toWX-3K","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":1054,"url":"https:\/\/chewett.co.uk\/blog\/1054\/raspberry-pi-tutorial-website-online-now\/","url_meta":{"origin":232,"position":0},"title":"Raspberry Pi Tutorial Website online now!","author":"Chewett","date":"March 10, 2018","format":false,"excerpt":"This post talks about the new page on my website with linking to all my Raspberry Pi Cluster tutorials. Raspberry Pi Cluster Tutorial Webpage Now I have a couple Raspberry Pi Cluster tutorials I decided to link to all of them on my website. This will form the basis of\u2026","rel":"","context":"In &quot;Raspberry Pi Cluster&quot;","block_context":{"text":"Raspberry Pi Cluster","link":"https:\/\/chewett.co.uk\/blog\/category\/raspberry-pi-cluster\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2018\/03\/rpi_tutorial_website_online.jpg?fit=800%2C800&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2018\/03\/rpi_tutorial_website_online.jpg?fit=800%2C800&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2018\/03\/rpi_tutorial_website_online.jpg?fit=800%2C800&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2018\/03\/rpi_tutorial_website_online.jpg?fit=800%2C800&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":738,"url":"https:\/\/chewett.co.uk\/blog\/738\/raspberry-pi-cluster-github-repository\/","url_meta":{"origin":232,"position":1},"title":"Raspberry Pi Cluster Github Repository","author":"Chewett","date":"October 14, 2017","format":false,"excerpt":"This post is a summary of the plan for the Raspberry Pi Cluster project. Progress so far So far I have been creating a variety of scripts to see what you I can easily do with the cluster project. At the moment these scripts are spread across the nodes and\u2026","rel":"","context":"In &quot;Raspberry Pi Cluster&quot;","block_context":{"text":"Raspberry Pi Cluster","link":"https:\/\/chewett.co.uk\/blog\/category\/raspberry-pi-cluster\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/10\/github_raspberry_pi_start.jpg?fit=800%2C800&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/10\/github_raspberry_pi_start.jpg?fit=800%2C800&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/10\/github_raspberry_pi_start.jpg?fit=800%2C800&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/10\/github_raspberry_pi_start.jpg?fit=800%2C800&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":658,"url":"https:\/\/chewett.co.uk\/blog\/658\/distributed-computing-raspberry-pi-cluster\/","url_meta":{"origin":232,"position":2},"title":"Distributed computing on the Raspberry Pi Cluster","author":"Chewett","date":"September 23, 2017","format":false,"excerpt":"This post describes the planned setup for distributed computing on the Raspberry Pi Cluster. A basic distributed system A very basic distributed system will have one Raspberry Pi asking as a master machine. This will coordinate work among all of the slaves. Eventually, we will move away from having a\u2026","rel":"","context":"In &quot;Raspberry Pi Cluster&quot;","block_context":{"text":"Raspberry Pi Cluster","link":"https:\/\/chewett.co.uk\/blog\/category\/raspberry-pi-cluster\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/09\/distributed_computing_design.jpg?fit=800%2C800&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/09\/distributed_computing_design.jpg?fit=800%2C800&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/09\/distributed_computing_design.jpg?fit=800%2C800&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/09\/distributed_computing_design.jpg?fit=800%2C800&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":307,"url":"https:\/\/chewett.co.uk\/blog\/307\/adding-a-creator-ci20-to-the-cluster\/","url_meta":{"origin":232,"position":3},"title":"Adding a Creator CI20 to the cluster","author":"Chewett","date":"October 17, 2018","format":false,"excerpt":"This post talks about the Creator CI20 and explains the differences between it and the Raspberry Pi. What is the Creator CI20? The Creator CI20 is a small Linux based computer that is similar to the Raspberry Pi. I have been looking at these for a while and decided to\u2026","rel":"","context":"In &quot;Electronics&quot;","block_context":{"text":"Electronics","link":"https:\/\/chewett.co.uk\/blog\/category\/electronics\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2018\/10\/creator_main_post_image.jpg?fit=800%2C800&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2018\/10\/creator_main_post_image.jpg?fit=800%2C800&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2018\/10\/creator_main_post_image.jpg?fit=800%2C800&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2018\/10\/creator_main_post_image.jpg?fit=800%2C800&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":2674,"url":"https:\/\/chewett.co.uk\/blog\/2674\/moving-the-raspberry-pi-cluster-to-a-raspberry-pi-4b-4gb\/","url_meta":{"origin":232,"position":4},"title":"Moving the Raspberry Pi Cluster to a Raspberry Pi 4B 4GB","author":"Chewett","date":"October 10, 2020","format":false,"excerpt":"Today I am talking about moving the Raspberry Pi Cluster primary node to a Raspberry Pi 4B 4GB model. I also discuss the improvements that the Raspberry Pi 4B has made over previous generations. Why move to a Raspberry Pi 4B Generally running the cluster does not require a large\u2026","rel":"","context":"In &quot;Raspberry Pi Cluster&quot;","block_context":{"text":"Raspberry Pi Cluster","link":"https:\/\/chewett.co.uk\/blog\/category\/raspberry-pi-cluster\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2020\/10\/raspberrypi_4_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2020\/10\/raspberrypi_4_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2020\/10\/raspberrypi_4_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2020\/10\/raspberrypi_4_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2020\/10\/raspberrypi_4_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":367,"url":"https:\/\/chewett.co.uk\/blog\/367\/overview-raspberry-pi-cluster\/","url_meta":{"origin":232,"position":5},"title":"Overview of the Raspberry Pi Cluster","author":"Chewett","date":"June 17, 2017","format":"gallery","excerpt":"Now I have the power supply and built the stacking system\u00a0I can show off the clusters initial state. Every Raspberry Pi in\u00a0the cluster Here are the 7 initial Pi's that are to become part of the cluster. Each have been given a codename to designate\u00a0its position in the cluster.\u00a0The Raspberry\u2026","rel":"","context":"In &quot;Raspberry Pi Cluster&quot;","block_context":{"text":"Raspberry Pi Cluster","link":"https:\/\/chewett.co.uk\/blog\/category\/raspberry-pi-cluster\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/05\/cluster_uncovered.jpg?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/05\/cluster_uncovered.jpg?resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/05\/cluster_uncovered.jpg?resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/05\/cluster_uncovered.jpg?resize=700%2C400 2x"},"classes":[]}],"_links":{"self":[{"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/posts\/232","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/comments?post=232"}],"version-history":[{"count":6,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/posts\/232\/revisions"}],"predecessor-version":[{"id":256,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/posts\/232\/revisions\/256"}],"wp:attachment":[{"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/media?parent=232"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/categories?post=232"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/tags?post=232"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}