{"id":613,"date":"2017-12-20T13:00:44","date_gmt":"2017-12-20T13:00:44","guid":{"rendered":"http:\/\/chewett.co.uk\/blog\/?p=613"},"modified":"2017-12-19T23:21:20","modified_gmt":"2017-12-19T23:21:20","slug":"backup-raspberry-pi-rsync","status":"publish","type":"post","link":"https:\/\/chewett.co.uk\/blog\/613\/backup-raspberry-pi-rsync\/","title":{"rendered":"Backup a Raspberry Pi with Rsync"},"content":{"rendered":"<p>I am going to upgrade our <a href=\"https:\/\/chewett.co.uk\/blog\/615\/compressing-raspberry-pi-backups-automatically\/\">previous Raspberry Pi backup script<\/a> that previously used Secure Copy to one that uses rsync. The primary reason for this is that rsync will only copy new or changed files over. This post goes through the reasons why you might want to use rsync\u00a0instead of scp when performing a backup.<\/p>\n<p><!--more--><\/p>\n<h2>Why do we want to use rsync over scp?<\/h2>\n<p>In our original backup script we used <code>scp<\/code> to copy files across the network. This works well when you are copying single files but doesn&#8217;t work well for our use case. We want to use something that when the file already exists it will see if it has changed, then only copy it if it has.<\/p>\n<p><code>rsync<\/code> is a Linux tool that can be used for synchronising file across two folders (which can be on different computers). The key feature that we are interested in is that it can check the timestamp, file size and contents to determine if it needs to copy the file or not. By checking to see if the file has changed it is able to copy only the files that have. Since we are going to be making our backups incrementally we would expect only a small number of files to change.<\/p>\n<p>The downside of using <code>rsync<\/code> is that it needs to determine if the files are different and will use a bit more processing power then <code>scp<\/code>. However typically copying files across a network is the primary slowdown for backing up systems. This means that for our use case rsync is ideally suited.<\/p>\n<h2>Changes to our script to use rsync<\/h2>\n<p>The first thing we need to do improving our script is to remove the first line that deletes the home and var directory. This is because we are going to use this to speed up our transfers.<\/p>\n<p>Once we have removed the line deleting the directories we need to change the scp command to a rsync command. My new backup commands are<\/p>\n<pre>rsync -rthvz --progress --delete bunker-master2:\/home .\r\nrsync -rthvz --progress --delete bunker-master2:\/var\/www var\/<\/pre>\n<p>Here I call rsync with a number of flags to backup my Raspberry Pi. The full details of the flags are below:<\/p>\n<ul>\n<li>-r Recursively copy directories<\/li>\n<li>-t Preserve the file modification times<\/li>\n<li>-h Output all numbers in a human readable format<\/li>\n<li>-v Verbose mode to increase the information during a copy<\/li>\n<li>-z Compress the data during the transfer<\/li>\n<li>&#8211;progress Show progress of the current transfers<\/li>\n<li>&#8211;delete Delete files in your backup directory that are not in the directory you are copying from. This ensures you don&#8217;t keep building up deleted files<\/li>\n<\/ul>\n<p>Using rsync with these flags will copy over the files from the Raspberry Pi. In addition, this will only copy the new or changed files which means the speed of copy is much faster.<\/p>\n<p>I\u00a0 haven&#8217;t\u00a0changed the final zipping part of the script as once the copy has finished with rsync\u00a0I still want it to create a folder archive.<\/p>\n<p>Now we have an archiving script that intelligently copies files over from the Raspberry Pi and archives them.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I am going to upgrade our previous Raspberry Pi backup script that previously used Secure Copy to one that uses rsync. The primary reason for this is that rsync will only copy new or changed files over. This post goes through the reasons why you might want to use rsync\u00a0instead of scp when performing a [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":918,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[122],"tags":[159,158,39,37,101,148,156],"class_list":["post-613","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-raspberry-pi-cluster","tag-backup","tag-bash","tag-linux","tag-raspberry-pi","tag-raspberry-pi-cluster","tag-windows-10","tag-windows-subsystem-for-linux"],"wppr_data":{"cwp_meta_box_check":"No"},"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/12\/rsync_backup_raspberry_pi.jpg?fit=800%2C800&ssl=1","jetpack_shortlink":"https:\/\/wp.me\/p2toWX-9T","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":2734,"url":"https:\/\/chewett.co.uk\/blog\/2734\/rsync-entire-raspberry-pi-to-windows\/","url_meta":{"origin":613,"position":0},"title":"Rsync Entire Raspberry Pi to Windows","author":"Chewett","date":"November 21, 2020","format":false,"excerpt":"This post includes a short snippet to copy your entire Raspberry Pi filesystem to Windows using Rsync. Copying and Archiving particular directories If you want to rsync specific directories rather than the entire Raspberry Pi I suggest you read my previous blog post. It details how to backup a Raspberry\u2026","rel":"","context":"In &quot;Raspberry Pi Cluster&quot;","block_context":{"text":"Raspberry Pi Cluster","link":"https:\/\/chewett.co.uk\/blog\/category\/raspberry-pi-cluster\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2020\/11\/raspberrypi_rsync_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2020\/11\/raspberrypi_rsync_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2020\/11\/raspberrypi_rsync_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2020\/11\/raspberrypi_rsync_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2020\/11\/raspberrypi_rsync_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":569,"url":"https:\/\/chewett.co.uk\/blog\/569\/creating-raspberry-pi-backup-script-scp-bash-windows\/","url_meta":{"origin":613,"position":1},"title":"Creating a Raspberry Pi backup script with scp and Bash on Windows","author":"Chewett","date":"August 30, 2017","format":false,"excerpt":"This post describes a simple way to keep backups of a Raspberry Pi or any other Linux computer. Why do I need backups of my Raspberry Pi? Some of the nodes in my cluster are going to be compute only, meaning that they will just run what is asked of\u2026","rel":"","context":"In &quot;Informational&quot;","block_context":{"text":"Informational","link":"https:\/\/chewett.co.uk\/blog\/category\/informational\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/08\/backing_up_a_rasperry_pi.jpg?fit=800%2C800&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/08\/backing_up_a_rasperry_pi.jpg?fit=800%2C800&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/08\/backing_up_a_rasperry_pi.jpg?fit=800%2C800&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/08\/backing_up_a_rasperry_pi.jpg?fit=800%2C800&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":615,"url":"https:\/\/chewett.co.uk\/blog\/615\/compressing-raspberry-pi-backups-automatically\/","url_meta":{"origin":613,"position":2},"title":"Compressing Raspberry Pi backups automatically","author":"Chewett","date":"November 22, 2017","format":false,"excerpt":"Today I will be improving the basic script used to backup my Raspberry Pi's so that it compresses the files after they have been copied. This will allow me to store several backups instead of having a single copy stored on my computer. This assumes you have followed the first\u2026","rel":"","context":"In &quot;Raspberry Pi Cluster&quot;","block_context":{"text":"Raspberry Pi Cluster","link":"https:\/\/chewett.co.uk\/blog\/category\/raspberry-pi-cluster\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/11\/compressing_raspberry_pi.jpg?fit=800%2C800&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/11\/compressing_raspberry_pi.jpg?fit=800%2C800&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/11\/compressing_raspberry_pi.jpg?fit=800%2C800&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/11\/compressing_raspberry_pi.jpg?fit=800%2C800&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":567,"url":"https:\/\/chewett.co.uk\/blog\/567\/upgrading-raspbian-jessie-raspbian-stretch\/","url_meta":{"origin":613,"position":3},"title":"Upgrading Raspbian Jessie to Raspbian Stretch","author":"Chewett","date":"November 4, 2017","format":false,"excerpt":"This post describes how you can upgrade Raspbian Jessie, version 8, to Raspbian Stretch, version 9. Raspbian Stretch was recently released and while they recommend installing from fresh you are able to upgrade the versions. Here I follow the steps of upgrading the Raspberry Pi's in my cluster including some\u2026","rel":"","context":"In &quot;Raspberry Pi Cluster&quot;","block_context":{"text":"Raspberry Pi Cluster","link":"https:\/\/chewett.co.uk\/blog\/category\/raspberry-pi-cluster\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/10\/upgrading_to_raspbian_stretch.jpg?fit=800%2C800&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/10\/upgrading_to_raspbian_stretch.jpg?fit=800%2C800&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/10\/upgrading_to_raspbian_stretch.jpg?fit=800%2C800&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/10\/upgrading_to_raspbian_stretch.jpg?fit=800%2C800&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":2002,"url":"https:\/\/chewett.co.uk\/blog\/2002\/raspberry-pi-cluster-node-12-automatic-start-with-rc-local\/","url_meta":{"origin":613,"position":4},"title":"Raspberry Pi Cluster Node \u2013 12 Automatic Start with rc.local","author":"Chewett","date":"March 6, 2019","format":false,"excerpt":"This post builds on\u00a0my previous posts in the Raspberry Pi Cluster series by creating a script which automatically starts the slave and master with rc.local. What is the rc.local file? The rc.local file is used on many Linux operating systems to schedule operations to run after services are started on\u2026","rel":"","context":"In &quot;Raspberry Pi Cluster&quot;","block_context":{"text":"Raspberry Pi Cluster","link":"https:\/\/chewett.co.uk\/blog\/category\/raspberry-pi-cluster\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/02\/rpi_cluster_12_automatic_start.jpg?fit=800%2C800&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/02\/rpi_cluster_12_automatic_start.jpg?fit=800%2C800&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/02\/rpi_cluster_12_automatic_start.jpg?fit=800%2C800&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/02\/rpi_cluster_12_automatic_start.jpg?fit=800%2C800&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":375,"url":"https:\/\/chewett.co.uk\/blog\/375\/setting-up-the-ci20-for-the-cluster\/","url_meta":{"origin":613,"position":5},"title":"Setting up the CI20 for the cluster","author":"Chewett","date":"October 27, 2018","format":false,"excerpt":"This post talks about the steps I have followed to set up my Creator CI20 for the Raspberry Pi Cluster. Burning Debian to the onboard NAND and configuring it To run the CI20 on the Raspberry Pi Cluster I am going to write Debian 8 to the NAND storage. First\u2026","rel":"","context":"In &quot;Raspberry Pi Cluster&quot;","block_context":{"text":"Raspberry Pi Cluster","link":"https:\/\/chewett.co.uk\/blog\/category\/raspberry-pi-cluster\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2018\/10\/ci40_setup.jpg?fit=800%2C800&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2018\/10\/ci40_setup.jpg?fit=800%2C800&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2018\/10\/ci40_setup.jpg?fit=800%2C800&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2018\/10\/ci40_setup.jpg?fit=800%2C800&ssl=1&resize=700%2C400 2x"},"classes":[]}],"_links":{"self":[{"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/posts\/613","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/comments?post=613"}],"version-history":[{"count":5,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/posts\/613\/revisions"}],"predecessor-version":[{"id":919,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/posts\/613\/revisions\/919"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/media\/918"}],"wp:attachment":[{"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/media?parent=613"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/categories?post=613"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/tags?post=613"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}