{"id":906,"date":"2017-12-16T13:00:13","date_gmt":"2017-12-16T13:00:13","guid":{"rendered":"http:\/\/chewett.co.uk\/blog\/?p=906"},"modified":"2017-12-12T23:54:17","modified_gmt":"2017-12-12T23:54:17","slug":"storing-bitcoin-price-data-raspberry-pi","status":"publish","type":"post","link":"https:\/\/chewett.co.uk\/blog\/906\/storing-bitcoin-price-data-raspberry-pi\/","title":{"rendered":"Storing Bitcoin price data with a Raspberry Pi"},"content":{"rendered":"<p>This post goes through the process of collecting and storing bitcoin price data using python, crontab, and a Raspberry Pi.<\/p>\n<p><!--more--><\/p>\n<h2>Collecting Bitcoin Price Data using Coinbase<\/h2>\n<p>To gather the price of bitcoin we are going to use Coinbase as our bitcoin price provider. The reason for this is they provide a free REST API to collect the price of bitcoin.<\/p>\n<p>I am going to use python 3 to access this API and log the current price of bitcoin every minute. This information will be logged to a file which I can then preprocess to store and display the data in different ways. I will run this on a Raspberry Pi using its crontab facility.<\/p>\n<h2>Accessing Coinbase API<\/h2>\n<p>The Coinbase API allows for a number of metrics including buy, sell and spot price. I am going to be downloading the spot price of bitcoin for this post.<\/p>\n<p>The API for spot price uses the URL <code>https:\/\/api.coinbase.com\/v2\/prices\/{coin}-{currency}\/spot<\/code><\/p>\n<p>Here there are two parameters that we can set:<\/p>\n<ul>\n<li>coin &#8211; This is set to whatever coin you want to retrieve the currency for. I am going to set this to BTC (Bitcoin)<\/li>\n<li>currency &#8211; This will return the value of the coin in the currency chosen. Bitcoin is typically measured in United States\u00a0Dollars so I will set this to USD<\/li>\n<\/ul>\n<p>By setting the URL to <code>https:\/\/api.coinbase.com\/v2\/prices\/BTC-USD\/spot<\/code> I will be able to retrieve the current price for bitcoin in United States Dollars.<\/p>\n<h2>Storing Bitcoin data over time<\/h2>\n<p>Using this API I can get the bitcoin data in python using the following code.<\/p>\n<pre class=\"brush: python; title: ; notranslate\" title=\"\">with urllib.request.urlopen(&quot;https:\/\/api.coinbase.com\/v2\/prices\/BTC-USD\/spot&quot;.format(coin=coin, currency=currency)) as response:\r\n    html = response.read().decode('utf-8')\r\n\r\ncurrent_data = json.loads(html)<\/pre>\n<p>Here I load up the bitcoin price data and then load it into a dict using <code>json.loads<\/code>.\u00a0I have to decode the\u00a0string returned from <code>response.read()<\/code> as this is received as bytes and <code>json.loads<\/code> requires a string. These three lines of code require importing the <code>json<\/code>\u00a0and <code>urllib.request<\/code>\u00a0module.<\/p>\n<p>Now we have the data we want to add the current time to it. I do this using <code>time.time()<\/code>.<\/p>\n<pre class=\"brush: python; title: ; notranslate\" title=\"\">current_data&#x5B;'time'] = time.time()<\/pre>\n<p>Similar to above this requires that\u00a0 the <code>time<\/code>\u00a0module is imported. Now I have the data I want to log I can write this to a file.<\/p>\n<pre class=\"brush: python; title: ; notranslate\" title=\"\">log_filename = os.path.join(os.path.dirname(os.path.realpath(__file__)), &quot;BTC-USD-spot-price.log&quot;)\r\nwith open(log_filename, &quot;a&quot;) as logfile:\r\n    logfile.write(json.dumps(current_data) + &quot;\\n&quot;)<\/pre>\n<p>The final three lines of code open a file called <code>BTC-USD-spot-price.log<\/code> in append mode in the directory of the file. This means that anything written to it will be added to the end of the file.<\/p>\n<p>I use the <code>json<\/code> module again to save the dict holding the data to the file and add a new line. This means that each json dump will be on a new line for easy processing.<\/p>\n<h2>Setting up crontab to run the data collection<\/h2>\n<p>To run a program at a regular interval you can use crontab. This can be edited by running <code>crontab -e<\/code>.\u00a0To run a program every minute you can enter the following crontab.<\/p>\n<pre>* * * * * python3 \/dev\/get_prices.py<\/pre>\n<p>The stars indicate that it should run every minute, of every hour, of every day of the month, every month of the year, and every day of the week. This is how you use the crontab format to specify it runs every minute.<\/p>\n<p>Now my program will run every minute on my Raspberry Pi so I will have a decent amount of collected data to review later.<\/p>\n<h2>Future work with Bitcoin data<\/h2>\n<p>Now I have a Raspberry Pi recording the price of Bitcoin over time I will be able to do some processing in the future. I am going to look at graphing this data and to visually see the changes over time.<\/p>\n<p>In addition to graphing the price data I am going to analyse the different ways of storing this price data. Comparing the speed of retrieval and size of the data.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This post goes through the process of collecting and storing bitcoin price data using python, crontab, and a Raspberry Pi.<\/p>\n","protected":false},"author":1,"featured_media":914,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[98],"tags":[202,199,184,198,37],"class_list":["post-906","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-software","tag-bitcoin","tag-coinbase","tag-python","tag-python-3","tag-raspberry-pi"],"wppr_data":{"cwp_meta_box_check":"No"},"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/12\/storing_bitcoin_price_with_raspi.jpg?fit=800%2C800&ssl=1","jetpack_shortlink":"https:\/\/wp.me\/p2toWX-eC","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":2780,"url":"https:\/\/chewett.co.uk\/blog\/2780\/raspberry-pi-cluster-node-18-raspberry-pi-temperature-monitoring\/","url_meta":{"origin":906,"position":0},"title":"Raspberry Pi Cluster Node \u2013 18 Raspberry Pi Temperature Monitoring","author":"Chewett","date":"February 20, 2021","format":false,"excerpt":"This post builds on\u00a0my previous posts in the Raspberry Pi Cluster series\u00a0by starting to log temperature with the RaspberryPiVcgencmd Python module. Installing RaspberryPiVcgencmd RaspberryPiVcgencmd is a small python module aimed to control vcgencmd and allow programmatic access to it. This can be installed with the following command. python3 -m pip\u2026","rel":"","context":"In &quot;Raspberry Pi Cluster&quot;","block_context":{"text":"Raspberry Pi Cluster","link":"https:\/\/chewett.co.uk\/blog\/category\/raspberry-pi-cluster\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2021\/02\/raspi_cluster_18_cputemperature_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2021\/02\/raspi_cluster_18_cputemperature_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2021\/02\/raspi_cluster_18_cputemperature_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2021\/02\/raspi_cluster_18_cputemperature_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2021\/02\/raspi_cluster_18_cputemperature_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":2744,"url":"https:\/\/chewett.co.uk\/blog\/2744\/raspberry-pi-cluster-node-17-influxdb-machine-stats-monitoring\/","url_meta":{"origin":906,"position":1},"title":"Raspberry Pi Cluster Node \u2013 17 InfluxDB Machine Stats Monitoring","author":"Chewett","date":"December 19, 2020","format":false,"excerpt":"This post builds on\u00a0my previous posts in the Raspberry Pi Cluster series\u00a0by performing a small refactor and storing cluster vitals data to InfluxDB. Refactoring the codebase a little To start with I will refactor the codebase a little so it is more split up. By introducing a few new package\u2026","rel":"","context":"In &quot;Raspberry Pi Cluster&quot;","block_context":{"text":"Raspberry Pi Cluster","link":"https:\/\/chewett.co.uk\/blog\/category\/raspberry-pi-cluster\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2020\/12\/raspi_cluster_17_influxdb_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2020\/12\/raspi_cluster_17_influxdb_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2020\/12\/raspi_cluster_17_influxdb_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2020\/12\/raspi_cluster_17_influxdb_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2020\/12\/raspi_cluster_17_influxdb_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":343,"url":"https:\/\/chewett.co.uk\/blog\/343\/raspberrypivcgencmd-python-library-access-raspberry-pi-vcgencmd-command\/","url_meta":{"origin":906,"position":2},"title":"RaspberryPiVcgencmd A python library to access Raspberry Pi vcgencmd command","author":"Chewett","date":"May 31, 2017","format":false,"excerpt":"To access some of the lower level aspects of the Raspberry Pi's config you can use the built in command vcgencmd. This is a useful on the command line but to easily access it in a program you need to parse the output. I am planning to use this command\u2026","rel":"","context":"In &quot;Raspberry Pi Cluster&quot;","block_context":{"text":"Raspberry Pi Cluster","link":"https:\/\/chewett.co.uk\/blog\/category\/raspberry-pi-cluster\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/05\/vcgencmd.png?fit=628%2C288&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/05\/vcgencmd.png?fit=628%2C288&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/05\/vcgencmd.png?fit=628%2C288&ssl=1&resize=525%2C300 1.5x"},"classes":[]},{"id":2680,"url":"https:\/\/chewett.co.uk\/blog\/2680\/raspberry-pi-cluster-node-16-python-3-codebase-refactor\/","url_meta":{"origin":906,"position":3},"title":"Raspberry Pi Cluster Node \u2013 16 Python 3 Codebase Refactor","author":"Chewett","date":"October 24, 2020","format":false,"excerpt":"This post builds on\u00a0my previous posts in the Raspberry Pi Cluster series\u00a0by improving the codebase for Python 3. Moving to Python 3 Python 2 was marked end of life on January 1st, 2020 and therefore applications should ideally be no longer using Python 2. There will still be a lot\u2026","rel":"","context":"In &quot;Raspberry Pi Cluster&quot;","block_context":{"text":"Raspberry Pi Cluster","link":"https:\/\/chewett.co.uk\/blog\/category\/raspberry-pi-cluster\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2020\/10\/raspi_cluster_16_python3refactor_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2020\/10\/raspi_cluster_16_python3refactor_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2020\/10\/raspi_cluster_16_python3refactor_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2020\/10\/raspi_cluster_16_python3refactor_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2020\/10\/raspi_cluster_16_python3refactor_posticon_OUTPUT.png?fit=1200%2C628&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":907,"url":"https:\/\/chewett.co.uk\/blog\/907\/fixing-importerror-no-module-named-winrandom\/","url_meta":{"origin":906,"position":4},"title":"Fixing ImportError: No module named &#8216;winrandom&#8217;","author":"Chewett","date":"December 13, 2017","format":false,"excerpt":"This post describes how you can fix the python error No module named 'winrandom'. This is typically found when you are using the pyCrypto or\u00a0coinbase python module. What is the No module named 'winrandom' error This python error may occur whenever you try and use a module that requires the\u2026","rel":"","context":"In &quot;Fixes&quot;","block_context":{"text":"Fixes","link":"https:\/\/chewett.co.uk\/blog\/category\/fixes\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/12\/python_fix_winrandom_error.jpg?fit=800%2C800&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/12\/python_fix_winrandom_error.jpg?fit=800%2C800&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/12\/python_fix_winrandom_error.jpg?fit=800%2C800&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2017\/12\/python_fix_winrandom_error.jpg?fit=800%2C800&ssl=1&resize=700%2C400 2x"},"classes":[]},{"id":2127,"url":"https:\/\/chewett.co.uk\/blog\/2127\/raspberry-pi-cluster-node-14-a-simple-webserver\/","url_meta":{"origin":906,"position":5},"title":"Raspberry Pi Cluster Node \u2013 14 A simple webserver","author":"Chewett","date":"April 17, 2019","format":false,"excerpt":"This tutorial focuses on creating a simple webserver that displays the status of the master using python Bottle. What will the webserver be used for? To interact with the cluster I am planning on making a small set of webpages. Initially these will just display information about the cluster but\u2026","rel":"","context":"In &quot;Raspberry Pi Cluster&quot;","block_context":{"text":"Raspberry Pi Cluster","link":"https:\/\/chewett.co.uk\/blog\/category\/raspberry-pi-cluster\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/04\/rpi_cluster_14_a_simple_webserver.jpg?fit=800%2C800&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/04\/rpi_cluster_14_a_simple_webserver.jpg?fit=800%2C800&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/04\/rpi_cluster_14_a_simple_webserver.jpg?fit=800%2C800&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/chewett.co.uk\/blog\/wp-content\/uploads\/2019\/04\/rpi_cluster_14_a_simple_webserver.jpg?fit=800%2C800&ssl=1&resize=700%2C400 2x"},"classes":[]}],"_links":{"self":[{"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/posts\/906","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/comments?post=906"}],"version-history":[{"count":4,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/posts\/906\/revisions"}],"predecessor-version":[{"id":915,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/posts\/906\/revisions\/915"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/media\/914"}],"wp:attachment":[{"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/media?parent=906"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/categories?post=906"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/chewett.co.uk\/blog\/wp-json\/wp\/v2\/tags?post=906"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}