Create a Simple Crypto Dashboard Using Python

-

A tutorial on web scraping, time series analysis, and web app deployment using Python and Streamlit.

Crypto Dashboard

Dashboard is an information management tool (or a Data Scientist's play tool) where you can keep track of the important metrics, performance indicators, etc.. relating to a business or a project.

Here, our aim is to build a tool that visualizes the time series price fluctuations of various cryptocurrencies over a span of a few years to daily price fluctuations with all the essential information.

A dashboard can be created in a number of different ways depending upon the use case. Since I want this to be a simple dashboard, I included the following aspects:

    1. A subset of cryptocurrencies - as there are thousands of them listed on the trading sites. I picked the top 25 cryptos.

    2. Financial charts like Candlestick charts - to display price features like open, close, high and low.

    3. Line chart to show intraday price fluctuations.

    4. And a table showing all cryptocurrencies with their latest price and stats.

This is everything that we will include in the dashboard.


Data Collection

After we are set with the plan for the dashboard structure, the next step naturally is the data collection. From my research, I found several ways to collect crypto price data:

    1. Crypto APIs - offered by different websites such as Binance, CoinMarketCap, Nomics, CoinGecko, etc...

    2. Python libraries like YFinance, DataReader, Fastquant to name a few.

    3. Good old web scraping using BeautifulSoup4.

I explored YFinance and web scraping to collect data for my dashboard.

Web scraping can be tricky as some websites make it difficult to scrape the data and these techniques should be used more for educational purposes.

I got some help on doing web scraping on Yahoo Finance from this site. Although, I modified the code to better suit my purpose.


The scraped data looks like this:


                                                    DataFrame scraped from CoinGecko

Time series data

The web scraped data shows the real-time price of each cryptocurrency but not the historical data. So, I used YFinance python library to collect the historical price data. The code is simple.


Period is the total duration of time series (5 years in the above code) and interval is the frequency of each value in the dataset (1day). Below shown is a screenshot of the dataset obtained:


Data Visualization

The trickest part of data collection is done and now we move to data visualization. As we are dealing with a financial time series dataset, a candlestick chart would be a good idea. We can also add simple moving averages over the candlestick chart to make it more informative.

I included 3 different visualizations in the dashboard:

1. Candlestick chart


2. Daily Trends Line chart

This line graph shows the up and down trends of the price fluctuation compared to the open price.


3. Table with a color-coded column

The color coding is set to green when the change is positive and red when the change is negative.


Streamlit Integration

So, we have all the essential components to build a dashboard and now, the job is to put the code together and integrate it with streamlit.

Streamlit is a framework to create beautiful and functional web apps using Python. It is simple to use compared to the Dash framework. The dashboard I created uses a sidebar on the left that lists the filters (enabling users to put the choice of tokens or fiat) and some explanations about the dashboard.


Instead of dumping the entire code, I made a code skeleton of the Streamlit framework that I used to develop the dashboard.

If you are new to Streamlit, these quick tips might help you get started:

    1. Everything in the sidebar can be controlled using st.sidebar method. I used selectbox to input the selections and expander to wrap additional information like a help guide.

    2. To display text, you can use st.title, st.write or st.markdown. It is simple to create unordered or ordered lists using markdown. There is a possibility to add HTML scripts inside the markdown method too.

    3. Add images using st.image and Plotly charts using st.plotly_chart method.

    4. If you want to place text and image side by side, horizontally, it can be done using st.columns.

Streamlit gives you a wide list of tools and methods which you can use to fancify the dashboard.

Additional Tools

I used Canva to generate the title image for free and this website to download good-quality logos for cryptocurrencies which added a bit for bling to the dashboard.

Deployment

After the Streamlit integration, all we need is a .py file (with our code) and a requirements.txt file on a GitHub repository. You can now choose any deployment service to deploy the web app.

Some of the options to deploy web apps for free are Streamlit Cloud, Heroku and Netlify. They also offer premium services but to deploy a simple app, the free tier works just fine. I used Streamlit Cloud to deploy my Crypto Dashboard.

Congrats! you've learned how to deploy a simple crypto dashboard.

Comments

Post a Comment

Popular posts from this blog

Sending Emails using Apache Airflow Email Operator

-
Image
  Automation reduces manual work and plays a key role in improving productivity in various industries. It is one of the fastest and less time-consuming techniques that most businesses practice to gain higher production rates and improve work efficiency. But, unsure how to perform, some of them fail to automate tasks and end up performing functions manually. Each IT expert has a different job or workflow to perform, right from collecting data from other sources to processing it, uploading, and creating reports. There are many tasks that experts need to perform manually on a daily basis. Thus, to trigger automatic workflow and reduce the time and effort of experts, we recommend using Apache Airflow. Apache Airflow is an open-source tool that assists in managing complex workflows. The powerful workflow management platform helps resolve issues and aids in programmatically authoring, scheduling, and monitoring daily tasks. Data Scientists or Data Engineers often find it helpful for thei...
Read more

Built a working Hadoop-Spark-Hive-Superset cluster on Docker

-
Image
Let’s get this thing started Now you can download my repo  hadoop-spark-hive-superset and from the directory where you placed it all it takes is this command to get the multi-container environment running: And you can break it all down again by going to that same directory and running this: All the containers will then be stopped and removed. But: the images and volumes stay! So don’t be surprised that the csv file you uploaded to HDFS will still be there. Quick starts Quick start HDFS Find the Container ID of the namenode. Copy  supermart_grocery_sales.csv  to the namenode. Go to the bash shell on the namenode with that same Container ID of the namenode. Create a HDFS directory /data//sales/ supermart_grocery_sales . Copy  supermart_grocery_sales.csv  to HDFS: Quick start Spark Go to http://<dockerhadoop_IP_address>:8080 or  http://localhost:8080/  on your Docker host (laptop). Here you find the spark:// master address: Go to the command line o...
Read more

ETL Process Using Airflow and Docker

-
Image
  Introduction In this article, I will create an ETL process for extracting Forex data and wrap the whole thing as a data pipeline using Airflow and Docker. The ETL process will extract data from  fixer.io  API, transform it, and load it to a PostgreSQL database. This project aims to have an automated process that constantly feeds the PostgreSQL database with data. Every 2 minutes, the ETL process will load an updated batch of Forex data. Note that this article assumes some knowledge of Airflow, Docker, Python, and SQL. I won't go into too many details to keep this article short. Project Steps     1. Setting up Airflow Architecture     2. 1st Dag - Check if API is available     3. 2nd Dag - Create a table     4. 3rd Dag - Extract     5. 4th Dag - Transform     6. 5th Dag - Load     7. Query data in the pgAdmin UI Step #1 - Setting up Airflow Architecture The first thing we should do is set up the basic ...
Read more