Last month I was watching HBO’s Silicon Valley (I highly recommend you to watch it) and my favourite character in the show, Gilfoyle had made this price alert for Bitcoin which rang with a very loud and obnoxious song every time Bitcoin went up or down a certain price point. I found the idea very funny and interesting so I decided to build my own version using Amazon Alexa. I will be discussing the process for the same in this article.
The project has these basic functionalities:
1. Ingesting real time cryptocurrency data and processing it.
2. Letting users create alerts using an API through a frontend client.
3. Alerts can be triggered when the specified cryptocurrency goes below or above the price specified by the user.
4. Alexa will announce that an alert has been triggered with the alert details.
I used the Binance Websocket API to get the Cryptocurrency data. The connection receives a message every 1000ms as JSON, which I parse into a Golang struct. Since the JSON has no dynamic fields, parsing it into a struct is not a very costly operation.
In order to make Alexa announce the alerts, I am using the following Bash Script that I found on GitHub. The main point of the project was trying to process real time data so I won’t be focusing on how the script works and my focus in this article would be on the design of the application. You can read more about the script from this blog by the script’s creator. (It is in German so don’t forget to translate, unless you know German already)
Let’s start by understanding the flow of the application:
Client sends a request to create an alert using the frontend application to the server. An alert request has the following parameters:
Symbol : The exchange that the user wants to set an alert for, eg BTCUSDT
Price: The price at which the alert will be triggered. If the current market price already satisfies the alert parameters, then the alert will be triggered instantly.
Comparison: “True” if the user wants to trigger the alert when the market price goes higher than the alert price, “False” if the user wants to trigger the alert when market price goes lower than the alert price.
Status: COMPLETE or PENDING. New alerts have a pending status by default.
The alert is saved in the database and for each new alert, the server checks the exchange symbol from the request body and finds if there is a listener already running for it from any previous incomplete alerts. If the new alert is the first one for its exchange symbol, then a new listener for that symbol is created.
Listeners are concurrent functions running as goroutines that are assigned all the alerts for one particular exchange symbol. Ie, BTCUSDT will have one listener and ETHUSDT will have another listener, both of which are running at the same time and processing their particular alerts.
Once a listener starts, the first step is to get all the alerts for its symbol and store them in memory efficiently. This is done by storing the alerts in a sorted set, ordered by their price. For each listener, 2 sorted sets are created. One stores all the alerts which need to be triggered when market price goes lower than the alert price and the other stores all the alerts which need to be triggered when the market price goes higher than the alert price.
The reason for using sorted sets over a simple array of alerts is because if we use an array, we would have to scan the entire array for each new data feed we receive to find if any alerts need to be triggered, making it an expensive operation. A sorted set on the other hand lets us automatically trigger all the alerts before or after the first alert we trigger, depending upon the set we are querying, saving us time.
Once the listener has stored its alerts in memory, it establishes a websocket connection to the Binance API and starts receiving data that contains real time market prices. For each new message, we query the sorted set based on whether the alert’s comparison is set to True or False. Once we find an alert that needs to be triggered, we push all the related alerts from the sorted set into a channel which is being listened to by our notifier. If all the alerts for our symbol are sent to the notifier, the websocket connection is closed and the listener is terminated. For simplicity’s stake, we discard all the alerts that do not trigger within 8 hours of creation.
The notifier is actively listening to a channel which receives all the alerts that need to be triggered. It consumes the alerts from the channel and runs our script with customised parameters, which leads to Alexa announcing our alert. Once the script has been run for an alert, the alert is marked as completed.
You can find a demo here: https://www.linkedin.com/feed/update/urn:li:activity:6997905065151721472
. . .
There are obviously better design choices and optimisations that can be made. I doubt that creating a listener goroutine for every distinct symbol would be very scalable if the application receives high number of alerts. One solution would be to create more instances of the same listener under heavy load. More instances of the whole application itself can be created as well, with each having multiple instances of listeners, therefore distributing load even more.
The possibility and existence of optimisations adds spice to software engineering and that is the only high we software engineers ever chase, so safe to say I will be working more on this project until some other side project catches my attention. You can follow me on GitHub to find my projects.
