AI (artifical intelligence) and neural networks are all over the news these days. Zuck and Elon are fighting over whether machines are going to kill us, Google's AI is beating world champions in the game of Go, and groundbreaking scientific work is now being done in collaboration with AI. In this series of blog posts we'll show you how to use AI to trade cryptocurrencies for fun and profit.
But first, let's take a step back and talk about what AI is. What's called AI today is a collection of statistical algorithms that can come up with good solutions to problems posed to them without requiring a human to instruct the machine on what exactly it should do. AI algorithms can be broken down into three categories:
Advanced AI systems combine all three approaches in order to achieve superhuman results. And today we're going to talk about the most exciting of them, reinforcement learning.
Our task is to train an AI on bitcoin price data to buy, sell or hold in a way that maximizes returns. We'll be using Google's awesome tensorflow library to do the mathematical heavy lifting, and we'll apply a technique called Deep Q learning pioneered by DeepMind. This model operates on the following concepts:
Our goal is to learn the Q function from historical data. The self learning comes from a concept of looping through a number of different states and actions many times, and each time update the Q function a little bit. So in each loop the Q function will know a little bit more about the world around it and should be able to approximate the real reward a little bit better for each possible action.
First things first, before we proceed, let's validate that our AI works on trivial examples. Let's start with learning how to trade a currency whose price chart looks like a straight line. Let's see what happens if we only let the model train for one iteration.
Clearly, our AI has no idea about what's going on. Let's let it train for 100 iterations instead.
Great! Our AI has learned what a straight line looks like. We only observe long trades after the model has seen the first two price points.
Let's turn it up a notch and see if our AI can learn how to behave in more complex scenarios. We'll replace the straight line with a sine curve and see what strategy our AI proposes.
Again, first let's see how an undertrained model performs, and then see how performance improves if we let the model train to its full potential.
Training for 10 iterations:
Again, no clue. 1000 iterations:
Fantastic! We've trained our AI to understand what a sine wave looks like and how to trade in a more complex scenario. It trades with confidence during strong ups and downs, and cautiously tries to anticipate sharp changes when it senses an upcoming trend change.
Now that I've convinced you that AI can learn to trade on theoretical examples, let's try it out on real data.
In the second part, we will take BTC/USD price, augment it with some technical indicators, feed that into our model, and see what strategy our AI would pick.
If you enjoyed reading please subscribe to our blog so that you don't miss our announcements and new posts where we will explore other machine learning techniques and how they can be applied to the world of cryptocurrency. Also, make sure you read through our whitepaper, and feel free to ask any questions you might have or suggest topics for a new blog post in the comments section below.