As I've discussed last time I implemented a Machine Learning Model which tries to predict whether a Monster is a good choice for the current ruleset, mana cap etc combination. I've extracted in total 400k entries to train my model. The main issue started there. Initially, my plan was to use TensorFlow to train a neural network. The training part worked. But for some reason CUDA refuses to work on my system. Therefor it's way to slow at classification. 0.02 seconds per classification. Tried the same with a Random Forest model, 0.002 seconds. A bit faster, but still too slow to use for the whole search tree where I need to classify a couple hundred thousand monsters each match. I settled for computing the first two team positions and pruned the dragon summoners with the model. It didn't lead to an improvement in ELO rank, but at least the teams were a lot more consistent and solid.
Overall the random forest reached an F1 score of 0.7 and the neural network of 0.6. It needs a lot of improvement if I want to use it in production. Other classifiers performed either on par with random forest or were lower. But none was faster. Sadly.
For those who are curious: my horrible chart of how F1 score improved.
Overfitting on the training set, stagnating on the evaluation set. And neither did it improve on the test set.
The lower chart shows the rating progression of AICU. Because I was debugging a lot with it, the chart shows a lot more ups and downs than the upper chart. The upper chart shows a mid-level account. I was curious how my bot would fare in lower-level leagues. Previously the account reached Gold I. But it wasn't played for a long time so I had to start from novice league. To my surprise, it reached Diamond II for a short while, finishing in diamond III league.
For now, I can say, that the bot has no problem advancing to champions league. With further improvements, I'm certain that it can reach Champion I and even the top 10 ranks. I just need to give it a little edge in team selection. The machine learning model might actually work. I just need to speed it up. Either by precomputing a lookup table for each monster/ruleset/mana cap/inactive splinter combination. Or by setting up CUDA on my computer. Which would allow my GPU to be used for training and classification of the neural network in Tensorflow.
Anyway, for now, let's finish with a smaller giveaway this time.
The rules for entering are the same as last time. Comment below and include your Steem Monsters username in the comment. After the payout date, the cards will be distributed.
| Gold Rusty Android | Daria Dragonscale | Manticore | Brownie | Prismatic Energy |
