The highest rating it has reached was 3856. At season end it stood at 3711, to which it recovered from 3630. Overall the bot was trending between 3600 and 3800 after the last change.
This approach, let's call it the "naive bot" was sufficient to reach Gold III and sometimes Gold II. Then I put some money on the table and upgraded some of my splinters to (almost) max level. Including Fire, Earth, Death, Dragon and Normal Splinter and some strong units from the other Splinters (e.g. Angel of Life). The new team, along with a Breadth-First Tree Initialization of depth 3 allowed the bot to rise to Gold II and stay there consistently. But for some reason, it couldn't progress beyond that point. Win streaks were followed by long losing streaks and prevented progress. The "naive initialization bot" needed an improvement.
The salvation for the versus bot was a weak "learning" approach. I wouldn't really call it learning, but it sort of is a very naive and weak learning approach. The bot stores both its own past team selections as well as the enemy teams. Not player specific, but in general. They get ranked per ruleset and splinter with their win/loss count. At the start of each Monte Carlo Tree Search, the bot does the usual Breath First Tree Initialization and afterwards does a couple of "Depth-First" initializations with the last 5 winning teams of each splinter for the ruleset. In total 90 leads for promising teams.
This new approach was doing good at first but seemed to taper off the longer the bot ran. The mistake was pretty obvious, I introduced a limit of 50 teams, but new teams were added to the end. This caused the bot to only remember teams with multiple wins (which rarely happens) and old teams which were added at the beginning. One quick rewrite later new teams were added at the start of those lists. When sorting this kept the strong teams still in front but propagated the new teams at the front of teams with equal win/loss counts. Making them "fresher" in memory. Old teams are forgotten and pushed off the list. With this change, the bot showed a much stronger play and generated weak teams much more rarely.
With this approach, the bot left Diamond III and II behind and fought its path through Diamond I, reaching the Champion League. That's where it got into trouble again. It pushed up to a rating of 3856 but stumbled back to the Edge of the Diamond League and stayed there. Walking in and out of Champion III. That was just a couple of hours ago. Now I need to think up new improvements.
With the current state of the bot, I suspect I need more drastic measures to advance. For now, I'll experiment with some sort of team decay. Making the bot forget teams after a sufficient number of new teams were added. This could lead to the bot improving its generated teams step by step. My thought process behind that is as follows: If a team stays long at the top it influences each team generation. If too many successful teams stay there, the bot becomes predictable and it notices way too late if teams are losing consistently. If the bot starts to forget them, let's say after 10 new teams were added those new teams are already influenced by this old team, and should, in theory, be better adapted to the current opponent situation.
Aside from that an easy quick fix would be to store teams player specific. If the bot encounters an opponent again this gives the bot a good idea which teams it will encounter.
Now onto more drastic measures. One further way of improving would be guiding the MCTS with a classification approach. For example a Random Forest or Neural Network which decides to reward, penalize or prune parts of the game tree. This would guide the MCTS to more efficient teams more quickly. The main issue with this approach is, that I'd need to lower the iterations of the MCTS because the new approach would take longer. On one hand, this can lead to improvements, but on the other, as seen in Googles Alpha Go it can also wrongly guide the search and overlook better solutions. Surely, this can lead to an overall improvement but that is something I need to evaluate before I speculate further. I can also go down a purer Reinforcement Learning approach or combine both approaches. I'll look into them over the coming days.
I'd like to thank you for staying/reading until this point. As my bot has been massively more successful I than I initially expected, I'd like to share some of its League rewards below. The first comment gets Daria Dragon Scale and a Black Dragon. All the other cards get distributed randomly to the next commenters in sequence. Just comment below and include your Steem Monsters username so that I can send them to you. I'll wait with the random distribution until at least ten people have commented. Or until tomorrow evening. Then I'll distribute the cards to the people who have commented until then.
First Commenter Prize
| Daria Dragonscale lvl 1 | Black Dragon lvl 1 |
| Naga Fire Wizard lvl 1 | Imp Bowman lvl 1 | Gold Wood Nymph lvl 1 | Javelin Thrower lvl 2 | Vampire lvl 3 |
| Daria Dragonscale lvl 1 | Skeletal Warrior lvl 2 | Highland Archer lvl 3 | Rusty Android lvl 3 | Goblin Mech lvl 3 |
