Starcraft AI blog

I decided to look more closely at the Win Percentage Over Time curves. For this post, “learning” means online learning during the tournament; bots which only learned offline at home are “non-learning” bots for the moment.

To start off, here are the bots whose curves are more or less flat over time. Of these, #1 SAIDA is the only learning bot. Its learning apparently enabled it to hold its ground at a high level, but not to rise further. The other 3 are #13 LastOrder, #26 CUNYBot, and tail-ender #27 Hellbot. Hellbot gradually lost win rate over time despite its low starting point. The other 2 are very nearly level over time, despite being non-learners in a field of enemies eagerly seeking weaknesses to exploit. I suppose that their play is in some way difficult to exploit by learning, whether highly adaptive, or random and unpredictable, or simply not exposing weaknesses that other bots were able to catch.

Here are all the non-learning bots, as best I could identify from yesterday’s findings. I also included #1 SAIDA to maintain the scale, which usefully goes to 1.1 to accomodate any bots which won more games than they played.

Most of the curves trend down over time. The exceptions are #13 LastOrder and #26 CUNYBot from the first graph. Here’s a rescaled graph to tease apart the dense clump from #16 LetaBot to #24 WillyT. It’s easier to see the downward trend. Of these, #17 Arrakhammer which has sophisticated play, and #20 XIMP whose weaknesses may be difficult for many opponents to exploit, leveled out after the early losses. (So did #9 Iron with its numerous adaptive reactions, from the chart above.) The others continued downward for the entire tournament. Apparently if your play is in some way good enough, you can avoid exploitation by other bots to an extent. But most non-learning bots seem doomed to keep losing win rate even over a long tournament.

Here are the learning bots which fell at first, then leveled out. It’s due to some combination of statistical fluctuation plus learning by their opponents, and no doubt bugs and whatever other random stuff. There are only 3 of them.

#15 MetaBot might belong in the graph above, but I gave it its own picture because it is in a class by itself when it comes to struggling at the start then recovering strongly. It fell hard (on the left its curve drops below #21 CDBot) and came back, but it did not level off! MetaBot rivals Steamhammer and AIUR for performance gains over time. I imagine it’s due to MetaBot’s 2-level learning ability, where it learns which of 3 heads is best against each opponent, and then 2 of the heads (AIUR and Skynet), when chosen, learn how best to play against that opponent. Like Steamhammer and AIUR, it has more scope to learn, and it learns more. The graph shows how many rivals MetaBot left in the dust—it came within an ace of surpassing #14 Tyr, and likely would have given 10 more rounds.

Here are the bots which gained win rate early, then largely leveled out—most continued to gain or lose a little for the duration. This is partly because the curves are cumulative. Only the left part of the curve can change quickly; each data point is the average of all the per round win rates to the left. The non-learning #13 LastOrder is included; the others are learning bots. #14 Tyr, which learns less because it only remembers one previous game, had the biggest decline from its peak. That’s interesting: The extremely simple method of learning from only one game is already a powerful form of learning, but it is not as powerful as, say, the UCB learning of #12 Microwave, which remembers summary statistics from many games. All these bots arguably could have done better if they had scope to learn more; their learning ceilings may not be high enough for a long tournament. Perhaps some are tuned for SSCAIT, where fast learning with limited scope helps performance.

Finally, here are the learning bots which kept learning for a long time. (#15 MetaBot has its own graph above and is left out.) #2 CherryPi started strongly and reduced its loss rate by 1/3 over the course of the tournament, which is impressive. #10 ZZZKBot started poorly, then has a clean smooth curve which approaches an asymptote after about 10 rounds. #11 Steamhammer also started poorly, and its slower improvement seems to approach an asymptote after around 30 games, but in fact Steamhammer kept on learning throughout, left Tyr, LastOrder, and Microwave behind, and came close to surpassing ZZZKBot. In a longer tournament, it likely would have; Steamhammer’s big repertoire of openings means it still has fresh ideas to try after 100 rounds. #22 AIUR struggled at first, then recovered and showed its usual strong learning gains.

I find that these performance curves are rich with insight. The top finishers have strong basics, and use learning to avoid being exploited (that seems to the only purpose of learning in SAIDA), or to exploit the weaknesses of other bots. Most bots that did not learn suffered for it, but some were difficult to exploit and could hold their ground—LastOrder was chief among these. Bots that did learn sometimes learned too little and could not keep up with their rivals. Steamhammer and MetaBot were remarkable for their comparatively weak foundations and slow but strong learning skills.

Next I’ll look into what specific bots learned about their opponents. Following tradition, I’ll start with AIUR.