Starcraft AI blog

Observation 1: CherryPi makes only zerglings, hydralisks, and mutalisks. That is it for combat units. Also it prefers the most basic units: Zerglings most, mutalisks least.

Observation 2: CherryPi seems to limit itself to a small number of opening build orders. For example, it seems to have 2 ZvZ builds, one nine pool into zergling pressure and one turtle into spire. The opening builds are varied to adapt to the situation in some way; I haven’t been able to discern how much is due to hand-coded reactions and how much to the learning system. Even so, neither build looks impressive in itself.

Observation 3: CherryPi has inconsistent micro skills. Some skills are outstanding, like storm dodging. Some are inferior, like zergling targeting. Micro doesn’t seem to have been a point of heavy emphasis.

And yet CherryPi is doing extremely well, keeping near the top of the rankings. I imagine that that is partly because (as has been pointed out) it is tuned to beat the strongest opponents, and voters like to match it against exactly those opponents. I don’t think that’s the whole story. I think the learning system deserves much of the credit.

All the observations support a story that the learning system is where the heavy development effort has gone. 1. Empirical learners need data. When the situation is simpler, they can get by with less data. CherryPi might support few combat units because the effort went elsewhere, or it might be a deliberate choice to make life easier for the learning system (at least for now). 2. Similarly for the small range of opening choices. Maybe they didn’t have time to polish more choices; maybe the learning system learns how to use each opening, and they need few enough choices that it is forced to learn a decent amount. 3. Poor micro in some situations probably means that they haven’t had time to work on it. The effort went elsewhere.

Of course this is speculation. You could argue the opposite, because the learning system is not mature either. We can tell because it has visible holes. For example, it struggles with Juno by Yuanheng Zhu, the cannon contain bot, and it has some trouble with heavy rushes like those by Wuli or Black Crow, which play easy-to-understand strategies that you might expect to be easy to learn to counter. In my story, that is of course because learning is hard.

My read on the project’s style is that they are pushing ahead hard and in consequence allowing bugs and sloppiness to creep in to a degree. If I am reading it right, then when the next round of tournaments comes up after the middle of the year, we can expect CherryPi to be formidably capable. It won’t matter if they lose 5% of their games to bugs, provided the long tournament allows time for the bot to learn how to win almost all the rest. And, you know, a sufficiently smart learning system might be able to learn how to work around bugs....