Starcraft AI blog

The CIG 2017 tournament results are out. So far we have a results table and a slideshow of the conference presentation with a few interesting details. I expect the full results will be out before long. Here I summarize the results table in a way that emphasizes the tight grouping of finishers 4 through 11; they all scored nearly the same. There is a wide gap between the bots with plus scores and the lower half.

The format this year was straight round robin with 20 entrants, 5 maps, and 125 rounds. With 125 games for each pair of opponents, 25 games on each map, slow machine learning algorithms had some data to bite into.

3 of the 5 maps are also used on SSCAIT: Tau Cross, Andromeda, and Python. The 2 player map Hitchhiker has a short rush distance and favors cheese. But if the game does not end early, then the narrow ravine between bases and the arrangement of map blocks calls for sophisticated play. I think Hitchhiker must be a difficult map for bots. The remaining map is 3 player Alchemist, which was also used last year. Each base has 2 ramp entrances, so a bot which wants to defend at “the” ramp may go wrong.

Discussion. The sophisticated 4-pooler ZZZKBot by Chris Coxe was the top winner. I imagine that having Hitchhiker in the map pool helped it. I’m curious to see its games and find out whether it had special-case strategies for specific opponents, as it has had in the past. Tscmoo played random for the first time and placed second. These 2 usually place high. PurpleWave came in third, an outstanding performance for a new entrant. Congratulations!

The pre-tournament favorite Iron did surprisingly poorly. It must have had a bug. I don’t know the cause, but my first guess is that it suffered on one of the maps. McRave is another bot which did not perform at its peak.

The most interesting detail in the slide show is a pair of graphs showing the effect of machine learning for opponent modeling. The first graph shows the win rates of the winners ZZZKBot and Tscmoo sagging toward the end of the tournament. The second shows MegaBot and SRBotOne soaring toward the end and says that they were the top scorers in the final rounds 120-125. In other words, if the tournament had continued long enough, the winners would have been completely different. One the one hand, this shows the power of machine leaning; on the other hand, it shows the slowness, because the long tournament was not long enough. In Steamhammer, I would like both fast adaptation and slow adaptation. The middlegame use of fast adaptation is almost working now, and I intend to use the same mechanism in a different way for opening selection. But there will be no time to add slow adaptation to fine-tune as more games accumulate.

I think that UAlbertaBot and Overkill were the 2015 versions. UAlbertaBot presumably had learning turned off. Those 2 and OpprimoBot have been constant for a while and can serve as benchmarks to judge progress. (AIUR is not as constant a benchmark because it has learning turned on.) In AIIDE 2015, the top finishers in order were Tscmoo, ZZZKBot, Overkill, UAlbertaBot. So in 2 years, former top bots have receded into the pack of above-average scorers.