map balance - bot balance in AIIDE 2015
I wrote Ye Usualle Little Perl Script to calculate map balance in AIIDE 2015, based on the the detailed game results (the “plaintext” link on that page). The results do not tell us what race random UAlbertaBot got each game, so its results don’t count in the analysis. UAlbertaBot was the only random bot.
map | TvZ | ZvP | PvT | |||
---|---|---|---|---|---|---|
wins | n | wins | n | wins | n | |
(2)Benzene.scx | 18% | 405 | 72% | 315 | 64% | 567 |
(2)Destination.scx | 19% | 405 | 73% | 315 | 64% | 567 |
(2)HeartbreakRidge.scx | 24% | 405 | 70% | 315 | 61% | 567 |
(3)Aztec.scx | 19% | 405 | 72% | 315 | 64% | 567 |
(3)TauCross.scx | 20% | 405 | 70% | 315 | 64% | 567 |
(4)Andromeda.scx | 22% | 405 | 69% | 315 | 60% | 567 |
(4)CircuitBreaker.scx | 20% | 405 | 69% | 315 | 67% | 567 |
(4)EmpireoftheSun.scm | 16% | 405 | 69% | 315 | 65% | 567 |
(4)Fortress.scx | 19% | 405 | 70% | 315 | 69% | 567 |
(4)Python.scx | 22% | 405 | 73% | 315 | 66% | 567 |
overall | 20% | 4050 | 71% | 3150 | 64% | 5670 |
In the table, n is the total number of games in the matchup, one of several crosschecks to make sure the analysis is right. The tournament had 5 zerg, 7 protoss, and 9 terran bots (plus random UAlbertaBot, which was not counted, making 22 participants). There were 90 rounds, each on one map (which over 10 maps means 9 times through the map pool). So for TvZ there should be 5*9*90 = 4050 games; for ZvP 5*7*90 = 3150 games; for PvT 7*9*90 = 5670 games. Good.
OK, from this exercise I learned more about race balance in this tournament than about map balance. Zerg came out on top because zerg bots won. Meanwhile terran bots were concentrated toward the bottom of the crosstable, while protoss were scattered throughout. Zerg crushed protoss 2:1 but annihilated terran 5:1. I had not realized that it was so extreme. The maps made small differences, the bots made big differences.
Bots analyze maps shallowly and try to play about the same on different maps. I had expected that that lack of adaptivity would cause maps to affect results strongly: Adapting means that the bot matters more; failing to adapt means that the map matters more. But if so, it’s not visible in this table. Maybe the maps are standardized enough that adaptation doesn’t matter at this level of play. Or maybe my original thinking is wrong, and adaptation is what allows the map to matter—Heartbreak Ridge has a narrow base entrance, so that you can easily block your enemy in or out, and high ground over the natural to proxy on, and I haven’t seen any bot take advantage of those features.
You can download the AIIDE 2015 map balance analysis script in a .zip file. I ran it on a *nix but it can probably be adapted to run under Windows with no more than a tweak or two.
Next: I’ll try to normalize the results and compare human map balance to bot map balance in relative terms. Though you can get an idea already by eyeballing the tables.
Comments