Starcraft AI blog

I counted the number of upsets in the crosstables for each map. An upset is when a lower-ranked player defeats one who ends up at a higher rank. On any given map you should expect more upsets than in the tournament overall, which sums over the maps and smooths out differences.

I expected that more standard maps would have fewer upsets, but it didn’t turn out that way. Less standard Heartbreak Ridge is the only map to tease me by acting as if I understood. I’m a little surprised by the high upset counts on standard Circuit Breaker and Fortress and the lower counts on less standard Aztec. I am very surprised that Destination has half the upsets of other maps! I can’t explain it.

Another theory you might try is: Many bots are tuned on SSCAIT, so SSCAIT maps might show more solid play and fewer upsets. The non-SSCAIT maps here are only Aztec and Fortress, which don’t support the hypothesis (though there’s little evidence either way).

I thought that per-map upset rate might be a way to measure the strategic fragility of bots on maps that they weren’t tuned for, but it may also measure code fragility. Look into yesterday’s crosstables to see where today’s totals came from. Heartbreak Ridge has many upsets not so much because bots weren’t ready for it, but because NUSBot wasn’t ready for it. LetaBot could not defend Fortress. Only Circuit Breaker’s high rate of upsets was not helped along by any single bot—and Circuit Breaker is as standard as they come. So we’re not learning as much about map characteristics as about bot characteristics.

Can anybody explain why Destination was more stable and less upset-prone than other maps? That’s the big mystery here.