Starcraft AI blog

I thought of another theory to explain why Steamhammer’s elo might have changed after the last mini-tournament.

Your rating depends on the mix of your opponents. Elo measures average strength. You will generally do better than your average against some opponents, and worse against others. And because of the voting system on SSCAIT, your mix of opponents depends on what people think of your bot. If they think it’s more fun to watch you lose to enemies that have your number, your elo goes down, or vice versa. Because of voting, elo depends on reputation.

Before the tournament, Steamhammer was getting more matches against Bereaver. Maybe it still had novelty value; maybe voters liked seeing the powerful Bereaver get crushed by zerg exploiting a narrow window of weakness. Anyway, that tended to drive up Steamhammer’s elo. Since the tournament, in which Steamhammer did less well than people may have expected, it gets fewer matches overall and more of them are against opponents that give it trouble, like Wuli. Its elo ought to have fallen some because of that.

Somebody who was interested might dig through the game history and try to figure out the bias that reputation lends to each bot’s elo. How much would the elo change if the mix of opponents were exactly even? I think that elo is a pretty good system, and despite reputation effects, elo depends mostly on strength. Am I right?

Randomhammer, on an unrelated note, had a good day today. It scored a win as protoss against Iron with a zealot rush; Igor Dimitrijevic is apparently trying to reduce Iron’s overreaction to early aggression from the opponent, and it underreacted instead. Then it won as zerg against Krasi0 when terran played an 8 rax into vulture build against fast mutalisks. Neither game was interesting, but it was satisfying to beat the leaders.