an advantage for playing random on BASIL
And another new bot, the strange random KangarooBot whose builds are... unconventional. Read the github, it’s funny. But that’s not what this post is about.
Here are graphs of Steamhammer’s win rates by matchup followed by Randomhammer’s for the same matchups. Randomhammer is Steamhammer, I upload the same file for both. Any differences in play are due to the learning data—or to the opponent.
The software has helpfully given the graphs different scales, so here’s the key data in a table.
| matchup | Steam | Random |
|---|---|---|
| ZvZ | 73% | 80% |
| ZvT | 73% | 75% |
| ZvP | 60% | 64% |
On BASIL, when you play random, the server chooses your race and so both players know it from the beginning. The idea is to negate any advantage from playing random. And yet Randomhammer performs better than Steamhammer in every zerg matchup. Why is that?
Maybe the issue is that other bots do not learn by matchup, but pool all their learning data for each opponent. (Steamhammer learns each matchup independently.) It’s a reasonable thing to do for a traditional random opponent, in which the game chooses the opponent’s race and you don’t know it at first. It amounts to pretending that you don’t know the opponent’s race for learning purposes, so the opponent appears more unpredictable than it should.
Any other theories?
PurpleWave and PurpleDestiny show the same effect for protoss, by the way. MadMix is not as simple. But Randomhammer is the only bot that I’m certain has no configuration differences when playing random.
Trackbacks
No Trackbacks
Comments
Dan on :
Jay Scott on :
Gathers on :
My guess is that MadMix has lower general combat skills than most bots around the same ranking (e.g. it still has no combat simulator, only rough unit-strength estimations etc) so it relies more on finding the correct opening vs each opponent. If this is the case then MadMixR should catch up as it learns more.
It is of course also possible there are bugs that interfere with play or learning as random.