Starcraft AI blog

Blue is good for Microwave, red is good for BananaBrain.

microwave strategies versus bananabrain strategies

	overall	10/12gate	1basespeedzeal	2basespeedzeal	4gate2archon	5gategoon	9/9gate	9/9proxygate	bisu	neobisu	sairdt	sairgoon	sairreaver	stove
overall	21/157 13%	3/86 3%	1/1 100%	1/2 50%	1/2 50%	5/30 17%	3/15 20%	1/2 50%	1/2 50%	1/2 50%	1/5 20%	1/5 20%	1/2 50%	1/3 33%
11Gas10PoolLurker	0/3 0%	0/2 0%	-	-	-	0/1 0%	-	-	-	-	-	-	-	-
11HatchTurtleHydra	0/1 0%	0/1 0%	-	-	-	-	-	-	-	-	-	-	-	-
11HatchTurtleMuta	0/1 0%	0/1 0%	-	-	-	-	-	-	-	-	-	-	-	-
12Hatch	0/1 0%	0/1 0%	-	-	-	-	-	-	-	-	-	-	-	-
12HatchTurtle	0/3 0%	0/1 0%	-	-	-	-	0/2 0%	-	-	-	-	-	-	-
12PoolMain	3/15 20%	1/10 10%	-	-	-	2/4 50%	-	0/1 0%	-	-	-	-	-	-
12PoolMuta	0/1 0%	-	-	-	-	-	0/1 0%	-	-	-	-	-	-	-
2HatchMuta	0/2 0%	0/1 0%	-	-	-	0/1 0%	-	-	-	-	-	-	-	-
2HatchMuta_Sparkle	0/1 0%	0/1 0%	-	-	-	-	-	-	-	-	-	-	-	-
3HatchHydra	1/7 14%	0/3 0%	-	-	-	0/3 0%	-	-	-	-	-	1/1 100%	-	-
3HatchLingBust	0/4 0%	0/2 0%	-	-	-	-	0/1 0%	-	-	-	-	0/1 0%	-	-
3HatchLurker	0/1 0%	-	-	-	-	0/1 0%	-	-	-	-	-	-	-	-
3HatchMuta	0/2 0%	0/2 0%	-	-	-	-	-	-	-	-	-	-	-	-
3HatchPoolHydra	0/3 0%	0/1 0%	-	-	-	0/1 0%	-	-	-	-	-	0/1 0%	-	-
4HatchBeforeGas	0/1 0%	-	-	-	-	0/1 0%	-	-	-	-	-	-	-	-
4HatchPool	0/1 0%	-	-	-	-	0/1 0%	-	-	-	-	-	-	-	-
4HatchPoolHydra	0/3 0%	0/2 0%	-	-	-	0/1 0%	-	-	-	-	-	-	-	-
4PoolHard	0/7 0%	0/3 0%	-	-	-	0/1 0%	0/1 0%	-	-	-	0/1 0%	-	-	0/1 0%
4PoolSoft	0/1 0%	-	-	-	-	0/1 0%	-	-	-	-	-	-	-	-
6Pool	0/1 0%	0/1 0%	-	-	-	-	-	-	-	-	-	-	-	-
7Pool	0/2 0%	0/2 0%	-	-	-	-	-	-	-	-	-	-	-	-
7PoolHydraRush7D	0/2 0%	0/1 0%	-	-	-	-	-	-	-	-	0/1 0%	-	-	-
8Pool	0/2 0%	0/1 0%	-	-	-	0/1 0%	-	-	-	-	-	-	-	-
9Hatch9Pool9Gas	0/2 0%	0/2 0%	-	-	-	-	-	-	-	-	-	-	-	-
9HatchTurtleHydra	0/1 0%	-	-	-	-	-	-	-	-	0/1 0%	-	-	-	-
9Pool	0/1 0%	-	-	-	-	0/1 0%	-	-	-	-	-	-	-	-
9PoolGasHatchSpeed8D	16/57 28%	2/24 8%	1/1 100%	1/2 50%	1/2 50%	3/9 33%	3/8 38%	-	1/2 50%	1/1 100%	1/3 33%	0/2 0%	1/1 100%	1/2 50%
9PoolHatchGasSpeed8D	1/11 9%	0/7 0%	-	-	-	0/1 0%	0/2 0%	1/1 100%	-	-	-	-	-	-
9PoolSpeed	0/1 0%	-	-	-	-	0/1 0%	-	-	-	-	-	-	-	-
9PoolSpeedLing	0/8 0%	0/7 0%	-	-	-	0/1 0%	-	-	-	-	-	-	-	-
Overpool	0/1 0%	0/1 0%	-	-	-	-	-	-	-	-	-	-	-	-
OverpoolTurtle	0/1 0%	0/1 0%	-	-	-	-	-	-	-	-	-	-	-	-
ZvP_10Hatch9Pool	0/1 0%	-	-	-	-	-	-	-	-	-	-	-	0/1 0%	-
ZvP_2HatchHydra	0/1 0%	0/1 0%	-	-	-	-	-	-	-	-	-	-	-	-
ZvP_9Hatch9Pool	0/1 0%	0/1 0%	-	-	-	-	-	-	-	-	-	-	-	-
ZvZ_Overgas11Pool	0/1 0%	0/1 0%	-	-	-	-	-	-	-	-	-	-	-	-
ZvZ_Overgas9Pool	0/1 0%	0/1 0%	-	-	-	-	-	-	-	-	-	-	-	-
ZvZ_Overpool9Gas	0/3 0%	0/3 0%	-	-	-	-	-	-	-	-	-	-	-	-
ZvZ_OverpoolTurtle	0/1 0%	0/1 0%	-	-	-	-	-	-	-	-	-	-	-	-

That 9PoolGasHatchSpeed8D (a variant of the Styx build) must be behind many of the NakedExpand wins. Well, we already knew that, because it is behind most of the wins altogether. The interesting discovery is that it had some success against a wide range of BananaBrain openings.

The overall impression from this table is that both bots were feeling around in the dark. And we know how they work, so we know it’s true!

microwave as seen by bananabrain

microwave played	#	bananabrain recognized
11Gas10PoolLurker	3	3 12pool
11HatchTurtleHydra	1	1 12hatch
11HatchTurtleMuta	1	1 12hatch
12Hatch	1	1 12hatch
12HatchTurtle	3	3 12hatch
12PoolMain	15	9 12pool | 4 unknown | 2 10hatch
12PoolMuta	1	1 12pool
2HatchMuta	2	2 12hatch
2HatchMuta_Sparkle	1	1 12hatch
3HatchHydra	7	7 12hatch
3HatchLingBust	4	4 12hatch
3HatchLurker	1	1 12hatch
3HatchMuta	2	2 12hatch
3HatchPoolHydra	3	3 12hatch
4HatchBeforeGas	1	1 12hatch
4HatchPool	1	1 12hatch
4HatchPoolHydra	3	2 12hatch | 1 unknown
4PoolHard	7	7 4/5pool
4PoolSoft	1	1 4/5pool
6Pool	1	1 4/5pool
7Pool	2	2 4/5pool
7PoolHydraRush7D	2	2 4/5pool
8Pool	2	2 9pool
9Hatch9Pool9Gas	2	2 10hatch
9HatchTurtleHydra	1	1 10hatch
9Pool	1	1 9pool
9PoolGasHatchSpeed8D	57	49 9pool | 8 overpool
9PoolHatchGasSpeed8D	11	6 9pool | 3 overpool | 2 unknown
9PoolSpeed	1	1 9poolspeed
9PoolSpeedLing	8	5 9poolspeed | 3 overpool
Overpool	1	1 overpool
OverpoolTurtle	1	1 overpool
ZvP_10Hatch9Pool	1	1 10hatch
ZvP_2HatchHydra	1	1 12hatch
ZvP_9Hatch9Pool	1	1 10hatch
ZvZ_Overgas11Pool	1	1 10hatch
ZvZ_Overgas9Pool	1	1 unknown
ZvZ_Overpool9Gas	3	3 overpool
ZvZ_OverpoolTurtle	1	1 overpool

BananaBrain’s recognition is generally close enough. 11 hatch can be treated like 12 hatch, though it is sometimes possible to exploit the slight slowness of 12 hatch. Treating 7 pool like 5 pool is probably close enough for bot play. The only harmful mistake was in recognizing overpool when facing the aggressive 9PoolGasHatchSpeed8D.

bananabrain as seen by microwave

bananabrain played	#	microwave recognized
10/12gate	86	61 HeavyRush | 16 Unknown | 8 NakedExpand | 1 SafeExpand
1basespeedzeal	1	1 Unknown
2basespeedzeal	2	1 NakedExpand | 1 Turtle
4gate2archon	2	1 Turtle | 1 NakedExpand
5gategoon	30	15 SafeExpand | 7 NakedExpand | 5 Turtle | 2 HeavyRush | 1 Unknown
9/9gate	15	9 HeavyRush | 4 Unknown | 2 NakedExpand
9/9proxygate	2	1 Unknown | 1 HeavyRush
bisu	2	1 SafeExpand | 1 Turtle
neobisu	2	1 NakedExpand | 1 SafeExpand
sairdt	5	3 HeavyRush | 2 Unknown
sairgoon	5	4 SafeExpand | 1 NakedExpand
sairreaver	2	1 Turtle | 1 NakedExpand
stove	3	1 Unknown | 1 HeavyRush | 1 NakedExpand

Here we see where BananaBrain’s NakedExpand losses came from: It didn’t simply play 12 nexus or whatever, but expanded behind the cover of pressure from some other build. Could it be that Microwave often didn’t scout the nexus unless it had defeated the pressure? Looking at this, it seems more like a BananaBrain weakness—eagerness to expand at a given time whether it is safe or not—than a Microwave strength. But then again, Microwave scored well against NakedExpand for almost every opponent that played it.

Microwave’s history files include large doses of training data prepared before the tournament. I snipped that data out, so the tables here include only tournament games and the 7 post-tournament games that are excluded from the official results.

Microwave has many strategies. I counted 79, compared to 73 last year (my first impression was that there were many more this year, but it was not true). Like Steamhammer, when losing badly it flails, trying anything.

See NakedExpand in the enemy table: Microwave was able to punish BananaBrain when BananaBrain made nexus before cannons. It’s a theme. I think it indicates general aggressiveness or rushiness in the early game. I get 6:52 for Microwave’s median game length when defeating BananaBrain.

Microwave had 3 builds that scored above 50%, and others that were close, but experimented too much. It would have scored much higher if it had exploited more and explored less. I think the lesson is that the learning algorithm’s exploration parameter should be set depending on the opponent’s expected strength. If you’re expecting to score 35% and you find a choice that scores 60%, reduce your exploration and exploit the winner unless and until the opponent adapts. If you have more than one high-scoring build (and they’re not too much alike), switch between them and your opponent will have more trouble adapting. If you’re expecting to score 70%, keep exploring if your best choice is only 60%.

Even against a higher-finishing zerg, Microwave punishes the fast expansion when it recognizes it. It’s striking. Zerg is not able to add static defense to a base before the base finishes, so this ability to restrict the oppoent’s strategy could be even more powerful in ZvZ.

And against terran, too. Against Microwave, if expanding early, apparently terran and protoss should add defenses at the natural first.

The first opponent that Microwave outscored, and it was a runaway. Steamhammer struggled versus DaQin, but the other zergs were fine. Later I’ll examine why to see if there are lessons for Steamhammer.

The strong results against NakedExpand show here too.

Like McRave but apparently for a different reason, Microwave had unnecessary trouble with UAlbertaBot. It takes more than a simpleminded learning algorithm to adapt to a random opponent with such different rushes for each race. The ideal answer is to adapt during the game after scouting. Steamhammer’s answer is a super-turtle build that defeats all of UAlbertaBot’s rushes. But still, see that 100% next to NakedExpand?

These tables tell more about McRave than about WillyT. Blue is good for McRave, red is good for WillyT.

mcrave strategies versus willyt strategies

	overall	1 rush	2 fe bio-mech	3 fe mech	4 tonk
overall	48/157 31%	16/41 39%	9/54 17%	12/47 26%	11/15 73%
HatchPool,12Hatch,2HatchMuta	29/89 33%	10/20 50%	6/32 19%	10/33 30%	3/4 75%
PoolHatch,12Pool,3HatchMuta	19/55 35%	6/18 33%	3/16 19%	2/12 17%	8/9 89%
PoolHatch,Overpool,2HatchMuta	0/13 0%	0/3 0%	0/6 0%	0/2 0%	0/2 0%

I find it strange that McRave’s overpool into 2 hatch muta failed in every case. Is it a reaction build that turned out to be a misreaction to what WillyT does? Probably not, it was tried against every WillyT opener. McRave’s other 2 builds were about equal, though the table shows that they were best in different cases. Switching between them was likely correct. The ratio that they were tried in also looks good to me: You want a ratio that leads to the final results being about equal.

WillyT would have done better without 15 tonk builds.

willyt as seen by mcrave

willyt played	#	mcrave recognized
1 rush	41	40 Unknown,Unknown,Unknown | 1 RaxCC,1RaxFE,Unknown
2 fe bio-mech	54	23 RaxCC,1RaxFE,Unknown | 20 RaxCC,1RaxFE,1FactTanks | 6 Unknown,Unknown,Unknown | 5 RaxCC,1RaxFE,5FactGoliath
3 fe mech	47	33 RaxCC,1RaxFE,5FactGoliath | 8 Unknown,Unknown,Unknown | 6 RaxCC,1RaxFE,Unknown
4 tonk	15	13 Unknown,Unknown,Unknown | 2 RaxFact,Unknown,5FactGoliath

Both the rush and the tonk build usually denied scouting, which seems like it should have been important because the builds call for opposite reactions. Yet McRave defeated the tanks and had less trouble with the rush than with WillyT’s expansion builds. RaxCC and 1RaxFE seem simple enough to recognize, and were. The followup seems harder to recognize, and was. I doubt that so many were actually 5FactGoliath.

Not much to see here, because the pairing was one-sided. But there are still a few points to note. In the tables, blue is good for BananaBrain, red is good for WillyT.

bananabrain strategies versus willyt strategies

	overall	1 rush	2 fe bio-mech	3 fe mech	4 tonk
overall	146/157 93%	43/46 93%	71/79 90%	17/17 100%	15/15 100%
10/12gate	41/44 93%	7/10 70%	19/19 100%	10/10 100%	5/5 100%
12nexus	5/6 83%	1/1 100%	1/2 50%	1/1 100%	2/2 100%
2gatedt	0/1 0%	-	0/1 0%	-	-
32nexus	21/24 88%	17/17 100%	2/5 40%	-	2/2 100%
9/9proxygate	76/77 99%	18/18 100%	46/47 98%	6/6 100%	6/6 100%
dtdrop	1/2 50%	-	1/2 50%	-	-
stove	2/3 67%	-	2/3 67%	-	-

It’s striking how quickly BananaBrain gave up on a build; it only had to fail once in six games. The reason is of course that the other builds were doing better than that. Now we see the relative success of WillyT’s build 2 bio-mech: It provided all of the wins in the builds that BananaBrain gave up on, and a few other wins as well. Otherwise, only WillyT’s rush was able to score a few wins, and then only against BananaBrain’s zealot play.

willyt as seen by bananabrain

willyt played	#	bananabrain recognized
1 rush	46	39 2rax | 7 unknown
2 fe bio-mech	79	40 fastexpand | 25 unknown | 14 2rax
3 fe mech	17	12 fastexpand | 2 unknown | 2 1fac | 1 2rax
4 tonk	15	10 1fac | 4 unknown | 1 2rax

BananaBrain seems to have diagnosed builds mostly correctly, when it was able to at all. WillyT’s build 2 seemed to be better at denying scouting. There is no sign that reading the opponent’s build helped BananaBrain play better; it’s the opposite if anything. But with the results so lopsided, we shouldn’t expect much of a sign anyway.

A middle group of bots finished close to each other, from #5 McRave at 41.70% to #8 DaQin at 39.63%. #6 WillyT is the second of the group.

WillyT’s learning files record the bot’s strategy as 01, 02, 03, or 04. Last year it only went up to 03. There may be an expectation of going up to 10 someday! Here is how I translated the strategy numbers into names, based on the numbering in the bot’s top-level README.

#	name	description
01	1 rush	2 rax bio + SCVs
02	2 fe bio-mech	1 rax expand into bio-mech
03	3 fe mech	1 rax expand into mech
04	4 tonk	slowly make many tanks

Stardust got special treatment, and it was still only good enough for 3%. The tonk build seems to have been specially devised to give a chance against Stardust. I checked on BASIL and found that the chance there was never high. But it’s over zero, that’s better than Steamhammer did!

Switching between the rush and bio-mech was able to squeeze a little blood from BananaBrain. It’s interesting that mech scored lower, though the expected win rate is so low that it’s hard to be sure the difference is real. Does the build suffer from a weak timing?

The author explained in a comment that WillyT is weak at TvT because it does not understand siege lines. With only one terran opponent, it wasn’t critical. This version of Dragon always makes a slow start to the game, so any slowness in WillyT’s bio-mech build did not matter, and having tanks likely helped.

WillyT could not outscore Steamhammer, but it made a good attempt. I find it distressing that the rush won so many games; it’s a strong rush but not that hard to hold. Again, bio-mech was better than mech. Well, that’s more expected versus zerg, but I wonder whether the reason is the same as versus BananaBrain?

McRave is meticulous in defense, which shows in these numbers. But it suffered against 2-base play. I think we can infer that WillyT has good mutalisk defense.

The tonk build had success against Microwave, but was neither successful nor much played other than here and versus Stardust. I’d say the build is overspecialized, useful only in a narrow range of situations. The rush was overwhelming, though.

And again, bio-mech over mech. It’s not conclusive, but I feel that something may be weak in the mech build. Maybe WillyT is just better with marines.

FreshMeat is newer and perhaps not ready yet to face the rush.

UAlbertaBot, with aggressive openers and no strong defensive skill, also fell to the rush. It got outrushed. It strikes me that WillyT scored nearly the same against #5 McRave, #7 Microwave, #9 FreshMeat, and #10 UAlbertaBot, even though the four are different in style and strength. WillyT did not crush any opponent. To me that suggests some kind of inconsistency in its play: It may have flaws that even weaker bots can exploit sometimes.

McRave recorded all 157 games versus Dragon, but Dragon recorded only 155. I was able to align the files anyway, because Dragon’s missing games were clearly due to Dragon’s 2 crashes. I manually removed the corresponding games from McRave’s records. Then there was one more fix: The game of round 97 that McRave and Dragon both recorded that they had lost. Officially, McRave had timed out and Dragon won, so I manually altered Dragon’s game record to give it the win.

Dragon doesn’t record anything but win/loss and its own strategy, so it has nothing to say about how McRave played.

mcrave strategies versus dragon strategies

	overall	1rax fe	2rax bio	2rax mech	bio	dirty worker rush	mass vulture	siege expand
overall	51/155 33%	4/20 20%	14/40 35%	6/17 35%	9/24 38%	2/2 100%	3/6 50%	13/46 28%
HatchPool,12Hatch,2HatchMuta	21/66 32%	0/7 0%	5/14 36%	3/5 60%	2/11 18%	-	2/2 100%	9/27 33%
HatchPool,12Hatch,2HatchSpeedling	0/1 0%	-	-	0/1 0%	-	-	-	-
PoolHatch,12Pool,2HatchMuta	1/9 11%	0/2 0%	0/2 0%	1/2 50%	-	-	-	0/3 0%
PoolHatch,12Pool,3HatchMuta	14/36 39%	1/3 33%	6/14 43%	1/2 50%	6/11 55%	-	0/3 0%	0/3 0%
PoolHatch,Overpool,2HatchMuta	9/26 35%	3/4 75%	1/7 14%	0/4 0%	1/2 50%	-	-	4/9 44%
PoolHatch,Overpool,2HatchSpeedling	2/2 100%	-	-	-	-	2/2 100%	-	-
PoolHatch,Overpool,3HatchMuta	4/15 27%	0/4 0%	2/3 67%	1/3 33%	-	-	1/1 100%	0/4 0%

Another reaction build: PoolHatch,Overpool,2HatchSpeedling was a reaction to Dragon’s worker rush, and taught Dragon not to do that.

dragon as seen by mcrave

dragon played	#	mcrave recognized
1rax fe	20	12 Unknown,Unknown,Unknown | 5 RaxCC,1RaxFE,5FactGoliath | 3 RaxCC,1RaxFE,Unknown
2rax bio	40	31 Unknown,Unknown,Unknown | 6 2Rax,Main,Unknown | 1 2Rax,Main,1FactTanks | 1 RaxCC,1RaxFE,Unknown | 1 2Rax,Expand,Unknown
2rax mech	17	14 Unknown,Unknown,Unknown | 2 2Rax,Main,Unknown | 1 2Rax,Proxy,Unknown
bio	24	11 RaxCC,1RaxFE,5FactGoliath | 7 Unknown,Unknown,Unknown | 3 RaxCC,1RaxFE,1FactTanks | 3 RaxCC,1RaxFE,Unknown
dirty worker rush	2	2 Unknown,Unknown,WorkerRush
mass vulture	6	5 Unknown,Unknown,Unknown | 1 RaxFact,Unknown,2PortWraith
siege expand	46	18 Unknown,Unknown,Unknown | 13 RaxFact,Unknown,5FactGoliath | 9 RaxCC,1RaxFE,5FactGoliath | 4 RaxCC,1RaxFE,Unknown | 2 RaxFact,Unknown,Unknown

Not much to see here; we already knew that McRave had trouble scouting Dragon. Much of what it did recognize was correct, at least. Though what McRave saw as a proxy, Dragon called “2rax mech”. The game is game 2767 from round 61 (replay file) on Heartbreak Ridge. Nothing in it resembles a proxy; it must have been a McRave bug.

McRave and BananaBrain both recorded all 157 of their mutual games. I chose to put McRave down the left side of the strategy cross, because its longer strategy names make the table hard to read otherwise. I also trimmed off the “PvZ_” and “Z_” prefixes from BananaBrain’s strategy names for compactness.

Blue is good for McRave, red is good for BananaBrain.

mcrave strategies versus bananabrain strategies

	overall	10/12gate	1basespeedzeal	2basespeedzeal	4gate2archon	5gategoon	9/9gate	9/9proxygate	bisu	neobisu	sairdt	sairgoon	sairreaver	stove
overall	28/157 18%	8/54 15%	1/3 33%	1/5 20%	1/1 100%	1/1 100%	1/1 100%	2/8 25%	1/5 20%	1/4 25%	1/3 33%	2/12 17%	1/1 100%	7/59 12%
HatchPool,12Hatch,2HatchMuta	16/86 19%	3/28 11%	1/1 100%	1/3 33%	1/1 100%	1/1 100%	1/1 100%	0/1 0%	-	1/4 25%	0/2 0%	2/10 20%	1/1 100%	4/33 12%
HatchPool,12Hatch,2HatchSpeedling	0/1 0%	-	-	-	-	-	-	0/1 0%	-	-	-	-	-	-
HatchPool,9Pool,2HatchSpeedling	2/2 100%	-	-	-	-	-	-	2/2 100%	-	-	-	-	-	-
PoolHatch,9Pool,2HatchMuta	3/15 20%	0/2 0%	-	-	-	-	-	-	0/3 0%	-	-	-	-	3/10 30%
PoolHatch,9Pool,2HatchSpeedling	0/3 0%	-	-	-	-	-	-	0/3 0%	-	-	-	-	-	-
PoolHatch,9Pool,3HatchMuta	4/19 21%	4/13 31%	-	0/1 0%	-	-	-	-	0/1 0%	-	-	-	-	0/4 0%
PoolHatch,9Pool,6HatchHydra	0/2 0%	-	-	-	-	-	-	-	-	-	-	0/1 0%	-	0/1 0%
PoolHatch,Overpool,2HatchMuta	0/4 0%	0/2 0%	0/1 0%	-	-	-	-	-	-	-	-	0/1 0%	-	-
PoolHatch,Overpool,2HatchSpeedling	0/1 0%	-	-	-	-	-	-	0/1 0%	-	-	-	-	-	-
PoolHatch,Overpool,3HatchMuta	3/14 21%	1/9 11%	0/1 0%	-	-	-	-	-	1/1 100%	-	1/1 100%	-	-	0/2 0%
PoolHatch,Overpool,6HatchHydra	0/10 0%	-	-	0/1 0%	-	-	-	-	-	-	-	-	-	0/9 0%

There we have the explanation for the 2 lonely HatchPool,9Pool,2HatchSpeedling games: The strategy was a successful reaction to proxy gates. I read it as meaning that the build is 9 hatch, 9 pool, and gas soon for the zergling speed upgrade.

BananaBrain split its effort between 10-12 gate and the Stove (a scout into dark templar build), very different builds. McRave answered both mostly with 12 hatch into 2 hatch muta. A hydralisk opening would have been a more natural way to counter both, but play what you’re good at.

mcrave as seen by bananabrain

mcrave played	#	bananabrain recognized
HatchPool,12Hatch,2HatchMuta	86	84 12hatch | 2 unknown
HatchPool,12Hatch,2HatchSpeedling	1	1 unknown
HatchPool,9Pool,2HatchSpeedling	2	2 12pool
PoolHatch,9Pool,2HatchMuta	15	8 9pool | 7 overpool
PoolHatch,9Pool,2HatchSpeedling	3	2 overpool | 1 9pool
PoolHatch,9Pool,3HatchMuta	19	13 9pool | 6 overpool
PoolHatch,9Pool,6HatchHydra	2	1 overpool | 1 9pool
PoolHatch,Overpool,2HatchMuta	4	4 overpool
PoolHatch,Overpool,2HatchSpeedling	1	1 overpool
PoolHatch,Overpool,3HatchMuta	14	14 overpool
PoolHatch,Overpool,6HatchHydra	10	10 overpool

BananaBrain was accurate in recognizing 12 hatch and overpool, but had trouble with 9 pool. It did not try to narrow the build down any further than that.

bananabrain as seen by mcrave

bananabrain played	#	mcrave recognized
10/12gate	54	31 2Gate,10/12,Corsair | 8 2Gate,10/12,DT | 4 2Gate,10/12,ZealotRush | 3 2Gate,10/17,Corsair | 2 2Gate,Unknown,Corsair | 2 2Gate,9/9,DT | 2 2Gate,9/9,Corsair | 1 2Gate,10/17,4Gate | 1 2Gate,10/17,DT
1basespeedzeal	3	2 1GateCore,2Zealot,DT | 1 1GateCore,2Zealot,Corsair
2basespeedzeal	5	3 FFE,Forge,Speedlot | 1 FFE,Nexus,Speedlot | 1 FFE,Nexus,5GateGoon
4gate2archon	1	1 FFE,Forge,5GateGoon
5gategoon	1	1 FFE,Nexus,5GateGoon
9/9gate	1	1 2Gate,9/9,Corsair
9/9proxygate	8	7 2Gate,Proxy,ZealotRush | 1 2Gate,9/9,Unknown
bisu	5	2 FFE,Forge,Unknown | 2 FFE,Forge,5GateGoon | 1 FFE,Nexus,Unknown
neobisu	4	4 FFE,Forge,Speedlot
sairdt	3	3 1GateCore,2Zealot,Corsair
sairgoon	12	6 FFE,Forge,5GateGoon | 2 FFE,Nexus,5GateGoon | 2 FFE,Forge,Unknown | 1 FFE,Gateway,5GateGoon | 1 FFE,Gateway,Unknown
sairreaver	1	1 FFE,Forge,Unknown
stove	59	27 1GateCore,2Zealot,Corsair | 10 1GateCore,Unknown,Corsair | 6 1GateCore,Unknown,DT | 6 1GateCore,2Zealot,DT | 3 2Gate,10/12,DT | 3 2Gate,10/12,ZealotRush | 2 2Gate,10/17,4Gate | 1 2Gate,10/17,DT | 1 2Gate,10/12,4Gate

In 1-base protoss plays, McRave tried to distinguish when the gates were made, and often got it right but had some trouble. It seems like something you can’t do perfectly, even if you combine direct scouting of the gates with inferences based on the enemy army. Recognizing the enemy build precisely doesn’t seem possible in general, though you can usually get close.

Working through the learning files of various bots, I ran into a strange discrepancy: One game between Dragon and McRave which both bots recorded as a loss. In the detailed results, the game ID is 4387 from round 97. The official result has Dragon winning after McRave timed out.

What really happened? I watched the replays recorded by both bots. On the surface, both replays of the game looked the same. McRave destroyed all of Dragon’s buildings, then as usual the replays continued a few more moments. According to OpenBW, Dragon’s replay ended at 22:10 and McRave’s replay at 22:09 (according to the official results, the game ended at 21:57). It’s reflected in the file sizes; Dragon has 1,553,196 bytes while McRave has 1,552,818 bytes (in other cases I checked, differences between replay sizes for the same game are less than 10 bytes). Here is a view at the end of Dragon’s replay, followed by the end of McRave’s. Notice that the valkyrie has moved a little farther in Dragon’s version, and Dragon shows supply 0 for both bots at the end (supporting that they both lost).

On the face of it, Dragon lost because all its buildings were destroyed, then in the brief runout of the game before Broodwar stopped, McRave lost by timing out (the tournament manager said it timed out, plus it recorded its own game result so it didn’t crash). That’s how they both believed they lost. The tournament manager, I have to assume, didn’t expect that situation and took the timeout as definitive, recording that Dragon won.

I wrote to Dave Churchill about it and got this answer:

I don’t have any time to work on this right now and it seems like a pretty small edge case, so if you could post about it and crowd source the fix that would be ideal. I’ll accept any pull request that makes the tournament better!

It’s not critical, it’s a rare case that is not even close to affecting the tournament finishing order. But it would still be nice to fix it.

The first job may be to read code and/or run experiments to figure out more exactly what actually happened. Regardless of the course of events, there are two issues to solve. 1. What should the game result be? 2. How can we tell both bots the correct result?

1. the game result

It was a misplay. Both bots messed up fatally. Don’t count it as a win for either, but skip the game.

McRave won. All terran buildings were destroyed, and that’s the winning condition, right? Never mind that McRave had trouble with time later. Why is there a later at all?

Dragon won. The tournament needs to control how much time bots take, no matter when they take it, as a matter of efficiency and fairness. Therefore the game is not over until Broodwar stops it.

Reasonable people can disagree. I don’t think the answer matters much. What’s more important is to make sure that the actual game, the tournament manager, and the bots all agree on the result as much as possible. I think that issue 1 and issue 2 are interrelated, and should be answered together, not separately.

2. notifying the bots

I don’t understand the technical details of how bots are notified that they won or lost much beyond “somebody calls onEnd()”. But I have poked at it a bit. Looking at it from the tournament end, the software includes a java tournament manager, a C++ tournament module that is part of BWAPI, and then the bots and game itself.

I think the outline is this: When a game completes normally, there is a short runout phase, and then the tournament module notifies both bots of the result, all good. If a bot times out, the tournament module kicks it out of the game immediately and notifies it of its loss. Then Broodwar realizes there is only one player left, calls the game over, and the remaining player wins. Usually good. In this case, I think Dragon lost and the game entered the runout phase. Then McRave timed out, was kicked out and notified of its loss. Then, when Broodwar ended the game slightly later (giving Dragon the longer replay), for whatever reason the tournament module told Dragon it had lost too. Meanwhile, the tournament manager took the timeout as definitive and recorded that Dragon won. Is my thinking correct? I think it’s close, but I may have details wrong.

The goal is to find a fix so that one of the game results of issue 1 is decided on and carried through consistently. It would be nice if the fix only affected the java tournament manager, but I don’t know if that’s possible.

I’m taking the bots in finishing order; McRave is next. Last year I analyzed McRave’s three-part strategy representation and learning algorithm. These apparently have not changed in outline, though details may have changed. The set of available strategies has been updated. For example, 6HatchHydra is new this year. It follows that the set of enabled strategy triples has also changed.

McRave is much stronger this year. It has become noted for dangerous mutalisk control.

9 pool into 2 hatch muta worked best, with 3 wins out of 22. That is not intuitive. The more natural 12 hatch into 2 hatch muta was tried more but was less successful. Did Stardust react inefficiently to the 9 pool? McRave appears to correctly understand that Stardust ends up with 4 gates despite taking different routes to get there. That’s kind of impressive.

Last year McRave scored 3 out of 150 against Stardust. This year it scored 8 out of 150 against a stronger Stardust—which seems to have updates specifically to defeat McRave, since McRave was the only bot to upset it in CoG 2021. Good progress!

McRave’s wins over BananaBrain are dominated by games where BananaBrain timed out. In game 2754, BananaBrained timed out with 320 frames over 55ms, when McRave had 318 frames over 55ms—close call! McRave had more timeouts than any other bot, but only 3 losses to BananaBrain by timeout. BananaBrain’s timeouts seem to be concentrated on Dragon and McRave, and to a lesser extent on Steamhammer.

If HatchPool,9Pool,2HatchSpeedling was tried twice and won twice, why wasn’t it tried more often? The first try was on game 30 of 0-156. I imagine that it was a reactive build, not enabled as an initial choice but switched to under given circumstances. I didn’t read the source to verify that.

The enemy table shows a complex set of strategies by BananaBrain.

Over half of McRave’s losses to Dragon were by timeout. I think Dragon is an especially easy bot to time out against, because its strong macro and big battles with light units put heavy demands on the opponent.

The enemy table shows 89 games with Unknown,Unknown,Unknown. Apparently Dragon often denied scouting. Presumably the scouting overlord was afraid to approach due to marines, and any scouting drone was turned away. Also, I wonder about 2Rax,Proxy,Unknown. Did Dragon really proxy once, or was it a misrecognition? On Python, bases can be close by air. If McRave measures proxy distance by air distance, it might take a barracks in the enemy main for a proxy.

McRave chose from the same fixed set of 3 strategies against all the zergs. Only the 1 hatch mutalisks were able to hold their own with Steamhammer.

Hurrying the mutas too much did not help against WillyT. The enemy table shows that WillyT sometimes countered with goliaths. Does McRave later make a hydra switch to fight the goliaths? WillyT sometimes goes for goliaths with 2 tanks, and it’s sensible to fight back with hydralisks. I didn’t see a hydra switch in the games I watched.

I noticed that McRave doesn’t clear the terran scout from its main until mutas come out. WillyT gets to know the exact timing for its turrets with no need to spend a scan.

The mutalisks did in DaQin. DaQin’s slow start puts 12 hatch ahead of other choices; DaQin makes cannons before nexus regardless of what the opponent does. DaQin defends its natural entrance with cannons, but not its nexus, so the mutalisks have a free hand and DaQin finds itself short of probes.

McRave met UAlbertaBot with the same strategies as last year (except for one stray PoolHatch,12Pool,2HatchMuta this year). The same strategies by name, that is. The actual play was different and performed far worse against UAlbertaBot’s pressure builds. I looked at some games. When McRave respected its enemy and defended itself, it generally won. Sometimes it seemed to arrogantly conclude “Pff, you’re not worth spending a sunken on” and got overrun. As far as I could tell from watching games, it wasn’t a scouting miss—though it’s easy to overlook things in watching games. It had the feel of a bug.

The submitted Steamhammer was mistakenly configured to retain 100 game records per opponent. I had thought it was set for 200, and didn’t double-check. So of the 157 games against each opponent, of which 150 counted in the tournament, I have records for only the final 100. That’s 93 tournament games plus the 7 extra at the end.

My prepared data was successful. For all opponents which I prepared for, the prepared openings were among the highest scoring (including the zero score versus Stardust). It’s notable that Steamhammer’s gas steal was not successful against any opponent, perhaps another sign of an elite tournament. It was either infeasible or abandoned as a failure against every opponent except DaQin, and did no good then.

Steamhammer’s game records are rich with data. To show a little bit more of it, I added a new feature in the opening table. There are new “wins” and “losses” columns showing the median time that winning and losing games with that opening lasted. The median is a better measure than the mean, because we can expect the distribution of game times to be right-tailed: Games are limited to between zero minutes and one hour, but we expect a hump nearer to zero and a long tail of slower games. That inflates the mean and makes it misleading. For the tournament, I turned off surrendering, so Steamhammer played its losses out to the end.

Steamhammer lost every game, but there is still valuable info here. If you’re losing all games, the game duration is a plausible proxy for how much trouble you caused the opponent. Especially so if you tried a rush opening and ended up in a long game—either the rush did some damage, or the opponent overreacted and was slowed down. Here, a couple of 7 pool builds were among the longest games. Steamhammer probably should have repeated them.

Notice the 4 pool and the hive rush. Steamhammer tried the whole range. Steamhammer recognized Stardust’s build in 2 games as nexus without cannons, a reaction that Stardust did not have last year. Otherwise, results are similar to last year’s.

In the 150 tournament games, Steamhammer scored 25 wins versus BananaBrain. Of those, 15 were due to BananaBrain suffering a frame timeout. Ouch. The game scores say that BananaBrain was ahead in 11 of the 15 games when it timed out. So the win percentages and times need to be interpreted carefully. The wins overall were longer games than the losses, possibly because BananaBrain was more likely to time out in a longer game with a larger game state to model and more units to control.

11HatchTurtleLurker scored over 50% in 15 games! Is it particularly good at prompting BananaBrain to time out? If I’d known about it ahead of time, I could have added it to my preparation and perhaps scored higher.

The build 2x10HatchSlow is shown as tried 7 times with no wins. I know from watching games that the opening scored wins earlier in the tournament, before the final 100 games; that is why it was tried so often later on. The build is very similar to Broken Horn’s 10 hatch-9 hatch-pool, but (I think) slightly more efficient. Apparently BananaBrain learned to avoid lines that lose to the mass of slow zerglings.

Successfully stealing its gas caused BananaBrain to take its gas sooner. I haven’t seen that before. In any case, it was only 3 games; Steamhammer found the gas steal unprofitable.

The most successful openings were 5HatchPool (5 hatcheries before pool, a supremely greedy build to exploit bots that never attack early) and ZvT_3HatchMutaExpo, the two openings I selected as preparation. For bots carried over from the previous year, good preparation is easier.

Dragon has a chaotic play style. Steamhammer’s wildest game of the tournament may be Steamhammer-Dragon on Longinus (replay file). Dragon played a V strategy: Vultures, valkyries, and vessels.

Again two of my prepared builds, 9PoolHatchSpeedAllIn and ZvZ_OverpoolTurtle, were the top choices. Both are tough for most zerg bots to handle. My other prepared build, ZvZ_Overgas9Pool, does not appear in these 100 games. Apparently it flopped and was abandoned early. Rushy builds ended up winning faster than they lost, and more macro builds were the reverse, as you might expect. The timing table shows that McRave went spire nearly every game (overlords do not count as “air units” there), and not slowly. That’s normal for ZvZ, of course, but it shows that McRave did not favor builds to overrun the opponent with zerglings.

WillyT has become much stronger over the past year. It is better at handling Steamhammer’s lurker builds—except for the especially early 9 pool lurker build, which apparently catches it unready. Steamhammer’s improvements in lurker play were important to keep up with progress. I think Steamhammer’s diverse mix of openings was essential to counter WillyT, which has its own diverse mix and will figure out how to counter anything that is too predictable.

Steamhammer’s closest game of the tournament was Steamhammer-WillyT on Empire of the Sun (replay file). Steamhammer decisively stopped WillyT from taking the nearby north island base, but allowed it to hold the distant south island despite scouting it the instant it started. Notice WillyT’s interesting but somewhat uncoordinated dropship play throughout the game.

Microwave had too many weaknesses. Of the 4 openings with 80% plus win rates, 3 were from preparation and one was Steamhammer’s discovery during the tournament. It’s interesting that the 12 hatch build ZvZ_12HatchMain was faster to win than to lose. I think that means it won with zerglings from its extra larvas.

The plan table shows that Microwave followed its own broad range of plans. In the timing table, see the wide and matching variation in Microwave’s gas timing and air unit timing. Did Microwave never get zergling speed in long games?

After this upset, I think I’ll take DaQin as a test opponent and finally figure out the skills to defeat it. DaQin is a Locutus fork, so beating it probably means doing better against other protoss bots.

The timing table shows that DaQin was remarkably late with air units. That includes both corsairs and observers—DaQin was late with both of them. In fact, I don’t remember whether it makes corsairs at all. Mutalisks might be a good choice to win.

When I was preparing opponent-specific data, Steamhammer had an overwhelming score against FreshMeat on BASIL. This result is good but not overwhelming; FreshMeat improved a lot in a short time. I had recognized that FreshMeat had made great strides, but there was not enough recent data to show what was working in the most recent games. So I made no preparation at all. These tables show an example of how Steamhammer figures out an opponent from scratch. I think it did OK.

Comparing this year to last year, Steamhammer actually did a little worse against UAlbertaBot. The skills I improved over the last year didn’t include skills to defeat UAlbertaBot’s pressure builds, or to adapt better to its random race.