Starcraft AI blog

Regardless of any other adaptation skills you may have, if you can predict your enemy’s opening build, you can better counter it. But it’s not simple. Steamhammer tries to distinguish between opponents that play a fixed build, and those that vary their openings. For those that vary their openings, there is much more: Some choose randomly. Some like to repeat the opening that won the last game, switching only on loss. Some stick with an opening that wins more than a given percentage. Some try openings systematically, “A didn’t work, B is next.” Some choose randomly with some probability distribution over the more successful choices. Sometimes openings are treated as black boxes distinguished only by their names, sometimes as strategies which are understood to counter other strategies (Steamhammer does both at different times).

I am wondering whether it makes sense to write a rock-paper-scissors analyzer that tries to tease out exploitable patterns in the opponent’s behavior (there are established techniques), and combine it with a game solver to make better initial opening choices. On the one hand, many bots have exploitable patterns that I know about. If an RPS analyzer can find the patterns too, Steamhammer might seem to gain the mysterious “star sense” to always play the right thing for no visible reason. On the other hand, it’s relatively easy to reduce your exploitability to a low level—use randomness wisely. Also, as Steamhammer gains skills to adapt its strategy during the game, the initial opening choices make less difference. The gain might be little. And by trying to exploit patterns, Steamhammer could itself become more exploitable; it might backfire.

The parts of the system would be:

1. Classify the enemy build. Steamhammer already does this, though it needs improvement.

2. Statistically analyze the sequence of (win/loss, my opening, your opening) under the assumption that the opponent is trying to counter what we’re doing. Knowing what-counters-what may factor in. The output should be a probability distribution over opening classes, “what are they likely to do?”

3. Knowing what-counters-what definitely factors in here: Solve the game. We start with a prior probability of winning for each of our openings against each opening class the opponent might play, and thanks to Bayes we can update it as we learn about the opponent. That gives us a game matrix with uncertainties in the payoffs. (Since Steamhammer knows a huge number of opening builds, making the game matrix too big to fill in, I would classify Steamhammer’s openings too so that the output only decides which class of opening to play.) Without an RPS analyzer, we can solve the game (I expect I would use a Monte Carlo method to handle the uncertainties) and play not far from a Nash equilibrium (i.e., almost perfectly assuming unexploitable play from the opponent). If an RPS analyzer can make a good estimate of the opponent’s plans, in the best case we can do better: We can exploit the opponent’s deviation from the Nash equilibrium to win more than if we played to a Nash equilibrium ourselves.

It’s unclear to me whether either the RPS analyzer or the game solver is worth the effort. Does anybody have an opinion? Perhaps some bot I haven’t looked at has similar ideas?

Here’s data from Microwave’s history files, using the same script as for BananaBrain with a little customization. Unlike Microwave’s learning files, which deliberately omit data and include information from pre-learning, the history files tell what Microwave actually did during the games. Microwave didn’t record information about the opponent’s strategy, so that table is left out. That made it look a little sparse, so I added columns giving the first and last games when the opening was tried, where the first game in the history file is game 0. We can see things like when a winning opening was found, and whether it kept winning. If there are fewer than 100 games recorded for an opponent because Microwave crashed, then the game numbers generally do not align with the tournament round numbers.

Against difficult opponents, Microwave experimented widely. Against some opponents that Microwave pre-trained against, it played whatever came out of pre-training. So I don’t have much to say about opponents in the top half of the post. But toward the bottom I’ve made some comments. Especially see the note to AITP.

Why are only 71 games recorded? According to the official results, Microwave crashed in 56 games throughout the tournament, and 29 of those crashes happened against ZZZKBot. Microwave recorded every game in which it did not crash. It’s a debugging opportunity. :-/

As soon as Microwave found that 7 pool worked, it played 7 pool exclusively.

Microwave was late to discover the success of the hydra bust opening. That’s why it was played so little. The example shows the importance of finding good ideas as early as possible. I am adding smarts to Steamhammer to make it better at finding the good tries fast.

It’s interesting that 2HatchMuta and 4PoolSoft have the same numbers, but were given up on at different times.

The choice against UAlbertaBot was determined by pre-training. From scratch, I expect Microwave would have tried a wider variety.

Apparently the initial choice against an unknown is random.

I wrote a script to analyze BananaBrain’s game history files, which record its experience with each opponent. For now, I had the script summarize the strategies played and the enemy strategies recognized. The history files also record the map and a value that represents the game duration. History files are rich with information, and there are many ways to summarize it. It would be interesting to see how strategy usage and win rate vary by map, among other possibilities.

The same script should work with minor changes to summarize Microwave’s history files.

BananaBrain had prepared history files for the opponents #1 Locutus, #2 PurpleWave, #5 Steamhammer, #6 ZZZKBot, #7 Microwave, and #8 Iron. Data from the prepared history files was not copied into the write directory. That is different from how Steamhammer and Locutus keep their game records, and it has the nice effect that the tables show exactly what happened in the tournament, from BananaBrain’s point of view.

For each opponent, the left table is BananaBrain’s choice. The right table is BananaBrain’s idea of what the opponent did. All the win rates are from BananaBrain’s point of view, so that, for example, when Locutus played P_1gatecore, BananaBrain won 5% of the time. Of course, the opponent’s view of its own strategy is likely to be more fine-grained than BananaBrain’s. To take the extreme case, Steamhammer played 30 different openings against BananaBrain, and BananaBrain recognized them in 8 categories.

As you might expect against Locutus, the best choice was a fast expansion.

Is the single game of enemy P_fastexpand a misrecognition? I suspect that Locutus played otherwise, and BananaBrain didn’t see everything and wasn’t able to draw the right conclusion. Or maybe it’s a bug somewhere. PurpleWave and McRave also show a single P_fastexpand game.

Against PurpleWave, different zealot rushes worked best. Maybe it is because zealot rushes depend for their success more on execution than on the enemy’s strategic reaction. PurpleWave is particularly good at reacting to the enemy strategy, and BananaBrain is good at execution.

BananaBrain made quite a variety of tries, and was most successful with... zealot rush and dark templars, which are kind of different. BananaBrain’s varied opening choice is a strength.

2 gate zealot openings work well against Steamhammer—but only when played by PurpleWave or BananaBrain. Steamhammer can usually defend versus a lesser protoss.

I like that BananaBrain varies its opening choice even when several openings win 100%. (Steamhammer does too; if more than one opening has scored 100% so far, Steamhammer chooses randomly among them.) Playing a strong opening gives the opponent one problem to solve (“how do I survive this?”). Unpredictably playing one of several strong openings sets the opponent two problems (“what is this fiend doing, and then how do I live through it?”) which must both be solved, more than twice as difficult.

Bulldog! That involves protoss dropping zealots, typically on cliff tanks, with a simultaneous attack by ground. When successful, a bulldog can abruptly break a terran defense that is sound against any purely ground attack. I don’t think I’ve seen BananaBrain play that; I should watch more games versus terran. Can anybody point out an example?

The Stove worked against XiaoYi? Again, XiaoYi shows weakness against tricks. The Stove involves making scouts to harass while teching to dark templar. It should not be hard for a good terran to defend against; notice that Iron dealt with it well enough.

It looks like most openings performed similarly against McRave, and BananaBrain struggled to identify what worked. I imagine a fierce learning battle, both trying to keep one step ahead.

BananaBrain apparently does not have a bunker rush recognizer.

AITP scored zero against over half of the participants, so its learning results are not deeply interesting. Also, its strategies are labeled with opaque sequences of letters and numbers. But it was easy to generate the tables, and they offer a little insight into AITP’s interesting design, so here they are.

Unlike other Steamhammer forks, AITP does not spell out concrete opening builds in the configuration file, at least not beyond 4 x SCV—start by making workers. The strategy names themselves are code sequences that tell what to do throughout the game. The letters A, B, C are stages of the game, and the combinations A1, A2 etc. are “modules” that may be active during the matching stage. Each module has its own update method to decide what to build, and the StrategyManager sometimes checks the current module for other decisions. There is module switching code in case of surprises (StrategyManager::shouldSwitchModule()); it also sets flags and updates other information.

I like it, it’s a flexible way to specify a plan for the whole game, and allows for changing plans on the fly. It’s an abstract strategy system, similar in principle to what I plan for Steamhammer. My implementation will look entirely different, though.

AITP has only 5 strategies configured. I gather that it can switch to other sequences on the fly if circumstances warrant. 5 is not many, though; I think they have only completed the basics. Here is the Steamhammer opening group it assigns to each strategy. It does not use the opening group strings, but they may have some heuristic value:

• A1-B3-C2 AntiRush
• A1-B1-B2-C2 Rush
• A3-B5-C1 NoneBunker
• A3-B7-C1 NoneBunker
• A4-B2-C1 8BB (does that mean BBS?)

AITP lost every game, but did not explore its possible strategies equally. It seems to have priorities. Maybe later I will look into how that works. AutoGasSteal is set true in the configuration file, but AITP did not record itself as stealing gas against any opponent. Presumably it is turned off in the code.

Steamhammer is the highest-ranked opponent that AITP scored wins against. It looks like a few scattered games, though.

It looks like ZZZKBot played its 4 pool in over half the games, and perhaps its guardian rush in the remainder. A1-B3-C2 is the strategy labeled AntiRush. AITP recorded more wins for itself than it actually scored, despite recording fewer games than it played. I suspect that AITP has changed the meaning of the numbers.

Again, AITP recorded fewer games and more wins than happened. Is it a bug, or is it intentionally over-recording wins for certain strategies to focus its search? Or what? AITP is interesting, it deserves a closer look into the code.

DaQin is derived from Locutus and also keeps 200 game records. But DaQin did not have pre-learned data. No games were left uncompleted; there are 100 against each opponent.

DaQin plays fewer builds than the other bots I’ve looked at so far.

DaQin had an enemy-specific strategy configured for Locutus, so it didn’t try anything else. Locutus is the only opponent that DaQin tried to prepare for, as far as I can see.

DaQin incorrectly recognized dark templar rush as Locutus’s strategy in most games, then correctly recorded that no cloaked units were seen during the game. See yesterday for Locutus’s play against DaQin, which did not include any DT build. I assume that the dark templar recognition is deliberately over-cautious, because DTs are dangerous. Locutus does have a fake dark templar build, where it adds a citadel of Adun to fool opponents into expecting dark templar (it works against most UAlbertaBot-derived bots).

Against PurpleWave, in contrast, DaQin less often foresaw dark templar, but apparently often faced them. (Arbiters can’t get out that fast.)

How did ZZZKBot upset DaQin? These numbers suggest zergling bust (it could be hydras, but DaQin does have a hydra bust recognizer which did not fire): Mostly “heavy rush,” few mutalisks, no lurkers. Steamhammer also settled on zergling bust as the best bet, but was much less successful. Microwave tried its zergling bust build versus DaQin without success. Maybe ZZZKBot’s extreme aggression is the key.

12NexusCarriers seems to be the default build versus terran. Apparently terrans, even Iron, were not able to punish the fast expand. Well, they’re not supposed to be able to without risk, that’s the point of cutting probes for nexus on 12, but it does require good play from protoss to ensure.

XiaoYi usually got air tech pretty fast, that’s unusual and interesting. I’m guessing it scouted the carriers coming and prepared wraiths.

McRave upset DaQin. Dark templar in 3 out of 4 games, and they came out pretty early. PurpleWave showed a similar pattern, but it wasn’t as salient because it wasn’t an upset. The dark templar rush recognizer did not seem to be fully effective, possibly because it was overridden by the fast rush recognizer. DaQin’s best counter was DT-back-atcha.

DaQin had some trouble adapting to random UAlbertaBot. This is a point where preparation for the opponent would have been valuable: Make a build that UAlbertaBot can’t beat and ensure that it is played. It can be a general-purpose build; PurpleWave included a cannon turtle build that is safe against all sorts of rushes.

Beating BunkerBoxeR with a build of fast expansion into carriers is... not the intuitive choice. But I guess it worked.

Locutus’s game records are in almost the same format as Steamhammer’s and can be summarized by the same script. I expect it will also work for DaQin and AITP.

Where Steamhammer was set to keep 100 game records per opponent, Locutus was set to keep 200. Since there were 100 rounds in the tournament, game counts over 100 mean that pre-learned data is included in the table alongside the tournament data. If Locutus was not trained on a near-final version of the opponent, then the two could be significantly different.

After seeing a few Locutus-PurpleWave games I got the impression that PurpleWave reacted adequately to Locutus’s trick strategy of cannoning the ramp and then dropping zealots. So I was surprised that Locutus considered it the best choice. But the overall win rate is high compared to the tournament results, so I suspect it is influenced by pre-learned data from games against a weaker version of PurpleWave.

The numbers in the “recognized” columns of the plan table show how widely Steamhammer cast its net for a solution to Locutus.

Locutus never tried to steal the gas of a zerg. Objectively, that makes sense. In the context of bot play, I’m not so sure; many bots of all races mess up their builds in the face of a gas steal.

Locutus declared an enemy-specific strategy against Iron. I’m not sure why it also had pre-learned data.

Proxy DT rush. That tends to confirm my picture of XiaoYi as vulnerable to tricks.

Locutus configured an enemy-specific strategy against UAlbertaBot. Openings other than CannonFirst4GateGoon are from pre-learned data, which was ignored in making the opening decision.

Today is Steamhammer. With a mid-rank finish and the widest range of builds, plus informative game records, Steamhammer may give us the best insight into how other bots played.

The tournament was 100 rounds, and Steamhammer was configured to remember the previous 100 game records, because in play there is no reason to remember more (earlier records are increasingly discounted). Steamhammer also had pre-learned game records for many opponents, so when the game record count reached 100, new records added caused old pre-learned records to drop away. Not all 100 tournament games happened for each opponent, but the pre-learned games filled in the small gaps so that Steamhammer ended up with exactly 100 game records per opponent in every case.

The “opening” table counts Steamhammer’s opening choices. The “plan” table shows the plan that Steamhammer first predicted that the opponent would play, then recognized that the opponent was playing. Both prediction and recognition can be wrong. The timing table is new this year, an attempt to get a little more information out of Steamhammer’s rich game records. For some events, it gives the count of games in which the event occurred, and the median time, earliest time, and latest time it occurred in those games when it did. The times are given under the assumption that 1 second of game time is exactly 24 frames, a simplification.

• gas steal attempt - When Steamhammer sent out the drone to steal gas (if it did).
• gas steal success - Whether the gas steal attempt succeeded in taking the opponent’s gas. Steamhammer doesn’t record the time it happens, so this is only a success count.
• enemy scout - When the enemy scout first reached Streamhammer’s base.
• enemy combat units - When the first enemy combat unit was seen.
• enemy air units - When the enemy is first known to have tech for flying units (except overlords).
• enemy cloaked units - When the enemy is first known to have tech for cloaked units.

It looks like Locutus opened forge-expand every game. It worked. Steamhammer desperately tried everything, including ZvZ builds and island builds, and finally squeezed 3 wins with a risky extreme macro opening, 4 hatcheries before spawning pool, which was able to win one game in five. I should add 5 and 6 hatch before pool and see if they help.

Locutus rarely made corsairs or dark templar. I wonder what its criteria are? Maybe it won before it got that far. The scout was usually quite early, and the first combat unit was seen late, as expected for a cannon-first opener.

I played over the 3 wins. They were in rounds 65, 70, and 73; after that, I expect that Locutus found a way to win. In 2 games, Steamhammer pulled ahead in early economy with its greedy opening, then struggled to defend and fell into a losing position. But Locutus got most of its units stuck in its base, and Steamhammer was able to turn it around and win after a hard fight with critical defiler support. In the third win, Locutus chose a zealot-archon unit mix that Steamhammer knows how to cope with, and zerg powered through.

PurpleWave in contrast went with mostly 2 gate openings against Steamhammer; that’s what “heavy rush” means for protoss. Steamhammer countered with early sunkens plus hydras or, less successfully, mutalisks (this version had a bug that weakened mutalisk play). There are also 3 wins with a ZvZ fast mutalisk opening. 2 gates should beat that, so protoss either played poorly or chose a different build in those games.

BananaBrain contrasts with both previous opponents in that it played a variety of builds. Steamhammer was unable to predict what was coming. It looks strange that the best reaction was an opening designed to counter terran factory-first builds that include a vulture runby, but in fact it is a mildly specialized 2 hatch mutalisk variant and not so surprising. BananaBrain made corsairs and dark templar in most games.

DaQin played forge-expand and has similar timings to Locutus, for the same reasons. The fast scout is to allow adjustment of the cannon count and timing, and the late combat units are due to getting a gateway later. Steamhammer couldn’t find any better reaction than to try to bust with zerglings, either early or late, and it was not particularly successful.

ZZZKBot mostly played a turtle-into-mutalisks strategy against Steamhammer, and was somewhat successful. You can read the idea straight out of the tables above. The 2:25 earliest timing but 4:06 median timing for combat units says that ZZZKBot sometimes rushed zerglings, but usually not.

Microwave played a 9 pool speed build into expansion and then spire, which you cannot read out of the plan table because Steamhammer didn’t recognize it accurately. But in the timing table you can see that combat units (zerglings) were early and air units (mutalisks) were not late.

Steamhammer was not able to steal Microwave’s gas. It probably should have stopped trying.

Look at that huge range of scout timings! 0:34 means that the scout SCV was sent immediately at the start of the game and went directly to the zerg base. 15:09 probably means that no enemy unit got into the base until the end of the game when Steamhammer lost (Steamhammer is on BWAPI 4.1.2 and cannot detect scans). Steamhammer prevented the scout entirely in 13 out of the 100 games by its own count; 15:09 is probably the same. Steamhammer was not able to steal Iron’s gas, and did eventually give up trying.

Steamhammer liked 7 pool against XiaoYi, just as Microwave did, but also liked its dawn hydra rush.

Thanks to pre-learning, I expected Steamhammer to play its overpool turtle build every game. I’m not sure why it didn’t. I also don’t know how it hit on the 5 scout build, which means send out a drone at 5 supply to scout very early, then leave all decisions to the strategy boss. It’s a logical try against a random opponent, especially one that has a single strategy for each race, and it was fairly successful. But it did not appear in the pre-learned data.

AITP scored zip against both mass zerglings (9HatchExpo9Pool9Gas) and against fast mutalisks (AntiFact_13Pool). And it successfully scouted Steamhammer’s base only 25% of the time. If you don’t scout reliably, it will be hard to withstand rushes.

Steamhammer was not able to judge whether BunkerBoxeR was playing a proxy (with its proxy bunker) or a worker rush (since it sent SCVs in support). But it didn’t matter. The reactions are nearly the same. Since BunkerBoxeR never wants gas, stealing its gas was a waste.

Steamhammer knows 142 different openings. In the whole tournament, it was only able to try 111 of them! It tried the most openings versus protoss, since it was looking everywhere for an escape from the overwhelming top protoss bots. Most openings were tried only a few times and lost every game, which means that Steamhammer would have performed better without them. That’s expected and even intentional; my plan is to add smarts until it is able to make good guesses about what to try. The work is underway.