Starcraft AI blog

The AIIDE 2017 win rate over time graph shows #1 ZZZKBot slowly and steadily learning throughout the tournament. It’s easiest to see if you turn off most of the lines of the graph (click on the bots in the legend). #2 PurpleWave shows a different pattern: Without prior knowledge, it starts lower, learns fast at first, then more slowly, and seems to level off before the end of the tournament (though it might be only a temporary plateau).

McRave

McRave upset ZZZKBot 64-46, so watching the games versus McRave lets us see the learning algorithm in action. ZZZKBot does not have a prior strategy versus McRave, possibly because none of its 4 strategies can win reliably against the early cannons. (There are ways to bust the cannons, so it is a limitation of ZZZKBot.)

There are 10 maps in the tournament, and they are played one map per round, so it takes 10 round robins to play all maps once. ZZZKBot played its 4 pool against McRave for the first 10 rounds to see how it did on each map. The answer: It won some and lost some, depending on whether it scouted protoss quickly and whether McRave pulled probes in time to shield the cannons when necessary.

On maps where the 4 pool won, ZZZKBot repeated it when the map came up again. On maps where the 4 pool lost, ZZZKBot switched to its next strategy, the overpool speedlings. The speedlings did not usually do well, because McRave had 3 cannons up in time. ZZZKBot tried to follow up with mutalisks, but McRave scouted that coming and was more than prepared.

I watched the sequence of games on Benzene. The speedlings mostly lost, except for occasional games where zerg managed to kill the scout probe and leave McRave in the dark and unable to react in time. But ZZZKBot kept trying the strategy, only occasionally switching back to 4 pool. I didn’t watch every game, but ZZZKBot’s other 2 strategies didn’t seem to come into play at all.

Iron

Versus Iron, ZZZKBot mostly stuck with its 1 hatch hydralisk strategy, an unusual opening. One odd point is that ZZZKBot researched hydralisk range before speed, which is rare and usually seen only when attacking a protoss wall. As we see in the per-map crosstables, ZZZKBot scored poorly against Iron on 2 player maps and tended to win on 3 and 4 player maps. The difference was that on 2 player maps, Iron was expecting to be rushed and was more willing to build a bunker, which held the hydras.

ZZZKBot sometimes fell back to its sunken defense into mutalisks, but that was less effective. Iron could stop the mutalisks, and its vultures were able to find gaps in the sunken defense.

Iron is the only opponent configured for the hydralisk build order. ZZZKBot doesn’t seem to use it at all, otherwise. I think the build must have been specially developed for Iron.

XIMP

ZZZKBot chose to bust XIMP’s cannons with its speedling build. The zerglings commonly arrived when XIMP had 2 cannons, versus McRave’s 3, and XIMP is not as skilled with probes. It didn’t help that XIMP likes to leave its gateway units in its main “to defend against drops.”

XIMP won only 4 games out of 110. In 3 of its wins, it got its first zealot into the fight in time to save the day. In the fourth, XIMP expanded to another base rather than its natural. ZZZKBot brilliantly scouted the undefended nexus and chose to attack it first, which allowed XIMP’s third cannon time to finish.

CherryPi

CherryPi is an interesting case because ZZZKBot’s “prior knowledge” was guesswork, “Guessing it is like tscmooz.” CherryPi consistently played a sunken defense into mutalisk build against ZZZKBot, with 2 to 4 sunkens. Where did this trend come from? In any case, Tscmoo doesn’t play any such build as far as I’ve seen.

ZZZKBot’s learning kicked in. It tried the 4 pool; no good. It tried the speedlings; no good. It tried its own sunken defense into mutalisks, building only 1 sunken, and that worked perfectly. The sunken was often poorly placed so ZZZKBot tended to lose a few drones, but its mutalisks were out faster.

the Steamhammer forks

ZZZKBot chose its sunken defense into mutalisks versus Steamhammer (with 5 sunkens), Microwave (3 sunkens), Arrakhammer (9 sunkens, because Arrakhammer likes to bust with zerglings), and KillAll (1 sunken), with great success. The sunken count is hand-configured for each opponent. I found it frustrating, because Steamhammer knows in principle how to respond: It makes drones instead of zerglings and goes air. Unfortunately, Steamhammer’s strategy adjustments are sloppy, and it almost always got its own mutalisks too late. It did things like start another hatchery when the spire finished, and then a queen’s nest, and then—well, then it was irretrievable. I knew all along that opponent modeling is crucial for tournaments.

conclusion

Watching games, I was struck that ZZZKBot’s builds are not tight. It doesn’t expand, and in the middle game (when it gets that far) it ends up with more drones than its hatcheries can support. It suffers mineral excess that it can’t spend, and gas shortage because it has only 1 geyser.

Its micro is not tight either. ZZZKBot doesn’t have a combat simulator (well, it would probably be in a subroutine, and as the ancient Greeks declared, only straight lines and circles will do). If the 4 pool leaves cannons alive, then the next 2 followup zerglings will die to the cannons, accomplishing nothing. Then the next 2, etc., until protoss moves out and wins. Followup is minimal; the bot is about winning right away.

ZZZKBot has a lot of clever little skills, but it is missing some big ones. The weaknesses mean that stronger cheese bots are possible. I don’t think the cheese era is over yet.

Next: CherryPi.

I read the source of the AIIDE 2017 version of ZZZKBot. It comes with a github repository but at the moment the repo is 2 years out of date.

As a reminder, #1 ZZZKBot was upset by McRave and Tyr which open with cannons to stop most cheeses, and by LetaBot which has extensive work to recognize and respond to different cheeses. ZZZKBot had a plus score against every other opponent, including #2 PurpleWave and #3 Iron. Interestingly, ZZZKBot scored only a little better than even against #26 Slin.

coding style

ZZZKBot is written in a straight-line style, as if subroutines had not been invented. Most of the code is in the onFrame() method. It’s kind of hard to read.

At one point, ZZZKBot checks the name that it is running under using Broodwar->self()->getName(), and if it is not exactly “ZZZKBot” it sets transitionOutOf4PoolFrameCountThresh to 0. The effect depends on the strategy, but it looks as though it usually switches out of its cheese build into late-game play. I have to suppose it is an obfuscation. [Update: Comments suggest that the reason is to avoid lazy copying.] Anyway, be aware that the bot may behave differently depending on the name it is playing under.

strategies and learning

This version of ZZZKBot has 4 strategies.

• its old standard 4 pool
• 9 pool speedlings
• a 1-hatchery hydralisk rush with 11 drones
• sunken defense into mutalisks

Besides the basic strategy choice, ZZZKBot keeps a few variables that control details of the strategy’s execution.

        bool isSpeedlingPushDeferred;
        bool isEnemyWorkerRusher;
        bool isNumSunkensDecidedAfterScoutEnemyRace;
        int numSunkens;

ZZZKBot also has hardcoded default strategy data for specific opponents, so that against a known bot it can choose a presumed good strategy on the first game. It has values for 17 of the 28 bots in AIIDE 2017, including itself (so 16 of the 27 possible opponents). Considering how thorough the list is, I suspect that Chris Coxe tested against every available bot and manually chose strategies against the ones that ZZZKBot did not defeat on the first attempt. (Against itself it sets all strategy variables to false—I didn’t check what that does.)

• UAlbertaBot
• ZZZKBot itself
• XIMP
• GarmBot
• Overkill
• CherryPi (comment “Guessing it is like tscmooz...”)
• Ziabot
• Skynet
• MegaBot (considered the same as Skynet)
• Steamhammer
• Arrakhammer
• Microwave
• KillAll
• ForceBot
• Juno
• Iron
• HannesBredberg

After each game, ZZZKBot may update its strategy data and save the result to a file. The algorithm is long and includes random choices, and I didn’t try to puzzle it out. There are special provisions for playing against random; it checks when the opponent’s race was discovered to see whether the stored info will be useful for the current game. It also, oddly, saves information about the processor it is running on.

scouting a zerg opponent

When your opponent is zerg, a couple tricks can make scouting easier. One is that if you spot the enemy’s first overlord, you may be able to infer the location of the enemy base. I added this knowledge to Steamhammer for the next version, and it frequently helps. Another is that you do not have to scout the enemy hatchery itself. The base is found when you see the creep, and a scouting unit can turn aside as soon as it sees bare ground where creep would be if the base were there. I didn’t add this knowledge to Steamhammer (yet), because it’s tricky. For one thing, I’m not sure of the exact rules for creep spreading; the way unbuildable ground blocks creep is not obvious (commonly creep extends beyond the geyser on both sides but not around it, leaving a notch). For another, a map could have neutral zerg buildings that spread static creep, and BWAPI won’t tell you where the static creep is; an example is Colosseum (though on that map the static creep doesn’t affect scouting). The complications were enough to keep the idea off the top of my priority list.

ZZZKBot implements both of these tricks. How does ZZZKBot know where the creep spreads to for each base location? Simple: It hardcodes the data for each starting position of all 10 maps!

ZZZKBot also tries to guess the location of the enemy base from other enemy units it sees. As a cheeser, ZZZKBot scouts early, so it has a good chance of guessing right.

a comment from the code

            // TODO: support making more than one building of a particular type.

Waste no time on the unimportant!

conclusion

ZZZKBot is very specific and narrowly coded. Its sole purpose is to defeat other bots, especially opponent bots that it knows playing on maps that it knows. Everything possible is hardcoded, above all prior knowledge about opponents and maps. ZZZKBot teaches strict lessons to its opponents, but there are not many lessons in the bot itself. It’s about the cheesiest, most exploitative bot possible.

If ZZZKBot had stuck with its classic 4 pool, I’m convinced that it would have finished in 4th place or worse rather than first. Too many of its opponents in this tournament were ready for the 4 pool (I know Steamhammer was). Next year, I expect that top bots will be prepared for all 4 of ZZZKBot’s strategies this year (it won’t be difficult), and ZZZKBot will have to go to even greater lengths if it is to finish strongly. I see it as a sign of progress: Even the most exploitative bot is forced to play multiple strategies and to rely on machine learning—it is forced to use smart methods to play better. The space for cheap exploits is gradually narrowing.

Next: Looking at how ZZZKBot played against different opponents.

The SCCAIT round of 8 was played last week as 4 best-of-3 matches. The loser of each match is out and the winner moves on. Today I’ll go over the first 2 matches of the 4, from the first video.

LetaBot vs WuliBot

This match was easy to call and I don’t have much to say about it. The forecast: LetaBot will wall in and zealot-heavy Wuli will be unable to cope.

As it turned out the first game was even less interesting—Wuli froze up and died with one pylon to its name. The second game went to script. Wuli’s zealot rush is strong but risks being hard countered, and that’s what happened.

ZZZKBot vs Iron

ZZZKBot’s 4 pool also risks a hard counter. And Iron knows a counter, but it is not as hard as it could be. When Iron sees the danger it stops any tech beyond barracks (often canceling gas) and pulls SCVs to block its entrance. As soon it can it builds a bunker behind the SCVs, and if it succeeds in getting marines in, it is usually safe.

As always, Iron knows some excellent micro tricks. When it has enough SCVs blocking, it will sometimes mineral walk damaged front SCVs back through its own blockade to the mineral line. That means that the SCV right-clicks minerals so that it passes through any intervening units, allowing Iron to rescue damaged units in front without opening its blockade.

Game 1 was on the level-ground 2-player map Heartbreak Ridge, ideal for the 4 pool. Iron saw it coming in time, got the bunker up, and got a marine into it, pulling nearly all SCVs to block. It was close, but Iron held and had more income. When it was safe, Iron switched back into its usual aggressive strategy and won easily.

Game 2 was on Icarus, a 4-player map with ramps which is not as good for the 4 pool. But the bases turned out to be close together, which is favorable for the rush. Both bots scouted each other in time. This time ZZZKBot broke the ramp and won—Iron got a bunker up but could not get a marine into it. ZZZKBot showed impressively smart targeting with its lings, switching smoothly between hitting the empty bunker and chasing away any terran units that came close. At one point zerg split its lings into 2 groups, one to chase the last marine and one to disrupt mining.

The deciding game 3 was on Empire of the Sun, a 4-player map but without ramps and with a wider entrance to defend than Heartbreak Ridge. ZZZKBot sent its overlord scout the right direction and did not need to make an extra drone to scout, which strengthens the rush slightly because 2 more zerglings fit under the supply limit. 3 drones mining are enough to keep up constant zergling production; more drones produce extra resources that are only useful if the rush fails, when the rusher is generally lost anyway.

But in any case, this time Iron narrowly held and won. Apparently the result depends more on random factors than on favorable or unfavorable conditions!

With good worker micro, it is more efficient to hold the rush in the mineral line, rather than at the entrance. Tscmoo knows how to do it. In pro games, attacking zerglings have to be cautious around workers and only pick off stragglers, because the workers fight so effectively.

Tomorrow: The second half of the round of 8. Maybe I’ll catch up with real time by the time the finals broadcast on Saturday!

One of the stories of SSCAIT 2016 is that LetaBot (playing under its author’s name, Martin Rooijackers) scored easy wins over certain tough opponents with SCV rushes. Classic LetaBot used to play bunker rushes and recent LetaBot versions still sometimes performed SCV-marine rushes, so the rush opening should not have been a total surprise. But it worked perfectly against Krasi0 and even against micro-virtuoso Iron. The losers tried hard to keep their own SCVs safe, lost too much mining time, and could not keep up.

I approve. That’s the way to do it. If you find a weakness, exploit it ruthlessly and don’t worry about whether people look down on you as a cheeser. In fact, that is ZZZKBot’s entire modus operandi, and ZZZKBot is doing well with it, departing from its traditional 4 pool often and effectively with opponent-specific builds. Ruthless wins are your opponent’s way of telling you what you need to fix, and that’s good for everybody.

LetaBot’s rush looked similar to the rush we used to see from Stone, before Stone evolved into Iron. LetaBot pulled back its damaged SCVs into loving mutual-repair couples before returning them to the fray. I rather wish Stone were still playing alongside Iron. Stone would constantly remind other bot authors of the importance of good worker micro—worker micro is difficult (and I sure haven’t gotten around to it in Steamhammer), but it’s key.

What is it with ZZZKBot? In most SSCAIT games it has stayed with its traditional 4 pool, but it has also been playing a variety of other builds against some opponents. Against Steamhammer it played mass sunkens into spire, a build that seemed tailored to beat Steamhammer’s anti-rush opening, which Steamhammer had never publicly played before the tournament, but which could have been discovered by local testing.

I set up ZZZKBot locally and tried it out. The github version is out of date, so I took the SSCAIT version. Does it win against Steamhammer’s standard ZvZ 12 pool? No, it loses, but what is this build it’s playing? I hadn’t seen that on the stream. It opened 9 pool, more drones, second hatchery in main, late gas, lair, queen’s nest—it was walking slowly toward greater spire! Steamhammer beat it before it got that far, though. ZZZKBot wrote no data file; it was not learning.

What about Steamhammer’s anti-rush build, as it is playing in SSCAIT? ZZZKBot played the same slow march toward greater spire. The anti-rush build is weak against 9 pool, but the greater spire build is worse and Steamhammer still got mutas out first and won.

Does it depend on the map? I ran 3 games on the same map as in the SSCAIT game, Roadrunner, versus the anti-rush build, and ZZZKBot aimed for greater spire and lost each game. In one game it managed to get out a couple guardians before dying. ZZZKBot never repeated its anti-anti-rush build.

ZZZKBot also never wrote a data file. I suspect that it keeps some extra static data that the SSCAIT server doesn’t serve up, an optional configuration file with counter-builds for specific matchups. That would explain its varied and effective build orders.