Starcraft AI blog

As Dan Gant let me know, CoG 2022 results are out today, complete with the detailed results file. The participants are the same as last year, except that MetaBot was dropped for unreliability that affecting the running of the tournament. The carryovers from last year are #6 XiaoYi, #7 CUNYbot, and #8 BetaStar. The others are updated for this year.

My version of the crosstable.

	overall	Bana	Purp	Star	McRa	Micr	XIAO	CUNY	Beta
#1 BananaBrain	85.40%		79%	60%	69%	90%	100%	100%	100%
#2 PurpleWave	75.24%	21%		84%	55%	74%	97%	97%	100%
#3 Stardust	73.02%	40%	16%		69%	90%	98%	100%	98%
#4 McRave	68.60%	31%	45%	31%		93%	81%	100%	100%
#5 Microwave	46.54%	10%	26%	10%	7%		74%	98%	100%
#6 XIAOYI	35.40%	0%	3%	2%	19%	26%		98%	100%
#7 CUNYBot	15.49%	0%	3%	0%	0%	2%	2%		100%
#8 BetaStar	0.32%	0%	0%	2%	0%	0%	0%	0%

There are surprises throughout, from top to bottom.

Stardust’s reign is over for the moment. Last year, Stardust scored over 90% in CoG and over 95% in AIIDE, crushing the competition. This time, #1 BananaBrain dominated with 85%, and #2 PurpleWave edged out #3 Stardust. The official results show that Stardust had 67 crashes and 7 frame timeouts in 3150 games. If Stardust had the same number of crashes (zero) and frame timeouts (1) as the two bots above it, it would have finished second by a razor-thin margin.

There is not a single upset, where a lower-ranked bot defeated a higher-ranked bot. The crosstable is very orderly. The lowest winning rate of a higher-ranked bot is 55% for #2 PurpleWave over #4 McRave.

Something went wrong with BetaStar. It is a strong bot and finished well ahead of CUNYbot last year. Head to head versus CUNYBot, it scored 40 wins out of 50 games. This year it scored 10 wins total against all opposition, and all wins were against Stardust and likely due to crashes. What went wrong? Did the new and improved map pool break it? Was there a rule change that it could not cope with?

race results

I made two versions of each table. The left one includes all results, the right one excludes BetaStar.

race	score
terran	35%
protoss	58%
zerg	44%

race	score
terran	25%
protoss	74%
zerg	34%

It’s not very informative, but I like to include it anyway. There was only one terran; we need more. Protoss dominated, as usual in recent years, even when including BetaStar’s debacle.

bot	race	overall	vT	vP	vZ
BananaBrain	protoss	85.40%	100%	80%	86%
PurpleWave	protoss	75.24%	97%	68%	75%
Stardust	protoss	73.02%	98%	51%	86%
McRave	zerg	68.60%	81%	52%	96%
Microwave	zerg	46.54%	74%	37%	53%
XIAOYI	terran	35.40%	-	26%	48%
CUNYBot	zerg	15.49%	2%	26%	1%
BetaStar	protoss	0.32%	0%	1%	0%

bot	race	overall	vT	vP	vZ
BananaBrain	protoss	82.96%	100%	70%	86%
PurpleWave	protoss	71.11%	97%	52%	75%
Stardust	protoss	68.89%	98%	28%	86%
McRave	zerg	63.37%	81%	36%	96%
Microwave	zerg	37.63%	74%	16%	53%
XIAOYI	terran	24.63%	-	1%	48%
CUNYBot	zerg	1.41%	2%	1%	1%

Again, not very informative with so few participants. Excluding BetaStar clarifies that CUNYbot was outclassed. XiaoYi was also outclassed by the remaining protoss, and was only able to fight against the zergs.

the surprising poor results

Stardust’s crash rate surprises me. It does not have a crashing problem on BASIL. There was something in the tournament environment that it was not ready for. I can’t guess whether that’s more due to Stardust, or more due to the tournament.

BetaStar essentially scored zero and added no information to the tournament results. To me it suggests that the tournament environment changed somehow (we know that at least the map pool changed), and the organizers did not test the carryover bots to make sure they still worked.

AIIDE 2022 registration is open.

Steamhammer will be competing.

A number of stronger terran bots make valkyries against Steamhammer. Not one of the terrans knows how to use them well. Steamhammer trades better against valkyries in nearly every game where they show up.

Valkyries are specialist units. Valkyries in small groups, when kept safe, can devastate mutalisks (or wraiths). A valkyrie wandering around on its own can’t devastate anything except one pair of scourge. Zerg will happily trade a pair of scourge costing 25/75 for a 250/125 valk. The valkyrie can run away from scourge, but only if it does not shoot. When the valk fires, it stops dead in the air!

In TvZ, defeating mutalisks is their only important role. In most situations they are not good for hunting overlords, because they are too vulnerable to scourge. To fight guardians, wraiths are better, because guardians have armor. Devourers have more armor, and goliaths and vessels are better.

Advice to terran bots: If you are facing mutalisks, consider keeping up to 3 valkyries in the air against a dozen mutas, maybe slightly more if the mutas mass up beyond that or pull ahead in armor upgrades. Valkyries are expensive; making too many is wasteful. Keep them together so they fire at nearly the same time. It makes them far more effective—seriously, mutas will melt away like light snow; zerg will be forced to spread them so that they cannot focus fire. Keep them near marines, goliaths, or turrets for protection from scourge. Make sure your anti-air units prioritize shooting down scourge; scourge are low in HP and expensive in gas, so they’re good targets whether you have air units nearby or not. Even with all that, valkyries may not pay for themselves unless you also implement valkyrie patrol micro as described in Liquipedia. It’s critical, and I’ve never seen a bot use it.

Valkyries can be worth it, but only for bots that know how to use them.

The bot Pinfel has been inactive for a long time. I last wrote it up in 2017. Pinfel played an entertaining zealot-probe all-in strategy, bringing every probe. Now a new bot named Pinfel 2 (BASIL link) has appeared. Its SSCAIT description says it uses STARTcraft.

Pinfel 2 makes 5 or 6 gates, accumulates a lot of zealots, and eventually attacks—if it is not attacked first. It’s rudimentary, but it’s a start. The zealots are many, but until Pinfel 2 starts its very late attack, the zealots only wait by their gates and only react if an enemy comes close.

Pinfel 2 is brand new. I approve of uploading it in a rudimentary state and watching how it does before getting down to the real work. Let’s see what becomes of it!

New protoss bot Pylon Puller is running on BASIL only, not SSCAIT. It was uploaded yesterday, and already has one update.

Pylon Puller plays at least three strategies. Its favorite is three gate zealot. It also knows builds for cannon defense followed by dark templar, and cannon defense followed by dragoon-reaver.

Pylon Puller tries to win in the opening. Once its unit mix is set, it does not seem to change. It is able to expand, but rarely does. It can do long distance mining, which I have seen more often than an expansion. Its micro is not strong. It sometimes leaves units on hold position while they’re under attack.

Pylon Puller has not always played the same build against the same opponent. I have only seen it play its dragoon-reaver build against Yuanheng Zhu aka Juno, a cannon bot: The reavers are theoretically effective versus cannons. Without more games, it’s impossible to tell whether it’s a strategy adaptation or a hardcoded build for this opponent. Against IceBot, it played zealots the first game, and Ice defended effortlessly and won. The second game, Pylon Puller went DT and won. Since there was an update, it’s impossible to know whether the switch was due to the bot learning, or the author coding. Similarly for two losses in a row against MadMixT, where Pylon Puller played different builds. But I doubt that a bot at Pylon Puller’s skill level has both strategy adaptation and learning. Learning does seem moderately likely.

Running on BASIL only, frequent updates, focus on winning early with little attention to the middle game, and a restricted but hand-adapted set of opening builds all strike me as characteristic of a Newbie Zerg production—though unlike many past Newbie Zerg bots, it does not seem to be based on Steamhammer. I expect that Pylon Puller will be unable to threaten top bots, but may become a danger to some opponents one tier lower if Pylon Puller’s hand-crafted builds target their weaknesses. Eh, maybe two tiers lower. So far, it does not look impressive.

But whatever. Every new bot is good! Weakness exposed by manual adaptation are still weaknesses that need fixing. Every bot that brings something new will teach us something new.

In this game, Steamhammer hit on a surprising plan that exploited a weakness in Stardust. Even so, Stardust had tricks up its sleeve.

It is strategically correct for the stronger bot to play conservatively, taking no risks. Steamhammer happened across an opening which Stardust answers too conservatively: It proxied in Stardust’s natural. It started the hatchery almost immediately after scouting the location of the protoss base. Correct play is for Stardust to smash the proxy before it can be defended. Stardust scouted the proxy and immediately assumed without evidence that it was contained, so it played conservatively and did not try to break out. The yellow dot in Steamhammer’s natural is the scouting probe: Stardust had all the information it needed to conclude that the proxy was indefensible, but assumed that it was too risky to attack.

The production queues tell each bot’s strategy. Stardust made a robo to escape the containment by air. Also notice the Citadel of Adun. The two cannons next to the nexus are unnecessary; zerg does not have a lair, and protoss could have scouted it but did not bother to. Steamhammer’s opening build assumes blindly that the opponent will not attack—it is a build specialized for defeating one-base protoss players who build up and attack late. Steamhammer is making drones now in preparation for sunkens at the proxy and then a large army.

Both sides have attack +1 already. The citadel was for zealot speed. A shuttle can carry four zealots but only two dragoons. Stardust cleverly elevatored zealots from its main to the north of its base where they would not be seen, and ran them across the map. Steamhammer was ready anyway. Zerglings are about to hatch, and once they joined the hydras the zealots were afraid to engage and ran away. The zealots tried to retreat to the protoss main through the proxy, where sunkens and the zerg army slaughtered them.

After that zerg was in charge. Steamhammer immediately invaded the protoss base, while Stardust airlifted a probe out for a distant hidden base. There was more fighting, but Steamhammer is reliable about winning when this far ahead.

On Jade with its low main, it’s important to defend above the ramp if you can. Otherwise you don’t see what’s coming and you have to fight uphill. But as far as I can see, it didn’t affect this game. Stardust scouted the proxy and did not try to defend its ramp, except for one cannon.

Update: Steamhammer played a second game against Stardust, on the map Python. It went very much the same way. I was right that Jade’s low-ground main did not matter.

Second update, 10 July: Steamhammer has since won a bunch of games that went the same way. Here’s the first game that went differently, on Andromeda. It shows both the strength and the fragility of a strong proxy position.

You have to take special measures to copy a simple object because... part of it is constant. How... deeply... intuitive? Bjarne Stroustrup, you have created a monstrosity!

CA.cpp:114:5: error: object of type 'BitVector' cannot be assigned because its copy assignment operator is implicitly deleted
  b = unpack();
    ^
./BitVector.h:19:15: note: copy assignment operator of 'BitVector' is implicitly deleted because field 'numWords' is of const-qualified type 'const sizet'
      (aka 'const unsigned long')
  const sizet numWords;             // set on initialization
              ^
1 error generated.

I call it duke typing: If it looks like a duke and talks like a duke—fuggedaboutit, there’s no nobility in this code.

Kudos to clang for clearly explaining the poor design decisions of C++.

Steamhammer-Krasi0P on Python (BASIL replay) was hilarious. Don’t miss the third zerg base in the upper left.

It’s easy to guess how the game went.

There was a somewhat similar game in March last year. In that game, Krasi0P put up only one containing cannon. In this game the protoss containment was effective for a long time.

CoG this year is a small, elite tournament, virtually the same as last year. The entrants, with their win rates in last year’s CoG:

bot	wins	author
Stardust	90.25%	Bruce Nielsen
BananaBrain	74.69%	Johan de Jong
McRave	68.17%	Christian McCrave
Microwave	54.14%	Micky Holdorf
PurpleWave	52.14%	Dan Gant

It’s a fair guess at the likely finishing order. Today’s BASIL ranks and 2021 CoG ranks are the same with one exception, McRave and Microwave are reversed. But PurpleWave has been playing for a long time with one bug that prevents it from doing any upgrades, including basics like dragoon range and zealot speed, and another bug that causes it to construct duplicate buildings. I think it’s a safe guess that the bugs will be fixed for the tournament, and PurpleWave may finish higher.

No terrans. That’s unfortunate.

The carryovers, also with their win rates from last year:

bot	wins	author
XiaoYi	40.10%	Benchang Zheng
BetaStar	39.29%	Ruo-Ze Luo
MetaBot	23.08%	Anderson Tavares
CUNYBot	7.5%	Bryan Weber

There’s a note that MetaBot may be dropped due to a tournament stability issue. The decision has not been announced yet.

All updated entrants are likely to outscore all carryovers. XiaoYi is the sole terran, and should not be much of a challenge for current bots. On BASIL, terran krasi0 has just in the last two days retaken its top spot. It would have been good to at least have Hao Pan or Dragon in the tournament.

But there is a silver lining. This is exactly the same participants as last year, assuming that MetaBot is kept. The biggest difference is that CUNYBot is carried over from last year rather than updated. It will be fun to compare relative progress.

The entry deadline for CoG 2022 is this Sunday.

CoG has an exciting new map pool. In past years they selected at tournament time a small number of maps from a large pool that included some clunkers. This year they gave that up and chose 9 maps ahead of time (see the tournament rules). The exciting part is that they include BWAPI 1.16.1 versions of the newer maps Eclipse, Neo Sylphid, and Polypoid. Neo Sylphid is available on SCHNAIL, but I think the interesting and popular maps Eclipse and Polypoid are new to the major competitions.

They also included the map Outsider, which is difficult for bots. Most bots should be able to play games, but skills to cope with the blocked-off side bases will be valuable.

Finally, at least a few maps that are modern and familiar to current human players! CoG is always interesting for its map variety, and this year it is better than ever.

Update: I see that the entry deadline has been extended to 19 June. It might be a sign that they’re not getting many entrants. I hope it’s only a typical delay; delays can come from anywhere.

Working at glacial speed, I pieced together a machine learning algorithm for use in Steamhammer. It passed basic tests today. When I’m sure that it works for real cases that I care about, I’ll release the code. It’s short.

I used a little-known idea and combined it with a standard algorithm employed in a slightly old-fashioned way. It took me a long time to put the thoughts together, but the result is that it fits in 374 lines of code, including blanks and comments, with no dependencies. (It will likely be longer when I release it.) It should be as accurate as a conventional (non-“deep”) neural network, but it learns faster and runs faster. It’s there in the literature, I just had to put the pieces together.

Hooking it into Steamhammer will take more code that the method itself. I’ve claimed before that the first use will be for an evaluation function, “Am I winning?” but that’s not strictly true. The first use will be to train the evaluation function! Once it has useful accuracy it will be put to work in opening selection, and it can be bootstrapped for all kinds of midgame strategy decisions.

Don’t hold your breath, I’m still working at glacial speed, and even global warming won’t change that right away.

MicRobot describes itself as “Student project based on UAlbertaBot with experimental micromanagement changes. Last updated 05/2021.” It was uploaded earlier in a version that did not work; now it has been fixed. As a UAlbertaBot fork, it plays random. MicRobot’s configuration file names its author as Robert Karpiel.

On a casual watch of its games, I didn’t see differences from UAlbertaBot. So I looked more closely, comparing Dave Churchill terran v Matej Istenik to MicRobot terran v Matej Istenik, very similar games. I still can’t tell the difference. Both bots seem to kite their marines the same way. Is MicRobot maybe a little better at keeping its front line organized? I’m not sure.

I downloaded the binaries. MicRobot adds a new section to the configuration file, with new options:

    "MicRobot" :
    {
        "ResetOKHPEveryNFrames"     : 0,
        "LastStandValue"            : 400,
        "StopAfterNGames"           : 0,

        "Ranged_AlwaysKite"         : false,
        "Ranged_CourageValue"       : 400,
        "Ranged_DeprioritizeWorkers" : false,
        "Ranged_KiteNOKPDPS"        : false,
        "Ranged_KiteNOK"            : false,
        "Ranged_KitePDPS"           : false,
        "Ranged_UseSquadDistance"   : false
    },

ResetOKHPEveryNFrames (reset OK HP ...) sounds like a feature to retreat a single damaged unit before it is destroyed. Just a guess. It looks as though most of the options affect ranged units only—and are turned off. The disabled options may be for experiments that didn’t work out. If the actual micro changes affect mostly ranged units, then that matters only for terran. UAlbertaBot and therefore MicRobot play zealot rush as protoss and zergling rush as zerg, and usually win or lose before making any ranged units. In fact, if zerg makes hydras then it will almost always lose because of BOSS bugs.

If the aim is to compare MicRobot with the UAlbertaBot version (named “Dave Churchill”) already running on SSCAIT and BASIL, then the experiment may not give valid results. SSCAIT runs a 2015 version of UAlbertaBot, and MicRobot is based on a newer version with code differences. Also, UAlbertaBot uses enemy-specific strategies (coded into the configuration file) for select old opponents that it otherwise has more trouble against. MicRobot does not use the feature.

Possibly it is a different experiment. Maybe the author will re-upload MicRobot with different options enabled once its BASIL rating stabilizes. In that case, my intuition is that improved micro alone, without changes to reduce other weaknesses, will probably produce a rating difference that is smaller than the noise. UAlbertaBot’s micro is not a bottleneck weakness; it usually loses for other reasons. Still, I could be wrong.

Maybe Robert Karpiel will enlighten us?

Infested Artosis describes itself as a “Zerg bot made from scratch, under active development.” It has a github repo. The README says that it was originally cloned from JavaBWAPI.

Infested Artosis is at a very early stage of development. It plays a fixed build order with zerglings and then hydralisks on one or two bases. It barely has combat skills, and does not seem to recognize enemy buildings as possible targets. Still, it can win occasional games against weak opponents.

One funny game is Infested Artosis vs Marine Hell on Python. Infested Artosis made zerglings and ran past Marine Hell’s ramp bunker to erase the SCV line. Then the zerglings ran out of the base again past the loaded bunker! It’s not a runby skill—the units simply fail to recognize that static defense can shoot at them. With nothing visible in the base except buildings, apparently the zerglings thought there was nothing to fight against (or maybe it was a bug).

It’s a start. I approve of live testing early on, before the bot can even play a decent game—you learn faster that way. Have fun with further development!

I re-enabled Randomhammer on SSCAIT. There are no updates, I only turned it back on.

Terminus is BananaBrain playing terran, and Brainiac is BananaBrain playing random. (Crona is BananaBrain playing zerg and has been around for a while. BananaBrain is of course BananaBrain playing protoss.)

Terran is the new race. In the games I watched, I saw it open with in-base BBS and proxy BBS (which it plays rigidly with no tech), Gundam rush, a different two-factory attack, bunker expand, factory expand, and wall its natural and defend with tanks. It’s a wide variety, and I doubt I saw everything. It knows something about tank lines, though its TvT skills did not seem polished. It knows how to scout with a floating building. It can play with barracks units or factory units, or both at once. It can make wraiths and vessels. It knows how to lay a dense carpet of mines in front of its base—sometimes too close in front. It’s not effective mine placement against a skilled opponent. It does not seem to go for the hyperactive vulture style popularized by Iron. Not as impressive a terran as Hao Pan or Krasi0, but strong skills overall.

This game versus Pathos was entertaining. Terminus tried to expand and failed.

It looks like another tough opponent. Way to go! As I write, Brainiac is ranked lower than its component races, but that is because it is too new. Its rating has not stabilized.

Steamhammer will be back. Give me time.

New bot Zerg Hell (BASIL) by NiteKat describes itself as a “C++ client bot written in 8 hours to see what could be done with limited development time. Uses no additional libraries.” It has a repository. The name refers to Marine Hell, a bot developed in 5 hours and left up as an example of what it is possible to do in a short time.

Zerg Hell plays a fixed plan: overpool, 4 sunkens placed wherever to turtle up on one base, collect hydras while researching the speed and range upgrades and adding drones and hatcheries, attack at a fixed time. It gets a lair solely to research overlord speed, and moves its overlords to counter cloaked units. It doesn’t have many other skills. It never expands.

When its plan comes off, the mass of hydras can be dangerous. The hydras are reasonably good at staying together and deciding what area to attack. If the sunkens are misplaced, which is common, it may lose all its drones helplessly. On SSCAIT, its build usually gets stuck at one point or another and it stops producing units. That doesn’t happen on BASIL, but against Skynet by Andrew Smith it had a production freeze making it very late to replace lost overlords.

It beats Marine Hell head-to-head. Does it take 11 hours of coding to beat Zerg Hell?