Starcraft AI blog

I took a preliminary look at how far creep spreads from a hatchery. Here’s a picture from my first experiment.

The numbers on each tile are the distance from the hatchery, determined by hatchery->getDistance() and scaled to tiles instead of pixels. Tiles for which BWAPI::Broodwar->hasCreep() is true get orange numbers. Boxed tiles are unwalkable according to Steamhammer’s walkability map: A 32x32 tile is walkable if all its constituent 8x8 walk tiles are walkable, taking into account terrain and fixed neutral units only.

Creep spreads in an oval pattern, similarly to power from a protoss pylon. Of course creep spreads bit by bit, unlike the instantaneous power from a pylon, but at the start of the game the initial hatchery has its full spread of creep, so in that case we can ignore the spreading process.

After trying this on a variety of maps, including some irregular Blizzard maps, I found that the initial hatchery’s creep, though its outline seems a little odd, takes on a tile-for-tile identical outline on all maps. I’ll test further, but it looks as though I can predict the creep for the initial hatchery without much trouble: Hardcode one pattern of creep spread, match it to the hatchery location, clip it at the edges of the map, and trim away the minerals, geyser, and any terrain features that don’t allow creep. So far I haven’t tried a map where this method would give a wrong answer.

If it continues to look good, then an upcoming 1.4.x version of Steamhammer will be a little faster to find the bases of zerg opponents. Early game scouting units will definitively identify the occupied base when they see creep, and definitively identify an empty base when they don’t see creep that should be there.

Later versions will also have the ability to infer the existence of a zerg base by seeing creep later in the game. It’s strange to see bots pass in sight of creep and not realize what it means—“must keep looking, where is that last enemy base?” That feature will be a little more complicated. We don’t want to infer the existence of a base which was just destroyed, or get confused because outlying creep colonies of one base extend the creep as far as another base.

Next: Other near-term plans for Steamhammer.

Mutalisk bounce attacks are complicated.

In lore, the mutalisk attacks with a glaive wurm. A glaive is a sword. BWAPI spells it “glave”, which is a less common alternate spelling. Anyway, the attack does 9 normal damage directly. If an enemy unit is close enough, it bounces and hits the enemy, doing 1/3 as much damage, 3. If a second enemy is close enough, it bounces a second time and does 1/3 as much damage again, 1.

upgrades

If you upgrade mutalisk attack +1, the bounces are upgraded too. The first attack is 9+1=10; the second attack is 10/3 = 3.33 (rounding to 2 digits); the second attack is 10/9 = 1.11. The total gain is 1.44; for +2 upgrade, 2.88; for +3 upgrade, 4.33.

Brood War does track fractional damage behind the scenes, though you can’t see it in the user interface. There is also a hard floor; every attack, even one that is smaller than the target’s armor, does at least 0.5 hp of damage. At least that’s my understanding of how it works.

If you’re under attack by mutalisks, +1 armor applies to all 3 bounces, so it prevents up to 3 points of damage per shot.

For ZvZ air battles, zerg players disagree on whether it is better to get +1 armor or +1 attack for mutalisks. +1 armor is more popular; it costs more (150/150 instead of 100/100) but prevents more damage (3) than +1 attack causes (1.44), and it also protects scourge and overlords. Nevertheless, some strong players prefer to get +1 attack. The best choice can depend on the game situation—for one thing, upgrades don’t use up a larva; for another, scourge have 25 hit points and are vulnerable to bounces.

you don’t have to attack an enemy

It doesn’t matter what the mutalisk attacks, the glaive wurm will bounce to an enemy if one is close enough. You can attack a harmless enemy building, or a neutral building or neutral critter, or your own unit or building. Mutalisks have a short range of only 3 tiles. By bouncing from a safe object, they can do some damage to enemies while keeping at longer range. In the best case, the bounces can hit marines while the mutalisks are out of range of marine fire.

Human players do this when a good opportunity presents itself. I’ve never seen a bot do it.

which way does it bounce?

This TeamLiquid translation from Korean gives 2 rules for how glaive wurms bounce. You can have some foreknowledge of which enemy each bounce will hit.

Rule 1: It bounces to the left if possible; otherwise, it bounces down if possible. If it can’t bounce left or down, it will go another direction. This rule applies no matter what direction the mutalisk attacks from. It means that separated mutalisks which attack the same target at the same time will get the same bounce pattern, which is good for the attacker because it focuses fire. It also means that the defender can arrange units to try to minimize the bounce harm, for example by putting healthier units or units with higher armor on the left of the formation.

Rule 2: If you attack from directly to the left, it bounces the minimum possible distance, hitting the closest enemy. If you attack from directly to the right, it bounces as far as it can. It can make a difference when fighting marines, if the mutalisks are numerous enough that the bounce can kill a marine. In that case, other things being equal, it’s better to attack from the left and kill a close marine that is in range to shoot back, rather than from the right and kill a distant marine that may be idle. You’ll escape taking that much less fire.

These rules aren’t enough to tell exactly which unit the bounces will hit. It’s coded into OpenBW, so there should be people who know....

Observation 1: CherryPi makes only zerglings, hydralisks, and mutalisks. That is it for combat units. Also it prefers the most basic units: Zerglings most, mutalisks least.

Observation 2: CherryPi seems to limit itself to a small number of opening build orders. For example, it seems to have 2 ZvZ builds, one nine pool into zergling pressure and one turtle into spire. The opening builds are varied to adapt to the situation in some way; I haven’t been able to discern how much is due to hand-coded reactions and how much to the learning system. Even so, neither build looks impressive in itself.

Observation 3: CherryPi has inconsistent micro skills. Some skills are outstanding, like storm dodging. Some are inferior, like zergling targeting. Micro doesn’t seem to have been a point of heavy emphasis.

And yet CherryPi is doing extremely well, keeping near the top of the rankings. I imagine that that is partly because (as has been pointed out) it is tuned to beat the strongest opponents, and voters like to match it against exactly those opponents. I don’t think that’s the whole story. I think the learning system deserves much of the credit.

All the observations support a story that the learning system is where the heavy development effort has gone. 1. Empirical learners need data. When the situation is simpler, they can get by with less data. CherryPi might support few combat units because the effort went elsewhere, or it might be a deliberate choice to make life easier for the learning system (at least for now). 2. Similarly for the small range of opening choices. Maybe they didn’t have time to polish more choices; maybe the learning system learns how to use each opening, and they need few enough choices that it is forced to learn a decent amount. 3. Poor micro in some situations probably means that they haven’t had time to work on it. The effort went elsewhere.

Of course this is speculation. You could argue the opposite, because the learning system is not mature either. We can tell because it has visible holes. For example, it struggles with Juno by Yuanheng Zhu, the cannon contain bot, and it has some trouble with heavy rushes like those by Wuli or Black Crow, which play easy-to-understand strategies that you might expect to be easy to learn to counter. In my story, that is of course because learning is hard.

My read on the project’s style is that they are pushing ahead hard and in consequence allowing bugs and sloppiness to creep in to a degree. If I am reading it right, then when the next round of tournaments comes up after the middle of the year, we can expect CherryPi to be formidably capable. It won’t matter if they lose 5% of their games to bugs, provided the long tournament allows time for the bot to learn how to win almost all the rest. And, you know, a sufficiently smart learning system might be able to learn how to work around bugs....

Steamhammer has played another good game against PurpleWave. This seems to happen regularly. See Steamhammer-PurpleWave games for a few older games (which are not as good as this one).

I don’t want to do the blow-by-blow for this game, but here’s a little of the story. 2 games ago, PurpleWave played its worker rush against Steamhammer and won. 1 game ago, Steamhammer countered the worker rush and, partly because the map was more favorable, held it off to win. PurpleWave’s worker attacks are coordinated and Steamhammer’s worker defense is not, so PurpleWave can often win even when Steamhammer plays an ideal counter build. Anyway, having lost, PurpleWave switched to its forge expand opening. Strategically, it’s the opposite of a worker rush.

Steamhammer continued to counter the opening of the previous game, and played its anti-worker rush build. It’s a cautious opening that expects cheese, and it goes so far as to make a sunken colony. It’s the opposite of how you want to play against a big economy build like forge expand. So, after an early fight where Steamhammer didn’t quite figure out how to exploit PurpleWave’s poor cannon placement, zerg was behind.

I want to make 2 points about the game. 1. Steamhammer’s ZvP is much improved. When PurpleWave moved out with its big army, zerg was still behind. Steamhammer put up a fierce fight and eventually wore down the attack and held, losing some drones but not enough to die. And zerg continued to play well after that. Earlier versions won the big fights only when PurpleWave overextended, which did not happen here—thank economy improvements. And earlier versions did not follow up as nicely after the big fight—thank tech improvements.

Point 2. Near the end of the game, Steamhammer expanded to a mined-out base. :-( This version was supposed to fix the last way that that could happen. There is a bug in choosing the base to expand to. I am walking the Path of Bugs and the path continues over the horizon.

Steamhammer’s web page is updated. You can get source and binary distributions there. Documentation to follow when I finish updating it!

The change list for release version Steamhammer 1.4. For the opponent model, see the opponent model in Steamhammer 1.4. The rest is here.

These are the changes since the tournament version. For older changes since the last release:

• 1.4a1
• 1.4a2 (first intended tournament version)
• 1.4a3 (actual tournament version with 2 last-minute fixes)

Overall, this version has substantially improved strategy and macro. There are only tiny changes to micro, and tactics are a lingering disaster area, nobody knows when the power will be restored. It will need to absorb some losses while it learns for itself how to beat various rushbots and others that it used to beat by hand configuration.

debug info

• In the game info in the upper left, the gas steal intention and enemy plan are mentioned, and more time information is displayed. More details in how Steamhammer is getting on.

• In some debugging displays, “_” is changed to “ “ for readability.

bug fixes

• A building manager bug could reserve resources forever. Fixed. The bug turned out to be astonishingly frequent and sometimes locked away thousands of minerals by the late game. Fixing it brought a clear improvement to Steamhammer’s macro. There is at least one more building manager bug that can over- or under-reserve resources, so I put in a workaround that corrects the reserves at the next opportunity. The workaround prints a message when it triggers, so stream watchers get to see the bugginess of Steamhammer in real time. These fixes are the biggest improvement in this version after the opponent model.

• Fixed another bug in gas stealing. There is one known bug remaining. I’ve seen it happen more than one game in a row, but at the same time it is rare and I couldn’t reproduce it when I tried to debug it.

• In inferring the enemy zerg base’s location from a sighting of the first enemy overlord, take into account the offset of the overlord from the hatchery. Older versions assumed that the overlord started from the hatchery location, which caused occasional dangerous scouting misses.

• When trying to expand to a gas base, don’t choose a base where the geyser is exhausted. An earlier version had a similar change to avoid expanding to bases with no remaining minerals. I hope Steamhammer will finally stop expanding to mined-out bases.

• Workers transferring between bases might formerly carry minerals or gas. They were being sent to mine minerals, and gas-carrying workers would lose their gas when they arrived. Fixed.

other improvements

• ProductionManager has a limited ability to reorder the production queue to avoid production delays. When there is not enough gas to produce the current item yet, and another item within 4 spots later in the queue requires no gas and has no dependencies, and there are enough minerals to produce both the current item and the later item, then the later item is moved up and produced immediately. This slight improvement in macro has little effect on Steamhammer’s strength, but it keeps me from scowling every time I see it waiting for gas to start the lair when the following item is the second extractor. On rare occasions reordering can move a hatchery ahead of a batch of scourge, which might save the odd game. Future versions may have more powerful reordering skills.

• Enemy observers are a higher priority target. Many lurkers owe their lives to this change.

code changes

• Made sure to catch all exceptions by reference. I happened to notice that Microsoft recommended it.

• The config file is slightly reorganized. Whenever each race is represented, the order of races is terran-protoss-zerg for consistency.

changes affecting terran and protoss

• In many openings, I changed the scouting command to "go scout once around" to give the opponent model enough scouting information to work.

• Losing a supply depot or pylon is no longer reason to break out of the opening build order. More supply will be built automatically if needed anyway.

• Emergency response: If there is no command center/nexus, make one immediately. Steamhammer will try to stay alive.

• Emergency response: Urgent marines and zealots are made more cautiously. It could queue up too many. The change sometimes makes play worse by not queueing up enough of them, if many workers have been lost. I figure that if you lost a lot of workers, it probably doesn’t matter how you react.

• Strategy reaction: If the opponent fast expanded and is not zerg, take the natural immediately. This is a rough-and-ready response and sometimes wrong, but I thought it improved play most of the time, since otherwise Steamhammer is too slow about the first expansions (which in turn is because it is weak at defending and doesn’t know what is coming). If the opening build order already includes an expansion, Steamhammer ends up double expanding in response to the enemy. It’s usually OK.

• Strategy reaction: If the opponent has air-to-ground units (other than long-range units like guardians or carriers), make turrets or photon cannons for defense. The number made depends on the number and type of the air-to-ground units. This reaction happens so slowly that it is rarely useful, unfortunately. Mutalisks can easily wipe out everything before the first turret starts.

• Terran: If the opponent is rushing and we have marines, make a bunker. Steamhammer has a better chance to survive rushes. How well it works depends on the kind of rush and on where the bunker ends up being placed.

• Terran: In the tanks opening group, I fixed a typo that prevented vultures from being included in the unit mix. They were originally intended. It makes a big difference, partly because Steamhammer is klutzy with tanks.

• Terran: When upgrading at the engineering bay, eventually upgrade infantry weapons to level 3. Older versions stopped upgrading at 1-1. Now it goes 1-0, 1-1 (the science facility is often built late), then continues to 3-1.

changes affecting zerg

The changes to drone production and ZvP unit mix work together. They combine to make Steamhammer react better to large protoss armies. Its economy is stronger and its tech switches over time are more appropriate. Its ZvP is now held back by the big tactical issues like moving overlords unsafely, failing to run drones from attacks, and doing stupid stuff with the army.

• I rewrote StrategyBossZerg::chooseTechTarget() almost from scratch, overwriting bugs and “it works as intended, but...” misbehaviors. When I looked closely, I was surprised how many subtle bugs were hiding in the old code. Tech switches are now smoother and more stable, with fewer bizarre decisions.

• Automatically morph a completed creep colony into a sunken, unless we see an explicit sunken or spore colony coming up soon in the production queue. This fixes jerky macro that used to happen when making reactive sunken colonies. The strategy boss used to do things like queue a creep colony + a drone + a sunken colony to make a sunken. The drone might delay the sunken if income was slow, or waiting for the creep colony to finish might delay later production if income was fast. Now the sunken is made automatically right after the creep colony finishes and other production is not delayed. It’s an important improvement; emergency defenses are no longer needlessly expensive. A few opening builds have been rewritten to make creep colonies and allow them to auto-morph into sunkens, but I left most openings alone for now; they all still work without change. If you want to make creep colonies and wait until danger approaches to morph them (Arrakhammer does this), then you’ll need to do some recoding. In a future version I intend to attach intentions to creep colonies, to handle all cases more transparently.

• Steamhammer has a bug that makes it believe that 2 mutalisks can defeat a spore colony. The 2 flyers attack, then it loses 1 and the survivor retreats, then a fresh muta arrives, etc., and soon the game is over. Someday I’ll fix the bug, but for now I put in a cheap workaround. For each spore being sent to the combat simulator, leave out 5 of our mutalisks—any 5, that’s all. It’s surprisingly effective, an important improvement to ZvZ play.

• I changed the rules for limiting the number of scourge made in ZvZ, a tricky point. Formerly, Steamhammer made 4 mutalisks and then allowed itself as much scourge as needed to combat the enemy air force. It often skipped scourge it needed because it couldn’t keep mutas in the air, and then overbuilt scourge when it could, using up all the gas and delaying production of mineral units. I changed it to a simple rule: It’s allowed to add more scourge if the current number of scourge is less than the number of mutalisks. So it can make a batch of scourge after only 1 muta if needed, but while keeping that scourge can’t make more until it has matching mutalisks. It’s far from ideal, but it’s an improvement because it allows scourge early without pushing the essential mutalisks off the table.

• In an emergency, continue making drones at a low rate. Older versions stopped making drones entirely to concentrate on defense, and continued pressure could leave Steamhammer strategically trapped, unable to reach essential tech.

• Steamhammer has long compensated for enemy static defense by making more drones. Now it also compensates for its own sunken colonies by making more drones. The sunkens make it safer, so it can safely add more drones, and they are expensive, so it needs more. When the enemy attacks and Steamhammer makes emergency defenses, the pushback after the attack comes later, but it is stronger.

• In ZvZ, reduce drone compensation by half, because making too many drones is fatal. There was already a partial method for this, but I made it broader and simpler. I put in this change after watching a test game in which the zerg opponent turtled hard and Steamhammer made 18 compensatory drones and lost without defending itself.

• Steamhammer will drop a queued hatchery if it has lost drones and no longer has enough to support production at the hatchery. I increased the limit of how many drones are needed. This is a slightly risky change that could cause blunders in rare cases, but in test games I saw it improve play.

• Tech evaluation changes: In ZvT, Steamhammer has a clearer idea that lurkers suck versus factory units and hydralisks die to tanks. In ZvZ, get the queen’s nest later, since the hive is delayed; it was often getting a useless queen’s nest. In ZvP, favor hydralisks more depending on the protoss unit mix, and favor lurkers more if the hydralisk upgrades are done. The change makes Steamhammer more able to cope with a big protoss army later in the game.

• Some openings scout earlier and/or more thoroughly, to collect data for the opponent model.

• Added openings 2HatchLurkerAllIn (fun against terran) and ZvZ_12Hatch. ZvP_10Hatch is renamed to Over10Hatch and revised; see optimizing one opening build. 11Gas10PoolSpire is renamed to 11Gas10PoolMuta for consistency with other opening names, and shortened and slightly revised.

• Fixed a rare bug that could cause 3 evolution chambers to be built, though Steamhammer is only able to upgrade using 2. It happened in far fewer than 1% of games.

• A bunch of minor strategy tweaks: Try harder to make more units when we are short of them. At the same time, be more cautious about spending gas when under attack. Try not to let a macro hatchery delay essential tech (the lair finished, for heaven’s sake make the spire next!). When building a spire solely to later make it into a greater spire, wait for the hive to start first.

• A few minor micro tweaks: When we are under attack, cancel a morphing sunken if it would finish with under 30 hp (after the 100 hp subtraction when it completes). When the attack is thought to be over, the creep colony auto-morphs into a sunken to start healing from the highest possible base (see the weird life history of the sunken colony). Irradiated melee units burrow to spare their companions, if they can; ranged units already did this. Steamhammer doesn’t research burrow, so they usually can’t. Badly injured lurkers are a little more eager to stay underground and heal.

missing from this version

A new version of FAP is out. Steamhammer doesn’t use it yet.

I tried to get shield batteries to work for protoss, but I had trouble producing behavior good enough to be worth it. I’m not sure how a shield battery can be more difficult to use than a bunker; I thought it was the other way around. Anyway, it was taking too much time so I postponed it until a later version.

The building manager seems to place each building more than once. Placing buildings is expensive, so that’s bad. Also I think it may delay construction of some buildings by causing extra worker movement. It may be an error in the last-second “can I really build here?” test. Fixing that might help protoss avoid overstepping the time limit when the main base fills up, though it wouldn’t solve the problem.

There’s a striking pattern in the SSCAIT round of 8 that I want to point out. First, the video and the Liquipedia page. I’ll discuss the results. The round was made up of 6 newer and frequently updated bots, which were all paired against each other, and 2 old hands which were paired together. Each pair played a best of 5 match.

The old hands were Killerbot and XIMP, not updated for this tournament. They played a balanced match which was decided by strategy and tactics and fine details of play which the bots didn’t take into account. There weren’t any obvious or decisive bugs, it was about good everyday play.

Taking the rest of the matches from the top, Iron-Microwave went to Microwave because the zerg bot was able to break the terran wall. Iron knew how to repair its wall and made marines to defend it. But the marines were afraid of the zerglings which could not attack them through the wall, and did not shoot. I suppose that the combat simulator doesn’t understand the wall and says “uh oh, we’ll lose if the melee units get close, keep a safe distance!” Microwave won by exploiting a bug in Iron.

Steamhammer swept Arrakhammer 3-0 with 3 zergling builds in a row. Nepeta didn’t point it out in the video, but in each game Arrakhammer’s own zerglings fought piecemeal, inefficiently engaging with a partial force and then retreating, suffering more damage than they dealt. Steamhammer won by exploiting a bug in Arrakhammer.

CherryPi beat McRave 3-2 in a close match. McRave showed strategy weaknesses which deserve some of the blame. Nepeta’s points in the video are valid: Expand earlier, get +1 attack for the zealots to counter zerglings (since with +1 a zealot kills an unupgraded zergling in 2 hits instead of 3, a giant difference), and don’t cower in your own base when you play an aggressive 2 gate opening (you should never be behind in units for long). But McRave’s biggest weakness was that its high templar usually did not cast psionic storm, and seemed happy to suicide themselves. CherryPi won by exploiting the bug. In one game that CherryPi lost, zerg did not build the macro hatcheries it needed to keep up its zergling-heavy unit mix, and CherryPi’s mineral bank grew into the thousands. McRave won that game by exploiting a bug in CherryPi.

Two conclusions are loud and clear. 1. 6 of the 8 participants that made it this far are frequently updated and fast evolving. Only 2 old timers could keep up. The hard work to make many improvements pays off. 2. The same frequent updates leave bots vulnerable to bugs. The old hands were solid (at least they looked solid this time), and the fast movers had fragile spots.

I’m not sure there are any lessons for bot authors, other than “hard work pays off” and “fix the worst problems first,” both of which we already knew. The pattern in the results was so striking that I couldn’t ignore it.

Next: Steamhammer 1.4 change list.

SSCAIT has reopened for submissions. I uploaded Steamhammer 1.4 (including a copy to play as Randomhammer). Its web page should be updated with the source tomorrow, and the day after I’ll post the change list here.

Today is Steamhammer 1.4’s opponent model. The features are:

• Recognize some enemy opening plans.
• Predict the enemy’s opening plan from experience against this opponent.
• React to the enemy’s predicted or actual opening plan.
• Choose openings based on the predicted plan.
• Decide whether to steal gas. Does experience suggest it may be worth trying against this opponent?

The code that implements the features has these parts:

1. The plan recognizer, which looks at the current game situation.
2. Game records, which save information about past games against this opponent, including the recognized plan.
3. The plan predictor and gas steal decider, which draw conclusions based on the game records.
4. Strategy reactions to predicted or recognized enemy plans are in various places–they are uses of the opponent model, not parts of the opponent model.
5. Opening choices to counter the predicted plan can be set in the configuration file.

1. The plan recognizer was first written up in December. It tries to understand the opponent’s intentions during the earliest part of the game. The code is in OpponentPlan.cpp and is less than 150 lines—it is rudimentary.

• Unknown - No plan recognized. The plans are not exhaustive, so this is common.
• Proxy - Enemy buildings in your base. This doesn’t include cannon contains or other more distant proxies.
• WorkerRush - Like Stone.
• FastRush - Basic units faster than 9 pool, 8 rax, or 9 gateway.
• HeavyRush - 2 gate zealots, 2 barracks marines, etc.
• SafeExpand - Static defense before expanding to the natural. Zerg can’t do this.
• NakedExpand - Expansion with no static defense.
• Turtle - Static defense without expanding.

My early impression of the plan recognizer was that it often failed to recognize plans, but was rarely wrong when it did. With more experience, I think it often misrecognizes plans severely. It’s crude and clumsy. Even so, when it helps it helps a lot. It’s a net win.

2. Game records are handled in GameRecord.cpp. Steamhammer writes a file for each opponent, like many learning bots. Here is one record from a file named om_UAlbertaBot.txt, where “om” stands for opponent model, with annotations so you can make sense of the list of numbers.

1.4                  <- record format version (from the Steamhammer version when it was first used)
ZvRZ                 <- matchup
(2)Destination.scx   <- map
Over10Hatch          <- Steamhammer’s opening
Heavy rush           <- initial predicted enemy plan
Fast rush            <- actual enemy plan
1                    <- we won, 1 or 0
0                    <- frame we sent a scout to steal gas, 0 if never
0                    <- gas steal happened, 1 or 0 (extractor was/was not queued)
2110                 <- frame enemy first scouted our base
3102                 <- frame enemy got first combat units
0                    <- frame enemy got first air units
0                    <- frame enemy got static air defense
0                    <- frame enemy got mobile air defense
0                    <- frame enemy got cloaked units
0                    <- frame enemy got static detection
1726                 <- frame enemy got mobile detection
12309                <- last frame of the game
END GAME             <- end mark

Theoretically, a single file could have records in more than one format. Of course, only one format exists so far. The “frame enemy got” times recorded are the time we first saw such a thing, which may be much later than it actually happened. For example, here we first saw an enemy overlord (a mobile detector) on frame 1726, but it existed the whole game. The END GAME mark is redundant. It gives us a way to recover in case data in the middle of a file is corrupted—we can skip ahead past the next END GAME and continue reading the file from there.

3. The plan predictor and gas steal decider look at the game records near the start of the game.

The gas steal decider was written up in December. Nothing important has changed since then.

The plan predictor runs once at the start of the game. It looks through the game records to see what recognized plans the enemy has played. It is close to the bare minimum: It counts the recognized plans in the game records for this matchup, ignoring unknown plans, and weighting recent games more using a discount factor so that the past is gradually forgotten. That way it reacts quickly when the enemy changes its play. Whether the game was won or lost, whether past predictions were correct or wrong—all the other information is ignored.

If the opponent was random, then when the opponent’s race is found out, the plan predictor runs again. In the first run, it counted all game records where the opponent went random. In the second run, it counts only games where the opponent was the same race as this game. The predicted plan may change.

4. Strategy reactions could be written in anywhere, but so far they are in StrategyManager for terran and protoss, and in StrategyBossZerg for zerg. So far, all the reactions are reactions to the enemy plan, not to any of the other information (even though it has obvious uses). Some strategy reactions are made when the enemy plan is recognized. Some reactions must begin in time, and happen when the plan is predicted. For example, against UAlbertaBot, the initial predicted plan is “Heavy rush”. If Steamhammer finds out that UAlbertaBot is zerg, it knows that zerg follows a different plan. The predicted plan changes to “Fast rush” and efforts to stop the zerglings begin immediately (if Steamhammer is zerg or terran; the protoss reaction is turned off because I didn’t get it working).

Good strategy reactions are the hard part. I find them much more difficult than the opponent model proper. Some of Steamhammer’s reactions are weak.

5. Openings to counter specific enemy plans can be written into the configuration file in a new subsection CounterStrategies. The opening is chosen once at the start of the game and can’t be changed (in this version), so only the initial predicted enemy plan matters.

Steamhammer first looks for counter strategies specific to the enemy race. The name is in the format “Counter [plan name] v[race character]”. The race character is “U” for Unknown if the opponent went random. You can use all the usual features of random opening selection. As you can see in the example, zerg is configured with a wider range of choices.

"Counter Safe expand vT" :
	{
		"Terran" : "14CCTanks",
		"Protoss" : "13Nexus",
		"Zerg" : [
			{ "Weight" :  1, "Strategy" : "FastPool", "Weight2" : 5 },
			{ "Weight" :  9, "Strategy" : "9PoolSpeed" },
			{ "Weight" :  0, "Strategy" : "ZvT_3HatchMuta", "Weight2" : 50 },
			{ "Weight" : 50, "Strategy" : "ZvT_3HatchMutaExpo", "Weight2" : 0 },
			{ "Weight" : 30, "Strategy" : "3HatchLurker" },
			{ "Weight" : 10, "Strategy" : "3HatchPoolMuta", "Weight2" : 0 }
		]
	},

If no counter strategy is found for the specific enemy race, Steamhammer next looks to see if there’s a general one for all races—leave out the “vX” string. This example points to a reusable strategy combo that specifies openings for each race Steamhammer might be playing, no matter the enemy race. The strategy combo feature has not changed.

"Counter Worker rush" : "AntiFastCheese",

If no counter strategy is found, Steamhammer falls back on its usual random opening selection. So if the regular repertoire is best in any given case, don’t specify a counter for that case.

For the following version Steamhammer 1.4.1, I will add at least one large piece to the opponent model. I'm likely to change my mind once I see how this version does in practice, but at the moment I'm thinking of fine-tuning plan prediction and opening selection based on wins and losses. That will raise Steamhammer's performance ceiling; with enough games, it will learn much more. Other possibilities include: • Make use of more of the information in the game records. This opponent doesn't get detection, use cloaked units; that opponent gets air units around frame X, add a spire for scourge just in time. • Have Steamhammer collect data on its own openings so it can generalize: Play a fast/slow opening; play a lurker/muta opening. • Revamp the plan recognizer with a richer set of plans, maybe a hierarchy so it can refine its conclusions over time. • Machine learning for a probabilistic plan recognizer and/or plan predictor. • Restore the originally planned extensive game records, making it possible to predict unit mixes over time throughout the game. • Add the ability to change openings during play instead of deciding ahead of time, so that decisions can be made as late as possible. • Move scout timing into the opponent model alongside the gas steal decider. • Have expansion decisions or hatchery placement influenced by the opponent model. “Hmm, historically in situations like this we do poorly if the third hatchery is at an expansion. Make it a macro hatchery instead.”

I have no shortage of ideas!

Next: Brief remarks on the SSCAIT round of 8.

Steamhammer 1.4 is more or less finished. I keep thinking I should package it up and be ready to go, and then I come up with another easy way to fix a weakness. I may yet fix a few more weaknesses, but progress so far is good. I’ve been running a lot of tests.

The simplest opponent for Steamhammer is Stone. Steamhammer now defeats Stone, from most openings and on most maps, without any learning needed, because its reactions to worker rushes are improved. (It loses if it plays 4 or 5 pool, because it doesn’t have enough drones to fight with.) I think a more aggressive worker rush will beat it, because Steamhammer’s micro is uncoordinated. But if you play a worker rush in the first game, win or lose, then Steamhammer will counter it in the second game.

The next are zerg rushbots and others that play fixed strategies that the plan recognizer is able to recognize. Steamhammer may lose the first game, or the first few—though maybe not, because its rush reactions are improved. As soon as it scouts well enough to recognize the rush (it’s not too reliable), the next game it switches to a counter and almost always wins from then on. It works against ZZZKBot, it works against Neo Edmund Zerg, it works against UAlbertaBot set to play zerg, it works against 2 different versions of the old bot 5 Pool that I have saved. I think it just works.

The next step up is UAlbertaBot without learning, which plays random and has a fixed strategy for each race. Steamhammer classifies UAlbertaBot’s zerg opening as “Fast rush” and its terran and protoss openings as “Heavy rush”, so once it has collected enough data it always plays to counter the heavy rush. It wins over 90% versus terran and protoss. And because of the improvements to rush defense, plus the fact that the opponent model knows to expect the fast rush as soon as it finds out that UAlbertaBot rolled zerg, it has a chance to hold the zerg too. In the end, the learned openings scored better than the tournament version with its hand-coded counter opening.

Learning opponents are more difficult. I played test matches against the opening learning bots Microwave and Zia. Microwave has the edge over the tournament Steamhammer. Against the new Steamhammer, the opponent model prevented Microwave from finding a stable way to win. When Microwave won a game, it tended to repeat the opening the next game, while Steamhammer tended to counter Microwave’s opening of the previous game. The match came out about even, with Microwave constantly switching its openings, unable to find one that worked consistently. Zia in contrast is weaker than the tournament Steamhammer, but the new Steamhammer was not able to come to grips with it. Zia plays a variety of different openings that look the same to Steamhammer’s plan recognizer, plus it rarely repeated an opening after winning only once, so Steamhammer was unable to choose good counters. Zia is the only opponent I tested that did better against the opponent model than against the random opening selection.

In one other test, I played a match of the new Steamhammer 1.4 versus the tournament version 1.4a3, both playing random. An opponent with random race and random choice of openings should be the worst realistic case for the opponent model opening selection (only a knowledgeable adversary that can predict Steamhammer’s predictions is worse). Also, since Steamhammer is still on BWAPI 4.1.2, the bot does not know when it starts up that it went random itself. The opponent model has no choice but to take separate statistics for each matchup, so when Steamhammer plays random it learns more slowly. The new version scored approximately 2:1. Even with tons of wrong predictions from the opponent model, the new version plays better.

These are only some of the tests I’ve run. There are opponents that the opponent model provides no leverage over. The plan recognizer has a limited repertoire, and if you step outside it can’t help. In the configuration file, I removed most but not all of the hand-selected enemy-specific openings. Removing the rest will be for a later version in the 1.4.x series.

Overall, the upcoming Steamhammer 1.4 is much improved over the tournament version. It is better in all matchups, with smoother macro and several small but useful skills for terran and protoss. It is especially improved in ZvZ and ZvP. It hasn’t caught up with McRave, but it should land upsets more often. I’m eager to see how it does in the wide world of the SSCAIT ladder. Are there more opponents like Zia that will leave it confused?

Next: How it works.

Some bots, like Willbot, take a passive line and build up a large force before taking any offensive action. Others, like Steamhammer, prefer to keep the pressure on and attack whenever possible, even at high risk.

Neither style of play shows real understanding of strategy. Here are 2 simple principles that should be uncontroversial:

1. The side with more stuff has an advantage. If you have more units, or better upgrades, or higher tech, that is an advantage. You get those things by spending minerals and gas, so ultimately the side with more resources has an advantage.

2. The side on the defensive has an advantage. If you have static defense built, tanks sieged, lurkers burrowed, or simply units deployed in good position to engage, that is an advantage. If you have a shorter route to your production buildings, then reinforcements arrive sooner and that is an advantage. If you can stop or delay or channel attacking forces with blocking buildings or chosen terrain, that is an advantage.

From these 2 axioms, we can act like Euclid and derive a theorem: If you control more resources, you don’t have to attack. You can attack if you spot a weakness, but you don’t need to. You can hang back, in a safe defensive position, until your theoretical resource advantage manifests as a practical battlefield advantage, and then attack. At least you can wait until you are maxed at 200 supply.

Containment is the common case we see in bot play. When Steamhammer has the opponent contained, it constantly tries to press forward to notice and immediately exploit any weakness. It’s usually a mistake. Typically you should contain at the best defensive location you can, take care with the forward units you risk for vision, and scout to make sure the opponent can’t sneak an expansion or bypass the containment with drops or air play. You control the resources on the rest of the map, and that is an advantage.

The general rule is: If you can take expansions and deny expansions to the opponent, you will win. You can win without ever entering a fair fight. If the opponent takes an undefended expansion, smash it. If the opponent moves their army to defend a new expansion, smash the enemy natural instead. At worst, you force the opponent to allocate forces accurately to defend all threats. Tscmoo is the bot which implements this rule the best, though it still seems a bit crude to me.

The extreme case is a map split, where each side ends up controlling about half the map—except one side controls an extra base or two. Humans sometimes play from the beginning of the game for a favorable map split. I don’t think any bot understands the idea.

How do we get from here, strategic ignorance, to there, understanding tactical force allocation risks and tradeoffs to meet the strategic goal? Well, I mis-stated it; bots don’t have to understand, they only have to take the right actions. Bots today that contain the enemy don’t (it seems to me) understand what they are doing. They are following rules that produce containments as an emergent behavior. It’s a valid approach. But I recommend more explicit knowledge representations, because I think it will lead to faster progress.

Sometime this year Steamhammer will get an evaluation function that tells it how good or bad a situation is. The first version may be a simple hand-written evaluator that is used for a few decisions. In time, I hope to create an accurate evaluator by machine learning, good for decisions throughout the game. Then the same underlying knowledge, encoded the evaluator, will let Steamhammer adapt its openings moment by moment, choose its unit mix, and maneuver its forces.

Why is the SSCAIT tournament re-running so many games in the elimination phase? There may be discussion of this on Facebook or somewhere, but I haven’t seen it. (I don’t use Facebook at all, because I don’t want to support their world domination plans. It would interfere with my world domination plans.)

I keep seeing games come up that are clearly tournament matches—then, later, the same matchups appear again. It looks exactly as though tournament matches are being replayed. Which games will be declared official?

They did the same thing on a smaller scale last year, and it caused some controversy. See the comments to the post Steamhammer vs LetaBot, SSCAIT round of 16 from last January. Last year it affected Steamhammer, and yet it didn’t bother me at all, partly because I saw the single elimination bracket more as entertainment than as a test of strength. There are legitimate administrative reasons to replay games. This year it doesn’t affect Steamhammer (games have been replayed, but with the same predictable results), but it bothers me more. There is an effort to make this phase of the tournament more rigorous, and replaying games undercuts that. When a learning bot is paired against a non-learning bot, such as CherryPi versus Iron, having more games against the opponent gives an advantage to the learning bot.

I doubt there’s favoritism behind the scenes, but how can I know? This comes up in politics all the time: It is not enough to avoid impropriety. If you want to be trusted then you also have to avoid the appearance of impropriety.

Yesterday I revised the ZvP_10Hatch opening. It was originally designed to counter 2 gate zealot rushes, and now (with the opponent model) it is used to counter expected heavy rushes by terran and random opponents too. I renamed it Over10Hatch. The basic build order is extractor trick for a 10th drone, overlord, hatchery, and a couple of sunkens to help hold off the mass zealot pressure so it’s safe to make drones and tech.

The changes are small. If you watch casually, you might not notice. The extractor trick is now before the overlord, not after, a careless oversight in the original. The first sunken is delayed slightly, and the second sunken needed to deter masses of zealots is delayed longer. The number of early zerglings is cut back to just the number needed in the worst case. The early game plays out much as before, it only looks as though Steamhammer might be cutting it a little close. (CherryPi’s versus-protoss opening often gives a similar impression. The team knows what it is doing.) You could miss that there is a slightly higher drone count.

A pro would cut it closer, but a pro has good judgment. The way the rest of the game plays out looks different. In the old build, Steamhammer would often narrowly hold the zealots, get lurkers just in time to survive, and use the opponent’s lack of tech to slowly push for victory. With the extra income from the new build, Steamhammer holds the zealot pressure easily, safely gets lurkers, and quickly smashes the protoss with a mass of lurkers and lings. The turnaround in the game looks completely different. The effect versus no-academy marine rushes like UAlbertaBot’s is similar.

I thought it was a good lesson in the importance of an efficient build.

Steamhammer has a scouting feature that works in ZvZ matches: When it sees the position of the enemy’s first overlord, it tries to infer where the enemy base is. See Steamhammer’s new scouting skills. It turned out that on some maps—Icarus and Roadrunner—the inference can be wrong. The reason is that Steamhammer measures the distance from the potential hatchery location, not from the overlord’s start location, which is offset. If 2 bases are at nearly the same distance, the offset can matter.

So I came to the question: At the start of the game, where is zerg’s overlord placed relative to its hatchery?

The answer turned out to be simple. The overlord’s x position is offset by either -99 or +99 from the hatchery, and its y position by either -65 or +65. The position is chosen to be whatever is closer to the center of the map. In code:

overlord.x = hatchery.x + ((hatchery.x < 32 * BWAPI::Broodwar->mapWidth() / 2) ? +99 : -99);
overlord.y = hatchery.y + ((hatchery.y < 32 * BWAPI::Broodwar->mapHeight() / 2) ? +65 : -65);

I don’t promise that this is exactly right. It works in the cases I tested. I imagine that somebody on the OpenBW project could tell us the exact condition.

backward games

I added a zerg-versus-zerg 12 hatch opening to Steamhammer. It’s the greediest common ZvZ opening, so Steamhammer only plays it when the opponent model expects that the opponent will also play a hatchery-first opening. But look what happened: In a test game today against Microwave, Steamhammer opened 12 hatch and Microwave played its 5 pool opening. Steamhammer saw it coming only after the spawning pool was started; all it could do was start a sunken as soon as possible. Microwave’s zerglings tore down the natural hatchery first, and that was enough time for the sunken to finish and 4 zerglings to hatch, just enough to hold the rush. Steamhammer had a huge drone advantage, went up to 3 hatcheries, and smashed down Microwave by brute force while fighting right in the face of Microwave’s emergency defensive sunken. That’s the way to play!

It’s no wonder that pros, with their much stronger defensive skills, are so willing to play a greedy opening like 12 hatch in ZvZ.

Microwave’s 5 pool, by the way, is a clever build. Microwave builds a second hatchery at its natural to keep the zergling numbers up, and if the pressure doesn’t work out it drones up and switches to mutalisks. The opponent has to keep playing well to stay ahead. In a different test game, Steamhammer predicted the 5 pool and played an anti-rush opening which held the zerglings easily—then Steamhammer lost despite its advantage because it didn’t tech fast enough: The lair finished, and it decided to build a macro hatchery before the spire; the spire finished and it decided to build a third base before any mutalisks.... That’s not the way to play!

Steamhammer 1.4 is frozen

Steamhammer is feature-frozen for the 1.4 release. I will keep on fixing bugs and minor weaknesses (lair finished...) and tweaking the configuration, but I won’t start anything bigger that might delay the release, though there are plenty of temptations. I should be able to upload within a day after SSCAIT reopens for submissions, and release after another day or so once I can see that I didn’t break anything obvious.

Steamhammer 1.4 is substantially stronger than the tournament version in some ways. I think it is especially improved at transitioning out of the opening in ZvP. It will start off slowly on SSCAIT, because it will have to learn for itself how to beat a number of opponents that it currently beats by hand configuration. Also the opponent model has weaknesses and blind spots that are sure to cause some surprises. Still, if I didn’t mess anything up, Steamhammer should at least keep up with the competition and maintain a place near the top. Its play continues to grow more complex and interesting.

the game info display

As an example of a minor change, I have updated the game info display that Steamhammer can draw in the upper left corner.

If we have decided to steal gas, whether in the opening build order or in the opponent model’s auto gas steal decider, the strategy line adds + steal gas.

Opp Plan is the opponent’s plan. It is often Unknown. If the opponent model has predicted the opponent’s plan and the plan recognizer hasn’t verified it yet, we get the orange word expect. The opponent model tends to be overconfident about what it expects. When the plan recognizer thinks it knows what is going on, expect disappears and the recognized plan shows. The plan recognizer also relies on inadequate information, but at least it looks first. Steamhammer uses both the expected and the recognized plans, to a limited degree, for different purposes and with different levels of confidence.

The Time: line gives the game time in frames and in minutes:seconds. The minutes:seconds display is both more compact and easier to read than the original XXm YYs display. Then come the mean and maximum time spent per frame, in milliseconds, so we can start to get an idea of when things are slow.

CherryPi has a striking habit of barely scouting before concluding that it knows what the opponent is going to do, and then seeming reluctant to ever change its mind. A pro player will often also barely scout, but pros probe for new information later in the game and are ready to draw new conclusions. I thought the CherryPi-TyrProtoss match from the SSCAIT 2017 round of 16 was a tantalizing example (see the video). It doesn’t tell us how CherryPi works, but it offers a small hint.

In the first game Tyr opened with 2 gateways for early pressure. CherryPi scouted the 2 gateways with a drone at about 1:50 into the game and reacted somewhat logically with an array of sunkens, behind which it built up a strong economy. Tyr saw the sunkens and did not seem to react to them at all, strategically. Protoss should have expanded immediately and taken steps to prevent zerg from expanding further. Tyr did neither, and it lost.

Seeing the opponent’s build and reacting is nice, but it’s not a special skill. In this game, CherryPi did well but didn’t show anything special in terms of strategy.

In the second game we saw something that might be more interesting, but it’s still unclear. At about 2:10 the scouting drone saw a forward pylon and immediately returned home without looking further. When I first watched the replay in OpenBW I thought that CherryPi had left before seeing the forge warping in behind the drone, but when I watched the replay in StarCraft with only CherryPi’s vision turned on I saw that it did just catch the forge starting. Still, the fact that the drone turned around immediately suggests that CherryPi had seen enough; the forward pylon was all it wanted to see to understand the opponent’s build.

CherryPi reacted with 3 hatcheries before pool, which is safe versus forge expand but allows good responses from protoss too. I imagine the CherryPi team chose the build because it’s unusual in that situation and a protoss bot might not know a good reaction (though any human player would have an idea). In this case they were right, and CherryPi got ahead and won.

What do you think happened? Did CherryPi see the forge and know there couldn’t be a gateway yet, so it could safely play a slow build? Or did CherryPi take a leap in the dark and make its decision after seeing only the pylon? Nothing stops protoss from building 2 gateways at the forward pylon and making an aggressive rush—a pro is more likely to do that than to build the 2 gateways in the main. The advantages are that the rush distance is shorter and it protects the natural.

opponent modeling skills

I don’t know what CherryPi is doing. Maybe there’s discussion about it somewhere, which I haven’t seen. But I can’t help comparing it to Steamhammer’s opponent model. When all the intended features of the opponent model are implemented, Steamhammer will be able to see the pylon and immediately conclude “I’ve seen you do that before, it was the start of a forge expand opening and you’re probably playing it again. Let’s counter the forge expand.” I suspect that may be what CherryPi did—probably not in exactly the same way, but maybe in a way that’s broadly similar.

Bots tend to be predictable, and their opponents can take advantage. It’s one of the ideas behind Steamhammer’s opponent model. Seeing a forward pylon narrows down what the opponent is doing, but doesn’t zero in on one possibility. But if the opponent tends to continue the same way as in the past, you can act as though there were only one possibility and start to counter it earlier, gaining an advantage. (Against a forge you make drones, against gateways you make zerglings and a sunken. If you make unnecessary zerglings, you set yourself back.)

The development Steamhammer version can already do this, in a limited way against a random opponent. For example, against UAlbertaBot, Steamhammer says “this is probably a heavy rush (with zealots or marines), so I’ll prepare for that.” If it finds out that UAlbertaBot rolled zerg, it immediately (well, within 8 frames) realizes “uh oh, I was wrong, it is going to be a fast rush which has a different counter.” It doesn’t wait to see early zerglings, or a spawning pool, or a drone count, it immediately starts to adapt its build. The exact reaction depends on the timing, but there is code that says, for example, to cancel the second hatchery if it will allow a spawning pool to get up faster. By reacting immediately, Steamhammer has a better chance to survive despite its weak defensive skills.

The opponent can thwart the opponent model, at least to an extent, by being genuinely unpredictable. That’s a countermeasure. Steamhammer’s opponent model should force top bots to vary their play more. That’s another idea behind the opponent model.