Starcraft AI blog

Today I tested a 12 hatchery in main opening for ZvZ. 12 hatch gets zerglings late, and it is risky against 9 pool openings, which are common in both human and bot play. Steamhammer in the past (after version 0.2 from 2016) rarely plays even the faster 12 spawning pool, because its poor defensive skills leave it vulnerable to normal play. I only added a 12 hatchery opening for ZvZ in version 1.4, to be played when the opponent model predicts that the opponent will also open slowly. Otherwise it seemed too risky.

The hatchery on 12 lets the player get more drones than any other opening that is playable in ZvZ. I realized that you can make a sunken colony and still end up with 11 drones, a strong count for ZvZ. So I made a 12 hatchery in main opening with a creep colony timed to finish a little before the spawning pool finishes (a deliberate production delay would make it more efficient). The sunken colony gives it a chance to survive early aggression despite poor drone defense skills. It uses the 2 hatcheries to make many zerglings, threatening any 1 hatchery opening, and at the same time techs to mutalisks fairly quickly, threatening zergling openings.

How did it work out? I tested with self-play versus different openings.

I started with Steamhammer’s 9 pool speed opening, which builds up to 3 hatcheries as it gets the money, going all-in on zerglings. Would the 12 hatchery survive? On 2 player maps, the 9 pool zerglings could arrive too early and win. On 3 player maps, the 12 hatchery seemed risky. On 4 player maps, the 12 hatchery usually won: It survived the early attack, matched zergling numbers later, and won with mutalisks. The plan worked.

It also got a plus score versus fast mutalisk openings like overgas 9 pool and 11 gas 10 pool. The 12 hatchery could not catch up in mutalisks and struggled to defend in the air, but still won the majority of games with mass zerglings. The one hatchery openings could not keep up in zergling count. The plan worked here too.

I didn’t bother to test against in-between openings like 9 hatchery, or overhatch, or the various overpool openings. The 12 hatch survived more extreme openings, and it has a superior economy to middle-of-the-road openings, so it should be competitive. I will try a few tests against other opponents, to see how it stands up against bots with different skills.

I already configured Steamhammer to play the 12 hatchery in main opening, on 4 player maps only, at a fairly high rate. I predict it will help. I also configured it for all maps as an alternative to the 12 hatch at natural opening when the opponent model says that it is playable.

I want Steamhammer to learn to select its own openings not only to suit the opponent, but to suit the maps. Next: Ideas for adapting to specific maps.

In a system inherited from UAlbertaBot, Steamhammer keeps a queue of items to produce (units, buildings, research, etc.). As soon as the prerequisites for the next item in the queue are satisfied, it produces the item: There have to be enough minerals and gas; for a spire the lair has to be finished; etc.

It’s kind of strange, but this isn’t always best! There are times when it is better to mine a little longer rather than sending a worker to construct a building right away. This is true even when the building is definitely the next thing to make, and you don’t want to schedule anything ahead of it.

I know a clear example among Steamhammer’s openings. In the terran BBS opening, terran builds 2 barracks at 8 supply. Steamhammer starts each one as the 150 minerals for the building become available. It’s best to start building the 2 barracks simultaneously: The SCV that would start the first barracks gets to mine a little longer, and the marines are not delayed (in fact, they’re stronger in a group, so there’s an advantage to producing them 2 by 2), so delaying the first barracks puts terran slightly ahead.

Building a spore colony because the enemy got mutalisks earlier is 2 examples in 1: First, as a game issue, you want to time the spore colony to finish just before mutalisks reach your base. Second, you need both an evolution chamber and a creep colony, and purely as an efficient production issue you would like to time the creep colony (300 frames to complete) to finish simultaneously with the evo (600 frames to complete), so that you waste as little mining time as possible. Another example is starting a hydralisk den to finish simultaneously with the lair, or to finish when you have just enough gas to start lurker research. I expect you can think of examples for other races.

How should this be implemented? Ideally, the bot should figure out for itself when it is better to delay production. Game issues like timing the spore colony should be decided by a tactical calculation outside the production system itself; somebody else needs to tell the production manager, “get this by time t.” For purely production issues, the bot might simulate the timings to find the most efficient, or it might set up a system of constraints and solve the optimization problem. These calculations are very elaborate—I might implement one of them someday, but not any time soon.

In the meantime, I’ve been wondering whether there is a good way to manually configure delays in common cases. It would be easy enough to add a command to the build order, like “wait for 300 minerals” that you’d put before starting the 2 barracks for BBS. The problem is that that is only one case. Sometimes you want things to start simultaneously (the barracks), sometimes to finish simultaneously (the evo and creep colony), sometimes to finish exactly when conditions are right for something else to start (hydra den finishes when you have 200 gas for lurker research). I haven’t thought of a simple way to specify the different conditions you might want.

Well, in the end, this is a minor optimization. Steamhammer has bigger problems with building stuff efficiently. Even so, I’d like to address it someday. So if anybody has ideas....

At some point I opened the istream of bugs, and now I can’t find the close method. I think the bugs are holding the valve open by force.

Yesterday I fixed a basic bug in choosing tech targets. I rewrote that code for current release version 1.4. It fixes a number of bugs, and in testing it came out better than the old version, especially in ZvP. After release, I started seeing puzzling mistakes in ZvT—Steamhammer refused to switch to lurkers after opening with mutalisks, no matter how much it needed them. Getting that decision correct was one of the reasons for the rewrite!

When I read the code closely, the cause was easy to see. A condition was reversed: Where it should have been “no need to do this step if we already have a hive,” it was written to mean “do this only if we have a hive.” I have to imagine that I made some late change and didn’t check it carefully. Quick tests show that the fix improves play in both ZvT and ZvP. With more experience, we’ll see how that holds up.

In other news, Steamhammer has been on a losing streak against Killerbot by Marian Devecka. I started to worry that I had introduced a bug, so I laid in a short match to find out. Steamhammer scored 11-4, good but not as good as in days past. I guess the losses were mostly due to a string of bad luck. I did spot a couple of contributing issues: The opponent model can sometimes cause a bad strategy reaction when it misdiagnoses Killerbot’s opening, and mutalisks which are busy destroying the enemy base are unable to return home to defend (due to a change that improves play on average, but harms it here). If I fix those plus a few longstanding defensive and tactical weaknesses, Steamhammer should score near 100% against Killerbot.

My list shows 18 improvements made for the next version 1.4.1 already, and yet I haven’t solved the uncommon crash bug. I’m thinking I’ll have to resort to dragnet methods.

Yesterday I thought, “You know, it would be easy to add a MacroLocation “center” so the bot could build proxies in the middle of the map. Why don’t I just do that?” It only took 10 minutes to hook all the pieces together, and a few more minutes to code up a quick version of center BBS to try it out with.

On the first try, I realized “Oh, 2 x barracks @ center doesn’t work because I never fixed that parsing bug.” You couldn’t have both a multiplier “2 x” and a macrolocation “@ center” at the same time, you had to say “barracks @ center” twice to get the same effect. So I went and fixed the bug, and now it works. That’s a good step.

On the second try, I noticed that the constructing SCVs were sent out from the base much too late. There is code that sends out builders early to try to start buildings as soon as possible, but it is not able to cope with buildings that are so far away. The result is that the barracks are constructed much later than they should be, which defeats the purpose of making them in the center of the map in the first place. The limitation does not seem easy to lift. The code to send builders early ties in with both the building manager and the worker manager, and it’s fragile; in the past, most things that touched it caused bugs. I need to figure out a more robust mechanism, throw out the old code and write it better. Or I could add a command to explicitly send builders early for proxy openings. Or something.

After a few more tries, I found an even worse problem. Where the center of the map is inaccessible, as on the map La Mancha, one or both of the barracks can be dropped. An SCV is assigned but it goes idle and never starts to build. Building placement is supposed to find the nearest clear area to build in, but in this case it apparently fails altogether, another bug.

Now you know why bots progress so slowly. Even an easy skill, only 10 minutes to code, can take days to debug. I’m not sure whether I’ll end up fixing the 2 problems any time soon. They definitely deserve to be fixed, but there are more serious bugs that deserve it more. In the meantime, “@ center” is available (well, it will be after the next release) but it is not useful.

In sort of related news, I ripped out the crufty old DistanceMap code inherited from UAlbertaBot and replaced it with shiny new DistanceMap code from UAlbertaBot—in a later version. When Dave Churchill removed BWTA he redid that part. The new version is much, much cleaner and was well worth borrowing. That portion of Steamhammer’s code will be easier to work with thanks to the improvement, and I loved being able to delete ugly stuff from MapTools. Though it was only a small refactoring, it took me a whole day to fix the bugs I introduced in making it. A valuable code improvement that is not even visible in the bot’s play can still take a lot of time.

Steamhammer got another mystery crash. It was assigned a crash loss with no crash record and no replay saved, while playing as zerg, the best-tested race, against Ecgberht at 2018-02-13 16:22:50 server time. There’s a longer than usual time gap between games, which might mean that the game went to late game and something rare went wrong, or might mean a server problem. Either way, with no information saved there’s nothing I can do about it except cry the news.

One of my age-old traditions, which has gone on for over a year now, is to announce my plans for upcoming versions before I change my mind. After I change my mind, I get to announce new plans. No wonder there are so many posts!

1.4.1 will be out as soon as I can fix the not-so-common crash that Antiga reported. Tracebacks are inconsistent, which points to memory corruption, which points to a bad pointer without admitting where it might be. Or it might be something else. So I’m not sure how long it will take. In between checks to catch the bug, I’m carrying on development of features. So far I’ve made only a couple small changes, including adding the new 21Nexus opening coded up by Antiga.

For 1.4.1 and/or 1.4.2 I want one of the opponent model features I mentioned earlier (see the last long paragraph of that post), plus progress on removing BWTA, plus progress in fixing at least one of the major weaknesses. For BWTA I will do the base information stuff like fetching the base’s geysers and so on. That will take me 90% of the way toward getting rid of BWTA, which by the standard rule of thumb means that the work will be 50% done. Having base info handy will make it easier to implement running workers away when a base is under attack, which is a major weakness, so that will be the other one. As always, I’m sure to do a lot of little fixes and improvements too.

I’m not sure how many more minor versions I’ll make, 1.4.3 and so on. However many there are, each of them will have one more important opponent model feature. I’ll keep it up until I find that the opponent model is Pretty Damn Good, whose definition is to be discovered empirically.

It’s also time to add new units and basic skills. It has been a while. Protoss will get shield batteries, which are already implemented almost correctly. Other top candidates are spider mine laying and tank micro for terran, and midgame drop and defilers for zerg. Drop and defilers are both complicated, so expect the first cut to be limited. I also want to give zerg overlords a sense of danger, because Steamhammer loses far too many of them, another major weakness. Before the 1.4.x series is over, I want to get shield batteries, at least one terran thing, and one or two zerg things. We’ll see how it goes.

Pretty vague plans, but this is what I’m thinking.

Steamhammer’s documentation is updated for version 1.4. I hope it will help people get started with the code and configuration.

Steamhammer has played another good game against PurpleWave. This seems to happen regularly. See Steamhammer-PurpleWave games for a few older games (which are not as good as this one).

I don’t want to do the blow-by-blow for this game, but here’s a little of the story. 2 games ago, PurpleWave played its worker rush against Steamhammer and won. 1 game ago, Steamhammer countered the worker rush and, partly because the map was more favorable, held it off to win. PurpleWave’s worker attacks are coordinated and Steamhammer’s worker defense is not, so PurpleWave can often win even when Steamhammer plays an ideal counter build. Anyway, having lost, PurpleWave switched to its forge expand opening. Strategically, it’s the opposite of a worker rush.

Steamhammer continued to counter the opening of the previous game, and played its anti-worker rush build. It’s a cautious opening that expects cheese, and it goes so far as to make a sunken colony. It’s the opposite of how you want to play against a big economy build like forge expand. So, after an early fight where Steamhammer didn’t quite figure out how to exploit PurpleWave’s poor cannon placement, zerg was behind.

I want to make 2 points about the game. 1. Steamhammer’s ZvP is much improved. When PurpleWave moved out with its big army, zerg was still behind. Steamhammer put up a fierce fight and eventually wore down the attack and held, losing some drones but not enough to die. And zerg continued to play well after that. Earlier versions won the big fights only when PurpleWave overextended, which did not happen here—thank economy improvements. And earlier versions did not follow up as nicely after the big fight—thank tech improvements.

Point 2. Near the end of the game, Steamhammer expanded to a mined-out base. :-( This version was supposed to fix the last way that that could happen. There is a bug in choosing the base to expand to. I am walking the Path of Bugs and the path continues over the horizon.

Steamhammer’s web page is updated. You can get source and binary distributions there. Documentation to follow when I finish updating it!

The change list for release version Steamhammer 1.4. For the opponent model, see the opponent model in Steamhammer 1.4. The rest is here.

These are the changes since the tournament version. For older changes since the last release:

• 1.4a1
• 1.4a2 (first intended tournament version)
• 1.4a3 (actual tournament version with 2 last-minute fixes)

Overall, this version has substantially improved strategy and macro. There are only tiny changes to micro, and tactics are a lingering disaster area, nobody knows when the power will be restored. It will need to absorb some losses while it learns for itself how to beat various rushbots and others that it used to beat by hand configuration.

debug info

• In the game info in the upper left, the gas steal intention and enemy plan are mentioned, and more time information is displayed. More details in how Steamhammer is getting on.

• In some debugging displays, “_” is changed to “ “ for readability.

bug fixes

• A building manager bug could reserve resources forever. Fixed. The bug turned out to be astonishingly frequent and sometimes locked away thousands of minerals by the late game. Fixing it brought a clear improvement to Steamhammer’s macro. There is at least one more building manager bug that can over- or under-reserve resources, so I put in a workaround that corrects the reserves at the next opportunity. The workaround prints a message when it triggers, so stream watchers get to see the bugginess of Steamhammer in real time. These fixes are the biggest improvement in this version after the opponent model.

• Fixed another bug in gas stealing. There is one known bug remaining. I’ve seen it happen more than one game in a row, but at the same time it is rare and I couldn’t reproduce it when I tried to debug it.

• In inferring the enemy zerg base’s location from a sighting of the first enemy overlord, take into account the offset of the overlord from the hatchery. Older versions assumed that the overlord started from the hatchery location, which caused occasional dangerous scouting misses.

• When trying to expand to a gas base, don’t choose a base where the geyser is exhausted. An earlier version had a similar change to avoid expanding to bases with no remaining minerals. I hope Steamhammer will finally stop expanding to mined-out bases.

• Workers transferring between bases might formerly carry minerals or gas. They were being sent to mine minerals, and gas-carrying workers would lose their gas when they arrived. Fixed.

other improvements

• ProductionManager has a limited ability to reorder the production queue to avoid production delays. When there is not enough gas to produce the current item yet, and another item within 4 spots later in the queue requires no gas and has no dependencies, and there are enough minerals to produce both the current item and the later item, then the later item is moved up and produced immediately. This slight improvement in macro has little effect on Steamhammer’s strength, but it keeps me from scowling every time I see it waiting for gas to start the lair when the following item is the second extractor. On rare occasions reordering can move a hatchery ahead of a batch of scourge, which might save the odd game. Future versions may have more powerful reordering skills.

• Enemy observers are a higher priority target. Many lurkers owe their lives to this change.

code changes

• Made sure to catch all exceptions by reference. I happened to notice that Microsoft recommended it.

• The config file is slightly reorganized. Whenever each race is represented, the order of races is terran-protoss-zerg for consistency.

changes affecting terran and protoss

• In many openings, I changed the scouting command to "go scout once around" to give the opponent model enough scouting information to work.

• Losing a supply depot or pylon is no longer reason to break out of the opening build order. More supply will be built automatically if needed anyway.

• Emergency response: If there is no command center/nexus, make one immediately. Steamhammer will try to stay alive.

• Emergency response: Urgent marines and zealots are made more cautiously. It could queue up too many. The change sometimes makes play worse by not queueing up enough of them, if many workers have been lost. I figure that if you lost a lot of workers, it probably doesn’t matter how you react.

• Strategy reaction: If the opponent fast expanded and is not zerg, take the natural immediately. This is a rough-and-ready response and sometimes wrong, but I thought it improved play most of the time, since otherwise Steamhammer is too slow about the first expansions (which in turn is because it is weak at defending and doesn’t know what is coming). If the opening build order already includes an expansion, Steamhammer ends up double expanding in response to the enemy. It’s usually OK.

• Strategy reaction: If the opponent has air-to-ground units (other than long-range units like guardians or carriers), make turrets or photon cannons for defense. The number made depends on the number and type of the air-to-ground units. This reaction happens so slowly that it is rarely useful, unfortunately. Mutalisks can easily wipe out everything before the first turret starts.

• Terran: If the opponent is rushing and we have marines, make a bunker. Steamhammer has a better chance to survive rushes. How well it works depends on the kind of rush and on where the bunker ends up being placed.

• Terran: In the tanks opening group, I fixed a typo that prevented vultures from being included in the unit mix. They were originally intended. It makes a big difference, partly because Steamhammer is klutzy with tanks.

• Terran: When upgrading at the engineering bay, eventually upgrade infantry weapons to level 3. Older versions stopped upgrading at 1-1. Now it goes 1-0, 1-1 (the science facility is often built late), then continues to 3-1.

changes affecting zerg

The changes to drone production and ZvP unit mix work together. They combine to make Steamhammer react better to large protoss armies. Its economy is stronger and its tech switches over time are more appropriate. Its ZvP is now held back by the big tactical issues like moving overlords unsafely, failing to run drones from attacks, and doing stupid stuff with the army.

• I rewrote StrategyBossZerg::chooseTechTarget() almost from scratch, overwriting bugs and “it works as intended, but...” misbehaviors. When I looked closely, I was surprised how many subtle bugs were hiding in the old code. Tech switches are now smoother and more stable, with fewer bizarre decisions.

• Automatically morph a completed creep colony into a sunken, unless we see an explicit sunken or spore colony coming up soon in the production queue. This fixes jerky macro that used to happen when making reactive sunken colonies. The strategy boss used to do things like queue a creep colony + a drone + a sunken colony to make a sunken. The drone might delay the sunken if income was slow, or waiting for the creep colony to finish might delay later production if income was fast. Now the sunken is made automatically right after the creep colony finishes and other production is not delayed. It’s an important improvement; emergency defenses are no longer needlessly expensive. A few opening builds have been rewritten to make creep colonies and allow them to auto-morph into sunkens, but I left most openings alone for now; they all still work without change. If you want to make creep colonies and wait until danger approaches to morph them (Arrakhammer does this), then you’ll need to do some recoding. In a future version I intend to attach intentions to creep colonies, to handle all cases more transparently.

• Steamhammer has a bug that makes it believe that 2 mutalisks can defeat a spore colony. The 2 flyers attack, then it loses 1 and the survivor retreats, then a fresh muta arrives, etc., and soon the game is over. Someday I’ll fix the bug, but for now I put in a cheap workaround. For each spore being sent to the combat simulator, leave out 5 of our mutalisks—any 5, that’s all. It’s surprisingly effective, an important improvement to ZvZ play.

• I changed the rules for limiting the number of scourge made in ZvZ, a tricky point. Formerly, Steamhammer made 4 mutalisks and then allowed itself as much scourge as needed to combat the enemy air force. It often skipped scourge it needed because it couldn’t keep mutas in the air, and then overbuilt scourge when it could, using up all the gas and delaying production of mineral units. I changed it to a simple rule: It’s allowed to add more scourge if the current number of scourge is less than the number of mutalisks. So it can make a batch of scourge after only 1 muta if needed, but while keeping that scourge can’t make more until it has matching mutalisks. It’s far from ideal, but it’s an improvement because it allows scourge early without pushing the essential mutalisks off the table.

• In an emergency, continue making drones at a low rate. Older versions stopped making drones entirely to concentrate on defense, and continued pressure could leave Steamhammer strategically trapped, unable to reach essential tech.

• Steamhammer has long compensated for enemy static defense by making more drones. Now it also compensates for its own sunken colonies by making more drones. The sunkens make it safer, so it can safely add more drones, and they are expensive, so it needs more. When the enemy attacks and Steamhammer makes emergency defenses, the pushback after the attack comes later, but it is stronger.

• In ZvZ, reduce drone compensation by half, because making too many drones is fatal. There was already a partial method for this, but I made it broader and simpler. I put in this change after watching a test game in which the zerg opponent turtled hard and Steamhammer made 18 compensatory drones and lost without defending itself.

• Steamhammer will drop a queued hatchery if it has lost drones and no longer has enough to support production at the hatchery. I increased the limit of how many drones are needed. This is a slightly risky change that could cause blunders in rare cases, but in test games I saw it improve play.

• Tech evaluation changes: In ZvT, Steamhammer has a clearer idea that lurkers suck versus factory units and hydralisks die to tanks. In ZvZ, get the queen’s nest later, since the hive is delayed; it was often getting a useless queen’s nest. In ZvP, favor hydralisks more depending on the protoss unit mix, and favor lurkers more if the hydralisk upgrades are done. The change makes Steamhammer more able to cope with a big protoss army later in the game.

• Some openings scout earlier and/or more thoroughly, to collect data for the opponent model.

• Added openings 2HatchLurkerAllIn (fun against terran) and ZvZ_12Hatch. ZvP_10Hatch is renamed to Over10Hatch and revised; see optimizing one opening build. 11Gas10PoolSpire is renamed to 11Gas10PoolMuta for consistency with other opening names, and shortened and slightly revised.

• Fixed a rare bug that could cause 3 evolution chambers to be built, though Steamhammer is only able to upgrade using 2. It happened in far fewer than 1% of games.

• A bunch of minor strategy tweaks: Try harder to make more units when we are short of them. At the same time, be more cautious about spending gas when under attack. Try not to let a macro hatchery delay essential tech (the lair finished, for heaven’s sake make the spire next!). When building a spire solely to later make it into a greater spire, wait for the hive to start first.

• A few minor micro tweaks: When we are under attack, cancel a morphing sunken if it would finish with under 30 hp (after the 100 hp subtraction when it completes). When the attack is thought to be over, the creep colony auto-morphs into a sunken to start healing from the highest possible base (see the weird life history of the sunken colony). Irradiated melee units burrow to spare their companions, if they can; ranged units already did this. Steamhammer doesn’t research burrow, so they usually can’t. Badly injured lurkers are a little more eager to stay underground and heal.

missing from this version

A new version of FAP is out. Steamhammer doesn’t use it yet.

I tried to get shield batteries to work for protoss, but I had trouble producing behavior good enough to be worth it. I’m not sure how a shield battery can be more difficult to use than a bunker; I thought it was the other way around. Anyway, it was taking too much time so I postponed it until a later version.

The building manager seems to place each building more than once. Placing buildings is expensive, so that’s bad. Also I think it may delay construction of some buildings by causing extra worker movement. It may be an error in the last-second “can I really build here?” test. Fixing that might help protoss avoid overstepping the time limit when the main base fills up, though it wouldn’t solve the problem.

Today is Steamhammer 1.4’s opponent model. The features are:

• Recognize some enemy opening plans.
• Predict the enemy’s opening plan from experience against this opponent.
• React to the enemy’s predicted or actual opening plan.
• Choose openings based on the predicted plan.
• Decide whether to steal gas. Does experience suggest it may be worth trying against this opponent?

The code that implements the features has these parts:

1. The plan recognizer, which looks at the current game situation.
2. Game records, which save information about past games against this opponent, including the recognized plan.
3. The plan predictor and gas steal decider, which draw conclusions based on the game records.
4. Strategy reactions to predicted or recognized enemy plans are in various places–they are uses of the opponent model, not parts of the opponent model.
5. Opening choices to counter the predicted plan can be set in the configuration file.

1. The plan recognizer was first written up in December. It tries to understand the opponent’s intentions during the earliest part of the game. The code is in OpponentPlan.cpp and is less than 150 lines—it is rudimentary.

• Unknown - No plan recognized. The plans are not exhaustive, so this is common.
• Proxy - Enemy buildings in your base. This doesn’t include cannon contains or other more distant proxies.
• WorkerRush - Like Stone.
• FastRush - Basic units faster than 9 pool, 8 rax, or 9 gateway.
• HeavyRush - 2 gate zealots, 2 barracks marines, etc.
• SafeExpand - Static defense before expanding to the natural. Zerg can’t do this.
• NakedExpand - Expansion with no static defense.
• Turtle - Static defense without expanding.

My early impression of the plan recognizer was that it often failed to recognize plans, but was rarely wrong when it did. With more experience, I think it often misrecognizes plans severely. It’s crude and clumsy. Even so, when it helps it helps a lot. It’s a net win.

2. Game records are handled in GameRecord.cpp. Steamhammer writes a file for each opponent, like many learning bots. Here is one record from a file named om_UAlbertaBot.txt, where “om” stands for opponent model, with annotations so you can make sense of the list of numbers.

1.4                  <- record format version (from the Steamhammer version when it was first used)
ZvRZ                 <- matchup
(2)Destination.scx   <- map
Over10Hatch          <- Steamhammer’s opening
Heavy rush           <- initial predicted enemy plan
Fast rush            <- actual enemy plan
1                    <- we won, 1 or 0
0                    <- frame we sent a scout to steal gas, 0 if never
0                    <- gas steal happened, 1 or 0 (extractor was/was not queued)
2110                 <- frame enemy first scouted our base
3102                 <- frame enemy got first combat units
0                    <- frame enemy got first air units
0                    <- frame enemy got static air defense
0                    <- frame enemy got mobile air defense
0                    <- frame enemy got cloaked units
0                    <- frame enemy got static detection
1726                 <- frame enemy got mobile detection
12309                <- last frame of the game
END GAME             <- end mark

Theoretically, a single file could have records in more than one format. Of course, only one format exists so far. The “frame enemy got” times recorded are the time we first saw such a thing, which may be much later than it actually happened. For example, here we first saw an enemy overlord (a mobile detector) on frame 1726, but it existed the whole game. The END GAME mark is redundant. It gives us a way to recover in case data in the middle of a file is corrupted—we can skip ahead past the next END GAME and continue reading the file from there.

3. The plan predictor and gas steal decider look at the game records near the start of the game.

The gas steal decider was written up in December. Nothing important has changed since then.

The plan predictor runs once at the start of the game. It looks through the game records to see what recognized plans the enemy has played. It is close to the bare minimum: It counts the recognized plans in the game records for this matchup, ignoring unknown plans, and weighting recent games more using a discount factor so that the past is gradually forgotten. That way it reacts quickly when the enemy changes its play. Whether the game was won or lost, whether past predictions were correct or wrong—all the other information is ignored.

If the opponent was random, then when the opponent’s race is found out, the plan predictor runs again. In the first run, it counted all game records where the opponent went random. In the second run, it counts only games where the opponent was the same race as this game. The predicted plan may change.

4. Strategy reactions could be written in anywhere, but so far they are in StrategyManager for terran and protoss, and in StrategyBossZerg for zerg. So far, all the reactions are reactions to the enemy plan, not to any of the other information (even though it has obvious uses). Some strategy reactions are made when the enemy plan is recognized. Some reactions must begin in time, and happen when the plan is predicted. For example, against UAlbertaBot, the initial predicted plan is “Heavy rush”. If Steamhammer finds out that UAlbertaBot is zerg, it knows that zerg follows a different plan. The predicted plan changes to “Fast rush” and efforts to stop the zerglings begin immediately (if Steamhammer is zerg or terran; the protoss reaction is turned off because I didn’t get it working).

Good strategy reactions are the hard part. I find them much more difficult than the opponent model proper. Some of Steamhammer’s reactions are weak.

5. Openings to counter specific enemy plans can be written into the configuration file in a new subsection CounterStrategies. The opening is chosen once at the start of the game and can’t be changed (in this version), so only the initial predicted enemy plan matters.

Steamhammer first looks for counter strategies specific to the enemy race. The name is in the format “Counter [plan name] v[race character]”. The race character is “U” for Unknown if the opponent went random. You can use all the usual features of random opening selection. As you can see in the example, zerg is configured with a wider range of choices.

"Counter Safe expand vT" :
	{
		"Terran" : "14CCTanks",
		"Protoss" : "13Nexus",
		"Zerg" : [
			{ "Weight" :  1, "Strategy" : "FastPool", "Weight2" : 5 },
			{ "Weight" :  9, "Strategy" : "9PoolSpeed" },
			{ "Weight" :  0, "Strategy" : "ZvT_3HatchMuta", "Weight2" : 50 },
			{ "Weight" : 50, "Strategy" : "ZvT_3HatchMutaExpo", "Weight2" : 0 },
			{ "Weight" : 30, "Strategy" : "3HatchLurker" },
			{ "Weight" : 10, "Strategy" : "3HatchPoolMuta", "Weight2" : 0 }
		]
	},

If no counter strategy is found for the specific enemy race, Steamhammer next looks to see if there’s a general one for all races—leave out the “vX” string. This example points to a reusable strategy combo that specifies openings for each race Steamhammer might be playing, no matter the enemy race. The strategy combo feature has not changed.

"Counter Worker rush" : "AntiFastCheese",

If no counter strategy is found, Steamhammer falls back on its usual random opening selection. So if the regular repertoire is best in any given case, don’t specify a counter for that case.

For the following version Steamhammer 1.4.1, I will add at least one large piece to the opponent model. I'm likely to change my mind once I see how this version does in practice, but at the moment I'm thinking of fine-tuning plan prediction and opening selection based on wins and losses. That will raise Steamhammer's performance ceiling; with enough games, it will learn much more. Other possibilities include: • Make use of more of the information in the game records. This opponent doesn't get detection, use cloaked units; that opponent gets air units around frame X, add a spire for scourge just in time. • Have Steamhammer collect data on its own openings so it can generalize: Play a fast/slow opening; play a lurker/muta opening. • Revamp the plan recognizer with a richer set of plans, maybe a hierarchy so it can refine its conclusions over time. • Machine learning for a probabilistic plan recognizer and/or plan predictor. • Restore the originally planned extensive game records, making it possible to predict unit mixes over time throughout the game. • Add the ability to change openings during play instead of deciding ahead of time, so that decisions can be made as late as possible. • Move scout timing into the opponent model alongside the gas steal decider. • Have expansion decisions or hatchery placement influenced by the opponent model. “Hmm, historically in situations like this we do poorly if the third hatchery is at an expansion. Make it a macro hatchery instead.”

I have no shortage of ideas!

Next: Brief remarks on the SSCAIT round of 8.

Steamhammer 1.4 is more or less finished. I keep thinking I should package it up and be ready to go, and then I come up with another easy way to fix a weakness. I may yet fix a few more weaknesses, but progress so far is good. I’ve been running a lot of tests.

The simplest opponent for Steamhammer is Stone. Steamhammer now defeats Stone, from most openings and on most maps, without any learning needed, because its reactions to worker rushes are improved. (It loses if it plays 4 or 5 pool, because it doesn’t have enough drones to fight with.) I think a more aggressive worker rush will beat it, because Steamhammer’s micro is uncoordinated. But if you play a worker rush in the first game, win or lose, then Steamhammer will counter it in the second game.

The next are zerg rushbots and others that play fixed strategies that the plan recognizer is able to recognize. Steamhammer may lose the first game, or the first few—though maybe not, because its rush reactions are improved. As soon as it scouts well enough to recognize the rush (it’s not too reliable), the next game it switches to a counter and almost always wins from then on. It works against ZZZKBot, it works against Neo Edmund Zerg, it works against UAlbertaBot set to play zerg, it works against 2 different versions of the old bot 5 Pool that I have saved. I think it just works.

The next step up is UAlbertaBot without learning, which plays random and has a fixed strategy for each race. Steamhammer classifies UAlbertaBot’s zerg opening as “Fast rush” and its terran and protoss openings as “Heavy rush”, so once it has collected enough data it always plays to counter the heavy rush. It wins over 90% versus terran and protoss. And because of the improvements to rush defense, plus the fact that the opponent model knows to expect the fast rush as soon as it finds out that UAlbertaBot rolled zerg, it has a chance to hold the zerg too. In the end, the learned openings scored better than the tournament version with its hand-coded counter opening.

Learning opponents are more difficult. I played test matches against the opening learning bots Microwave and Zia. Microwave has the edge over the tournament Steamhammer. Against the new Steamhammer, the opponent model prevented Microwave from finding a stable way to win. When Microwave won a game, it tended to repeat the opening the next game, while Steamhammer tended to counter Microwave’s opening of the previous game. The match came out about even, with Microwave constantly switching its openings, unable to find one that worked consistently. Zia in contrast is weaker than the tournament Steamhammer, but the new Steamhammer was not able to come to grips with it. Zia plays a variety of different openings that look the same to Steamhammer’s plan recognizer, plus it rarely repeated an opening after winning only once, so Steamhammer was unable to choose good counters. Zia is the only opponent I tested that did better against the opponent model than against the random opening selection.

In one other test, I played a match of the new Steamhammer 1.4 versus the tournament version 1.4a3, both playing random. An opponent with random race and random choice of openings should be the worst realistic case for the opponent model opening selection (only a knowledgeable adversary that can predict Steamhammer’s predictions is worse). Also, since Steamhammer is still on BWAPI 4.1.2, the bot does not know when it starts up that it went random itself. The opponent model has no choice but to take separate statistics for each matchup, so when Steamhammer plays random it learns more slowly. The new version scored approximately 2:1. Even with tons of wrong predictions from the opponent model, the new version plays better.

These are only some of the tests I’ve run. There are opponents that the opponent model provides no leverage over. The plan recognizer has a limited repertoire, and if you step outside it can’t help. In the configuration file, I removed most but not all of the hand-selected enemy-specific openings. Removing the rest will be for a later version in the 1.4.x series.

Overall, the upcoming Steamhammer 1.4 is much improved over the tournament version. It is better in all matchups, with smoother macro and several small but useful skills for terran and protoss. It is especially improved in ZvZ and ZvP. It hasn’t caught up with McRave, but it should land upsets more often. I’m eager to see how it does in the wide world of the SSCAIT ladder. Are there more opponents like Zia that will leave it confused?

Next: How it works.

Yesterday I revised the ZvP_10Hatch opening. It was originally designed to counter 2 gate zealot rushes, and now (with the opponent model) it is used to counter expected heavy rushes by terran and random opponents too. I renamed it Over10Hatch. The basic build order is extractor trick for a 10th drone, overlord, hatchery, and a couple of sunkens to help hold off the mass zealot pressure so it’s safe to make drones and tech.

The changes are small. If you watch casually, you might not notice. The extractor trick is now before the overlord, not after, a careless oversight in the original. The first sunken is delayed slightly, and the second sunken needed to deter masses of zealots is delayed longer. The number of early zerglings is cut back to just the number needed in the worst case. The early game plays out much as before, it only looks as though Steamhammer might be cutting it a little close. (CherryPi’s versus-protoss opening often gives a similar impression. The team knows what it is doing.) You could miss that there is a slightly higher drone count.

A pro would cut it closer, but a pro has good judgment. The way the rest of the game plays out looks different. In the old build, Steamhammer would often narrowly hold the zealots, get lurkers just in time to survive, and use the opponent’s lack of tech to slowly push for victory. With the extra income from the new build, Steamhammer holds the zealot pressure easily, safely gets lurkers, and quickly smashes the protoss with a mass of lurkers and lings. The turnaround in the game looks completely different. The effect versus no-academy marine rushes like UAlbertaBot’s is similar.

I thought it was a good lesson in the importance of an efficient build.

backward games

I added a zerg-versus-zerg 12 hatch opening to Steamhammer. It’s the greediest common ZvZ opening, so Steamhammer only plays it when the opponent model expects that the opponent will also play a hatchery-first opening. But look what happened: In a test game today against Microwave, Steamhammer opened 12 hatch and Microwave played its 5 pool opening. Steamhammer saw it coming only after the spawning pool was started; all it could do was start a sunken as soon as possible. Microwave’s zerglings tore down the natural hatchery first, and that was enough time for the sunken to finish and 4 zerglings to hatch, just enough to hold the rush. Steamhammer had a huge drone advantage, went up to 3 hatcheries, and smashed down Microwave by brute force while fighting right in the face of Microwave’s emergency defensive sunken. That’s the way to play!

It’s no wonder that pros, with their much stronger defensive skills, are so willing to play a greedy opening like 12 hatch in ZvZ.

Microwave’s 5 pool, by the way, is a clever build. Microwave builds a second hatchery at its natural to keep the zergling numbers up, and if the pressure doesn’t work out it drones up and switches to mutalisks. The opponent has to keep playing well to stay ahead. In a different test game, Steamhammer predicted the 5 pool and played an anti-rush opening which held the zerglings easily—then Steamhammer lost despite its advantage because it didn’t tech fast enough: The lair finished, and it decided to build a macro hatchery before the spire; the spire finished and it decided to build a third base before any mutalisks.... That’s not the way to play!

Steamhammer 1.4 is frozen

Steamhammer is feature-frozen for the 1.4 release. I will keep on fixing bugs and minor weaknesses (lair finished...) and tweaking the configuration, but I won’t start anything bigger that might delay the release, though there are plenty of temptations. I should be able to upload within a day after SSCAIT reopens for submissions, and release after another day or so once I can see that I didn’t break anything obvious.

Steamhammer 1.4 is substantially stronger than the tournament version in some ways. I think it is especially improved at transitioning out of the opening in ZvP. It will start off slowly on SSCAIT, because it will have to learn for itself how to beat a number of opponents that it currently beats by hand configuration. Also the opponent model has weaknesses and blind spots that are sure to cause some surprises. Still, if I didn’t mess anything up, Steamhammer should at least keep up with the competition and maintain a place near the top. Its play continues to grow more complex and interesting.

the game info display

As an example of a minor change, I have updated the game info display that Steamhammer can draw in the upper left corner.

If we have decided to steal gas, whether in the opening build order or in the opponent model’s auto gas steal decider, the strategy line adds + steal gas.

Opp Plan is the opponent’s plan. It is often Unknown. If the opponent model has predicted the opponent’s plan and the plan recognizer hasn’t verified it yet, we get the orange word expect. The opponent model tends to be overconfident about what it expects. When the plan recognizer thinks it knows what is going on, expect disappears and the recognized plan shows. The plan recognizer also relies on inadequate information, but at least it looks first. Steamhammer uses both the expected and the recognized plans, to a limited degree, for different purposes and with different levels of confidence.

The Time: line gives the game time in frames and in minutes:seconds. The minutes:seconds display is both more compact and easier to read than the original XXm YYs display. Then come the mean and maximum time spent per frame, in milliseconds, so we can start to get an idea of when things are slow.

CherryPi has a striking habit of barely scouting before concluding that it knows what the opponent is going to do, and then seeming reluctant to ever change its mind. A pro player will often also barely scout, but pros probe for new information later in the game and are ready to draw new conclusions. I thought the CherryPi-TyrProtoss match from the SSCAIT 2017 round of 16 was a tantalizing example (see the video). It doesn’t tell us how CherryPi works, but it offers a small hint.

In the first game Tyr opened with 2 gateways for early pressure. CherryPi scouted the 2 gateways with a drone at about 1:50 into the game and reacted somewhat logically with an array of sunkens, behind which it built up a strong economy. Tyr saw the sunkens and did not seem to react to them at all, strategically. Protoss should have expanded immediately and taken steps to prevent zerg from expanding further. Tyr did neither, and it lost.

Seeing the opponent’s build and reacting is nice, but it’s not a special skill. In this game, CherryPi did well but didn’t show anything special in terms of strategy.

In the second game we saw something that might be more interesting, but it’s still unclear. At about 2:10 the scouting drone saw a forward pylon and immediately returned home without looking further. When I first watched the replay in OpenBW I thought that CherryPi had left before seeing the forge warping in behind the drone, but when I watched the replay in StarCraft with only CherryPi’s vision turned on I saw that it did just catch the forge starting. Still, the fact that the drone turned around immediately suggests that CherryPi had seen enough; the forward pylon was all it wanted to see to understand the opponent’s build.

CherryPi reacted with 3 hatcheries before pool, which is safe versus forge expand but allows good responses from protoss too. I imagine the CherryPi team chose the build because it’s unusual in that situation and a protoss bot might not know a good reaction (though any human player would have an idea). In this case they were right, and CherryPi got ahead and won.

What do you think happened? Did CherryPi see the forge and know there couldn’t be a gateway yet, so it could safely play a slow build? Or did CherryPi take a leap in the dark and make its decision after seeing only the pylon? Nothing stops protoss from building 2 gateways at the forward pylon and making an aggressive rush—a pro is more likely to do that than to build the 2 gateways in the main. The advantages are that the rush distance is shorter and it protects the natural.

opponent modeling skills

I don’t know what CherryPi is doing. Maybe there’s discussion about it somewhere, which I haven’t seen. But I can’t help comparing it to Steamhammer’s opponent model. When all the intended features of the opponent model are implemented, Steamhammer will be able to see the pylon and immediately conclude “I’ve seen you do that before, it was the start of a forge expand opening and you’re probably playing it again. Let’s counter the forge expand.” I suspect that may be what CherryPi did—probably not in exactly the same way, but maybe in a way that’s broadly similar.

Bots tend to be predictable, and their opponents can take advantage. It’s one of the ideas behind Steamhammer’s opponent model. Seeing a forward pylon narrows down what the opponent is doing, but doesn’t zero in on one possibility. But if the opponent tends to continue the same way as in the past, you can act as though there were only one possibility and start to counter it earlier, gaining an advantage. (Against a forge you make drones, against gateways you make zerglings and a sunken. If you make unnecessary zerglings, you set yourself back.)

The development Steamhammer version can already do this, in a limited way against a random opponent. For example, against UAlbertaBot, Steamhammer says “this is probably a heavy rush (with zealots or marines), so I’ll prepare for that.” If it finds out that UAlbertaBot rolled zerg, it immediately (well, within 8 frames) realizes “uh oh, I was wrong, it is going to be a fast rush which has a different counter.” It doesn’t wait to see early zerglings, or a spawning pool, or a drone count, it immediately starts to adapt its build. The exact reaction depends on the timing, but there is code that says, for example, to cancel the second hatchery if it will allow a spawning pool to get up faster. By reacting immediately, Steamhammer has a better chance to survive despite its weak defensive skills.

The opponent can thwart the opponent model, at least to an extent, by being genuinely unpredictable. That’s a countermeasure. Steamhammer’s opponent model should force top bots to vary their play more. That’s another idea behind the opponent model.

I’ve been testing out Steamhammer’s new opening selection algorithm, part of the opponent model.

CasiaBot plays a hand-made anti-Steamhammer build, sunken turtle into fast spire. (It does this 80% of the time, and 9 pool speed the other 20%. It’s hard to counter both at once.) Steamhammer’s strategy reactions to it are in the right direction, but are too slow and sloppy. For SSCAIT 2017, I hand-made an anti-CasiaBot build that holds off the mutalisks with a slightly faster spire and gets a second gas to win with numbers, while preventing CasiaBot from ever expanding. CasiaBot’s build is hard for Steamhammer to beat, much tougher than CherryPi’s similar turtle build. But the counter usually works, and Steamhammer scored 2-0 in the round robin.

CasiaBot was a simple first test. In the first game, Steamhammer made its usual random opening selection and lost to the turtle build. But the plan recognizer recognized the build. In the second game, Steamhammer played its counter and won. The system worked.

I didn’t test against CherryPi, but I would expect this sequence as the most likely: Steamhammer wins the first game, because its random opening selection usually beats CherryPi’s first ZvZ opening. CherryPi switches to the turtle build for the second game and evens it at 1-1. (That’s what happened in the round robin.) Steamhammer switches to the counter-turtle build and pulls ahead again to 2-1. I don’t know what happens after that. CherryPi might have another winning build and even it again at 2-2, which would set up a learning race.

UAlbertaBot is a much tougher test because it plays random, and Steamhammer doesn’t have the smarts to fully adapt to each of the strategies UAlbertaBot follows. My hand-made counter scores about 2/3, almost always winning against zerg, mostly winning against terran, and usually losing against the protoss zealot rush.

I turned off the hand-made answer and let Steamhammer learn. In the first game, UAlbertaBot rolled terran and lost to Steamhammer’s random choice. In the second game, UAlbertaBot was protoss and Steamhammer played a counter to the terran opening it had seen: Steamhammer doesn’t need to lose to learn. Stopping the marines and stopping the zealots can be done in a similar way, and Steamhammer won again. Then UAlbertaBot rolled zerg twice in a row, and Steamhammer mis-countered both times. In game 5 UAlbertaBot was protoss again, while Steamhammer (having been hit with 5 pool twice in a row) thought its opponent had switched to fast rushes and played to stop the early zerglings, losing again. But at the end of 15 games the score was 9-6, not distinguishable from the 2/3 winning rate of my hand-made counter. The openings played were completely different, most wins were against protoss and terran instead of zerg and terran, and the reasoning behind the choices was the practical “this is what I saw, now beat it” instead of the theoretical “this is what ought to work” of my hand-made counter, but the end result was the same. Good enough!

Only longer experience will show how well the system works in the wider world of the SSCAIT ladder, and whether it can hold up when bot authors look for and exploit its weaknesses. I have thought of a lot of ways it might break down. I know it doesn’t work against Juno by Yuanheng Zhu. But I have also thought of a lot of ways to improve it. I have ideas in mind for the plan recognizer, the opening selector itself, and the reactions to recognized plans. A lot of information is not exploited yet. Some ideas I will get to before release, some will wait for later versions in the 1.4.x series, and some will wait a long time.

Near the 1.4 release I’ll write a description of how it works, up to date with the release. There are several working parts, but for what it does, it’s not complex. It acts for all 3 races, retains configurability, allows for random choice of openings in every situation if the author wants, and expects that some opponents will change their behavior over time. I imagine that getting all these features is simpler than you expect.