Starcraft AI blog

Question for stream watchers: Would you like some debug drawing turned on? Steamhammer traditionally runs showing only the game info and production queue. Maybe it would be fun to see more? Possibilities: Unit orders (what each unit is doing all the time—including what mineral patches are doing, namely nothing!), squad info (each squad’s order, target location, and retreat status), zerg strategy boss info (basics of zerg strategy decisions). I don’t want to turn on more than one of these at a time, but maybe I could change the settings now and again.

It seems that SSCAIT is down, so maybe I’ll upload the new Steamhammer there tomorrow.

In the meantime, here is Steamhammer 1.4.5, the CIG version. This is the same file I uploaded to CIG, not even renamed. It includes source and binary and the configuration file. Unlike an SSCAIT upload, it does not include BWAPI. Following CIG rules, the ability to configure the game speed and frame skip is disabled—the configuration options are ignored. This and a few other minor details will be changed in version 1.4.6 for SSCAIT.

CIG 2018’s info is updated with the latest. The “was this bot submitted?” column in the table of registered entrants is on the far right.

maps

CIG 2018 has cranked the random number generator and chosen their 5 maps from the pool of 20:

• (2)Destination
• (3)Tau Cross
• (3)Great Barrier Reef
• (4)Andromeda
• (4)Python

None of the difficult maps was chosen. 5 maps are too few to even out any imbalances that might exist, but except for that it seems like a fine map pool. The maps have some special features: Destination has a mineral-blocked back door to each main. Great Barrier Reef has mineral line blocks around the edge of the map. Andromeda and Python have island bases. None of the special features causes severe problems for bots which are unprepared (Destination’s mineral block is the worst), and bots which are prepared can show off their skills and gain an advantage.

race balance

I count 16 submisions and 11 old bots carried over from last year, for 27 participants. For the submissions:

terran	3
protoss	6
zerg	6
random	1
total	16

The total:

terran	8
protoss	7
zerg	10
random	2
total	27

Zerg and protoss are equally popular this year, and terran is in a slump. Among the bots carried over from last year, protoss was the unpopular race, so the overall numbers are not too uneven, with a bit of a bump for zerg. All in all, a pretty good range of challenges for everyone.

unknowns and favorites

The unknowns are terran TitanIron, protoss ISAMind, and zerg Stormbreaker. ISAMind and Stormbreaker are both “independent” affiliation, lone wolves, and a quick look did not turn up more about them. TitanIron I can guess more about: It is from the Chinese Academy of Sciences, the same larger organization under which Casiabot and NLPRBot were made, and it is credited to a large team that includes some of the same authors. It looks as though serious effort and expertise went into it. To generalize, the past bots from CAS were Steamhammer forks with either ambitious and not too successful, or quick-and-dirty but strong changes—none had staying power, but some placed high in the tournament they were tuned for. Playing terran and having a name that includes “iron” suggests that TitanIron may be a fork of Iron. Putting together the whole picture, I think TitanIron has the shape of a strong contender.

The familiar favorites are PurpleWave which we know from experience does better in tournaments that it was put in shape for, McRave which has been maturing in secret, new Locutus which is strong but which other bots may have prepared specifically against, and the perennials Tscmoo and ZZZKBot, which can never be counted out. You might think you can count out ZZZKBot because this year everyone will be ready for its rush, but so far it always has something new and successful.

CIG submission is closed. I received no acknowledgement other than “upload successful” with the second browser I tried, so I don’t have full confidence in the process. Anyway, here is the change list for Steamhammer 1.4.5, the CIG 2018 version. Instead of laying foundations as I have been doing in recent releases, this time I concentrated on making the biggest play improvements I could in a short time. Steamhammer should play visibly better, though I can’t judge how much. The most important changes are in bold.

As I mentioned yesterday, the CIG version is not quite right for SSCAIT. I need to undo small changes that uphold unhelpful CIG rules, and to test terran and protoss. Expect version 1.4.6 tomorrow or thereabouts.

overlord behavior

• Release the scout overlord when the enemy is seen to have anti-air. Against terran, almost any enemy unit or building implies anti-air tech, so the overlord tends to be released early. Normally, an overlord released early in the game leaves the enemy base; later in the game, it may not.

• Always send out a scout overlord. The exception of sending no overlord against terran is dropped, because the overlord is now safe enough. A terran base may be found sooner, and the response to a terran worker rush is improved because the overlord hangs around the enemy base.

• Overlords not assigned other duty now go into an Overlord squad. The overlord squad furnishes 2 easy skills: 1. It assigns 1 overlord to each base, so that Steamhammer stays ready for cloaked units. 2. If the enemy has flying overlord hunters, and Steamhammer has spore colonies, gather free overlords at the spores. The skills are crude; if neither condition applies, the overlord is left where it is, even if it is under attack from the ground—and even if it is a released scout overlord in the enemy base. Nevertheless, it’s a clear improvement, because Steamhammer used to have no overlord survival skills whatever. Better would be to retreat from danger and seek out safe places to watch from.

• If the enemy has no cloaked units, and the enemy does have anti-air, then assign no overlord to the Ground, Flying, or Recon squads. Missing the overlord is unfortunate, because the overlord provides a wider vision range and can see uphill, but keeping overlords safe has priority. The change saves overlords from being shot down by marines or dragoons. The game commonly proceeds: Overlords are assigned; enemy gets anti-air so overlords are released, and may be shot down but at least no further overlords are sent to die; enemy gets cloaked units so overlords are assigned again, because now it is worth the risk.

• Assign a detector to each defense squad, under the same conditions of cloaked enemies and anti-air. Defense squads used to simply hope that overlords were hanging around the base in detection range of intruding dark templar, and the hope was often dashed.

other tactical changes

• Predict enemy movement. When moving to attack a specific enemy unit, most units now move toward a predicted enemy position 8 frames in the future, rather than the enemy’s current position. Mutalisks and vultures don’t use prediction, because they are controlled by special-case code that works differently. The main purpose may sound surprising: To get rid of the enemy scouting worker! Using prediction, even a slow zergling can catch and kill an evading worker. I think Arrakhammer was the first zerg to demonstrate the skill, and Stone may have been the earliest bot to predict enemy unit movement. Besides more effective chases, unit micro is improved in general, because units come into firing range sooner.

• Squads defend and retreat toward the starting base, if we still own it, not the ever-changing “main” base where we add buildings. This change prevents one major cause of retreating through the enemy formation. There are other causes, so it still happens.

• When a unit in retreat finds itself blocked by an enemy (according to a simple geometry calculation), instead of continuing the retreat it stops to fight on the spot. I think this is probably a net win because it tends to delay the enemy advance, but sometimes units that could slip by safely instead stop to fight and are lost unnecessarily.

• When regrouping, air units stick closer together. Historically, regrouping treated air units and ground units the same. But air units stack, and often gain an advantage by being stacked, so now they get a special case. This modestly improves mutalisk play (on average) by grouping them tightly when they reach their retreat point.

• Defend is a regroupable order, or in other words, a defense squad retreats when the enemy is superior. I’m not sure whether this is a net win by itself; I didn’t have time to add the smarts that would make it a definite gain. It prevents the severe mistake of making a last stand outside of static defense range, but causes the severe mistake of fleeing from a stronger enemy when it should engage to delay as long as possible.

• Static defense is never considered 100% adequate; when under attack, Steamhammer always assigns at least 2 mobile defenders. Formerly, Steamhammer would calculate “eh, the sunkens will hold.” It was usually right, but that did not help when the enemy maneuvered to harass while staying out of sunken range. Iron does that, for example.

• If an enemy attacking a base has no anti-air, prefer to assign flying defenders. Mutalisks are better than zerglings against vultures, but also against tanks, zealots, and firebats.

• Because I was doing a bunch of work on squads, I clarified the hard-to-read squad info display for debugging. Map coordinates are converted to tiles rather than shown in pixels, which makes them shorter and easier to interpret. The Base Defense squads are renamed to Base squads for brevity. Most importantly, each squad shows its attack/retreat status at the top of the screen on its line of the squad info display, rather than at the bottom of the screen overlapping with the status of other squads.

player methods, not type methods

UAlbertaBot always got unit ability information from the unit’s type, and I copied that in Steamhammer when adding code. I was slow to realize the advantage of getting the information from the player object, which knows whether the unit has been upgraded.

After these changes, I can see the difference in hydralisk play, but I am not sure how important it is in practice. There was no time for a long test to find out.

• Changed unittype->topSpeed() to player->topSpeed(unittype) in several cases. The biggest effect is on kiting of hydras (since hydras have a speed upgrade). There is also a small effect on the decision of whether to chase after an enemy unit. Vultures have a speed upgrade but are not affected, because they are kited using the same special code as mutalisks, and that code works differently.

• UnitUtil::GetAttackRange() uses attacker->getPlayer()->weaponMaxRange(weapon), which simplifies the code and is probably accurate in more cases.

• Micro::KiteTarget() also uses unit->getPlayer()->weaponMaxRange(weapon) to find weapon ranges for both the kiting unit and its target. It affects hydra and dragoon kiting, because those units have range upgrades. It also affects whether kiting happens, which depends on whether the target unit has a longer range.

other stuff

• MacroLocation::Front added to represent “the front of the base,” the spot in either the main or natural where defenses should go. The calculation is done in a simplified way. If there is no natural and the main is the front, then because Steamhammer doesn’t understand chokes yet, the front is in the direction of the natural, which is often (but not always) close to the direction of the ramp to the natural. If the front is the natural, then for the same reason the front is opposite the minerals. See Bases::frontPoint() for the easy calculations. Passing units disturb building placement, and units often pass the front, so unexpected placements are possible.

In most cases, creep colony @ front is a much better place to put defenses. Sunkens are usually set close together instead of spread around the hatchery, and on most maps tend to be somewhat toward the natural choke and somewhat toward the ramp, defending both. There are times when Steamhammer blocks its own ramp with sunkens, though. You can get the old behavior with creep colony @ macro or creep colony @ natural.

MacroLocation::Front is used in most openings and in strategy reactions in the zerg strategy boss. It is not used for placing spore colonies, which should typically not be all the way in front.

• There was a serious bug in PlayerSnapshot::takeSelf(), which counts our units by type: All units had count zero. It worked when I wrote it, but suffered an editing slip later. It formerly wasn’t used, so the bug didn’t affect play, but now it is used.

• Recognize a cybernetics core as anti-air tech only when it completes. This is a tweak to convince the scout overlord to stick around an enemy protoss base until the cyber core finishes, so the overlord watches as long as it can without putting itself in danger.

• Added a new method InformationManager::enemyCloakedUnitsSeen(), which recognizes a different level of danger than enemyHasCloakTech() or enemyHasMobileCloakTech(). It’s used to make overlord decisions.

• The cloak tech recognizers now recognize cloaked units that they can’t identify by type because they are not detected. What an odd oversight.

• Certain air tech and cloak tech recognizers cross-set each other’s flags when appropriate: Yes, if you have overlord hunters, then you necessarily have air tech. This is a speedup optimization that does not affect play.

• The resource box is fixed. I changed BuildingPlacer::computeResourceBox(), which computed for the starting base only a box around the minerals where buildings should not be placed, to reserveSpaceNearResources() which marks as reserved those tiles at any base which should not be built on to avoid mineral or gas mining problems. Since sunkens are now normally placed @ front instead of surrounding the hatchery, the effect is not big. But spores are still placed around the hatchery, and this prevents spores from messing up mining.

• MicroRanged understands that dark swarm does not protect buildings. Units mistakenly believed they could not attack a building under swarm. Oops.

zerg strategy and reactions

• If too far behind in army size, make more army. Marian Devecka called out army size as a major weakness (too little army due to making too many drones), and Proxy suggested an easy way to repair it. I knew it was a weakness, but I had resisted making changes to the drone/army balance because it is difficult to do right. This way is not right; it is rough and will go wrong in common cases. Besides being crude, it is probably not tuned well due to lack of testing. But it is still much better than what Steamhammer did before. Play against protoss especially is clearly improved, though Steamhammer sometimes expands too little instead of too much.

• When rebuilding drones to recover from a severe setback, mix in zerglings if the enemy army is stronger. It should improve the odds to survive some major emergencies, especially in ZvZ.

• Be slower to make macro hatcheries, and place fewer of them at expansions. This was a critical weakness that made it difficult to survive heavy rushes, even given other adaptations. Steamhammer is now less likely to overbuild hatcheries, and more likely to float minerals. It can’t balance correctly without prediction skills that it does not have, but now it balances closer to correctly on average. It’s a net gain.

• Steamhammer classically made up to 4 defensive sunkens when necessary. I bumped the limit up to 5. It helped more than I expected.

• Be a little slower to expand solely to get more gas.

• Spore colony placement is a little smarter. Since it first gained the feature over a year ago, Steamhammer has always made only 1 spore colony, at whichever base is currently designated as the place to add buildings. Now it makes 1 spore when it sees a threat, and adds a second spore if the threat is high. Versus protoss, the first spore goes at the natural in case it is needed to help defend against dark templar, and the second spore goes at the current “main” (which may not be the starting base). Otherwise the placement is the other way around: Main then natural. There are cases where it builds the 2 spores at the same base. The spore colony change works together with the overlord changes to improve both overlord safety and dark templar defense.

protoss

• Added a one-base CorsairZealot opening for protoss, which I used to test zerg overlord reactions. The opening turned out to be strong against Steamhammer’s zerg, so I configured it to be played.

code changes

• Fixed a latent bug in CombatCommander::updateIdleSquad(). In aviation safety terms, this is a major hazard, which is defined as “you’re still OK as long as nothing else goes wrong.” Make programs as safe as flying!

• Various stuff related to bases moved from InformationManager to Bases. The process is still underway.

• Deleted the unused methods InformationManager::getIndex() and nearbyForceHasCloaked().

• Clearing of neutral “blocker” buildings was written for the map Sparkle. It turned out to have a bug on the map Pathfinder. At the natural of the top base, one of the neutral eggs was recognized as a blocker and destroyed—not ideal, but not much of a problem either. The bug is that Steamhammer failed to recognize that the egg had been destroyed, and units were permanently assigned to keep trying to destroy it. Instead of fixing the bug, I turned off the tactical reaction to blockers. It was easier and has bad effects only on rare maps like Sparkle.

Steamhammer 1.4.5 is submitted to CIG. Because of an obscure CIG rule, the version for SSCAIT will be slightly different (the rules curiously require Steamhammer to comment out a couple of minor configuration features, which is not how I want to distribute it). But in any case, it’s done. Working right up to the deadline (around midday tomorrow in my time zone) is an algorithm for causing more problems than it solves.

CIG rules also say that they will choose 5 maps pseudo-randomly (one 2-player, two 3-player, two 4-player) from a pool of 20. Since starting out on CIG work, I’ve been playing test games exclusively on CIG maps. The SSCAIT maps have been feeling stale, so it was a welcome break. Here are my thoughts on the CIG map pool.

7 maps are familiar from SSCAIT. SSCAIT doesn’t include version numbers in its map names (except La Mancha for some reason), so I’m not sure which of these are the exact same version. In any case, they are all close.

• (2)Destination1.1.scx
• (2)NeoHeartbreakerRidge.scx (Heartbreak Ridge)
• (3)TauCross1.1.scx
• (4)Andromeda1.0.scx
• (4)CircuitBreakers1.0.scx
• (4)FightingSpirit1.3.scx
• (4)Python1.3.scx

9 maps are different from SSCAIT maps but provide no special features that challenge bots (at least not that I noticed). I enjoyed all these maps. My favorites are Chupung Ryeong and Arcadia II.

• (2)MatchPoint1.3.scx. Sometimes compared to Benzine, because the main bases and expansions are laid out similarly. But the middle is not nearly so open.

• (2)NeoChupungRyeong2.1.scx. Multiple narrow paths between bases. Played properly, I think the map leads to complicated and interesting games.

• (2)RideofValkyries1.0.scx. The main bases are across from each other, relatively close with the naturals pointing toward each other, and expansions beyond the natural are away from the direct path.

• (3)GreatBarrierReef1.0.scx (originally named El Niño, then revised). The map has blocking mineral lines in between bases. Humans can push units through the minerals, or drop behind them if you dare to mine them. Since the blocking mineral paths run around the edge of the map, all bots that I have tried ignore them, and they don’t cause a problem.

• (3)NeoAztec2.1.scx. Low ground main and high ground natural (compare Jade). Bases beyond the natural are a little farther away than on some maps.

• (3)Pathfinder1.0.scx. The looping layout of the bases seems intuitive when I look at the picture, but for some reason in following games I can’t get my head around it. Bots ought to pay more attention to attacking around the back way.

• (4)ArcadiaII2.02.scx. The natural base and mineral-only are near each other, both inside the natural entrance. Compare Andromeda, where the mineral-only is above the ramp but can be attacked from outside.

• (4)LunaTheFinal2.3.scx. A classic macro map, sometimes accused of making the game boring.

• (4)NeoSniperRidge2.0.scx. Ridges all through the center of the map, somewhat like Heartbreak Ridge. Bots don’t really understand how to use the ridges, except that I think Locutus is starting to get the idea.

The remaining 4 maps have features that bots struggle with. I didn’t enjoy them as much. I vividly remember the struggle to play on Sparkle, and I did not adapt Steamhammer to play on any of these maps. Since they’re selecting 5 of the 20 maps, any given map is unlikely to end up in the tournament. I thought effort was best spent elsewhere.

• (2)BlueStorm1.2.scx. The closest exit to the center is a narrow one that larger units can’t fit through. Steamhammer gets lurkers and ultralisks trapped behind it, not realizing it could take a longer path to the center. Iron breaks down severely, because vultures can’t fit through.

• (2)Hitchhiker1.1.SCX. This map has a narrow direct path between the 2 mains, a ravine. It also has numerous destructible neutral buildings that open other paths. Steamhammer sometimes plays OK on this map, and sometimes gets a large army stuck trying to pass through neutral buildings.

• (3)Alchemist1.0.scm. Each base has 2 entrances, and map is laid out in a loop so that the enemy might approach from either direction. Many bots cannot cope. Also, the map is poorly made and looks ugly.

• (3)Plasma1.0.scx. An awesomely difficult map. The main bases are small, so not many buildings fit. The ramp down from each is narrow, so not all units can pass. The map is divided into sections which are separated by blocks of neutral zerg eggs. Workers can mineral walk through the eggs in either direction using handy minerals placed just beyond. Other units can be pushed through one at a time; otherwise, you have to destroy enough eggs to open a path. For the icing on the cake, the left-side egg block between the 2 left bases leaves an open path 1 walk tile wide, not wide enough for any unit to pass, but wide enough for Steamhammer to conclude that the bases are reachable from each other. “Oh, hey, this is not an island map!” it thinks. My hope is that Steamhammer will learn to go air and opponents will be even less able than it to come to grips with the map.

I have wrapped up work on the last feature for the CIG version of Steamhammer. No more changes unless testing turns up a last-minute critical bug.

I’m pleased with how much I got done in a short time. With perfect planning I could have done more, but hindsight is like that. Looking through my change log, I feel I’ve done a good job improving play as much as I could with small changes. I also found a few good bug fixes. Steamhammer should do well enough in CIG.

Zerg got more work than the other races, of course, but most of the changes are not specific to zerg. The changes were intended to help zerg and were only tested with zerg, but the other races should benefit too. I even added a strong new protoss opening versus zerg (which I used in testing zerg).

Expect the SSCAIT upload of this version on Wednesday, more or less. I’ll test with terran and protoss first, and make any necessary tweaks (so it might be a slightly different version). In recent releases most of the important work has been on the opponent model and points of strategy. Those changes made Steamhammer smarter, but to a stream watcher, the play looked as though it had hardly changed. This release will be different. Plenty of familiar old weaknesses remain, because there was only so much I could do, but some very visible problems are reduced and I have changed long-standing patterns of play. Watchers should notice. Steamhammer’s play should look different.

Here is a question of virtually no importance that nevertheless has me puzzled: What should I call my learning files?

For now, Steamhammer has an opponent model file for each opponent, named om_[opponent].txt. OM for opponent model. It makes sense, or anyway it makes as much sense as an arbitrary filename can. I like names that make sense.

The next learning data I want to add is opening models, so that Steamhammer can know the timings. With knowledge of both opponent strategies and its own opening strategies, it will be able to directly compare and find counters: Your attack comes at this time, which opening is ready at that timing? It will also be able to recognize which openings are similar to others, so that when it can’t match a strategy, finding good openings by trial and error is quicker.

What should I name the opening model files? Opponent models and opening models are both empirical data, and should be updated as games are played. But the unfortunate words “opponent” and “opening” are too much alike to abbreviate nicely. Should I start with “enemy” and “build order”? “Bot” and “strategy”? Most abbreviations seem unintuitive. My best idea so far is to rename the om_* files to bot_* and use the prefix open_ for openings. Maybe OK?

What’s your idea? Because, according to the bikeshed principle, everyone should have an opinion on this....

Humans naturally keep track of progress toward their goals. It doesn’t matter what you’re trying to do: If you reach out for something but don’t get a good grasp, you notice. If you’re making a point in conversation, you try to tell whether the other person understood. In Starcraft, people notice (unless they’re swamped with multitasking) when a unit gets stuck, or an army takes a bad path, or really when anything interferes with a plan. And having noticed a problem, they can try to solve it.

Isn’t it obviously a good idea? And yet it seems rare in bots. I think nearly all bots only notice problems that they are explicitly coded to look for. They don’t notice when their units run into Iron’s wall and start to move aimlessly, seeking a firing position that they can’t reach. Even a novice human will realize that something is wrong, but bots don’t register that progress is stalled and keep trying to execute the failing plan.

I’ve been thinking about adding goal progress monitoring throughout Steamhammer, at every level: Strategy, operations, tactics, micro. First, I want to rewrite everything with explicit goals anyway, because I think it is clearer and more flexible. Carrying out a goal consists of choosing a plan (either ahead of time or piece by piece on the fly) and executing the plan. Then, goal monitoring means being able to tell whether the plan is executing as intended. Firing at Iron’s marine behind the wall is a 2-step plan, get into range and fire a shot. Getting into range is a subgoal to move to a position that is in range. And we can tell whether a movement plan is executing as intended: Is the range closing with time? Does the movement take about as long as predicted? If not, then the plan is going wrong and we may want to patch the plan, or try a different plan, or back up further and try a different goal.

It seems like a lot of detail work if done by hand, and I’m sure I will do part of it by hand. But it means that the bot will always react to problems. If Steamhammer is beating its head against Iron’s wall, it will notice. Even if it doesn’t have a wall recognizer and doesn’t know how to react, it will know that its plan is failing and that is should try something different—choose another target to shoot at, maybe that will work. After several tries, it will be sure to find that shooting the wall itself succeeds. Even without specific knowledge, having general adaptivity seems valuable.

It also provides a clear task structure. Today, Steamhammer’s structure is ad hoc—the underlying principle might as well be “let’s code up some behavior!” With a structure of goals and plans, the amorphous behavior becomes a programming pattern to be reused over and over in the code. Each behavior is made up of a fixed set of parts: Choose goals, plan how to meet each goal, monitor the progress of each plan, back up and try something else if the plan is failing.

The clear structure also helps with machine learning. “OK, system, now learn to behave!” is a hard problem. “Subsystems, go learn to choose goals, plan, monitor, and retry” is an easier problem, because the relationships between the parts are fixed. There is less to learn, and better information about which parts are working well.

Well, that’s my long-term vision. I expect I will get there in a slow and messy way, as always.

When I uploaded Steamhammer 1.4.3 to SSCAIT on 11 June, I erased its learned data from the server. Its elo immediately plunged, partly because the voters wanted to put it through its paces against strong opponents, and partly because it needs its learning data to cope. Most rushbots, and many others, won their first or first few games against Steamhammer. That didn’t change when I uploaded 1.4.4 a week later—the improvements weren’t many.

Finally, only in the past several days, I’ve started to feel that Steamhammer has learned enough that it is closing in on its equilibrium elo. It has been wavering around the high 2100’s, not able to break above 2200 but not falling far either. It seems about right, at least for SSCAIT conditions.

The findings. As I’ve mentioned, clearing the learning data was a deliberate test to see how well the learning system works when learning from scratch. I’m fairly pleased. I see weaknesses, but only weaknesses I expected. Against XIMP, Steamhammer has settled on an opening that has won every game so far, but is not as strong as the 3 hatch before pool opening that I hand-chose for it in the old days. Steamhammer only sees that it wins. It can’t tell the difference between openings that win nearly all games because it sees only the winning rate; it needs an evaluation function that can tell it “this one wins more convincingly.” Against Proxy, it won one game with its unusual 6 pool opening. Then it played another game and recorded another win—because Proxy crashed. Steamhammer thought it had found a winner, and had to lose some games before it realized that the 6 pool was not a reliable counter. (It would be, if not for Proxy’s powerful worker defense.) Possibly 5 pool or 4 pool would succeed, but Steamhammer does not know that some openings are related to others. When I teach it that, it will be able to realize that if one opening shows promise but is not quite successful, it should try related openings.

In some cases, Steamhammer hit on surprising counters. The most striking example is against TyrProtoss, which had been winning every game with its cannon turtle into timing attack strategy. Steamhammer tried its 2 hatch lurker all-in attack, which did not make sense to me—Steamhammer’s lurkers suck when cannons are around, it has little idea how to break the cannons and no idea how to bypass them. But it won a game. We’ll see if it keeps winning!

I expected the weaknesses, and I expected the surprising counters. I feel as though I understood the learning system and its limitations fairly well. It gives me confidence that my planned improvements, when I finally get around to them, will be real improvements.

I continue to prepare Steamhammer for CIG as best I can in the short time since I relaid my plans. I had to abandon 2 more ambitious changes when they busted their time boxes, but my log shows about 20 minor improvements. Each one helps in some situations—probably. Nothing is as well tested as it should be. Steamhammer does do better against some test opponents. I think that overall the improvement is definite though not great, and I have a little time left.

MicroDK mentioned somewhere (maybe it was in Microwave’s update note?) that Arrakhammer fixed a problem in the resource box used to avoid placing buildings in the mineral line. Steamhammer loves to manner its own minerals and gas (that means putting buildings in the way so they interfere with mining), but fixing it had never come to the top of my queue. Today I decided it was the issue with the best effort/payoff ratio.

Oh wow, this code inherited from UAlbertaBot is not at all good for Steamhammer! It calculates a single rectangle “avoid placing buildings here” for the main base minerals. It pays no attention to the gas geyser or to other bases. It was good enough for UAlbertaBot, which barely expands and doesn’t build much, but not so for Steamhammer.

My fix is similar in idea to Arrak’s, though my implementation looks completely different. I did away with the single-purpose resource box and relied on the more general building tile reservation map. I marked certain tiles near the minerals and geyser as reserved, so that nothing can be built there. For the geyser, I was careful to reserve the adjacent clockwise tiles which cause gas mining workers to take long paths around, and I tried to leave other tiles alone. For example, if the gas is straight up from the hatchery, then a building in front of the extractor on the left looks as though it should be in the way, but doesn’t interfere with mining. A building touching on the right, even at the corner, can cause bizarre paths for a drone leaving the extractor. There are some tricky cases of diagonally placed geysers where Steamhammer reserves too many tiles, but it’s not harmful.

Looking at the pattern of reserved tiles for different bases, I was able to spot one case where theoretically Steamhammer might be able to manner its minerals. But I couldn’t get it to happen in practice. This should completely fix an irritating problem.

I’ll release Steamhammer 1.4.5 after CIG submission closes. It’s a change of plans; I decided there was no reason not to.

Proxy has become legitimately strong. Versus terran, protoss, and random, Proxy goes 12 hatch with early sunken defense if needed, then hydras with a growing economy, keeping only 1 geyser and slowly shifting toward zerglings as the gas runs low. In ZvZ, it opens 12 pool (12 hatchery is too risky) and makes mass zerglings, again with an ever-growing economy. The ZvT strategy is not efficient, and Proxy suffers losses to weaker terrans that follow a one-big-push game plan, but still does well overall. The ZvP strategy is the most convincing. The ZvZ is not optimal, and Proxy does nothing to defend itself against mutalisks, but even so the mass lings are genuinely difficult for a bot to hold off.

For me, it’s natural to compare Proxy to Steamhammer. Steamhammer is far ahead in versatility and reliability (Proxy crashes sometimes). At everything that Proxy knows how to do, it is at least as skilled: It is at about equal in zergling micro, ahead in hydralisk micro, better at worker defense, less blunder-prone in tactics, and superior at macro. I admire its ability to switch between army and drone production in ZvP—Steamhammer desperately needs that.

According to the SSCAIT crosstable page, the head-to-head score as I write is Proxy 5-4 Steamhammer—they are about even in ZvZ, which is Steamhammer’s best matchup.

To improve further, Proxy needs more skills and more adaptivity. To do well against terran, it needs lair tech units. Proxy does less well against the top protoss bots, Locutus, PurpleWave, and BananaBrain, and even against XIMP by Tomas Vajda. To move further up the rankings, I think Proxy needs to recognize better what the opponent is doing, and adapt its play. And it needs new skills to cope with cloaked units, zerg air attack, and other basics.

Proxy has only shown itself in public for a short time. How much development did it get behind the scenes? Rather a lot, I imagine, to become as good as it is in such a short time. Still, it comes across to me as a young bot. Proxy shows that it is possible to climb high with few skills, as long as those skills are exceptionally strong. But to challenge the very top, it needs more.

Just thought I’d point this out, to remind everyone how far we’ve come.

IceBot’s elo on SSCAIT today is 2032. The rating system is designed so that the average rating is 2000, so IceBot is not significantly different from average. 30 elo points corresponds to a winning rate of 54%, not enough to notice when watching games.

IceBot was the strongest player on SSCAIT for over a year, from late 2013 to early 2015. When I ran the numbers in 2016, it was the longest-reigning champion: See 7 eras of SSCAIT champions. (Since then Krasi0 may have surpassed it.) In only a few years, we have made such progress that the one-time grandmaster with exceptional skills that other bots could barely touch now stands in the middle of a crowded field, showing as many weaknesses as strengths.

Next: Proxy, honest. These little interruptions don’t count!

Today there was a short string of comments about retreating. Steamhammer makes mistakes in retreating: It retreats to poor positions, and the purpose of retreating is to disengage so it issues a move command—but it doesn’t check whether it is moving to a safe place. It often loses by retreating into or through an enemy force without fighting. It’s such a severe weakness that I think almost every Steamhammer fork has implemented some improvement to the behavior.

I am planning a simple improvement for CIG. For the following AIIDE tournament, I should have time to implement actual factual honest-to-goodness tactical analysis and make maneuver decisions properly. So here are my thoughts on what has to go into tactical analysis.

1. Distinguish different kinds of tactics. Maneuvering is one kind: Deciding when and where to move, when and where to attack, what locations need static defense or mobile defenders or scouting units, all that stuff. In maneuvering tactics, you do planning to decide on goals, and then carrying out the goals requires reacting to the situation but doesn’t require planning as such. The planning is all up front; if the bot hits a problem, the solution is to replan and set new goals. There are also what I think of as sequencing tactics, like clearing a map block or carrying out a drop or performing a coordinated attack from different directions. They require a sequence of steps. The bot has to recognize a need—clear the neutral buildings to open the path to expand to that base; select and collect resources—use lurkers because the buildings are stacked, otherwise it will take too long; plan paths; and execute steps in order—first clear the neutral buildings, then send the drone to expand. The response to problems seems more complicated than “just replan from scratch;” minor problems can be solved by assigning more resources. I don’t think there’s necessarily a sharp line between maneuvering tactics and sequencing tactics, but they do seem to call for different planning skills.

2. I am increasingly thinking that the 3 levels of analysis strategy, tactics, micro are not enough. I think I want 4 levels, which I will call strategy, operations, tactics, and micro. To a certain extent, this already shows in Steamhammer, where decisions made at the squad level sometimes look like tactics and sometimes look like micro. Retreating is a perfect example: It is a tactical decision which is made by the squad based on local tactical analysis of the situation of that squad only. Roughly speaking, high level tactics are in the combat commander, low level tactics in the squad, and micro in the squad’s micro managers. I want to reorganize it somewhat so that it is clear what level each decision is made at.

Strategy is about resources. It decides on the drone count, how much gas to collect, when to expand, and what units to make. The unit count it achieves and the unit mix it chooses have big effects on the lower levels. Strategy also includes the highest-level understanding of the game situation, like “I need to stay defensive until I’ve built up a big enough force” or “I’m ahead in economy, so an even trade is good.”

Operations is about choosing high-level tactical goals and assigning units to achieve them. Raiding squad, go hit that enemy base; blocking squad, delay any enemies from arriving there. I think that attack and retreat should be decided at this level. Sometimes it is right to retreat the raiding squad when strong enemies arrive; sometimes it is right to sacrifice the raiding squad to do as much damage as possible. The decision is best made based on the whole situation. You want to be able to reason, “sacrificing the raiders to delay the enemy a little longer means the upcoming drop is more likely to work.”

Tactics tries to achieve the operational goals. It chooses roles for each unit in a squad, or perhaps subdivides the squad into subsquads with different subgoals: Hydras engage the cannons, lurkers run past and reach the mineral line as fast as possible.

Micro is individual unit control. Where should the lurkers aim to kill as much stuff as possible?

As always, the different levels of analysis should ideally feed information to each other. See levels of abstraction.

3. How to analyze maneuver tactics efficiently? MaasCraft showed the way: Cluster units together and analyze at the level of clusters. At the operational level, it doesn’t matter exactly where individual units are. The analysis has to remain valid for at least a few seconds, and units will move around in that time. Draw a bounding box or a circle around each cluster and represent the contents of the cluster as summary measures of unit count, total hit points, speed or range of speeds, or whatever other values are useful. I believe many bots do this kind of analysis, but Steamhammer doesn’t.

With that information in hand, you can make quick estimates of when an army can arrive, how much force can be brought to bear on a given point, who wins the fight, and so on. The estimates may not be accurate, but combat simulators aren’t accurate either. You just have to be good enough to improve your decisions.

4. What should tactical analysis look like? MaasCraft uses a form of MCTS, a logical choice of search algorithm. A simpler search would already improve Steamhammer’s tactics. I expect that, with a smart enough evaluation function, a brief search that looks ahead a short time would produce strong tactical play. Surely machine learning should be able to produce an evaluator that smart. But I have ideas about how to write one by hand, and I may try that first.

Still next: Oops, somehow Proxy slipped down a slot.

Still here. I’ve been hamstering on the codemill and not leaving myself enough time for anything else. It seems like the right thing to do, but based on history it probably isn’t. I usually go wrong when I rush for a deadline.

I want to make as much progress as possible by the CIG deadline on 10 July. And I have made several clear improvements. But my latest feature may be an overreach. After several days of work, the development version of Steamhammer lost every test game against the current release version. It will be an improvement if I can fix the problems....

Next: Proxy has improved fast.

CombatCommander::updateIdleSquad() does this:

    for (const auto unit : _combatUnits)
    {
        // if it hasn't been assigned to a squad yet, put it in the low priority idle squad
        if (_squadData.canAssignUnitToSquad(unit, idleSquad))
        {
            idleSquad.addUnit(unit);
        }
    }

The code is correct as it stands. It cannot assign 1 unit to 2 squads. But I consider it a latent bug: Only the lowest-priority squad can be updated this way. It would be easy and natural to make changes to squad priorities and cause an error, or to copy and paste this code under the assumption that it would be correct in a different case. You would get an irritating bug with no obvious cause. In the spirit of defensive programming, I recommend changing it to this:

    for (const auto unit : _combatUnits)
    {
        // if it hasn't been assigned to a squad yet, put it in the low priority idle squad
        if (_squadData.canAssignUnitToSquad(unit, idleSquad))
        {
            _squadData.assignUnitToSquad(unit, idleSquad);
        }
    }

assignUnitToSquad() ensures that the unit is removed from any previous squad before it is added to the new squad. Any squad other than the lowest-priority squad might steal a unit from another squad, and you can’t steal it without taking it away. Assigning 1 unit to 2 squads would cause problems, and Steamhammer checks later and will throw.

At a minimum, I recommend commenting the code so you don’t copy-paste it heedlessly when adding a new squad type.

Randomhammer terran played its first public game with the vulture-wraith opening against zerg ZurZurZur. As I have mentioned, I wrote vulture-wraith originally as an extra test of Steamhammer’s anti-factory zerg opening. I found that the vultures and wraiths bring terror to unprepared zergs, so I made it a permanent part of the terran arsenal.

ZurZurZur, it turned out, was not prepared at all. It could not get a sunken colony up, and it did not get hydralisks into the fight until its natural was destroyed. By then it had lost all overlords so it could not reinforce and the hydras were quickly defeated. I can see how Tscmoo terran does so well with a similar strategy.

I’m working on zerg stuff now for the upcoming tournaments, but I have a note to revisit this opening. It shows its legacy as a test opening; the execution and followup could be much stronger.

Next: A latent bug.