Starcraft AI blog

Squad unit clustering is the Steamhammer 2.0 change with the biggest effect on play. It was also the change that took the most effort to get working well, because it adds flexibility, which means more ways to go wrong.

Many bots form squads dynamically by clustering: Each cluster is a squad. Steamhammer forms squads as usual, and clusters the units in each squad into groups which act somewhat independently. The two arrangements are not necessarily different in effect; what matters is how decisions are made. Steamhammer’s system is meant to keep a clean distinction between levels of abstraction, the operational level at which squads are formed and given orders, and the tactical level at which units are maneuvered to carry out the orders. Each cluster in the squad makes decisions based on its local situation, but it is ultimately trying to carry out the same order as the other clusters in the squad.

Clustering is driven by a simple imperative: Units must react to their local environment. In the unified squad structure that Steamhammer inherited from UAlbertaBot, the squad runs a single combat simulation for front-line units which are in contact with the enemy, and the entire squad is told to advance or retreat based on the result. Units which are away from the front lines, but in contact with the enemy, are given nonsensical orders as often as not. For example, if the front line is advancing because the enemy army is away from home, then small groups of reinforcing units which run into the enemy army are also told to attack, and they end up dying. And conversely, if the front line is afraid to attack, then units behind the scenes which meet lone enemies are also afraid of them. Units must react to the situation they are in, not the situation some other unit is in.

With clustered squad units, each cluster that comes into contact with the enemy runs its own combat simulation, and advances or retreats independently depending on the result. When the army is large, the squad members tend to be spread over a wide area, and independent behavior is a necessity. The full set of cluster behaviors is complex; keep reading.

the clustering algorithm

The clustering algorithm itself is implemented in OpsBoss (intended to eventually replace CombatCommander), which is probably not the right place for it. Methods cluster either a given set of units (like the members of a squad) or all units of a player. The feature to cluster all units is intended for clustering enemy units, and not currently used, but I expect it will be valuable for deeper tactical analysis when I get that far.

A cluster is either an air cluster or a ground cluster. Air units and ground units have different movement possibilities—different decisions available—so they should be treated differently. Only an air cluster can retreat over a cliff.

Steamhammer makes circular clusters. I expect that rectangular clusters are as good, and probably a tad cheaper to calculate, but also a tad more complex. Pick an arbitrary unit as the seed of a cluster, find nearby units, calculate the center and radius of the cluster so far, and expand the radius by a fixed amount to see if that draws in any more units. If so, recalculate the center and radius, and so on; if not, the cluster is complete. Repeat until all units are in clusters. A cluster of size 1 is fine.

clustering in squads

Each squad reclusters its units every frame. That means that clusters have no continuity; there is no such thing as “the same cluster” the next frame. It’s a limitation. Some units, like overlords and defilers, are left out of clusters and handled separately. To save time, not every cluster makes decisions on every frame, especially in the late game when there may be many clusters. See Squad::update() for all this stuff.

The clusters then make decisions independently. Each is trying to reach the squad’s order position, and decides what to do based on its local situation. The clusters know that they are working together, so if they see a way, they will try to join up into larger clusters. If you turn on Config::Debug::DrawClusters in the configuration file (it is currently turned on for the SSCAIT stream), you’ll see (among other things) a status string for each cluster: Join Up, Advance, Attack, Retreat, and so on. The decisions work like this:

• If there is no enemy nearby that the cluster can attack, or that can attack the cluster, then the cluster is in a “no fight” situation—combat is impossible for the moment.

• If a “no fight” cluster finds itself in the vanguard, its status string is “Advance” and it moves toward the squad’s order position.

• If a “no fight” cluster is behind other clusters, its status string is “Join Up” and it tries to merge with a cluster that is ahead of it. If the cluster ahead is advancing, they will form a train; if the cluster ahead is stopped or in combat, the trailing cluster may be able to join it.

• Other clusters have to decide whether to attack or retreat (aka regroup). First, the code checks several shortcuts to see if combat sim can be skipped.

• If Steamhammer is near max supply, the status string is “Banzai!” and the decision is to attack..Every cluster of every squad with an attack order will try to get into the fight.

• If the cluster is near a static defense building that can help in the fight, the decision is to attack. There are a couple of different cases, which get different strings mentioning static defense. The behavior is not as good as it could be (most zerg units should move behind the static defense before turning around to fight, instead of standing in front), but it’s an improvement over past versions.

• If the cluster has retreated as far as it can, back into its base which is now under attack, the decision is to attack and the string is “Back to the wall”.

• If none of those checks fires, there is nothing for it but to run the combat sim. Depending on the result, the status string will be “Attack” or “Retreat”. On the enemy side, all the nearby enemies are added to the combat sim, as usual. On the friendly side, only the cluster’s own units are added; clusters assume that they are unable to cooperate. This works better than adding all nearby friendly units, because in fact different friendly units often do not cooperate. The downside is that clusters cannot cooperate when they should, for example to carry out a sandwich maneuver. A better system would be to identify top-down which clusters have the same enemies in their sights, and run one combat sim for all those clusters. I expect to do that at some point after I turn on enemy unit clustering.

• If the decision is to retreat, where should the cluster retreat to? There are several rules for this, checked in Squad::calcRegroupPosition(). I have said that Steamhammer’s old tactical misbehaviors are gone, but it’s not true: Clusters sometimes still try to retreat through the enemy force. There is plenty of room to improve retreating. At some point after I turn on enemy unit clustering, I’ll teach the retreat calculation more.

• Retreating: If there is nearby static defense in any direction, retreat toward it.

• Retreating: If part of the cluster is in range of the enemy (so we did a combat sim) and part is out of range (so it is a safe place to retreat to), retreat to the position of the cluster unit out of enemy range which is closest to the squad’s order position. This is a cluster variant of the classic UAlbertaBot retreat behavior. Restricting it to one cluster at a time makes it behave much more nicely—no more retreating to a random position because some reinforcing unit happens to be approaching from an unexpected angle.

• Retreating: Look for another cluster nearby to join up with. Favor nearby clusters that are closer to the order position and clusters which are attacking. It’s good in general for clusters to join up, but this rule also tries to fix a specific weakness: If a small cluster does its combat sim with its few units, it may be told to retreat, even though a large cluster in front of it has done its combat sim and decided to attack. It is much better for the small cluster to join the large one in its attack, so in this case Steamhammer tries to merge the clusters, or in other words, it retreats toward the enemy. It doesn’t always work; cases still occur of small clusters which are needlessly fearful.

• Retreating: If all else fails, retreat toward the main base.

other details

The call Squad::unitNearEnemy() decides whether a unit is “in range” of the enemy. In the past, Steamhammer often did not retreat far enough for safety, especially if the enemy had sieged tanks. I improved it in 2 ways:

• It uses the information manager’s records of all enemy units last known positions to decide whether an enemy is near, instead of using the MapGrid information about visible enemies only. Having to see the enemy to know to stay out of its firing range was especially bad for zerglings, which have a short sight range. It was also bad against sieged tanks, which can hit from out of sight range.

• If the enemy is known to have tanks with siege mode, then ground units make sure they stay outside of sieged tank range. Before, the assumed safe retreat distance was simply not far enough to be safe from tanks! It was painful to see a squad retreat to just inside siege range, lose the frontmost units to tank fire, and—move forward to stay centered on the retreat point. Ouch!

I made the minimum changes to Squad to get all this to work. The code is getting increasingly messy with special cases. For example, the micro managers which handle clustered units have to be told about the cluster and do a set intersection to figure out which units they should issue commands to, and the micro managers that handle unclustered units do not—squad code and micro code have to be coordinated, making refactoring more difficult. I will eventually rewrite the squads with a new and cleaner design, but probably not this year.

overall

This was a needed change. Units simply must pay attention to their own situations, there is no way around it.

Play versus protoss is vastly stronger. Voters have been feeding Steamhammer a lot of games against top protoss bots, which zerg mostly loses. Even in these losing games, it is easy to see that Steamhammer is putting up a much tougher fight than it used to. Since my home tests against DaQin, which Steamhammer learned to defeat, DaQin has been updated to be more aggressive and Steamhammer can no longer win (and I don’t know what version was entered into AIIDE, so we’ll see how the tournament goes). I can see the path to stronger zerg play, to earning wins over Locutus and PurpleWave. I can tell what needs to be done and I have confidence that it will succeed.

Play versus terran varies, and play versus zerg is weaker because of poor zergling behavior. A large part of the poor zergling behavior is due to clustering changes. In the past, when Steamhammer retreated a squad, it would not rerun the combat sim and consider attacking again until a time limit had passed (I changed the details thoroughly, but the behavior was inherited from UAlbertaBot). Since clusters do not persist from one frame to the next, it was impossible to retain this feature. The result is more indecision, and the indecision strongly affects zergling play, and causes other harm as well. It’s not easy to fix, and I’m still thinking about ways.

Part of the weakness is in retreating from a superior force that then advances. Not all of the enemy force is in sight range, and the rest is assumed to be still in its last-seen positions, so the combat sim says “ha, we can turn around and fight now!” With no retreat time limit, and the short sight range of zerglings, this happens fast and causes serious losses. The weakness was always there, and now it is worse.

Next: The big change list with everything else.

I rewrote important parts of base defense. I fixed minor bugs in passing, but I had 2 main reasons: 1. Squad unit clustering caused bad defensive behaviors that had to be fixed. 2. The original base defense, as inherited from UAlbertaBot, was actually region defense. It iterated through regions of the map, and defended any region where the bot owned buildings. On some maps, one region can contain more than one base. Examples are the mineral-only bases on Python which are all in one big center region, and the center bases on Fortress which are laid out similarly. Base defense needed changes to defend these bases successfully.

• Iterate through bases, not regions, to check for areas that need defense. This means that buildings built outside any base area are no longer defended—not a serious loss, they were rarely defended well in the first place. See CombatCommander::updateBaseDefenseSquads(). The squads created to defend bases are called Base squads.

• A base is deemed to be in need of defense when an enemy unit (other than a single worker scout) is within a specified baseDefenseRadius distance of the base, OR if it is in the same region as the base and is within baseDefenseRadius + 300. The base region may be very large (and may contain multiple bases), in which case we don’t want to panic and send defenders when they aren’t needed. Or the base region may be very small, so that attackers outside the region can hit the base, a historical problem for Steamhammer. This condition covers both cases.

• A base is considered to be no longer under attack if there are no enemy units meeting the same criterion, except that the radius is larger by baseDefenseHysteresis. The hysteresis reduces indecisiveness. The numbers baseDefenseRadius and baseDefenseHysteresis are constants defined at the top of CombatCommander::updateBaseDefenseSquads(); at some point I may move them to the configuration file.

• Steamhammer used to ignore overlords, observers, and floating terran buildings in deciding whether the base needs defense. Now it counts them, and will send anti-air units if needed and available. It will also assign a detector if needed to target the observer. Thou Shalt Not Scout For Free.

• A Base squad is disbanded and the squad data structure destroyed when the squad is no longer needed. The squads used to be kept around even when empty. Dropping the squad mainly makes the debug display easier to understand.

• The Base squad’s order is centered on the enemy unit which is closest to the base location (the upper left corner of the command center, nexus, or hatchery). This guarantees that the Base squad won’t sit idle in the middle of the base while the enemy rampages on the outskirts.

• Release defending units that can’t attack any of the remaining intruders. For example, if the enemy shows up with both air and ground units, we might assign both zerglings and mutalisks for defense. If the enemy ground units are destroyed or leave and only air units remain, the zerglings become useless for defense and are released. The useless zerglings used to remain in the Base squad until the whole squad was released.

• Don’t double-count the same unit as both a ground and an air defender. Steamhammer used to think, “I have 4 hydralisks to drive off these corsairs, and I have the same 4 hydralisks to stop these 2 zealots. I have enough defenders!” The calculation to avoid double-counting is kind of involved.

• Fixed a bug introduced in a recent version which could overassign defenders. It would, for example, always assign at least 2 anti-ground defenders when under air attack.

pulling workers

Deciding whether to pull workers for defense—to place them in a Base squad—is also in this code. Pulling workers is sometimes necessary, but it has to be strictly regulated because it puts workers in danger. (Worker self-defense doesn’t involve putting them into a Base squad, and is not handled here.) I tried to fix the biggest weaknesses.

• Pull workers that are within a smaller radius: 18 tiles instead of 24. This is about the location of the worker. That should mostly prevent Steamhammer from pulling workers from another base to defend this one.

• Pull workers only when the enemy is very close to the mineral line. This is about the location of the enemy. Steamhammer used to pull workers to meet the enemy too far away from the mineral line, wasting time and putting workers in unnecessary danger.

• Release pulled workers much sooner. Steamhammer used to retain pulled workers in the Base squad until the squad was disbanded. Now it releases the workers when the “must pull workers!” condition is not satisfied.

Next: Squad unit clustering.

The game Steamhammer-Bereaver is a good example of Steamhammer’s defiler skills, both the good and bad points. Steamhammer gained an early advantage and contained Bereaver to its main, so the outcome of the game was not in doubt, hive tech or not. Bereaver mounted a strong defense at its ramp and held on for a long time. A timid zerg might have feared to attack and lost on points when the game time ran out—but of course Steamhammer is not timid.

The defiler dithered for a long time. Consume research finished at about 15:45 into the game, and the defiler filled up on energy within 15 seconds after, so that part was OK. A first attempt to break up the ramp was repelled with storm; the defiler was too far back and unable to help, due to lacking squad coordination. The second attempt to break up the ramp was at 18:20, and this time the defiler was in range. That’s about 2 and a half minutes from when the defiler was ready to act until it finally did; it had plenty of time in between to swarm or plague. It would have been worth it to sacrifice the defiler to plague the defenders.

Excellent swarm placement! The defiler had energy to cast a second swarm farther into the enemy base, but did not. Even so, the one swarm nullified the dragoons and cannons and ensured that the attack would succeed if pressed. Thanks to my modifications, FAP understood that it would succeed and Steamhammer did press until it broke through. The attack had to be under way before the defiler could support it; Steamhammer has no ability to swarm (or plan to swarm) in advance of an attack. Steamhammer attacks when maxed, so in most winning games an eventual attack is guaranteed. When the swarm does come, it is generally well-positioned for where the units are at the moment, rather than where they are going or where they want to be.

The defiler seemed to get confused and did not contribute again until the very end, when it laid down a perfectly-placed plague over a few protoss buildings that were about to be destroyed anyway. The laser-precise plague hit as many protoss buildings as possible while missing zerg units. It’s impressive in a way, and it may even have shortened the game... by a fraction of a second.

Still next: Base defense.

Defilers are complicated to use. I implemented a pile of features to get them to work acceptably, and I suspect that a bug has crept in since they don’t seem to work as well as when I first finished the implementation. They don’t swarm or plague as often as they used to. A likely source of the bug is the time control work.

The first step is to get defiler tech. The strategy boss decides when to get defilers. Currently, it delays defilers until fairly late versus terran that has a high ratio of tanks and/or vultures in its force, especially with spider mines, and otherwise uses the same criteria for all races: Do I have a hive, enough drones, and a few more items? When the defiler mound finishes, the strategy boss eagerly orders consume research. It is lazier about ordering plague, and later metasynaptic node (+50 energy). According to conventional wisdom, the energy upgrade is useful for defilers because then a full-energy defiler can swarm once and plague once before it has to consume again.

The strategy boss currently orders the production of 1 defiler at a time. A second defiler is not made unless the first one is lost. The defiler is sent toward the front in the hope that it will find something useful to do there.

consume

The consume implementation is on 3 levels. Strategy: If there are not enough zerglings on the field for defilers to consume, because the unit mix calls for other units, then the strategy boss orders up a few more. It makes sure that there exist at least 4 zerglings for each defiler. Tactics: A small number of zerglings are assigned to the MicroDefilers unit controller alongside any defilers. The number assigned depends on how much energy the defilers need; see Squad::addUnitsToMicroManagers(). The controller orders the zerglings to catch up to and stay near the defilers; each seeks the closest hungry defiler. Micro: If a defiler is hungry and finds an assigned zergling standing by, it’s lunch time.

A defiler counts as “hungry” if it has less than 150 energy. Defilers don’t consume anything other than zerglings which have been assigned for consumption. It’s not urgent, but it would be nice if they could consume irradiated units, or any units nearby that are destined to die. In an emergency, they should be willing to consume overlords. In the very late game, when minerals are gone and gas is plentiful, it may be more efficient to consume scourge.

plague

Plague is a spell of opportunity. If there is a big ball of enemies that aren’t already plagued and have a lot of hit points to lose, it virtually always pays to plague them if you can. Steamhammer is not smart enough to intentionally seek out good opportunities, but it is smart enough to recognize opportunities that arise.

So defilers keep their eyes open. If a defiler passes a cheap check (I have plague researched, I have enough energy, there are a bunch of enemies nearby) then it performs an expensive check to decide whether and where to cast plague. It draws a square of each place it can reach without moving far, and does a raster scan through the square, scoring each possible plague it might cast. For enemy units, it adds up the hit points the enemy will lose, counting it as 0 if the enemy is already plagued and adjusting for a few special cases: Buildings other than static defense get a discount, they are less important to plague, while a bonus goes to cloaked units (which are revealed by plague) and carrier interceptors (which will get stuck in the carrier until plague wears off, as the carrier tries to repair them). It also subtracts plague on friendly units, so that it is willing to plague itself if it covers enough enemies too. If the best score passes a threshold, cast the plague there.

Death rule: If the defiler thinks it is about to die, then plaguing any enemy is good enough. Theoretically, in this case Steamhammer might plague 1 stray probe that happens to be close enough.

This exhaustive search method is startlingly accurate at deciding where to plague (though I think I should increase the discount for buildings). I’m sure there are computationally cheaper ways to get good results.

dark swarm

Dark swarm is way more difficult to use well than plague. Swarm use ought to be part of a coordinated plan. A simple plan might be: Start the attacking units moving, then cast swarm around the time the attackers come into cannon range. The decision to attack should be informed by the intention to cast swarm during the attack—with enough swarms, one zergling can be the Klingon running man and kill all the marines. You need different plans for assaulting a fixed position, fighting mobile enemies that can evade swarm, or warding off an attack on your base. You have to take into account what units on both sides are able to do damage under swarm.

Steamhammer can’t plan like that. Those skills are far in the future. All zerg bots are weak with swarm.

Steamhammer chooses where to swarm with an exhaustive search similar to the plague search, and it is not as effective. First, the quick check only allows swarm over enemy buildings; Steamhammer can use swarm to attack, and not to maneuver or to defend, a major limitation. Protect the army in the middle of the map from air attack? No can do, boss: Besides the defiler skill, that would require more smarts in unit control. In the exhaustive search, it scores the units that end up under dark swarm, trying to cover friendly units that can hit under swarm and enemy units that can’t. It’s not the score you want; it ought to take into account nearby units that will want to move into the swarm, or will be unable to hit swarmed units from outside, or will have to suddenly flee.

Death rule: If the defiler is about to die, it will swarm wherever it thinks best, even if it doesn’t cover an enemy building. If the defiler doesn’t have any better idea, it will cast swarm over itself, which may save it depending on the situation. If the dying defiler could either swarm or plague, the choice depends on which happens to be checked first.

I implemented consume, plague, and swarm as micro skills, decided at the level where the code is controlling a single defiler. It makes sense for consume and plague. I think swarm should be decided at the squad level, because its use depends on the tactical situation and the squad’s goals. That will require a lot more cleverness in the squad, though.

combat simulation and micro

I modified the combat simulator FAP with a rough understanding of dark swarm. FAP performs a loose approximation of combat, and the dark swarm knowledge I added is equally loose. If a unit is under dark swarm at the start of the simulation, it is assumed to remain under dark swarm throughout, no matter how it may move around. (Movement ignores terrain and other units too, so FAP is already loose that way.) It understands which units can do damage under swarm, and has simulated units seek out only targets that they can damage.

All this happens only if dark swarm is already cast. There is no provision for planning ahead, “when the units start shooting, I’ll cast swarm here.” In other words, dark swarm influences the combat sim, but the combat sim does not influence dark swarm; Steamhammer doesn’t extract the full value. See the discussion above about coordinated plans.

Steamhammer also understands dark swarm at the unit micro level. Melee units seek out targets under dark swarm, on the theory that then they will be protected under swarm too (“ha ha, you’re under swarm, I’m perfectly safe... I hope?”). Ranged units skip targets under swarm that they can’t hit. The ranged unit controller understands which ranged units can do damage under dark swarm.

There is room for more smarts in exploiting dark swarm at the tactical level. Units under air attack, for example, or facing marines, should move into swarm for their own protection, and mostly they don’t. I’ve seen Steamhammer make serious mistakes because of that ignorance.

time control

My first implementation made 2 defilers at once and performed all the computation steps in the same frame. A disadvantage was that sometimes the 2 defilers would swarm the same place at the same time (scoring ensures that if they swarm at different times, they swarm different places). A bigger disadvantage is that it was too slow, with some frames taking over 150ms. I changed it to make only 1 defiler at a time, and arranged that consume, plague, and dark swarm are examined in different frames. That made it fast enough. There are ways to speed up the calculations, but slicing them apart seemed easier.

Overall, the defiler system is not industrial strength, but it’s a strong start. As I mentioned at the start, I suspect a bug due to the time controls. Steamhammer implements all the basic skills, which if you go through the above text you can estimate for yourself required 1.5 zillion different software features. Now that the pieces are in place, they can be improved one by one.

One of the emergent properties of the defiler system is that it is unpredictable. The defiler may plague this and then plague that, or swarm here and then swarm there—it gives the impression of running in streaks. And I can’t judge ahead of time what it will do, it surprises me.

Next: Base defense.

Steamhammer 2.0 is uploaded at SSCAIT. As I have mentioned, it is zerg only; Randomhammer is not updated. The only difference from the AIIDE 2018 version is that I turned on a few debug options, notably the new option to draw unit clusters.

I erased the learning data, so Steamhammer 2.0 is figuring out its opponents from scratch. It’s necessary since this version has different strengths and weaknesses. I expect its elo to plummet until it gets to know the different kinds of rushbot (just like the last time I erased the learning data).

Today I list the big changes in Steamhammer 2.0 without going into detail. There are tons of smaller changes. Details will follow.

The most effective changes:

• squad units are clustered, and each cluster makes attack/retreat decisions independently
• mutalisk and especially hydralisk micro is crisper
• a base under attack does not receive transferred workers, improving resilience
• greatly reduce cases where the bot gets a spire/lurker aspect and then delays getting mutas/lurkers

The squad unit clustering allows Steamhammer to play a big army macro game without the constant tactical blunders that it used to make (it makes smaller and mostly different blunders). Many bots use a similar system, with good reason. The tactics rework is substantial, and deserves the 2.0 version number by itself. In combination with the other improvements, Steamhammer is incomparably stronger at ZvP, which may now be its best matchup. I reported on DaQin, and Steamhammer (like Proxy) can open with 12 hatchery and still cope with Wuli’s rush (Proxy’s author called it easy, but it requires a combination of several skills that are not so easy).

I put in a ton of work, but the decisions that unit clusters make are still not as polished as they need to be. An interaction with the combat simulator causes dangerous mistakes in retreating from superior forces. It can happen in all matchups, but it is most obvious in zergling fights in ZvZ. What used to be Steamhammer’s most successful style of play in its best matchup is now its least successful in what may be its weakest matchup. It’s pretty funny, or at least it will be after I figure out how to fix it.

All these items deserve their places on the list. I fixed the case where Steamhammer’s spire used to finish and it would make a bunch of drones instead of mutalisks, and that may sound like the usual modest improvement to the strategy boss. In fact it is a critical fix that makes midgame tech switches more rapid and decisive. It wins games.

Other important changes:

• mineral locking
• base defense was reworked extensively
• new openings added to exploit possibilities and old openings tweaked to fix flaws that had crept in
• opponent model exploration and strategy randomization improved
• defiler support with all spells: consume, dark swarm, plague

Many aspects of play have moved forward. As always, there are new bugs like the retreating problem I mentioned above. And as I blathered about yesterday, old weaknesses remain. The overall result is as I’ve reported before, hugely stronger play against some opponents and weaker play against others. Well, that’s in my testing, which is never enough. We’ll see how it goes in the wide world! Since we’re in a time of protoss domination, I’m thinking that strength against protoss is a good sign.

For AIIDE in particular, I expect that my changes to the opponent model are key. I made the couple needed improvements of a previous post. The academic papers I found turned out to be unhelpful, and my changes are ad hoc, but they address the problems. If the opponent chooses strategies unpredictably, Steamhammer will too. If the opponent model doesn’t find a winning counter, it will explore more and more widely until it is randomly trying anything it knows, including openings which are otherwise not played in any context. On the upside, I’m hoping it will eventually hit on surprise counters to the toughest opponents. On the downside, in the worst case it will explore a lot of bad ideas, and the convergence time could be longer than the tournament!

Later today: Notice after Steamhammer 2.0 is uploaded to SSCAIT. Tomorrow: Defiler support in detail.

Today, a post about things I wanted to do and didn’t get to, or started to do and rolled back or didn’t finish. I accomplished a lot, but the to-do list is functionally infinite.

• UAlbertaBot executes unit orders as they are issued during the frame, retaining no state beyond what BWAPI remembers. Steamhammer inherited the system. I am in the process of changing it to record the orders and save status info, then issue any needed orders at the end of the frame. Adding the Micro object in the last version was one step. The Micro object now records the order and status of each of our units, and has all the infrastructure it needs to issue orders, including its own update() method with TimerManager calls to track how long it takes. But it doesn’t actually issue orders; there is no effective change to the micro system.

When this is done, it will be easier or possible to unstick units, to make sure units don’t get stuck in the first place, and in general to control units precisely.

• I worked a bit on analyzing Steamhammer’s use of the combat simulator. Improvements are there for the taking, but my first try was not successful and I left it for later.

• McRave suggested that micro to avoid big damage attacks was key. Equally important is to pull off big damage attacks yourself. Besides plague, the zerg big damage attack is scourge suicide, and Steamhammer sucks with scourge. Scourge control is on my list, but never came to the top.

• I implemented mine dodging to reduce the danger of spider mines, but the way I did it was not useful for zerg. Reacting to mines is tricky: If a mine pops up, sometimes you should drag it to the enemy, sometimes drag it away from your other units, sometimes shoot it down. I lost the “just shoot it” case, and it was a net disadvantage. The feature is in the code but is disabled.

• Steamhammer’s lurker skills were adequate for when they were implemented, but have fallen behind the times. It needs to stop doing things like burrowing one lurker at a time in cannon range.

• I haven’t done anything to improve overlord safety, dark templar reactions, or any of that stuff. Many weaknesses there.

• I planned and promised a skill to put pressure on the enemy in the early or middle game, but did not finish one. I had several ideas, and all of them hinged on what I call raiding: Harassing where possible, moving around but staying near the enemy, not retreating to rejoin the main army. Harassing mutalisks can be considered raiders in that sense, and so can Krasi0’s vultures and Bereaver’s reaver drops. Runby units and dropped units should act as raiders, and Steamhammer doesn’t have the skill—with a Hold order, they go idle if they clear their area to hold, and with an Attack order they may try to retreat to the main base. Picking up dropped units is also a good raiding skill.

• I implemented a Watch squad, but did not polish it up enough that it became useful. It is turned off. It had several purposes. It could keep watch on expansions and chokes, sometimes making it possible to disband the Recon squad. It could prevent enemy expansions in some cases, and clear mines and verify the safety of expansions that we wanted to take.

• I thought of adding pre-learned opponent model data for the fixed opponents carried over from last year, plus opponent model hint configuration (like “try this first and see how it works”) for opponents that I think I know something about. PurpleWave, we’re told, did an elaborate job of that for CIG this year. The idea of hints is that the opponent model can save exploration time if the hints are right, winning more games, and correct itself with little loss if the hints are wrong. I decided that, since AIIDE is a long tournament with plenty of time for learning to converge, other work would probably help more.

upcoming

Done with the testing, today I am resting. I’ll upload zerg-only Steamhammer 2.0 to SSCAIT tomorrow. On the blog side, I think I’ll start with an overall evaluation post, what I’ve done and how successful it seems, and follow up with posts about specifics like defiler skills and squad restructuring, and then the big detailed change list. At some point soon Steamhammer 2.1 will slip in. 2.1 will likely have a few minor fixes and deserve its own little change list.

Shortly, CIG should release its detailed results. They promised source and replays. If they also provide a game result log, I’ll analyze it and post my usual colorful summary tables. Not sure how that will fit in with the other posts, the blog may have a busy time.

Steamhammer 2.0 is submitted for AIIDE 2018. All I have left is the waiting.

Since yesterday I fixed one bug, a newly introduced bug in the worker manager that could cause a small number of drones to temporarily go idle early in the game, when it matters a lot. I made an attempt to fix a lingering weakness related to cooperating with static defense, but had to roll it back. The fix was a 2-line change, but somehow it caused a hard-to-understand bug, and I can’t accept a risk like that so close to submission time. Also, since an entire day is too long to go without working on the openings, I added a new opening that helps Steamhammer sidestep weaknesses in ZvZ, allowing it to beat more opponents. And I ran all the tests I had time for, looking for new bugs, checking my fixes, and verifying that the new opening functions as intended and earns wins.

It’s hard to understand why, but now I feel tired....

New bot DaQin on SSCAIT matches an AIIDE entrant. It may be the AIIDE entry, or it may be a test version shortly before the entry is made. It looks like a Locutus fork. The .dll is larger than the Locutus .dll by about 24K out of 2.5M—a fair amount of code but no drastic rewrite. The play I’ve seen so far looks similar, lots of dragoons with strong Locutus micro, so without digging deeper I’m not sure what the changes are.

As one of my tests today, I ran a short match Steamhammer 2.0 - DaQin. Steamhammer took several games to find a way to win (not long at all), and after that it won every game, though I didn’t play many and one of the games was excitingly close. I didn’t try the old version Steamhammer 1.4.7, but with Locutus skills I expect that DaQin would have won every game. I don’t think Steamhammer 1.4.x has a way to beat DaQin. It was satisfying to see Steamhammer 2.0 score convincingly, and it helped my confidence.

We’ll see how much Locutus has improved since August!

Steamhammer is almost ready for AIIDE. I have fixed all the major bugs and weaknesses that I have time for—the last was a newly-introduced bug in mutalisks caused by tightening up the micro, it was a bad one. The rest will be simple low-risk fixes and testing to make sure I don’t have another disaster like in CIG (it’s a risk, I wrote a bunch of new code and my change log records over 80 items).

Steamhammer is still weaker against some opponents, though not by as wide a margin as in my last update. It still crushes some opponents that it used to lose to. I can’t foresee which way the balance will fall, definitely not in this tournament with so many new names. My ambition remains to finish in the top 3rd, about the same as last year, which will mean that I am keeping up with the blazing fast progress in the scene.

There are changes at all levels. I have the highest hopes for improvements to the opponent model (key for a long tournament), the squad structure (sounder in principle than the old one), and certain of the micro optimizations (not fully exploited yet, but a firmer foundation).

Regardless how well Steamhammer performs in AIIDE, looking back over the past year I feel I’ve gotten a good amount done. Steamhammer is still short on basic skills, but compared to a year ago it is smarter in most ways. There is a mountain left to move, but I am that much closer to having moved it.

Steamhammer 2.0 for AIIDE will be specialized for zerg only. There won’t be stuff for the other races in the config file, and I haven’t tested that terran and protoss work at all. I’ll upload it to SSCAIT shortly after the AIIDE deadline, only turning on a few of the screen drawing options. Since it plays so differently, I’ll erase the learned opponent data again. A few days after that, depending on how wiped out I feel and whether any surprises turn up, expect Steamhammer 2.1 and its identical twin Randomhammer that plays all races. Along with, of course, a series of posts about the details of my work, the unexpected problems that cropped up, and the known problems that I would have solved if only there were more time.

The good news is that I have definitely made enough changes to Steamhammer that the next release deserves to be called Steamhammer 2.0. In some situations it plays much more strongly. The bad news is that I still haven’t fixed all the bugs introduced in making such extensive changes. Changing the squad structure, which has little direct effect on unit control, had surprising indirect effects on the micro of different units—and I also made changes to micro itself. There are situations where it plays much worse, and some are difficult to debug. I may have bitten off more than I can chew.

The effect is big: Some formerly difficult opponents are easy, some formerly beatable opponents are difficult. With all the new names in the AIIDE entrant list, it’s impossible to guess whether Steamhammer will face opponents it finds easy or difficult.

Steamhammer also has new skills like defiler use, a new layer of micro infrastructure, more smarts in the opponent model (time-consuming to test), improvements to the strategy boss, and a bunch of new, fixed, or retuned openings. And stuff. Special preparations against Steamhammer might backfire.

Well, regardless of how well Steamhammer performs in AIIDE, I feel that I’ve made progress that I can believe in. The new squad structure is more complex so it allows more ways to go wrong, but it fixes fundamental weakesses that are inherent to the legacy design and it does not create new fundamental weaknesses. What I mean is that, in principle and if polished enough, it could be superior in many situations and worse in none. It’s not that polished yet....

Anyway, there is still over a week left. I have other things to do too, but I should be able to make fixes. Outlook: Moderately good.

Simplicity has been getting updates, and an update yesterday was apparently when it gained overlord drop skills. (There’s another update today.) It is the first zerg bot that I have seen use drops (well, in public at least). The drops look a bit stereotyped, with about 8 overlords loaded up in the main and sent (usually along the edge of the map) toward an enemy mineral line—it looks as though the drop happens when the overlords find targets to drop on. That makes Simplicity the first bot I have seen do mass drops, a unique skill.

I found 3 drop games on SSCAIT. Curiously, all 3 games are on Moon Glaive, although Simplicity does research drop on other maps. The less impressive games are Simplicity-Juno aka Yuanheng Zhu and Simplicity-NiteKat terran. Drops definitely did not change the outcome of those games. The best drop game is Simplicity-WillBot, in which WillBot randomed to terran.

The first drop started to load up at the zerg main just before 16:00 into the game—while a terran attack on the natural was underway, not the most auspicious time. Apparently Simplicity does not put much analysis into the drop decision. In the picture, the two players are roughly even (objectively, with no defilers in sight, the big terran army should win, but neither bot has the skills). The selected overlords are exactly those that are loading units. In the minimap, notice the long tail of terran units moving toward the zerg natural; the terran attack was kind of piecemeal.

Loading proceeded slowly, I guess since the ground units were distracted by the terran activity, and the overlords set out at about 16:40. The overlords proceeded counterclockwise around the edge of the map toward the terran main. Meanwhile, terran diverted to annihilate the zerg base at 9 o’clock before returning its attention to the zerg natural, and zerg units that were still on the ground bypassed the marines and tanks and headed for the terran natural to counter. Simplicity’s decision was good. Attacking the terran bases from both sides put WillBot into a difficult situation where it was tough to make the right choices.

The drop started to land at about 17:40. WillBot defended the ground maneuvers poorly; it left tanks forward to pressure the zerg, and lost them to zerglings, and sent marines back to defend, where they ran full tilt into burrowed lurkers. In the picture, notice that both sides have lost workers and WillBot has already fallen behind before the drop happened. The white dots in the terran natural are lurkers which are in range of the command center.

WillBot did finally clear the attack, but was far behind. Simplicity made 2 follow-up drops to finish off the terran main.

All in all, the mass drop skill could use improvement—but you could say that about almost any skill of any player. In Steamhammer, I have held off on zerg drop because I think it requires tactical analysis that the bot can’t do yet. In this game we see that mass drop can be a scary ability even with little tactical calculation behind it.

Steamhammer still has occasional devastating bugs in its reactions. Steamhammer-Sungguk Cha is an example. Sungguk Cha is an opponent Steamhammer usually has little trouble against. In this game, zerg played a not-very-appropriate lurker rush and handily held the initial marines. The zerg economy was weak, but the tech was there to cope with the next phase of vultures and wraiths, and zerg was in position to fight on. Instead Steamhammer fell into a loop, spawning zergling-overlord-zergling-overlord etc. With massive oversupply of overlords, a broken unit mix, and money piling up with no second hatchery in sight, only terran was in the game.

There are actually 2 bugs. The main bug is that an emergency reaction (“Oh no I have no army hurry hurry spawn stuff”) didn’t recognize that it had already occurred and fired repeatedly, dumping the long sequence of zergling-overlord into the queue. There are checks to prevent that sort of thing, and I haven’t spotted the mistake yet. If I don’t find it soon I’ll let it go until after the AIIDE deadline, because there are more urgent fixes to make.

As often happens, the primary bug triggered a secondary bug—bugs tend to cause unusual conditions, and unusual conditions tend to bring out more bugs. After a few extra overlords, any further overlords should have been recognized as excess and dropped—the idea is to drop planned overlords that are no longer needed after the army is destroyed in battle. That bug I have fixed. The issue is that Steamhammer was still technically in its opening book, though it was nearly at the end of the opening line. The opening book sometimes deliberately makes extra overlords (to save up larvas and produce mutalisks or lurkers in number all at once), so the check was turned off in that case. It was easy enough to add back a check that is loose enough that it doesn’t break openings.

As Willy Wonka said, so much time and so little to do. That must be why I haven’t posted in a week.

The development Steamhammer version still makes many facepalm-level mistakes—the guardians press forward into enemy corsairs while the devourers run away, that level. Even so, testing today shows that its tactical play has improved so much that it can win long difficult macro games against opponents that used to smash it every time in those games. The old blunders where Steamhammer might at any moment throw away its army by suddenly retreating through the enemy, or the like—those blunders are completely gone. In restructuring squads I introduced an all-new stockpile of different mistakes, and those mistakes I have cleaned up enough that they are minor by comparison. Defense squads especially needed a lot of attention.

Since late last year, Steamhammer has been able to play with strong macro. It has finally gained the ability to use the large number of units it can make without wasting too many. It can freely enter into situations that formerly I tuned its openings to avoid. The work took a tremendous amount of time, but the time was well spent.

I still have stuff to fix and features to add, and the time is wearing on. In AIIDE, I hope that Steamhammer can keep up with the field and finish about as high as it did last year. Progress has been blistering, and if I can keep up with it then I feel I’m doing great.

Today, trying to track down a mysterious regression, I noticed this condition in an if in CombatCommander::getAttackOrder():

enemy.type == enemy.type == BWAPI::UnitTypes::Protoss_High_Templar

Hmm, it looks a little different than I intended. == is left-associative, so this always evaluates to true. The effect is that Steamhammer might make a poor decision of which enemy base to attack. The decision is pretty rough anyway, though, so the buggy decision is not much worse.

Gotta love edit slip bugs, they are so creative. In this case, the original intention is also wrong, because high templar are excluded by an earlier check—the condition would have always been false.

If only I could find the bugs that matter, instead of the bugs that don’t hurt much....

One of the earliest improvements I made for the upcoming Steamhammer 2.0 for AIIDE was superior zergling micro. In an equal fight, zerglings on zerglings, Steamhammer tended to come out ahead of Killerbot more often than behind. Now, after many other changes, I have somehow broken it, and Steamhammer loses equal fights. I tried undoing all the changes that seem as though they could be related, but no go. I don’t know what I have done wrong. It’s amazing how fragile some skills are.

Weaknesses are stubborn. The first wipe doesn’t clear away the dirt, you have to scrub repeatedly. I made changes to improve kiting, and hydralisk play is noticeably sharper. But mutalisk micro is worse, because the changes caused seemingly unrelated code to misbehave. I think it will be easy to fix, but what weaknesses did I introduce without noticing?

In another example. I have been systematically working through Steamhammer’s openings to fix faults that have crept in because of other changes: Queue reordering, queue jam clearing rules, mineral locking, and so on. Most openings are OK, but a surprising number need minor adjustments. I also found a subtle one-frame-late timing bug in the interaction between the building manager and the strategy boss that broke a single rarely-played opening. Everything is connected.

Strong interdependence seems to characterize Starcraft skills. When you change one point, it doesn’t matter if it’s a big improvement, you have to also align other points to realize the improvement. As Dan Gant expressed it, a bot is usually at a local maximum in skill. When you nudge it in any direction, it is no longer at a maximum, and you have to hunt around for a new local peak, which you hope is higher.

It makes progress bumpy. On the one hand, that drives me to machine learning: I don’t want to get all these details right by hand! It’s too much! On the other hand, it poses tough problems for machine learning algorithms, too, whether learning holistically or piecemeal. Such a complex and difficult fitness landscape is hard to optimize over, whether by hand or by machine.

The new zerg Simplicity has been getting a lot of games. Taking a peek at the binary, I see that it is a Java bot, and it lives up to its name: The compiled code is quite small, and most of the space used is in libraries.

Simplicity’s play is reminiscent of Proxy; it plays a similar mass hydralisk build with a similar late game addition of ultralisks. It’s a good general-purpose choice for a new bot, so no complaints there. At the same time, it puts on a direct and predictable kind of pressure. Proxy switched away from that build for both terran and zerg opponents, with good reason. (Well, Proxy still goes hydra against terran mech.)

Simplicity is weaker than Proxy, though (including the early Proxy before it was improved). Simplicity has been scoring around 50%, but many of its games are against low-level opposition. I forecast that its initial elo will come in below 1900. Simplicity does not have strong macro, and its tactics are clumsy. Also it appears to build a fixed amount of static defense, not adapting to the situation.

Simplicity does midgame scouting with reconnaissance in force, the same idea as Steamhammer—not common among bots, and unheard of among new bots made from scratch. It looked effective in games. I thought that that was Simplicity’s most impressive feature.

All in all, nothing dazzling, but good for a brand new bot, especially such a small one. After only one day, the bot got an update that I thought improved it, so development is proceeding rapidly. Any bot that improves rapidly has a chance to climb high.

Next: Analysis of potential paths.