Starcraft AI blog

My plans for replacing the squad infrastructure are in a vague state. Here’s what I’m thinking.

• Above all, turn the worker manager and scout manager into ordinary squads. All the special cases of delegating units to managers should go away, subsumed by a general assignment mechanism. Other design features follow from that decision.

• A new bookkeeping class for squad information. SquadData as it stands seems awkward to me.

• Possibly create a hierarchy of squads. In that case, the root is the supersquad of all units and acts as the bookkeeping class for its children, which are the base-level squads that exist now. Each squad could subdivide its units into subsquads with different roles or purposes. The future worker squad might have a subsquad for each base to mine, a combat squad might have subsquads corresponding to the current unit micro managers, and so on.

• Alternatively, have a flat squad structure but allow roles in each squad to be assigned by a tag attached to each unit. This allows for the same features, it’s just a different code design. I’m still thinking through which idea makes better sense. I can think of a case for allowing both....

• In either case, provide the features through a tidy object oriented design so that a combat squad doesn’t have to worry about accidentally assigning a unit to mine gas. Look up the names of the design patterns if you want; I only remember the patterns themselves. No matter the details of the design, the current unit micro managers will become more general role managers. The way that role managers coordinate with the larger squad seems like the key to working out the undecided design details. I’ll need to work through cases in detail to foresee what’s needed in the implementation.

• Information wants to flow both up and down. Top-down control means that the tactics boss gives tasks to squads, which in turn give more detailed tasks to its units. Bottom-up control means that units examine their environment and decide what to do accordingly, perhaps self-organizing into larger task groups. As I mentioned in a comment yesterday, you want both. Units should examine their environments in light of their tasks, and the information they produce should work its way back up to the tactics boss. I have to think more about how to manage the upward flow of information.

• Currently, some bits of code that aren’t much interested in unit control end up issuing orders directly to units: The building manager takes temporary ownership of a worker to construct a building, the combat commander directly orders comsat scans on its own initiative, and a few more cases. It probably makes sense to consolidate all these somewhere in the squad code. Put all unit orders in the same module and it will be easier to cope with latency frames and the like. That will involve adding some mechanism to issue these more detailed commands to a squad.

Whatever the decisions, the goal is cleaner, more regular code that at the same time better supports the features needed for good play. More regular code will make it possible to apply machine learning uniformly across tactical and micro decisions. The purpose of investing time to rewrite working code is to earn back that time in the long run and reach the goal sooner.

UAlbertaBot’s squad structure, still used in Steamhammer with little change, is not rich enough to easily support good play. It can be made to work, but things get awkward. When I reach the 1.5.x series of versions, which will be about strong mutalisk play, I intend to do a big rewrite, to make it easier to support good tactical play and good cooperation between units in a squad. I’ve been thinking about what the plan should be.

Today’s post is about the awkward points in the squad infrastructure. Next I’ll write up my thoughts on how to do it better.

Too many special cases. In Steamhammer, in theory every unit is assigned to a squad. I’m not sure it’s true in practice, but it’s close. Workers busy mining or building are in the Idle squad, because by 1984 doublethink, work is idleness. Well, I guess the real reason is that workers are not controlled by orders to their squad, but by the worker manager (or briefly by the building manager)—the squad itself is idle, though its members mostly aren’t. The worker manager has its own special case task system, and assigns workers to mine minerals or mine gas or repair or whatever. A worker sent to fight is taken out of the Idle squad and put into a combat squad. The squad code itself has a special case to treat the Idle squad a little differently.

Another special case is the scouting worker, and for zerg, the scouting overlord. Their movements are controlled not by the squad, but by the scout manager. The scout manager also has its own system to keep track of goals and tasks. I would like to unify the different systems at least enough that they can be seen as different variants of the same thing, not as unique subsystems that share little in common.

Weak squad orders. The special cases come about because the squad order system is not powerful enough to represent all the goals and tasks needed to control workers or scouts. That means it’s also not powerful enough to represent everything that a squad wants to do in combat. There are a bunch more special cases in the micro managers that are part of the combat squad code, working around the limitations in ad hoc ways.

Internal coordination. Suppose that Steamhammer’s marines and tanks meet mass sunkens and decide to attack. The tanks siege out of sunken range and the marines charge in. It’s fine if the force is strong enough to defeat the sunkens with marines left over, but if the marines all die, then zerglings or mutalisks kill the tanks. Something similar happens when vultures and tanks meet photon cannons. The squad splits up its members into unit types that may behave differently, but it provides no mechanism for the unit types to cooperate depending on the situation. There are many, many cases where Steamhammer comes to grief because its squad members don’t coordinate among themselves, and without any general mechanism, each case has to be implemented from scratch.

With the current code, it’s somewhat awkward to implement features like positioning other units in front of sieged tanks to provide vision. The division of unit behaviors by unit type is not helpful here; you should be able to pick any suitable unit to provide vision, whether a floating building like Krasi0, or wraiths, or vultures, or marines. We care about the role of providing vision, not the unit type playing the role. And similarly for many other combat roles.

Coordination between squads. Steamhammer’s main combat squads are the Ground squad and the Flying squad, separating ground and air units because they often want to act separately. But when they do come together in the same place with the same goal, they still don’t cooperate; see limitations of combat simulation (tactical coordination section). The squads could be merged together in this case, but that doesn’t improve the behavior! The internal coordination weakness is just as bad.

Different roles with the same goal. Here is another example of poor coordination. When new fighting units are produced, they are put immediately into a combat squad, but they may start out far from the front lines. They are reinforcements and have to make their way across unseen terrain to reach the front lines. Steamhammer has almost no understanding that the different units in the same squad are in different situations and need to do different things—only a few bits of special case code to provide different behavior. It often loses the reinforcements unnecessarily, or sees them distracted by passing targets.

One fix would be to break the units into two or more squads and give each squad a different order: The vanguard squad fights as usual, the reinforcement squads attempt to join up (or are simply given the order to attack the same target as the vanguard squad, so that they end up going to the same place and then are merged, perhaps by a clustering algorithm). Another fix would be to retain the single squad, but allow it to be partitioned into a vanguard group and reinforcement groups. The 2 plans differ mainly in the code organization; they’re the same otherwise.

Next: My ideas for fixing these issues.

If a skill is effective in games and not too hard to implement, and yet isn’t implemented in many bots, then surely it is an underused skill. For example, I think that evacuating workers from a base under attack is underused. It is easy to tell whether to evacuate workers: If you have a combat simulator or an army strength estimator or any method to tell whether you’ll win or lose a fight, then run it when the enemy reaches a base of yours, at least if you have another base that can keep mining. It is not as easy to tell where to evacuate workers to, since they might flee toward the enemy army. But you know the current place is bad, and running anywhere else is probably no worse.

Steamhammer doesn’t evacuate workers, and suffers for it. It’s high on my list. The 7 bots above it on the bot ladder do, except I think for XIMP by Tomas Vajda; Steamhammer may be the strongest bot missing the skill. Killerbot by Marian Devecka is particularly good at it, if as ask me. Most other bots don’t; it’s a fairly rare skill, which is why I say it is underused.

There is a special case that is even easier to implement: Terran can evacuate SCVs by simply lifting the command center. If terran has another command center landed anywhere else (reachable by land), then the SCVs will return their cargo there. If not, the SCVs will stay near the minerals or gas, not the best place to hide but better than nothing. As a side benefit, if the attacking force includes units that can’t shoot up, the command center will often take longer to destroy—even if it only floats there and doesn’t move away.

PenaBot by Henri Kumpulainen used to lift its command centers, if I remember from so long ago. Iron does it sometimes when losing, though I don’t think the purpose is to evacuate SCVs. LetaBot by Martin Rooijackers will float a command center, but only to move it into position. I can’t think of any current bot that lifts its command center to protect itself.

I nominate lifting the command center for safety as the #1 most underused skill. What do you think are underused skills?

New terran bot WillyT is easier to write about than GuiBot. I can’t see anything in common with WillBot. “T” in “WillyT” might stand for terran, and “Bot” in “WillBot” is semantically empty, so the names are nearly the same—but apparently it is a coincidence. The internet knows a bunch of things named Willy T. Anyway, WillBot is Java and WillyT is C++.

WillyT plays similarly against all races. It builds 2 barracks, gets medics, stim, and marine range, then goes for an all-in attack with marines, firebats, medics, and 8 SCVs. It’s a scary attack; it beat Tscmoo random which rather randomly decided to make carriers. ForceBot made 3 sunkens a little late and could not hold, while Arrakhammer diagnosed the situation and made 5 sunkens. WillyT sends reinforcements as it produces them, but the attack truly is all-in. If the opponent spoils it ahead of time, or is ready and holds it off without much damage, then WillyT has little chance. As I write, WillyT is ranked a little below average.

Its unit mix seems to vary by the race of the opponent. Versus terran, it mixes in a few firebats. Versus protoss and zerg, it makes far more firebats, I guess expecting mass zealots or zerglings.

WillyT has some micro skills. It scans for cloaked units. It has a win over Stone, so it has an idea how to defend itself against tricks. It keeps marines in the mineral line until it is time to move out, somewhat like Tscmoo, for safety against rushes and to make sure that the opponent can get in to see everything. After it moves out, it makes attempts to keep its force from spreading out too much.

WillyT doesn’t expand. If it destroys the enemy main, it doesn’t know how to search for more enemy bases. In this absurd game versus NiteKatP, WillyT destroys the protoss main and then neither side can figure out what to do next.

WillyT is the opposite of GuiBot in a way. GuiBot will benefit from bug fixes; WillyT will benefit from new features so it can cope with a wider range of situations.

Today I discovered that Steamhammer’s OverpoolSpeed opening is broken in version 1.4.1. It was still correct in the previous version 1.4. I had been experimenting with an overpool +1 build similar to the one in Killerbot by Marian Devecka. Somehow a weak, unfinished version of the +1 build got into the distributed configuration file under the name of the overpool speed opening. It must have been a copy-paste slip or something.

It’s fixed for the next version, of course. With so many openings, how many others have gross mistakes? Few, I hope, but I know there are many that could benefit from refinement.

GuiBot has been getting extremely frequent updates, so anything I say about it may be obsolete by tomorrow. With that caveat....

Protoss GuiBot crashes in a large proportion of its games. When it doesn’t crash, it often performs poorly. On SSCAIT, it is currently ranked 4th bot from the bottom, making it extremely weak. But GuiBot shows some promising signs. It knows how to use a wide variety of units, including complex units like shuttles, reavers, and high templar. It gets many upgrades in a sensible order. Some of its skills are excellent. If the author can fix the crashes and a few of the biggest weaknesses, GuiBot will move quickly up the ranking.

I think GuiBot’s favorite opponent race is zerg. Against zerg, GuiBot opens with a forge expand build. It likes to place the forge beneath the gateway in a ling-tight arrangement, the most popular placement among human players, and for some reason unusual among bots. I imagine that the author knows the game. It also places the cannons farther behind than necessary, which is odd—it makes busts easier. Zealots plug the gaps in the wall. It makes corsairs and sets about clearing the skies of undefended overlords. A strong start.

Then when it comes to the fight, GuiBot tends to fall down. When the corsairs meet mutalisks, their natural prey, the corsairs don’t win as they should but fly around in panicked curves until they are shot down. They were serious when they said it was a good day to die. (Probes have a similar panic behavior, losing too much mining time.) Zealots group into a long line and chase stray enemies this way and that without making any decisive attack. If GuiBot had good combat skills, it would become strong. In this game, just played, ForceBot busts GuiBot’s wall, taking advantage of the rear placement of the cannons and the overeagerness of 2 zealots.

I don’t want to draw conclusions about GuiBot’s mix of openings. I have seen it play different builds, but the bot is updated so frequently that I can’t tell whether it is GuiBot or its author that is choosing the build.

I think GuiBot’s favorite unit is the shuttle. It likes to build a robo facility early, and the first thing it makes in the robo fac is a shuttle. If shuttles are shot down, it makes more shuttles. 1 or 2 shuttles accompany the army, and it uses them to micro units in battle: When a unit is in danger of dying, the shuttle picks up the unit and pulls it back a little before letting it re-enter the fight. This is a unique skill among bots and can be valuable, but I don’t think it is worth making a shuttle just for that.

Once reavers come out, somewhat later than I expected in watching the games, GuiBot’s shuttles switch to reaver dropping. GuiBot is good at it, about the same level as other good reaver bots. It does have a tendency to spend too much time at home waiting for new reavers to finish; a shuttle carrying 1 reaver will drop what it’s doing and run home as soon as a second reaver starts in the robo fac. Reaver dropping is not a rare skill any more, but it’s still deadly. The bot’s reaver targeting looks fairly good too; it doesn’t only shoot at the closest enemy, like Steamhammer or CerkoBot by Tomas Cere. See this game versus Ecgberht for an example (though the game was decided in a boring way, by a production freeze).

GuiBot also knows how to use psionic storm, though like most bots it is not skilled at the when and where. But a bot could use 100 good skills in a row and lose the game in the next moment with one big mistake. As I've said before, the law is: Adding features decreases elo; fixing bugs increases elo. I know from experience that after adding a big feature that will be valuable in the long run, I may have to fix a lot of problems to get back to zero in the short run. The greater part of bot development is not about adding skills, it is about fixing the weaknesses in the skills you have, and in the interactions between them.

I decided I was tired of seeing Steamhammer lose helplessly to Iron’s vulture runby and Tscmoo’s factory openings. Terran can start with vultures and force zerg to defend: Zerglings quickly become useless, drones are at risk, and terran can run by defenses into the main as Iron does. Steamhammer’s strategy boss understands that a sunken is needed at the front, and has a vague idea that hydralisks may be useful if the book line happens to end, but in the end it doesn’t defend adequately. Then terran can follow up with more factory units or with wraiths, and zerg has to be ready for a range of possibilities. Against Steamhammer, Tscmoo likes to follow up with goliaths, which is effective against Steamhammer’s mutalisk openings. Too many crushing losses.

Zerg has a standard way to defend: Place a sunken at the front as a vulture obstacle and keep a modest number of unupgraded hydralisks in a defensive position. The hydras are general purpose and can fight off vulture runbys, wraith harassment, small drops—all the natural followups. Zerg then continues to mutalisks as usual. The mutas are delayed, but terran is playing a slow strategy so it’s OK.

Today I set about developing an “AntiFactory” opening that implements the standard defense. I decided on development tests against Iron to see whether the runby defense works. In the very first test, when the opening timings were still all screwy, Steamhammer put up a persistent defense and didn’t lose until well into the middlegame. An improvement already, good. I made a few fixes and tried again. Steamhammer held the early aggression cleanly and fought a long level game until Iron finally mined out and collapsed.

I did not expect that! Steamhammer lost drones by the barrel and miscontrolled scourge and chased 1 SCV with 3 ultralisks, all its usual mistakes, but Iron struggled too and could neither stop zerg from expanding nor keep a fourth base up. I guess Iron depends heavily on the success of the early vulture runby.

Well, one win does not make a champion. We’ll see how it goes as I polish the opening. It’s just that the test game was so long that by the time I finished watching it, I had forgotten what needed fixing in the opening....

When the opening is ready, will the opponent model be able to select it when it should? I think the answer is no. I think I will have to extend the plan recognizer to recognize early vulture plans. Then Steamhammer should be able to counter the vultures in the second game, and ideally cope with the followup. It would be nice if Steamhammer could get back into the learning war against Tscmoo terran, and come out on top this time.

AntiFactory branches fairly late from related openings. When the opening tree is available, Steamhammer should be able to recognize the vulture plan during the game and choose the right build on the fly, more like a human player. It might need some scouting improvements first, though.

Next: The new bots GuiBot and WillyT.

I like the BWAPI ladder. It doesn’t seem to have an official name; I’ll just call it “the ladder”.

I’ve enjoyed following the games. I play over most of Steamhammer’s games on the ladder every day. The ladder makes random pairings, so it feeds me a wider variety of opponents and I see more strengths and weaknesses. Also the ladder plays more games in total, because it plays games at full speed, not slowed down for streaming.

Providing accurating rankings and elo is the primary purpose of the ladder, at least as I see it. With a diet of randomly chosen opponents, Steamhammer’s rank stabilized at #8, behind CherryPi and ahead of TyrProtoss, with elo steady in the 2230s. On SSCAIT, the same version’s ranking is not stable—it has varied by over a factor of 2, with the elo tending to rise into the low 2200s, then fall rapidly, then slowly rise again, depending on the whims of the voters. When the voters see Steamhammer with a high rank, they tend to pair it against opponents that it will lose rating points to; after it has lost the rating points, they tend to pay less attention. Steamhammer ends up below its equilibrium elo, and the popular opponents that defeat it end up overrated. The ladder pairs bots fairly, so it better predicts tournament performance.

Randomhammer’s rank can’t be compared across competitions so neatly, though, because the competitions treat random players differently. The difference in rules makes it less useful for predicting the tournament performance of a random player.

File I/O seems to work a little differently in each competition. They are all based on Dave Churchill’s tournament manager software, but each competition uses a different version or tweaks it differently, and the behavior is not exactly the same. They all share in common a read directory and a write directory, with read-only access to read and write-only to write, and copy the contents of write to read. They differ in whether and/or when they clear directories. AIIDE proceeds in all-play-all rounds, and clears write at the end of each round of many games. SSCAIT and the ladder proceed by single games, and can’t do exactly the same thing. I believe that SSCAIT never clears write. I don’t know what the ladder does, but it has different behavior, and Steamhammer’s code doesn’t work correctly.

Steamhammer’s problem, I saw immediately when I requested and received the stored data, is that its record of games against each opponent only extends back 1 game. Instead of the whole history, the opponent model has to draw conclusions based on the one previous game. Data is being cleared at some point; perhaps write is cleared before each game. Steamhammer appends data to the opponent’s file after each game, which works on SSCAIT. I think if I change it to rewrite the entire data file (originally read from read), instead of only appending the new game record, it will work everywhere, including the ladder and the AIIDE tournament. I won’t know for sure until it happens, though, because the details are not documented. The change will be in the next version, 1.4.2.

Call it a bug in Steamhammer. The bug means that Steamhammer’s rank and elo can’t be compared between SSCAIT and the ladder, even though the opponents are mostly the same in both. It’s possible that Steamhammer plays better with the bug, so its higher rank on the ladder is justified. The point about the stability of the rank stands, though.

I’ve narrowed down some of the decisions of what to do in the 1.4.x series, which is about improving the opponent model until it becomes more clever than Clever Hans. Of course my plans tend to change frequently, so no promises. This is what I’m thinking now. I wrote up less specific plans in February.

As always, I’ll do a lot of things that I don’t call my primary goals. Here are some that I feel pretty sure about.

• Make some necessary improvements to the plan recognizer.
• Speed up and improve building placement. In particular, fix the bug with protoss.
• Make a start on reworking the production system to accept more general goals. I thought of ways to take it step by step.
• Try a few experiments with squad control, so that I have a toe on the ground when I move on to the 1.5.x series, which will be about mutalisk control. I might teach units to hold position in locations where they can hit targets from outside enemy range, for example, which seems like an important basic skill.
• Protoss shield batteries.
• Overlord safety! Steamhammer loses far too many.
• At least 1 new terran skill and 1 or 2 new zerg skills, just to keep progress up.
• Probably a few skills made easier by the new base and map code, such as running workers away from a base under attack.

There are also things I want to try but feel less sure about fitting in. There’s already too much. One is Antiga’s ideas about ramps and chokes and stuff.

1.4.2

• Fine-tuning of opening choice: Tend to repeat successful openings, regardless of other considerations.
• The opponent model will try to understand whether the opponent plays a single fixed strategy, or varies its play. Just knowing that gives a lot of leverage to learning.
• New base placement code, a major step in dropping BWTA.
• Make the opponent model work on BWAPI bot ladder and other tournaments. There are slight technical differences between SSCAIT and other competitions in when directories are cleared.

With Steamhammer’s large number of opening build orders, it would be very slow to learn which one to use against each opponent using an undirected algorithm like UCB. Every opening would have to be tried a few times—even a long tournament like AIIDE would not be long enough to find good openings against all opponents. We have to use knowledge to direct the search to try more promising builds first. The instant learning of the opponent model is one piece of that knowledge.

For this version, fine-tuning will consist of noticing which openings have been more successful against this opponent, and tending to repeat them. I may also add exploration, but only a limited amount because there is too much to explore.

1.4.3

• Collect opening data.
• Start to adapt opening choice to the maps.

Another piece of knowledge to direct the search for a good build is knowledge of which openings are similar to each other. If a lurker rush opening gets smashed hard, maybe try a mutalisk opening next, or at least a slower mass lurker opening. That’s only the simplest way to use the knowledge. I want Steamhammer to collect data about its own openings in real games, so it learns the timings of when production and tech get completed, and the variation of the timings due to maps and to pressure from real opponents. It’s a model of each build, you could say. The bot should be able to compare the opening models against the opponent model, so it can choose an opening likely to upset the opponent’s plan.

1.4.4

• Represent the tree of openings, and make choices at each branch.

I expect to implement the opening tree for zerg only at first. I think that by combining the opponent model, the opening models, and the smarts of the zerg strategy boss, it should be possible to make good enough choices at each branch point in the tree. The idea is that the opponent model tells us what the opponent is planning, the opening models tell us what we’ll do for each branch we might choose, and the strategy boss understands counters and can tell which result is best. I could be wrong, and in that case I’ll have to find more knowledge to add.

If successful, this could be an amazing skill. I’m determined to try. We’ll see how it goes!

Steamhammer 1.4.1 source and documentation are now available on Steamhammer’s page.

I uploaded Steamhammer 1.4.1 today. As promised, the occasional crash bug is fixed—I never identified it, but I can no longer reproduce it, so I think my changes have fixed it or at least made it much less common. Also as announced, the primary goal of improving the opponent model is put off until the next version, 1.4.2. New map code is largely written, but it is not finished so it is turned off. As always, I have made a lot of minor fixes and improvements, because working on what I said I would work on would be almost like having a job. Key fixes are to limit protoss gateways to 10 (idea borrowed from Locutus by Bruce Nielsen) and a serious zerg bug in choosing the tech target, which suppressed tech switches in the midgame.

People will want some of these fixes, so I’ll put up source approximately tomorrow.

maps

• Fixed a crash on a certain seemingly messed-up version of Bloody Ridge.

• Nicer code for distance maps, borrowed from a later UAlbertaBot. The new code is much cleaner and less bug-prone.

configuration file

• The debug option DrawBWTAInfo is replaced with DrawMapInfo, since I’m preparing to drop BWTA. For now, DrawMapInfo doesn’t do anything, since the new map code is turned off. The old BWTA info drawing code is removed.

• Bug fix: "n x item @ macrolocation" finally works, e.g. "2 x photon cannon @ natural". It was a bug in parsing.

• Added a new macrolocation @ center. Don’t use it though, except maybe on specific maps where you’ve tested it out; it doesn’t work consistently due to building placement bugs. If you ask for 2 buildings in the center, you may get 1 or none.

opponent model

• A new strategy reaction for all races: If the opponent is following a heavy macro strategy and we are still in the opening, drop any planned static defense. We won’t need it right off. Static defense added later (as a reaction to seeing scary enemy stuff) is built as normal.

tactics

• A squad that can’t find any buildings or units to attack has always been sent to explore the map. The change is, if the squad includes air units, it can be sent to explore tiles which are only reachable by air. It will help... if any opponent ever expands to an island or builds on a cliff, which hasn’t been seen so far.

• Steamhammer decides which enemy base to attack based on how much static defense it has. The change is, the enemy static defense is counted only up to a smaller distance. The longer distance was causing some mistakes in choosing targets.

• Bug fix: Units like scourge, which could belong to either the ground squad or the flying squad, could sometimes be mistakenly stuck permanently in the ground squad. The change is to CombatCommander::updateAttackSquads(). I saw a game today where this change would have helped, so I’m hopeful that it is a useful fix (but it’s certainly a fix).

micro

• Melee units (other than workers) prefer targets under dark swarm, in hope of also staying under swarm. This mainly helps protoss for now, but it is also preparation for when Steamhammer gets defiler support.

• Dark templar target zerg spore colonies with higher priority. I saw a bad game.

other stuff

• A few more opponent-specific strategies were dropped from the configuration, after experience showed that they were not needed any longer. My plan is to drop the remaining ones in the next version, 1.4.2, and rely solely on learning.

• Seeing a unit in the act of burrowing now tells Steamhammer that the enemy has burrow tech. It used to have to detect the burrowed unit to know. So many details to catch....

• The old ProductionManager::goOutOfBook() is renamed to goOutOfBookAndClearQueue(), which is what it does. A new goOutOfBook() goes out of the opening book if we are in it, and nothing else. This simplifies a few bits of code and makes them easier to understand.

• The unused BuildOrderQueue.queueItem() is removed.

terran and protoss

• Emergency response: If we have fewer than 3 workers, urgently make more if possible. This provides a small chance of surviving an extreme emergency—maybe the opponent is broken too.

• New 21Nexus opening, borrowed from Antiga.

• In tank openings, get vulture speed earlier. It was too late.

• Limit gateways to 10 total, an idea borrowed from the excellent Steamhammer fork Locutus. There are 2 benefits: 1. BOSS likes to order far too many gateways, so limiting the total prevents unnecessary expenditure. 2. What’s even more important is, it works around the building placement bug that causes the bot to exceed the per-frame time limit and lose games. The bug comes up when Steamhammer runs out of space to add buildings, so adding fewer buildings prevents it from happening. It is a super successful mitigation; it works better than all my other tries put together. In fact, I’ve been thinking of removing the vertical building placement workaround, which now probably causes more problems than it solves—maybe next version.

zerg

• When dropping excess upcoming hatcheries, count unstarted hatcheries in the limit. Steamhammer still often makes excess hatcheries, but it’s a little less severe. Steamhammer can’t time its hatcheries accurately, because it depends on future production timings and future unit mix, which Steamhammer doesn’t try to predict. The heuristics it uses instead are usually not far off, and this change is a small improvement to one of them.

• Supply counting for overlord production simulates how supply changes as queue production proceeds. The simulation is not exact, because it doesn’t account for timings such as how long hatcheries take to finish, but it is as exact as possible otherwise. This makes it possible for opening lines to order overlords at exact points and expect that runtime execution will follow the instructions. In practice there’s no gain for most openings, because overlord production was already fairly accurate. For occasional openings that depend on producing later overlords at precise timings, the difference is dramatic. Overlord production outside the opening is not improved, because overlords aren’t queued ahead of time then. (They aren’t always queued in the opening either. It’s optional, and overlords are only included when they are to be made at specific timings.)

• Important macro improvement: In midgame and later, make overlords faster. This fixes Steamhammer’s tendency to repeatedly supply block itself once the drone count grows over 40 or 50. Steamhammer still has some zerg macro weaknesses, but they are getting to be so slight that they aren’t worth attention for now. Well, except for one problem where it accumulates minerals versus terran....

• If the lair, spire, or hydralisk den is lost or cannot be made, break out of the opening. The plan has gone awry, so give it up and hand over control to the strategy boss.

• If the enemy has air tech or cloak tech, order a lair even if we don’t otherwise need a lair yet. The lair is a precondition for existing rules to fire, adding a spire or researching overlord speed to react to the enemy tech. I added this after watching Steamhammer lose a game where it thought that fast hydras with slow overlords were a good counter to dark templar (they could be good enough if it knew the right way to use them...).

• If hydras are needed in a hurry, get hydras in a hurry. If Steamhammer finds that zerglings are bad and hydralisks are good, and lair tech is still far away, then get hydras immediately without considering the longer term plan. This is a simplified way of accounting for the time it takes to get tech, as well as the value of the tech. In some future version, Steamhammer will calculate the time to get each tech and the urgency with which it is needed, and take all into account uniformly.

• Critical bug fix: Tech target calculation was wrong because a condition was coded as “if we have hive” instead of “unless we have hive”. This stopped many tech switches that should have happened in the middle game. This should improve play in many games, especially versus terran—though I have seen one or two games where Steamhammer probably played better because of the bug. Steamhammer’s understanding of tech switches remains shallow.

• Slight tweaks to ZvT tech choices.

• All zerg openings are updated to allow sunken colonies to auto-morph.

• The overhatch openings played versus random are switched from the in-base hatchery to the expo hatchery versions. The expo hatchery is better versus terran and protoss and still playable versus zerg, so it should be an improvement.

• New openings: Over10Hatch1Sunk ZvZ_12PoolMain ZvZ_12PoolLing 2.5HatchMuta. The openings offer new choices against all races and should make Steamhammer even harder to prepare against. Also renamed a few existing openings with more specific names, to fit with the new ones.

Coming soon (in one order or another): More concrete plans for the following version. The new bot GuiBot. Experience with the BWAPI bot ladder. Squad structure.

Time for another Steamhammer update. The key bug still has me frustrated. Some of my recent progress:

Base placement. The UAlbertaBot base placement algorithm that I borrowed works perfectly on most bases, but turns out to have a flaw. For some unusual base layouts, it places the resource depot far from the minerals. For example, all other bases are good on Moon Glaive, but the 1 o’clock base sets the geyser at an unusual angle to the intended base location. The algorithm places the base level with the geyser and far from the minerals, away from the intended location.

I’ll look into it more before deciding how to fix it. I might keep the algorithm as is and add an optimization step that slides the base to a better location. Or it might be simpler to ditch it and use a distance minimization measure directly.

New openings. Looking ahead to the near (ish) future when Steamhammer gets its tree of openings, I added 4 new openings. In the tree, they will be forks of existing openings and will offer opportunities to make late decisions based on scouting information. I added 1 ZvT, 1 ZvP, and 2 ZvZ choices. Before the tree arrives, I’m likely to add more forks. It would be natural to have a wide variety of overpool followups for different situations.

Even in the upcoming version, once I add fine-tuning of opening choice to the opponent model, the new openings should be worth having. They will counter some opponents more sharply.

Squad mixup bug. Today, tired of other things, I arbitrarily decided to fix a bug in squad membership. It’s not an important weakness, but it’s been bothering me for months, so I picked it.

The problem is that sometimes units that should be in the flying squad get stuck permanently in the ground squad. 3 unit types are special for squad assignment: Scourge, devourers, and carriers. If there is a flying squad with other flying units, then these types go into the flying squad, otherwise into the ground squad. The theory is that these units should coordinate with whatever squad exists—especially devourers, which are support units. The bug is that, if one of these units is assigned to the ground squad—usually because the flying squad was temporarily wiped out—it stays there. When the flying squad is reconstituted, it doesn’t move back.

The bug was easy to find, but I decided that the underlying problem was that the code was too complicated. I simplified and rewrote it until it was Obviously Correct and there could not be any bugs left. Nevertheless, there was one bug left, and I had to fix that too. Not quite simple enough; maybe next lifetime.

I’m not sure whether this fix improves strength. If you’re playing with mutalisks and zerglings and the two tend to act independently and go to different places, maybe scourge should not always be with the mutalisks. At least, stream watchers are not likely to notice the different behavior; it’s hard to see unless you know what to look for. Anyway, now it works as designed, and since the code is clearer it should be easier to change the behavior if it turns out to be weak. Still a net improvement.

I like the new protoss bot Locutus by Bruce Nielsen. It is a Steamhammer fork which uses BWEB for building placement. That means it includes BWEM, which BWEB depends on, though I don’t notice any use of BWEM for pathfinding. (If it used BWEM for pathfinding, probes would not have gotten stuck behind a mineral block in this game against Bereaver.) It also has some good extensions to Steamhammer.

Locutus has a favorite build order that it uses against all races, a forge expand build that was originally designed for use against zerg. It uses BWEB to place a reasonably good wall with forge, cannons, and gateway in front of the natural, not a wall that is optimized for the terrain but one that is about as good as the walls of other protoss bots. As a new bot, it has little ability to adapt its unit mix to the situation, so it follows up with universal units, dragoons. Like Steamhammer, it eventually adds reavers and observers and a few dark templar. The dragoon micro inherited from UAlbertaBot is one of Steamhammer’s best skills, so that’s another reason dragoons are a good choice.

Locutus does know other build orders, and uses certain ones against specific opponents. [Update: I misread the configuration! It does not actually play enemy-specific build orders; the feature is turned off.] It does not use Steamhammer’s opponent model.

The forge expand build was designed for zerg opponents, but in bot world it works well against other races too. Locutus has wins over strong opponents like Bereaver and TyrProtoss. Building cannons willy-nilly has an advantage: The cannons stop most early rushes, common bot strategies. For example, PurpleSpirit tried its SCV-marine rush and failed hard.

Also visible in the PurpleSpirit game is Locutus’s changed probe micro. Probes come to defend the wall when necessary, a critical skill that Steamhammer does not have. Unfortunately, they also come to defend when it is not necessary, and Locutus tends to send too many probes and delay mining too much. Sometimes Locutus pulls probes into the middle of the map and loses them for no apparent reason.

Locutus appears to still be using BOSS for production, because it adds more gateways than it can keep busy, a typical BOSS misbehavior. But Locutus has good macro, so if so, it is using BOSS more carefully than Steamhammer (I’m going to dump BOSS so I don’t care about using BOSS carefully). Locutus can almost keep up with BananaBrain in macro, which points to good skill.

Using BWEB for building placement means that buildings are laid our more nicely than in Steamhammer. Locutus’s base is more pleasant to look at and fits more buildings. It is still possible for dragoons to get trapped between buildings, though. It may be a bug in BWEB, or in Locutus’s use of it. The 2 dragoons on the left in the picture cannot get out.

Locutus retains Steamhammer’s recon squad that performs reconnaissance in force. It’s usually good at finding and knocking down undefended expansions. Locutus also retains Steamhammer’s inability to defend its own expansions, leaving it vulnerable to an opponent that also scouts and destroys bases.

Looking into the configuration file, I see 2 nice extensions to Steamhammer’s configuration syntax. The build order is given in the usual concrete Steamhammer way, but a new location allows it to place buildings in its wall. I may adopt the same syntax for Steamhammer itself, when I get to walls.

"pylon @ wall"

There is also new “then” syntax that lets you chain actions together. I take it to mean “then with the same probe, do this next action.” Otherwise the code will assign a new probe, causing inefficient probe movement. You can chain as many actions as you like (or at least, I see an example where Locutus chains 3 actions). I can only assume that it is smart enough to assign a new probe if the first one is lost in the middle of the chain. This is a cool idea that I didn’t think of. It only seems useful for protoss, though.

"pylon @ wall then go scout location"

or

"photon cannon @ wall then photon cannon @ wall"

Overall, Locutus is a promising new bot. It has good ideas and good execution, and it already plays well. The author’s note says that new builds are coming. As Locutus gains more skills, it could become fearsome.

Steamhammer has 2 favorite lurker dysfunctions. One of them is kind of interesting.

Its #1 favorite blunder happens versus protoss. Lurkers act somewhat independently of the squad they belong to, because their burrowing ability lets them be more aggressive. But they’re not clever about it; they don’t understand detection. When a lurker sees a photon cannon, it understands it as a high priority target, walks up to it, and burrows in range. Then the cannon kills it. Not very interesting. I haven’t thought of a way to fix it without adding or rewriting too much code, so I put it off. I’ll fix it when I work on micro, likely in the 1.5.x series while I’m doing mutalisk stuff.

The second favorite blunder is less common but more interesting. It usually comes up versus terran. Lurkers approach a ramp with a bunker on top. They can’t see uphill, but when they get in range the bunker starts firing, so the lurkers can see the bunker and they burrow and start shooting back. Then the bunker can no longer see the lurkers, so it stops firing. The lurkers no longer see their target, so they unburrow, becoming visible... and the cycle repeats. Terrans that know how to place a bunker on top of a ramp generally know how to repair it, so the bunker usually wins. If the overlord catches up to provide vision, the overlord flies into bunker range and holds position... and doesn’t provide vision for long. This game versus Ecgberht shows it happening twice, before another lurker attack finally breaks the ramp because there were enough zerglings.

To fix the overlord behavior, overlords have to worry about threats. That’s on my menu for soon-ish. I may fix the lurker misbehavior with a rule like “if you’re on low ground and the target is on high ground and you want to burrow, move to burrow on high ground if possible,” that is, on the ramp. Ideally, the combat simulator would be able to tell whether the lurker can burrow successfully. Eventually I want to teach Steamhammer how to reason about visibility, instead of following rules.

I made an improvement to overlord production in Steamhammer, and it reminded me: Making overlords is a difficult skill. Supply is tricky for terran and protoss too, but zerg overlords are trickier because overlords have more uses and are more vulnerable.

When to make overlords. If you know ahead of time what units you will produce when, you can plan overlords at exactly the right times. Of course, you usually have only a general idea. If you scout that the enemy is getting mutalisks much earlier than you, then you have to make spores and that is that; an overlord in the egg will turn out to be unnecessary.

The tactical situation matters: Are corsairs out? Are you planning a doom drop? If you’re at risk of losing overlords, you may want extras to avoid being supply blocked. Ideally, bots should weigh the risk of supply blockage (which depends on events in the game) against the cost of extra overlords. I don’t know whether any bots do that; Steamhammer certainly doesn’t. When CherryPi loses overlords and gets supply blocked, it likes to hugely overproduce overlords and try to stay ahead by sheer volume. I don’t think that’s good play, but it seems at least as successful as Steamhammer’s method of producing the minimum number of overlords.

Conversely, if you find yourself ahead in supply, you may be able to take bigger risks with your overlords, at least for a while. Sacrifice one to scout and you won’t miss it.

The strategic situation also weighs in on overlord production. If you get supply blocked, instead of producing overlords, you could start a fight to reduce your army size. You want to compare the cost of the fight versus the cost of the overlords. Even if you’re not supply blocked, the cost of buying supply to feed your growing army can provide some impetus to attack. If you suddenly lose a lot of army in battle, you may even want to cancel overlords in the egg.

The result of all the complications is that bots have to rely on heuristics or approximations. Most bots do well enough with overlord production despite living by simple rules of thumb. But the stronger the bot, the more the fine details matter.

Where to make overlords. Since overlords have uses, it also matters where you produce them, especially before overlord speed is researched. In the early game, strong players plan ahead where to place and how to maneuver overlords to see as much as possible of key areas. Where each overlord is produced figures into the plan. Later, bases and armies may need overlord coverage to spot dark templar or to gain vision of high ground. New overlords should be made near where they will be needed—unless other units are higher priority. If corsairs are marauding, most overlords may have to be produced near air defenses (lacking this skill hurts Steamhammer).

Overlords are a tiny aspect of the game. It’s dizzying how complicated it all is.

A heuristic: If the enemy has one medic and some marines, target the medic first. Nobody will heal the medic, and the marines will be softer targets. If the enemy has more than one medic, target the marines. Medics have armor and more hit points, they are happy to heal each other and they do no damage. The heuristic is not always right, but it seems good most of the time.

Another heuristic: If you are tearing down a base and may be interrupted, then a building that the enemy has only 1 of is more important than otherwise. If you find 2 x gateway and 1 x nexus, it may be better to target the nexus. If you find 2 x nexus and 1 x gateway, maybe the gateway is a better target. It’s not as firm as the first heuristic, but it seems to point in the right direction.

A similar idea is taking out one pylon that powers more than one enemy building. These can all be seen as special cases of key point analysis, where you recognize key points that are especially valuable to take out—or, if they are yours, to protect. The general idea is to understand what depends on what. You want to destroy the enemy dependencies that are the most important and take the least effort: The one medic, the one gateway, the one pylon.

Well, I happened to think of the first 2 heuristics and realized that they were related. It’s probably not time to implement this kind of reasoning in bots; I think for now it is enough to implement a few special cases as heuristics. But someday bots will be strong and will get into late game situations where precise reasoning makes a big difference: When to use your last minerals to make a cannon and surround it with dark templar, when to hide a pylon on a cliff, when to shepherd your probes with an arbiter for long-distance mining—that way of protecting your key points.