Starcraft AI blog

It’s taking a long time to write up the uses of burrow, because there are so many, so here is a note on queue reordering.

Steamhammer retains UAlbertaBot’s production system, with a queue of items to produce. The queue lays out the exact order of units to make, research to do, and so on. In the beginning of the game, the queue is filled by an opening build from the configuration file. Later, the build order planner BOSS (for terran and protoss) fills the queue with long sequences, or the zerg strategy boss fills it with shorter sequences.

Most of the opening builds, though not all, are tightly written. Executing the build in order is normally efficient. But game events can intervene. For example, if the enemy scout harasses gas workers and slows down gas mining, maybe it will be more efficient to rearrange the order slightly. Maybe the 100 gas for the lair is delayed, and it makes sense to pull forward the drone that was going to be built afterward. The zerg strategy boss will often blindly misorder items the same way: It will ask for a lair that there is not enough gas to make yet, then (say) a second extractor that could be made immediately. And BOSS creates long and poorly-designed sequences with many items out of order.

I started experimenting with queue reordering in version 1.4 to mitigate these problems. The first version of queue reordering is extremely conservative: It recognizes only one case of misordering, the gas case where we are waiting for gas to start the first item in the queue, and we can pull forward a later item (up to 3 items later in the queue) that requires no gas and has no dependencies other than a worker to build it (a resource depot, a supply unit, a refinery building, or a zerg creep colony—the supply unit might be an overlord, which requires a larva instead). Also it doesn’t try to predict mineral income; there have to be enough minerals to build both items before it will reorder the queue. Because of this conservatism, queue reordering happens rarely, about once per game on average, and has virtually no effect on the bot’s strength. But it doesn’t seem to introduce any bugs despite its trickiness, so it proved the concept.

I got e-mail from a Steamhammer user who suggested that I should make the queue production “multithreaded”. To me that meant reordering the queue more freely. Maybe a better analogy is a compiler’s peephole optimizer.

I extended the lookahead in the queue from 3 items to 4, allowing reordering in a slightly wider window. I removed the requirement that the item moved forward have no dependencies; now it checks that the dependencies are met. Most importantly, I added a second case: If the next item can’t be produced because no producer is available for it (you asked for a tank, but every factory with a machine shop is already busy producing something else), or because tech dependencies are not met (you asked for a goliath but the armory is not finished yet), then a later item whose dependencies are met can be pulled forward and produced next. I kept the conservative requirement that there must be enough resources for both, to reduce the risk that the reordering will delay production overall. To relax the requirement, Steamhammer would have to predict future income, and it doesn’t do that yet. There is still some risk that reordering will delay production, if an unimportant item is moved ahead of an important item that is produced from the same source—you desperately needed scourge to survive the air harass? Too bad, the larva was used up making another drone that could be started a little earlier.

So far, my tests have not turned up any bugs, although reordering the queue is delicate and seems easy to mess up. I also haven’t noticed any cases of important items being delayed in favor of unimportant ones, though I’m sure it must happen occasionally. In practice, it looks like all win.

The zerg queue is still not often reordered. The strategy boss is fairly accurate in its production, and also it usually puts out short sequences that don’t have many items to reorder. Protoss, and especially terran, see frequent queue reordering, and it helps noticeably. BOSS likes to do silly things like order 4 SCVs in a row when there are 2 command centers, delaying everything else. Now Steamhammer is able to start 2 SCVs, then look into the future and say “OK, the next SCV has to wait, but these marines can go ahead.” Protoss and terran macro is still not strong, but it is visibly less bad than before. Also some production jams are mitigated, since Steamhammer can move items ahead of the jammed item.

Of course I intend to drop BOSS when I get that far. But I don’t think the queue reordering work is wasted. It helps occasionally in the opening, and it allows even the more careful zerg strategy boss to be a little sloppy about its sequences. I think there is a fair chance that I’ll keep the queue reordering even after I replace BOSS.

My plans for replacing the squad infrastructure are in a vague state. Here’s what I’m thinking.

• Above all, turn the worker manager and scout manager into ordinary squads. All the special cases of delegating units to managers should go away, subsumed by a general assignment mechanism. Other design features follow from that decision.

• A new bookkeeping class for squad information. SquadData as it stands seems awkward to me.

• Possibly create a hierarchy of squads. In that case, the root is the supersquad of all units and acts as the bookkeeping class for its children, which are the base-level squads that exist now. Each squad could subdivide its units into subsquads with different roles or purposes. The future worker squad might have a subsquad for each base to mine, a combat squad might have subsquads corresponding to the current unit micro managers, and so on.

• Alternatively, have a flat squad structure but allow roles in each squad to be assigned by a tag attached to each unit. This allows for the same features, it’s just a different code design. I’m still thinking through which idea makes better sense. I can think of a case for allowing both....

• In either case, provide the features through a tidy object oriented design so that a combat squad doesn’t have to worry about accidentally assigning a unit to mine gas. Look up the names of the design patterns if you want; I only remember the patterns themselves. No matter the details of the design, the current unit micro managers will become more general role managers. The way that role managers coordinate with the larger squad seems like the key to working out the undecided design details. I’ll need to work through cases in detail to foresee what’s needed in the implementation.

• Information wants to flow both up and down. Top-down control means that the tactics boss gives tasks to squads, which in turn give more detailed tasks to its units. Bottom-up control means that units examine their environment and decide what to do accordingly, perhaps self-organizing into larger task groups. As I mentioned in a comment yesterday, you want both. Units should examine their environments in light of their tasks, and the information they produce should work its way back up to the tactics boss. I have to think more about how to manage the upward flow of information.

• Currently, some bits of code that aren’t much interested in unit control end up issuing orders directly to units: The building manager takes temporary ownership of a worker to construct a building, the combat commander directly orders comsat scans on its own initiative, and a few more cases. It probably makes sense to consolidate all these somewhere in the squad code. Put all unit orders in the same module and it will be easier to cope with latency frames and the like. That will involve adding some mechanism to issue these more detailed commands to a squad.

Whatever the decisions, the goal is cleaner, more regular code that at the same time better supports the features needed for good play. More regular code will make it possible to apply machine learning uniformly across tactical and micro decisions. The purpose of investing time to rewrite working code is to earn back that time in the long run and reach the goal sooner.

UAlbertaBot’s squad structure, still used in Steamhammer with little change, is not rich enough to easily support good play. It can be made to work, but things get awkward. When I reach the 1.5.x series of versions, which will be about strong mutalisk play, I intend to do a big rewrite, to make it easier to support good tactical play and good cooperation between units in a squad. I’ve been thinking about what the plan should be.

Today’s post is about the awkward points in the squad infrastructure. Next I’ll write up my thoughts on how to do it better.

Too many special cases. In Steamhammer, in theory every unit is assigned to a squad. I’m not sure it’s true in practice, but it’s close. Workers busy mining or building are in the Idle squad, because by 1984 doublethink, work is idleness. Well, I guess the real reason is that workers are not controlled by orders to their squad, but by the worker manager (or briefly by the building manager)—the squad itself is idle, though its members mostly aren’t. The worker manager has its own special case task system, and assigns workers to mine minerals or mine gas or repair or whatever. A worker sent to fight is taken out of the Idle squad and put into a combat squad. The squad code itself has a special case to treat the Idle squad a little differently.

Another special case is the scouting worker, and for zerg, the scouting overlord. Their movements are controlled not by the squad, but by the scout manager. The scout manager also has its own system to keep track of goals and tasks. I would like to unify the different systems at least enough that they can be seen as different variants of the same thing, not as unique subsystems that share little in common.

Weak squad orders. The special cases come about because the squad order system is not powerful enough to represent all the goals and tasks needed to control workers or scouts. That means it’s also not powerful enough to represent everything that a squad wants to do in combat. There are a bunch more special cases in the micro managers that are part of the combat squad code, working around the limitations in ad hoc ways.

Internal coordination. Suppose that Steamhammer’s marines and tanks meet mass sunkens and decide to attack. The tanks siege out of sunken range and the marines charge in. It’s fine if the force is strong enough to defeat the sunkens with marines left over, but if the marines all die, then zerglings or mutalisks kill the tanks. Something similar happens when vultures and tanks meet photon cannons. The squad splits up its members into unit types that may behave differently, but it provides no mechanism for the unit types to cooperate depending on the situation. There are many, many cases where Steamhammer comes to grief because its squad members don’t coordinate among themselves, and without any general mechanism, each case has to be implemented from scratch.

With the current code, it’s somewhat awkward to implement features like positioning other units in front of sieged tanks to provide vision. The division of unit behaviors by unit type is not helpful here; you should be able to pick any suitable unit to provide vision, whether a floating building like Krasi0, or wraiths, or vultures, or marines. We care about the role of providing vision, not the unit type playing the role. And similarly for many other combat roles.

Coordination between squads. Steamhammer’s main combat squads are the Ground squad and the Flying squad, separating ground and air units because they often want to act separately. But when they do come together in the same place with the same goal, they still don’t cooperate; see limitations of combat simulation (tactical coordination section). The squads could be merged together in this case, but that doesn’t improve the behavior! The internal coordination weakness is just as bad.

Different roles with the same goal. Here is another example of poor coordination. When new fighting units are produced, they are put immediately into a combat squad, but they may start out far from the front lines. They are reinforcements and have to make their way across unseen terrain to reach the front lines. Steamhammer has almost no understanding that the different units in the same squad are in different situations and need to do different things—only a few bits of special case code to provide different behavior. It often loses the reinforcements unnecessarily, or sees them distracted by passing targets.

One fix would be to break the units into two or more squads and give each squad a different order: The vanguard squad fights as usual, the reinforcement squads attempt to join up (or are simply given the order to attack the same target as the vanguard squad, so that they end up going to the same place and then are merged, perhaps by a clustering algorithm). Another fix would be to retain the single squad, but allow it to be partitioned into a vanguard group and reinforcement groups. The 2 plans differ mainly in the code organization; they’re the same otherwise.

Next: My ideas for fixing these issues.

Today I discovered that Steamhammer’s OverpoolSpeed opening is broken in version 1.4.1. It was still correct in the previous version 1.4. I had been experimenting with an overpool +1 build similar to the one in Killerbot by Marian Devecka. Somehow a weak, unfinished version of the +1 build got into the distributed configuration file under the name of the overpool speed opening. It must have been a copy-paste slip or something.

It’s fixed for the next version, of course. With so many openings, how many others have gross mistakes? Few, I hope, but I know there are many that could benefit from refinement.

I decided I was tired of seeing Steamhammer lose helplessly to Iron’s vulture runby and Tscmoo’s factory openings. Terran can start with vultures and force zerg to defend: Zerglings quickly become useless, drones are at risk, and terran can run by defenses into the main as Iron does. Steamhammer’s strategy boss understands that a sunken is needed at the front, and has a vague idea that hydralisks may be useful if the book line happens to end, but in the end it doesn’t defend adequately. Then terran can follow up with more factory units or with wraiths, and zerg has to be ready for a range of possibilities. Against Steamhammer, Tscmoo likes to follow up with goliaths, which is effective against Steamhammer’s mutalisk openings. Too many crushing losses.

Zerg has a standard way to defend: Place a sunken at the front as a vulture obstacle and keep a modest number of unupgraded hydralisks in a defensive position. The hydras are general purpose and can fight off vulture runbys, wraith harassment, small drops—all the natural followups. Zerg then continues to mutalisks as usual. The mutas are delayed, but terran is playing a slow strategy so it’s OK.

Today I set about developing an “AntiFactory” opening that implements the standard defense. I decided on development tests against Iron to see whether the runby defense works. In the very first test, when the opening timings were still all screwy, Steamhammer put up a persistent defense and didn’t lose until well into the middlegame. An improvement already, good. I made a few fixes and tried again. Steamhammer held the early aggression cleanly and fought a long level game until Iron finally mined out and collapsed.

I did not expect that! Steamhammer lost drones by the barrel and miscontrolled scourge and chased 1 SCV with 3 ultralisks, all its usual mistakes, but Iron struggled too and could neither stop zerg from expanding nor keep a fourth base up. I guess Iron depends heavily on the success of the early vulture runby.

Well, one win does not make a champion. We’ll see how it goes as I polish the opening. It’s just that the test game was so long that by the time I finished watching it, I had forgotten what needed fixing in the opening....

When the opening is ready, will the opponent model be able to select it when it should? I think the answer is no. I think I will have to extend the plan recognizer to recognize early vulture plans. Then Steamhammer should be able to counter the vultures in the second game, and ideally cope with the followup. It would be nice if Steamhammer could get back into the learning war against Tscmoo terran, and come out on top this time.

AntiFactory branches fairly late from related openings. When the opening tree is available, Steamhammer should be able to recognize the vulture plan during the game and choose the right build on the fly, more like a human player. It might need some scouting improvements first, though.

Next: The new bots GuiBot and WillyT.

I’ve narrowed down some of the decisions of what to do in the 1.4.x series, which is about improving the opponent model until it becomes more clever than Clever Hans. Of course my plans tend to change frequently, so no promises. This is what I’m thinking now. I wrote up less specific plans in February.

As always, I’ll do a lot of things that I don’t call my primary goals. Here are some that I feel pretty sure about.

• Make some necessary improvements to the plan recognizer.
• Speed up and improve building placement. In particular, fix the bug with protoss.
• Make a start on reworking the production system to accept more general goals. I thought of ways to take it step by step.
• Try a few experiments with squad control, so that I have a toe on the ground when I move on to the 1.5.x series, which will be about mutalisk control. I might teach units to hold position in locations where they can hit targets from outside enemy range, for example, which seems like an important basic skill.
• Protoss shield batteries.
• Overlord safety! Steamhammer loses far too many.
• At least 1 new terran skill and 1 or 2 new zerg skills, just to keep progress up.
• Probably a few skills made easier by the new base and map code, such as running workers away from a base under attack.

There are also things I want to try but feel less sure about fitting in. There’s already too much. One is Antiga’s ideas about ramps and chokes and stuff.

1.4.2

• Fine-tuning of opening choice: Tend to repeat successful openings, regardless of other considerations.
• The opponent model will try to understand whether the opponent plays a single fixed strategy, or varies its play. Just knowing that gives a lot of leverage to learning.
• New base placement code, a major step in dropping BWTA.
• Make the opponent model work on BWAPI bot ladder and other tournaments. There are slight technical differences between SSCAIT and other competitions in when directories are cleared.

With Steamhammer’s large number of opening build orders, it would be very slow to learn which one to use against each opponent using an undirected algorithm like UCB. Every opening would have to be tried a few times—even a long tournament like AIIDE would not be long enough to find good openings against all opponents. We have to use knowledge to direct the search to try more promising builds first. The instant learning of the opponent model is one piece of that knowledge.

For this version, fine-tuning will consist of noticing which openings have been more successful against this opponent, and tending to repeat them. I may also add exploration, but only a limited amount because there is too much to explore.

1.4.3

• Collect opening data.
• Start to adapt opening choice to the maps.

Another piece of knowledge to direct the search for a good build is knowledge of which openings are similar to each other. If a lurker rush opening gets smashed hard, maybe try a mutalisk opening next, or at least a slower mass lurker opening. That’s only the simplest way to use the knowledge. I want Steamhammer to collect data about its own openings in real games, so it learns the timings of when production and tech get completed, and the variation of the timings due to maps and to pressure from real opponents. It’s a model of each build, you could say. The bot should be able to compare the opening models against the opponent model, so it can choose an opening likely to upset the opponent’s plan.

1.4.4

• Represent the tree of openings, and make choices at each branch.

I expect to implement the opening tree for zerg only at first. I think that by combining the opponent model, the opening models, and the smarts of the zerg strategy boss, it should be possible to make good enough choices at each branch point in the tree. The idea is that the opponent model tells us what the opponent is planning, the opening models tell us what we’ll do for each branch we might choose, and the strategy boss understands counters and can tell which result is best. I could be wrong, and in that case I’ll have to find more knowledge to add.

If successful, this could be an amazing skill. I’m determined to try. We’ll see how it goes!

Steamhammer 1.4.1 source and documentation are now available on Steamhammer’s page.

I uploaded Steamhammer 1.4.1 today. As promised, the occasional crash bug is fixed—I never identified it, but I can no longer reproduce it, so I think my changes have fixed it or at least made it much less common. Also as announced, the primary goal of improving the opponent model is put off until the next version, 1.4.2. New map code is largely written, but it is not finished so it is turned off. As always, I have made a lot of minor fixes and improvements, because working on what I said I would work on would be almost like having a job. Key fixes are to limit protoss gateways to 10 (idea borrowed from Locutus by Bruce Nielsen) and a serious zerg bug in choosing the tech target, which suppressed tech switches in the midgame.

People will want some of these fixes, so I’ll put up source approximately tomorrow.

maps

• Fixed a crash on a certain seemingly messed-up version of Bloody Ridge.

• Nicer code for distance maps, borrowed from a later UAlbertaBot. The new code is much cleaner and less bug-prone.

configuration file

• The debug option DrawBWTAInfo is replaced with DrawMapInfo, since I’m preparing to drop BWTA. For now, DrawMapInfo doesn’t do anything, since the new map code is turned off. The old BWTA info drawing code is removed.

• Bug fix: "n x item @ macrolocation" finally works, e.g. "2 x photon cannon @ natural". It was a bug in parsing.

• Added a new macrolocation @ center. Don’t use it though, except maybe on specific maps where you’ve tested it out; it doesn’t work consistently due to building placement bugs. If you ask for 2 buildings in the center, you may get 1 or none.

opponent model

• A new strategy reaction for all races: If the opponent is following a heavy macro strategy and we are still in the opening, drop any planned static defense. We won’t need it right off. Static defense added later (as a reaction to seeing scary enemy stuff) is built as normal.

tactics

• A squad that can’t find any buildings or units to attack has always been sent to explore the map. The change is, if the squad includes air units, it can be sent to explore tiles which are only reachable by air. It will help... if any opponent ever expands to an island or builds on a cliff, which hasn’t been seen so far.

• Steamhammer decides which enemy base to attack based on how much static defense it has. The change is, the enemy static defense is counted only up to a smaller distance. The longer distance was causing some mistakes in choosing targets.

• Bug fix: Units like scourge, which could belong to either the ground squad or the flying squad, could sometimes be mistakenly stuck permanently in the ground squad. The change is to CombatCommander::updateAttackSquads(). I saw a game today where this change would have helped, so I’m hopeful that it is a useful fix (but it’s certainly a fix).

micro

• Melee units (other than workers) prefer targets under dark swarm, in hope of also staying under swarm. This mainly helps protoss for now, but it is also preparation for when Steamhammer gets defiler support.

• Dark templar target zerg spore colonies with higher priority. I saw a bad game.

other stuff

• A few more opponent-specific strategies were dropped from the configuration, after experience showed that they were not needed any longer. My plan is to drop the remaining ones in the next version, 1.4.2, and rely solely on learning.

• Seeing a unit in the act of burrowing now tells Steamhammer that the enemy has burrow tech. It used to have to detect the burrowed unit to know. So many details to catch....

• The old ProductionManager::goOutOfBook() is renamed to goOutOfBookAndClearQueue(), which is what it does. A new goOutOfBook() goes out of the opening book if we are in it, and nothing else. This simplifies a few bits of code and makes them easier to understand.

• The unused BuildOrderQueue.queueItem() is removed.

terran and protoss

• Emergency response: If we have fewer than 3 workers, urgently make more if possible. This provides a small chance of surviving an extreme emergency—maybe the opponent is broken too.

• New 21Nexus opening, borrowed from Antiga.

• In tank openings, get vulture speed earlier. It was too late.

• Limit gateways to 10 total, an idea borrowed from the excellent Steamhammer fork Locutus. There are 2 benefits: 1. BOSS likes to order far too many gateways, so limiting the total prevents unnecessary expenditure. 2. What’s even more important is, it works around the building placement bug that causes the bot to exceed the per-frame time limit and lose games. The bug comes up when Steamhammer runs out of space to add buildings, so adding fewer buildings prevents it from happening. It is a super successful mitigation; it works better than all my other tries put together. In fact, I’ve been thinking of removing the vertical building placement workaround, which now probably causes more problems than it solves—maybe next version.

zerg

• When dropping excess upcoming hatcheries, count unstarted hatcheries in the limit. Steamhammer still often makes excess hatcheries, but it’s a little less severe. Steamhammer can’t time its hatcheries accurately, because it depends on future production timings and future unit mix, which Steamhammer doesn’t try to predict. The heuristics it uses instead are usually not far off, and this change is a small improvement to one of them.

• Supply counting for overlord production simulates how supply changes as queue production proceeds. The simulation is not exact, because it doesn’t account for timings such as how long hatcheries take to finish, but it is as exact as possible otherwise. This makes it possible for opening lines to order overlords at exact points and expect that runtime execution will follow the instructions. In practice there’s no gain for most openings, because overlord production was already fairly accurate. For occasional openings that depend on producing later overlords at precise timings, the difference is dramatic. Overlord production outside the opening is not improved, because overlords aren’t queued ahead of time then. (They aren’t always queued in the opening either. It’s optional, and overlords are only included when they are to be made at specific timings.)

• Important macro improvement: In midgame and later, make overlords faster. This fixes Steamhammer’s tendency to repeatedly supply block itself once the drone count grows over 40 or 50. Steamhammer still has some zerg macro weaknesses, but they are getting to be so slight that they aren’t worth attention for now. Well, except for one problem where it accumulates minerals versus terran....

• If the lair, spire, or hydralisk den is lost or cannot be made, break out of the opening. The plan has gone awry, so give it up and hand over control to the strategy boss.

• If the enemy has air tech or cloak tech, order a lair even if we don’t otherwise need a lair yet. The lair is a precondition for existing rules to fire, adding a spire or researching overlord speed to react to the enemy tech. I added this after watching Steamhammer lose a game where it thought that fast hydras with slow overlords were a good counter to dark templar (they could be good enough if it knew the right way to use them...).

• If hydras are needed in a hurry, get hydras in a hurry. If Steamhammer finds that zerglings are bad and hydralisks are good, and lair tech is still far away, then get hydras immediately without considering the longer term plan. This is a simplified way of accounting for the time it takes to get tech, as well as the value of the tech. In some future version, Steamhammer will calculate the time to get each tech and the urgency with which it is needed, and take all into account uniformly.

• Critical bug fix: Tech target calculation was wrong because a condition was coded as “if we have hive” instead of “unless we have hive”. This stopped many tech switches that should have happened in the middle game. This should improve play in many games, especially versus terran—though I have seen one or two games where Steamhammer probably played better because of the bug. Steamhammer’s understanding of tech switches remains shallow.

• Slight tweaks to ZvT tech choices.

• All zerg openings are updated to allow sunken colonies to auto-morph.

• The overhatch openings played versus random are switched from the in-base hatchery to the expo hatchery versions. The expo hatchery is better versus terran and protoss and still playable versus zerg, so it should be an improvement.

• New openings: Over10Hatch1Sunk ZvZ_12PoolMain ZvZ_12PoolLing 2.5HatchMuta. The openings offer new choices against all races and should make Steamhammer even harder to prepare against. Also renamed a few existing openings with more specific names, to fit with the new ones.

Coming soon (in one order or another): More concrete plans for the following version. The new bot GuiBot. Experience with the BWAPI bot ladder. Squad structure.

Time for another Steamhammer update. The key bug still has me frustrated. Some of my recent progress:

Base placement. The UAlbertaBot base placement algorithm that I borrowed works perfectly on most bases, but turns out to have a flaw. For some unusual base layouts, it places the resource depot far from the minerals. For example, all other bases are good on Moon Glaive, but the 1 o’clock base sets the geyser at an unusual angle to the intended base location. The algorithm places the base level with the geyser and far from the minerals, away from the intended location.

I’ll look into it more before deciding how to fix it. I might keep the algorithm as is and add an optimization step that slides the base to a better location. Or it might be simpler to ditch it and use a distance minimization measure directly.

New openings. Looking ahead to the near (ish) future when Steamhammer gets its tree of openings, I added 4 new openings. In the tree, they will be forks of existing openings and will offer opportunities to make late decisions based on scouting information. I added 1 ZvT, 1 ZvP, and 2 ZvZ choices. Before the tree arrives, I’m likely to add more forks. It would be natural to have a wide variety of overpool followups for different situations.

Even in the upcoming version, once I add fine-tuning of opening choice to the opponent model, the new openings should be worth having. They will counter some opponents more sharply.

Squad mixup bug. Today, tired of other things, I arbitrarily decided to fix a bug in squad membership. It’s not an important weakness, but it’s been bothering me for months, so I picked it.

The problem is that sometimes units that should be in the flying squad get stuck permanently in the ground squad. 3 unit types are special for squad assignment: Scourge, devourers, and carriers. If there is a flying squad with other flying units, then these types go into the flying squad, otherwise into the ground squad. The theory is that these units should coordinate with whatever squad exists—especially devourers, which are support units. The bug is that, if one of these units is assigned to the ground squad—usually because the flying squad was temporarily wiped out—it stays there. When the flying squad is reconstituted, it doesn’t move back.

The bug was easy to find, but I decided that the underlying problem was that the code was too complicated. I simplified and rewrote it until it was Obviously Correct and there could not be any bugs left. Nevertheless, there was one bug left, and I had to fix that too. Not quite simple enough; maybe next lifetime.

I’m not sure whether this fix improves strength. If you’re playing with mutalisks and zerglings and the two tend to act independently and go to different places, maybe scourge should not always be with the mutalisks. At least, stream watchers are not likely to notice the different behavior; it’s hard to see unless you know what to look for. Anyway, now it works as designed, and since the code is clearer it should be easier to change the behavior if it turns out to be weak. Still a net improvement.

Steamhammer has 2 favorite lurker dysfunctions. One of them is kind of interesting.

Its #1 favorite blunder happens versus protoss. Lurkers act somewhat independently of the squad they belong to, because their burrowing ability lets them be more aggressive. But they’re not clever about it; they don’t understand detection. When a lurker sees a photon cannon, it understands it as a high priority target, walks up to it, and burrows in range. Then the cannon kills it. Not very interesting. I haven’t thought of a way to fix it without adding or rewriting too much code, so I put it off. I’ll fix it when I work on micro, likely in the 1.5.x series while I’m doing mutalisk stuff.

The second favorite blunder is less common but more interesting. It usually comes up versus terran. Lurkers approach a ramp with a bunker on top. They can’t see uphill, but when they get in range the bunker starts firing, so the lurkers can see the bunker and they burrow and start shooting back. Then the bunker can no longer see the lurkers, so it stops firing. The lurkers no longer see their target, so they unburrow, becoming visible... and the cycle repeats. Terrans that know how to place a bunker on top of a ramp generally know how to repair it, so the bunker usually wins. If the overlord catches up to provide vision, the overlord flies into bunker range and holds position... and doesn’t provide vision for long. This game versus Ecgberht shows it happening twice, before another lurker attack finally breaks the ramp because there were enough zerglings.

To fix the overlord behavior, overlords have to worry about threats. That’s on my menu for soon-ish. I may fix the lurker misbehavior with a rule like “if you’re on low ground and the target is on high ground and you want to burrow, move to burrow on high ground if possible,” that is, on the ramp. Ideally, the combat simulator would be able to tell whether the lurker can burrow successfully. Eventually I want to teach Steamhammer how to reason about visibility, instead of following rules.

Steamhammer is doing relatively poorly. The new version 1.4 is at about the same level as the tournament version 1.4a3, maybe a little worse. (It’s hard to tell, because the opponents have improved.) It is at the low end of my expectations. A slow start was only natural, since it had to learn for itself about many opponents that it previously was hand-configured to defeat, but it seems to be genuinely not improved. In reality, it is stronger in many ways, but it also has a few new weaknesses. The new weaknesses are from the opponent model’s mistakes and limitations.

The next opponent model feature, I’ve decided, should be fine-tuning the choice of openings against each opponent according to results. The opening recognition will provide instant counters when it successfully recognizes the opponent’s opening, and fine-tuning will sharpen the selection over time. Details still to be decided, though. After enough games to accumulate data, that should solve Steamhammer’s newly-added weaknesses.

More opponent model features will follow, as mentioned at the end of the original post on the opponent model’s workings and the post on a tree of openings.

Today’s overlong game against PurpleSwarm shows bugs on both sides. Steamhammer correctly predicted PurpleSwarm’s opening and played 5 pool, which put Steamhammer ahead and should have won. Instead of keeping the pressure on, Steamhammer droned up, went to 3 bases, and started to switch into mutalisks—it does that occasionally, it’s a choice. PurpleSwarm took advantage of the delay and recovered, overrunning Steamhammer’s bases with zerglings. Just after its natural fell, Steamhammer made an extractor there, which seemed wasteful but turned out to be key. Then, just before its main fell, Steamhammer hatched its first and only 2 mutalisks. There was not much play left for Steamhammer.

PurpleSwarm destroyed everything it found, but never found the extractor that was barely out of sight. Zerglings roamed the map checking bases, but didn’t look everywhere. PurpleSwarm also never mounted a defense against the 2 mutalisks. That makes 2 basic errors by PurpleSwarm. Steamhammer’s mutalisks slowly wore down drones and buildings, like water dripping on stone (watch the whole game if you have geological patience). The mutalisks sometimes went idle, which should never happen for more than 8 frames. And the mutalisks did not search for other bases, but only destroyed those already seen. That’s 2 bugs for Steamhammer. Steamhammer’s bugs proved less severe: Steamhammer won on points when the game timed out, since its mutalisks destroyed so much stuff over the long long endgame.

Steamhammer’s bugs appear to be in the tactical goal selection. These are features which usually work but failed this game (and I have seen other failures too). The code does not look that complicated, but I have fixed bugs in it before. I guess it is trickier than it appears. I did make one change to exploration: Air squads are now allowed to explore places which are not reachable by ground. That would not have helped this game, but it could help against an opponent whose last base is an island base, or which is hiding floating buildings in an inaccessible spot.

This came up (again) in comments to yesterday’s post. Steamhammer too often exceeds the time limit when playing a long game as protoss. It happens when the bot’s base fills up with buildings and there is no room for a new building. The building placer doesn’t understand the situation and keeps retrying, and meanwhile no buildings can be constructed. Building placement is slow so after a while it exceeds the time limit. Even if it didn’t exceed the time limit, the bot would likely lose the game because inability to place a building freezes the production queue so that nothing can be produced.

In principle. the same problem affects all races. In practice, terran can place buildings freely and zerg makes so few buildings that they almost always fit on available creep. If zerg wants a lot of sunkens, it doesn’t build them all at once, and each one extends the creep so that there is more space.

There are 3 different mitigations in the code that reduce the problem but don’t come close to solving it.

• Protoss buildings are allowed to touch each other vertically, blocking horizontal movement. This reduces the space taken, but occasionally causes units to get locked in enclosures.
• Steamhammer can randomly choose a new “main base” where future buildings will be placed. It considers doing this whenever a pylon is placed and more than one base is available to place it. This often causes buildings to spill unnecessarily into the center of the map.
• A special case in the production code recognizes when building construction is being retried, and retries less often.

Other simple mitigations that I have considered also don’t solve the problem.

A full solution, in my view, has 2 parts. The first part will be to make building placement much faster. I have 2 ideas that together I expect to speed up building placement by an order of magnitude or more. These ideas are made easier by the base data infrastructure that I’m writing now (see yesterday, new base placement code). I’m currently working on Steamhammer version 1.4.1. My plan is that building placement will be fast no later than version 1.4.3, I hope sooner. Expect a production freeze, but that is better than an instant loss.

The second part is to reason about building placement. The absolute minimum is to recognize: “Oh, building placement failed. It’s going to keep failing until a pylon finishes.” Then at a minimum it can stop adding buildings. Better, it could find a good place with space around it, and put a pylon there. I should be able to do that in the 1.4.x series, and it will solve the problem for good. There are complications that I will have to deal with. For example, if there is free supply then Steamhammer shouldn’t let unit production stop just because building construction is stopped or delayed.

Another step could be to recognize when a base is full, and stop trying to place buildings there. Put them somewhere else. The new base data should make that easier, too.

I want to go further, and write a general placement planner to optimize the placement of buildings and units. You specify goals: Place buildings compactly, while leaving space for units to pass. Form a wall at the front with as few gaps as possible. Place static defense to defend both the natural and the ramp to the main. Set lurkers just far enough apart that only 1 at a time gets hit by storm. Set tanks out of sunken range and in siege range and vision range. Spread out tanks to defend this choke. Prefer to put tanks on high ground. The placement planner figures out good places. It will be complicated, but I expect that with the speedups I have in mind it will be fast enough.

Time control in general is an issue that Steamhammer should tackle eventually. As it stands, near the start of the game the maximum time per frame is commonly at least 25 times greater than the mean time per frame, and it tends to increase as the game goes on. Steamhammer gets by because it does almost everything fast enough, so that it doesn’t have to worry much about breaking the time limit. When I add tactical analysis and other expensive features, that will probably change. It will become necessary to check whether there is time for each task, and defer or suspend or abort tasks that threaten to go over. I’ll figure that out when I get there.

Today I tested Steamhammer’s new base placement code, a major step in getting rid of BWTA2. There are still a couple of big bugs, but it works well most of the time. It’s getting close to ready.

As I mentioned when I first wrote about map analysis plans last June, Dave Churchill removed BWTA as a dependency from UAlbertaBot after Steamhammer forked from it. Newer UAlbertaBot versions do the necessary map analysis themselves, and when it comes to unnecessary map analysis, they simply delete some less important features that BWTA used to provide. I won’t delete features that add strength, but for the necessary features my plan has always been to borrow from UAlbertaBot.

Here’s how placing bases works in general. A base consists of a location for the resource depot, plus a set of minerals and vespene geysers belonging to the base (plus any other info you find useful). For starting positions that a player may begin the game in, the map specifies where the resource depot goes. BWAPI passes the information through. You still have to figure out which resources belong to the base. I had Steamhammer assign all the resources within a fixed distance to each starting base, and it works fine.

For other bases, software is on its own. First, group the resources on the map into clusters, expecting that each cluster will become a base location. You probably want to skip over blocking minerals and any clusters that don’t provide enough resources to be worth placing a base at, or at least handle them differently. Then, for each cluster, figure out the best place to put the resource depot so that mining will be efficient. Details matter, but it’s not complicated, at least if you ignore weird maps like Crystallis.

I wrote my own clustering, which is simpler than UAlbertaBot’s (though I wonder if I made it too simple and crude). For placing the resource depot, I followed UAlbertaBot’s way. UAlbertaBot provides 2 methods, one of which is commented out. The commented out method finds the center of the resources by averaging their locations. (Technically, it’s the centroid.) It then uses UAlbertaBot’s building placement code to find the nearest location to the center at which it is legal to build a resource depot—you have to be at a minimum distance from any minerals and gas. The building placement code calculates a spiral of tiles centered at the desired location and tries each one until it finds one that works.

The active method in UAlbertaBot, which I took for Steamhammer, is to find the center of the bounding box of the resources. That’s the desired location, and UAlbertaBot proceeds as above.

In practice, Steamhammer’s new code usually finds exactly the same base locations as BWTA. There are some exceptions. It finds 2 bases in the center base of Fighting Spirit (as some other bots do), which I think is due to a bug that assigns the same mineral patch to more than one base. I don’t expect it to be hard to fix.

Good progress so far.

This is an idea I’ve talked about before. As I get closer to starting the implementation, I’ve worked out a little more detail.

It’s getting close to the time to start distinguishing between build orders and opening strategies. Steamhammer today only knows exact build orders: Spawning pool exactly now, then exactly 3 drones, then exactly 3 pairs of zerglings, etc. It has some ability to vary the build order; it can insert static defense, and react to emergencies, and a few other things. The variations are shallow and Steamhammer does not know how to adjust the build order to compensate for them. For example, Steamhammer will often substitute drones for zerglings that it believes it does not need, and suffer from a mineral excess as the build order continues. The opening build order reaches its end, and suddenly Steamhammer comes to life and starts 4 hatcheries simultaneously to get enough larvas to burn off the extra minerals. Ouch!

A pro can easily do things like open with 9 pool and then transition into the usual 3 hatch mutalisk play. (Strategically, the intent is that the 9 pool should delay the opponent enough to make up for the delay in the zerg build.) Steamhammer only knows the standard 12 hatchery into 3 hatch mutalisk build order. I would like to give Steamhammer the ability to start with any opening stem and then transition into any reasonable middlegame followup. It should keep its ability to follow an exact build order when it knows one, and add the ability to devise a build order to reach a goal situation like 3 hatch mutalisk. I think the flexibility is necessary.

I think of the goal situations as abstract opening strategies. They are abstract and not concrete, because many details have to be decided to turn the strategy description into a specific build order. “3 hatch muta” is how humans talk about strategy, so we know it’s a useful level of abstraction.

What does it do? First, you need a way to specify a goal situation, like “3 hatch mutalisk.” At a minimum, you need to be able to specify the required tech and the number of production buildings, so you can ask for things like “2 port wraith with cloak.”

The goal will look somewhat like a BOSS goal—not surprising, since BOSS’s purpose is to devise build orders to reach goal situations. For BOSS, you give the number you want of each unit and building and tech to research. That’s too specific, and not good enough for my intention. For example, normally terran should build SCVs whenever possible in the early game, but if playing BBS you want marines to have priority over SCVs. For another example, if you’re going pure lurker-ling, then your ratio of lurkers to zerglings should not be fixed but should depend on the situation: Typically, spend all gas on lurkers, any remaining minerals on zerglings. In the goal, you usually want to specify exact numbers of only a few things and priorities of other things (“SCVs first” and “gas goes to lurkers”). It’s complicated, and I won’t be surprised if the final goal specification ends up more complicated.

Planning is still at an early stage. Right now, I’m imagining that the configuration file will declare a set of goals separate from the opening lines. An opening line may implement a goal: For example, the current 3 hatch muta line will follow the 12 hatchery opening stem and implement the 3 hatch muta goal in a specific way. There will have to be notation to declare that. Or the opening line may stop short at the stem. In that case, the strategy boss (consulting with the opponent model) will be responsible for choosing or inventing a followup goal.

When this is done, the tree of openings will become much bushier, which is to say, Steamhammer will have many more choices at each point. At each choice point, it could choose any compatible goal (an opening stem that makes 3 hatcheries is not compatible with a goal that calls for 2 hatcheries). Even if a goal is already underway, in case of surprise the bot could change its mind. If the strategy boss is smart enough to keep up, then Steamhammer should gain much, much greater ability to adapt.

How does it work? Steamhammer retains UAlbertaBot’s production system, which assumes a uniform list of items to produce. This system, or one like it, must be kept if Steamhammer is to play hand-optimized opening lines, which is a useful skill (humans do it too). I may rewrite it, because it has limitations that harm performance: It can produce only 1 item per frame, and it requires latency compensation. Goals do not fit into the uniform production queue. The exact production sequence to achieve a goal has to be re-evaluated every time the situation changes unexpectedly—even the rate of mineral mining can’t be predicted exactly, because there is a random component, and you never know when you might have to suddenly spend money on an emergency response. The natural implementation is to create a separate software layer, the production boss, which turns goals into production queues.

Turning goals into production queues is also what BOSS does. BOSS tends to react slowly to changes, because BOSS likes to create a long detailed production queue (though of course it depends on what goal you feed it). The production boss (it’s funny, but it will be keeping nearly the same name) needs to avoid that problem. It will have to look ahead to make good plans, but at the same time it will have to feed items one or a few at a time to the queue. Either that, or it will have to be prepared to frequently clear and recalculate a longer queue. In any case, the natural implementation is as a separate software layer. I may rename the current ProductionManager to ProductionPeon and add a component ProductionBoss... that should make the layering clear. :-)

I’ll implement it piece by piece. The first piece will come during the 1.4.x series, but it will be a tiny part. I’ll have the production system take care of the trivial prerequisites for terran addons and zerg static defense: If you ask for a machine shop, it will build the factory if necessary and add the machine shop when it can and without blocking other production unnecessarily. Similarly if you ask for a spore colony. The system will replace zerg automatic sunken morphing, moving it from the strategy boss to the production system where it belongs, and extend the benefits to a few more buildings. It will help terran significantly. Terran openings that build an addon deliberately include extra production and produce the addon later than they usually can, because a bug messes up production if the addon can’t be completed when ordered.

With that done, it should be easy to add more goals. As the production system gets more capable, the zerg strategy boss will become smarter to match. It will become much easier to write strategy bosses for terran and protoss, so I’ll do that and dump BOSS, which should bring a big jump in skills for terran and protoss. The whole process will take a long time.

I have a simple idea that will both make it easy to restructure Steamhammer’s openings as a tree of opening build orders, and at the same time make the opening lines shorter and easier to understand. There are 2 parts.

Part one: Add a touch of new syntax. If the first item in an opening build order is the name of another opening, then substitute in the other opening’s build in its place. This applies recursively, of course.

For example, I could then recode Steamhammer’s 12 hatch openings to factor out the 12 hatch part. These 3 openings

"ZvZ_12HatchExpo" : { "Race" : "Zerg", "OpeningBuildOrder" : ["4 x drone", "overlord", "4 x drone", "go scout once around", "hatchery", "spawning pool", "extractor", "3 x drone", "4 x zergling", "Lair", "drone", "creep colony @ natural", "metabolic boost", "2 x zergling", "spire", "9 x zergling"] },

"ZvT_2HatchMuta" : { "Race" : "Zerg", "OpeningBuildOrder" : ["4 x drone", "overlord", "4 x drone", "go scout once around", "hatchery", "spawning pool", "extractor", "4 x drone", "3 x zergling", "Lair", "drone", "metabolic boost", "2 x drone", "spire", "3 x drone", "hatchery", "drone", "extractor", "2 x overlord", "12 x mutalisk", "hydralisk den", "5 x drone"] },

"2HatchLurker" : { "Race" : "Zerg", "OpeningBuildOrder" : ["4 x drone", "overlord", "4 x drone", "go scout once around", "hatchery", "spawning pool", "extractor", "go gas until 1075", "3 x drone", "3 x zergling", "Lair", "2 x drone", "metabolic boost", "hydralisk den", "extractor", "drone", "lurker aspect", "drone", "2 x zergling", "5 x hydralisk", "overlord", "5 x lurker", "drone", "hatchery", "3 x zergling"] },

could become these 5 openings

"12Hatch" : { "Race" : "Zerg", "OpeningBuildOrder" : ["4 x drone", "overlord", "4 x drone", "go scout once around", "hatchery", "spawning pool", "extractor", "3 x drone"] },

"ZvZ_12HatchExpo" : { "Race" : "Zerg", "OpeningBuildOrder" : ["12Hatch", "4 x zergling", "Lair", "drone", "creep colony @ natural", "metabolic boost", "2 x zergling", "spire", "9 x zergling"] },

"12HatchTech" : { "Race" : "Zerg", "OpeningBuildOrder" : ["12Hatch", "drone", "3 x zergling", "Lair", "drone", "metabolic boost"] },

"ZvT_2HatchMuta" : { "Race" : "Zerg", "OpeningBuildOrder" : ["12HatchTech", "2 x drone", "spire", "3 x drone", "hatchery", "drone", "extractor", "2 x overlord", "12 x mutalisk", "hydralisk den", "5 x drone"] },

"2HatchLurker" : { "Race" : "Zerg", "OpeningBuildOrder" : ["12HatchTech", "go gas until 1075", "hydralisk den", "extractor", "drone", "lurker aspect", "drone", "2 x zergling", "5 x hydralisk", "overlord", "5 x lurker", "drone", "hatchery", "3 x zergling"] },

I think of it as stems plus branches. Currently, you have to repeat the opening stems in every opening. Doing it the new way, the stems will be factored out and only written once each. It’s shorter and could prevent mistakes.

If you read closely, you’ll see that one detail of the original 3 openings is lost: The ZvZ opening scouts earlier. After recoding, all openings scout at the same timing. I want to change scout timing to be decided by the opponent model rather than by the opening book, removing the decision from the opening altogether. Also the number of drones made in the lurker opening is changed in this version; it’s not quite the same opening, though it’s close. When I do it for real, I will probably recode the openings differently than in this example.

One other detail is changed in an unimportant way. The 2 hatch lurker opening limits how much gas it collects, because once it has its initial set of lurkers, it wants to make drones and hatcheries to transition into a longer game. The original version specifies "go gas until 1075" right after the extractor is started, while the stem-and-branch version gives it after the opening stem. The change doesn’t affect how the opening plays out in any way.

I’m also thinking of factoring the openings into 3 subsections, one for each race. Then you don’t have to repeatedly specify the race. Instead of

"2HatchLurker" : { "Race" : "Zerg", "OpeningBuildOrder" : ["12HatchTech", "go gas until 1075", "hydralisk den", "extractor", "drone", "lurker aspect", "drone", "2 x zergling", "5 x hydralisk", "overlord", "5 x lurker", "drone", "hatchery", "3 x zergling"] },

it would become

"2HatchLurker" : { "OpeningBuildOrder" : ["12HatchTech", "go gas until 1075", "hydralisk den", "extractor", "drone", "lurker aspect", "drone", "2 x zergling", "5 x hydralisk", "overlord", "5 x lurker", "drone", "hatchery", "3 x zergling"] },

I can’t reduce it to a bare list until I change how terran and protoss openings work. They specify an opening group that says what unit mix to build in the middle game. Someday, they’ll have their own strategy boss that figures that out for them. Though I can think of other reasons to add tags to opening lines.

Part two: Interpret the stems and branches as a tree. Why allow opening names only in the first item of the build order? Because then you get a tree of openings.

Here’s how Steamhammer will make opening decisions. There will be a bunch of initial opening stems, 4 pool, 5 pool, 9 pool, and so on up through 3 hatch before pool. Steamhammer can choose an opening stem first, and go from there. Alternately, it can pick any opening it wants, just as it does now, and backtrack to the stem as a starting point. Anyway, at first it plays only the stem of the opening, then it is time for another decision: It can follow any of the opening lines branching from that stem, or it can stop following any opening and let the strategy boss take over.

Suppose it starts off choosing 2 hatch lurker. First it plays out the 12 hatch opening stem, then it thinks: “I found out that my random opponent is terran and not zerg, so I’ll continue down my original path.” It plays from there, treating the 12 hatch tech branch as the stem of an opening subtree. At the end of the 12 hatch tech branch, it thinks again: “I see an early factory, so lurkers might be bad. Better follow the mutalisk branch.” (I think the best reaction to an early factory is to get a hydralisk den and a few hydras for defense against vultures and/or wraiths, then continue into mutalisks. Steamhammer won’t be that smart at first.)

Or it reaches the end of the 12 hatch opening stem, then “Uh oh! Marines already!” It can bail out immediately instead of following any branch, and let the strategy boss take over and try to survive. Whatever opening line the bot finished on will be recorded in the data for the opponent model. Every opening line, stem or branch, can be treated identically, except that some of them have further branches and some don’t. In effect, at the end of each line is an implicit branch “now let the strategy boss do its job,” and that branch can be chosen like any other. The idea of treating every opening line identically is key; it gives the learning algorithms flat data that is easy to digest.

Using scouting information at each decision point will make Steamhammer’s openings much more reactive. Making decisions later, when more information is available, means the decisions will be better. (Project management works the same way!)

I’ll implement this in the 1.4.x series. It won’t be in 1.4.1 or 1.4.2, but it might be the version immediately after. The idea is simple and it provides necessary flexibility that Steamhammer has always missed. At first, only the zerg strategy boss will have the smarts to make decisions between branches of the opening tree, though. Terran and protoss will have to wait until they have more software infrastructure.

Some time in the future I’ll add an abstraction layer to openings, and cleanly distinguish between specific opening build orders and general opening strategies. Next: Abstract opening strategies.

Instead of configuring all of Steamhammer’s opening probabilities by hand, I want it to figure them out for itself. The starting point is data: For each matchup, keep overall statistics of all the openings in a file. Another goal is to have Steamhammer adapt its strategy to maps based on experience. So I thought, why not combine the two? For each matchup and map, keep a file with numbers for all the openings. Or for each matchup, keep a file for all the pairs (map, opening)—that way the bot has the data to generalize between maps.

Someday I also want Steamhammer to adapt its strategy reactions and its tactics to the map. At first it will analyze the map and decide what looks good (“look at that cliff over the mineral line—I should go air or drop”); later it will learn from experience what works well. I don’t expect to get to that for a long time, though.

Steamhammer has over 60 zerg openings (it doesn’t play all of them), and the count will increase. SSCAIT has 14 maps and other tournaments use fewer, so I think I should be ready for on the order of 1000 pairs (map, opening) if I keep them in one file. Each pair would be one line of data, something like “<opening> <map> <# games> <# wins>” and maybe a few more numbers like mean game length, or a statistical summary of evaluations once there is an evaluation function, or whatever else. If I want a cache of map data like number of starting positions and rush distances and so on, to use in generalizing across maps, that would be in a separate file.

In that case there would be 12 matchup files, including data for when the opponent went random: TvT TvP TvZ TvR, PvT PvP PvZ PvR, ZvT ZvP ZvZ ZvR. With up to 1000 lines per file, it seems like a reasonable amount of data. In every game, Steamhammer would read one file in onStart() and write it in onEnd(), which doesn’t seem excessive. There is one complication. If the opponent goes random, and after I give Steamhammer the ability to change its mind on the fly (which I will do), then when Steamhammer finds out the opponent’s race it may want to read that matchup file too. Reading and analyzing that much data may take more than one frame time, so it might have to go into a separate thread. Another solution for when the opponent goes random might be to read 4 matchup files during onStart() when we are allowed more time. Well, when it comes up I’ll figure out a plan. Maybe nothing special will be needed (seek time for a hard drive could exceed one frame time, but reading from SSD is faster).

That’s the data. How to use it? I haven’t decided on details. The opponent model keeps detailed records for each game against a given opponent, including the map. When we play the opponent for the first time, decisions have to be made without the opponent model, solely on the basis of the matchup+map+opening statistics. I’ll figure out a more-or-less sound way to turn the raw numbers into probability-to-play-this values, including an exploration policy. There are choices. After we’ve played an opponent many times, the opponent model will have more information (since it records more data, and it is specific to the opponent), so it can decide on its own. In between, I’ll need some kind of evidence combination procedure to blend the 2 sources of information together. I expect that a simple procedure would work fine, even weighted averaging of probabilities.

Steamhammer’s configuration file will become much shorter. I expect I’ll retain the manual configuration options for openings, for those bot authors who want to do it that way, but Steamhammer itself will rely on the data it collects. Once I have a good set, I’ll distribute it with the bot.

I’m not sure when I’ll get to all this stuff. Maybe or maybe not in the 1.4.x series.

Next: A tree of openings. It ties in.