Starcraft AI blog

Two big posts today! Antiga announced the AI Starcraft Tournament (AIST), Season 1, to be held next month. The tournament is to be played on the same professional maps as the ongoing Afreeca Starleague season 5, aka ASL5: The old map Gladiator and the new maps Sparkle, Third World, and Transistor.

I’m in favor of tournaments like this. The new maps present difficult challenges that Brood War bots have not faced before, and confronting them should force bots to become more robust and capable. The competition is labeled Season 1, so the intention is to hold more competitions in the same vein, with further tough maps. Over time, that should also impel bots to become more general and flexible. It’s good.

The deadline is soon! The submission deadline is 10 June, less than a month away. The new maps are ambitious targets for current bots, and I believe it will not be possible for authors to fully adapt their bots in time. I expect that participants will be taking shortcuts or accepting poor play.

trying out the maps

Should Steamhammer participate? The first part of the question is, can Steamhammer participate? Can it be adapted to the demanding professional maps before the deadline, or is the amount of work infeasible? I imagine I could do it if I rely on map-specific hacks (“if the map is named Sparkle, use this build order, kill this neutral building at this time, ...”), but I see hacks as wasted work that will have to be redone properly later. I want more general techniques to cope naturally with particular map features wherever they come up.

I tried the maps in test games to see what happens. In alphabetical order:

• Gladiator was fine. Steamhammer played normally. It’s a classic map from 2010, and that is what I expected.

• Sparkle, the island map, caused less trouble than I expected. Well, I expected a lot. I gave Steamhammer a game against the built-in AI (which can cope with islands), and made it ZvZ to be sure that it would go air. Drone scouting failed, obviously. The bot was unable to expand, even on its own island, and played the game with 1 base. It made a ton of zerglings (“it’s ZvZ, I know how to play this”) which had little use since they could not get off the island. The Recon squad is ground-only, so Steamhammer didn’t find enemy island expansions until after it reduced the enemy main. Steamhammer did finally win, though; the broken stuff was not absolutely essential. The problems are likely solvable by adding basic island skills, plus some map feature skills so it can recognize and take its own expansions.

• Third World caused more trouble than I expected. I thought that play would ignore the “third world” upper left part of the map, since it is reachable only by drone mineral-walking tricks, and proceed normally on the resource-poor “first world” remainder. But no, again Steamhammer was unable to expand beyond its starting base. I don’t know why. Drone scouting again failed, apparently because BWTA failed to calculate regions, or did it in some way differently than Steamhammer expects (I didn’t dig into the cause). It’s hard to say how long it might take to debug and fix or work around the issues. Every use of BWTA has to be removed eventually, but this one was going to wait....

• Transistor crashed Steamhammer’s new map analysis. I’ve tested the code on dozens of maps, from irregular Blizzard maps to intricate pro maps, and this is the first crash. The map doesn’t look technically difficult except for the weird crystals at the starting locations, and I had expected it would be OK. Likely it will turn out fine once I fix the crash.

My preliminary conclusion is that Steamhammer can play, but it will not be well adapted to the maps. I will have to spend time surviving tricky map features and won’t have much for thriving with the important play features like mineral-walking drones into the third world, or linking up island bases with nydus canals. The play features are what make it worthwhile, so I’m feeling hesitant. I’ll look into it more, though. This is only a first look, and there is more to find out.

How many participants will there be? How ready will they be? If people are saying “I suffer on these maps too, but it is still worth it” then it becomes self-fulfilling, it is still worth it.

Steamhammer’s elo is plummeting, a clear sign that the voters have taken an interest in the new release and are wielding their monkey wrench. At this rate, they’ll lose interest again pretty soon....

Here is the change list. Opening selection in the opponent model should help all races choose stronger openings against each opponent, when the bot has gathered enough experience. The map analysis changes have no effect on play (yet), but are part of the effort to jettison the terrain analysis library BWTA2 and will make a variety of analyses easier in the future. Protoss should no longer break the time limit after its base fills up with buildings. Terran tanks don’t siege and unsiege so insanely often. Terran building addons are built more quickly and reliably. Many of the other changes help all races play better. Macro for terran and protoss is much improved. Micro is a trifle crisper, especially for melee units.

It’s a complicated update, and there is a higher than usual risk of new bugs.

The most important changes are in bold type.

opponent model

• Select openings which have been successful against this opponent before. Take the map into account; with experience, Steamhammer will play different openings on different maps (though with 14 maps to learn about, map adaptation is slow as a snail). If past opening choices are not fully successful, try out different openings. With an increasing number of games, this increasingly includes openings that aren’t tagged as suitable for the opponent’s recognized plan, or for the opponent’s race, or even openings that Steamhammer otherwise won’t play at all. We will occasionally see openings that Steamhammer has never played in public before, because they were written in but not configured to be played. This is more or less as described in my earlier plans. I’ll post a more detailed description separately.

• When recording the opponent’s game record to a file at the end of each game, rewrite the entire file instead of appending to it. Appending works on SSCAIT, but most other competitions require the file to be rewritten.

• The plan recognizer recognizes a new Factory plan that can be followed only by terran. It means terran is starting with a mech opening. Zerg is configured to counter the Factory plan with an AntiFactory opening. In tests against different opponents, the plan is accurately recognized, and the change produces much stronger zerg play against terran mech openings. Terran and protoss are not configured with anti-mech openings, but could be.

• The plan recognizer is smarter in one way: If the opponent plays a fast opening plan, the recognizer doesn't change its mind later when it sees signs of a slow opening plan; it wants to recognize the first thing the opponent does. The plan recognizer was doing things like correctly recognizing a 5 pool as Fast rush, then later seeing the 5 pooler add a second hatchery, and deciding that it was a macro opening after all. In the next game, Steamhammer could react incorrectly.

• There was a typo in the UCB code! Fixed. This affected the auto gas steal, if it was turned on.

• Separately, the auto gas steal is a little more conservative. It wants to see more evidence that it is useful before it is repeated.

• Try to detect whether the opponent always plays the same strategy. If so, the game info display notes it as (say) “surely Fast rush” instead of “expect Fast rush”. The opponent model uses the information behind the scenes (a little bit).

map analysis

• New code calculates base locations and stores related information. It works, and is used for some purposes, but most of Steamhammer still relies on BWTA. The information manager indexes bases by their BWTA::BaseLocation, and it turns out to be something that should be redone in one sweep. Later I’ll move the information manager functions into the new Bases class.

openings

• The opponent model is now in charge of choosing openings, so the CounterStrategies section of the configuration file is now used only when the opponent model wants to try out openings that respond to the enemy's expected plan, rather than whenever the enemy has an expected plan. I made the probabilities more even, so that stronger counters are still preferred but weaker counters will come up more often than they used to. That will help the opponent model collect the data it needs.

• I haven’t documented it before, but Steamhammer for a time has had some openings which are included in the configuration file but not configured to be played. Bot authors might choose them for their own bots, or take them as starting points for developing new openings. The opponent model is now able to select these openings. It will occasionally choose one at random and try it out against an opponent which it has played many games against, and if the opening is successful it will keep playing it against that opponent. For this release, I added more unconfigured openings.

• Included in the openings provided but not configured to be played are a bunch of new protoss openings from Antiga. They deserve attention, but I ran out of time.

• OverpoolSpeed was accidentally replaced with a bad opening in the previous version. I restored the good version.

• The classic Over10Hatch opening was restored under that name. It makes 2 sunkens at earlier timing than the newer Over10Hatch2Sunk, and builds fewer drones. The idea is that restoring it should allow old learned data at SSCAIT to remain useful—it’s useless when it refers to an opening name that no longer exists. Plus there are times when the low-economy version works better.

• New zergling rushes 5PoolHard2Player which hits harder on 2 player maps, and 6PoolSpeed which hits differently.

• Zerg has a new AntiFactory opening to combat early vultures and their followup. It works great. Terran has a corresponding VultureWraith opening for TvZ. I wrote it as one of the tests of AntiFactory, and found that it is effective against zerg opponents which aren’t ready. And the opponent model should now be able to tell which opponents are not ready.

• New zerg openings Over10HatchBust, Over10HatchSlowLings, and Over10Hatch2SunkHard are variations of the anti-zealot rush build and may be suitable for different opponents. They are intended as food for the opponent model. Overpool+1 is a weaker version of the ZvZ opening played by Marian Devecka’s Killerbot. Because of a limitation of the strategy boss, it gets carapace +1 instead of melee attack +1, which reduces its effectiveness. None of these openings is particularly good, but I expect they will have value against some opponents.

production

• No more queue deadlocks. Steamhammer avoids most production jams by recognizing and dropping jammed items. This affects terran and protoss; zerg already had virtually no production jams.

• Effective queue reordering. The previous version occasionally reordered queue items for more efficient production. This version generalizes the technique and benefits from it much, much more often. If the next queue item can’t be started immediately, and a later item up to 4 places ahead can be, and other conditions are satisfied, the later item is moved to the front of the queue and produced immediately. This affects all races, terran the most, protoss somewhat less, zerg the least.

With production unjamming and queue reordering working together, terran and protoss macro is way smoother. Production unjamming is aggressive; it drops any item which will cause a long delay. BOSS will often order (say) a cyber core and then dragoon range immediately afterward. It caused a long delay, though not a deadlock. Now when that happens, Steamhammer usually finds when it gets to dragoon range that the cyber core is still in the building manager’s queue and hasn’t been started yet, so it drops the research. BOSS will order the research again on its next cycle; not ideal but a big improvement overall. Meanwhile, the next cycle comes sooner because queue reordering makes production of this cycle faster. If the cyber core was started and hasn’t finished, other queue items will be pulled in front of dragoon range and produced first. Unfortunately, BOSS is as happy as ever to order all buildings and no units. Queue reordering can only mitigate the flaw to an extent.

• It’s possible in principle for queue reordering to be too thorough and break an opening’s intended build order. To cope with that, I added a new command that can be included in a build order, "go queue barrier". Nothing after "go queue barrier" will be reordered to happen before it. It was just a tiny little bit trivial to implement, because it’s a command that does nothing; queue items are never reordered across any command. No openings use "go queue barrier" currently; I tested the likely candidates and found that they worked fine with queue reordering.

• Protoss should not overstep the time limit when its base fills up with buildings. This is a major improvement for protoss. There are 2 contributors to the slowness: Failing building placement in the building manager, and failing building placement in the production manager when it tries to predict when a building can be started (it needs to know where, so it can figure out how far ahead of time to send a worker). The change is applied in both places; both are fixed. The production manager’s problem was already mitigated by a workaround which retries building placement less often, and I kept the mitigation since placement can fail for other reasons. The problem won’t be 100% solved and shelved until building placement is fast and reliable.

If the debug option Config::Debug::DrawBuildingInfo is turned on, then buildings which are currently stalled and cannot be built are in red. After a pylon finishes and the buildings unstall, they turn back to the normal yellow color.

• Earlier versions tried to work around the protoss building placement problem in part by allowing certain types of protoss buildings to touch each other vertically. This made building layout more compact, but had the side effect of sometimes trapping units in enclosures. This version drops the workaround. Protoss base layout is loose to the point that it is awkward and causes problems, but that is not as bad as trapping units.

• Terran addons are produced via production goals, which construct them more reliably and earlier. It is a significant improvement for terran. Some terran openings are improved to take advantage; there is more room to improve them. I removed a workaround for a BOSS comsat bug, since production goals work around the bug automatically.

tactics and micro

• Deciding which base to attack now takes into account some of the enemy units seen near that base. Earlier versions took into account static defense only: Attack the base with the least static defense. This version also takes into account certain slow-moving units that are good for defense: Tanks, high templar, reavers, lurkers, guardians. The aim is still to attack the least-defended base. It’s smart enough that an air squad with no ground units only worries about defenses which can attack air, and a ground squad with no air units doesn’t worry about turrets or spore colonies.

• Deciding where to drop is more flexible. Drop navigation—moving the transport around the edge of the map—is rewritten, and its code is far simpler. The main practical effect of this work is that the dropship or shuttle always takes the shortest path around the map to its target. It used to be right only half the time, despite an attempt to figure it out. The protoss dark templar drop will land earlier up to half the time. With help from the addon production improvement (see above), the terran vulture drop always lands earlier.

I did not finish my plans to improve drop decisions. It’s a little smarter about where to drop, not as smart as I wanted. If no enemy base is known to drop on (which only happens in rare circumstances when other things have gone wrong), the transport is sent to drop on an unscouted base—and if the base is empty, it still drops there. Furthermore, the dropped units then sit around and do nothing. The issue is so rare that I didn’t fix it.

• Some rework of micro targeting. Zealots and zerglings are particularly improved. Melee units are allowed (as ranged units always have been) to ignore bad targets, which makes them less distractible. All units ignore targets outside of a maximum range limit, to make them less distractible. The method of prodding units to tend to move toward the squad goal is changed; this is responsible for most of the improvement in melee units. And there are various minor changes in detail.

• Units that can’t attack under dark swarm, don’t try to. This may help occasionally against those few opponent bots that use dark swarm. Mainly, it is preparation for when Steamhammer knows how to use defilers itself.

• Enemy nydus canals were usually given priority as important targets. Now they are always given yet higher priority. I think it is likely that they are still not high enough in priority, but Steamhammer has yet to need the knowledge, so it’s hard to be sure.

configuration file

• The debug option Config::Debug::DrawQueueFixInfo prints information to the screen when the queue is reordered or a production jam is averted. For terran and protoss, this happens a lot. If you only watch the queue on screen and don’t turn on this option, it is hard to visually catch all of the changes.

• The IO option Config::IO::MaxGameRecords limits the maximum number of game records that are stored for each opponent. When over the limit, the oldest records are dropped. The bot can be left running indefinitely on a server like SSCAIT without risk of using up its allotted space or growing slow because the opponent model files are too big.

• If you don’t want to use the plan recognizer, set Config::Strategy::UsePlanRecognizer to false. The enemy plan will always be Unknown. I expect this may be useful for some authors of forks.

• The configuration option Config::Micro::KiteLongerRangedUnits is removed. Not only was it a confusing name, I decided it wasn’t something that bot authors needed to be able to reconfigure. You can still change the behavior in the code at Micro::AlwaysKite().

• The command "go queue barrier" can be used in build orders to prevent queue reordering. It was explained above.

code

• CombatCommander::getMainAttackLocation() renamed to getAttackLocation(). If I keep this up, eventually all the names will make sense.

terran and protoss

• Tank siege and unsiege are much improved. There’s less “Siege! Unsiege! Siege! Maybe fire a shot? No, unsiege!” Terran tank play is considerably stronger—it is like putting dirt in your eye, instead of blinding yourself with a red-hot poker. There are other minor changes to tank behavior.

• Wraiths now use the same kiting code as mutalisks.

• Goliaths, wraiths, and scouts prefer air targets over ground targets. Suddenly they realize that their air weapons are much stronger. This change and the previous one were made first of all to improve terran wraith play, now that there is a VultureWraith opening. Overlords beware.

• Protoss carriers do not target enemy interceptors. In carrier-on-carrier battles, Steamhammer’s carriers were ineffective because they tried to hit the swooping enemy interceptors, and failed—“I’ll hit that one... no. That one... no. That one...”.

zerg

• If we need a macro hatchery and adding it won’t delay important production, just add it! There was a bug where Steamhammer thought that adding a macro hatchery would prevent it from getting critical tech units in time, even when it had plenty of resources and adding the hatchery could only help. This fixes what I think is the one biggest cause of breakdowns in zerg macro. (There are more.)

• If we have a big mineral excess, be more eager to add extractors. This should improve gas-starved late game play, which is especially noticeable versus XIMP by Tomas Vajda.

• If we scout the enemy doing a fast rush, and we’re doing a fast rush ourselves—don’t react. Steamhammer’s current reaction is not appropriate for the situation; it’s better to stay the course.

• The gas steal sometimes failed because the strategy boss incorrectly dropped the extractor from the queue. Fixed.

• Fixed a bug that prevented 4 pool from working. I think this one has been around for a while; I have occasionally noticed odd games that were probably caused by this bug, but never diagnosed it because 4 pool is played so rarely.

• I made a minor change to the details of unburrowing a severely hurt unit after it heals up some.

• A minor change to defense against proxy buildings: Steamhammer no longer feels a need to defend against a terran proxy which lifts off and floats out of sight.

Vapor no longer, Steamhammer 1.4.2 is uploaded on SSCAIT. As I’ve mentioned, expect Randomhammer’s terran and protoss especially to play better. The usual change list and source to follow.

Once I debugged the opponent model opening selection, a long careful test revealed that—drum roll—it didn't work as intended. When the number of games against a given opponent went over 30 or so, the "clever" system for trying out new openings became overactive and gummed up the mechanism, with poor game results. It was a design mistake, and I had to redesign it with less cleverness and rewrite a large section. Now it tests as working reasonably well against both predictable and difficult opponents, over both short and long runs of games.

Tests located the stubborn bug in the gas steal, which sometimes caused the gas steal drone to sit around next to the enemy extractor doing nothing. I fixed it. Is that the last gas steal bug? Based on history, probably not....

More tests to run, and the wild shots so far are par for the course. Everything is on track. Expect Steamhammer 1.4.2 to be uploaded late tonight or early tomorrow (in my time zone).

Tomorrow: Steamhammer 1.4.2 change list. It's kind of long.

I want Steamhammer to infer the existence of enemy buildings that it has not seen. Right now, the plan recognizer explicitly encodes its conditions: If I see an early factory, or a unit that came from a factory, or an armory, or a starport, then I can guess that the enemy is playing a factory opening. The zerg strategy boss uses related reasoning. Every bit of code that wants to know if the enemy has a factory has to also look for things that depend on the factory. It’s more general and powerful to decouple the inferences. The information manager should notice when it sees things that can’t be explained by known enemy structures, and infer what was necessary to produce them. If it sees a battlecruiser, it should infer the science facility with physics lab, the starport with control tower, and all the way down. Then the rest of Steamhammer can rely on the inferences, and not have to do any extra work on its own. In principle, it could draw conclusions like “Wait, I’ve seen the whole enemy base and there is no physics lab there. I should look harder at the rest of the map.”

Research counts too. If I see a wraith cloak, I know there is a control tower, even though a control tower is not necessary to make wraiths. It’s necessary to research cloaking. If I see a unit with an attack upgrade, I know what buildings were needed to research the upgrade.

It occurs to me that there are tricky cases. Suppose I break into the enemy base and find a factory—and destroy it. Later, I reach farther into the base and find an armory. I can’t infer that there is a second or new factory; the armory could have been started when the first factory still existed. To interpret the armory correctly, I have to remember that the enemy used to have a factory, and without more information I can’t be sure whether it has a factory now. This is building inference level 1; I have to know history to draw true conclusions.

But sometimes I can infer that there is another factory. If the armory is in a place that I know was empty when the first factory was destroyed, then I know that it is a new armory. There must have been another factory, or another factory was built after the first one was killed; in any case, I can infer a factory that I did not see. To do building inference perfectly, you have to not only remember the history of enemy buildings, you have to remember the history of places. This is building inference level 2; if I understand it, I can draw more conclusions.

Building inference level 3 (I’ll call it) is drawing conclusions about the number of enemy production buildings from the number of units that they are seen to have produced. “Hmm, my scout saw your barracks under construction, and by now it could theoretically have produced n marines. You have more than n marines, so you definitely have at least 2 barracks.” Resource counting can figure into this: If your scout counted the enemy minerals, or if you have a table “it’s possible to mine x amount by time t,” or if you simply know the maximum number of marines that can be produced by one barracks without being sure when it was started, you can draw inferences. In the most general case, you know all the good build orders and their production potential, and can rule out the ones which are not consistent with what you see. Later in the game it should at least be possible to make a rough estimate of the enemy’s income, correlate it to the types and/or number of units you see, and get a fair idea of how many production buildings are likely. At this point you’re making estimates instead of drawing definite inferences.

How many levels of building inference does PerfectBot have? “Oh, this army movement suggests that you are trying to draw my forces forward. Maybe you are trying to set up a drop in the rear? I’d say that means a 20% chance of a robo support bay....”

The opponent model changes in the upcoming Steamhammer version 1.4.2 are proving tedious to debug. I haven’t hit any serious problems and I don’t expect to, but simple bugs take time too and there are a lot of details to double-check. The new opening selection feature of the opponent model has many moving parts, and each part has to be tested and inspected and shown to work with the rest of the system.

All other aspects are passing tests with flying colors, even complicated stuff. The release might happen in 2 or 3 days. Terran and protoss are substantially stronger, so stand by!

Bot authors do a lot of creative stuff. Sometimes they are so creative that I can’t figure out the idea. Like this game between Locutus and Iron: Why did Locutus’s scout probe build pylons all over Iron’s territory?

It’s not a bug. It’s an intentional feature in this version of Locutus—here is the commit. It’s called “pylon harassment” and in this version it is hardcoded to happen only against Iron. A comment says:

    // We want to build a pylon. Do so when:
    // - We have enough resources
    // - We are not close to the enemy mineral line
    // - We are in sight range of an enemy building
    // - Nothing is in the way

“In sight range of an enemy building.” The pylons are meant to be seen. The intention must be to direct the opponent’s attention, to somehow divert it from doing something Locutus doesn’t want it to do, such as leave its base and attack Locutus.

In this game, Iron was not diverted at all, and won easily since Locutus had wasted a ton of minerals on pylons. I can imagine that some bots would go wrong. Those that pull workers to defend against proxy buildings might pull too many workers and stop mining, for example. Still, if your bot does make a mistake like that, it shouldn’t be hard to fix. So if pylon harassment is useful at all, I guess it must be a trick to easily defeat some weaker bots, or bots which make a certain class of blunder and are not being updated. Or possibly it is an unfinished feature, and the way it worked in this game is not how it is intended to work. I like the second theory better.

Another comment says

    // In the future, recognize how opponents react to pylon harass and store it in the opponent model

So in its final state, it is not intended to be hand-configured, but automatically selected. It could use code similar to the auto gas steal code already in Steamhammer, which decides whether to steal gas. Or it might use code similar to the plan recognizer to decide whether the opponent made a sensible or a silly reaction, and repeat if the opponent was silly.

Randomhammer crashed—as zerg, no less. It got knocked back to 1 base with a few drones, and started to recover as normal, but after making a couple units... kaboom. Zerg has been recovering reliably from these upsets for a long time now, so it’s a surprise. I’m trying to triangulate the source of the problem from not-quite-adequate information.

The upcoming version 1.4.2 is closing in on being ready. The opponent model changes are finished, except for small details. The remaining work is a little bit of smoothing and a bunch of testing (and probably fixing). In between an untraced crash and a new and fairly complex addition to the opponent model, there’s no guessing how long debugging will take.

Ack, Steamhammer has a typo in its UCB formula! A parenthesis is misplaced. What a blunder! In UCB1_bound(), change this inexcusable mistake:

	return sqrt(2.0 * log(double(total)/tries));

to this:

	return sqrt(2.0 * log(total) / tries);

The typecast has no formal effect in C++11 and later, and made it harder to see the error.

The current Steamhammer 1.4.1 uses UCB only for deciding whether to steal gas, when AutoGasSteal is turned on. I had been wondering why it chose to steal gas so often against so many opponents. Was the gas steal really that effective? When I looked again at the code, I soon spotted the mistake.

The behavior is approximately right when the number of games is small. That’s how it passed my end-to-end tests. As the number of games goes up, it gets more and more wrong. It’s impossible to be too careful in testing. :-/

The upcoming version will use UCB for opening selection—not in the most direct way, like most bots, but with a twist to cope with the large number of openings, too many to explore. Good thing I caught the bug in time.

I wrote up Steamhammer’s protoss building problem in March as Steamhammer’s time limit problem with protoss. Since then, I borrowed the smart Locutus idea of limiting the number of gateways to 10, a workaround that nearly eliminated the problem—though it doesn’t solve it. It worked so well that I decided to remove a different workaround, a rule in building placement that many protoss buildings are allowed to touch vertically, blocking horizontal movement. That rule sometimes caused units to get trapped in between buildings and the edge of the map, so it was now causing more harm than good.

Then, in a test game of protoss versus Killerbot by Marian Devecka, the building problem came up again. Steamhammer laid out its base poorly, and no buildings could be added without ordering a new pylon first, which it didn’t know to do. The bot slowed to a crawl, trying over and over again to place buildings that, under its rules, there was no room for. The space powered by pylons was filled up.

Too infuriating! That same evening I put together an elaborate system to solve it, connecting the building manager, the strategy manager, and the information manager. It’s a version of the “absolute minimum” fix that I wrote about before. When the building manager sees that a protoss building location cannot be found, and the building requires pylon power, it sets a flag “stalled for lack of space” and refuses to make any more attempts to place buildings that require pylon power. It can still place a pylon, nexus, or assimilator. The strategy manager recognizes a building manager stall as a production emergency and orders an emergency pylon. Finally, the information manager, which keeps track of units, now keeps a set of our pylons. When a new pylon completes, it notifies the building manager to clear the “stalled for lack of space” flag if it is set.

The system seems to work. As one test, I made an opening which ordered 20 forges in a row. It was fun to write "20 x forge". Everything worked as designed: The building manager stalled each time it came to a forge that could not be placed. In the stalled state it does little work, and there was no slowdown. The strategy manager saw the stall and ordered an emergency pylon, which the building manager started. The information manager saw the pylon finish and told the building manager to unstall, and construction proceeded. It was not too efficient in terms of game play, but nothing broke or froze or ran over the time limit. And it worked repeatedly until all 20 forges were made, and the bot switched into normal play. Still, it would be surprising if such a complicated arrangement didn’t cause any bugs....

The problem to solve is a slowdown, and this change is an optimization that speeds things up: Technically it is a fix, not a workaround. But it is not a complete fix. Building placement can fail due to bugs, rather than lack of space in range of pylon power, and this fix doesn’t take that into account. A building placement bug could cause unnecessary stalls. Also I’m not convinced it will always play nice with the “oh, hey, let’s choose here as my new main base” code. I need to put a lot more work into the building code to make it fast and reliable, and then to teach it more sensible building layout.

The next version is almost finished—nominally. I need to complete some coding in the opponent model and batter it with heavy tests until it doesn’t fall down, and that is all that is in the plan. The problem, to change metaphors, is that every time I kick the ball, the goal moves away, carried by creeping features.

Here’s a new table I haven’t generated before, at least not in this form: Map balance for each race, from the 20000 SAIL games. The left-side columns give win rates for the 3 matchups. The right-side columns give game counts and win rates for each race in all non-mirror matchups. (A mirror matchup always has 50% win rate, one winner and one loser of the same race, so including those would only pull the numbers closer to 50%.)

map	TvZ	ZvP	PvT	T games	T %	P games	P %	Z games	Z %	R games	R %
Benzene	50%	54%	48%	516	51%	654	48%	657	51%	231	50%
Destination	46%	49%	50%	498	49%	652	50%	627	52%	205	45%
HeartbreakRidge	43%	55%	41%	533	50%	588	44%	637	56%	222	50%
NeoMoonGlaive	40%	51%	48%	546	46%	652	49%	656	53%	232	53%
TauCross	47%	48%	48%	529	49%	668	51%	678	51%	229	47%
Andromeda	44%	49%	51%	506	47%	627	52%	646	52%	223	47%
CircuitBreaker	43%	51%	52%	550	46%	693	51%	692	52%	233	49%
EmpireoftheSun	47%	52%	48%	492	50%	612	49%	613	52%	227	49%
FightingSpirit	45%	52%	57%	535	45%	676	52%	694	52%	217	49%
Icarus	42%	54%	47%	543	48%	600	47%	650	54%	207	53%
Jade	44%	55%	50%	519	49%	611	48%	655	54%	211	46%
LaMancha	48%	51%	43%	501	52%	619	47%	613	52%	213	47%
Python	44%	49%	50%	521	47%	624	50%	642	52%	197	51%
Roadrunner	47%	46%	50%	557	49%	648	52%	716	51%	231	42%
overall	45%	51%	49%	7346	48%	8924	49%	9176	52%	3078	48%

Compare this to the race balance tables posted a couple days ago. Overall, there are no giant imbalances; the largest imbalance is 40%-60%, a 3:2 win rate for zerg over terran on Neo Moon Glaive. Terran has a small but consistent disadvantage against zerg on all but one map (Benzene came out even). That is a genuine race imbalance in current bot play: Z > T. But averaged across all the maps, the imbalance is only 55% zerg to 45% terran, 11:9, not a value to complain about. You can look back at the older tables to see which bots are responsible for the imbalance. By no means are all terran bots worse against zerg, or all zerg better against terran, but Krasi0 and Iron do show the pattern, and so does Steamhammer.

The other matchups have some pink and some blue, but tend to approximately balance out on average. Past investigation showed that the map race balance for pro players and bot bot players were pretty much uncorrelated in AIIDE 2015: See comparing pro and bot balance. I expect that it is still true. Bots have not gotten much better at exploiting map features.

More SAIL data: How well each bot performed on each map.

	games	overall	Benzen	Destin	Heartb	NeoMoo	TauCro	Androm	Circui	Empire	Fighti	Icarus	Jade	LaManc	Python	Roadru
100382319	40	0.00%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	-	0%	0%
AILien	484	66.32%	50%	52%	66%	70%	64%	69%	76%	62%	76%	61%	73%	69%	64%	71%
Alice	233	19.31%	20%	7%	16%	24%	24%	40%	9%	24%	19%	14%	23%	13%	15%	28%
AndrewSmith	479	66.81%	56%	67%	74%	62%	69%	68%	77%	79%	51%	79%	61%	59%	64%	71%
AndreyKurdiumov	464	69.83%	61%	54%	71%	79%	75%	67%	69%	71%	67%	71%	74%	76%	77%	70%
Antiga	472	70.97%	73%	74%	69%	70%	66%	60%	72%	86%	74%	69%	57%	62%	78%	86%
Arrakhammer	467	65.31%	62%	56%	68%	70%	73%	59%	64%	64%	61%	62%	75%	65%	66%	73%
AurelienLermant	472	31.36%	38%	45%	35%	18%	35%	24%	18%	25%	32%	37%	31%	32%	38%	36%
BananaBrain	469	73.99%	61%	66%	71%	67%	85%	70%	79%	78%	76%	78%	70%	78%	81%	70%
Bereaver	501	71.06%	88%	75%	66%	85%	62%	74%	79%	63%	78%	80%	56%	51%	56%	80%
BlackCrow	459	62.09%	62%	45%	63%	75%	52%	56%	59%	74%	56%	63%	73%	62%	55%	73%
BryanWeber	460	13.48%	19%	3%	18%	14%	8%	16%	17%	8%	12%	19%	17%	30%	8%	7%
CarstenNielsen	467	56.32%	37%	55%	58%	44%	63%	58%	50%	70%	56%	51%	50%	56%	70%	68%
CasiaBot	458	51.31%	55%	58%	52%	50%	48%	48%	41%	55%	51%	48%	62%	52%	50%	46%
CherryPi	482	76.56%	84%	88%	74%	73%	83%	82%	67%	85%	78%	76%	69%	83%	70%	67%
ChrisCoxe	476	70.38%	56%	64%	80%	83%	62%	67%	66%	55%	61%	80%	79%	69%	88%	72%
Cimex	41	17.07%	0%	20%	0%	-	50%	0%	0%	0%	50%	0%	33%	33%	0%	33%
cpac	6	66.67%	100%	-	67%	-	-	-	-	100%	-	-	0%	-	-	-
CruzBot	476	19.12%	18%	26%	18%	26%	18%	26%	14%	19%	11%	26%	10%	12%	17%	24%
DAIDOES	477	27.25%	21%	26%	12%	8%	32%	37%	33%	27%	40%	26%	29%	26%	30%	37%
DaveChurchill	474	60.97%	63%	59%	64%	69%	59%	65%	58%	65%	63%	63%	63%	49%	48%	63%
DawidLoranc	458	43.01%	49%	57%	34%	57%	33%	54%	27%	57%	39%	44%	42%	38%	44%	32%
Ecgberht	472	61.23%	57%	66%	52%	64%	56%	65%	62%	60%	74%	55%	65%	71%	48%	67%
Flash	492	60.37%	63%	61%	48%	63%	68%	78%	50%	50%	70%	62%	53%	53%	61%	61%
FlorianRichoux	462	36.36%	58%	25%	43%	31%	26%	41%	36%	31%	32%	34%	45%	30%	41%	40%
ForceBot	403	48.88%	55%	57%	40%	41%	43%	52%	45%	56%	49%	37%	58%	55%	42%	55%
GaoyuanChen	481	44.49%	29%	46%	40%	51%	42%	45%	50%	53%	51%	48%	41%	39%	44%	39%
Goliat	43	11.63%	0%	0%	0%	25%	20%	0%	33%	0%	0%	14%	0%	0%	100%	0%
GuiBot	50	32.00%	0%	0%	50%	50%	33%	33%	20%	43%	100%	20%	40%	67%	0%	50%
HannesBredberg	443	31.15%	32%	40%	44%	37%	22%	33%	32%	23%	28%	29%	35%	45%	21%	19%
HOLDZ	340	20.00%	6%	12%	18%	30%	20%	9%	33%	17%	24%	24%	24%	15%	35%	14%
ICELab	481	64.03%	52%	67%	56%	59%	57%	78%	57%	71%	65%	72%	76%	73%	53%	61%
Ironbot	485	89.48%	94%	81%	92%	91%	91%	95%	88%	95%	81%	90%	87%	84%	94%	89%
JakubTrancik	481	34.93%	54%	35%	37%	38%	35%	48%	38%	9%	21%	39%	40%	42%	25%	26%
JohanKayser	486	17.28%	14%	14%	10%	36%	21%	29%	21%	15%	11%	14%	16%	4%	24%	6%
Juno	4	50.00%	-	-	-	-	50%	0%	100%	-	-	-	-	-	-	-
KaonBot	460	31.30%	39%	28%	34%	32%	44%	15%	29%	33%	40%	24%	30%	38%	22%	31%
KillAlll	481	53.01%	45%	63%	55%	57%	52%	44%	56%	64%	59%	45%	56%	59%	39%	45%
Korean	478	20.50%	22%	18%	32%	27%	25%	0%	31%	0%	26%	33%	21%	21%	0%	31%
krasi0	495	94.34%	95%	94%	94%	89%	95%	90%	90%	97%	97%	94%	97%	100%	93%	97%
Kruecke	41	24.39%	0%	0%	0%	33%	0%	33%	67%	50%	0%	33%	0%	50%	0%	50%
Locutus	321	73.21%	80%	82%	77%	59%	78%	74%	71%	68%	85%	60%	81%	61%	77%	76%
LukasMoravec	472	31.99%	45%	26%	20%	38%	28%	32%	38%	30%	29%	30%	37%	23%	39%	29%
MadMixP	484	48.35%	45%	58%	48%	46%	41%	36%	52%	38%	42%	50%	59%	61%	53%	46%
MadMixT	452	30.75%	29%	29%	28%	24%	31%	32%	25%	31%	33%	28%	28%	46%	34%	28%
MadMixZ	471	32.06%	40%	23%	26%	27%	38%	39%	46%	26%	41%	33%	19%	19%	31%	29%
MarekKadek	472	20.13%	12%	21%	17%	12%	25%	21%	26%	29%	10%	24%	26%	15%	19%	30%
MarianDevecka	463	83.59%	85%	85%	82%	91%	77%	78%	77%	84%	87%	89%	93%	86%	76%	84%
MarineHell	483	23.60%	25%	15%	25%	21%	23%	35%	19%	21%	28%	16%	14%	31%	33%	21%
MartinRooijackers	455	67.91%	67%	70%	71%	69%	72%	77%	64%	66%	64%	66%	69%	72%	68%	59%
MatejIstenik	465	31.61%	32%	39%	29%	31%	50%	33%	25%	38%	23%	42%	16%	25%	36%	26%
MegaBot2017	442	49.32%	42%	58%	41%	47%	56%	53%	45%	63%	61%	45%	26%	57%	43%	57%
Microwave	466	76.82%	86%	72%	69%	78%	78%	87%	88%	68%	80%	66%	79%	84%	71%	70%
MiddleSchoolStrats	288	48.96%	39%	67%	67%	50%	55%	43%	54%	39%	39%	41%	43%	43%	59%	38%
Myscbot	473	34.04%	57%	60%	31%	25%	27%	24%	34%	36%	46%	6%	35%	29%	27%	44%
NeoEdmundZerg	475	73.05%	80%	74%	64%	76%	69%	81%	66%	70%	69%	75%	78%	81%	80%	57%
NielsJustesen	315	31.43%	19%	22%	22%	50%	24%	46%	32%	30%	32%	30%	39%	36%	25%	40%
NiteKatP	458	33.84%	29%	36%	41%	27%	45%	24%	34%	41%	36%	18%	27%	55%	28%	29%
NiteKatT	458	56.11%	50%	80%	55%	45%	62%	53%	58%	48%	58%	60%	61%	35%	62%	63%
NLPRbot	460	62.83%	66%	70%	63%	56%	65%	57%	85%	70%	54%	63%	67%	48%	63%	59%
NUSBot	500	23.60%	23%	34%	0%	34%	27%	31%	21%	22%	31%	11%	37%	23%	29%	8%
OpprimoBot	308	13.31%	12%	6%	11%	9%	12%	8%	22%	13%	21%	19%	10%	19%	13%	14%
PeregrineBot	471	45.01%	39%	48%	34%	48%	52%	52%	47%	50%	35%	50%	47%	36%	49%	45%
PineappleCactus	490	38.78%	21%	27%	46%	31%	45%	40%	39%	45%	27%	40%	43%	55%	36%	48%
PurpleSpirit	441	50.34%	48%	48%	53%	50%	47%	51%	44%	60%	41%	61%	35%	57%	68%	44%
PurpleSwarm	451	69.84%	79%	59%	70%	65%	67%	69%	84%	70%	74%	76%	69%	60%	76%	57%
PurpleWave	456	80.04%	69%	78%	75%	85%	87%	82%	82%	74%	82%	85%	81%	80%	82%	74%
Randomhammer	496	55.65%	59%	68%	64%	69%	53%	57%	45%	56%	50%	59%	50%	42%	74%	42%
RomanDanielis	489	32.92%	24%	29%	17%	33%	39%	42%	30%	25%	38%	32%	42%	21%	47%	36%
SijiaXu	479	57.83%	57%	51%	56%	61%	59%	54%	53%	60%	57%	53%	61%	54%	64%	68%
SimonPrins	446	72.20%	79%	79%	61%	69%	57%	70%	82%	64%	74%	63%	76%	73%	73%	84%
Sling	509	26.52%	32%	22%	29%	28%	17%	33%	25%	22%	40%	11%	22%	30%	25%	33%
SoerenKlett	455	49.01%	55%	38%	49%	39%	44%	47%	55%	50%	42%	61%	53%	27%	56%	58%
Sparks	474	35.23%	35%	28%	37%	45%	42%	38%	33%	41%	32%	26%	31%	33%	32%	42%
SRbotOne	4	25.00%	-	0%	-	0%	0%	-	-	-	-	-	-	-	100%	-
Steamhammer	466	76.18%	84%	72%	81%	65%	70%	86%	67%	81%	81%	82%	73%	71%	76%	78%
Stone	481	53.22%	68%	64%	69%	40%	44%	39%	45%	57%	55%	59%	56%	48%	63%	51%
SunggukCha	463	35.42%	38%	31%	37%	35%	28%	36%	31%	38%	41%	36%	38%	39%	29%	38%
TomasCere	485	36.91%	38%	30%	19%	31%	28%	34%	53%	39%	30%	34%	56%	45%	44%	31%
TomasVajda	459	72.11%	55%	90%	74%	71%	65%	72%	78%	79%	69%	74%	73%	73%	73%	63%
TravisShelton	536	12.50%	19%	10%	5%	13%	9%	17%	12%	17%	5%	18%	14%	18%	16%	7%
tscmoo	489	72.39%	80%	77%	74%	63%	77%	69%	73%	68%	82%	76%	63%	79%	68%	67%
tscmoop	468	75.64%	82%	73%	76%	79%	81%	73%	83%	66%	78%	75%	57%	76%	69%	86%
tscmoor	485	74.85%	84%	77%	76%	80%	72%	71%	78%	76%	84%	67%	68%	66%	87%	61%
tscmooz	446	54.48%	46%	57%	53%	45%	61%	61%	62%	59%	52%	59%	50%	57%	48%	54%
TyrProtoss	477	67.51%	82%	43%	57%	62%	75%	67%	72%	86%	72%	71%	68%	63%	57%	70%
UC3ManoloBot	36	2.78%	0%	0%	0%	0%	0%	0%	0%	50%	0%	0%	0%	0%	0%	0%
UPStarCraftAI2016	451	39.25%	35%	37%	52%	39%	54%	33%	31%	33%	50%	50%	30%	36%	32%	45%
WillBot	521	44.53%	49%	43%	39%	50%	40%	38%	57%	43%	52%	49%	38%	45%	33%	50%
WillyT	140	42.86%	60%	45%	29%	60%	100%	43%	0%	22%	21%	50%	50%	78%	31%	50%
WuliBot	486	72.02%	65%	72%	79%	78%	76%	79%	67%	57%	73%	66%	74%	78%	66%	74%
Xelnaga	456	32.24%	26%	43%	40%	31%	32%	18%	24%	38%	42%	25%	33%	28%	44%	20%
YuanhengZhu	488	37.91%	28%	24%	28%	40%	38%	36%	46%	37%	41%	60%	46%	24%	36%	46%
Zercgberht	471	35.03%	32%	48%	32%	40%	31%	29%	23%	37%	31%	28%	28%	49%	40%	39%
Ziabot	478	53.97%	50%	46%	70%	39%	65%	53%	57%	51%	57%	50%	52%	48%	62%	65%
ZurZurZur	483	61.08%	61%	60%	73%	73%	58%	61%	55%	61%	62%	67%	67%	53%	58%	52%

Many bots show dramatic performance differences between maps. The differences definitely mean something, even though I often find it hard to say what. What in your bot’s play interacts well or poorly with the map features? If your bot performs poorly on a given map, maybe it should choose a different strategy there. But it’s possible that the reason has to do with specific opponents, and you have to dig into details to find the answer.

Steamhammer’s results surprised me. I think of Heartbreak Ridge as Steamhammer’s worst map, but these numbers say it is a good map. After I fix the map block issues on the map, which will probably be in the version after next, maybe Heartbreak Ridge will become Steamhammer’s best map. In any case, the next version will try to adjust its play for each opponent according to the map, once it accumulates enough data from experience. It will be interesting to see whether that successfully evens out the performance across maps.

Next: Map balance.

SAIL is still down. I decided to analyze the game data anyway. I grabbed their file of the last 20,000 game results and modified my tournament result analyzer to handle it.

Here is the overall race balance. It’s nice and even. Terran has a little trouble against zerg, and random has a little trouble against protoss, but it’s all within expectations. Of course this averages together bots of all skill levels (at first I typed “kill levels,” which I guess means the same thing). We know that there’s a good mix of participants of each race, so these numbers mean that none of the races is finding its job much easier or harder than the others.

	vT	vP	vZ	vR
terran		51%	45%	52%
protoss	49%		49%	54%
zerg	55%	51%		50%
random	48%	46%	50%

The table of how each bot performs against the different races is more interesting. This is in alphabetical order, so the number of the left is just to show how many there are. There are few random players, so that column is less informative. (Of course on SAIL, “random” only means that both players learn the bot’s race when the game starts.)

#	bot	race	games	overall	vT	vP	vZ	vR
1	100382319	terran	40	0.00%	0%	0%	0%	0%
2	AILien	zerg	484	66.32%	64%	65%	66%	80%
3	Alice	zerg	233	19.31%	18%	22%	14%	32%
4	AndrewSmith	protoss	479	66.81%	65%	72%	63%	71%
5	AndreyKurdiumov	random	464	69.83%	65%	70%	74%	69%
6	Antiga	protoss	472	70.97%	60%	62%	83%	79%
7	Arrakhammer	zerg	467	65.31%	69%	59%	71%	53%
8	AurelienLermant	zerg	472	31.36%	36%	38%	16%	48%
9	BananaBrain	protoss	469	73.99%	69%	68%	82%	80%
10	Bereaver	protoss	501	71.06%	73%	77%	64%	72%
11	BlackCrow	zerg	459	62.09%	59%	61%	68%	55%
12	BryanWeber	zerg	460	13.48%	19%	14%	9%	14%
13	CarstenNielsen	protoss	467	56.32%	50%	57%	58%	65%
14	CasiaBot	zerg	458	51.31%	62%	39%	61%	24%
15	CherryPi	zerg	482	76.56%	75%	74%	85%	57%
16	ChrisCoxe	zerg	476	70.38%	79%	69%	64%	79%
17	Cimex	zerg	41	17.07%	8%	21%	17%	33%
18	cpac	zerg	6	66.67%	100%	33%	-	-
19	CruzBot	protoss	476	19.12%	24%	26%	8%	18%
20	DAIDOES	protoss	477	27.25%	17%	29%	33%	25%
21	DaveChurchill	random	474	60.97%	57%	54%	67%	76%
22	DawidLoranc	zerg	458	43.01%	46%	43%	39%	46%
23	Ecgberht	terran	472	61.23%	59%	68%	55%	68%
24	Flash	protoss	492	60.37%	45%	64%	67%	61%
25	FlorianRichoux	protoss	462	36.36%	43%	38%	26%	54%
26	ForceBot	zerg	403	48.88%	44%	52%	50%	46%
27	GaoyuanChen	protoss	481	44.49%	50%	46%	39%	45%
28	Goliat	terran	43	11.63%	12%	18%	0%	0%
29	GuiBot	protoss	50	32.00%	36%	28%	45%	0%
30	HannesBredberg	terran	443	31.15%	37%	31%	21%	53%
31	HOLDZ	zerg	340	20.00%	33%	22%	7%	26%
32	ICELab	terran	481	64.03%	73%	79%	43%	71%
33	Ironbot	terran	485	89.48%	88%	96%	85%	88%
34	JakubTrancik	protoss	481	34.93%	43%	41%	22%	46%
35	JohanKayser	terran	486	17.28%	12%	23%	17%	12%
36	Juno	protoss	4	50.00%	-	100%	0%	100%
37	KaonBot	terran	460	31.30%	37%	38%	22%	32%
38	KillAlll	zerg	481	53.01%	59%	64%	37%	54%
39	Korean	zerg	478	20.50%	25%	30%	7%	29%
40	krasi0	terran	495	94.34%	95%	98%	91%	91%
41	Kruecke	terran	41	24.39%	43%	31%	0%	0%
42	Locutus	protoss	321	73.21%	79%	66%	77%	73%
43	LukasMoravec	protoss	472	31.99%	40%	29%	29%	40%
44	MadMixP	protoss	484	48.35%	50%	46%	47%	62%
45	MadMixT	terran	452	30.75%	46%	28%	22%	29%
46	MadMixZ	zerg	471	32.06%	36%	32%	31%	24%
47	MarekKadek	terran	472	20.13%	12%	17%	29%	22%
48	MarianDevecka	zerg	463	83.59%	87%	85%	85%	69%
49	MarineHell	terran	483	23.60%	13%	29%	22%	29%
50	MartinRooijackers	terran	455	67.91%	52%	81%	69%	61%
51	MatejIstenik	terran	465	31.61%	33%	31%	29%	41%
52	MegaBot2017	protoss	442	49.32%	60%	53%	38%	47%
53	Microwave	zerg	466	76.82%	76%	77%	76%	84%
54	MiddleSchoolStrats	zerg	288	48.96%	62%	42%	47%	56%
55	Myscbot	protoss	473	34.04%	22%	31%	44%	34%
56	NeoEdmundZerg	zerg	475	73.05%	74%	68%	74%	83%
57	NielsJustesen	protoss	315	31.43%	35%	30%	29%	34%
58	NiteKatP	protoss	458	33.84%	33%	19%	49%	34%
59	NiteKatT	terran	458	56.11%	63%	56%	51%	62%
60	NLPRbot	zerg	460	62.83%	82%	45%	61%	70%
61	NUSBot	protoss	500	23.60%	40%	18%	16%	25%
62	OpprimoBot	random	308	13.31%	14%	18%	7%	13%
63	PeregrineBot	zerg	471	45.01%	55%	48%	39%	28%
64	PineappleCactus	zerg	490	38.78%	40%	34%	41%	42%
65	PurpleSpirit	terran	441	50.34%	58%	47%	50%	46%
66	PurpleSwarm	zerg	451	69.84%	60%	69%	79%	61%
67	PurpleWave	protoss	456	80.04%	73%	84%	82%	74%
68	Randomhammer	random	496	55.65%	62%	42%	66%	33%
69	RomanDanielis	protoss	489	32.92%	27%	33%	34%	42%
70	SijiaXu	zerg	479	57.83%	56%	59%	60%	49%
71	SimonPrins	terran	446	72.20%	61%	79%	74%	74%
72	Sling	zerg	509	26.52%	15%	34%	25%	37%
73	SoerenKlett	terran	455	49.01%	58%	55%	36%	56%
74	Sparks	terran	474	35.23%	42%	38%	28%	38%
75	SRbotOne	terran	4	25.00%	100%	0%	0%	-
76	Steamhammer	zerg	466	76.18%	85%	70%	80%	58%
77	Stone	terran	481	53.22%	34%	65%	55%	53%
78	SunggukCha	terran	463	35.42%	54%	31%	25%	42%
79	TomasCere	protoss	485	36.91%	47%	38%	29%	37%
80	TomasVajda	protoss	459	72.11%	62%	81%	72%	74%
81	TravisShelton	random	536	12.50%	13%	11%	13%	10%
82	tscmoo	terran	489	72.39%	83%	58%	80%	72%
83	tscmoop	protoss	468	75.64%	78%	76%	73%	78%
84	tscmoor	random	485	74.85%	81%	74%	68%	91%
85	tscmooz	zerg	446	54.48%	71%	53%	42%	61%
86	TyrProtoss	protoss	477	67.51%	53%	63%	78%	97%
87	UC3ManoloBot	terran	36	2.78%	10%	0%	0%	0%
88	UPStarCraftAI2016	zerg	451	39.25%	49%	46%	27%	34%
89	WillBot	random	521	44.53%	39%	50%	45%	38%
90	WillyT	terran	140	42.86%	37%	44%	48%	33%
91	WuliBot	protoss	486	72.02%	58%	77%	76%	78%
92	Xelnaga	protoss	456	32.24%	38%	36%	25%	30%
93	YuanhengZhu	protoss	488	37.91%	44%	41%	28%	51%
94	Zercgberht	zerg	471	35.03%	37%	39%	31%	30%
95	Ziabot	zerg	478	53.97%	51%	51%	60%	50%
96	ZurZurZur	zerg	483	61.08%	63%	53%	68%	59%

A lot of interesting details can be seen in which bot is better against which race. Iron does well against all races, but smashes protoss into the ground. I find it surprising that MarineHell scores better against protoss. Zerg KillAlll has a striking pattern in that it does well against terran and protoss, and can’t cope with another zerg. Zerg NLPRbot does much better against terran.

I’m pleased with Steamhammer’s numbers. In the last AIIDE tournament, Steamhammer played well against zerg, but unconvincingly against terran and protoss. This version looks good against all races, though still weaker against protoss.

Next: Data broken down by map.

A question for bot authors.

One of the changes Dave Churchill made to UAlbertaBot after I forked off Steamhammer was to make it work either as a DLL with direct access, or as a separate EXE. For running in a competition I think the DLL is more natural, because it should have lower overhead. But there are reasons to prefer the EXE. Dave Churchill said that it made his debugging much easier, and he strongly recommended the change.

For my debugging style, I don’t see an advantage. I could borrow code from UAlbertaBot, and unless I’m missing something it wouldn’t be much trouble to implement. Nevertheless, I haven’t been in a hurry. But maybe I don’t get it, or maybe I am some kind of freak. So my question is:

Do you see it as a useful feature? I expect I could implement it for the version after next, if people want it.

Update: After painstaking examination of the quantity and tenor of the comments, spending entire seconds of deep concentration on the task, I conclude that there is other than a groundswell of support. Steamhammer will stay DLL only.

Working on the opponent model today, I made one of the key changes for the next version:

    "UseEnemySpecificStrategy" : false,
    "EnemySpecificStrategy" :
    {
    },

No more openings hand-configured for known opponents. Steamhammer has to figure out everything on its own. I’ve been working toward this for a long time, and it’s good to finally take the step. I expect play to become more varied—Steamhammer is likely to discover surprising solutions for some opponents. Play should also become stronger, especially in tournaments where opponents like to prepare specially against select enemies. They’ll have to look for ways to exploit Steamhammer’s tactical and micro mistakes, because the game plans will be too adaptive.

I also wrote the terran vulture-first recognizer for the plan recognizer today. It recognizes a plan called Factory that can only be followed by terran, and Steamhammer zerg is configured to counter the plan with the AntiFactory opening. Testing against Iron, it worked perfectly: The first game, Iron won easily. The second game, Steamhammer countered and fought back hard (and happened to win). That’s how it’s supposed to work.

The recognizer was easy to write. Maybe I should write a few more recognizers and counters.

Iron should be a good test case, because Iron is strong enough to usually defeat the counter—AntiFactory puts up a tough battle, but still mostly loses. Opening learning success looks like this: Steamhammer realizes that AntiFactory is probably best, though not all that good, and explores other openings sometimes but not too often. I think I should be able to get that right.

Will playing better games against Iron entice voters on SSCAIT? I think it might happen. If so, I will quickly grow bored with similar Iron-Steamhammer games, but stream watchers may be pleased. Iron would likely lose a few elo points to Steamhammer on average, instead of gaining as it does now.

The upcoming version 1.4.2 has important improvements for all races, including some improvements I haven’t mentioned. Strategy, macro, and micro are better. Look forward to higher rankings for Steamhammer and Randomhammer.

The next version will be Steamhammer 1.4.2. I still have a bunch of work to do before it will be ready to release. I’ve implemented “side features” (meaning stuff I added for this version beside my main goals of map analysis and the opponent model) that will improve macro for all races, but especially terran and protoss. Micro is improved for all races, especially zerg and protoss. But I haven’t finished most of the main features that I promised! The map analysis works but is not well tied in to the rest of the code yet. The opponent model features will improve strategic play for all races—I expect they will improve play a lot. But the opponent model features are not ready at all.

So I’ve decided to get back to my main goals and skip further side features for now. Well, I’ll fix a broken improvement to wraith micro that I already wrote, but that’s all I plan. Pushing ahead on the main goals will get the release out sooner.

Earlier on this release, I thought I was working a lot on openings. I made a zerg AntiFactory opening to counter terran vulture openings and their followup, and it does its job. (By the way, more experience shows that even with AntiFactory, Steamhammer loses most games to Iron. Steamhammer stops the vultures, but has other weaknesses that Iron exploits. Still, zerg puts up a fight and does win occasionally, and that’s good enough for now.) As part of testing it, I made a terran VultureWraith opening, which is why I wanted an improvement to wraith micro. (VultureWraith seems to work well against unsuspecting zergs.) Plus I added various other openings, and fixed the accidentally broken OverpoolSpeed.

Now I feel as though I need to put in more time than I have on openings. I started working through Antiga’s new openings, beginning at the beginning with 6Gate. I got distracted after realizing that different followups for 6Gate put different pressure on the opponent, and had to stop when I saw that it was going to take too much time. I expect that I will slot Antiga’s openings into the configuration file, but I will probably not set any of them to be played, because I don’t have the time. Well, the following version 1.4.3 is planned to collect opening data itself and decide on its own how often to play openings, so I guess it’s OK. And for 1.4.2 I plan to play “never play this” openings when all else fails, so even in this version the new openings may help a little. Krasi0 will at least get to exercise its different reactions.

Since production goals are working well, upcoming versions will emphasize production goal features, alongside the map analysis features that will let me turn off BWTA. Each new production goal feature should further improve macro, until I can finally drop BOSS from Steamhammer altogether. In both BOSS and BWTA, I’ve come to feel that the B stands for Big and Bad.