Starcraft AI blog

This picture of Steamhammer versus XIMP by Tomas Vajda is a perfect illustration of why Steamhammer needs a smarter tactical analyzer.

The hydras decided that they had to retreat from the carriers at the top. Where to retreat to? “Hey, look, those scourge have found a great location! It’s a safe spot, it’s close to the enemy base—let’s go there!” It follows a heuristic of trying to keep its units together. It hardly needs saying that the entire hydra force was lost trying to run past the cannons. In big army games, Steamhammer makes similar blunders nearly every game.

The poor scourge play is another issue....

The picture also illustrates the silver lining to Steamhammer’s excessive retreating weakness in this version. Because Steamhammer now plays weakly when it is locally behind, it lost games playing the merely strong 3 hatch hydra opening that it had settled on in the previous version. It was forced to find an even stronger counter to XIMP, 3 hatch before pool into hydras. Look at the worker counts—even after losing this army, zerg is still ahead because its massive economy can replace the losses while protoss will remain unable to expand. Thanks more to XIMP’s objectively bad strategy than to Steamhammer’s play, and despite the severe blunder, Steamhammer put the game away easily.

Steamhammer is coming along steadily for AIIDE. It is already noticeably stronger than the release version now playing on SSCAIT.

Defiler skills may never be as good as I want; defilers are difficult. But the skills are there, and zerg can lay down the law in the late game. I can’t always predict whether Steamhammer will swarm or plague in a given situation. It’s satisfying to see a carrier plagued, and there is a bonus in the scoring function to favor exploiting plague-on-interceptor. XIMP has no hope.

Next I made some micro changes. After playing games against several opponents at very slow speed to watch the fast decisions, I’m convinced that Steamhammer is making fewer wasted movements and fighting more efficiently. I want to make further improvements, but it’s enough for the moment. I’ll see how the time works out.

I’m thinking through a more detailed plan for the tactical analyzer. While that is crystallizing, today I am working on the retreat problem which has left the current Steamhammer weaker than the previous version. Once retreat is ironed out, I should have the strongest Steamhammer yet and a good foundation for the next changes.

Meanwhile, I notice that PurpleWave has learned how to delay the zerg natural in the early game with its scouting probe by denying the hatchery from starting. It’s the classical reason usually cited for why zergs switched from 12 hatchery to overpool against protoss. I deliberately chose 12 hatchery for many of Steamhammer’s openings because no protoss bots had the hatchery denial skill, and now that has changed. Besides opening pool first, other standard reactions are to make the second hatchery elsewhere (at what would otherwise be the third base) or to fight a little micro war of drone versus probe—or both at once. Hmm, so many choices....

Steamhammer 1.4.7 source and binary downloads are available from Steamhammer's web page. This is the one with the crashing bug fix—don’t use 1.4.6. Sorry about the delay!

I am still narrowing down my AIIDE plans. I don’t know how long steps will take, so it makes no sense to plan in detail, but I want to lay down enough priorities that I don’t leave important stuff until too late.

I have optimistically given the next Steamhammer the provisional version number 2.0, on the theory that I’ll make that much progress. I may roll it back to 1.5, we’ll see!

So far, I have implemented defilers—not the most urgent skill, but an essential skill that will help. As it stands, they are modestly effective, enough that I tweaked the queen’s nest timing to get the hive a little sooner. The code to plague clumped enemies and avoid our own units worked on the first try, and was very satisfying. Once I add a few micro skills and teach FAP about dark swarm, defilers should revolutionize Steamhammer’s late game.

I will also choose at least one new skill to put on pressure in the early or middle game. Most bots are not prepared for runbys, that would be a logical choice for the early game. For the middle game, no bot does large drops, or proper lurker contains (though OpprimoBot does try to contain with 4 lurkers). I have other ideas.

I also need to do a bunch of ordinary micro improvements and weakness fixes. It’s a large amount of work in total, but it breaks into pieces so it’s easier to fit in.

Do it right, don’t do it fast is still the principle that I instinctively want to follow, especially after I’ve deliberately deviated from it and come face to face with crashes and poor play. Going with defilers shows that attitude. Meanwhile, the overall development plan says that now is the time for more tactical and micro skills, because strategy skills are stalled waiting for them.

What I most want to do is offline machine learning for strategy, operations, tactics, and micro. (The opponent model uses online learning.) I have selected my method and I’ve been reading up on it, and I’ve written about half the code I need to start playing with it. It is not a priority for AIIDE. The first offline learning feature will be the easiest and the most foundational, an evaluation function “am I winning?” It will be fun for stream watchers, and it will be good for opening selection and unit mix decisions—strategy items that are not chief next steps. Also, offline learning will take many days of computer time while Steamhammer crunches games and tries to draw conclusions from them, and I might have to spend that large amount of time repeatedly to get industrial-strength results. I’ll keep working on it because it’s what I want to do, but I shouldn’t let it distract me too much.

A tactics sequencer would be good for clearing map blocks, breaking through backdoor entrances, complex drops, taking islands, planning how to expand against opposition (instead of sending drones out to die), and other stuff that takes multiple steps. I may do that, it’s tempting.

More urgent is probably a tactics analyzer that is better at figuring out how to organize forces, how to maneuver, where to attack, and especially when to retreat and where to retreat to. Steamhammer loses a lot of games to mistakes on those points. Unless I think of a cheap fix to the major problems, that will be my choice for AIIDE. It’s a big enough feature to justify a 2.0 version number.

This Steamhammer version is not performing as well as I had hoped. It seems that the new weaknesses more or less offset the new strengths. Also, every time Steamhammer’s play changes, its past learned data becomes less effective. This change was big enough that some counters which used to work against a given opponent no longer do.

• The new retreat behavior in defense looks like a net loss. As I mentioned in the change list, it prevents the severe mistake of making a last stand outside of static defense range, but causes the severe mistake of fleeing from a stronger enemy when it should engage. I think the mistake caused is probably worse than the mistake prevented. For example, Steamhammer lost its ability to reliably defeat XIMP by Tomas Vajda, and excess retreating by defenders is the main cause.

The upside is that it’s relatively easy to reduce excess retreating without causing premature last stands. It was always on my list, I didn’t have time.

• The other defense changes are good, but don’t help greatly. Steamhammer is a little more effective at assigning defenders, and I may be the only person who can tell.

• Predicting enemy unit movement seems to be a win overall. Enemy scouting workers are caught, and units trying to slip past screening zerglings rarely get through. Zergling versus zergling micro is sometimes dramatically stronger—attack with fewer and no advantage in formation, win anyway. The disadvantage is that units are more likely to get stuck, either stuck in place on their own or stuck on each other. I have to dig in and solve that.

• The improvements in overlord safety help against many opponents, but do not help enough to make a big difference. Probably Steamhammer can defeat weaker enemies a little more consistently. Against protosses that make corsairs but don’t have strong overlord hunting skills, like Skynet, the improvements can save games. Against those with strong skills, like BananaBrain, Steamhammer holds on well at first and survives longer into the game, but the slaughter begins as soon as observers or dark templar come out and the overlords leave the safety of their spore colonies. Alongside better overlord movement, Steamhammer needs better air defense skills with hydras and scourge.

• I suspect that the heuristic “the enemy is ahead in army, make more combat units” rule is not tuned well. I think it helps on average, but not as much as it could. And there are times when it is wrong.

• The “oops, expanding failed, I’ll turn it into a macro hatchery” rule causes Steamhammer to make many extra macro hatcheries when it is contained. It may be an improvement over losing an infinite stream of drones, but it’s still awful. There is an obvious quick-and-dirty fix of canceling excess macro hatcheries in the building manager before they are started—but I think that will cause Steamhammer to go back to losing an infinite stream of drones, while doing unnecessary work in between. Have to bite the bullet on this one and perform serious surgery.

• The better sunken and spore placement is all win, despite occasional odd results. “Ooh, sunkens right up against my ramp! Units can’t get out? So what, it’s safe!” Now I need to add the production goal to put the right static defense at every base that is vulnerable to vulture or air harass.

Next: AIIDE plans in a little more detail.

Whew, crashes are fixed. Stand by for the source release. For CIG, the question is whether Steamhammer will do poorly, or be disqualified outright.

I plan the next release well in the future, after the AIIDE deadline on 1 October. That is enough time to make major improvements. I haven’t decided what improvements to make, but I have a shortlist. There are the usual suspects, overlord skills, worker skills, decent micro, mineral locking and other mining tricks. I’ve thought of several zerg skills to bring despair to the hearts of Steamhammer’s enemies—bots have big gaps in their knowledge, and maybe I will bring a tractor to drive through the gaps. I may write a capable tactics analyzer. Also I’ve selected the machine learning method I want to use for offline learning of strategy, operations, and tactics, so I may work on that. The shortlist is ironically too long to finish much of, but every item is important so any selection should make a big difference.

For CIG, I made many small changes in a short time. For AIIDE, with more time to prepare, I plan a smaller number of larger changes. I hope to replace major parts, and make enough progress to call it Steamhammer 2.0. We’ll see how it goes!

Steamhammer 1.4.7 is uploaded to SSCAIT. Change list: The crashes should be fixed. A few trivial code cleanups with no effect on play. A few comments rewritten.

Argh, crashes!

I ran a ton of test games, against both stronger and weaker opponents of all races on rotating maps, and saw no sign of any crashing bug. I eliminated nearly all crashes from Steamhammer by about April last year, and since then there have been only a handful of crashes total on SSCAIT. Every release has been reliable. But on SSCAIT, the new Steamhammer is crashing frequently, in up to 1/3 of games. If that happens in CIG, Steamhammer will finish in the bottom half if it is not disqualified altogether. I have contacted the CIG organizers to warn them of the possible disaster.

The symptoms indicate a severe bug which happens in many games and causes a crash or not depending on details of the environment. It’s likely a pointer or iterator thing. Crash records point to squad update routines for different squads. Those were all changed to assign or unassign detectors depending on the situation, so I know where to start looking.

Please stand by while I employ my fine-tooth comb. It’s too late for CIG, but the rest of the world can be saved. I’ll issue a new version when it’s fixed, and I hope it won’t be long.

I’m eager to get on with the real work, but crashes come first. Argh!

Update: I found bugs of the general kind I expected in the places I expected. As usual, the bugs were obvious once I read my own code carefully. Lesson: I was in too much hurry. Code review works even with only 1 coder. I’ll upload version 1.4.7 after reading through again and retesting.

Steamhammer 1.4.6 is uploaded to SSCAIT. Steamhammer's web page is updated with source and binary downloads.

For an idea how much this version improves over 1.4.4, see its first game Steamhammer-Krasi0.

There are only a few visible changes from the CIG version 1.4.5. Terran bunkers and protoss photon cannons in the opening are built @ front in a defensive location, which is usually better. A terran bunker built as a strategy reaction by StrategyManager is also @ front. Reactive photon cannons and turrets are in their usual questionable locations, because they are meant to cover the base, not to defend the front.

I also adjusted the rules for dropping items from the production queue so that terran and protoss openings will execute as written. Formerly, a few openings might drop items and break. See MacroAct::hasTech().

Both changes affect the protoss ForgeExpand opening. It’s str0nger.

Predicting enemy unit movement is easy and fun, and it helps micro a lot. Now that I’ve implemented a version in Steamhammer, I recommend it for everyone.

Adding this feature dramatically improved Steamhammer’s micro in some situations. It affects almost every fight; with prediction, your units do less moving and more shooting. It’s most obvious in a chase. When chasing the enemy scouting worker, zerglings with prediction don’t pursue endlessly, but catch it. The most extreme case may be playing against Stone once zerglings come out. Stone’s SCVs flee like the wind, or they twist and turn trying to evade, but they can’t escape. Steamhammer wins with much less useless chasing.

making the prediction

Here’s my implementation in Steamhammer: Assume that the enemy unit will keep moving in a straight line at a constant speed, and predict a fixed number of frames ahead. It’s dead simple. ClipToMap() makes sure that we return a valid position—we may predict that the enemy falls off the edge of the world. ClipToMap() simply restricts x and y to valid values,. We predict a position closer to where the enemy will end up if it bumps into the edge of the map.

// Predict a visible unit's movement a given number of frames into the future,
// on the assumption that it keeps moving in a straight line.
// If it is predicted to go off the map, clip the prediction to a valid position on the map.
BWAPI::Position PredictMovement(BWAPI::Unit unit, int frames)
{
	UAB_ASSERT(unit && unit->getPosition().isValid(), "bad unit");

	BWAPI::Position pos(
		unit->getPosition().x + int(frames * unit->getVelocityX()),
		unit->getPosition().y + int(frames * unit->getVelocityY())
	);
	ClipToMap(pos);
	return pos;
}

There are fancier ways to predict. You can take terrain into account; an enemy ground unit will probably not pass through unwalkable terrain (though there are ways to do that). You may want to take the chasing unit into account—can it catch up? I believe Iron calculates the closest interception point where the chasing unit can catch the enemy, and heads straight there. That is a more elaborate calculation, but not too difficult, and there are several ways to do it.

using the prediction

How you use the prediction depends on your code, of course. In Steamhammer, it is as simple as making the prediction. Steamhammer normally calls Micro::AttackUnit() to issue an attack command. I added Micro::CatchAndAttackUnit() which attacks if we are able to, and otherwise issues a move command toward the predicted enemy position 8 frames in the future. Here is the working part (I took out irrelevant stuff like error checking and debug drawing).

	if (!target->isMoving() || !attacker->canMove() || attacker->isInWeaponRange(target))
	{
		AttackUnit(attacker, target);
	}
	else
	{
		BWAPI::Position destination(PredictMovement(target, 8));	// the number is how many frames to look ahead
		Move(attacker, destination);
	}

Then I substituted Micro::CatchAndAttackUnit() for Micro::AttackUnit() throughout the code wherever it made sense. The condition !attacker->canMove() means there is no problem if you call it for an inappropriate unit, like a burrowed lurker or a sieged tank, because then it behaves just like the original call. (But I didn’t call it for inappropriate units.)

I chose 8 frames fairly arbitrarily. I think a range of values will work. With 8 frames, we aim for a spot a little ahead of where the enemy is, and not so far ahead that they can throw us off by turning. Dan Gant points out in a comment that PurpleWave predicts a variable number of frames ahead depending on when the chasing unit will be able to attack; the goal is to come into range at the same moment that we are ready to attack.

It makes sense to predict further into the future if the enemy is far away. Calculating the interception point like Iron is the ultimate version of that. Maybe the ideal method is to find the interception point when far away, and switch to the Purple idea when approaching attack range.

There is a disadvantage to the change in Steamhammer. There seems to be a higher rate of units getting stuck, unable to move. I haven’t yet looked into why. It affects protoss the most. It’s possible that the change makes protoss play worse overall.

Question for stream watchers: Would you like some debug drawing turned on? Steamhammer traditionally runs showing only the game info and production queue. Maybe it would be fun to see more? Possibilities: Unit orders (what each unit is doing all the time—including what mineral patches are doing, namely nothing!), squad info (each squad’s order, target location, and retreat status), zerg strategy boss info (basics of zerg strategy decisions). I don’t want to turn on more than one of these at a time, but maybe I could change the settings now and again.

It seems that SSCAIT is down, so maybe I’ll upload the new Steamhammer there tomorrow.

In the meantime, here is Steamhammer 1.4.5, the CIG version. This is the same file I uploaded to CIG, not even renamed. It includes source and binary and the configuration file. Unlike an SSCAIT upload, it does not include BWAPI. Following CIG rules, the ability to configure the game speed and frame skip is disabled—the configuration options are ignored. This and a few other minor details will be changed in version 1.4.6 for SSCAIT.

CIG submission is closed. I received no acknowledgement other than “upload successful” with the second browser I tried, so I don’t have full confidence in the process. Anyway, here is the change list for Steamhammer 1.4.5, the CIG 2018 version. Instead of laying foundations as I have been doing in recent releases, this time I concentrated on making the biggest play improvements I could in a short time. Steamhammer should play visibly better, though I can’t judge how much. The most important changes are in bold.

As I mentioned yesterday, the CIG version is not quite right for SSCAIT. I need to undo small changes that uphold unhelpful CIG rules, and to test terran and protoss. Expect version 1.4.6 tomorrow or thereabouts.

overlord behavior

• Release the scout overlord when the enemy is seen to have anti-air. Against terran, almost any enemy unit or building implies anti-air tech, so the overlord tends to be released early. Normally, an overlord released early in the game leaves the enemy base; later in the game, it may not.

• Always send out a scout overlord. The exception of sending no overlord against terran is dropped, because the overlord is now safe enough. A terran base may be found sooner, and the response to a terran worker rush is improved because the overlord hangs around the enemy base.

• Overlords not assigned other duty now go into an Overlord squad. The overlord squad furnishes 2 easy skills: 1. It assigns 1 overlord to each base, so that Steamhammer stays ready for cloaked units. 2. If the enemy has flying overlord hunters, and Steamhammer has spore colonies, gather free overlords at the spores. The skills are crude; if neither condition applies, the overlord is left where it is, even if it is under attack from the ground—and even if it is a released scout overlord in the enemy base. Nevertheless, it’s a clear improvement, because Steamhammer used to have no overlord survival skills whatever. Better would be to retreat from danger and seek out safe places to watch from.

• If the enemy has no cloaked units, and the enemy does have anti-air, then assign no overlord to the Ground, Flying, or Recon squads. Missing the overlord is unfortunate, because the overlord provides a wider vision range and can see uphill, but keeping overlords safe has priority. The change saves overlords from being shot down by marines or dragoons. The game commonly proceeds: Overlords are assigned; enemy gets anti-air so overlords are released, and may be shot down but at least no further overlords are sent to die; enemy gets cloaked units so overlords are assigned again, because now it is worth the risk.

• Assign a detector to each defense squad, under the same conditions of cloaked enemies and anti-air. Defense squads used to simply hope that overlords were hanging around the base in detection range of intruding dark templar, and the hope was often dashed.

other tactical changes

• Predict enemy movement. When moving to attack a specific enemy unit, most units now move toward a predicted enemy position 8 frames in the future, rather than the enemy’s current position. Mutalisks and vultures don’t use prediction, because they are controlled by special-case code that works differently. The main purpose may sound surprising: To get rid of the enemy scouting worker! Using prediction, even a slow zergling can catch and kill an evading worker. I think Arrakhammer was the first zerg to demonstrate the skill, and Stone may have been the earliest bot to predict enemy unit movement. Besides more effective chases, unit micro is improved in general, because units come into firing range sooner.

• Squads defend and retreat toward the starting base, if we still own it, not the ever-changing “main” base where we add buildings. This change prevents one major cause of retreating through the enemy formation. There are other causes, so it still happens.

• When a unit in retreat finds itself blocked by an enemy (according to a simple geometry calculation), instead of continuing the retreat it stops to fight on the spot. I think this is probably a net win because it tends to delay the enemy advance, but sometimes units that could slip by safely instead stop to fight and are lost unnecessarily.

• When regrouping, air units stick closer together. Historically, regrouping treated air units and ground units the same. But air units stack, and often gain an advantage by being stacked, so now they get a special case. This modestly improves mutalisk play (on average) by grouping them tightly when they reach their retreat point.

• Defend is a regroupable order, or in other words, a defense squad retreats when the enemy is superior. I’m not sure whether this is a net win by itself; I didn’t have time to add the smarts that would make it a definite gain. It prevents the severe mistake of making a last stand outside of static defense range, but causes the severe mistake of fleeing from a stronger enemy when it should engage to delay as long as possible.

• Static defense is never considered 100% adequate; when under attack, Steamhammer always assigns at least 2 mobile defenders. Formerly, Steamhammer would calculate “eh, the sunkens will hold.” It was usually right, but that did not help when the enemy maneuvered to harass while staying out of sunken range. Iron does that, for example.

• If an enemy attacking a base has no anti-air, prefer to assign flying defenders. Mutalisks are better than zerglings against vultures, but also against tanks, zealots, and firebats.

• Because I was doing a bunch of work on squads, I clarified the hard-to-read squad info display for debugging. Map coordinates are converted to tiles rather than shown in pixels, which makes them shorter and easier to interpret. The Base Defense squads are renamed to Base squads for brevity. Most importantly, each squad shows its attack/retreat status at the top of the screen on its line of the squad info display, rather than at the bottom of the screen overlapping with the status of other squads.

player methods, not type methods

UAlbertaBot always got unit ability information from the unit’s type, and I copied that in Steamhammer when adding code. I was slow to realize the advantage of getting the information from the player object, which knows whether the unit has been upgraded.

After these changes, I can see the difference in hydralisk play, but I am not sure how important it is in practice. There was no time for a long test to find out.

• Changed unittype->topSpeed() to player->topSpeed(unittype) in several cases. The biggest effect is on kiting of hydras (since hydras have a speed upgrade). There is also a small effect on the decision of whether to chase after an enemy unit. Vultures have a speed upgrade but are not affected, because they are kited using the same special code as mutalisks, and that code works differently.

• UnitUtil::GetAttackRange() uses attacker->getPlayer()->weaponMaxRange(weapon), which simplifies the code and is probably accurate in more cases.

• Micro::KiteTarget() also uses unit->getPlayer()->weaponMaxRange(weapon) to find weapon ranges for both the kiting unit and its target. It affects hydra and dragoon kiting, because those units have range upgrades. It also affects whether kiting happens, which depends on whether the target unit has a longer range.

other stuff

• MacroLocation::Front added to represent “the front of the base,” the spot in either the main or natural where defenses should go. The calculation is done in a simplified way. If there is no natural and the main is the front, then because Steamhammer doesn’t understand chokes yet, the front is in the direction of the natural, which is often (but not always) close to the direction of the ramp to the natural. If the front is the natural, then for the same reason the front is opposite the minerals. See Bases::frontPoint() for the easy calculations. Passing units disturb building placement, and units often pass the front, so unexpected placements are possible.

In most cases, creep colony @ front is a much better place to put defenses. Sunkens are usually set close together instead of spread around the hatchery, and on most maps tend to be somewhat toward the natural choke and somewhat toward the ramp, defending both. There are times when Steamhammer blocks its own ramp with sunkens, though. You can get the old behavior with creep colony @ macro or creep colony @ natural.

MacroLocation::Front is used in most openings and in strategy reactions in the zerg strategy boss. It is not used for placing spore colonies, which should typically not be all the way in front.

• There was a serious bug in PlayerSnapshot::takeSelf(), which counts our units by type: All units had count zero. It worked when I wrote it, but suffered an editing slip later. It formerly wasn’t used, so the bug didn’t affect play, but now it is used.

• Recognize a cybernetics core as anti-air tech only when it completes. This is a tweak to convince the scout overlord to stick around an enemy protoss base until the cyber core finishes, so the overlord watches as long as it can without putting itself in danger.

• Added a new method InformationManager::enemyCloakedUnitsSeen(), which recognizes a different level of danger than enemyHasCloakTech() or enemyHasMobileCloakTech(). It’s used to make overlord decisions.

• The cloak tech recognizers now recognize cloaked units that they can’t identify by type because they are not detected. What an odd oversight.

• Certain air tech and cloak tech recognizers cross-set each other’s flags when appropriate: Yes, if you have overlord hunters, then you necessarily have air tech. This is a speedup optimization that does not affect play.

• The resource box is fixed. I changed BuildingPlacer::computeResourceBox(), which computed for the starting base only a box around the minerals where buildings should not be placed, to reserveSpaceNearResources() which marks as reserved those tiles at any base which should not be built on to avoid mineral or gas mining problems. Since sunkens are now normally placed @ front instead of surrounding the hatchery, the effect is not big. But spores are still placed around the hatchery, and this prevents spores from messing up mining.

• MicroRanged understands that dark swarm does not protect buildings. Units mistakenly believed they could not attack a building under swarm. Oops.

zerg strategy and reactions

• If too far behind in army size, make more army. Marian Devecka called out army size as a major weakness (too little army due to making too many drones), and Proxy suggested an easy way to repair it. I knew it was a weakness, but I had resisted making changes to the drone/army balance because it is difficult to do right. This way is not right; it is rough and will go wrong in common cases. Besides being crude, it is probably not tuned well due to lack of testing. But it is still much better than what Steamhammer did before. Play against protoss especially is clearly improved, though Steamhammer sometimes expands too little instead of too much.

• When rebuilding drones to recover from a severe setback, mix in zerglings if the enemy army is stronger. It should improve the odds to survive some major emergencies, especially in ZvZ.

• Be slower to make macro hatcheries, and place fewer of them at expansions. This was a critical weakness that made it difficult to survive heavy rushes, even given other adaptations. Steamhammer is now less likely to overbuild hatcheries, and more likely to float minerals. It can’t balance correctly without prediction skills that it does not have, but now it balances closer to correctly on average. It’s a net gain.

• Steamhammer classically made up to 4 defensive sunkens when necessary. I bumped the limit up to 5. It helped more than I expected.

• Be a little slower to expand solely to get more gas.

• Spore colony placement is a little smarter. Since it first gained the feature over a year ago, Steamhammer has always made only 1 spore colony, at whichever base is currently designated as the place to add buildings. Now it makes 1 spore when it sees a threat, and adds a second spore if the threat is high. Versus protoss, the first spore goes at the natural in case it is needed to help defend against dark templar, and the second spore goes at the current “main” (which may not be the starting base). Otherwise the placement is the other way around: Main then natural. There are cases where it builds the 2 spores at the same base. The spore colony change works together with the overlord changes to improve both overlord safety and dark templar defense.

protoss

• Added a one-base CorsairZealot opening for protoss, which I used to test zerg overlord reactions. The opening turned out to be strong against Steamhammer’s zerg, so I configured it to be played.

code changes

• Fixed a latent bug in CombatCommander::updateIdleSquad(). In aviation safety terms, this is a major hazard, which is defined as “you’re still OK as long as nothing else goes wrong.” Make programs as safe as flying!

• Various stuff related to bases moved from InformationManager to Bases. The process is still underway.

• Deleted the unused methods InformationManager::getIndex() and nearbyForceHasCloaked().

• Clearing of neutral “blocker” buildings was written for the map Sparkle. It turned out to have a bug on the map Pathfinder. At the natural of the top base, one of the neutral eggs was recognized as a blocker and destroyed—not ideal, but not much of a problem either. The bug is that Steamhammer failed to recognize that the egg had been destroyed, and units were permanently assigned to keep trying to destroy it. Instead of fixing the bug, I turned off the tactical reaction to blockers. It was easier and has bad effects only on rare maps like Sparkle.

I have wrapped up work on the last feature for the CIG version of Steamhammer. No more changes unless testing turns up a last-minute critical bug.

I’m pleased with how much I got done in a short time. With perfect planning I could have done more, but hindsight is like that. Looking through my change log, I feel I’ve done a good job improving play as much as I could with small changes. I also found a few good bug fixes. Steamhammer should do well enough in CIG.

Zerg got more work than the other races, of course, but most of the changes are not specific to zerg. The changes were intended to help zerg and were only tested with zerg, but the other races should benefit too. I even added a strong new protoss opening versus zerg (which I used in testing zerg).

Expect the SSCAIT upload of this version on Wednesday, more or less. I’ll test with terran and protoss first, and make any necessary tweaks (so it might be a slightly different version). In recent releases most of the important work has been on the opponent model and points of strategy. Those changes made Steamhammer smarter, but to a stream watcher, the play looked as though it had hardly changed. This release will be different. Plenty of familiar old weaknesses remain, because there was only so much I could do, but some very visible problems are reduced and I have changed long-standing patterns of play. Watchers should notice. Steamhammer’s play should look different.

Here is a question of virtually no importance that nevertheless has me puzzled: What should I call my learning files?

For now, Steamhammer has an opponent model file for each opponent, named om_[opponent].txt. OM for opponent model. It makes sense, or anyway it makes as much sense as an arbitrary filename can. I like names that make sense.

The next learning data I want to add is opening models, so that Steamhammer can know the timings. With knowledge of both opponent strategies and its own opening strategies, it will be able to directly compare and find counters: Your attack comes at this time, which opening is ready at that timing? It will also be able to recognize which openings are similar to others, so that when it can’t match a strategy, finding good openings by trial and error is quicker.

What should I name the opening model files? Opponent models and opening models are both empirical data, and should be updated as games are played. But the unfortunate words “opponent” and “opening” are too much alike to abbreviate nicely. Should I start with “enemy” and “build order”? “Bot” and “strategy”? Most abbreviations seem unintuitive. My best idea so far is to rename the om_* files to bot_* and use the prefix open_ for openings. Maybe OK?

What’s your idea? Because, according to the bikeshed principle, everyone should have an opinion on this....

Humans naturally keep track of progress toward their goals. It doesn’t matter what you’re trying to do: If you reach out for something but don’t get a good grasp, you notice. If you’re making a point in conversation, you try to tell whether the other person understood. In Starcraft, people notice (unless they’re swamped with multitasking) when a unit gets stuck, or an army takes a bad path, or really when anything interferes with a plan. And having noticed a problem, they can try to solve it.

Isn’t it obviously a good idea? And yet it seems rare in bots. I think nearly all bots only notice problems that they are explicitly coded to look for. They don’t notice when their units run into Iron’s wall and start to move aimlessly, seeking a firing position that they can’t reach. Even a novice human will realize that something is wrong, but bots don’t register that progress is stalled and keep trying to execute the failing plan.

I’ve been thinking about adding goal progress monitoring throughout Steamhammer, at every level: Strategy, operations, tactics, micro. First, I want to rewrite everything with explicit goals anyway, because I think it is clearer and more flexible. Carrying out a goal consists of choosing a plan (either ahead of time or piece by piece on the fly) and executing the plan. Then, goal monitoring means being able to tell whether the plan is executing as intended. Firing at Iron’s marine behind the wall is a 2-step plan, get into range and fire a shot. Getting into range is a subgoal to move to a position that is in range. And we can tell whether a movement plan is executing as intended: Is the range closing with time? Does the movement take about as long as predicted? If not, then the plan is going wrong and we may want to patch the plan, or try a different plan, or back up further and try a different goal.

It seems like a lot of detail work if done by hand, and I’m sure I will do part of it by hand. But it means that the bot will always react to problems. If Steamhammer is beating its head against Iron’s wall, it will notice. Even if it doesn’t have a wall recognizer and doesn’t know how to react, it will know that its plan is failing and that is should try something different—choose another target to shoot at, maybe that will work. After several tries, it will be sure to find that shooting the wall itself succeeds. Even without specific knowledge, having general adaptivity seems valuable.

It also provides a clear task structure. Today, Steamhammer’s structure is ad hoc—the underlying principle might as well be “let’s code up some behavior!” With a structure of goals and plans, the amorphous behavior becomes a programming pattern to be reused over and over in the code. Each behavior is made up of a fixed set of parts: Choose goals, plan how to meet each goal, monitor the progress of each plan, back up and try something else if the plan is failing.

The clear structure also helps with machine learning. “OK, system, now learn to behave!” is a hard problem. “Subsystems, go learn to choose goals, plan, monitor, and retry” is an easier problem, because the relationships between the parts are fixed. There is less to learn, and better information about which parts are working well.

Well, that’s my long-term vision. I expect I will get there in a slow and messy way, as always.