Starcraft AI blog

At its most general, a learning system consists of a data structure, a learning algorithm to update the data structure with new data, and a retrieval algorithm to answer questions from the learned data structure. You could say it is a specialized kind of memory, with a data store and ways to get information in and out. The information you get out is not the same as what you put in, and that is part of what makes it useful. Or you could say that ordinary computer memory is a learning system with no loss and no generalization.

One of my goals for Steamhammer’s opponent model is to learn from a single game. There is a well-known class of learning systems which makes it easy, because their data store is nothing more than a record of the original input data, called instance-based learning or memory-based learning. This simplest example is the k-nearest neighbors algorithm family: Your data is a set of points (x, y) , input x gives output y (where x and y might be, say, vectors). To learn from a new data point, simply store it along with the others. To answer questions of the form “what’s the output for input x?” you find a given number, k, of x‘s nearest neighbors according to some distance measure, and average their outputs (using whatever kind of average may be appropriate for your problem). A fancier class of systems that can draw complex conclusions from a single example goes under the name case-based reasoning, where the data store is a database of structured “cases”, or examples.

Anyway, I thought I should use a method in the nearest neighbor family. It’s the simplest way to meet my goals.

What should my data points look like? Well, what information does Steamhammer use to make strategy decisions? It looks at the enemy’s current unit mix. I want to be able to predict the enemy’s unit mix at a given time: “Oh no, those zerglings are early, it’s a rush!” or “this enemy switches to goliaths and hardly any tanks, I should build up hydralisks next.” Both are nothing more than unit mix @ time.

My data points are games, boiled down to sequences of unit mixes. In the first implementation, Steamhammer takes a snapshot of the unit mixes of both sides every 30 seconds, “this many drones, that many zerglings, ....” I also threw in some supplementary information: The map, the opening chosen, and as shortcuts for opening selection the times at which the enemy got various things, such as the first combat units, the first flyers, the first detection, and so on. And it simply appends all the data to a file named after that opponent.

To answer the question “what will the enemy’s unit mix be at time t?” the first implementation finds the nearest neighbor. It looks through the game records to find the best match game, the past game against the same opponent which is most like the current game, according to a similarity measure which adds up differences in unit mixes over time, up to the current time in the current game. (So the best match will change at most once every 30 seconds.) Having found the best match, it looks up the recorded enemy unit mix in the best match game record which is closest to time t and calls that the prediction. It’s dead simple.

That was my motivation. In fact, the game records have endless uses beyond predicting the enemy unit mix. For example, to figure out whether an opening is safe to play against this opponent, run the timings of the opening against the timings of the game records. If the opening always gets defenders in time, then the enemy will not smash you with a rush (or at least it will only be a surprise once). Or if you notice that the enemy never gets detection, then go lurkers and get an easy win. And so on.

Einstein, hand me the simplicity!

You can see why I thought the method was obvious. With clear goals and the right background knowledge, it is obvious. And you can see why I thought I could get it working within a few weeks; there is nothing complicated here. If I were a better coder, I would have succeeded.

Of course, it may turn out that the simplest option is not good enough. For the first cut I wanted to take the easiest way. If some part turns out to work poorly, I have improvements up my sleeve. The possible improvements are as endless as the possible uses.

• The recorded unit mixes include buildings. Buildings are especially important for predicting what the opponent is up to, but my first cut similarity measure does not understand that. It treats the difference between 1 barracks or 2 the same as it treats the difference between 1 marine or 2, and that is obviously not ideal.
• For some purposes, it may be better to record the total units ever made (or ever seen, if the enemy’s) instead of the current unit mix, because the current mix depends on the outcome of battles as well as the strategy followed.
• If the best match is not close at all, maybe it should be ignored.
• If there are a number of good matches, maybe they should be averaged together.
• Surely the current unit mix should have a role in predicting the next unit mix. In the first cut, it is ignored.

The bottom line is that my first implementation may or may not work adequately. But I’m confident it can be improved until it does work.

You can download the archive I submitted to AIIDE 2017, which is Steamhammer 1.3.1. It follows the AIIDE rules: It includes a compiled binary, the configuration file, and source. Unlike an SSCAIT submission, it does not include the BWAPI 4.1.2 library.

The configuration file is specialized for the tournament. The terran and protoss opening lines are removed, and it is intended to play zerg only. I did all my testing with zerg, and mostly on the AIIDE maps. Everything is optimized for the best AIIDE performance, at the expense of anything else.

Next is Steamhammer 1.3.2 for SSCAIT. It will restore the terran and protoss configuration. I also made a minor change that should improve devourer play (at least a little), and may make a few other small fixes. I need to test that protoss and terran are working well. It shouldn’t take more than a day or two, and then I’ll be right back on opponent modeling.

Steamhammer 1.3.1 is the AIIDE 2017 tournament version. I made the submission.

Here’s the change list. This version is configured specially for the tournament. On Saturday I’ll release the link to the tournament submission, and not long after I’ll upload a version to SSCAIT with a configuration tailored for SSCAIT.

I list a lot of changes, but they are small. I allowed only changes that were quick and had a low risk of introducing bad side effects, since I was concentrating on opponent modeling. I expect the net effect will be that Steamhammer plays a slightly but distinctly cleaner game. I think that people watching the games won’t be able to tell the difference unless they know what to look for, but if they watch long enough they’ll realize that the new version makes fewer strange blunders.

configuration

• Stuff related to I/O moved to a new IO configuration section. The ReadDirectory and WriteDirectory used to be under Strategy. This is where you turn the opponent model on or off. It is currently off, because it isn’t working yet.

• Added one new command, "go scout once around", which tries to send the scout on one circuit of the enemy base and then return it home. The waypoints can get messed up, so in some positions on some maps the scout follows a strange path.

• I added a half-dozen zerg openings. They were intended as options for the opponent modeler to choose among. They are 2 hatch and 3 hatch lurker openings to fill out the lurker selections, turtle openings to hold against zealot rushes, and a couple additional anti-4 pool openings to give more choices against hard zergling rushes.

buildings

• Steamhammer has a current “main base” where new buildings go. It frequently chooses a random new main base to spread out its buildings. This especially helps protoss avoid filling up its starting base and dying. The change is to not shift to a new main base while in the opening book. The effect is that the opening line is executed more efficiently, with less drone movement. See InformationManager::maybeChooseNewMainBase().

• Release the worker when a building is canceled or fails to construct. BuildingManager::undoBuildings(). There seems to be at least one more case where the building manager doesn’t release a worker when it should.

• Keep track of how long it has been since a building was ordered, and how many workers have been assigned to build it (a new workers gets assigned when the previous one is lost). If the building has waited too long, or too many workers have died trying to make it, then cancel the building. This prevents Steamhammer from sending all its workers through the enemy army to start a building (it still sends a few). It also stops the building manager from accumulating buildings that can’t be started, the cause of more than one serious bug.

tactics

• Bftjoe suggested dropping the “is any enemy unit in range?” check from Squad::needsToRegroup(). Good idea. It was originally an optimization in UAlbertaBot, and started to grow complicated as I tried to fix bugs. Let the combat simulation handle it; that way is simpler and better.

• I dropped the “we just retreated, don’t attack yet” timer from 3 seconds to 2. Also part of Squad::needsToRegroup().

FAP combat simulation

• The author N00byEdge fixed a bug in FAP in calculating concussive damage. This especially helps it understand how badly zerglings lose to vultures.

• Don’t pass carrier interceptors to FAP. As bftjoe pointed out, FAP is designed to treat the carrier as if it were doing the damage directly.

• The past Steamhammer 1.3 did not add workers to the combat sim. I changed it to add workers, whether ours or the enemy’s, which have attack orders. A worker which is actively engaged in combat will be simulated, at least while it is attacking rather than blocking or fleeing, and one which is busy working will be ignored. It should be more accurate.

• Other minor changes to which units are passed in to the combat simulator. For example, SparCraft understands detectors while FAP does not, so Steamhammer no longer passes mobile detectors to the combat sim—it won’t do anything with them. Overall, the unit selection code is simpler and cleaner. As part of this, I deleted the now unused InformationManager::isValidUnit(); the job goes to UnitUtil, which is responsible for classifying units.

other stuff

• Steamhammer had several routines which selected a mineral worker to complete some job. It failed if there were no mineral workers, even if there were idle workers that could be assigned. These routines can now choose idle workers too, fixing several bugs.

• I fixed the drone dance bug in WorkerManager::handleGasWorkers(). The bug was located by Arrak.

• Tracking of bases is not always accurate—even our own bases. I added InformationManager::updateTheBases() to periodically double-check and correct errors. So far, only the case of “we thought this was our base, but it isn’t really” is written; the other cases are unfinished. The fix corrects some serious misbehaviors.

• UnitUtil::IsValidUnit() now considers a unit “valid” when it is loaded into a bunker or transport. A loaded unit does not have a valid position, so it used to say that the unit was invalid.

• UnitUtil::GetAllUnitCount() has the job of counting all units of a given type, whether complete or not. It now counts uncompleted morphed units, meaning lurkers in the egg and guardians and devourers in the cocoon.

project settings

• Don’t try to link SparCraft, which is long gone. SparCraft was still included in a Visual Studio linker setting, and I didn’t notice because I had an old copy still lying around. Sorry about that. :-(

zerg strategy boss

• If our tech target is a lair tech, make sure we have at least 2 extractors. If it is a hive tech, make sure we have 3. This corrects some strategy freezes where Steamhammer would stay at a low tech level for a long time, not getting more gas because the low tech level didn’t need it, and not teching up because the higher tech level wanted more gas. It was a gap in the rules.

• Don’t try to research hydra upgrades and lurker aspect simultaneously. Oops. This caused a production freeze until the research finished.

• The previous Steamhammer 1.3 considers that a ground emergency lasts 5 seconds after the last enemy unit is seen off. Then it says, “whew, time to make drones.” But if it was a close defense, then it was left with few combat units and often lost to the next attack. I bumped the timer to up 15 seconds, enough time to rebuild combat squads before switching to drones. The change makes the biggest difference in games versus protoss.

• Fixed a minor bug in deciding whether to make an aux unit (an extra unit type added in small numbers to the regular unit mix).

• Also relaxed the condition for creating an aux unit, so Steamhammer is quicker to make them. The biggest effect is that if it is going hydra-ling and also has lurker tech, it is more likely to add the 1 aux lurker (or so) to its unit mix, increasing its fighting effectiveness.

• Other adjustments to the unit mix, to what-counters-what, and the like. The code needed a little refactoring to allow for the (removed) hookup to the opponent model, but changes to the logic are minor.

• Fixed a typo that caused Steamhammer to try to make ultralisks when it wanted guardians as a main unit. Oops. The effect was usually to get no hive units, since there was usually no ultra cavern.

• When Steamhammer has a tech target, it sometimes techs too fast. I tuned it down in certain cases. It used to often research lurkers and then immediately start a spire instead of using the gas to make its first lurkers, a harmful delay. Now it holds off on the spire and queen’s nest until it has “enough” combat units (according to an arbitrary low limit).

I am bitterly disappointed. I wrote my opponent modeling code. I kept it as simple as I could, since it’s the first draft of a major new feature. When it seemed to work I hooked it into the strategy boss, which then made decisions based on the enemy’s predicted future unit mix, not based on the current observed unit mix. I ran a few test games without visible problems. Then I tested more thoroughly, and turned up severe bugs. It is not close to solid enough to enter into a major tournament. And as far as I am concerned, frantic last minute debugging is not a plan.

I got what I deserved for hurrying to meet the deadline. I should have run careful tests earlier. I still think I had enough time to polish up the basic features, if I had only worked wisely. It’s not that complicated. It’s a simple idea that nobody happens to have implemented before (that I know of).

Anyway, for the AIIDE version I’ll turn off opponent modeling and remove the hookup into the strategy boss. I’ll release it as version 1.3.1, with a configuration file specially tuned for the tournament. It does have bug fixes and other improvements over the current version 1.3, so it should play a little better. (I count 19 bug fixes and small improvements plus 1 minor new feature, as well as new 2 hatch and 3 hatch lurker openings.) Then I’ll take my time with opponent modeling and release version 1.4 when it is good and ready.

The opponent model has strong capabilities in principle. It can meet most of the goals I laid out. If there is data—enough data from past games and enough from the current game—it can estimate the enemy unit mix at any future time during the game, and can predict the times of important events like “enemy gets air units” and “enemy gets detection.” An opponent that plays a fixed strategy should turn transparent by the second game. An opponent with a modest number of fixed strategies should become predictable as soon as Steamhammer has seen all the strategies—and when the current game has progressed far enough to distinguish them. I do expect trouble predicting opponents which successfully hide their tech, or which make important decisions in the middle game randomly, or which (like Steamhammer itself) play a wide range of strategies.

The downside is that it is only a data structure. Each use of the prediction capability has to be coded into the bot separately. I coded the prediction routine to estimate the enemy’s future unit mix. I coded 2 uses of the prediction in the strategy boss, one to set the tech target and one to set the unit mix (using different time horizons). When I want to use predictions to make spore colonies in time, or to get the right number of scourge at the right time to counter XIMP’s carriers as they show themselves, I’ll have to separately code more uses into the strategy boss, because of the scattered way the strategy boss works. When I want to use it for opening selection, I’ll have to code a different prediction routine and use it. And so on. The ability to predict is powerful, and you have to pay for the power.

Since I kept things simple for the first cut, predictions may be a little unsteady. It should be good enough to be useful, but whether it is or not, there is room for improvement.

I’m bitterly disappointed not to get opponent modeling into AIIDE. It’s an easy idea—to me it’s obvious, and I have to imagine that nobody else has done it only because they’re concentrating on other aspects. But it’s also new and I expect it to be successful. If I’m right, then other authors will soon be borrowing my opponent modeling code or writing their own along similar lines. The actively developed bots may take a step up in ability.

Devourers are not an everyday unit. They are specialist anti-air support units for when the enemy goes mass air. Devourers shoot slowly and do little damage for their cost, but their acid spores splash on enemy air units and make mutalisk or hydralisk fire much more effective.

Only today did Steamhammer play its first devourer game, the first game in which it both made devourers and had opportunity to put them to good use. XIMP by Tomas Vajda certainly goes mass air, and with devourers Steamhammer could have made short work of the carriers and corsairs.

But it didn’t. Steamhammer’s devourer play turned out to be terribly weak. They sometimes hung back ignoring the enemy and sometimes pushed forward on their own and took fire without reacting. In the picture, the 4 devourers are flying at an angle past the corsairs, ignoring them instead of loading them up with purple goop. Shortly after, they flew around and past the carriers, taking fire and still not engaging.

Devourers are not easy to use well, but I was surprised at how poorly they turned out. I didn’t put any special effort into guardian play, and Steamhammer uses guardians effectively as support units, with fewer micro and coordination issues than I expected. Mixing guardians into the late game army poses problems for the enemy, and the devourers did not.

At some point (if nobody beats me to it) I will teach FAP about the effects of acid spores, and then the real anti-air units in the army will know when to pour on the fire. But that’s not the underlying problem in this game. Devourers moved around foolishly and did not shoot as often as they should have.

One of the themes of today’s SSCAIT broadcast by Nepeta is that too much static defense is bad for you. He repeatedly showed in ZvZ that the side which made too many sunkens tended to fall behind. Static defense can’t go attack; it costs resources and it offers initiative to the opponent. In a well-played game, static defense has to pay for itself with a countervailing advantage: You have to use the temporary safety it brings to get ahead in economy, as in a protoss forge-expand opening or as Killerbot by Marian Devecka tries to do, or to get ahead in tech, as when in ZvZ you make a sunken to tide yourself over until your spire finishes.

But there is a reason that so many bots play turtle strategies. Turtling is strong against bots which do not adapt, which is most of them. For Steamhammer, I added 1 base and 2 base turtle openings which crush Wuli, an opponent that Steamhammer otherwise struggles against. Wuli does not adapt.

Steamhammer does adapt to static defense and pulls ahead of opponents that turtle too much. When the opponent makes static defense, you have 3 broad strategic choices. They are the same 3 choices you always have, but the opponent has offered you initiative so any of them might give you an advantage. 1. You can pull ahead in army and bust the defenses. Example: Hydralisk bust versus protoss forge expand. It can win outright, but it is risky. 2. Use the opponent’s passivity to pull ahead in tech. If you made a sunken, other things being equal I can safely skip a few lings and get a faster spire. 3. Or similarly, pull ahead in economy. If you made sunkens, I can make drones.

Steamhammer primarily adapts by making drones. A protoss or terran bot might expand sooner instead, so it can build workers faster. But whatever the choice, you want to adapt so that you don’t end up crushed like Wuli.

The game Steamhammer vs KillAll (correcting the unfortunate typo in the bot’s name) is an example. KillAll opened 9 pool with extractor trick to get 10 drones, and Steamhammer unluckily chose 5 pool. It is a build order win for KillAll—if both sides play well, the 9 pool wins with little risk.

But instead of making a second hatchery to win with mass lings, or getting quick gas to win with mutalisks, KillAll turtled. It threw away its advantage.

You may want 1 sunken, because the 5 pooler starts making lings sooner and can be a little ahead. 5 sunkens are 4 too many and put KillAll behind despite its stronger opening. They are not well placed; they don’t protect either the approaches or the mineral line.

If I had been playing, I would have made a moderate number of drones, teched fast, expanded once for a second gas to guarantee that I would stay ahead, and aimed to win with mutalisks. Steamhammer made many drones and teched slowly (fast enough not to fall behind), spending the extra income on zerglings. It went up to 5 hatcheries to produce a biblical flood of zerglings. The picture shows 3 of the 5 hatcheries; the other 2 are in the natural. Steamhammer is slightly ahead in economy and tech and way ahead in army and production capacity. Steamhammer can only lose if it blunders.

The zerglings were so many that they cracked the turtle shell and overran KillAll. GG. KillAll lost drones right off because the sunkens were misplaced, but it took only a fraction of the zerglings to finish off the sunkens. Steamhammer had a completed spire but did not need any mutalisks (see how many purple zerglings are already en route).

Steamhammer’s reaction to static defense is effective. You can beat it with a turtle strategy, but only by exploiting weaknesses. The reaction to static defense is not a weakness.

While reviewing WorkerManager to find a different bug, I happened to notice a subtle mistake in WorkerManager::getBuilder(), the routine that chooses a worker to construct a building. If there is a worker whose job is to move into position, it prefers that worker. But if there is not, it picks a mineral worker:

		if (unit->isCompleted() && (workerData.getWorkerJob(unit) == WorkerData::Minerals))

It looks correct. If you have workers then should always have mineral workers, unless they are all pulled for other jobs, and then you probably prefer to wait to start the building. But it’s not correct. There is an important case where there are no mineral workers: You may have no bases mining minerals because they were destroyed or mined out, so all the former mineral workers are marked Idle.

		if (unit->isCompleted() &&
			(workerData.getWorkerJob(unit) == WorkerData::Minerals || workerData.getWorkerJob(unit) == WorkerData::Idle))

Without this fix, when Steamhammer loses its only hatchery, it is unable to rebuild. I ran into that bug in January and always wondered about the cause. Theoretically the bug could also strike in the late game, after all surviving hatcheries have mined out their minerals. I’ve never seen it happen, but then there would be no mineral workers and Steamhammer would be unable to construct a building.

It’s not a critical bug. Losing your last hatchery usually means you lose the game. But if the opponent is also in dire straits, you still have a chance. It happens most often in zerg versus zerg.

Now I need to go find any other places where code looks for a mineral worker and make sure it can find an idle worker too. WorkerManager::isFree() was already correct, but there might be more....

Update: The other routines which get a mineral worker and skip over idle workers are WorkerManager::getClosestMineralWorker(), WorkerManager::getGasWorker(), and WorkerManager::getMoveWorker(). Routines outside WorkerManager which do similar jobs all call isFree() as they should. I rewrote these three to call isFree() too.

Here’s how I rewrote WorkerManager::handleGasWorkers() to solve the drone dance bug whose cause Arrak located. It’s a deadly bug and not fixed in any released Steamhammer version, so if you’re running a fork you probably want this.

First, a pseudocode version to show the structure. The “resource depot” is the command center, nexus, or hatchery. If the resource depot is lost, we need to stop mining from the refinery because the enemy army is likely still around and will kill all our workers 3 at a time as we send them over. (Code elsewhere handles the case where the refinery is destroyed.)

if collecting gas
  for each refinery
    if its resource depot still exists
      add workers to the refinery
    else
      remove workers from the refinery
else
  remove workers from all refineries

Then the actual code. The pseudocode is good, but bugs may lurk in the details. It is new code and poorly tested, especially removing workers from a refinery whose depot is lost. Let me know if you find any remaining bugs.

// Move gas workers on or off gas as necessary.
// NOTE A worker inside a refinery does not accept orders.
void WorkerManager::handleGasWorkers() 
{
	if (_collectGas)
	{
		// Gather gas where possible. Check each refinery.
		for (const auto refinery : BWAPI::Broodwar->self()->getUnits())
		{
			if (refinery->getType().isRefinery() && refinery->isCompleted())
			{
				if (refineryHasDepot(refinery))
				{
					// This is a good refinery. Gather from it.
					// If too few workers are assigned, add more.
					int numAssigned = workerData.getNumAssignedWorkers(refinery);
					for (int i = 0; i < (Config::Macro::WorkersPerRefinery - numAssigned); ++i)
					{
						BWAPI::Unit gasWorker = getGasWorker(refinery);
						if (gasWorker)
						{
							workerData.setWorkerJob(gasWorker, WorkerData::Gas, refinery);
						}
						else
						{
							return;    // won't find any more, either for this refinery or others
						}
					}
				}
				else
				{
					// The refinery has no depot to return gas to. Remove any gas workers.
					std::set gasWorkers;
					workerData.getGasWorkers(gasWorkers);
					for (const auto gasWorker : gasWorkers)
					{
						if (refinery == workerData.getWorkerResource(gasWorker) &&
							gasWorker->getOrder() != BWAPI::Orders::HarvestGas)  // not inside the refinery
						{
							workerData.setWorkerJob(gasWorker, WorkerData::Idle, nullptr);
						}
					}
				}
			}
		}
	}
	else
	{
		// Don't gather gas: If workers are assigned to gas anywhere, take them off.
		std::set gasWorkers;
		workerData.getGasWorkers(gasWorkers);
		for (const auto gasWorker : gasWorkers)
		{
			if (gasWorker->getOrder() != BWAPI::Orders::HarvestGas)    // not inside the refinery
			{
				workerData.setWorkerJob(gasWorker, WorkerData::Idle, nullptr);
				// An idle worker carrying gas will become a ReturnCargo worker,
				// so gas will not be lost needlessly.
			}
		}
	}
}

Here’s a little story for anyone who doubts the importance of fixing bugs: Randomhammer’s last 4 games were all decided by bugs.

4. Versus IceBot, Randomhammer rolled terran and went vultures. The vulture attack almost broke through IceBot’s wall, but IceBot got its bunker up and managed to keep 1 marine alive long enough to get in. Randomhammer’s hard-coded switch to tanks was late and its tactics were weak, and IceBot was soon winning. The hostile tank push started, then—IceBot crashed, game to Randomhammer.

3. Oleg Ostroumov crashed at 5 SCVs, before the game was properly underway.

2. Hardcoded crashed on start.

1. Versus zerg ZurZurZur there was at least a game. Randomhammer again rolled terran and this time went for a vulture drop. ZurZurZur sent 4 hydralisks across for a poke, where they met Randomhammer’s small force of marines and vultures, whose purpose is to keep the enemy occupied out front while the drop happens in back. It worked; ZurZurZur’s forces were far out of position, and the vultures killed all but 1 drone before being cleaned up. Then Randomhammer fell into a production freeze and did not produce another unit for the rest of the game, allowing ZurZurZur to slowly recover with its 1 drone and win.

There is a sub-theme here: Crash bugs are the worst. You crash, you lose. Randomhammer could have defeated ZurZurZur on points if it had better vulture micro, even with the production freeze. (It couldn’t have won outright, though, because there were sunkens that existing forces could not have brought down.)

Steamhammer has no known crashing bugs. I went on campaign and exterminated them. Its last crashes due to fixable bugs were in April, and its last recorded crashes on SSCAIT were in May and appear to have been due to bugs in the infrastructure, not in Steamhammer. Some of the bugs I fixed are obscure and their triggers have never occurred in real games. For example, Steamhammer has never had a unit mind controlled, as far as I have seen. The normal unit validity check done on every frame should catch mind controlled squad units and workers before Steamhammer tries to send them orders—though it hasn’t been tested, so I don’t promise it’s correct. But I noticed a loophole: The scouting worker is not in a squad and is not treated like a regular worker. It is controlled directly by ScoutManager, which does not run the validity check. I left a comment in the code where I fixed it. How many years would it have been, do you suppose, before somebody mind controlled the scouting worker and brought about a crash?

Opponent modeling is coming along, though never as fast as I would like. Steamhammer now records a pretty informative model. Making best use of the model is not as easy—it has a ton of uses. If it works as well as I hope, Steamhammer will gain scary predictive abilities, even against opponents with multiple strategies.

Opponent modeling depends on good scouting. The more Steamhammer finds out about what the opponent does, the better the model. So today I added a new scouting command, "go scout once around" which sends the worker scout on a single circuit of the enemy base and then returns it home. (Usually. There are some funny cases because the waypoint numbering is not quite clean.) In a lot of openings, I used it to replace "go scout location" which only finds the enemy base and doesn’t look to see what’s there. I’m thinking of also adding "go scout while safe".

The command is a minor addition, but while hooking up the wiring I saw awkwardness in the communication among ProductionManager which executes the commands, GameCommander which initiates scouting, and ScoutManager which does the work. I ended up spending the whole afternoon refactoring it for simplicity and testing to make sure I hadn’t broken anything.

Is that what I should be spending my time on? And yet it makes Steamhammer better.

Production freezes are one of the most serious classes of bugs in UAlbertaBot. Its descendants like Steamhammer and Arrakhammer have mitigated the problem by solving various subclasses of production freeze item by item, but have never solved all of them.

There are two main kinds of production freeze, permanent production freezes caused by deadlocks and temporary production freezes caused by waiting for a slow prerequisite to finish.

There are many ways for production to deadlock. You want hydra speed, so you order it up. Then the hydra den is destroyed and the queue hangs. Steamhammer solves that for zerg by canceling items whose prerequisites are missing, which is itself complicated and bug-prone. I think most remaining permanent deadlocks are caused by bugs in the strategy boss, the information manager, the production manager, or the building manager, or in their interactions with each other. Steamhammer tries to mitigate these by timing out and clearing the queue if nothing has been produced for too long, but that is not enough to save the game against a strong opponent.

A temporary production freeze is caused by waiting unnecessarily for a prerequisite to finish. For example, an old version of Steamhammer once froze frequently when it wanted a hive next, because it didn’t realize that it was still researching overlord speed. It had to wait until research finished before it could morph the hive, and in the meantime it was making no units and probably losing the game. The live version solves most issues like that, but it does have an unsolved production freeze that occurs while waiting for the spire (it sounds easy to solve, but I’m not finding it). Another example is: Suppose you want vulture speed and spider mines, and you made 2 machine shops to do the work, but 1 shop has been destroyed. No prerequisites are missing, so there is no deadlock, but you’ll end up researching the upgrades one after another instead of simultaneously, in the meantime making no units. Or even: You want something that requires gas next, so you wait for the gas to accumulate and produce nothing else even though you could make mineral units while waiting. I may add queue reordering to mitigate this, but the underlying issue remains.

Production freezes happen in other bots too. Watch the production tab in OpenBW. I’ve seen tscmoo suffer badly from temporary production freezes.

Software engineering is the solution. So far I’ve been fixing issues one by one. The underlying problem is that the software architecture is fragile. The right kind of fix is to redesign the production system in a way that is not prone to freezes. I don’t know what the redesign will look like, but some of its features are already visible from a distance. For example, I expect it will take into account low-level details like “which building will do this research?” before ordering the research; currently, the details are left to ProductionManager after the order is made, leaving room for slippage.

Anyway, my plans are always changing. This month I am on opponent modeling. After that, I plan to work on the mutalisk control that I’ve always promised. A new macro manager and re-architected production would be a logical task after that, especially since it would improve play for all races. At the rate I’ve been going, I may or may not get to it this year.

Since uploading version 1.3, I’ve been bemused to watch Steamhammer’s elo fall to near 2100, while Randomhammer’s rose to over 2150. I’m sure that they haven’t really switched places, since Randomhammer has been winning as zerg. The rating swap is some combination of luck and the effects of the voting system. The reversal has already partly reversed itself.

I have gotten a clearer view of new weaknesses, though. One weakness is caused by making emergencies too short, so that after a difficult defense Steamhammer makes drones and tech instead of replenishing its combat squads, and may lose to the next attack. (Someday I’ll solve that in a more principled way.)

I have to put the big effort into opponent modeling, though. I boasted big goals and I have to meet at least some of them.

Next up for Steamhammer is opponent modeling. Some version of it will be ready for AIIDE, deadline 1 September. Opening learning methods that we have seen so far implicitly assume that the bot knows a small number of strategies and that one of them is the best counter for the opponent’s play. I want Steamhammer to be able to cope with opponents that are reactive (Iron), multi-strategy (Zia), or both (Krasi0). My goals for opponent modeling are something like this:

• Play a wide range of strategies. It has been part of my plan from the first.
• Learn from the events of the game, not only from the outcome. That way the bot can learn more and faster.
• Learn from one trial: See an opponent’s strategy in the first game and counter it in the second, or at least make a good try. Fixed-strategy bots that Steamhammer knows a counter for should stand at a disadvantage from the second game on.
• React by both choosing a counter opening and later choosing a counter unit mix in the middle game.
• As more games are completed, recognize the range of the opponent’s play.
• If no one opening strategy counters the full range of the opponent’s play, use game theory to estimate the best mix of openings.

It’s fancier than the strategy learning we’ve seen before, but it doesn’t seem hard to me. It’s straightforward, at least in principle. The key element is a model of game strategy. Already last night I wrote a GameRecord class that keeps a simplified description of what happens. That will be the basis for reading the opponent’s strategy over the game. As soon as we have one game record for an opponent, we can check our openings against it to predict which openings will succeed, and we can also use the record to make middle game decisions about what to produce.

Against opponents with a fixed play style, like XIMP, I expect this to be fast-acting poison. Steamhammer won’t react “Oh, carriers, I’d better make some scourge,” it will prepare ahead of time, “4 carriers will be coming in a minute or so, let me make the right amount of scourge to explode them all.” Against opponents that vary their play, it will take more games to formulate effective venom.

If I have time, I’ll do more. There is no chance that I can get all of these ideas implemented in time for AIIDE, but I’ll make what progress I can.

• Generalize across opponents, so that an unknown opponent faces play that has proved strong before. If you play a lot of openings, then some of them are weak in most circumstances; so far I have accepted that.
• Take more information into account, including map and starting positions.
• Integrate scouting information with the opponent model to get the best possible prediction of what the opponent is aiming for this game.
• Arrange the known opening lines into a tree, and use the integrated prediction to make decisions at every branch. Opening play will become reactive moment by moment, more like pro play.
• Use the same mechanism to decide when to break out of the opening book.
• Record the decisions made just after leaving the opening book as a new opening line to be added to the book and possibly played against other opponents. With breaking out plus adding opening lines, Steamhammer gains the ability to invent its own openings.

Steamhammer 1.3 is uploaded. This is the release version. I’ll try to put up the source code and documentation today, so that I can hurry on to opponent modeling.

I’m convinced that Steamhammer 1.3 is stronger overall than the last release, 1.2.3. Some new bugs and weaknesses were added, as always, but more than enough old ones were taken away. Most of the new weaknesses fall into two groups: One, new macro problems revealed by fixing the last round of macro problems. Two, mistuning and general klutziness in the strategy boss because it has more tech choices. Adding lurkers, guardians, and devourers meant that I had to replace the subsystem that chooses the unit mix and tech target, and there are rough edges.

The change list since the last test version:

• Drop SparCraft and use FAP for combat simulation. SparCraft was occasionally breaking the time limit per frame in long games, so it had to go. I can’t find any practical disadvantage to the change. FAP is faster, smaller, simpler, more comprehensive, and less buggy. The CombatSimulation class responsible for calling it became tiny. Its results seem a little more accurate even in simple cases like unupgraded zerglings versus unupgraded zealots, and of course they are much more accurate for units that SparCraft does not support, like bunkers. SparCraft has the advantage of a basic understanding of terrain, so theoretically it might do a better job in assaulting ramps. But both combat simulators ignore vision issues and allow simulated units to pass through each other, so terrain effects are not well simulated anyway—the biggest difficulties in assaulting up a ramp are that you can’t see the top, and that not many units fit on the ramp.

• Fix some potential production freezes related to subtle interactions between the strategy boss and the building manager.

• Fix a potential production freeze related to ordering an impossible extractor.

• A worker sent early to construct a building is no longer prematurely swept up as idle. In the final test version, this was sometimes causing buildings to start a little later than they should have. It’s a recurring bug, and luckily this iteration was not as severe as the last.

• Turrets are a higher targeting priority for melee units.

• In ZvZ, don’t ever make hydralisks, and only make lurkers after hive. Steamhammer was always intended to play this way, and now the rule is strictly enforced.

• In ZvZ, deliberately undercompensate for enemy static defenses and expansions.

• If we’re making hydras now and planning lurkers later, maybe get hydra upgrades if there’s time. Steamhammer prioritizes researching lurker aspect, and the priority was too strict.

• If all lair tech looks useless, go straight to hive. Earlier test versions brought up a lair tech, either mutas or lurkers, even if they judged that all lair tech was unhelpful. Sometimes it’s better to stay on hydra-ling until you can get hive tech.

• More adjustments to what-counters-what. In general, more ultralisks and fewer lurkers.

• Several new zerg openings added with low probability. There are now 3 different 1 hatch lurker builds, even though there is still no 2 hatch or 3 hatch lurker build (all things in due time).

Other changes since the last release, by test version number:

• 1.2.6
• 1.2.5
• 1.2.4 (the long list)

Steamhammer has a particularly weird bug on the map Jade. Jade is a 4-player map with bases in the corners. Each base is on low ground, surrounded by a ring of high ground. A ramp leads up to the natural at one end of the ring of high ground, and at the other end of the ring is a third base.

If Steamhammer starts in the lower left base, then it is unable to take its third on the high ground. A drone on its way to start the third will travel most of the way there, then turn around and go home. BuildingManager then sends another drone, which behaves the same, and so on forever. Starting at a different location is OK. Taking the third of another start location is OK. Only that specific start location has trouble with taking that specific base.

The bug has affected Steamhammer since early days. I’ve never looked into the cause because it only comes up 1/4 of the time on 1/14 of the maps. It’s a game-losing bug, but low on the list. Presumably some detail of the map, or maybe some error in BWTA, is triggering a mistaken decision.

It’s such a surprising bug that I thought it deserved its own post. Maybe somebody has an idea about the cause? I could be mistaken, but I vaguely recall that I’ve seen other bots with the same error.

In the meantime, Steamhammer lost against Roman Danielis today by starting off strongly. Roman Danielis repeatedly tried to take its natural, and Steamhammer repeatedly stopped the nexus from warping in. The strategy boss mistakenly went into a panic, “Oh no, the enemy just took a ton of bases, I’d better make a ton of drones to catch up!” Steamhammer made all drones and no combat units and lost. Now that’s a bug I’ll dig into.

Igor Dimitrijevic has uploaded a new version of Stone, a version which attacks buildings instead of workers. I guess it’s because worker defense has been steadily improving, but many bots still don’t react when a building comes under attack.

Steamhammer doesn’t react to attacks on buildings. Even so, its usual anti-rush opening defended successfully in this game. Buildings have a lot of hit points, so the spawning pool finished before it was destroyed. In the picture, the spawning pool is about to die before any zerglings are spawned, but the sunken colony is already morphing.

The sunken protected the hatchery completely, so there was little risk. Steamhammer replaced the spawning pool in the same position, which was not safe because the sunken protection extended to only one side of the pool. Stone did not seem to understand the range of the sunken, and half the SCVs sent to attack the pool were killed while standing just inside the danger zone. Steamhammer actually won the game more easily than against the older Stone version, because Stone threw away so many SCVs. Stone did point out weaknesses. Steamhammer should defend buildings which are in danger of being lost, and it needs better building placement.

This game versus Skynet by Andrew Smith was fun. Skynet, ahead of its time as usual, does react when buildings are attacked, and pulled many probes to defend. Skynet was unworried by the skirmishes and calmly continued with its build: Late gateway, gas, cyber core... all before the first combat unit.

My impression is that the change to Stone has left it weaker overall. Buildings are too hard to kill. Still, it does easily wipe out some opponents, and every change is a new challenge.