Starcraft AI blog

After defilers, I did smaller micro and tactics fixes, corrected a few openings, and made other quick improvements. Then I looked over my to-do list and marked the items that would benefit from having on hand the results of a tactical analysis—wow, that’s quite a few, tactics are next! So far I have implemented clustering as mentioned here. I think that within a few days I should be getting practical results like decisions on which direction to retreat in, and then I’ll start seeing where my tactics plan needs revisions.

spider mines

In the meantime, I am also thinking about spider mines. In spider mine placement I wrote about the complicated considerations behind laying mines. Coping with the enemy’s mines is also complicated. These 5 counters to spider mines are all common in pro games.

1. Sweep the mines. The most obvious counter is to bring a detector and ranged units to safely clear the mines.

2. Force the mines. With good micro and the right choice of units, you can try to get by without a detector: You can shoot down spider mines after they pop up. It’s the most common way for terrans to cross a minefield. Dragoons and hydralisks can do it too, with more care, and so can dark templar, or zealots with high enough attack upgrade (+2 for 20 damage, enough to kill a mine in one hit), or in fact any combination of units which are quick and coordinated enough to do the damage in time. One risk that too many mines will pop up at the same time, and you’ll have to dodge them to reduce the damage. Another risk is that the enemy may appear, possibly forcing you back since you won’t be able to cope with mines and enemy units at once.

3. Dodge the mines. Find out which unit of yours the mine is targeting (the mine’s getOrderTarget() will tell you, as Purple Dan pointed out). Simply move the target unit away from the mine—and ideally away from any other units of yours which might be in splash damage range. There is a delay between when the mine decides to stop and explode and when the explosion occurs, and if you are moving away during that delay then you can suffer only splash damage instead of the full effect. (The same idea works for scarabs.) It makes a big difference.

4. Sacrifice to clear the mines. If you want to attack through a minefield, or if you are otherwise in a big hurry to destroy the mines, you won’t be able to do any of the above counters. Send a few zealots or zerglings through the minefield on move command, to trigger as many mines as possible. Being on move command is important; a unit that fires on a mine will stop moving and won’t trigger as many mines. The mine clearing units will be lost, but attackers can follow behind. I’ve occasionally seen terrans sacrifice marines to clear mines, but it is not as common or as effective. This technique is probably worth using during an attack when there is any suspicion of a minefield.

5. Drag the mines. If you are sacrificing a unit on move command to clear mines, what is the best place for your unit to end up? Right next to the enemy, making the enemy’s weapon backfire. A great mine drag can blow up a mass of tanks in moments. Pros know to place spider mines away from where their units intend to go, and to destroy their own mines when their units go there anyway. Bots tend to be careless about mine placement, and are acutely vulnerable to mine drags. Zealots from a shuttle can drag mines too; I think that variation is most valuable when you already know where the mines are.

Less common counters to spider mines exist. You can clear mines without damage by dropping units from a transport and picking them up again with exact timing. Psionic storm will clear mines from an area, but is generally not worth it. Disruption web stops mines from triggering, a use that I have never seen.

One other idea comes to mind, not a common technique as far as I have seen: Play tricks with the mines. You can deliberately leave mines in place and use them to show the enemy what you want the enemy to see. For example, an observer can spot mines without being seen in turn. Notice some mines and leave them unmolested and seemingly undiscovered. Then, when you need a diversion, send an empty shuttle over the mines and it will look convincingly like a drop (a fun use for hallucinate). Someday bots will be smart enough to be fooled!

Knowing that there are spider mines on the map affects your tactical planning from the beginning. Even units like workers that don’t trigger mines will be seen by them, giving away information. Some mines you may spot ahead of time; some you will not. If you’re smart, maybe you can guess where mines are likely. No matter what, you know that moving around the map will rely on the 5 counters.

I would like to get all 5 into Steamhammer for AIIDE, but I’m not sure how much progress I’ll make. I guess mine dodging will be first, since it is so similar to scarab dodging that it can probably be done at the same time.

This picture of Steamhammer versus XIMP by Tomas Vajda is a perfect illustration of why Steamhammer needs a smarter tactical analyzer.

The hydras decided that they had to retreat from the carriers at the top. Where to retreat to? “Hey, look, those scourge have found a great location! It’s a safe spot, it’s close to the enemy base—let’s go there!” It follows a heuristic of trying to keep its units together. It hardly needs saying that the entire hydra force was lost trying to run past the cannons. In big army games, Steamhammer makes similar blunders nearly every game.

The poor scourge play is another issue....

The picture also illustrates the silver lining to Steamhammer’s excessive retreating weakness in this version. Because Steamhammer now plays weakly when it is locally behind, it lost games playing the merely strong 3 hatch hydra opening that it had settled on in the previous version. It was forced to find an even stronger counter to XIMP, 3 hatch before pool into hydras. Look at the worker counts—even after losing this army, zerg is still ahead because its massive economy can replace the losses while protoss will remain unable to expand. Thanks more to XIMP’s objectively bad strategy than to Steamhammer’s play, and despite the severe blunder, Steamhammer put the game away easily.

Steamhammer is coming along steadily for AIIDE. It is already noticeably stronger than the release version now playing on SSCAIT.

Defiler skills may never be as good as I want; defilers are difficult. But the skills are there, and zerg can lay down the law in the late game. I can’t always predict whether Steamhammer will swarm or plague in a given situation. It’s satisfying to see a carrier plagued, and there is a bonus in the scoring function to favor exploiting plague-on-interceptor. XIMP has no hope.

Next I made some micro changes. After playing games against several opponents at very slow speed to watch the fast decisions, I’m convinced that Steamhammer is making fewer wasted movements and fighting more efficiently. I want to make further improvements, but it’s enough for the moment. I’ll see how the time works out.

I’m thinking through a more detailed plan for the tactical analyzer. While that is crystallizing, today I am working on the retreat problem which has left the current Steamhammer weaker than the previous version. Once retreat is ironed out, I should have the strongest Steamhammer yet and a good foundation for the next changes.

Meanwhile, I notice that PurpleWave has learned how to delay the zerg natural in the early game with its scouting probe by denying the hatchery from starting. It’s the classical reason usually cited for why zergs switched from 12 hatchery to overpool against protoss. I deliberately chose 12 hatchery for many of Steamhammer’s openings because no protoss bots had the hatchery denial skill, and now that has changed. Besides opening pool first, other standard reactions are to make the second hatchery elsewhere (at what would otherwise be the third base) or to fight a little micro war of drone versus probe—or both at once. Hmm, so many choices....

At any given time, a unit has an order: See BWAPI::Orders Namespace Reference. That page is the only documentation I know of, and it doesn’t explain what any of the orders means.

There are a lot of possible orders. A great many are self-explanatory—Die, I think I know what that one is for. Some are familiar because they are common or because I have dealt with them before, like PlayerGuard and ResetCollision. Some I can guess are probably used by neutral units, like OpenDoor. And some don’t appear to make sense at all. What is CarrierIgnore2? What is JunkYardDog? I have no idea.

It would be nice to have a short explanation of each order. Anybody have a list, or a start on one?

Steamhammer has a configuration option to draw each unit’s order in cyan underneath the unit on the screen. It’s handy for seeing what the units are doing, and it makes it easy to figure out what the common orders are. It is not so helpful for rare orders, or orders which are active for such a short time that you don’t notice the order name flashing by.

Update: Commenter Yeg pointed out what I was looking for: Orders.dat Entry Listing.

In SSCAIT news, Locutus is back on top at #1. The commit log shows a large number of relatively minor changes that were apparently enough. For a day or so, Locutus won every single game, including a good number voted in against Krasi0. As usual, Krasi0 was updated quickly and is winning games against Locutus again, but Locutus is still #1 in elo. (Being #1 for a long time is an advantage in itself for staying #1, because generally only one opponent at a time is able to surpass you, and then it’s relatively easy to patch the right weaknesses.)

In pro news, the KSL is underway; see the round of 16 preview on Team Liquid.

The ASL6 maps have been announced (in Korean; see the Team Liquid thread for ongoing discussion). Contrary to predictions, Transistor was carried over (in a new version) rather than Third World.

Just like last season, the map pool is remarkable: There are no 4 player maps! Of the 4 maps, 1 is 2 player, 2 are 3 player, and 1 is 5 player. All the maps have unusual features. The strangest may be the 5 player map Labyrinth, which has 2 geysers in each main base and 2 entrances to the base, each special: One is blocked by a gate that can only be passed by workers using mineral walk (and it’s a ramp, so an enemy worker without vision of your minerals cannot pass), the other by a flag that apparently must be picked up by a worker before other units can enter or leave. Early game scouting will be tricky!

Update: Apparently this is not the final ASL6 map pool? I’m not sure how it works, but it seems that these are candidate new maps. I guess some new maps will be chosen and some old maps will be carried over. If that is true, then we will probably see Third World after all.

Tscmoo zerg uses defilers and casts dark swarm, one of few bots with the skill. And... it’s clumsy at it. The swarms usually end up in unhelpful places, or overlap wastefully. Well, Steamhammer can’t place swarms either. The skill is extremely difficult for bots, for 2 related reasons.

One reason is that dark swarm is a coordination skill, and bots suck at coordinating their armies. For an attack, the defiler has to be charged up and in position as the attackers move forward, and place its swarms not only in the right places, but also at the right times, to neutralize as much enemy fire as possible. To do it well, you have to plan or predict how the fight will go, and set up the pieces in good locations ahead of time. One way would be with an abstract combat simulator or planner that could accurately foresee the general movements without working out the details—quite different from today’s concrete combat simulators that inaccurately foresee the details of the fight. Another way would be a large learning system that would be time-consuming to train. I don’t think any current bot has either.

The other reason is that dark swarm is a tactical analysis skill. Suppose you spot a group of marines in the middle of the map. You can’t simply drop a swarm on them and mop them up; if the enemy is any good, the marines will step aside. As long as you have zerglings around, the marines had better stay out of the swarm, but you can only swarm a small portion of the map. Or suppose carriers find your hydras, and you don’t have enough hydras to win, so you lay a swarm over them. Good, the hydralisks are safe, but the carriers can fly away and attack somewhere else. Dark swarm doesn’t protect buildings.

To do it right, the bot has to analyze the tactical situation. If you’re defending a narrow space from marines, great: Swarm the choke, put some lurkers under the ground, and you’re cool until swarm wears off or everything gets irradiated. If there’s more room, then you need to analyze possible maneuvers. If I swarm here, the marines can back up or go around. If they back up, are they farther away from where they want to go, or closer? If they go around, can I trap them? Will the defiler be safe, or is there a risk that this will be my last swarm until I can get another defiler over here? Strong play is super complex.

It just so happens—I’m sure it’s a coincidence—that I am planning to write a tactics analyzer next. Maybe that will help.

Steamhammer 1.4.7 source and binary downloads are available from Steamhammer's web page. This is the one with the crashing bug fix—don’t use 1.4.6. Sorry about the delay!

I am still narrowing down my AIIDE plans. I don’t know how long steps will take, so it makes no sense to plan in detail, but I want to lay down enough priorities that I don’t leave important stuff until too late.

I have optimistically given the next Steamhammer the provisional version number 2.0, on the theory that I’ll make that much progress. I may roll it back to 1.5, we’ll see!

So far, I have implemented defilers—not the most urgent skill, but an essential skill that will help. As it stands, they are modestly effective, enough that I tweaked the queen’s nest timing to get the hive a little sooner. The code to plague clumped enemies and avoid our own units worked on the first try, and was very satisfying. Once I add a few micro skills and teach FAP about dark swarm, defilers should revolutionize Steamhammer’s late game.

I will also choose at least one new skill to put on pressure in the early or middle game. Most bots are not prepared for runbys, that would be a logical choice for the early game. For the middle game, no bot does large drops, or proper lurker contains (though OpprimoBot does try to contain with 4 lurkers). I have other ideas.

I also need to do a bunch of ordinary micro improvements and weakness fixes. It’s a large amount of work in total, but it breaks into pieces so it’s easier to fit in.

Do it right, don’t do it fast is still the principle that I instinctively want to follow, especially after I’ve deliberately deviated from it and come face to face with crashes and poor play. Going with defilers shows that attitude. Meanwhile, the overall development plan says that now is the time for more tactical and micro skills, because strategy skills are stalled waiting for them.

What I most want to do is offline machine learning for strategy, operations, tactics, and micro. (The opponent model uses online learning.) I have selected my method and I’ve been reading up on it, and I’ve written about half the code I need to start playing with it. It is not a priority for AIIDE. The first offline learning feature will be the easiest and the most foundational, an evaluation function “am I winning?” It will be fun for stream watchers, and it will be good for opening selection and unit mix decisions—strategy items that are not chief next steps. Also, offline learning will take many days of computer time while Steamhammer crunches games and tries to draw conclusions from them, and I might have to spend that large amount of time repeatedly to get industrial-strength results. I’ll keep working on it because it’s what I want to do, but I shouldn’t let it distract me too much.

A tactics sequencer would be good for clearing map blocks, breaking through backdoor entrances, complex drops, taking islands, planning how to expand against opposition (instead of sending drones out to die), and other stuff that takes multiple steps. I may do that, it’s tempting.

More urgent is probably a tactics analyzer that is better at figuring out how to organize forces, how to maneuver, where to attack, and especially when to retreat and where to retreat to. Steamhammer loses a lot of games to mistakes on those points. Unless I think of a cheap fix to the major problems, that will be my choice for AIIDE. It’s a big enough feature to justify a 2.0 version number.

This Steamhammer version is not performing as well as I had hoped. It seems that the new weaknesses more or less offset the new strengths. Also, every time Steamhammer’s play changes, its past learned data becomes less effective. This change was big enough that some counters which used to work against a given opponent no longer do.

• The new retreat behavior in defense looks like a net loss. As I mentioned in the change list, it prevents the severe mistake of making a last stand outside of static defense range, but causes the severe mistake of fleeing from a stronger enemy when it should engage. I think the mistake caused is probably worse than the mistake prevented. For example, Steamhammer lost its ability to reliably defeat XIMP by Tomas Vajda, and excess retreating by defenders is the main cause.

The upside is that it’s relatively easy to reduce excess retreating without causing premature last stands. It was always on my list, I didn’t have time.

• The other defense changes are good, but don’t help greatly. Steamhammer is a little more effective at assigning defenders, and I may be the only person who can tell.

• Predicting enemy unit movement seems to be a win overall. Enemy scouting workers are caught, and units trying to slip past screening zerglings rarely get through. Zergling versus zergling micro is sometimes dramatically stronger—attack with fewer and no advantage in formation, win anyway. The disadvantage is that units are more likely to get stuck, either stuck in place on their own or stuck on each other. I have to dig in and solve that.

• The improvements in overlord safety help against many opponents, but do not help enough to make a big difference. Probably Steamhammer can defeat weaker enemies a little more consistently. Against protosses that make corsairs but don’t have strong overlord hunting skills, like Skynet, the improvements can save games. Against those with strong skills, like BananaBrain, Steamhammer holds on well at first and survives longer into the game, but the slaughter begins as soon as observers or dark templar come out and the overlords leave the safety of their spore colonies. Alongside better overlord movement, Steamhammer needs better air defense skills with hydras and scourge.

• I suspect that the heuristic “the enemy is ahead in army, make more combat units” rule is not tuned well. I think it helps on average, but not as much as it could. And there are times when it is wrong.

• The “oops, expanding failed, I’ll turn it into a macro hatchery” rule causes Steamhammer to make many extra macro hatcheries when it is contained. It may be an improvement over losing an infinite stream of drones, but it’s still awful. There is an obvious quick-and-dirty fix of canceling excess macro hatcheries in the building manager before they are started—but I think that will cause Steamhammer to go back to losing an infinite stream of drones, while doing unnecessary work in between. Have to bite the bullet on this one and perform serious surgery.

• The better sunken and spore placement is all win, despite occasional odd results. “Ooh, sunkens right up against my ramp! Units can’t get out? So what, it’s safe!” Now I need to add the production goal to put the right static defense at every base that is vulnerable to vulture or air harass.

Next: AIIDE plans in a little more detail.

Whew, crashes are fixed. Stand by for the source release. For CIG, the question is whether Steamhammer will do poorly, or be disqualified outright.

I plan the next release well in the future, after the AIIDE deadline on 1 October. That is enough time to make major improvements. I haven’t decided what improvements to make, but I have a shortlist. There are the usual suspects, overlord skills, worker skills, decent micro, mineral locking and other mining tricks. I’ve thought of several zerg skills to bring despair to the hearts of Steamhammer’s enemies—bots have big gaps in their knowledge, and maybe I will bring a tractor to drive through the gaps. I may write a capable tactics analyzer. Also I’ve selected the machine learning method I want to use for offline learning of strategy, operations, and tactics, so I may work on that. The shortlist is ironically too long to finish much of, but every item is important so any selection should make a big difference.

For CIG, I made many small changes in a short time. For AIIDE, with more time to prepare, I plan a smaller number of larger changes. I hope to replace major parts, and make enough progress to call it Steamhammer 2.0. We’ll see how it goes!

Steamhammer 1.4.7 is uploaded to SSCAIT. Change list: The crashes should be fixed. A few trivial code cleanups with no effect on play. A few comments rewritten.

Argh, crashes!

I ran a ton of test games, against both stronger and weaker opponents of all races on rotating maps, and saw no sign of any crashing bug. I eliminated nearly all crashes from Steamhammer by about April last year, and since then there have been only a handful of crashes total on SSCAIT. Every release has been reliable. But on SSCAIT, the new Steamhammer is crashing frequently, in up to 1/3 of games. If that happens in CIG, Steamhammer will finish in the bottom half if it is not disqualified altogether. I have contacted the CIG organizers to warn them of the possible disaster.

The symptoms indicate a severe bug which happens in many games and causes a crash or not depending on details of the environment. It’s likely a pointer or iterator thing. Crash records point to squad update routines for different squads. Those were all changed to assign or unassign detectors depending on the situation, so I know where to start looking.

Please stand by while I employ my fine-tooth comb. It’s too late for CIG, but the rest of the world can be saved. I’ll issue a new version when it’s fixed, and I hope it won’t be long.

I’m eager to get on with the real work, but crashes come first. Argh!

Update: I found bugs of the general kind I expected in the places I expected. As usual, the bugs were obvious once I read my own code carefully. Lesson: I was in too much hurry. Code review works even with only 1 coder. I’ll upload version 1.4.7 after reading through again and retesting.

Steamhammer 1.4.6 is uploaded to SSCAIT. Steamhammer's web page is updated with source and binary downloads.

For an idea how much this version improves over 1.4.4, see its first game Steamhammer-Krasi0.

There are only a few visible changes from the CIG version 1.4.5. Terran bunkers and protoss photon cannons in the opening are built @ front in a defensive location, which is usually better. A terran bunker built as a strategy reaction by StrategyManager is also @ front. Reactive photon cannons and turrets are in their usual questionable locations, because they are meant to cover the base, not to defend the front.

I also adjusted the rules for dropping items from the production queue so that terran and protoss openings will execute as written. Formerly, a few openings might drop items and break. See MacroAct::hasTech().

Both changes affect the protoss ForgeExpand opening. It’s str0nger.

Predicting enemy unit movement is easy and fun, and it helps micro a lot. Now that I’ve implemented a version in Steamhammer, I recommend it for everyone.

Adding this feature dramatically improved Steamhammer’s micro in some situations. It affects almost every fight; with prediction, your units do less moving and more shooting. It’s most obvious in a chase. When chasing the enemy scouting worker, zerglings with prediction don’t pursue endlessly, but catch it. The most extreme case may be playing against Stone once zerglings come out. Stone’s SCVs flee like the wind, or they twist and turn trying to evade, but they can’t escape. Steamhammer wins with much less useless chasing.

making the prediction

Here’s my implementation in Steamhammer: Assume that the enemy unit will keep moving in a straight line at a constant speed, and predict a fixed number of frames ahead. It’s dead simple. ClipToMap() makes sure that we return a valid position—we may predict that the enemy falls off the edge of the world. ClipToMap() simply restricts x and y to valid values,. We predict a position closer to where the enemy will end up if it bumps into the edge of the map.

// Predict a visible unit's movement a given number of frames into the future,
// on the assumption that it keeps moving in a straight line.
// If it is predicted to go off the map, clip the prediction to a valid position on the map.
BWAPI::Position PredictMovement(BWAPI::Unit unit, int frames)
{
	UAB_ASSERT(unit && unit->getPosition().isValid(), "bad unit");

	BWAPI::Position pos(
		unit->getPosition().x + int(frames * unit->getVelocityX()),
		unit->getPosition().y + int(frames * unit->getVelocityY())
	);
	ClipToMap(pos);
	return pos;
}

There are fancier ways to predict. You can take terrain into account; an enemy ground unit will probably not pass through unwalkable terrain (though there are ways to do that). You may want to take the chasing unit into account—can it catch up? I believe Iron calculates the closest interception point where the chasing unit can catch the enemy, and heads straight there. That is a more elaborate calculation, but not too difficult, and there are several ways to do it.

using the prediction

How you use the prediction depends on your code, of course. In Steamhammer, it is as simple as making the prediction. Steamhammer normally calls Micro::AttackUnit() to issue an attack command. I added Micro::CatchAndAttackUnit() which attacks if we are able to, and otherwise issues a move command toward the predicted enemy position 8 frames in the future. Here is the working part (I took out irrelevant stuff like error checking and debug drawing).

	if (!target->isMoving() || !attacker->canMove() || attacker->isInWeaponRange(target))
	{
		AttackUnit(attacker, target);
	}
	else
	{
		BWAPI::Position destination(PredictMovement(target, 8));	// the number is how many frames to look ahead
		Move(attacker, destination);
	}

Then I substituted Micro::CatchAndAttackUnit() for Micro::AttackUnit() throughout the code wherever it made sense. The condition !attacker->canMove() means there is no problem if you call it for an inappropriate unit, like a burrowed lurker or a sieged tank, because then it behaves just like the original call. (But I didn’t call it for inappropriate units.)

I chose 8 frames fairly arbitrarily. I think a range of values will work. With 8 frames, we aim for a spot a little ahead of where the enemy is, and not so far ahead that they can throw us off by turning. Dan Gant points out in a comment that PurpleWave predicts a variable number of frames ahead depending on when the chasing unit will be able to attack; the goal is to come into range at the same moment that we are ready to attack.

It makes sense to predict further into the future if the enemy is far away. Calculating the interception point like Iron is the ultimate version of that. Maybe the ideal method is to find the interception point when far away, and switch to the Purple idea when approaching attack range.

There is a disadvantage to the change in Steamhammer. There seems to be a higher rate of units getting stuck, unable to move. I haven’t yet looked into why. It affects protoss the most. It’s possible that the change makes protoss play worse overall.

Question for stream watchers: Would you like some debug drawing turned on? Steamhammer traditionally runs showing only the game info and production queue. Maybe it would be fun to see more? Possibilities: Unit orders (what each unit is doing all the time—including what mineral patches are doing, namely nothing!), squad info (each squad’s order, target location, and retreat status), zerg strategy boss info (basics of zerg strategy decisions). I don’t want to turn on more than one of these at a time, but maybe I could change the settings now and again.

It seems that SSCAIT is down, so maybe I’ll upload the new Steamhammer there tomorrow.

In the meantime, here is Steamhammer 1.4.5, the CIG version. This is the same file I uploaded to CIG, not even renamed. It includes source and binary and the configuration file. Unlike an SSCAIT upload, it does not include BWAPI. Following CIG rules, the ability to configure the game speed and frame skip is disabled—the configuration options are ignored. This and a few other minor details will be changed in version 1.4.6 for SSCAIT.