Starcraft AI blog

I rewrote important parts of base defense. I fixed minor bugs in passing, but I had 2 main reasons: 1. Squad unit clustering caused bad defensive behaviors that had to be fixed. 2. The original base defense, as inherited from UAlbertaBot, was actually region defense. It iterated through regions of the map, and defended any region where the bot owned buildings. On some maps, one region can contain more than one base. Examples are the mineral-only bases on Python which are all in one big center region, and the center bases on Fortress which are laid out similarly. Base defense needed changes to defend these bases successfully.

• Iterate through bases, not regions, to check for areas that need defense. This means that buildings built outside any base area are no longer defended—not a serious loss, they were rarely defended well in the first place. See CombatCommander::updateBaseDefenseSquads(). The squads created to defend bases are called Base squads.

• A base is deemed to be in need of defense when an enemy unit (other than a single worker scout) is within a specified baseDefenseRadius distance of the base, OR if it is in the same region as the base and is within baseDefenseRadius + 300. The base region may be very large (and may contain multiple bases), in which case we don’t want to panic and send defenders when they aren’t needed. Or the base region may be very small, so that attackers outside the region can hit the base, a historical problem for Steamhammer. This condition covers both cases.

• A base is considered to be no longer under attack if there are no enemy units meeting the same criterion, except that the radius is larger by baseDefenseHysteresis. The hysteresis reduces indecisiveness. The numbers baseDefenseRadius and baseDefenseHysteresis are constants defined at the top of CombatCommander::updateBaseDefenseSquads(); at some point I may move them to the configuration file.

• Steamhammer used to ignore overlords, observers, and floating terran buildings in deciding whether the base needs defense. Now it counts them, and will send anti-air units if needed and available. It will also assign a detector if needed to target the observer. Thou Shalt Not Scout For Free.

• A Base squad is disbanded and the squad data structure destroyed when the squad is no longer needed. The squads used to be kept around even when empty. Dropping the squad mainly makes the debug display easier to understand.

• The Base squad’s order is centered on the enemy unit which is closest to the base location (the upper left corner of the command center, nexus, or hatchery). This guarantees that the Base squad won’t sit idle in the middle of the base while the enemy rampages on the outskirts.

• Release defending units that can’t attack any of the remaining intruders. For example, if the enemy shows up with both air and ground units, we might assign both zerglings and mutalisks for defense. If the enemy ground units are destroyed or leave and only air units remain, the zerglings become useless for defense and are released. The useless zerglings used to remain in the Base squad until the whole squad was released.

• Don’t double-count the same unit as both a ground and an air defender. Steamhammer used to think, “I have 4 hydralisks to drive off these corsairs, and I have the same 4 hydralisks to stop these 2 zealots. I have enough defenders!” The calculation to avoid double-counting is kind of involved.

• Fixed a bug introduced in a recent version which could overassign defenders. It would, for example, always assign at least 2 anti-ground defenders when under air attack.

pulling workers

Deciding whether to pull workers for defense—to place them in a Base squad—is also in this code. Pulling workers is sometimes necessary, but it has to be strictly regulated because it puts workers in danger. (Worker self-defense doesn’t involve putting them into a Base squad, and is not handled here.) I tried to fix the biggest weaknesses.

• Pull workers that are within a smaller radius: 18 tiles instead of 24. This is about the location of the worker. That should mostly prevent Steamhammer from pulling workers from another base to defend this one.

• Pull workers only when the enemy is very close to the mineral line. This is about the location of the enemy. Steamhammer used to pull workers to meet the enemy too far away from the mineral line, wasting time and putting workers in unnecessary danger.

• Release pulled workers much sooner. Steamhammer used to retain pulled workers in the Base squad until the squad was disbanded. Now it releases the workers when the “must pull workers!” condition is not satisfied.

Next: Squad unit clustering.

The game Steamhammer-Bereaver is a good example of Steamhammer’s defiler skills, both the good and bad points. Steamhammer gained an early advantage and contained Bereaver to its main, so the outcome of the game was not in doubt, hive tech or not. Bereaver mounted a strong defense at its ramp and held on for a long time. A timid zerg might have feared to attack and lost on points when the game time ran out—but of course Steamhammer is not timid.

The defiler dithered for a long time. Consume research finished at about 15:45 into the game, and the defiler filled up on energy within 15 seconds after, so that part was OK. A first attempt to break up the ramp was repelled with storm; the defiler was too far back and unable to help, due to lacking squad coordination. The second attempt to break up the ramp was at 18:20, and this time the defiler was in range. That’s about 2 and a half minutes from when the defiler was ready to act until it finally did; it had plenty of time in between to swarm or plague. It would have been worth it to sacrifice the defiler to plague the defenders.

Excellent swarm placement! The defiler had energy to cast a second swarm farther into the enemy base, but did not. Even so, the one swarm nullified the dragoons and cannons and ensured that the attack would succeed if pressed. Thanks to my modifications, FAP understood that it would succeed and Steamhammer did press until it broke through. The attack had to be under way before the defiler could support it; Steamhammer has no ability to swarm (or plan to swarm) in advance of an attack. Steamhammer attacks when maxed, so in most winning games an eventual attack is guaranteed. When the swarm does come, it is generally well-positioned for where the units are at the moment, rather than where they are going or where they want to be.

The defiler seemed to get confused and did not contribute again until the very end, when it laid down a perfectly-placed plague over a few protoss buildings that were about to be destroyed anyway. The laser-precise plague hit as many protoss buildings as possible while missing zerg units. It’s impressive in a way, and it may even have shortened the game... by a fraction of a second.

Still next: Base defense.

Defilers are complicated to use. I implemented a pile of features to get them to work acceptably, and I suspect that a bug has crept in since they don’t seem to work as well as when I first finished the implementation. They don’t swarm or plague as often as they used to. A likely source of the bug is the time control work.

The first step is to get defiler tech. The strategy boss decides when to get defilers. Currently, it delays defilers until fairly late versus terran that has a high ratio of tanks and/or vultures in its force, especially with spider mines, and otherwise uses the same criteria for all races: Do I have a hive, enough drones, and a few more items? When the defiler mound finishes, the strategy boss eagerly orders consume research. It is lazier about ordering plague, and later metasynaptic node (+50 energy). According to conventional wisdom, the energy upgrade is useful for defilers because then a full-energy defiler can swarm once and plague once before it has to consume again.

The strategy boss currently orders the production of 1 defiler at a time. A second defiler is not made unless the first one is lost. The defiler is sent toward the front in the hope that it will find something useful to do there.

consume

The consume implementation is on 3 levels. Strategy: If there are not enough zerglings on the field for defilers to consume, because the unit mix calls for other units, then the strategy boss orders up a few more. It makes sure that there exist at least 4 zerglings for each defiler. Tactics: A small number of zerglings are assigned to the MicroDefilers unit controller alongside any defilers. The number assigned depends on how much energy the defilers need; see Squad::addUnitsToMicroManagers(). The controller orders the zerglings to catch up to and stay near the defilers; each seeks the closest hungry defiler. Micro: If a defiler is hungry and finds an assigned zergling standing by, it’s lunch time.

A defiler counts as “hungry” if it has less than 150 energy. Defilers don’t consume anything other than zerglings which have been assigned for consumption. It’s not urgent, but it would be nice if they could consume irradiated units, or any units nearby that are destined to die. In an emergency, they should be willing to consume overlords. In the very late game, when minerals are gone and gas is plentiful, it may be more efficient to consume scourge.

plague

Plague is a spell of opportunity. If there is a big ball of enemies that aren’t already plagued and have a lot of hit points to lose, it virtually always pays to plague them if you can. Steamhammer is not smart enough to intentionally seek out good opportunities, but it is smart enough to recognize opportunities that arise.

So defilers keep their eyes open. If a defiler passes a cheap check (I have plague researched, I have enough energy, there are a bunch of enemies nearby) then it performs an expensive check to decide whether and where to cast plague. It draws a square of each place it can reach without moving far, and does a raster scan through the square, scoring each possible plague it might cast. For enemy units, it adds up the hit points the enemy will lose, counting it as 0 if the enemy is already plagued and adjusting for a few special cases: Buildings other than static defense get a discount, they are less important to plague, while a bonus goes to cloaked units (which are revealed by plague) and carrier interceptors (which will get stuck in the carrier until plague wears off, as the carrier tries to repair them). It also subtracts plague on friendly units, so that it is willing to plague itself if it covers enough enemies too. If the best score passes a threshold, cast the plague there.

Death rule: If the defiler thinks it is about to die, then plaguing any enemy is good enough. Theoretically, in this case Steamhammer might plague 1 stray probe that happens to be close enough.

This exhaustive search method is startlingly accurate at deciding where to plague (though I think I should increase the discount for buildings). I’m sure there are computationally cheaper ways to get good results.

dark swarm

Dark swarm is way more difficult to use well than plague. Swarm use ought to be part of a coordinated plan. A simple plan might be: Start the attacking units moving, then cast swarm around the time the attackers come into cannon range. The decision to attack should be informed by the intention to cast swarm during the attack—with enough swarms, one zergling can be the Klingon running man and kill all the marines. You need different plans for assaulting a fixed position, fighting mobile enemies that can evade swarm, or warding off an attack on your base. You have to take into account what units on both sides are able to do damage under swarm.

Steamhammer can’t plan like that. Those skills are far in the future. All zerg bots are weak with swarm.

Steamhammer chooses where to swarm with an exhaustive search similar to the plague search, and it is not as effective. First, the quick check only allows swarm over enemy buildings; Steamhammer can use swarm to attack, and not to maneuver or to defend, a major limitation. Protect the army in the middle of the map from air attack? No can do, boss: Besides the defiler skill, that would require more smarts in unit control. In the exhaustive search, it scores the units that end up under dark swarm, trying to cover friendly units that can hit under swarm and enemy units that can’t. It’s not the score you want; it ought to take into account nearby units that will want to move into the swarm, or will be unable to hit swarmed units from outside, or will have to suddenly flee.

Death rule: If the defiler is about to die, it will swarm wherever it thinks best, even if it doesn’t cover an enemy building. If the defiler doesn’t have any better idea, it will cast swarm over itself, which may save it depending on the situation. If the dying defiler could either swarm or plague, the choice depends on which happens to be checked first.

I implemented consume, plague, and swarm as micro skills, decided at the level where the code is controlling a single defiler. It makes sense for consume and plague. I think swarm should be decided at the squad level, because its use depends on the tactical situation and the squad’s goals. That will require a lot more cleverness in the squad, though.

combat simulation and micro

I modified the combat simulator FAP with a rough understanding of dark swarm. FAP performs a loose approximation of combat, and the dark swarm knowledge I added is equally loose. If a unit is under dark swarm at the start of the simulation, it is assumed to remain under dark swarm throughout, no matter how it may move around. (Movement ignores terrain and other units too, so FAP is already loose that way.) It understands which units can do damage under swarm, and has simulated units seek out only targets that they can damage.

All this happens only if dark swarm is already cast. There is no provision for planning ahead, “when the units start shooting, I’ll cast swarm here.” In other words, dark swarm influences the combat sim, but the combat sim does not influence dark swarm; Steamhammer doesn’t extract the full value. See the discussion above about coordinated plans.

Steamhammer also understands dark swarm at the unit micro level. Melee units seek out targets under dark swarm, on the theory that then they will be protected under swarm too (“ha ha, you’re under swarm, I’m perfectly safe... I hope?”). Ranged units skip targets under swarm that they can’t hit. The ranged unit controller understands which ranged units can do damage under dark swarm.

There is room for more smarts in exploiting dark swarm at the tactical level. Units under air attack, for example, or facing marines, should move into swarm for their own protection, and mostly they don’t. I’ve seen Steamhammer make serious mistakes because of that ignorance.

time control

My first implementation made 2 defilers at once and performed all the computation steps in the same frame. A disadvantage was that sometimes the 2 defilers would swarm the same place at the same time (scoring ensures that if they swarm at different times, they swarm different places). A bigger disadvantage is that it was too slow, with some frames taking over 150ms. I changed it to make only 1 defiler at a time, and arranged that consume, plague, and dark swarm are examined in different frames. That made it fast enough. There are ways to speed up the calculations, but slicing them apart seemed easier.

Overall, the defiler system is not industrial strength, but it’s a strong start. As I mentioned at the start, I suspect a bug due to the time controls. Steamhammer implements all the basic skills, which if you go through the above text you can estimate for yourself required 1.5 zillion different software features. Now that the pieces are in place, they can be improved one by one.

One of the emergent properties of the defiler system is that it is unpredictable. The defiler may plague this and then plague that, or swarm here and then swarm there—it gives the impression of running in streaks. And I can’t judge ahead of time what it will do, it surprises me.

Next: Base defense.

Steamhammer 2.0 is uploaded at SSCAIT. As I have mentioned, it is zerg only; Randomhammer is not updated. The only difference from the AIIDE 2018 version is that I turned on a few debug options, notably the new option to draw unit clusters.

I erased the learning data, so Steamhammer 2.0 is figuring out its opponents from scratch. It’s necessary since this version has different strengths and weaknesses. I expect its elo to plummet until it gets to know the different kinds of rushbot (just like the last time I erased the learning data).

Today I list the big changes in Steamhammer 2.0 without going into detail. There are tons of smaller changes. Details will follow.

The most effective changes:

• squad units are clustered, and each cluster makes attack/retreat decisions independently
• mutalisk and especially hydralisk micro is crisper
• a base under attack does not receive transferred workers, improving resilience
• greatly reduce cases where the bot gets a spire/lurker aspect and then delays getting mutas/lurkers

The squad unit clustering allows Steamhammer to play a big army macro game without the constant tactical blunders that it used to make (it makes smaller and mostly different blunders). Many bots use a similar system, with good reason. The tactics rework is substantial, and deserves the 2.0 version number by itself. In combination with the other improvements, Steamhammer is incomparably stronger at ZvP, which may now be its best matchup. I reported on DaQin, and Steamhammer (like Proxy) can open with 12 hatchery and still cope with Wuli’s rush (Proxy’s author called it easy, but it requires a combination of several skills that are not so easy).

I put in a ton of work, but the decisions that unit clusters make are still not as polished as they need to be. An interaction with the combat simulator causes dangerous mistakes in retreating from superior forces. It can happen in all matchups, but it is most obvious in zergling fights in ZvZ. What used to be Steamhammer’s most successful style of play in its best matchup is now its least successful in what may be its weakest matchup. It’s pretty funny, or at least it will be after I figure out how to fix it.

All these items deserve their places on the list. I fixed the case where Steamhammer’s spire used to finish and it would make a bunch of drones instead of mutalisks, and that may sound like the usual modest improvement to the strategy boss. In fact it is a critical fix that makes midgame tech switches more rapid and decisive. It wins games.

Other important changes:

• mineral locking
• base defense was reworked extensively
• new openings added to exploit possibilities and old openings tweaked to fix flaws that had crept in
• opponent model exploration and strategy randomization improved
• defiler support with all spells: consume, dark swarm, plague

Many aspects of play have moved forward. As always, there are new bugs like the retreating problem I mentioned above. And as I blathered about yesterday, old weaknesses remain. The overall result is as I’ve reported before, hugely stronger play against some opponents and weaker play against others. Well, that’s in my testing, which is never enough. We’ll see how it goes in the wide world! Since we’re in a time of protoss domination, I’m thinking that strength against protoss is a good sign.

For AIIDE in particular, I expect that my changes to the opponent model are key. I made the couple needed improvements of a previous post. The academic papers I found turned out to be unhelpful, and my changes are ad hoc, but they address the problems. If the opponent chooses strategies unpredictably, Steamhammer will too. If the opponent model doesn’t find a winning counter, it will explore more and more widely until it is randomly trying anything it knows, including openings which are otherwise not played in any context. On the upside, I’m hoping it will eventually hit on surprise counters to the toughest opponents. On the downside, in the worst case it will explore a lot of bad ideas, and the convergence time could be longer than the tournament!

Later today: Notice after Steamhammer 2.0 is uploaded to SSCAIT. Tomorrow: Defiler support in detail.

Today, a post about things I wanted to do and didn’t get to, or started to do and rolled back or didn’t finish. I accomplished a lot, but the to-do list is functionally infinite.

• UAlbertaBot executes unit orders as they are issued during the frame, retaining no state beyond what BWAPI remembers. Steamhammer inherited the system. I am in the process of changing it to record the orders and save status info, then issue any needed orders at the end of the frame. Adding the Micro object in the last version was one step. The Micro object now records the order and status of each of our units, and has all the infrastructure it needs to issue orders, including its own update() method with TimerManager calls to track how long it takes. But it doesn’t actually issue orders; there is no effective change to the micro system.

When this is done, it will be easier or possible to unstick units, to make sure units don’t get stuck in the first place, and in general to control units precisely.

• I worked a bit on analyzing Steamhammer’s use of the combat simulator. Improvements are there for the taking, but my first try was not successful and I left it for later.

• McRave suggested that micro to avoid big damage attacks was key. Equally important is to pull off big damage attacks yourself. Besides plague, the zerg big damage attack is scourge suicide, and Steamhammer sucks with scourge. Scourge control is on my list, but never came to the top.

• I implemented mine dodging to reduce the danger of spider mines, but the way I did it was not useful for zerg. Reacting to mines is tricky: If a mine pops up, sometimes you should drag it to the enemy, sometimes drag it away from your other units, sometimes shoot it down. I lost the “just shoot it” case, and it was a net disadvantage. The feature is in the code but is disabled.

• Steamhammer’s lurker skills were adequate for when they were implemented, but have fallen behind the times. It needs to stop doing things like burrowing one lurker at a time in cannon range.

• I haven’t done anything to improve overlord safety, dark templar reactions, or any of that stuff. Many weaknesses there.

• I planned and promised a skill to put pressure on the enemy in the early or middle game, but did not finish one. I had several ideas, and all of them hinged on what I call raiding: Harassing where possible, moving around but staying near the enemy, not retreating to rejoin the main army. Harassing mutalisks can be considered raiders in that sense, and so can Krasi0’s vultures and Bereaver’s reaver drops. Runby units and dropped units should act as raiders, and Steamhammer doesn’t have the skill—with a Hold order, they go idle if they clear their area to hold, and with an Attack order they may try to retreat to the main base. Picking up dropped units is also a good raiding skill.

• I implemented a Watch squad, but did not polish it up enough that it became useful. It is turned off. It had several purposes. It could keep watch on expansions and chokes, sometimes making it possible to disband the Recon squad. It could prevent enemy expansions in some cases, and clear mines and verify the safety of expansions that we wanted to take.

• I thought of adding pre-learned opponent model data for the fixed opponents carried over from last year, plus opponent model hint configuration (like “try this first and see how it works”) for opponents that I think I know something about. PurpleWave, we’re told, did an elaborate job of that for CIG this year. The idea of hints is that the opponent model can save exploration time if the hints are right, winning more games, and correct itself with little loss if the hints are wrong. I decided that, since AIIDE is a long tournament with plenty of time for learning to converge, other work would probably help more.

upcoming

Done with the testing, today I am resting. I’ll upload zerg-only Steamhammer 2.0 to SSCAIT tomorrow. On the blog side, I think I’ll start with an overall evaluation post, what I’ve done and how successful it seems, and follow up with posts about specifics like defiler skills and squad restructuring, and then the big detailed change list. At some point soon Steamhammer 2.1 will slip in. 2.1 will likely have a few minor fixes and deserve its own little change list.

Shortly, CIG should release its detailed results. They promised source and replays. If they also provide a game result log, I’ll analyze it and post my usual colorful summary tables. Not sure how that will fit in with the other posts, the blog may have a busy time.

Steamhammer 2.0 is submitted for AIIDE 2018. All I have left is the waiting.

Since yesterday I fixed one bug, a newly introduced bug in the worker manager that could cause a small number of drones to temporarily go idle early in the game, when it matters a lot. I made an attempt to fix a lingering weakness related to cooperating with static defense, but had to roll it back. The fix was a 2-line change, but somehow it caused a hard-to-understand bug, and I can’t accept a risk like that so close to submission time. Also, since an entire day is too long to go without working on the openings, I added a new opening that helps Steamhammer sidestep weaknesses in ZvZ, allowing it to beat more opponents. And I ran all the tests I had time for, looking for new bugs, checking my fixes, and verifying that the new opening functions as intended and earns wins.

It’s hard to understand why, but now I feel tired....

New bot DaQin on SSCAIT matches an AIIDE entrant. It may be the AIIDE entry, or it may be a test version shortly before the entry is made. It looks like a Locutus fork. The .dll is larger than the Locutus .dll by about 24K out of 2.5M—a fair amount of code but no drastic rewrite. The play I’ve seen so far looks similar, lots of dragoons with strong Locutus micro, so without digging deeper I’m not sure what the changes are.

As one of my tests today, I ran a short match Steamhammer 2.0 - DaQin. Steamhammer took several games to find a way to win (not long at all), and after that it won every game, though I didn’t play many and one of the games was excitingly close. I didn’t try the old version Steamhammer 1.4.7, but with Locutus skills I expect that DaQin would have won every game. I don’t think Steamhammer 1.4.x has a way to beat DaQin. It was satisfying to see Steamhammer 2.0 score convincingly, and it helped my confidence.

We’ll see how much Locutus has improved since August!

Steamhammer is almost ready for AIIDE. I have fixed all the major bugs and weaknesses that I have time for—the last was a newly-introduced bug in mutalisks caused by tightening up the micro, it was a bad one. The rest will be simple low-risk fixes and testing to make sure I don’t have another disaster like in CIG (it’s a risk, I wrote a bunch of new code and my change log records over 80 items).

Steamhammer is still weaker against some opponents, though not by as wide a margin as in my last update. It still crushes some opponents that it used to lose to. I can’t foresee which way the balance will fall, definitely not in this tournament with so many new names. My ambition remains to finish in the top 3rd, about the same as last year, which will mean that I am keeping up with the blazing fast progress in the scene.

There are changes at all levels. I have the highest hopes for improvements to the opponent model (key for a long tournament), the squad structure (sounder in principle than the old one), and certain of the micro optimizations (not fully exploited yet, but a firmer foundation).

Regardless how well Steamhammer performs in AIIDE, looking back over the past year I feel I’ve gotten a good amount done. Steamhammer is still short on basic skills, but compared to a year ago it is smarter in most ways. There is a mountain left to move, but I am that much closer to having moved it.

Steamhammer 2.0 for AIIDE will be specialized for zerg only. There won’t be stuff for the other races in the config file, and I haven’t tested that terran and protoss work at all. I’ll upload it to SSCAIT shortly after the AIIDE deadline, only turning on a few of the screen drawing options. Since it plays so differently, I’ll erase the learned opponent data again. A few days after that, depending on how wiped out I feel and whether any surprises turn up, expect Steamhammer 2.1 and its identical twin Randomhammer that plays all races. Along with, of course, a series of posts about the details of my work, the unexpected problems that cropped up, and the known problems that I would have solved if only there were more time.