Starcraft AI blog

The tournament version Steamhammer 2.1.4 suffers from a command jam bug which reissues commands far too often, causing many to be dropped. It’s a critical bug with devastating effects, causing units to ignore their orders—to freeze in place, to wander past the enemy taking fire without noticing, and so on. It starts having an effect often before the zerg supply reaches 50, and the effect grows worse as supply increases. By the late game, large groups of units are sitting uselessly around the map doing nothing. It’s a critical bug and intolerably severe.

But how critical is it really? I look at every game that Steamhammer plays. Based on tournament losses, I estimate that if I had fixed the bug before SSCAIT started, Steamhammer’s rank would not be #10 as now, but #7—not much gain considering how closely the ranks are spaced, only a few percentage points up in win rate.

How can such a calamitous bug have so little practical effect? In Steamhammer’s early days, one version had a bug that subtly caused building construction to be delayed. I doubt any stream viewer noticed; I didn’t notice either, until surprised by unexpected losses. Experience and test games proved that it was a critical bug that caused a high rate of losses against early aggression. By comparison, the command jam bug is identifiable as the cause of loss in only a few games, like the one loss against XIMP by Tomas Vajda. In other games where the bug struck hard, as against ICEbot and MadMix, Steamhammer struggled more than it should have but won regardless.

Apparently the bug causes losses only against a narrow range of opponents which play macro games and are strong enough to exploit the weak play that the bug causes. There is no effect against a strong opponent like SAIDA, or against the weakest opponents which lose to Steamhammer’s first 6 zerglings. One explanation is that most opponents either prefer early aggression, or else fall to Steamhammer’s early aggression. Another explanation is that I may underestimate the damage the bug causes; maybe it leads to losses that are not clearly attributable.

forge expand reaction

Why is it still called “forge fast expand”? It was a fast expansion when invented, but by today’s standards it’s not fast at all. That’s why I say “forge expand.” (It has the same number of syllables as FFE).

Though I still have region work to do, today I decided to make an important improvement to how Steamhammer reacts to forge expand and other safe macro openings. The tournament Steamhammer makes 3 attempts to adapt: 1. If the enemy’s opening plan was predicted, it tries to select a good counter opening. 2. Otherwise, having missed the prediction and gone down a poor path, it cancels any planned static defense which is now unnecessary, and 3. makes extra drones to catch up in economy. If it’s still in its opening book, it stays the course and tries to minimize the disruption by changing planned zerglings into drones, which cost the same.

Its plans are still disrupted, though, because the extra drones and the omitted static defense cause minerals to build up. Steamhammer waits until the opening is over before it makes macro hatcheries and otherwise spends down its excess resources, and that is often too late. Zerg can’t keep up with the enemy’s economy and falls far behind.

Today I added 2 new reactions that happen when we want extra drones so that resources threaten to build up: 4. If possible, take gas early (or take another gas early). Putting drones on gas slows down mineral accumulation and may speed up tech openings, so that mutas or lurkers come out sooner. If gas accumulates too much, Steamhammer will stop gas collection, so there’s little downside. 5. Make extra hatcheries as conditions seem ripe. One or all of the extra hatcheries may be placed at expansions, depending on the situation. The rules are more cautious than the macro hatchery rules that apply once the opening is over, because they’re still trying not to disrupt the opening line. The overall effect of the new reactions is that Steamhammer pursues the tech of its opening line, sometimes faster since it has more income, gas, and larvas than expected, and transitions into the middle game in a stronger position. It’s making a big difference in test games, including wins from positions that were sure losses otherwise.

The fix is inspired by recent losses, especially the 2 losses to Skynet by Andrew Smith when it unexpectedly (to Steamhammer) switched from zealot rushes and DT rushes to forge expand. To my intuition, the forge expand reaction seems much less important than the command jam fix, which is a critical bug fix that affects far more games. And yet, taking into account test games and the rate at which Steamhammer was surprised by macro openings in the tournament, I estimate that it will save about 2/3 as many losses—in terms of improving elo, both seem almost equal. How does that happen?

Apparently you have to measure the severity of weaknesses, because intuition does not seem accurate. Unfortunately, to measure with an A/B test, first you need to fix the weakness. Maybe that is an advantage of machine learning, which does its entire job by measuring weaknesses and correcting them.