Today is Steamhammer 1.4’s opponent model. The features are:
- Recognize some enemy opening plans.
- Predict the enemy’s opening plan from experience against this opponent.
- React to the enemy’s predicted or actual opening plan.
- Choose openings based on the predicted plan.
- Decide whether to steal gas. Does experience suggest it may be worth trying against this opponent?
The code that implements the features has these parts:
- The plan recognizer, which looks at the current game situation.
- Game records, which save information about past games against this opponent, including the recognized plan.
- The plan predictor and gas steal decider, which draw conclusions based on the game records.
- Strategy reactions to predicted or recognized enemy plans are in various places–they are uses of the opponent model, not parts of the opponent model.
- Opening choices to counter the predicted plan can be set in the configuration file.
1. The plan recognizer was first written up in December. It tries to understand the opponent’s intentions during the earliest part of the game. The code is in OpponentPlan.cpp and is less than 150 lines—it is rudimentary.
Unknown - No plan recognized. The plans are not exhaustive, so this is common.Proxy - Enemy buildings in your base. This doesn’t include cannon contains or other more distant proxies.WorkerRush - Like Stone.FastRush - Basic units faster than 9 pool, 8 rax, or 9 gateway.HeavyRush - 2 gate zealots, 2 barracks marines, etc.SafeExpand - Static defense before expanding to the natural. Zerg can’t do this.NakedExpand - Expansion with no static defense.Turtle - Static defense without expanding.
My early impression of the plan recognizer was that it often failed to recognize plans, but was rarely wrong when it did. With more experience, I think it often misrecognizes plans severely. It’s crude and clumsy. Even so, when it helps it helps a lot. It’s a net win.
2. Game records are handled in GameRecord.cpp. Steamhammer writes a file for each opponent, like many learning bots. Here is one record from a file named om_UAlbertaBot.txt, where “om” stands for opponent model, with annotations so you can make sense of the list of numbers.
1.4 <- record format version (from the Steamhammer version when it was first used)
ZvRZ <- matchup
(2)Destination.scx <- map
Over10Hatch <- Steamhammer’s opening
Heavy rush <- initial predicted enemy plan
Fast rush <- actual enemy plan
1 <- we won, 1 or 0
0 <- frame we sent a scout to steal gas, 0 if never
0 <- gas steal happened, 1 or 0 (extractor was/was not queued)
2110 <- frame enemy first scouted our base
3102 <- frame enemy got first combat units
0 <- frame enemy got first air units
0 <- frame enemy got static air defense
0 <- frame enemy got mobile air defense
0 <- frame enemy got cloaked units
0 <- frame enemy got static detection
1726 <- frame enemy got mobile detection
12309 <- last frame of the game
END GAME <- end mark
Theoretically, a single file could have records in more than one format. Of course, only one format exists so far. The “frame enemy got” times recorded are the time we first saw such a thing, which may be much later than it actually happened. For example, here we first saw an enemy overlord (a mobile detector) on frame 1726, but it existed the whole game. The END GAME mark is redundant. It gives us a way to recover in case data in the middle of a file is corrupted—we can skip ahead past the next END GAME and continue reading the file from there.
3. The plan predictor and gas steal decider look at the game records near the start of the game.
The gas steal decider was written up in December. Nothing important has changed since then.
The plan predictor runs once at the start of the game. It looks through the game records to see what recognized plans the enemy has played. It is close to the bare minimum: It counts the recognized plans in the game records for this matchup, ignoring unknown plans, and weighting recent games more using a discount factor so that the past is gradually forgotten. That way it reacts quickly when the enemy changes its play. Whether the game was won or lost, whether past predictions were correct or wrong—all the other information is ignored.
If the opponent was random, then when the opponent’s race is found out, the plan predictor runs again. In the first run, it counted all game records where the opponent went random. In the second run, it counts only games where the opponent was the same race as this game. The predicted plan may change.
4. Strategy reactions could be written in anywhere, but so far they are in StrategyManager for terran and protoss, and in StrategyBossZerg for zerg. So far, all the reactions are reactions to the enemy plan, not to any of the other information (even though it has obvious uses). Some strategy reactions are made when the enemy plan is recognized. Some reactions must begin in time, and happen when the plan is predicted. For example, against UAlbertaBot, the initial predicted plan is “Heavy rush”. If Steamhammer finds out that UAlbertaBot is zerg, it knows that zerg follows a different plan. The predicted plan changes to “Fast rush” and efforts to stop the zerglings begin immediately (if Steamhammer is zerg or terran; the protoss reaction is turned off because I didn’t get it working).
Good strategy reactions are the hard part. I find them much more difficult than the opponent model proper. Some of Steamhammer’s reactions are weak.
5. Openings to counter specific enemy plans can be written into the configuration file in a new subsection CounterStrategies. The opening is chosen once at the start of the game and can’t be changed (in this version), so only the initial predicted enemy plan matters.
Steamhammer first looks for counter strategies specific to the enemy race. The name is in the format “Counter [plan name] v[race character]”. The race character is “U” for Unknown if the opponent went random. You can use all the usual features of random opening selection. As you can see in the example, zerg is configured with a wider range of choices.
"Counter Safe expand vT" :
{
"Terran" : "14CCTanks",
"Protoss" : "13Nexus",
"Zerg" : [
{ "Weight" : 1, "Strategy" : "FastPool", "Weight2" : 5 },
{ "Weight" : 9, "Strategy" : "9PoolSpeed" },
{ "Weight" : 0, "Strategy" : "ZvT_3HatchMuta", "Weight2" : 50 },
{ "Weight" : 50, "Strategy" : "ZvT_3HatchMutaExpo", "Weight2" : 0 },
{ "Weight" : 30, "Strategy" : "3HatchLurker" },
{ "Weight" : 10, "Strategy" : "3HatchPoolMuta", "Weight2" : 0 }
]
},
If no counter strategy is found for the specific enemy race, Steamhammer next looks to see if there’s a general one for all races—leave out the “vX” string. This example points to a reusable strategy combo that specifies openings for each race Steamhammer might be playing, no matter the enemy race. The strategy combo feature has not changed.
"Counter Worker rush" : "AntiFastCheese",
If no counter strategy is found, Steamhammer falls back on its usual random opening selection. So if the regular repertoire is best in any given case, don’t specify a counter for that case.
For the following version Steamhammer 1.4.1, I will add at least one large piece to the opponent model. I'm likely to change my mind once I see how this version does in practice, but at the moment I'm thinking of fine-tuning plan prediction and opening selection based on wins and losses. That will raise Steamhammer's performance ceiling; with enough games, it will learn much more. Other possibilities include: • Make use of more of the information in the game records. This opponent doesn't get detection, use cloaked units; that opponent gets air units around frame X, add a spire for scourge just in time. • Have Steamhammer collect data on its own openings so it can generalize: Play a fast/slow opening; play a lurker/muta opening. • Revamp the plan recognizer with a richer set of plans, maybe a hierarchy so it can refine its conclusions over time. • Machine learning for a probabilistic plan recognizer and/or plan predictor. • Restore the originally planned extensive game records, making it possible to predict unit mixes over time throughout the game. • Add the ability to change openings during play instead of deciding ahead of time, so that decisions can be made as late as possible. • Move scout timing into the opponent model alongside the gas steal decider. • Have expansion decisions or hatchery placement influenced by the opponent model. “Hmm, historically in situations like this we do poorly if the third hatchery is at an expansion. Make it a macro hatchery instead.”
I have no shortage of ideas!
Next: Brief remarks on the SSCAIT round of 8.