archive by month
Skip to content

Steamhammer opponent modeling goals

Next up for Steamhammer is opponent modeling. Some version of it will be ready for AIIDE, deadline 1 September. Opening learning methods that we have seen so far implicitly assume that the bot knows a small number of strategies and that one of them is the best counter for the opponent’s play. I want Steamhammer to be able to cope with opponents that are reactive (Iron), multi-strategy (Zia), or both (Krasi0). My goals for opponent modeling are something like this:

  • Play a wide range of strategies. It has been part of my plan from the first.
  • Learn from the events of the game, not only from the outcome. That way the bot can learn more and faster.
  • Learn from one trial: See an opponent’s strategy in the first game and counter it in the second, or at least make a good try. Fixed-strategy bots that Steamhammer knows a counter for should stand at a disadvantage from the second game on.
  • React by both choosing a counter opening and later choosing a counter unit mix in the middle game.
  • As more games are completed, recognize the range of the opponent’s play.
  • If no one opening strategy counters the full range of the opponent’s play, use game theory to estimate the best mix of openings.

It’s fancier than the strategy learning we’ve seen before, but it doesn’t seem hard to me. It’s straightforward, at least in principle. The key element is a model of game strategy. Already last night I wrote a GameRecord class that keeps a simplified description of what happens. That will be the basis for reading the opponent’s strategy over the game. As soon as we have one game record for an opponent, we can check our openings against it to predict which openings will succeed, and we can also use the record to make middle game decisions about what to produce.

Against opponents with a fixed play style, like XIMP, I expect this to be fast-acting poison. Steamhammer won’t react “Oh, carriers, I’d better make some scourge,” it will prepare ahead of time, “4 carriers will be coming in a minute or so, let me make the right amount of scourge to explode them all.” Against opponents that vary their play, it will take more games to formulate effective venom.

If I have time, I’ll do more. There is no chance that I can get all of these ideas implemented in time for AIIDE, but I’ll make what progress I can.

  • Generalize across opponents, so that an unknown opponent faces play that has proved strong before. If you play a lot of openings, then some of them are weak in most circumstances; so far I have accepted that.
  • Take more information into account, including map and starting positions.
  • Integrate scouting information with the opponent model to get the best possible prediction of what the opponent is aiming for this game.
  • Arrange the known opening lines into a tree, and use the integrated prediction to make decisions at every branch. Opening play will become reactive moment by moment, more like pro play.
  • Use the same mechanism to decide when to break out of the opening book.
  • Record the decisions made just after leaving the opening book as a new opening line to be added to the book and possibly played against other opponents. With breaking out plus adding opening lines, Steamhammer gains the ability to invent its own openings.

Trackbacks

No Trackbacks

Comments

jtolmar on :

"See an opponent’s strategy in the first game and counter it in the second." This implies knowing what the opponent's strategy in the first game was. Could you share your thoughts on how to do that? Comparing to a fixed book of known openings?

Jay Scott on :

I did explain it, though super briefly. Your observation is only partly right. If I’m playing against protoss, it may help me to know how many gateways the enemy made and when, but I can get by with knowing how many zealots appear in my path and when. A pro can use the zealot count and timing to infer the gateway count and understand how it shapes the enemy’s future play, but for this cut I’ll be satisfied with learning the results of the enemy strategy, not its internals.

krasi0 on :

Wow! All of this is very ambitious! Far beyond what I've got under my belt so far. If you implement even half of what's planned, you'd be further ahead of what we've seen from any bots until now.
Let the opening learning wars begin! Good luck! :D

Jay Scott on :

Tscmoo’s neural network strategy learning is in one way more ambitious: It in effect learns its own game model instead of being given one, so in principle it can be more flexible in coping with the enemy’s reactivity. The downside is that it needs more games before it learns much. My take on strategy learning in “new Overkill” is similar; it is too purely empirical. To apply their observations immediately, bots need a game model known in advance.

krasi0 on :

And how far have tscmoo and the new Overkill gone with that? What has the outcome been so far? I am not familiar with their internals and am just curious. All 3 tscmoo bots are in top 7, so the author must be doing something right :)

Jay Scott on :

I wrote up Overkill’s method in a series “Overkill’s new learning” starting on 13 November 2016, findable in the Overkill category: http://satirist.org/ai/starcraft/blog/categories/4-Overkill . I haven’t seen anything new on it since then. As for tscmoo, I’m not sure what it is doing now. The first try at neural network learning did not seem successful. I don’t know whether it has been improved since then, or abandoned, or what.

MicroDK on :

I agree... this seems very ambitious and to me it does not sound to be straightforward to implement. First you need to find out what kind of information you want to record in your GameRecord class. Next you need to find out if you can use that information to find correct counter measures for openings and middle game unit mix. But if you can pull this off SH is in very good shape! I like that your goal is to have SH play like a pro: Reactive openings and creating new opening lines. :D

Joseph Huang on :

Your goals are quite ambitious. I will be happy with trying to stop production freezes and have more reactive unit mixes for now. I will also probably shorten openings or exit early in case of detected ffe builds.

Add Comment

E-Mail addresses will not be displayed and will only be used for E-Mail notifications.

To prevent automated Bots from commentspamming, please enter the string you see in the image below in the appropriate input box. Your comment will only be submitted if the strings match. Please ensure that your browser supports and accepts cookies, or your comment cannot be verified correctly.
CAPTCHA

Form options

Submitted comments will be subject to moderation before being displayed.