archive by month
Skip to content

Steamhammer progress so far

Steamhammer is coming along steadily for AIIDE. It is already noticeably stronger than the release version now playing on SSCAIT.

Defiler skills may never be as good as I want; defilers are difficult. But the skills are there, and zerg can lay down the law in the late game. I can’t always predict whether Steamhammer will swarm or plague in a given situation. It’s satisfying to see a carrier plagued, and there is a bonus in the scoring function to favor exploiting plague-on-interceptor. XIMP has no hope.

Next I made some micro changes. After playing games against several opponents at very slow speed to watch the fast decisions, I’m convinced that Steamhammer is making fewer wasted movements and fighting more efficiently. I want to make further improvements, but it’s enough for the moment. I’ll see how the time works out.

I’m thinking through a more detailed plan for the tactical analyzer. While that is crystallizing, today I am working on the retreat problem which has left the current Steamhammer weaker than the previous version. Once retreat is ironed out, I should have the strongest Steamhammer yet and a good foundation for the next changes.

Meanwhile, I notice that PurpleWave has learned how to delay the zerg natural in the early game with its scouting probe by denying the hatchery from starting. It’s the classical reason usually cited for why zergs switched from 12 hatchery to overpool against protoss. I deliberately chose 12 hatchery for many of Steamhammer’s openings because no protoss bots had the hatchery denial skill, and now that has changed. Besides opening pool first, other standard reactions are to make the second hatchery elsewhere (at what would otherwise be the third base) or to fight a little micro war of drone versus probe—or both at once. Hmm, so many choices....

Trackbacks

No Trackbacks

Comments

MicroDK on :

How many times have you written on your blog that the next version of Steamhammer is stronger than the current one on SSCAIT? ;) Anyway, I always look forward to a stronger Steamhammer.

Jay Scott on :

It’s a refrain. Trying to measure progress is important. And I’m usually right, but not always!

Joseph S Huang on :

Don't retreat while a sunken is attacking.

Jay Scott on :

That’s part of it. Actual good play often does include retreating while in static defense range, but one step at a time.

Tully Elliston on :

Much greater commitment to attacks using melee units (eg. don't move into range then retreat then move into range then retreat) - if SH is uncertain of the odds, it shouldn't be diving; if it has already dived, it shouldn't be so fast to abort.

Jay Scott on :

In the long run, I will have a combat evaluator that comes up with smoothly varying estimates of the odds. For now, we are all stuck with combat simulators that give unstable results. I may look into doing some kind of smoothing or sensitivity analysis as a stopgap.

Jay Scott on :

But yes, even some hysteresis in the results would probably be a gain.

Dan on :

I've experimented a lot with hysteresis. It helps. But the band of instability in combat sim results far outranges any reasonable hysteresis thresholds. The whole simulation approach is a dead end -- the approach is inherently unstable and the squad-level granularity of fight/flight decisions is far too coarse. You can slather it in band-aids but you can't fix it without ripping out. We need something new.

Jay Scott on :

Maybe that should be my first hard problem to try to solve with machine learning, once I get it working for easier problems like a global evaluation.

McRave on :

Before diving into a ML approach, I think next-step is to move away from "my ball-of-units attacks your ball-of-units" style simulation. Dan's comment about the coarseness of squad level decisions is problematic for many bots.

Jay Scott on :

You and jtolmar both suggested so. It’s the same line of thinking I was following when I proposed separating operations from tactics. I think the two steps can be done in either order: 1. Structure squads into subsquads, perhaps with one level or perhaps hierarchically, by some kind of rule system. Then apply learning to control the subsquads, and perhaps to subsume the rules. That’s what I had been thinking of so far. 2. Apply a learning method whose output is a structure over the squad with instructions for what each part should do. I wonder if the more direct method 2 might not be better? There are times when complex outputs are easier to learn than simple outputs like “A is better than B.”

jtolmar on :

I think focus fire groups would be a better level of granularity to target. That is, all units targeting the same enemy make a decision based on whether they'll kill it and how many will die in the process.

A full squad grouping is still needed to get units into the right part of the map and to decide if it's time to go all in on an attack, but that choice can afford to be more timid if smaller groups are still accomplishing things.

McRave on :

I wouldn't be surprised if Dan is adding these features to test them versus CherryPi in the future. I'm on to him!

Dan on :

I've had hatchery blocking since AIIDE last year, though I caught and fixed a major bug in it a few months ago. It's certainly much better now.

Tully Elliston on :

I disagree with you guys, the combat sim is a wonder - better than any method we had for Zero-K AIs. It just needs to be used differently IMO.

If combat sim shows its worth attacking, that attack should be locked in for a period, and the threshold for retreating should be much lower than just "will we win?" Eg. SH may attack if the combat sim shows it will win with 10% remaining. If the combat sim then reports 2 seconds in actually it will lose, with opponent having 10% remaining, because marines killed some zergling on the way in - that is not a sufficient cause to retreat, even though it would not have been sufficient cause to attack. Moreover, with such tight odds SH should not be attacking at all. SH needs to be accepting greater or worse than 1:1 losses based on the macro situation, and it should be ready to trade forces to the death if it has resources and larva - zerg can rebuild fast.

Combat sim can provide the info needed, the info just needs to be interpreted correctly rather than treated as binary signal to dive or flee.

Jay Scott on :

Unfortunately it can be a wonder and still be not good enough. :-/

Dan on :

I've experimented extensively with hysteresis in PurpleWave. In the current version, it requires a tradeoff of >= ~0.5 (enemy losses divided by total losses) to pick a fight. Once the group changes decisions between fight/flight, future decisions receive a time-decaying hysteresis -- currently about .15 -- adjustment to that 0.5 in favor of continuing that decision.

It helps but doesn't fix the problem. There are no values that produce good behavior in all circumstances. Some fights need tiny hysteresis (zealots vs. zealots) and some need massive hysteresis (Big Protoss army vs. Big Terran army). And it's hard to know which.

But even with well-tuned parameters, the fluctuations in sim output are *still* of high enough variance (and inaccuracy) that this is insufficient for combat decisions approaching those of a decent human player. Pick any hysteresis threshold or duration, and you will see different flavors of stupidity.

There are multiple sources of instability in combat simulation. They largely boil down to the inability of simulations to accurately simulate collisions or to give units intelligent behaviors, both for performance reasons. So if you had 4 Zealots vs. 4 Zealots in nearly-symmetrical positions, in one simulation you might wind up with emergent focus-firing by one side, and in the next simulation by the other, producing radically different outcomes in consecutive frames for what's obviously an equal fight.

Beyond that, an even bigger problem with combat sim is that it fails to consider contextual factors. Maybe you're at 0.5 and ready to fight, but if you wait two seconds for more units to catch up, you'll be at 0.8.

Or say you're parked outside a Terran base loaded up with tanks that you have no hope of beating. A human thinks "Okay, I'll just sit outside until they move out". A bot thinks "LET'S RUN COMBAT SIM EVERY FRAME" and it only takes a few bad sim results over the 2,000+ you run in that time to momentarily decide to suicide your whole army.

Marian on :

My bot only uses quite simple heuristics with some power vector sum. Ground, air, damage, armor types, static defences are taken into account. While fighting I remember the starting power vector of enemy army and have some special constants(similar to what Dan has mentioned) that are applied for retreating, reinforcements...
Whole thing is very fast so can be ran many times per frame.
Obvious drawbacks are that positioning, shape of map, range advantage(for not to retreat) are not well accounted for.. although I have some ideas for improvement.
I have also created a bunch of UMS test maps where various combat features are tested like melee vs melee, melee vs ranged, melee + static vs attacker ... I can only recommend to use something like this.

Add Comment

E-Mail addresses will not be displayed and will only be used for E-Mail notifications.

To prevent automated Bots from commentspamming, please enter the string you see in the image below in the appropriate input box. Your comment will only be submitted if the strings match. Please ensure that your browser supports and accepts cookies, or your comment cannot be verified correctly.
CAPTCHA

Form options

Submitted comments will be subject to moderation before being displayed.