archive by month
Skip to content

BWAPI and information leaks

One of the goals of BWAPI is information fairness: Don’t provide information to a bot that a human player in the same situation would not have. It makes a serious attempt. BWAPI won’t tell you about an enemy unit that you do not have sight of. But it also leaks information through many channels. BWAPI is an unofficial add-on, and the game was not designed to include a programmatic interface. I think it’s difficult to design an interface that provides everything necessary and nothing extra.

Here are some of the leaks that stand out to me.

Screen visibility. A human player sees what is in the minimap plus what is visible on the screen, which shows only a small part of the whole map. A bot does not look through a peephole, but sees everything visible at once. It makes bots easier to code, so it is a correct tradeoff for early days.

Cloaked units. A bot cannot overlook the presence of a cloaked unit in its vision. By design, humans can easily miss seeing the blur, either by not putting the screen over it or by missing the subtle visual cue—even pros sometimes lose all their workers at a base before realizing that a dark templar has sneaked in. I don’t know how to implement this in a way that is 100% equal between humans and bots, because bots do not have the complicated perceptual apparatus. But something could be rigged up to make it approximately fair.

The same goes for other events that are easy to miss: Nuke targets marked by a tiny red dot, scans marked by an conspicuous but fleeting swirl of sparkles.

Unit IDs. BWAPI labels enemy units so you can tell them apart. A human player sees a mass of zerglings, a bot sees a mass of zerglings each of which has a name. A human may be able to distinguish units in certain cases, by hit points or because the unit shows a visible effect like ensnare, but in general, one unit of a given type looks like any other. A non-leaking design would give fixed IDs to your own units, and for the enemy provide a bag of units distinguishable only by their properties, with temporary IDs and/or unit pointers that remain usable only while the unit remains in view. The change would add complexity.

In practice, bots use the extra information to draw inferences that a human player cannot draw. “I saw these units standing guard over that entrance—now I see them here. They are no longer guarding the entrance.” A human player might guess that, the bot knows for sure.

Unit orders and targets. Mutalisks sweep in and notice an SCV scurrying across the terran base. Where is the SCV going? Will it construct or repair, or is it only moving? A human player has to see the SCV start to build or start to repair to be sure; a bot knows the SCV’s orders immediately. Steamhammer (like many others, no doubt) exploits the information to target SCVs that have important tasks.

When a unit wanders past in the middle of the map, a human will guess (or at least wonder) where it may be going. A bot knows where it has been ordered to go—though that may not be its ultimate destination. In a fight, a human can judge which enemy unit is targeting which friendly unit when the enemy fires, or sometimes as the enemy prepares to fire. A bot knows as soon as the order is given, or as soon as the enemy comes into sight range.

Lesson for bot authors: Tell your units to move to their destinations before handing down their orders about what to do there. You give away less information to enemy bots. Also, make long distance moves with intermediate waypoints, so that the enemy doesn’t know your ultimate destination (though mineral walking has to be an exception). Alternately, you can actively deceive the enemy by ordering a destination where the initial path is the same as the path you want, then changing the destination after that.

Information fairness is of course only one form of fairness. A computer with a programmatic connection to a computer game enjoys direct perception that humans can never match; it is intrinsically unfair in other ways too. You can get perfect fairness of a certain kind if the computer has to use a camera and robot hands, interacting with the game as intended by the game designers. But that’s no scene for students and hobbyists. I think that BWAPI is a good compromise for now between being fair and making bot coding easy. As bots become stronger, the fairness issues will loom larger—and bots are getting stronger fast. We ought to be thinking about it.

Do you have a favorite information leak that I didn’t mention?

Trackbacks

No Trackbacks

Comments

Dan on :

There are some that go in the reverse direction. Humans can hear Siege Tanks sieging in fog; bots can't. There are a variety of animations which reveal unit state more precisely than orders do.

There's also information from outside the game which humans incorporate but to which bots lack access in current tournament settings, such as your record, your opponent's record, and what they've done in games against other opponents. A bot can't tell whether or not their opponent knows their race or whether they entered a game as random.

Ultimately unless you've got a bot that plays by firing muscles in a human body holding a keyboard and mouse -- and perhaps even then -- there will be arguments to be made about fairness. No arrangement will satisfy everyone.

Jay Scott on :

All good points. The game was not written with a programmatic interface in mind.

IlIllIllIIlIlIIIll on :

Another fun one: based on ID detecting if the enemy skips building workers (e.g. 4/5pool) at the start of the game by checking the ID increments of your own workers.
(At least I think this is possible, but if it is, not ethical :))

Jay Scott on :

I have wondered whether that works, but never checked it out (since I’m definitely not implementing it). Does anybody know?

Ideally, unit IDs should be randomly generated. And they sure don’t look that way.

IIlIlIIllIlllIlI on :

A short testscript gives these results (3 SCV's created after the game start):

Type | ID |
Terran_SCV | 154 |
Terran_SCV | 156 |
Terran_SCV | 158 |

Confirming the potential exploit.

From the unit ID's one can also infer how many unit the opponent has at the start (e.g. inferring if the opponent is Zerg or not if it spawned as Random).

Bwapi5 should randomize units ID's for sure

lIllIIlIlIlIIlIIl on :

I forgot to add that if the opponents bot does nothing (not building drones, we get)

Type | ID |
Terran_SCV | 154 |
Terran_SCV | 155 |
Terran_SCV | 156 |

Jay Scott on :

It looks pretty damning. :-(

Jay Scott on :

Feed all available information to a machine learning system, and the system will learn to use all the information. These systems are generally opaque, so even with a code inspection it won’t be easy to see that the bot has learned how to cheat. The author might not know, or might be aware and have the excuse: “All I did was pass along what BWAPI gave me.” We have no way to tell. :-(

Jay Scott on :

I have verified that the unit IDs leak information, and that a bot can use the information to judge the enemy build. It is easy to recognize a 4 pool without scouting. I opened an issue with BWAPI.

https://github.com/bwapi/bwapi/issues/799

Marian on :

Interesting idea, if this works we need to think of reporting this to Adam as it is a serious exploit. I would preferably not like to lose id matching. But maybe if we won't have it will drive the community to develop opponent planning model along with unit distributions.

Antiga / Iruian on :

It's a multi layered issue imo, not just with BWAPI but with comparing humans and bots. . 1. Making sure humans / bots both play on LF6 or lan lat in order to have a steady competitive ping target. 2. Concerns over bot income levels with mineral locking 3. concerns over bot command spam impacting the opponent (I expect at some point we will see a 20k-30K bot apm limit to prevent this).

Jay Scott on :

Yes, at some point we likely want command rate limits and reaction time floors so that instant reactions don't seem unfair. It may be that latency makes a good floor. I notice that OpenAI set their reaction time floor to 200 ms in their recent DOTA matches against top players. 200 ms is similar to the fastest human reactions to new information.

MicroDK on :

Is the purpose that bots only should have access to the same information as Humans? It is not that bots can use keyboard and mouse. Bots are a whole different thing and it can not be compared exactly to humans. But I agree the most severe information leaks should be fixed: IDs of units and unit orders. Randomize the IDs that units gets and do not provide unit orders of enemy units. Maybe also bullet information and cloaked units should be changed. Make bullets and cloaked units appear randomly to the opponent so it is much more difficult to track... just like human vision.

krasi0 on :

There have already been plenty of discussions on the same topic and the conclusions have always been the following: the current trade offs with BWAPI are still the best we could hope for for now.
The only new finding here is the Zerg case where IDs leak information which could be addressed by Unit IDs randomizations (while still maintaining uniqueness). For now, we could add a rule to each tournament that bans such uses of Unit IDs.

Add Comment

E-Mail addresses will not be displayed and will only be used for E-Mail notifications.

To prevent automated Bots from commentspamming, please enter the string you see in the image below in the appropriate input box. Your comment will only be submitted if the strings match. Please ensure that your browser supports and accepts cookies, or your comment cannot be verified correctly.
CAPTCHA

Form options

Submitted comments will be subject to moderation before being displayed.