archive by month
Skip to content

SSCAIT scores - summary by bot

I’m looking into the scores recorded in the SSCAIT game data, which I have up to 27 September. So far I haven’t found anything too interesting, but it’s not entirely useless either.

According to the SSCAIT rules, a player’s score is the sum of units killed plus buildings razed: BWAPI::Player::getKillScore() + BWAPI::Player::getRazingScore().

Here’s basic score information for the dates between 17 August 2016 and 27 September 2016. It’s the same date range I used in the SSCAIT crosstables, chosen so as not to smear too many different bot versions into one table. The difference and ratio columns are all arithmetic means. They give the difference or the ratio between the winner’s and loser’s scores. (Well, the loss score ratio is the ratio between the loser’s and winner’s scores, to make it easier to compare by eye.) Games in which either side had a score of 0 (no kills) or -100 (crash) are left out.

botgameswin %mean
score
mean
win
score
mean
loss
score
win
score
diff
loss
score
diff
win
score
ratio
loss
score
ratio
krasi029886.91%54850556764936845776930615.583.06
Iron bot24280.17%23967248352045918644-769916.774.06
Marian Devecka21192.42%211202221377989442-295788.978.94
Martin Rooijackers26079.23%25077287221117022046-276869.1413.59
tscmooz24773.68%2032225134684712118-152039.8410.71
tscmoo27271.69%35035415911843523035-240155.496.53
LetaBot CIG 201625673.05%26635302291689622995-218109.034.24
WuliBot23465.81%8441862780846814-1805717.2212.69
Simon Prins22465.62%24256280581699821971-1239117.6812.18
ICELab24266.53%36286430992274528818-263506.4710.22
Sijia Xu23663.98%121191459877159913-2346015.798.28
LetaBot SSCAI 2015 Final23664.41%1940024725976418854-1742711.045.24
Dave Churchill23956.49%6656841243776394-1637017.4111.89
Chris Coxe17557.71%2989388617633468-379823.596.49
Tomas Vajda23064.78%34027396272372734048-1923436.525.11
Flash25164.14%114351387270758335-253057.789.89
Zia bot23651.69%130821764082058544-1551812.127.28
PeregrineBot13151.91%3560541515584651-658722.7712.65
tscmoop25852.33%156532312874488055-307189.9514.85
Andrew Smith25756.03%15896183911271712180-265088.847.42
Florian Richoux22353.36%147752280455888518-228056.3011.30
Carsten Nielsen27550.91%101471212181008140-2005711.686.89
Soeren Klett22845.18%39869573192549043732-129739.3211.69
Jakub Trancik20445.10%15552159651521211133290914.556.41
Tomas Cere28144.13%1831732027748818054-289587.1316.25
MegaBot18555.68%1666122464937216042-174739.2116.45
Aurelien Lermant28837.15%18580366697886-12822-226411.2512.13
Odin201413146.56%125521675788878998-1675716.2311.67
Gaoyuan Chen25839.53%114171685578618691-266987.1313.62
DAIDOES10127.72%1079423254601517054-1009312.4313.22
Igor Lacik10535.24%1318628378492014545-144837.667.40
Matej Istenik26029.62%17509326521113719086-115097.547.64
NUS Bot9632.29%70691288542958953-147548.7313.61
Roman Danielis24422.95%19086432681188320858-271513.8310.30
ZerGreenBot666.67%97621355421802291-80453.4410.87
Ian Nicholas DaCosta11616.38%40481035828126223-87957.5612.45
AwesomeBot11526.96%81001871241849280-168983.1120.40
Johan Kayser24917.67%1296535732807823303-1020710.8711.14
Martin Vlcak10231.37%1138620558719410658-1545912.3414.06
Rob Bogie5950.85%127771624891869555-2266213.034.92
Christoffer Artmann19717.26%771224928412115025-151944.9416.67
Marek Gajdos7211.11%518716426378212820-96715.8016.60
Travis Shelton10616.98%79421781659229374-94553.5117.10
Bjorn P Mattsson19018.95%49951473027197300-152373.9928.59
Vladimir Jurenka14228.17%95611533372978970-116915.607.47
neverdieTRX13714.60%849321777622213883-107834.5511.22
OpprimoBot21812.39%15672252581431717671-565420.1813.84
Sungguk Cha12225.41%1526334072885521740-114686.508.50
Jacob Knudsen9219.57%728519250437511994-121935.0818.43
HoangPhuc1291.67%151481631623009550-223453.1410.72
ButcherBoy156.67%1777565515003105-65432.2218.09

The most striking point is that Krasi0 was ahead in points, on average, in the games that it lost. So was Jakub Trancik’s cannon bot. The data that I have does not record the cause of losses. It’s perfectly possible to lose while ahead on points, when you fight efficiently and destroy masses of enemy stuff before dying. But you may also be ahead on points when you crash or overstep the time limit.

Score increases as the game goes on. I think that the score diff columns mostly tell us how long the games were. So, for example, Marian Devecka’s Killerbot often won short games and hung on through a long fight in lost games. The score ratio columns seem more informative about how far ahead the winner was at the end of the game. Killerbot tended to win with about the same point ratio that it lost with. Krasi0 won with a huge point ratio and lost with a small ratio, which might reflect its defensive style. Iron, which is super-aggressive, also won with a huge ratio and lost with a small ratio, which in its case might mean that it won after a long series of pinpricks or lost in a sudden collapse.

The numbers are not easy to interpret! But they must mean something.

Tomorrow: I’ll try to dig out something about the rate of failing to start up.

Trackbacks

No Trackbacks

Comments

krasi0 on :

Yeah, the explanation wrt to my low LSR seems plausible. The defensive style means that I didn't give up without a proper fight.

OTOH, the A Lermant case appears to be rather interesting: WSR 1.25, LSR 12.13. Does that mean that it's always on the verge of losing even when it wins?

Jay Scott on :

I’ve thought about it some more. Today my interpretation is that a high ratio between the winning score ratio and the losing score ratio (if anybody can make sense of that) means that the bot fights efficiently: It kills a lot, win or lose. A very low value of that ratio for Aurelien Lermant’s GarmBot means that GarmBot fights extremely inefficiently. And it’s easy to see that that is true: For most of the game it throws away units freely to distract the opponent! And a middling value for Killerbot also makes sense to me: Killerbot wins by tenacity, not by efficiency.

Jay Scott on :

So yes, GarmBot is usually on the verge of losing. Against opponents focused enough to find and kill its bases, undistracted by the random units flitting around, it DOES lose.

Jay Scott on :

To interpret the scores right and get out all the info that’s in them, I think we’d need more info. If we knew the game duration (not in the data that I have) and understood how the scores tend to evolve over a game, it would all make more sense.

Jay Scott on :

Hmm, after looking longer at the data, I wish I had used geometric mean for the score ratio columns instead of arithmetic mean, or something else that would give a fairer idea of the values. The values are mostly low but with some extremely high.

krasi0 on :

I remember reading somewhere in the past why the geometric mean might provide more insight than an arithmetic mean, but I can't remember the reason exactly ATM. Of course, I could google the answer, but if you have it off the top of your head, you might want to share it here for future readers' reference. :)

Jay Scott on :

In this case, because it gives less weight to the few outliers and better represents what the “typical” value is. On third thought, the harmonic mean makes even more sense, because this is the average of ratios.... If I were a statistician, I would remember to always look into the distribution of values before I tried to average them.

krasi0 on :

Thanks for the explanation! I hadn't even thought of the harmonic mean as an option...

Add Comment

E-Mail addresses will not be displayed and will only be used for E-Mail notifications.

To prevent automated Bots from commentspamming, please enter the string you see in the image below in the appropriate input box. Your comment will only be submitted if the strings match. Please ensure that your browser supports and accepts cookies, or your comment cannot be verified correctly.
CAPTCHA

Form options

Submitted comments will be subject to moderation before being displayed.