archive by month
Skip to content

AIIDE 2017 the learning bots

In March 2016 I analyzed which bots learned during the AIIDE 2015 tournament by looking at the data files. Here’s a similar analysis for AIIDE 2017.

I looked at the “write” directory for each bot, to see if it wrote files there, and if so, what the files looked like. Writing data doesn’t mean that the bot actually learned anything—it may not have used the data. Bots not listed in the table did not write anything interesting (maybe log files, nothing more). The table includes 15 bots of the 28 entrants, over half of them.

#botinfo
1ZZZKBotvaried info about each game, including tidbits like the time zone and the processor it was played on
2PurpleWavefor each opponent, a log of games with info including the sequence of strategies followed
5Microwavesame format as UAlbertaBot (Microwave has more strategies)
6CherryPiopening data for each opponent
9Tyrfor each opponent, seems to save info only about the previous game: win or loss, and a flag or two like "FLYERS" or "CANNONS"
11AILien10 values per opponent: scores for zerg unit types, a few odds and ends like macroHeavyness and supplyISawAir
12LetaBotone file which records a log of games, with opponent, race, map, and 3 numbers per game
14UAlbertaBota file for each opponent, giving for each strategy the count of wins and losses; learning was turned on this year
15Aiur91 values per opponent: strategy x map size
17Skyneta file for each opponent, with lines like "build_0_2 14 12"
19MegaBotmany files; the important ones seem to be "MegaBot-vs-[bot name].xml" which give scores for each bot MegaBot can imitate: Skynet, NUSBot, Xelnaga
20Xelnagaa file for each opponent with a single number: -1, 0, 2, or 3
21Overkillmany files with neural network data, reinforcement learning data, and opening learning data for each opponent (more than I thought!)
24Myscbotsame format as UAlbertaBot, but only 1 strategy was played for each opponent; nothing was learned
25HannesBredberg2 numbers per opponent, approximately (not exactly) the win and loss counts

LetaBot seems worth looking into, to see whether its log is learning data and, if so, how it is used. PurpleWave also recorded data essentially as a log, which could be used for a wide range of purposes. And AILien has a unique learning method that I should spend some time on.

UAlbertaBot had learning turned on this year. It has sometimes left learning off because its default strategies were dominant. It’s also notable that Ziabot skipped learning this year. In the past it has learned. Ziabot also finished last.

Next: What AIUR learned.

AIIDE 2017 crosstables per map

The AIIDE 2017 official results include a table of how each bot performed on each map. Some bots performed evenly across maps, and some liked or disliked particular maps. The left “overall” column of each table corresponds to one column in the official bot-map table; the rest of the table breaks it down by opponent. With 10 maps and 110 rounds, each cell ideally represents 11 games, so the tables are noisy. Some games are missing, so some cells represent fewer games.

First, the complete crosstable for reference. Its contents are the same as the official crosstable, but shown as percentages rather than as wins/games.

complete

overallZZZKPurpIroncpacMicrCherMcRaArraTyrSteaAILiLetaXimpUAlbAiurIceBSkynKillMegaXelnOverJunoGarmMyscHannSlinForcZiab
ZZZKBot83.11%67%60%66%87%89%42%85%37%96%83%45%96%99%86%97%96%96%92%100%100%99%100%72%100%52%100%100%
PurpleWave82.35%33%65%93%88%78%75%95%81%86%55%61%91%92%49%93%94%56%91%90%98%75%100%94%98%95%100%99%
Iron81.52%40%35%67%68%7%42%94%96%87%100%63%99%84%98%100%100%61%93%85%98%99%97%98%100%99%97%91%
cpac71.01%34%7%33%95%55%99%36%5%70%85%77%99%99%58%97%67%83%82%64%83%80%100%69%85%83%74%100%
Microwave70.86%13%12%32%5%72%28%67%49%55%47%84%58%83%92%92%85%99%91%83%96%77%100%95%100%99%98%100%
CherryPi69.08%11%22%93%45%28%54%29%89%35%42%89%50%53%92%92%50%66%67%98%93%100%99%79%95%99%97%100%
McRave67.07%58%25%58%1%72%46%83%63%77%70%51%67%55%36%76%64%75%67%77%50%85%96%100%96%67%95%98%
Arrakhammer65.95%15%5%6%64%33%71%17%93%27%68%60%75%59%62%94%68%99%64%47%96%100%100%64%98%96%99%100%
Tyr65.91%63%19%4%95%51%11%37%7%84%60%55%96%65%58%65%81%90%80%76%89%95%95%85%86%64%95%75%
Steamhammer64.14%4%14%13%30%45%65%23%73%16%89%49%86%80%35%89%54%93%55%72%95%99%100%85%76%100%92%100%
AILien58.29%17%45%0%15%53%58%30%32%40%11%13%40%39%55%82%84%87%74%87%76%60%99%92%95%97%94%99%
LetaBot56.92%55%39%37%23%16%11%49%40%45%51%87%68%45%57%5%77%87%72%44%78%56%80%98%47%98%85%87%
Ximp54.19%4%9%1%1%42%50%33%25%4%14%60%32%51%88%10%99%65%86%75%59%86%99%86%93%100%95%99%
UAlbertaBot53.40%1%8%16%1%17%47%45%41%35%20%61%55%49%75%74%67%54%65%100%58%91%59%61%87%65%89%100%
Aiur50.46%14%51%2%42%8%8%64%38%42%65%45%43%12%25%81%89%62%33%56%89%22%74%70%92%75%86%75%
IceBot45.62%3%7%0%3%8%8%24%6%35%11%18%95%90%26%19%62%75%53%35%62%85%68%87%97%98%67%88%
Skynet43.78%4%6%0%33%15%50%36%32%19%46%16%23%1%33%11%38%28%17%67%92%70%99%55%100%96%95%96%
KillAll43.04%4%44%39%17%1%34%25%1%10%7%13%13%35%46%38%25%72%77%89%45%100%89%64%78%44%61%94%
MegaBot42.83%8%9%7%18%9%33%33%36%20%45%26%28%14%35%67%47%83%23%37%75%44%75%66%92%71%86%77%
Xelnaga37.10%0%10%15%36%17%2%23%53%24%28%13%56%25%0%44%65%33%11%63%87%45%1%75%80%75%80%45%
Overkill32.69%0%2%2%17%4%7%50%4%11%5%24%22%41%42%11%38%8%55%25%13%75%67%66%36%87%83%87%
Juno29.57%1%25%1%20%23%0%15%0%5%1%40%44%14%9%78%15%30%0%56%55%25%4%42%95%68%50%85%
GarmBot27.09%0%0%3%0%0%1%4%0%5%0%1%20%1%41%26%32%1%11%25%99%33%96%79%6%97%77%73%
Myscbot25.94%28%6%2%31%5%21%0%36%15%15%8%2%14%39%30%13%45%36%34%25%34%58%21%61%9%68%44%
HannesBredberg21.26%0%2%0%15%0%5%4%2%14%24%5%53%7%13%8%3%0%22%8%20%64%5%94%39%75%66%29%
Sling21.09%48%5%1%17%1%1%33%4%36%0%3%2%0%35%25%2%4%56%29%25%13%32%3%91%25%71%9%
ForceBot17.97%0%0%3%26%2%3%5%1%5%8%6%15%5%11%14%33%5%39%14%20%17%50%23%32%34%29%85%
Ziabot17.21%0%1%9%0%0%0%2%0%25%0%1%13%1%0%25%12%4%6%23%55%13%15%27%56%71%91%15%

(2) Benzene

overallZZZKPurpIroncpacMicrCherMcRaArraTyrSteaAILiLetaXimpUAlbAiurIceBSkynKillMegaXelnOverJunoGarmMyscHannSlinForcZiab
ZZZKBot74.07%36%9%55%91%82%27%82%9%100%91%0%100%100%91%100%91%100%91%100%100%100%100%18%100%27%100%100%
PurpleWave84.85%64%64%100%100%73%82%91%82%100%82%73%82%100%18%91%73%73%100%100%100%82%100%100%91%82%100%91%
Iron85.86%91%36%82%82%9%45%100%100%100%100%36%91%100%100%100%100%100%82%73%100%100%100%100%100%100%100%91%
cpac71.28%45%0%18%91%82%100%36%0%55%100%73%100%100%45%100%82%91%90%45%64%91%100%27%100%91%100%100%
Microwave72.30%9%0%18%9%82%18%82%55%27%82%82%64%82%73%91%100%100%100%91%100%100%100%91%100%100%100%100%
CherryPi66.67%18%27%91%18%18%45%18%82%36%36%91%64%64%100%73%73%73%64%100%91%100%91%55%100%91%82%100%
McRave72.05%73%18%55%0%82%55%91%55%82%91%55%82%73%36%64%82%100%82%100%36%100%100%100%100%36%100%100%
Arrakhammer63.85%18%9%0%64%18%82%9%100%36%91%73%64%36%18%100%73%100%70%64%100%100%100%18%91%91%100%100%
Tyr66.67%91%18%0%100%45%18%45%0%100%64%36%100%73%27%55%91%100%73%73%100%100%91%100%73%55%100%73%
Steamhammer62.63%0%0%0%45%73%64%18%64%0%100%45%91%82%36%73%45%82%45%64%100%100%100%64%100%100%100%100%
AILien48.48%9%18%0%0%18%64%9%9%36%0%0%18%45%18%73%91%55%82%82%73%45%100%100%82%91%100%91%
LetaBot60.68%100%27%64%27%18%9%45%27%64%55%100%27%27%55%0%91%82%100%55%100%18%100%100%82%100%82%91%
Ximp46.60%0%18%9%0%36%36%18%36%0%9%82%73%55%0%9%100%73%88%55%36%0%100%73%64%100%100%100%
UAlbertaBot53.54%0%0%0%0%18%36%27%64%27%18%55%73%45%73%73%55%82%73%100%55%100%73%55%91%73%82%100%
Aiur60.27%9%82%0%55%27%0%64%82%73%64%82%45%100%27%91%91%55%55%82%91%18%73%36%91%91%82%64%
IceBot49.32%0%9%0%0%9%27%36%0%45%27%27%100%91%27%9%82%82%80%27%55%100%73%82%82%100%91%73%
Skynet38.31%9%27%0%18%0%27%18%27%9%55%9%9%0%45%9%18%0%0%82%100%91%91%9%100%100%91%82%
KillAll35.69%0%27%0%9%0%27%0%0%0%18%45%18%27%18%45%18%100%82%100%36%100%73%0%73%18%55%73%
MegaBot35.02%9%0%18%10%0%36%18%30%27%55%18%0%12%27%45%20%100%18%11%45%45%75%18%100%55%88%80%
Xelnaga31.19%0%0%27%55%9%0%0%36%27%36%18%45%45%0%18%73%18%0%89%82%45%0%55%73%36%64%0%
Overkill33.67%0%0%0%36%0%9%64%0%0%0%27%0%64%45%9%45%0%64%55%18%82%73%64%0%91%100%64%
Juno29.63%0%18%0%9%0%0%0%0%0%0%55%82%100%0%82%0%9%0%55%55%18%0%0%100%73%64%82%
GarmBot26.53%0%0%0%0%0%9%0%0%9%0%0%0%0%27%27%27%9%27%25%100%27%100%64%0%100%100%64%
Myscbot46.46%82%0%0%73%9%45%0%82%0%36%0%0%27%45%64%18%91%100%82%45%36%100%36%73%9%100%100%
HannesBredberg24.24%0%9%0%0%0%0%0%9%27%0%18%18%36%9%9%18%0%27%0%27%100%0%100%27%100%100%18%
Sling25.25%73%18%0%9%0%9%64%9%45%0%9%0%0%27%9%0%0%82%45%64%9%27%0%91%0%91%0%
ForceBot8.50%0%0%0%0%0%18%0%0%0%0%0%18%0%18%18%9%9%45%12%36%0%36%0%0%0%9%0%
Ziabot24.66%0%9%9%0%0%0%0%0%27%0%9%9%0%0%36%27%18%27%20%100%36%18%36%0%82%100%100%

(2) Destination

overallZZZKPurpIroncpacMicrCherMcRaArraTyrSteaAILiLetaXimpUAlbAiurIceBSkynKillMegaXelnOverJunoGarmMyscHannSlinForcZiab
ZZZKBot82.83%36%9%73%100%82%27%91%55%100%91%27%100%100%100%100%100%100%82%100%100%100%100%91%100%73%100%100%
PurpleWave78.38%64%64%91%55%73%91%91%91%73%27%36%91%73%36%100%100%91%80%91%82%55%100%91%100%73%100%100%
Iron84.51%91%36%55%64%0%27%100%91%100%100%55%100%100%100%100%100%100%100%82%100%100%100%100%100%100%100%82%
cpac73.90%27%9%45%100%82%100%55%18%91%82%100%100%100%82%91%27%82%89%82%64%82%100%18%100%100%73%100%
Microwave74.32%0%45%36%0%100%73%73%82%64%82%100%27%73%100%82%73%100%100%55%91%91%100%82%100%91%91%100%
CherryPi62.96%18%27%100%18%0%55%0%82%55%9%73%55%45%91%100%36%64%64%100%82%100%100%36%91%100%100%100%
McRave62.63%73%9%73%0%27%45%73%91%73%64%55%45%64%9%64%73%45%64%73%27%100%100%100%100%45%100%100%
Arrakhammer63.64%9%9%0%45%27%100%27%100%27%73%18%73%45%64%82%82%100%100%45%73%100%100%36%100%91%91%100%
Tyr52.36%45%9%9%82%18%18%9%0%36%45%27%64%55%45%64%73%64%70%73%82%91%91%100%100%27%45%73%
Steamhammer66.33%0%27%0%9%36%45%27%73%64%100%55%100%73%36%91%55%100%64%91%100%91%100%73%91%100%91%100%
AILien60.27%9%73%0%18%18%91%36%27%55%0%18%55%27%82%82%82%91%64%82%64%91%100%82%100%100%82%100%
LetaBot65.31%73%64%45%0%0%27%45%82%73%45%82%64%64%82%0%100%91%91%45%100%100%91%100%18%100%91%100%
Ximp59.26%0%9%0%0%73%45%55%27%36%0%45%36%73%100%18%100%73%100%91%27%100%100%100%100%100%91%100%
UAlbertaBot53.54%0%27%0%0%27%55%36%55%45%27%73%36%27%64%64%64%36%64%100%45%91%82%64%91%82%91%100%
Aiur41.16%0%64%0%18%0%9%91%36%55%64%18%18%0%36%73%91%64%62%100%55%9%18%27%82%0%91%36%
IceBot52.88%0%0%0%9%18%0%36%18%36%9%18%100%82%36%27%91%73%73%73%67%100%100%100%100%100%73%91%
Skynet39.39%0%0%0%73%27%64%27%18%27%45%18%0%0%36%9%9%36%18%73%82%45%100%9%100%73%82%91%
KillAll41.55%0%9%0%18%0%36%55%0%36%0%9%9%27%64%36%27%64%90%100%45%100%82%45%91%27%64%91%
MegaBot38.03%18%20%0%11%0%36%36%0%30%36%36%9%0%36%38%27%82%10%18%82%60%70%55%70%55%100%90%
Xelnaga34.80%0%9%18%18%45%0%27%55%27%9%18%55%9%0%0%27%27%0%82%100%91%0%64%100%64%82%18%
Overkill36.90%0%18%0%36%9%18%73%27%18%0%36%0%73%55%45%33%18%55%18%0%64%50%45%36%82%82%91%
Juno23.31%0%45%0%18%9%0%0%0%9%9%9%0%0%9%91%0%55%0%40%9%36%0%27%100%55%9%100%
GarmBot24.41%0%0%0%0%0%0%0%0%9%0%0%9%0%18%82%0%0%18%30%100%50%100%9%27%100%91%18%
Myscbot41.41%9%9%0%82%18%64%0%64%0%27%18%0%0%36%73%0%91%55%45%36%55%73%91%73%0%100%100%
HannesBredberg18.24%0%0%0%0%0%9%0%0%0%9%0%82%0%9%18%0%0%9%30%0%64%0%73%27%64%64%36%
Sling29.29%27%27%0%0%9%0%55%9%73%0%0%0%0%18%100%0%27%73%45%36%18%45%0%100%36%82%9%
ForceBot19.87%0%0%0%27%9%0%0%9%55%9%18%9%9%9%9%27%18%36%0%18%18%91%9%0%36%18%100%
Ziabot17.57%0%0%18%0%0%0%0%0%27%0%0%0%0%0%64%9%9%9%10%82%9%0%82%0%64%91%0%

(2) Heartbreak Ridge

overallZZZKPurpIroncpacMicrCherMcRaArraTyrSteaAILiLetaXimpUAlbAiurIceBSkynKillMegaXelnOverJunoGarmMyscHannSlinForcZiab
ZZZKBot81.82%91%36%64%82%91%9%91%9%100%64%36%100%100%100%100%100%100%100%100%100%100%100%91%100%45%100%100%
PurpleWave85.52%9%45%100%91%91%82%91%91%100%64%73%100%100%55%91%91%82%82%91%100%82%100%100%100%100%100%100%
Iron84.18%64%55%64%55%18%36%100%100%82%100%36%100%100%100%100%100%100%91%91%100%100%100%100%100%100%100%82%
cpac72.73%36%0%36%91%36%100%18%9%73%73%73%100%100%82%100%73%82%73%73%91%91%100%91%100%73%91%100%
Microwave71.62%18%9%45%9%82%36%82%55%55%36%73%55%82%91%100%73%100%90%64%100%82%100%100%100%100%100%100%
CherryPi70.71%9%9%82%64%18%64%36%91%18%55%100%27%55%100%91%55%73%82%100%100%100%100%91%91%100%100%100%
McRave66.67%91%18%64%0%64%36%73%55%100%91%55%73%0%9%91%64%55%73%45%45%100%100%100%100%100%100%100%
Arrakhammer68.01%9%9%0%82%18%64%27%100%9%64%91%82%82%55%82%73%100%73%64%100%100%100%73%100%82%100%100%
Tyr68.92%91%9%0%91%45%9%45%0%91%64%27%100%64%0%91%82%100%100%82%91%100%100%91%100%100%100%91%
Steamhammer64.65%0%0%18%27%45%82%0%91%9%91%45%64%91%27%82%73%91%55%73%100%100%100%100%82%100%100%100%
AILien56.90%36%36%0%27%64%45%9%36%36%9%9%9%27%64%64%100%82%82%64%91%55%100%100%91%100%100%100%
LetaBot55.44%64%27%64%27%27%0%45%9%73%55%91%82%18%18%9%100%64%50%18%91%18%82%100%64%100%100%100%
Ximp60.88%0%0%0%0%45%73%27%18%0%36%91%18%91%100%36%100%100%89%91%36%91%100%100%100%100%100%100%
UAlbertaBot54.05%0%0%0%0%18%45%100%18%36%9%73%82%9%64%64%82%55%70%100%64%100%73%73%91%45%91%100%
Aiur60.94%0%45%0%18%9%0%91%45%100%73%36%82%0%36%91%100%91%82%73%100%9%100%91%100%82%100%91%
IceBot45.73%0%9%0%0%0%9%9%18%9%18%36%91%64%36%9%91%91%43%0%55%91%100%100%100%100%64%91%
Skynet43.00%0%9%0%27%27%45%36%27%18%27%0%0%0%18%0%9%36%43%64%100%91%100%91%100%91%100%100%
KillAll39.53%0%18%0%18%0%27%45%0%0%9%18%36%0%45%9%9%64%80%100%55%100%91%91%64%36%55%100%
MegaBot34.83%0%18%9%27%10%18%27%27%0%45%18%50%11%30%18%57%57%20%43%91%40%75%78%73%90%50%0%
Xelnaga38.91%0%9%9%27%36%0%55%36%18%27%36%82%9%0%27%100%36%0%57%100%36%0%82%91%100%45%36%
Overkill29.39%0%0%0%9%0%0%55%0%9%0%9%9%64%36%0%45%0%45%9%0%82%91%55%9%91%80%100%
Juno29.05%0%18%0%9%18%0%0%0%0%0%45%82%9%0%91%9%9%0%60%64%18%0%36%100%73%45%100%
GarmBot23.81%0%0%0%0%0%0%0%0%0%0%0%18%0%27%0%0%0%9%25%100%9%100%100%0%82%91%82%
Myscbot16.27%9%0%0%9%0%9%0%27%9%0%0%0%0%27%9%0%9%9%22%18%45%64%0%36%0%64%73%
HannesBredberg21.55%0%0%0%0%0%9%0%0%0%18%9%36%0%9%0%0%0%36%27%9%91%0%100%64%64%55%55%
Sling18.24%55%0%0%27%0%0%0%18%0%0%0%0%0%55%18%0%9%64%10%0%9%27%18%100%36%45%0%
ForceBot19.11%0%0%0%9%0%0%0%0%0%0%0%0%0%9%0%36%0%45%50%55%20%55%9%36%45%55%100%
Ziabot14.81%0%0%18%0%0%0%0%0%9%0%0%0%0%0%9%9%0%0%100%64%0%0%18%27%45%100%0%

(3) Aztec

overallZZZKPurpIroncpacMicrCherMcRaArraTyrSteaAILiLetaXimpUAlbAiurIceBSkynKillMegaXelnOverJunoGarmMyscHannSlinForcZiab
ZZZKBot84.12%91%91%55%100%91%9%91%0%100%91%73%91%91%82%100%100%100%90%100%100%100%100%91%100%36%100%100%
PurpleWave82.15%9%55%91%91%82%73%100%73%100%73%73%91%82%55%100%91%45%91%73%100%82%100%91%100%100%100%100%
Iron77.36%9%45%73%55%0%27%82%100%100%100%27%100%82%91%100%100%55%90%82%100%100%100%91%100%100%100%82%
cpac73.06%45%9%27%100%64%100%45%0%55%82%82%100%100%73%100%73%91%82%55%82%82%100%82%73%91%82%100%
Microwave70.03%0%9%45%0%91%27%73%27%45%27%82%27%91%91%100%91%91%100%82%100%91%100%100%100%100%100%100%
CherryPi70.27%9%18%100%36%9%36%36%91%64%45%91%64%45%91%90%45%55%91%100%100%100%100%91%91%100%100%100%
McRave70.51%91%27%73%0%73%64%91%73%91%91%55%55%64%36%73%45%73%89%82%45%91%82%100%100%45%100%100%
Arrakhammer65.54%9%0%18%55%27%64%9%100%0%55%64%82%64%73%100%64%100%70%36%100%100%100%91%91%100%100%100%
Tyr66.55%100%27%0%100%73%9%27%0%91%45%64%100%64%73%55%91%100%80%100%91%91%91%64%73%45%100%45%
Steamhammer65.08%0%0%0%45%55%36%9%100%9%91%45%100%82%27%100%73%100%56%82%82%100%100%91%73%100%100%100%
AILien60.34%9%27%0%18%73%55%9%45%55%9%0%64%36%36%73%91%100%67%100%91%82%91%100%100%100%100%100%
LetaBot57.43%27%27%73%18%18%9%45%36%36%55%100%82%45%55%0%100%91%70%9%91%73%73%100%64%91%64%100%
Ximp51.85%9%9%0%0%73%36%45%18%0%0%36%18%27%91%0%100%73%91%64%55%100%100%73%91%100%91%100%
UAlbertaBot54.88%9%18%18%0%9%55%36%36%36%18%64%55%73%91%64%64%82%55%100%73%91%64%36%91%55%91%100%
Aiur47.47%18%45%9%27%9%9%64%27%27%73%64%45%9%9%55%91%27%18%64%91%27%73%82%100%82%64%73%
IceBot43.05%0%0%0%0%0%10%27%0%45%0%27%100%100%36%45%91%64%40%45%27%100%9%73%100%100%18%100%
Skynet42.03%0%9%0%27%9%55%55%36%9%27%9%0%0%36%9%9%0%0%91%91%82%100%82%100%100%91%100%
KillAll42.76%0%55%45%9%9%45%27%0%0%0%0%9%27%18%73%36%100%82%45%36%100%82%64%100%45%55%91%
MegaBot44.48%10%9%10%18%0%9%11%30%20%44%33%30%9%45%82%60%100%18%64%80%55%91%64%100%73%82%55%
Xelnaga38.05%0%27%18%45%18%0%18%64%0%18%0%91%36%0%36%55%9%55%36%100%27%0%91%82%91%73%36%
Overkill33.78%0%0%0%18%0%0%55%0%9%18%9%9%45%27%9%73%9%64%20%0%91%82%73%36%91%82%91%
Juno25.93%0%18%0%18%9%0%9%0%9%0%18%27%0%9%73%0%18%0%45%73%9%9%73%100%36%45%100%
GarmBot31.65%0%0%0%0%0%0%18%0%9%0%9%27%0%36%27%91%0%18%9%100%18%91%100%18%100%91%91%
Myscbot20.88%9%9%9%18%0%9%0%9%36%9%0%0%27%64%18%27%18%36%36%9%27%27%0%73%9%55%27%
HannesBredberg17.23%0%0%0%27%0%9%0%9%27%27%0%36%9%9%0%0%0%0%0%18%64%0%82%27%73%27%18%
Sling22.90%64%0%0%9%0%0%55%0%55%0%0%9%0%45%18%0%0%55%27%9%9%64%0%91%27%82%0%
ForceBot22.22%0%0%0%18%0%0%0%0%0%0%0%36%9%9%36%82%9%45%18%27%18%55%9%45%73%18%91%
Ziabot18.52%0%0%18%0%0%0%0%0%55%0%0%0%0%0%27%0%0%9%45%64%9%0%9%73%82%100%9%

(3) Tau Cross

overallZZZKPurpIroncpacMicrCherMcRaArraTyrSteaAILiLetaXimpUAlbAiurIceBSkynKillMegaXelnOverJunoGarmMyscHannSlinForcZiab
ZZZKBot83.84%82%91%73%82%91%0%82%73%82%91%9%100%100%73%100%91%82%91%100%100%100%100%73%100%100%100%100%
PurpleWave80.07%18%64%73%82%73%64%100%36%73%82%27%91%91%91%82%100%45%90%100%100%91%100%91%100%100%100%100%
Iron83.16%9%36%100%64%18%45%100%100%100%100%91%100%73%100%100%100%36%91%100%100%100%100%100%100%100%82%100%
cpac67.00%27%27%0%91%82%100%36%0%73%82%64%91%100%18%100%27%73%73%45%91%82%100%91%73%82%82%100%
Microwave70.03%18%18%36%9%82%9%36%45%82%55%82%55%82%91%91%82%100%82%82%100%73%100%82%100%100%100%100%
CherryPi67.34%9%27%82%18%18%55%36%100%9%55%100%45%45%91%91%27%73%64%100%100%100%100%82%91%100%100%100%
McRave71.62%100%36%55%0%91%45%100%45%91%55%36%36%82%55%73%55%91%80%73%55%91%100%100%100%91%100%100%
Arrakhammer64.86%18%0%0%64%64%64%0%100%55%82%64%55%64%45%100%91%100%30%27%100%100%100%27%100%100%100%100%
Tyr69.59%27%64%0%100%55%0%55%0%91%55%55%100%55%91%55%82%64%80%100%91%91%100%100%100%82%100%91%
Steamhammer62.16%18%27%0%27%18%91%9%45%9%100%36%82%55%36%82%45%82%60%73%100%100%100%100%91%100%91%100%
AILien54.21%9%18%0%18%45%45%45%18%45%0%0%27%45%27%91%82%82%73%91%91%36%100%100%91%91%91%100%
LetaBot62.29%91%73%9%36%18%0%64%36%45%64%100%73%55%55%9%100%91%45%64%91%64%91%100%27%100%82%100%
Ximp57.77%0%9%0%9%45%55%64%45%0%18%73%27%36%100%18%100%64%80%73%55%91%100%100%100%100%100%100%
UAlbertaBot57.58%0%9%27%0%18%55%18%36%45%45%55%45%64%100%55%55%82%82%100%73%100%55%91%82%73%91%100%
Aiur51.18%27%9%0%82%9%9%45%55%9%64%73%45%0%0%73%91%36%36%73%91%36%82%91%91%100%73%82%
IceBot46.28%0%18%0%0%9%9%27%0%45%18%9%91%82%45%27%100%73%10%36%91%73%64%91%100%100%27%100%
Skynet45.79%9%0%0%73%18%73%45%9%18%55%18%0%0%45%9%0%82%9%55%100%91%100%45%100%100%91%91%
KillAll45.79%18%55%64%27%0%27%9%0%36%18%18%9%36%18%64%27%18%82%73%73%100%100%64%55%64%82%100%
MegaBot49.30%9%10%9%27%18%36%20%70%20%40%27%55%20%18%64%90%91%18%36%64%67%73%90%100%100%64%100%
Xelnaga37.71%0%0%0%55%18%0%27%73%0%27%9%36%27%0%27%64%45%27%64%82%73%0%64%73%91%64%73%
Overkill27.95%0%0%0%9%0%0%45%0%9%0%9%9%45%27%9%9%0%27%36%18%82%55%64%55%73%82%91%
Juno25.42%0%9%0%18%27%0%9%0%9%0%64%36%9%0%64%27%9%0%33%27%18%9%45%100%82%0%91%
GarmBot26.60%0%0%0%0%0%0%0%0%0%0%0%9%0%45%18%36%0%0%27%100%45%91%91%9%100%64%82%
Myscbot20.27%27%9%0%9%18%18%0%73%0%0%0%0%0%9%9%9%55%36%10%36%36%55%9%36%55%18%18%
HannesBredberg21.21%0%0%0%27%0%9%0%0%0%9%9%73%0%18%9%0%0%45%0%27%45%0%91%64%64%27%55%
Sling14.19%0%0%0%18%0%0%9%0%18%0%9%0%0%27%0%0%0%36%0%9%27%18%0%45%36%91%36%
ForceBot25.93%0%0%18%18%0%0%0%0%0%9%9%18%0%9%27%73%9%18%36%36%18%100%36%82%73%9%100%
Ziabot10.77%0%0%0%0%0%0%0%0%9%0%0%0%0%0%18%0%9%0%0%27%9%9%18%82%45%64%0%

(4) Andromeda

overallZZZKPurpIroncpacMicrCherMcRaArraTyrSteaAILiLetaXimpUAlbAiurIceBSkynKillMegaXelnOverJunoGarmMyscHannSlinForcZiab
ZZZKBot81.82%55%100%45%82%91%91%91%18%91%73%9%100%100%82%82%91%100%91%100%100%91%100%73%100%55%100%100%
PurpleWave83.39%45%82%91%91%64%91%100%100%73%27%64%91%91%55%91%100%55%89%91%100%73%100%100%91%100%100%100%
Iron74.41%0%18%36%82%9%36%82%91%55%100%64%100%73%100%100%100%27%100%91%100%100%73%100%100%100%100%73%
cpac74.32%55%9%64%100%18%100%18%9%73%100%73%100%100%64%100%73%100%80%91%91%64%100%64%91%82%91%100%
Microwave70.03%18%9%18%0%64%18%82%45%55%45%82%73%64%100%100%64%100%91%91%100%73%100%100%100%100%100%100%
CherryPi73.40%9%36%91%82%36%73%36%100%36%73%91%18%64%91%100%91%45%45%100%82%100%100%82%100%100%100%100%
McRave64.86%9%9%64%0%82%27%91%45%82%45%9%73%45%45%73%64%91%60%82%82%82%100%100%100%91%100%100%
Arrakhammer67.91%9%0%18%82%18%64%9%91%18%73%45%73%55%91%100%55%100%90%64%100%100%100%82%100%100%100%100%
Tyr62.96%82%0%9%91%55%0%55%9%82%73%73%100%45%73%82%64%100%82%73%82%82%91%64%55%9%100%73%
Steamhammer69.02%9%27%45%27%45%64%18%82%18%64%27%73%91%64%100%55%100%73%100%91%100%100%91%100%100%100%100%
AILien63.30%27%73%0%0%55%27%55%27%27%36%18%45%55%100%100%64%91%91%100%73%73%100%82%100%100%91%100%
LetaBot63.64%91%36%36%27%18%9%91%55%27%73%82%91%64%45%0%82%100%100%91%82%45%73%100%45%100%91%64%
Ximp54.88%0%9%0%0%27%82%27%27%0%27%55%9%45%100%0%100%64%100%91%73%82%91%91%91%100%100%91%
UAlbertaBot53.20%0%9%27%0%36%36%55%45%55%9%45%36%55%73%82%73%73%45%100%64%91%27%64%82%55%100%100%
Aiur45.45%18%45%0%36%0%9%55%9%27%36%0%55%0%27%100%91%73%18%64%82%36%64%73%82%73%82%73%
IceBot46.28%18%9%0%0%0%0%27%0%18%0%0%100%100%18%0%55%91%90%73%73%64%82%91%100%91%82%73%
Skynet47.14%9%0%0%27%36%9%36%45%36%45%36%18%0%27%9%45%36%18%91%91%73%100%82%100%100%100%100%
KillAll43.05%0%45%73%0%0%55%9%0%0%0%9%0%36%27%27%9%64%100%100%36%100%82%64%100%55%82%100%
MegaBot36.01%9%11%0%20%9%55%40%10%18%27%9%0%0%55%82%10%82%0%36%64%20%64%73%90%9%90%82%
Xelnaga23.57%0%9%9%9%9%0%18%36%27%0%0%9%9%0%36%27%9%0%64%64%36%0%73%55%18%91%27%
Overkill33.67%0%0%0%9%0%18%18%0%18%9%27%18%27%36%18%27%9%64%36%36%82%73%100%27%100%64%91%
Juno33.78%9%27%0%36%27%0%18%0%18%0%27%55%18%9%64%36%27%0%80%64%18%0%45%100%82%64%91%
GarmBot29.97%0%0%27%0%0%0%0%0%9%0%0%27%9%73%36%18%0%18%36%100%27%100%73%0%100%82%73%
Myscbot22.22%27%0%0%36%0%18%0%18%36%9%18%0%9%36%27%9%18%36%27%27%0%55%27%55%9%82%18%
HannesBredberg22.64%0%9%0%9%0%0%0%0%45%0%0%55%9%18%18%0%0%0%10%45%73%0%100%45%55%91%27%
Sling25.93%45%0%0%18%0%0%9%0%91%0%0%0%0%45%27%9%0%45%91%82%0%18%0%91%45%73%9%
ForceBot12.84%0%0%0%9%0%0%0%0%0%0%9%9%0%0%18%18%0%18%10%9%36%36%18%18%9%27%100%
Ziabot19.87%0%0%27%0%0%0%0%0%27%0%0%36%9%0%27%27%0%0%18%73%9%9%27%82%73%91%0%

(4) Circuit Breaker

overallZZZKPurpIroncpacMicrCherMcRaArraTyrSteaAILiLetaXimpUAlbAiurIceBSkynKillMegaXelnOverJunoGarmMyscHannSlinForcZiab
ZZZKBot76.35%36%55%64%73%91%36%64%27%100%73%18%91%100%64%91%100%82%80%100%100%100%100%73%100%45%100%100%
PurpleWave83.45%64%64%91%100%73%100%82%91%82%73%64%91%100%18%100%100%18%90%91%100%73%100%91%100%100%100%100%
Iron80.13%45%36%9%27%9%45%100%91%73%100%64%100%91%100%100%100%100%100%82%100%100%100%91%100%100%100%100%
cpac72.64%36%9%91%100%36%100%36%0%64%91%91%100%91%45%100%82%91%90%55%100%91%100%82%82%64%36%100%
Microwave68.35%27%0%73%0%36%18%36%36%36%36%82%82%82%91%82%100%100%91%82%100%55%100%100%100%100%100%100%
CherryPi70.37%9%27%91%64%64%55%36%91%18%36%82%36%45%91%100%36%91%55%100%91%100%100%91%91%100%100%100%
McRave66.67%64%0%55%0%82%45%82%64%82%73%55%64%45%55%100%91%82%45%64%55%91%82%100%100%55%82%91%
Arrakhammer65.66%36%18%0%64%64%64%18%64%45%64%18%82%45%73%100%73%100%45%27%91%100%100%82%100%100%100%100%
Tyr68.01%73%9%9%100%64%9%36%36%91%73%64%100%64%45%55%91%91%82%64%82%100%100%73%82%91%100%55%
Steamhammer65.32%0%18%27%36%64%82%18%55%9%91%45%100%91%36%82%36%100%45%55%100%100%100%100%82%100%91%100%
AILien56.42%27%27%0%9%64%64%27%36%27%9%0%27%36%45%73%82%91%80%100%73%36%100%100%100%91%100%100%
LetaBot66.89%82%36%36%9%18%18%45%82%36%55%100%100%45%91%0%55%100%90%91%91%55%91%100%82%100%100%100%
Ximp54.88%9%9%0%0%18%64%36%18%0%0%73%0%45%100%0%100%64%91%91%91%100%100%91%91%100%91%100%
UAlbertaBot55.22%0%0%9%9%18%55%55%55%36%9%64%55%55%82%73%82%45%64%100%73%82%64%55%100%55%100%100%
Aiur53.22%36%82%0%55%9%9%45%27%55%64%55%9%0%18%91%91%64%22%27%100%18%82%91%100%82%100%100%
IceBot42.23%9%0%0%0%18%0%0%0%45%18%27%100%100%27%9%0%55%60%36%45%82%73%73%100%100%73%91%
Skynet44.26%0%0%0%18%0%64%9%27%9%64%18%45%0%18%9%100%27%20%64%73%64%100%64%100%100%100%100%
KillAll42.76%18%82%0%9%0%9%18%0%9%0%9%0%36%55%36%45%73%55%91%55%100%91%64%73%36%91%100%
MegaBot47.16%20%10%0%10%9%45%55%55%18%55%20%10%9%36%78%40%80%45%18%82%40%82%70%100%100%100%100%
Xelnaga43.10%0%9%18%45%18%0%36%73%36%45%0%9%9%0%73%64%36%9%82%82%45%0%82%100%100%100%91%
Overkill29.97%0%0%0%0%0%9%45%9%18%0%27%9%9%27%0%55%27%45%18%18%73%45%64%55%82%82%91%
Juno31.08%0%27%0%9%45%0%9%0%0%0%64%45%0%18%82%18%36%0%60%55%27%0%55%100%73%73%45%
GarmBot27.61%0%0%0%0%0%0%18%0%0%0%0%9%0%36%18%27%0%9%18%100%55%100%82%9%100%82%82%
Myscbot21.28%27%9%9%18%0%9%0%18%27%0%0%0%9%45%9%27%36%36%30%18%36%45%18%73%0%64%9%
HannesBredberg17.57%0%0%0%18%0%9%0%0%18%18%0%18%9%0%0%0%0%27%0%0%45%0%91%27%100%82%9%
Sling18.64%55%0%0%36%0%0%45%0%9%0%9%0%0%45%18%0%0%64%0%0%18%27%0%100%0%55%18%
ForceBot14.14%0%0%0%64%0%0%18%0%0%9%0%0%9%0%0%27%0%9%0%0%18%27%18%36%18%45%82%
Ziabot16.22%0%0%0%0%0%0%9%0%45%0%0%0%0%0%0%9%0%0%0%9%9%55%18%91%91%82%18%

(4) Empire of the Sun

overallZZZKPurpIroncpacMicrCherMcRaArraTyrSteaAILiLetaXimpUAlbAiurIceBSkynKillMegaXelnOverJunoGarmMyscHannSlinForcZiab
ZZZKBot84.46%73%64%73%100%91%73%82%45%91%73%91%82%100%73%100%91%100%90%100%100%100%100%64%100%27%100%100%
PurpleWave85.08%27%73%100%80%91%82%100%91%82%55%73%100%100%55%91%100%64%100%91%100%73%100%82%100%91%100%100%
Iron78.04%36%27%82%64%0%45%100%91%82%100%64%100%73%91%100%100%9%90%55%100%100%100%100%100%100%100%100%
cpac69.70%27%0%18%91%45%100%55%0%73%82%73%100%100%55%82%91%82%82%82%82%82%100%82%91%55%55%100%
Microwave70.95%0%20%36%9%45%27%82%45%45%45%82%73%91%82%91%100%100%82%82%91%82%100%100%100%100%100%100%
CherryPi68.35%9%9%100%55%55%45%27%82%27%27%82%55%45%82%100%45%73%45%91%100%100%100%91%100%100%100%100%
McRave62.29%27%18%55%0%73%55%55%73%45%55%64%91%55%36%82%45%55%45%100%91%55%100%100%73%45%91%100%
Arrakhammer66.78%18%0%0%45%18%73%45%91%45%55%64%91%64%64%91%55%100%44%64%100%100%100%73%100%100%100%100%
Tyr68.35%55%9%9%100%55%18%27%9%91%55%55%100%82%91%55%73%100%82%73%91%100%91%82%91%73%100%82%
Steamhammer61.15%9%18%18%27%55%73%55%55%9%73%45%91%73%27%91%27%91%50%64%100%100%100%55%64%100%82%100%
AILien60.81%27%45%0%18%55%73%45%45%45%27%0%64%36%55%82%91%100%70%82%64%36%100%100%100%100%82%100%
LetaBot54.42%9%27%36%27%18%18%36%36%45%55%100%73%45%55%9%64%91%50%36%64%55%91%100%36%100%91%100%
Ximp51.35%18%0%0%0%27%45%9%9%0%9%36%27%36%100%18%100%64%90%82%64%100%100%82%91%100%82%100%
UAlbertaBot48.82%0%0%27%0%9%55%45%36%18%27%64%55%64%73%91%64%18%55%100%64%82%27%27%73%55%91%100%
Aiur49.16%27%45%9%45%18%18%64%36%9%73%45%45%0%27%73%82%64%9%36%100%27%64%55%91%73%100%91%
IceBot41.89%0%9%0%18%9%0%18%9%45%9%18%91%82%9%27%27%82%40%0%82%91%27%82%100%91%91%73%
Skynet44.41%9%0%0%9%0%55%55%45%27%73%9%36%0%36%18%73%27%22%36%82%18%100%64%100%100%100%100%
KillAll47.14%0%36%91%18%0%27%45%0%0%9%0%9%36%82%36%18%73%82%91%45%100%100%82%100%36%64%91%
MegaBot50.18%10%0%10%18%18%55%55%56%18%50%30%50%10%45%91%60%78%18%70%91%30%55%91%100%78%89%82%
Xelnaga39.53%0%9%45%18%18%9%0%36%27%36%18%64%18%0%64%100%64%9%30%82%36%0%100%73%73%91%45%
Overkill29.97%0%0%0%18%9%0%9%0%9%0%36%36%36%36%0%18%18%55%9%18%82%45%55%36%91%100%91%
Juno34.12%0%27%0%18%18%0%45%0%0%0%64%45%0%18%73%9%82%0%70%64%18%0%73%82%82%73%64%
GarmBot31.31%0%0%0%0%0%0%0%0%9%0%0%9%0%73%36%73%0%0%45%100%55%100%100%0%100%73%73%
Myscbot22.90%36%18%0%18%0%9%0%27%18%45%0%0%18%73%45%18%36%18%9%0%45%27%0%64%0%64%27%
HannesBredberg22.90%0%0%0%9%0%0%27%0%9%36%0%64%9%27%9%0%0%0%0%27%64%18%100%36%91%73%18%
Sling23.05%73%9%0%45%0%0%55%0%27%0%0%0%0%45%27%9%0%64%22%27%9%18%0%100%9%73%9%
ForceBot16.27%0%0%0%45%0%0%9%0%0%18%18%9%18%9%0%9%0%36%11%9%0%27%27%36%27%27%100%
Ziabot16.84%0%0%0%0%0%0%0%0%18%0%0%0%0%0%9%27%0%9%18%55%9%36%27%73%82%91%0%

(4) Fortress

overallZZZKPurpIroncpacMicrCherMcRaArraTyrSteaAILiLetaXimpUAlbAiurIceBSkynKillMegaXelnOverJunoGarmMyscHannSlinForcZiab
ZZZKBot90.57%91%73%73%82%91%91%91%73%100%91%100%100%100%100%100%100%100%100%100%100%100%100%82%100%9%100%100%
PurpleWave78.45%9%55%91%91%82%36%100%73%82%27%55%91%91%55%82%91%55%100%91%100%73%100%91%100%100%100%100%
Iron83.84%27%45%100%100%0%27%91%100%82%100%100%100%64%100%100%100%45%100%100%82%100%100%100%100%100%100%100%
cpac69.15%27%9%0%91%55%100%27%9%64%82%91%100%100%64%100%82%64%89%55%82%73%100%73%82%91%64%100%
Microwave71.33%18%9%0%9%55%36%55%55%73%27%100%64%90%100%91%91%100%100%100%100%73%100%100%100%100%91%100%
CherryPi71.72%9%18%100%45%45%55%18%91%45%45%91%64%64%91%91%45%73%73%91%91%100%100%91%100%100%100%100%
McRave69.59%9%64%73%0%64%45%82%64%73%55%100%91%91%27%82%55%82%80%91%45%45%100%100%100%91%82%91%
Arrakhammer69.02%9%0%9%73%45%82%18%82%27%91%100%64%73%73%91%64%100%55%36%100%100%100%73%100%100%100%100%
Tyr70.71%27%27%0%91%45%9%36%18%100%82%100%100%64%100%55%100%100%91%64%91%91%100%82%100%55%100%82%
Steamhammer63.18%0%18%18%36%27%55%27%73%0%82%91%91%82%36%91%73%91%30%64%82%100%100%100%55%100%82%100%
AILien60.94%9%73%0%18%73%55%45%9%18%18%73%55%27%73%82%73%100%55%82%73%73%100%73%100%100%91%100%
LetaBot24.75%0%45%0%9%0%9%0%0%0%9%27%9%18%55%0%9%82%33%0%0%82%36%91%0%91%45%18%
Ximp53.20%0%9%0%0%36%36%9%36%0%9%45%91%55%100%0%100%36%55%82%82%100%100%64%100%100%91%100%
UAlbertaBot52.70%0%9%36%0%10%36%9%27%36%18%73%82%45%55%91%73%36%64%100%27%91%82%64%91%100%64%100%
Aiur48.15%0%45%0%36%0%9%73%27%0%64%27%45%0%45%82%82%73%9%27%91%36%100%73%100%100%82%73%
IceBot44.11%0%18%0%0%9%9%18%9%45%9%18%100%100%9%18%45%64%55%9%64%55%82%91%100%100%73%91%
Skynet46.13%0%9%0%18%9%55%45%36%0%27%27%91%0%27%18%55%9%27%55%100%64%100%73%100%100%100%100%
KillAll43.92%0%45%55%36%0%27%18%0%0%9%0%18%64%64%27%36%91%50%91%27%100%100%73%73%45%36%100%
MegaBot48.23%0%0%0%11%0%27%20%45%9%70%45%67%45%36%91%45%73%50%27%70%40%100%73%100%70%91%90%
Xelnaga41.75%0%9%0%45%0%9%9%64%36%36%18%100%18%0%73%91%45%9%73%91%27%9%64%100%91%91%18%
Overkill39.19%0%0%18%18%0%9%55%0%9%18%27%100%18%73%9%36%0%73%30%9%91%91%73%55%91%73%82%
Juno31.08%0%27%0%27%27%0%55%0%9%0%27%18%0%9%64%45%36%0%60%73%9%9%36%100%55%55%100%
GarmBot20.20%0%0%0%0%0%0%0%0%0%0%0%64%0%18%0%18%0%0%0%91%9%91%73%0%91%18%73%
Myscbot24.92%18%9%0%27%0%9%0%27%18%0%27%9%36%36%27%9%27%27%27%36%27%64%27%82%0%55%45%
HannesBredberg18.24%0%0%0%18%0%0%0%0%0%45%0%100%0%9%0%0%0%27%0%0%45%0%100%18%64%55%9%
Sling19.59%91%0%0%9%0%0%9%0%45%0%0%9%0%0%0%0%0%55%30%9%9%45%9%100%36%64%9%
ForceBot25.25%0%0%0%36%9%0%18%0%0%18%9%55%9%36%18%27%0%64%9%9%27%45%82%45%45%36%82%
Ziabot19.93%0%0%0%0%0%0%9%0%18%0%0%82%0%0%27%9%0%0%10%82%18%0%27%55%91%91%18%

(4) Python

overallZZZKPurpIroncpacMicrCherMcRaArraTyrSteaAILiLetaXimpUAlbAiurIceBSkynKillMegaXelnOverJunoGarmMyscHannSlinForcZiab
ZZZKBot91.22%82%73%91%82%91%55%91%64%100%91%82%100%100%100%100%100%100%100%100%100%100%100%64%100%100%100%100%
PurpleWave82.15%18%82%100%100%82%55%100%82%100%36%73%82%91%55%100%91%36%91%82%100%64%100%100%100%100%100%100%
Iron83.67%27%18%73%91%10%82%82%100%100%100%91%100%82%100%100%100%36%89%100%100%91%100%100%100%91%91%100%
cpac66.33%9%0%27%91%55%91%36%0%82%82%55%100%100%55%100%64%73%73%55%82%64%100%82%55%100%64%100%
Microwave69.70%18%0%9%9%82%18%73%45%73%36%73%64%100%100%91%73%100%82%100%82%55%100%100%100%100%100%100%
CherryPi69.05%9%18%90%45%18%55%45%82%36%36%91%73%55%91%82%45%45%90%100%91%100%100%80%100%100%91%100%
McRave63.85%45%45%18%9%82%45%91%64%55%82%27%64%36%55%64%64%82%60%64%18%91%100%100%91%73%100%100%
Arrakhammer64.19%9%0%18%64%27%55%9%100%9%36%64%91%64%64%91%55%91%60%45%100%100%100%82%100%100%100%100%
Tyr64.98%36%18%0%100%55%18%36%0%64%45%45%100%82%36%91%64%82%64%64%91%100%91%91%91%100%100%91%
Steamhammer61.82%0%0%0%18%27%64%45%91%36%100%55%73%82%27%100%55%91%70%55%100%100%100%73%27%100%82%100%
AILien61.28%9%64%0%18%64%64%18%64%55%0%9%36%55%55%100%82%82%73%91%73%73%100%82%91%100%100%100%
LetaBot58.25%18%27%9%45%27%9%73%36%55%45%91%82%64%64%18%73%82%82%27%73%55%73%91%55%100%100%100%
Ximp51.18%0%18%0%0%36%27%36%9%0%27%64%18%45%91%0%91%36%82%36%73%100%100%91%100%100%100%100%
UAlbertaBot50.51%0%9%18%0%0%45%64%36%18%18%45%36%55%73%82%64%27%82%100%45%82%45%82%82%64%91%100%
Aiur47.47%0%45%0%45%0%9%45%36%64%73%45%36%9%27%82%82%73%27%18%91%0%82%82%82%73%91%64%
IceBot44.41%0%0%0%0%9%18%36%9%9%0%0%82%100%18%18%36%82%33%55%64%91%73%91%91%100%82%100%
Skynet47.30%0%9%0%36%27%55%36%45%36%45%18%27%9%36%18%64%27%20%64%100%82%100%27%100%100%91%100%
KillAll48.15%0%64%64%27%0%55%18%9%18%9%18%18%64%73%27%18%73%73%100%45%100%91%91%55%73%27%91%
MegaBot44.64%0%9%11%27%18%10%40%40%36%30%27%18%18%18%73%67%80%27%45%78%45%70%50%90%91%100%100%
Xelnaga42.42%0%18%0%45%0%0%36%55%36%45%9%73%64%0%82%45%36%0%55%91%36%0%82%55%82%100%100%
Overkill32.54%0%0%0%18%18%9%82%0%9%0%27%27%27%55%9%36%0%55%22%9%27%64%73%55%82%91%82%
Juno32.32%0%36%9%36%45%0%9%0%0%0%27%45%0%18%100%9%18%0%55%64%73%9%27%73%73%73%73%
GarmBot28.72%0%0%0%0%0%0%0%0%9%0%0%27%0%55%18%27%0%9%30%100%36%91%100%0%100%82%91%
Myscbot22.71%36%0%0%18%0%20%0%18%9%27%18%9%9%18%18%9%73%9%50%18%27%73%0%45%9%82%18%
HannesBredberg28.72%0%0%0%45%0%0%9%0%9%73%9%45%0%18%18%9%0%45%10%45%45%27%100%55%73%91%45%
Sling13.80%0%0%9%0%0%0%27%0%0%0%0%0%0%36%27%0%0%27%9%18%18%27%0%91%27%55%0%
ForceBot15.49%0%0%9%36%0%9%0%0%0%18%0%0%0%9%9%18%9%73%0%0%9%27%18%18%9%45%100%
Ziabot12.88%0%0%0%0%0%0%0%0%9%0%0%0%0%0%36%0%0%9%0%0%18%27%9%82%55%100%0%

These tables surface tons of details, many of which are hard to interpret. For example, ZZZKBot shows different patterns of upsets on different maps, suggesting that ZZZKBot’s strategy selection was a little fragile and sometimes got exploited.

LetaBot hated the map Fortress, doing poorly against most opponents. It apparently failed to cope with something about the map. Myscbot liked the 2-player maps Benzene and Destination (but for some reason not Heartbreak Ridge). The tables reveal that it is because Myscbot upset specific opponents on these maps in particular. Something narrow and specific about the maps and strategies aligned for Myscbot.

I will poke at some of the bots to tease out their secrets. Maybe that will reveal the meaning behind some of the mysterious numbers.

Next: Looking at ZZZKBot.

AIIDE 2017 race balance

The race representation in AIIDE 2017 was very unbalanced, with 13 zergs and only 4 terrans. But the results were closely balanced by race. If the colors in the table look white, well, one of them is and the others nearly are.

racescore
terran51%
protoss50%
zerg49%
random53%

All races did equally well overall. At the top of the rankings too, the 3 winners represent each race and their scores are virtually equal. I take it to mean that there is no good reason for the preponderance of zergs. In a way, the balance is a coincidence; if one race had stronger entrants, maybe for reasons unrelated to Starcraft, there would be an imbalance. And yet the point is made: It doesn’t matter what race you choose for your bot.

Only 1 bot played random, UAlbertaBot. That leaves the vRandom statistics not very interesting, so I left them out of the other tables.

Since the overall balance is virtually level, I added a matchup table.

vTvPvZ
terran51%52%
protoss49%51%
zerg48%49%

Again, the balance is virtually level. Terran wasn’t balanced because terran did well against zerg and poorly against protoss or vice versa; everything was equal all around. Well, protoss did a smidge better versus zerg and a smidge worse versus terran, but it’s hardly noticeable.

#botraceoverallvTvPvZ
1ZZZKBotzerg83.11%75%79%88%
2PurpleWaveprotoss82.35%79%82%83%
3Ironterran81.52%88%85%78%
4cpaczerg71.01%73%63%75%
5Microwavezerg70.86%77%67%71%
6CherryPizerg69.08%92%70%62%
7McRaveprotoss67.07%70%65%68%
8Arrakhammerzerg65.95%65%59%72%
9Tyrprotoss65.91%52%70%68%
10Steamhammerzerg64.14%57%54%74%
11AILienzerg58.29%48%61%62%
12LetaBotterran56.92%30%61%61%
13Ximpprotoss54.19%34%63%55%
14UAlbertaBotrandom53.40%58%60%47%
15Aiurprotoss50.46%54%49%52%
16IceBotterran45.62%64%50%40%
17Skynetprotoss43.78%40%32%54%
18KillAllzerg43.04%39%55%34%
19MegaBotprotoss42.83%43%41%45%
20Xelnagaprotoss37.10%54%38%34%
21Overkillzerg32.69%25%30%37%
22Junoprotoss29.57%39%35%24%
23GarmBotzerg27.09%15%34%24%
24Myscbotprotoss25.94%19%25%27%
25HannesBredbergterran21.26%18%11%31%
26Slingzerg21.09%8%28%19%
27ForceBotzerg17.97%21%15%20%
28Ziabotzerg17.21%26%21%13%

Individual bots, of course, are not as balanced. Some of the table cells have striking numbers. First of all, with many zergs and few terrans, the vZ column carries the most weight. Sure enough, #3 Iron’s relative weakness versus zerg (“only” winning 3:1) allowed competitors to squeeze in front.

The largest number in any cell is #6 CherryPi’s 92% versus terran. CherryPi crushed the 4 terrans: #3 Iron, #12 LetaBot, #16 ICEbot, #25 HannesBredberg. #26 Sling, in contrast, rolled over and died for terrans but had some chance against other races. It makes sense that the matchup with the fewest participants, terran, would give us the most extreme numbers.

#12 LetaBot and #13 XIMP struggled against terran, while #16 ICEbot and #20 Xelnaga were happy to accept terran customers. #10 Steamhammer and #17 Skynet only played well against zerg, while #18 KillAll liked protoss victims.

Next: The per-map crosstables. Prepare for data overload.

AIIDE 2017 replays

1. Dave Churchill let me know: The AIIDE 2017 results now include replays. You can watch individual games from the html result listing (the page comes with other nice features too) or you can download bulky replay packs of each bot.

2. The race balance post will follow as soon as my parsing script can reproduce the official results. There seem to be subtle points I haven’t gotten quite right yet.

AIIDE 2017 results discussion

The AIIDE 2017 results include the usual thorough tables and graphs and data files. I don’t see replays, though, so I will only be guessing about play styles. Except, of course, for what I can see in the 7 videos of sample games.

The finishing order was #1 ZZZKBot, #2 PurpleWave, and #3 Iron, all with extremely close winning percentages of 82% - 83%. If you look at the winning percentages over time, Iron started well ahead early in the tournament and the other 2 caught up over time due to learning, finally edging just in front. This year, learning is winning. Iron, we know, has superior micro and an adaptive play style that can cope with most challenges—but it does not learn about its opponents. Opponent modeling has become a feature which helps you win tournaments (and I’m still disappointed I didn’t finish it in time). Steamhammer’s random opening choice, an anti-learning feature, held Steamhammer’s results steady over time as many other non-learning bots lost ground to the learners. #4 cpac in particular lost a lot of ground and came within a hair’s breadth of falling behind #5 Microwave.

#1 ZZZKBot supposedly added new cheese builds and learned which ones to play when. I will dig into it later and see what’s going on.

ZZZKBot, cpac, and CherryPi are notable for the short durations of their games. Cpac and CherryPi apparently like cheese or at least low-econ pressure builds (we saw some of that in the videos). The average duration of games was about the same as last year, even though with many zergs this year there should have been more fast ZvZ games. That may reflect a higher level of play—or that bots don’t know how to play ZvZ.

I get a crash rate this year of 3.8% of games, compared to 6.7% of games last year. Apparently bots are more reliable (though some crashes may be due to the infrastructure, which could have improved too). The crash column does show 2 orders of magnitude difference between the most and least reliable bots, so no doubt a few bad apples can spoil the number. According to Dave Churchill’s comment yesterday, the crash column this year includes games in which a bot went idle for 60 seconds while last year presumably didn’t, so the improvement is bigger than it may appear.

the re-entrants

Even though some new entrants were weak bots which lost most of their games, the overall level of play was higher. 6 bots played in identical versions in 2016 and 2017. In 2016, 5 of the 6 scored above 50%; in 2017, only XIMP and Aiur were able to hold a little above 50%, and all the re-entrants found themselves in a tougher field.

bot20162017
AIUR61.22%50.46%
Garmbot42.52%27.09%
ICEbot57.43%45.62%
Skynet55.03%43.78%
Xelnaga56.98%37.10%
XIMP64.54%54.19%

race distribution

Nearly half of the entrants this year were zerg.

race#
terran4
protoss10
zerg13
random1
total28

If we look at new entrants, excluding the 2016 re-entries, it’s even more extreme. Zerg is the popular race. How much is that my fault for making Steamhammer a zerg?

race#
terran3
protoss6
zerg12
random1
total22

Looking at the results, zergs clustered toward the top and bottom of the rankings. Terran and protoss are more evenly spread.

heritage

This is based on a quick troll through the source code, and I probably missed things. #4 cpac, #5 Microwave, #8 Arrakhammer, #10 Steamhammer, and #18 KillAll are Steamhammer and its forks. Steamhammer itself finished behind most of them, which may represent the cost of open source. All the successful forks, of course, represent the benefit of open source. The direct UAlbertaBot forks—leaving aside Steamhammer and its family—seem to be #11 Ailien, #12 LetaBot, #14 UAlbertaBot itself, #21 Overkill, and #24 Myscbot, and possibly #26 Sling (it seems to borrow some code, at least).

Lesson 1: This is a lot of bots. UAlbertaBot is extremely influential, and its influence is still growing.

Lesson 2: The Steamhammer Branch was mostly more successful than the Old UAlbertaBot Family that it stems from. Apparently I did some things right.

Next: The massive per-map crosstables. Race balance.

CIG 2017 race balance

I thought of another CIG 2017 analysis I wanted to do. Here is how bots of each race scored in the tournament.

racescore
terran43%
protoss55%
zerg48%
random68%

The table is not interesting in itself. It reflects the participants more than anything else. Random bots scored well because the 2 random players both finished high; terran scored poorly because 5 of the 7 terrans were in the bottom half. But the numbers make it easier to interpret the next table, which is how each bot scored against opponents of each race.

botraceoverallvTvPvZvR
ZZZKBotzerg75.43%80%75%74%67%
tscmoorandom73.50%81%74%67%54%
PurpleWaveprotoss66.51%69%74%69%36%
LetaBotterran62.75%63%55%76%40%
UAlbertaBotrandom61.67%69%51%64%46%
MegaBotprotoss61.06%75%45%59%52%
Overkillzerg59.65%74%45%63%37%
CasiaBotzerg58.32%56%60%65%47%
Ziabotzerg58.49%70%51%60%32%
Ironterran58.11%65%54%57%50%
AIURprotoss56.73%71%53%50%35%
McRaveprotoss47.20%53%42%52%23%
Tyrprotoss45.32%48%36%49%43%
SRbotOneterran45.24%54%35%50%29%
TerranUABterran38.58%55%27%42%10%
Bonjwaterran33.04%42%30%35%10%
Bigeyesterran30.90%31%32%33%24%
OpprimoBotterran31.90%41%24%37%10%
Slingzerg26.07%28%26%28%15%
Salsazerg9.52%9%10%10%7%

The “versus random” column means versus Tscmoo and UAlbertaBot, so the other columns tell us more about strategy. We can see that Tscmoo was a little weak versus zerg, but LetaBot was strong. MegaBot and Overkill struggled versus protoss, while CasiaBot was strong against protoss and had some trouble against terran.

I don’t think it’s necessary to normalize the table entries to draw conclusions from them. But if people are interested, I could produce a version of the table with the score of each race (in the first table above) effectively subtracted out so that we can see how each bot did against each race relative to how well it “should” have.

CIG Elo ratings

Elo as calculated by Remi Coulom’s bayeselo program. The # column gives the official ranking, so you can see how it differs from the rank by Elo. The bayeselo ranking should be slightly more accurate because it takes into account all the information in the tournament results, but unfortunately there are missing games so the Elo is computed from slightly less data than the official results. The “better?” column tells how likely each bot is to be superior to the one below it.

#botscoreElobetter?
1ZZZKBot75%174998.8%
2tscmoo74%1731100%
3PurpleWave67%166099.9%
4LetaBot63%162697.9%
5UAlbertaBot62%161185.1%
6MegaBot61%160496.7%
7Overkill60%159189.3%
8CasiaBot59%158255.2%
9Ziabot59%158158.8%
10Iron58%157997.0%
11AIUR57%1566100%
12McRave47%147697.6%
13Tyr45%146279.9%
14SRbotOne45%145699.9%
15TerranUAB38%139799.9%
16Bonjwa33%134794.9%
18OpprimoBot32%133569.0%
17Bigeyes32%133199.9%
19Sling26%1275100%
20Salsa9%1041-

Looking at the better? column, we see that the top 3 are well separated; the places are virtually sure to be accurate. ZZZKBot and Tscmoo are close, but bayeselo thinks they are separated enough. Farther down, CasiaBot, Ziabot, and Iron are statistically hard to distinguish; there is not strong evidence that they finished in the correct order. Also OpprimoBot and Bigeyes are not well separated—as you might guess since their results are reversed from the official results.

Is this all the analysis we want of CIG 2017? I also have a script for the map balance, to check whether any race is favored. But it tells more about who competed than about the maps or bot skills.

CIG 2017 bots x maps

How well did each bot do on each map? This table is nice and compact and easy to read conclusions out of. Hitchhiker is a 2 player map and a complex one. Alchemist and Tau Cross are 3 players. Andromeda and Python are 4 players.

overallHitchhAlchemTauCroAndromPython
ZZZKBot75.43%81%76%73%73%75%
tscmoo73.50%78%69%73%74%73%
PurpleWave66.51%82%70%56%57%67%
LetaBot62.75%62%58%63%68%62%
UAlbertaBot61.67%63%65%60%61%59%
MegaBot61.06%55%66%64%60%61%
Overkill59.65%66%59%59%57%57%
CasiaBot58.32%62%61%59%55%55%
Ziabot58.49%66%59%55%57%56%
Iron58.11%5%60%77%75%74%
AIUR56.73%61%56%57%55%55%
McRave47.20%9%49%62%62%54%
Tyr45.32%52%42%44%46%43%
SRbotOne45.24%54%41%43%44%44%
TerranUAB38.58%47%40%33%39%33%
Bonjwa33.04%44%33%32%27%30%
Bigeyes30.90%35%30%29%28%33%
OpprimoBot31.90%37%33%30%30%29%
Sling26.07%23%27%24%25%32%
Salsa9.52%19%7%6%7%8%

If one bot loses on a given map, then another bot has to win. The BWEM blowouts on Hitchhiker lift the other numbers in that column about equally. It’s striking that the top 3 finishers all liked Hitchhiker above the other maps. Maybe it’s a sign that map smarts make a difference? Well, ZZZKBot presumably liked Hitchhiker because the map has a short rush distance, and PurpleWave may have gotten a similar effect if it used its probe rush on the map. But UAlbertaBot got no boost from the map, although it plays rushes.

I noted earlier that PurpleWave struggled surprisingly on Tau Cross and Andromeda, and that MegaBot did not like Hitchhiker. It’s also notable that LetaBot did not like Alchemist and McRave did not like Python.

Next: The bayeselo ratings.

CIG 2017 without the problem maps

What if CIG 2017 had included only maps that Iron plays properly on? Then Iron would have won, at least according to its results on the remaining maps. Leaving out Hitchhiker and Alchemist leaves only 3 other maps, Andromeda, Python, and Tau Cross. A tournament with only 3 maps is hardly satisfying, but it’s the best estimate of how Iron might have done if it had been prepared for the map pool.

overallIronZZZKtscmLetaMegaPurpUAlbMcRaOverCasiZiabAIURTyrSRboTerrBigeOpprBonjSlinSals
Iron75.37%55%57%28%80%63%80%51%85%84%77%84%55%85%91%95%95%87%91%91%
ZZZKBot73.46%45%45%41%77%47%79%84%93%29%64%91%81%81%91%84%95%88%89%91%
tscmoo73.33%43%55%83%69%88%55%69%64%53%71%77%68%69%89%77%96%89%83%95%
LetaBot64.49%72%59%17%60%60%71%47%81%85%73%63%57%43%69%77%40%72%84%95%
MegaBot61.40%20%23%31%40%35%75%52%51%40%59%36%57%75%89%91%97%100%97%100%
PurpleWave60.07%37%53%12%40%65%52%25%89%44%35%76%84%71%87%92%63%60%64%92%
UAlbertaBot60.07%20%21%45%29%25%48%80%57%63%69%55%44%71%89%75%83%89%84%93%
McRave59.44%49%16%31%53%48%75%20%39%49%72%35%41%88%93%55%89%84%97%95%
Overkill57.89%15%7%36%19%49%11%43%61%45%85%61%43%84%95%93%93%89%80%91%
CasiaBot56.28%16%71%47%15%60%56%37%51%55%35%60%73%32%51%95%68%75%85%89%
Ziabot56.04%23%36%29%27%41%65%31%28%15%65%56%57%72%81%79%91%91%83%95%
AIUR55.72%16%9%23%37%64%24%45%65%39%40%44%71%79%77%69%89%93%80%93%
Tyr44.28%45%19%32%43%43%16%56%59%57%27%43%29%28%45%36%65%57%48%93%
SRbotOne43.65%15%19%31%57%25%29%29%12%16%68%28%21%72%39%100%43%59%72%95%
TerranUAB35.09%9%9%11%31%11%13%11%7%5%49%19%23%55%61%57%73%77%55%91%
Bigeyes29.96%5%16%23%23%9%8%25%45%7%5%21%31%64%0%43%68%40%48%88%
OpprimoBot29.68%5%5%4%60%3%37%17%11%7%32%9%11%35%57%27%32%40%80%92%
Bonjwa29.47%13%12%11%28%0%40%11%16%11%25%9%7%43%41%23%60%60%57%93%
Sling27.09%9%11%17%16%3%36%16%3%20%15%17%20%52%28%45%52%20%43%92%
Salsa7.23%9%9%5%5%0%8%7%5%9%11%5%7%7%5%9%12%8%7%8%

Iron narrowly edges out ZZZKBot and Tscmoo. PurpleWave falls back and MegaBot, which also did poorly on Hitchhiker, jumps upward. McRave moves from the lower half to the upper half but does not break into the top ranks.

Here is another version of the same chart, with bots in the official finishing order. It makes some comparisons easier.

overallZZZKtscmPurpLetaUAlbMegaOverCasiZiabIronAIURMcRaTyrSRboTerrBonjBigeOpprSlinSals
ZZZKBot73.46%45%47%41%79%77%93%29%64%45%91%84%81%81%91%88%84%95%89%91%
tscmoo73.33%55%88%83%55%69%64%53%71%43%77%69%68%69%89%89%77%96%83%95%
PurpleWave60.07%53%12%40%52%65%89%44%35%37%76%25%84%71%87%60%92%63%64%92%
LetaBot64.49%59%17%60%71%60%81%85%73%72%63%47%57%43%69%72%77%40%84%95%
UAlbertaBot60.07%21%45%48%29%25%57%63%69%20%55%80%44%71%89%89%75%83%84%93%
MegaBot61.40%23%31%35%40%75%51%40%59%20%36%52%57%75%89%100%91%97%97%100%
Overkill57.89%7%36%11%19%43%49%45%85%15%61%61%43%84%95%89%93%93%80%91%
CasiaBot56.28%71%47%56%15%37%60%55%35%16%60%51%73%32%51%75%95%68%85%89%
Ziabot56.04%36%29%65%27%31%41%15%65%23%56%28%57%72%81%91%79%91%83%95%
Iron75.37%55%57%63%28%80%80%85%84%77%84%51%55%85%91%87%95%95%91%91%
AIUR55.72%9%23%24%37%45%64%39%40%44%16%65%71%79%77%93%69%89%80%93%
McRave59.44%16%31%75%53%20%48%39%49%72%49%35%41%88%93%84%55%89%97%95%
Tyr44.28%19%32%16%43%56%43%57%27%43%45%29%59%28%45%57%36%65%48%93%
SRbotOne43.65%19%31%29%57%29%25%16%68%28%15%21%12%72%39%59%100%43%72%95%
TerranUAB35.09%9%11%13%31%11%11%5%49%19%9%23%7%55%61%77%57%73%55%91%
Bonjwa29.47%12%11%40%28%11%0%11%25%9%13%7%16%43%41%23%60%60%57%93%
Bigeyes29.96%16%23%8%23%25%9%7%5%21%5%31%45%64%0%43%40%68%48%88%
OpprimoBot29.68%5%4%37%60%17%3%7%32%9%5%11%11%35%57%27%40%32%80%92%
Sling27.09%11%17%36%16%16%3%20%15%17%9%20%3%52%28%45%43%52%20%92%
Salsa7.23%9%5%8%5%7%0%9%11%5%9%7%5%7%5%9%7%12%8%8%

Next: The chart of bot x map.

CIG 2017 crosstables per map

CIG 2017 was played on only 5 maps. In my view, that’s not enough maps for fairness, but the advantage is that each map is played enough times that the crosstables have ample data. Each cell in these tables represents 25 games (minus the missing games, of course).

(2)Hitchhiker

overallZZZKtscmPurpLetaUAlbMegaOverCasiZiabIronAIURMcRaTyrSRboTerrBonjBigeOpprSlinSals
ZZZKBot80.80%76%0%96%88%68%92%40%68%92%100%96%92%92%84%92%84%92%96%88%
tscmoo78.27%24%96%80%56%76%64%48%52%100%71%96%76%92%96%92%96%88%92%92%
PurpleWave82.07%100%4%52%80%96%88%84%60%100%96%100%96%84%88%68%80%92%96%96%
LetaBot62.37%4%20%48%60%32%88%88%72%96%33%92%52%12%76%84%96%44%92%96%
UAlbertaBot63.08%12%44%20%40%28%56%40%68%92%62%100%52%68%96%92%60%100%80%88%
MegaBot54.97%32%24%4%68%72%72%28%72%100%42%100%32%88%88%12%17%96%16%80%
Overkill66.03%8%36%12%12%44%28%68%80%96%96%92%36%80%92%96%92%96%92%100%
CasiaBot61.95%60%52%16%12%60%72%32%32%96%67%92%64%16%44%92%96%84%92%100%
Ziabot65.61%32%48%40%28%32%28%20%68%96%71%96%68%76%76%92%92%96%88%100%
Iron4.65%8%0%0%4%8%0%4%4%4%0%32%4%0%4%4%0%4%4%4%
AIUR60.88%0%29%4%67%38%58%4%33%29%100%92%75%83%88%92%91%96%88%92%
McRave8.88%4%4%0%8%0%0%8%8%4%68%8%12%0%12%20%0%8%0%4%
Tyr51.90%8%24%4%48%48%68%64%36%32%96%25%88%16%44%64%72%64%88%96%
SRbotOne54.22%8%8%16%88%32%12%20%84%24%100%17%100%84%28%48%92%76%100%92%
TerranUAB47.47%16%4%12%24%4%12%8%56%24%96%12%88%56%72%76%88%56%96%100%
Bonjwa43.76%8%8%32%16%8%88%4%8%8%96%8%80%36%52%24%83%76%96%100%
Bigeyes34.76%16%4%20%4%40%83%8%4%8%100%9%100%28%8%12%17%17%88%96%
OpprimoBot37.21%8%12%8%56%0%4%4%16%4%96%4%92%36%24%44%24%83%92%100%
Sling22.57%4%8%4%8%20%84%8%8%12%96%12%100%12%0%4%4%12%8%24%
Salsa18.57%12%8%4%4%12%20%0%0%0%96%8%96%4%8%0%0%4%0%76%

(3)Alchemist

overallZZZKtscmPurpLetaUAlbMegaOverCasiZiabIronAIURMcRaTyrSRboTerrBonjBigeOpprSlinSals
ZZZKBot76.00%48%44%44%88%84%80%28%76%100%92%68%84%72%84%88%92%96%84%92%
tscmoo69.26%52%92%84%48%52%68%24%64%64%40%68%64%76%96%84%68%92%84%96%
PurpleWave70.32%56%8%40%72%76%88%56%40%64%88%80%96%72%84%56%96%92%76%96%
LetaBot57.89%56%16%60%36%32%48%68%56%64%32%96%48%72%64%60%64%40%92%96%
UAlbertaBot65.05%12%52%28%64%44%80%72%72%56%76%56%44%56%76%92%80%88%96%92%
MegaBot66.11%16%48%24%68%56%52%36%56%60%72%44%44%92%96%100%96%96%100%100%
Overkill58.53%20%32%12%52%20%48%72%44%76%44%32%44%88%92%92%92%80%84%88%
CasiaBot60.84%72%76%44%32%28%64%28%32%88%64%52%68%32%40%92%84%88%72%100%
Ziabot58.74%24%36%60%44%28%44%56%68%72%52%16%60%72%76%80%84%80%68%96%
Iron59.58%0%36%36%36%44%40%24%12%28%92%96%56%88%96%84%92%92%88%92%
AIUR55.79%8%60%12%68%24%28%56%36%48%8%32%84%88%92%88%60%88%88%92%
McRave48.63%32%32%20%4%44%56%68%48%84%4%68%84%60%12%24%96%4%92%92%
Tyr41.89%16%36%4%52%56%56%56%32%40%44%16%16%20%44%52%64%56%56%80%
SRbotOne41.05%28%24%28%28%44%8%12%68%28%12%12%40%80%32%80%92%12%60%92%
TerranUAB40.21%16%4%16%36%24%4%8%60%24%4%8%88%56%68%76%56%60%64%92%
Bonjwa33.05%12%16%44%40%8%0%8%8%20%16%12%76%48%20%24%44%80%56%96%
Bigeyes29.89%8%32%4%36%20%4%8%16%16%8%40%4%36%8%44%56%76%56%96%
OpprimoBot33.26%4%8%8%60%12%4%20%12%20%8%12%96%44%88%40%20%24%64%88%
Sling26.53%16%16%24%8%4%0%16%28%32%12%12%8%44%40%36%44%44%36%84%
Salsa7.37%8%4%4%4%8%0%12%0%4%8%8%8%20%8%8%4%4%12%16%

(3)Tau Cross

overallZZZKtscmPurpLetaUAlbMegaOverCasiZiabIronAIURMcRaTyrSRboTerrBonjBigeOpprSlinSals
ZZZKBot72.84%28%60%28%84%80%96%12%72%28%84%92%92%92%92%92%88%92%88%84%
tscmoo72.84%72%88%88%64%64%52%64%68%40%60%60%60%76%80%92%80%100%88%88%
PurpleWave56.21%40%12%40%48%56%88%36%36%36%88%12%92%56%80%52%100%56%48%92%
LetaBot63.16%72%12%60%72%56%72%88%76%68%56%28%52%56%72%64%68%44%88%96%
UAlbertaBot59.79%16%36%52%28%32%52%68%84%8%56%76%40%64%96%88%72%80%88%100%
MegaBot63.58%20%36%44%44%68%64%40%60%20%48%48%60%72%96%100%88%100%100%100%
Overkill59.37%4%48%12%28%48%36%36%92%12%48%84%44%80%100%88%92%96%84%96%
CasiaBot58.95%88%36%64%12%32%60%64%36%4%64%52%84%24%76%76%92%64%92%100%
Ziabot54.95%28%32%64%24%16%40%8%64%8%52%24%52%84%80%96%80%100%92%100%
Iron77.47%72%60%64%32%92%80%88%96%92%84%28%48%84%92%88%96%96%92%88%
AIUR57.26%16%40%12%44%44%52%52%36%48%16%76%72%80%80%92%60%88%88%92%
McRave62.32%8%40%88%72%24%52%16%48%76%72%24%48%92%96%88%56%88%100%96%
Tyr44.42%8%40%8%48%60%40%56%16%48%52%28%52%20%56%56%48%60%52%96%
SRbotOne42.95%8%24%44%44%36%28%20%76%16%16%20%8%80%32%64%100%48%56%96%
TerranUAB33.05%8%20%20%28%4%4%0%24%20%8%20%4%44%68%72%52%84%56%92%
Bonjwa31.79%8%8%48%36%12%0%12%24%4%12%8%12%44%36%28%64%76%72%100%
Bigeyes29.26%12%20%0%32%28%12%8%8%20%4%40%44%52%0%48%36%44%60%88%
OpprimoBot29.68%8%0%44%56%20%0%4%36%0%4%12%12%40%52%16%24%56%84%96%
Sling24.21%12%12%52%12%12%0%16%8%8%8%12%0%48%44%44%28%40%16%88%
Salsa5.89%16%12%8%4%0%0%4%0%0%12%8%4%4%4%8%0%12%4%12%

(4)Andromeda

overallZZZKtscmPurpLetaUAlbMegaOverCasiZiabIronAIURMcRaTyrSRboTerrBonjBigeOpprSlinSals
ZZZKBot72.84%44%64%20%84%72%92%36%56%60%92%68%76%76%92%84%84%96%92%96%
tscmoo74.32%56%84%84%60%72%64%44%76%44%88%68%68%68%92%92%76%92%88%96%
PurpleWave56.63%36%16%32%40%72%88%40%28%12%60%20%72%80%88%68%92%60%84%88%
LetaBot68.42%80%16%68%84%72%92%84%68%76%64%56%48%44%68%84%80%32%84%100%
UAlbertaBot61.47%16%40%60%16%20%60%72%76%28%48%88%56%80%84%88%76%92%76%92%
MegaBot60.00%28%28%28%28%80%48%36%48%20%36%52%60%72%80%100%100%96%100%100%
Overkill57.47%8%36%12%8%40%52%48%88%24%72%48%40%84%92%88%96%88%84%84%
CasiaBot54.53%64%56%60%16%28%64%52%40%20%36%36%64%44%36%80%96%84%80%80%
Ziabot57.26%44%24%72%32%24%52%12%60%36%68%32%52%64%88%92%80%88%76%92%
Iron74.74%40%56%88%24%72%80%76%80%64%76%56%60%88%88%88%96%92%96%100%
AIUR54.95%8%12%40%36%52%64%28%64%32%24%48%64%76%72%96%80%84%68%96%
McRave62.11%32%32%80%44%12%48%52%64%68%44%52%40%88%96%80%68%92%96%92%
Tyr45.68%24%32%28%52%44%40%60%36%48%40%36%60%28%28%48%44%72%52%96%
SRbotOne43.79%24%32%20%56%20%28%16%56%36%12%24%12%72%40%64%100%44%84%92%
TerranUAB38.74%8%8%12%32%16%20%8%64%12%12%28%4%72%60%76%68%72%72%92%
Bonjwa26.95%16%8%32%16%12%0%12%20%8%12%4%20%52%36%24%52%44%56%88%
Bigeyes27.79%16%24%8%20%24%0%4%4%20%4%20%32%56%0%32%48%76%56%84%
OpprimoBot30.32%4%8%40%68%8%4%12%16%12%8%16%8%28%56%28%56%24%88%92%
Sling24.63%8%12%16%16%24%0%16%20%24%4%32%4%48%16%28%44%44%12%100%
Salsa7.37%4%4%12%0%8%0%16%20%8%0%4%8%4%8%8%12%16%8%0%

(4)Python

overallZZZKtscmPurpLetaUAlbMegaOverCasiZiabIronAIURMcRaTyrSRboTerrBonjBigeOpprSlinSals
ZZZKBot74.68%64%16%76%68%80%92%40%62%48%96%92%76%76%88%88%80%96%88%92%
tscmoo72.84%36%92%76%40%72%76%52%68%44%84%80%76%64%96%84%76%96%72%100%
PurpleWave67.37%84%8%48%68%68%92%56%40%64%80%44%88%76%92%60%84%72%60%96%
LetaBot61.89%24%24%52%56%52%80%84%76%72%68%56%72%28%68%68%84%44%80%88%
UAlbertaBot58.95%32%60%32%44%24%60%48%48%24%60%76%36%68%88%92%76%76%88%88%
MegaBot60.63%20%28%32%48%76%40%44%68%20%24%56%52%80%92%100%84%96%92%100%
Overkill56.84%8%24%8%20%40%60%52%76%8%64%52%44%88%92%92%92%96%72%92%
CasiaBot55.37%60%48%44%16%52%56%48%28%24%80%64%72%28%40%68%96%56%84%88%
Ziabot55.91%38%32%60%24%52%32%24%72%24%48%28%68%68%76%84%76%84%80%92%
Iron73.89%52%56%36%28%76%80%92%76%76%92%68%56%84%92%84%92%96%84%84%
AIUR54.95%4%16%20%32%40%76%36%20%52%8%72%76%80%80%92%68%96%84%92%
McRave53.89%8%20%56%44%24%44%48%36%72%32%28%36%84%88%84%40%88%96%96%
Tyr42.74%24%24%12%28%64%48%56%28%32%44%24%64%36%52%68%16%64%40%88%
SRbotOne44.21%24%36%24%72%32%20%12%72%32%16%20%16%64%44%48%100%36%76%96%
TerranUAB33.47%12%4%8%32%12%8%8%60%24%8%20%12%48%56%84%52%64%36%88%
Bonjwa29.68%12%16%40%32%8%0%8%32%16%16%8%16%32%52%16%64%60%44%92%
Bigeyes32.84%20%24%16%16%24%16%8%4%24%8%32%60%84%0%48%36%84%28%92%
OpprimoBot29.05%4%4%28%56%24%4%4%44%16%4%4%12%36%64%36%40%16%68%88%
Sling32.42%12%28%40%20%12%8%28%16%20%16%16%4%60%24%64%56%72%32%88%
Salsa8.42%8%0%4%12%12%0%8%12%8%16%8%4%12%4%12%8%8%12%12%

observations

The difficult maps are Hitchhiker, which has complex map features that bots are not ready for and which BWEM apparently does not support, and Alchemist, which has 2 entrances into each base, a nonstandard strategic feature. Sure enough, on Hitchhiker we see that BWEM users #10 Iron and #12 McRave were virtually helpless. #6 MegaBot suffered to a lesser extent. Other extreme results on Hitchhiker: #3 PurpleWave blanked #1 ZZZKBot with 100% wins, a surprise since Hitchhiker is a 2 player map with a short rush distance which should favor the 4 pool. I guess the defensible narrow path between bases made up for that. Then PurpleWave got smashed by #2 tscmoo with 4% wins. The map also brought #20 Salsa most of its wins, with plus scores versus #10 Iron and #12 McRave and, in a surprise, #19 Sling; on every other map, Salsa lost heavily in all matchups. It’s not the same syndrome as the BWEM users, though.

Alchemist is a 3 player map with a circular path all the way around the map which passes through each base. Out one of your base entrances is the short path to your enemy’s base, and out the other entrance is the long path. You get a natural expansion outside each entrance. It’s bad for bots which want to set up one defensive line, whether by walling the ramp or cannoning the natural, because the enemy can bypass the defenses. It favors bots which reason about map topology and distances—if there are any. Comparing across maps, we see that the map cost Iron (which makes a wall at the ramp or at the natural) over 20% win rate and cost #12 McRave perhaps half as much. #3 PurpleWave, #5 UAlbertaBot, and #6 MegaBot did well on the map. I’m not sure why PurpleWave liked the map. I don’t think any of the bots does much map analysis; most bots were not much affected by the nonstandard map, probably because they are blind to its possibilities.

Curiously, #3 PurpleWave had some trouble on Tau Cross and Andromeda. Those maps are also played on SSCAIT, so I expected PurpleWave would be well tuned for them.

There’s a lot more in these crosstables, but that’s enough for now.

Next: What would have happened without Hitchhiker and Alchemist?

CIG 2017 crosstable

My version of the CIG 2017 crosstable. I have small differences from the official results—see the explanation below. My results reverse the finishing places of #8 CasiaBot and #9 Ziabot as well as #17 Bigeyes and #18 OpprimoBot, because even small differences affect ranking.

The format of the results file has changed since last year. There is no documentation, so I don’t know what all the columns mean, but I only needed a few of them and I was able to pick them out. It turns out that column 6 is true if the first player won, otherwise false. It looks like each game is recorded twice, with the same winner, loser, and map but some differences in other data. I imagine that each player is running in its own instance, and each instance records its own data. The games are numbered so the duplicates can be recognized, and sometimes the games are recorded out of order. I had to rewrite parsing code, but only a handful of lines.

The results file turned out to have a section of corrupted data in the middle. Information about a small number of games is missing or corrupted, and I had to delete it from my input. The tournament format of 125 rounds with 20 participants called for 125 * 20 * 19 / 2 = 23750 games. Each game was recorded twice, so there should be 47500 lines in the results file. One game was expected to be missed because the tournament manager software has an off-by-one bug and doesn’t play one game. The last and highest-numbered game recorded in the results file is 23721, and numbering starts from 0 so it looks as though 3 games in fact went unplayed, or at least uncounted. There are 47487 lines remaining in the input file, accounting for 23722 games or 99.88% of the ideal 23750, or 99.89% of the expected and claimed 23749 games.

Anyway, my winning percentages are different from the official numbers mostly starting in the 3rd decimal place, which is what you expect with a discrepancy in game count in the third decimal place. Apparently the official numbers don’t suffer from corrupted data. I have written to the organizers to see if they can provide a clean result file.

overallZZZKtscmPurpLetaUAlbMegaOverCasiZiabIronAIURMcRaTyrSRboTerrBonjBigeOpprSlinSals
ZZZKBot75.43%52%37%53%82%77%90%31%67%66%93%83%84%82%88%89%86%94%90%90%
tscmoo73.50%48%90%82%54%67%65%46%66%58%69%74%69%75%92%89%79%94%85%94%
PurpleWave66.51%63%10%42%62%74%89%54%41%55%82%51%89%74%86%61%90%74%73%94%
LetaBot62.75%47%18%58%62%49%76%82%70%75%51%66%54%42%70%72%78%41%87%95%
UAlbertaBot61.67%18%46%38%38%30%62%60%70%42%60%79%46%67%88%90%73%87%86%92%
MegaBot61.06%23%33%26%51%70%55%37%61%44%44%60%50%81%90%82%77%97%82%96%
Overkill59.65%10%35%11%24%38%45%55%76%43%65%62%42%84%94%91%93%91%83%92%
CasiaBot58.32%69%54%46%18%40%63%45%34%46%62%59%70%29%47%82%93%75%84%94%
Ziabot58.49%33%34%59%30%30%39%24%66%47%58%39%60%73%79%89%82%90%81%96%
Iron58.11%34%42%45%25%58%56%57%54%53%69%56%45%69%74%70%76%76%73%74%
AIUR56.73%7%31%18%49%40%56%35%38%42%31%64%74%81%82%92%72%90%83%93%
McRave47.20%17%26%49%34%21%40%38%41%61%44%36%44%65%61%59%52%56%77%76%
Tyr45.32%16%31%11%46%54%50%58%30%40%55%26%56%24%45%58%49%63%58%91%
SRbotOne45.24%18%25%26%58%33%19%16%71%27%31%19%35%76%35%61%97%43%75%94%
TerranUAB38.58%12%8%14%30%12%10%6%53%21%26%18%39%55%65%77%63%67%65%93%
Bonjwa33.04%11%11%39%28%10%18%9%18%11%30%8%41%42%39%23%61%67%65%95%
Bigeyes30.90%14%21%10%22%27%23%7%7%18%24%28%48%51%3%37%39%60%58%91%
OpprimoBot31.90%6%6%26%59%13%3%9%25%10%24%10%44%37%57%33%33%40%79%93%
Sling26.07%10%15%27%13%14%18%17%16%19%27%17%23%42%25%35%35%42%21%77%
Salsa9.52%10%6%6%5%8%4%8%6%4%26%7%24%9%6%7%5%9%7%23%

observations

Newcomers #3 PurpleWave and #8 CasiaBot were the only players with positive scores versus #1 ZZZKBot. When I have time I’ll look into the replays and see if ZZZKBot was ready for the old-timers with special builds, or if it was the old-timers who weren’t ready for ZZZKBot. Perhaps ZZZKBot now has a learning feature and switches to a backup build if its 4 pool doesn’t work? I’m interested to find out.

#2 Tscmoo was edged out by #1 ZZZKBot and #8 CasiaBot, but otherwise had plus scores across the board. It showed stable performance across opponents—except for its crushing 90% win over #3 PurpleWave.

#3 PurpleWave did reasonably well against all except #2 tscmoo (10% wins) and #9 Zia. (I count 42% against #4 LetaBot as reasonably good, though it’s technically an upset.) But there were several weaker opponents that it edged out more narrowly than you might expect. My conclusion: Strong, but a little lacking in the solidity needed to defeat weaker opponents consistently. With more maturity it will likely become even stronger.

#8 CasiaBot seems to have the most uneven results, with severe upsets in both directions—69% versus #1 ZZZKBot and 29% against #14 SRbotOne.

The biggest upset is #18 OpprimoBot at 59% against #4 LetaBot.

The winning rates of #10 Iron and #12 McRave versus tail-ender #20 Salsa, which lost nearly all games against other opponents, are 74% and 76%. It backs up the claim that the two played only about 75% of games due to map problems with BWEM.

What do you notice in the crosstable?

Next: Per-map crosstables. We can expect dramatic numbers in some table cells thanks to Iron and McRave.

Update: I heard back from the organizers. They say that they had to alter the results file to compensate for errors in the tournament manager. And they think that all the problems are slight given the large number of games played. I think that’s true as far as it goes, but it leaves me feeling a little uneasy about the official results.

AIIDE 2017 outlook

AIIDE submission closes after today. Some bots may yet withdraw at the last moment or otherwise disappear, but we pretty much know the roster. I want to look ahead with the information we have.

race distribution

This year we have a zerg plurality. That happens to be good for Steamhammer, because ZvZ is Steamhammer’s best matchup.

race#%
terran721%
protoss1031%
zerg1443%
random13%

The percentages add up to 98% instead of 100% due to rounding.

BWAPI versions

I was surprised how many bots were already on BWAPI 4.2.0.

BWAPI#%
3.7.4928%
4.1.21753%
4.2.0618%

The 2 bots listed as withdrawn were also on BWAPI 4.2.0. Does anybody know if or when SSCAIT will support the newer version? I know resources are limited, so I won’t be surprised if it takes time.

6 of the 9 bots still on BWAPI 3.7.4 are the legacy bots carried over from last year. Only 3 are active entrants. Surprisingly, one of the 3 is ForceBot, which seems to be a recent bot since it only recently appeared on SSCAIT and suffered some teething troubles there.

the newcomers

There are unusually many absolute newcomers, bots whose names or authors I don’t recognize. History says that most newcomers will do poorly; it takes a long time to turn an empty repository into a contender. But there are exceptions. This year PurpleWave reached #3 in CIG 2017. Last year Bereaver was a sensation. It will be a surprise if an unknown wins—but surprises happen.

  • bonjwAI
  • CherryPi
  • cpac
  • DeepTerran
  • HOLD
  • Inspir
  • Myscbot
  • Sling

I think many are especially looking forward to the Facebook entry CherryPi by old hand Gabriel Synnaeve. I’ve read the papers, and my hopes are not high. My impression is that the project is not far along. I expect CherryPi to finish in the lower half, and I mainly hope that it will show some interesting behaviors along the way.

the unpredictable

A few bots are both potential top finishers and potential also-rans. Arrakhammer and McRave have shown inconsistent results, strong play when in their best form but many losses when problems creep into the codebase. AILien was often impressive on SSCAIT, but stopped playing there after updating to BWAPI 4.2.0. I don’t know what updates it may have seen since.

the known contenders

Iron is of course the favorite. In CIG, Iron suffered because it was unready for the new and troublesome maps. In AIIDE, the maps are standard and are the same as last year. I see a strong chance that Iron will pull in well ahead.

Bots that are likely to finish high are LetaBot, Microwave, PurpleWave, Steamhammer, UAlbertaBot, and ZZZKBot. ZZZKBot won CIG 2017 and is a perpetual contender. This tournament has more new bots than CIG, which favors ZZZKBot’s exceptionally well-implemented 4 pool—it smashes opponents which are not perfectly prepared. ZZZKBot is my pick as the most likely #2 finisher. Beyond that, it gets harder for me to forecast.

Steamhammer I think is likely to finish fairly high, but outside the top 3. It will lose almost all games versus Iron and PurpleWave, and some versus UAlbertaBot (which I expect to be stronger with the new SparCraft). Without opponent modeling and unable to adjust its strategy mixes, Steamhammer is also at risk of losing many games to newcomer bots. I see the opponent model as a necessary feature for a tournament, because unknown opponents might do anything, and known opponents might have surprises in hand.

But really, we don’t know. The tournament exists to find out.

Update: The roster page now says that 4 of the newcomer bots, plus AILien, either withdrew or did not submit. 27 contenders remain.

Update 2: Now AILien is listed as submitted after all. Whew.

choosing tournament maps

I doubt anyone will take my advice, and I certainly won’t take it myself, so I can offer it with complete freedom. Here’s what I would take into account in choosing maps if I were running a tournament.

Balance. You want the maps to be fair across races. We can’t use statistics from pro games to judge balance, because bot balance and pro balance are unrelated. Also bots are improving rapidly and the participant pools are small, plus we may choose some new maps in each tournament, so past tournament statistics are not too helpful for balance either. But the same graph I linked above, showing that bot balance and pro balance are different, also shows that bots have narrower imbalances. Or to say it differently: The maps may have imbalances, but bots suck at exploiting the imbalances. Choose enough maps, and the balance differences will average out statistically; it’s the same principle as balancing a portfolio of stocks. The 5 maps of CIG 2017 are not enough to convince me, but the 10 maps of AIIDE are probably enough.

Number of starting positions. For this year, maps were chosen like this:

tournament2 player3 player4 player
CIG1 (20%)2 (40%)2 (40%)
AIIDE3 (30%)2 (20%)5 (50%)
SSCAIT3 (21%)2 (14%)9 (64%)

Those all seem reasonable to me. I like SSCAIT’s ratios best. 2 player maps favor rush strategies and 4 player maps favor macro strategies, and you don’t want to emphasize either too much. One issue is that there aren’t many good 3 player maps (though the best ones are quite good).

Novelty versus consistency. If you carry some maps over from year to year, we can use them to (at least try to) measure balance changes and progress. If you introduce new maps, you pose a stronger test of adaptability. If I were choosing, I would pick some old standbys and a few unusual maps that the bots might not have played on before (or else run specialized tournaments and do both separately). CIG has done a good job of this, though I think it’s only a side effect of their process and not a deliberate decision.

Prodding bots to improve. Since bots are poor at exploiting map features, I want to include some maps with exploitable features to encourage authors to step up. Think of Iron failing on the map Hitchhiker at CIG 2017 because (as the author explained) BWEM did not grasp all the map features; do you think Iron will fail the same way next year? I proposed the map Namja Iyagi, which has 6 islands, as a map with exploitable features which is still playable by bots that do not understand islands. PurpleFistJadian suggested Outsider, which has the exploitable feature of pushing units through mineral lines and remains playable without. There are a lot of choices; the universe of pro maps is large.

No appearance of cheating. Sometimes the tournament organizers participate in the tournament. When not, the organizers may have a real or apparent interest in some participants: “Bot X uses method Y, which I’ve been pushing. So if X does well....” To avoid controversy, we may want map selection to make favoritism visibly difficult. So divide your universe of maps into classes depending on the other goals, and choose randomly from each class. We’ve seen the procedure of accepting a number from each participant, XORing the numbers together, and using that as the random number seed for a known generator, so that the process is transparent and tamper-resistant. It has never been clear to me whether the organizers actually follow the elaborate process. We often see map pools reused from year to year, probably to save time. Well, if there are no suspicions, then there is no reason to allay them. I have no reason to suspect that the tournaments are unfair, even unintentionally.

CIG 2017 results first look

The CIG 2017 tournament results are out. So far we have a results table and a slideshow of the conference presentation with a few interesting details. I expect the full results will be out before long. Here I summarize the results table in a way that emphasizes the tight grouping of finishers 4 through 11; they all scored nearly the same. There is a wide gap between the bots with plus scores and the lower half.

placewin ratebots
175%ZZZKBot
274%tscmoo
367%PurpleWave
4-1163-57%LetaBot, UAlbertaBot, MegaBot, Overkill, CasiaBot, Ziabot, Iron, Aiur
12-2046-9%McRave, Tyr, SRBotOne, TerranUAB, Bonjwa, Bigeyes, OpprimoBot, Sling, Salsa

The format this year was straight round robin with 20 entrants, 5 maps, and 125 rounds. With 125 games for each pair of opponents, 25 games on each map, slow machine learning algorithms had some data to bite into.

3 of the 5 maps are also used on SSCAIT: Tau Cross, Andromeda, and Python. The 2 player map Hitchhiker has a short rush distance and favors cheese. But if the game does not end early, then the narrow ravine between bases and the arrangement of map blocks calls for sophisticated play. I think Hitchhiker must be a difficult map for bots. The remaining map is 3 player Alchemist, which was also used last year. Each base has 2 ramp entrances, so a bot which wants to defend at “the” ramp may go wrong.

Discussion. The sophisticated 4-pooler ZZZKBot by Chris Coxe was the top winner. I imagine that having Hitchhiker in the map pool helped it. I’m curious to see its games and find out whether it had special-case strategies for specific opponents, as it has had in the past. Tscmoo played random for the first time and placed second. These 2 usually place high. PurpleWave came in third, an outstanding performance for a new entrant. Congratulations!

The pre-tournament favorite Iron did surprisingly poorly. It must have had a bug. I don’t know the cause, but my first guess is that it suffered on one of the maps. McRave is another bot which did not perform at its peak.

The most interesting detail in the slide show is a pair of graphs showing the effect of machine learning for opponent modeling. The first graph shows the win rates of the winners ZZZKBot and Tscmoo sagging toward the end of the tournament. The second shows MegaBot and SRBotOne soaring toward the end and says that they were the top scorers in the final rounds 120-125. In other words, if the tournament had continued long enough, the winners would have been completely different. One the one hand, this shows the power of machine leaning; on the other hand, it shows the slowness, because the long tournament was not long enough. In Steamhammer, I would like both fast adaptation and slow adaptation. The middlegame use of fast adaptation is almost working now, and I intend to use the same mechanism in a different way for opening selection. But there will be no time to add slow adaptation to fine-tune as more games accumulate.

I think that UAlbertaBot and Overkill were the 2015 versions. UAlbertaBot presumably had learning turned off. Those 2 and OpprimoBot have been constant for a while and can serve as benchmarks to judge progress. (AIUR is not as constant a benchmark because it has learning turned on.) In AIIDE 2015, the top finishers in order were Tscmoo, ZZZKBot, Overkill, UAlbertaBot. So in 2 years, former top bots have receded into the pack of above-average scorers.