Starcraft AI blog

SSCAIT 2016 is beyond the halfway point, and Steamhammer’s score ratio has been holding steady near 2:1, good enough to place it in the bottom half of the top 16. It will probably make it into the finals (no guarantee; some tough games are still ahead). Its rate of upsets suggests a 30% chance that it could win in the round of 16 and make it through to the round of 8. It is unlikely to get further than that.

I’ve seen a few people wonder how a brand-new bot could do so well. Steamhammer 0.2 has less than 3 weeks of my development effort in it. I can answer that!

1. Steamhammer builds on a strong foundation in UAlbertaBot. I changed the openings and the strategy followup, and fixed bad play where I had time to, but most of Steamhammer’s play comes straight from UAlbertaBot. Tactics are only slightly changed; micro has modestly improved targeting but is otherwise nearly identical.

2. I know the game and I’ve been following the bot scene for a long time. I was able to make choices that both fit within UAlbertaBot’s limitations and pose challenges to other bots. Knowledge is power. Most new bots play inefficient builds; Steamhammer builds inefficiently only to work around other problems.

3. Strong new bots usually show good results at first and then decline as other bots adapt. I’ve seen it before and I see it now with Steamhammer. The opening learning bots find the right openings to play; I played a test match versus Zia in which Steamhammer scored 1 1 1 1 1 0 0 0 0, switching from wins to losses when Zia hit on the right opening. The actively developed bots get fixes to any weaknesses that the new bot reveals; both Krasi0 and LetaBot have mentioned improvements to mutalisk defense.

I got my SSCAIT wish and saw the first Steamhammer versus Killerbot and Steamhammer versus XIMP games today.

Steamhammer versus Killerbot by Marian Devecka: Would my counter-build work? In my tests Steamhammer wins with it about half the time. Killerbot went with its +1 speed lings and Steamhammer opened 11 pool and held with its own slow lings while teching to lair. Timing is critical—12 pool is too slow on some maps! In this game Steamhammer managed to sneak in a drone kill while Killerbot’s lings were busy chasing the scouting drone (UAlbertaBot doesn’t do that natively, I had to improve the decision making). Steamhammer added 2 sunkens for protection while getting mutalisks—it still had only 1 hatchery, while Killerbot was headed for 3 hatcheries, and it is impossible to hold without static defense. Steamhammer’s first 3 mutalisks hatched and headed for the enemy base, and as soon as the flyers were safely out of range to intervene, Killerbot’s zerglings attacked! Steamhammer pulled drones to defend and lost every single drone, but the sunkens held and the base was left barren but secure for the moment. The 3 mutalisks killed all of Killerbot’s drones in return and then tore down the enemy zerg base with excruciating slowness—Steamhammer won a close game. Steamhammer-Killerbot games often go down to the wire like this in my testing; I’ve seen one side or the other win with only 1 or 2 buildings left bleeding onto the map. But without the special-purpose counter-build, Steamhammer loses every game; Killerbot is far superior in tactics and robustness.

Steamhammer versus XIMP by Tomas Vajda: Steamhammer opened with 3 hatcheries before pool, the only bot I’ve seen play this logical response to forge-expand. (Though I saw auxanic open unsoundly with 5 hatcheries before pool!) XIMP cannoned up for carriers, as always. I hadn’t tested this matchup, so I wasn’t sure what would happen. Steamhammer went hydras with +1 attack, and soon had enough to tear down the cannon wall (slightly before +1 finished, annoyingly), wipe out XIMP’s natural, and start on the main. That’s what I intended, and I’m pleased it worked. XIMP seemed about to fall, but suddenly the first carriers started to make a difference—they finally had enough interceptors. The hydras did not have critical mass to fight carriers and were cleared from the main. At this point, Steamhammer had 6 bases though not enough drones to saturate them, and XIMP was mining minerals with 6 probes and gas with 3. Steamhammer was still winning by a mile.

As the game went on, the carriers moved to the middle of the map and Steamhammer trickled hydras toward them, losing them a few at a time for no apparent gain. There were 2 obvious problems. First, Steamhammer only attacked interceptors; as I watched, I realized “carriers don’t attack, so they’re at the bottom in the target priority list.” The code is easy to rewrite, though I’m not sure what solution is best. And second, trickling the hydras in was suicidal. I concluded while watching that SparCraft must not know about carriers. It ignores units that it does not know about, so when asked to predict the combat outcome, SparCraft always reported “zerg wins,” and Steamhammer moved its hydras confidently to their deaths. That may be harder to fix.

Steamhammer then won when XIMP overstepped a time limit, taking too long on a frame. That was disappointing. :-( I’m not sure how the game would have turned out. Steamhammer was still ahead but was playing like an idiot, and XIMP did finally re-expand to its natural and may have been able to add carriers before Steamhammer could take the rest of the map and start to make progress.

After these two uncertain wins, Steamhammer played against protoss Ian Nicholas DaCosta, a weaker bot which it had beaten in their previous game. Steamhammer got unlucky with base positions and scouting, placed its 3rd hatchery directly on the path that zealots were taking toward its main, and lost. Luck runs both ways!

For convenience, links related to the SSCAIT 2016 tournament that’s going on now.

• SSCAIT front page
• official stream
• official results
• unofficial crosstable (very useful)
• OpenBW replays (I found this not so useful)

LetaBot posted the unofficial crosstable link in a comment.

I haven’t seen a plain explanation of the tournament format, although many people seem to understand it. There are 45 players. The tournament will be in 2 stages. The first stage or “student division” is 1980 games in a double round robin: Each bot plays each other twice. Top students in the student division can win prizes. Then the top 16 go into a second stage, the “mixed division,” played in single elimination style.

Among SSCAIT tournament watchers, the talk of the moment is of Iron’s two surprise defeats against zergs Steamhammer by me and GarmBot by Aurelien Lermant. Iron got unlucky 2 games in a row.

In the Steamhammer game, Iron became overcautious against the few early zerglings, bunkered its ramp, and was late with its vulture attack. Provoking reactions like that is one of the reasons Steamhammer makes zerglings. When the mutalisks came out, they soon had enough numbers to safely break the bunker and then were free to ravage Iron’s base (they could have gone ravaging from the start, but they were obsessed with the bunker). Iron’s vultures moved in and quickly wiped out every drone, but the mutalisk swarm split up, some cleaning vultures and the rest cleaning the terran base. Steamhammer won with empty bases but unchallenged air supremacy. In my testing, Steamhammer has never won against Iron with this build order—Iron got unlucky, or perhaps a late change weakened its play in this situation.

In the GarmBot game, GarmBot randomly expanded to Iron’s natural and built a sunken that covered the ramp. Iron was again overcautious: It double bunkered its ramp and blocked with vultures and many SCVs, and sent no vultures past the sunken. It seemed in no hurry to get tanks and blow away the sunken. Mutalisks showed up and started picking at a supply depot; nothing stopped them. But the lethal blow did not come until lurkers arrived and burrowed in good position on the ramp. There was no detection. Lurkers defeated the blocking units and cleared the bunkers while mutalisks killed the engineering bay, and then there was no hope. Iron reacted poorly to GarmBot’s bizarre expansion and GarmBot followed up well.

It’s a hard game! Even a bot as well-rounded as Iron is not prepared for everything.

The deadline is past. Here are the surprise eleventh-hour SSCAIT entrants this year:

• Aman Zargarpur - terran
• auxanic - zerg
• BeeBot - terran
• Tommy Fang - terran
• XelnagaII - protoss

Of these, XelnagaII is the only name I recognize. It finished in the middle of the pack in AIIDE 2016. As LetaBot says, BeeBot had a bug in its first upload which was fixed before the deadline, explaining its inconsistent results. It may be a dangerous mech bot. Auxanic has been around long enough for me to get an impression of its style: It is a single-minded macro zerg which doesn’t believe in making units for early defense. Auxanic can recover and win after having its main and tech destroyed, so at least it’s robust. Aman Zargarpur has played 1 game so far and Tommy Fang none yet.

I think we’re in for some variety, and that is good.

As LetaBot notes in the comments, they have returned to calling this the 2016 tournament, so that the sequence of numbers is unbroken. They called it 2017 until lately.

LetaBot is organizing another tournament on short notice: SSCAI man vs machine practice tournament. The format calls for human versus machine games only, a choice that I like.

The tournament is Saturday—tomorrow already! I hope bot authors will hurry and sign up! (My bot is unfortunately not ready; it can’t play all 3 matchups yet.)

The SSCAIT tournament deadline this year is 18 December. Like the 2015 edition last year, it will start in December and I expect final results in January. Curiously, they seem to have decided to skip a year in the numbering scheme, so this is the 2017 edition.

The final rush is on. The Usual Suspects are updating their bots. Marian Devecka, I’ve already noted, is back in the game. Tyr has been reuploaded for the first time since the middle of the year. Possibly a few bot authors are waiting until the last moment, hoping to gain an advantage with surprise updates.

The admins have been at work too. Some previously disabled bots are alive again: Awesomebot, DAIDOES, Ian Nicholas DaCosta’s bot, JompaBot, the 2013 bot by Oleg Ostroumov, PeregrineBot. The number of games used to track the current winning rate has been bumped from 20 to 50, so that the ranking changes more slowly but ends up more accurate.

I will be jumping in too. I got work underway a week ago and I’m rushing to finish a first primitive version. Stand by for the announcement.

Here is a csv file of rating differences and score ratios. It contains all the data that went into yesterday’s graph, plus the names of the opponents so we can see which bots are behind each dot. It is over 4MB for 80,798 games, hardly Big Data territory but not something you want to look through by eye either.

Krasi0 asked about the outliers in the upper left, where the weaker bot upset the stronger with a huge score ratio. I pulled out the 44 games where the weaker bot was at least 50 Elo behind and won with a score ratio of 100 or more. There’s quite a variety of bots and it looks like most of the losers are genuine strong opponents, but we can’t tell from this why they lost so badly.

winner	loser	rating_diff	score_ratio
Odin2014	Soeren Klett	-121.241105546074	102
Maja Nemsilajova	Gaoyuan Chen	-87.5441567081255	133.75
EradicatumXVR	Martin Rooijackers	-176.159988871814	134.75
Florian Richoux	Andrew Smith	-75.4167275722925	139.75
EradicatumXVR	Martin Rooijackers	-160.787872111647	123.6
Jakub Trancik	tscmooz	-234.397880730261	107.85
EradicatumXVR	Tomas Vajda	-310.393156864715	238.75
Odin2014	tscmoo	-341.595996071025	111.2
Jakub Trancik	Krasimir Krystev	-206.879443535575	118.5
Martin Rooijackers	Tomas Vajda	-185.169798041552	178
Martin Rooijackers	Tomas Vajda	-196.771893565481	196.5
Jakub Trancik	Florian Richoux	-201.533152887389	146
Martin Rooijackers	Tomas Vajda	-214.947508799781	194.5
EradicatumXVR	Soeren Klett	-59.609552771464	121.625
Martin Rooijackers	Tomas Vajda	-197.317290128813	173.75
EradicatumXVR	Martin Rooijackers	-153.322771550638	134
NUS Bot	Tomas Cere	-217.342932814072	102.25
EradicatumXVR	Martin Rooijackers	-299.534513717939	140
Martin Rooijackers	Tomas Vajda	-177.764917764824	171.5
Jakub Trancik	tscmoo	-114.300240687596	113.25
Martin Rooijackers	Tomas Vajda	-261.726998366137	185.5
Tomas Vajda	ICELab	-61.7649472971907	110.3
Maja Nemsilajova	Gaoyuan Chen	-444.702632047678	148.5
Jakub Trancik	Aurelien Lermant	-96.3561952340121	144
Adrian Sternmuller	Serega	-165.832601805773	127.6
Ian Nicholas DaCosta	Tomas Cere	-180.057678684451	109.25
OpprimoBot	Radim Bobek	-107.746261964435	187.75
Martin Rooijackers	Tomas Vajda	-276.603234707109	108.2
Florian Richoux	Martin Rooijackers	-117.736046527607	126
Igor Lacik	Gaoyuan Chen	-94.2338975266132	121.75
Florian Richoux	Andrew Smith	-186.917595581173	123.25
Radim Bobek	UPStarcraftAI	-62.9693177806437	136.5
Tomas Vajda	Andrew Smith	-79.8133863519054	152.771739130435
Jakub Trancik	Florian Richoux	-84.9235344515139	128
Martin Rooijackers	Tomas Vajda	-78.9815583248117	207
Jakub Trancik	PeregrineBot	-69.7933587952098	121.25
Dave Churchill	Andrew Smith	-94.5789654176522	180.75
Tomas Vajda	WuliBot	-52.5760915924302	182.5
Soeren Klett	Dave Churchill	-155.311160590808	361.25
Odin2014	PeregrineBot	-94.3129452431417	124.55
DAIDOES	MegaBot	-178.609296376345	115.875
OpprimoBot	Martin Rooijackers	-786.274344230815	307.75
Soeren Klett	Dave Churchill	-68.2015666358432	109

This scatter chart shows rating differences versus score ratios. For each SSCAIT game in the dataset which ended with both scores above zero (about 80,000 games), the x-axis has the rating difference, winner’s rating minus loser’s rating. The ratings are calculated as of that game, so changing versions don’t mess things up (much). Check the Elo table to see what the rating differences mean. The y-axis has the score ratio, winner’s score divided by loser’s score. The y-axis is on a logarithmic scale and shows ratios from 0.2 to 200. A small number of points are off the top or bottom; no points are off the left or right. You can click through the graph to get the same image on its own, which may make it easier to zoom in and out.

Since it’s from the winner’s point of view, most games have rating difference > 0 (the higher rated bot won) and score ratio > 1 (the bot with the higher score won).

There is complex structure here, but I’m at a loss to interpret much of it. Horizontal lines show that some score ratios are popular, which seems like a quirk of the game. Beyond the sharp lines, some fuzzier stratification in the score ratios is visible. When the stronger bot wins, it is usually by a score ratio of at least 2, increasing slowly as the strength differences goes up. The slowly rising “soft floor” in the score ratio is interesting and surprising. There are other clear structures in the chart, but I don’t know what they mean. It is mildly interesting that the left side, when the stronger bot lost, looks less structured.

Games where the score ratio is less than 1 are games where stopping early and adjudicating by score would give the wrong answer. It’s rare... but not rare enough to give me confidence in the timeout adjudication procedure. Some points are off the bottom, so some bots won despite having less than 1/5 the score by SSCAIT rules. The adjudication procedure will make occasional extreme mistakes.

Next: The winning attitude.

In the SSCAIT data, some scores are recorded as 0 or -100. I think 0 means that the bot scored no kills and no razings, so the game finished very early. Bots might legitimately lose with no kills against a rush, but I have to guess that in many cases it means the bot failed to start up or failed to do anything. I think -100 means the bot crashed. I also think that exceeding the time limit per frame ends the game without changing the scores, so -100 only means crashes, not time infractions. I don’t promise that my interpretation of the numbers is correct! I could be wrong!

Anyway, assuming that my interpretation is right, this table of -100 scores should tell us about reliability. Like yesterday, I include only games after 17 August, so that distant versions of the same bot are not lumped together. The game counts are different from yesterday, because yesterday’s table excludes games in which either side has a score of 0 or -100.

“Crashes” are games in which the bot had -100 score. “Double crashes” are games in which both sides had -100 score. “Fast crashes” are games in which the opponent had 0 score, so the crash must have happened very early or when the opponent failed to achieve anything. “Max opp score” is the largest score that the opponent had in any game where the bot crashed. “Other crashes” are crashes where the opponent has a positive score (all crashes except double crashes and fast crashes). “Mean pos opp score” is the average of the opponent’s score in these other crashes. Large opponent scores mean that the game went on for a long time before the crash. Since a crash scores as -100, there is no way to tell whether the bot was ahead or behind when it crashed.

bot	games	crashes	crash %	double crash %	fast crash %	max opp score	other crash %	mean pos opp score
krasi0	394	1	0.25%	0.25%	0.00%	-100	0.00%	-
Iron bot	331	8	2.42%	1.51%	0.00%	7225	0.91%	4367
Marian Devecka	302	30	9.93%	0.33%	0.00%	129110	9.60%	52320
Martin Rooijackers	336	2	0.60%	0.60%	0.00%	-100	0.00%	-
tscmooz	330	3	0.91%	0.91%	0.00%	-100	0.00%	-
tscmoo	332	1	0.30%	0.30%	0.00%	-100	0.00%	-
LetaBot CIG 2016	318	1	0.31%	0.31%	0.00%	-100	0.00%	-
WuliBot	304	3	0.99%	0.99%	0.00%	-100	0.00%	-
Simon Prins	318	19	5.97%	2.52%	3.46%	0	0.00%	-
ICELab	334	12	3.59%	0.30%	0.30%	3500	2.99%	2960
Sijia Xu	285	1	0.35%	0.35%	0.00%	-100	0.00%	-
LetaBot SSCAI 2015 Final	338	28	8.28%	0.89%	0.59%	60850	6.80%	16196
Dave Churchill	323		0.00%	0.00%	0.00%	-		-
Chris Coxe	223	1	0.45%	0.45%	0.00%	-100	0.00%	-
Tomas Vajda	317	16	5.05%	0.63%	0.32%	118435	4.10%	44220
Flash	306	2	0.65%	0.65%	0.00%	-100	0.00%	-
Zia bot	327	33	10.09%	0.92%	1.22%	45095	7.95%	23120
PeregrineBot	157		0.00%	0.00%	0.00%	-		-
tscmoop	328	2	0.61%	0.61%	0.00%	-100	0.00%	-
Andrew Smith	334	5	1.50%	1.50%	0.00%	-100	0.00%	-
Florian Richoux	308	31	10.06%	0.97%	0.32%	57405	8.77%	14962
Carsten Nielsen	347	4	1.15%	1.15%	0.00%	-100	0.00%	-
Soeren Klett	284	2	0.70%	0.35%	0.35%	-100	0.00%	-
Jakub Trancik	318		0.00%	0.00%	0.00%	-		-
Tomas Cere	345	3	0.87%	0.00%	0.58%	81825	0.29%	81825
MegaBot	305	54	17.70%	1.31%	2.95%	92450	13.44%	16394
Aurelien Lermant	349	4	1.15%	1.15%	0.00%	-100	0.00%	-
Odin2014	183	17	9.29%	0.55%	1.64%	38600	7.10%	22077
Gaoyuan Chen	329		0.00%	0.00%	0.00%	-		-
DAIDOES	136	8	5.88%	0.74%	0.74%	1750	4.41%	4742
Igor Lacik	144	13	9.03%	0.69%	0.00%	54100	8.33%	13421
Matej Istenik	323	1	0.31%	0.31%	0.00%	-100	0.00%	-
NUS Bot	137	3	2.19%	0.00%	0.00%	34175	2.19%	22400
Roman Danielis	304	3	0.99%	0.99%	0.00%	-100	0.00%	-
ZerGreenBot	36	25	69.44%	2.78%	8.33%	37025	58.33%	22087
Ian Nicholas DaCosta	150	1	0.67%	0.67%	0.00%	-100	0.00%	-
AwesomeBot	164	16	9.76%	1.22%	1.83%	27850	6.71%	7968
Johan Kayser	306	1	0.33%	0.00%	0.33%	0	0.00%	-
Martin Vlcak	151	19	12.58%	0.66%	11.92%	0	0.00%	-
Rob Bogie	135		0.00%	0.00%	0.00%	-		-
Christoffer Artmann	343	52	15.16%	0.58%	11.95%	700	2.62%	300
Marek Gajdos	175	68	38.86%	26.29%	12.00%	100	0.57%	100
Travis Shelton	155		0.00%	0.00%	0.00%	-		-
Bjorn P Mattsson	288	36	12.50%	0.00%	12.50%	0	0.00%	-
Vladimir Jurenka	349	37	10.60%	1.43%	2.87%	400	6.30%	264
neverdieTRX	170	1	0.59%	0.59%	0.00%	-100	0.00%	-
OpprimoBot	351	51	14.53%	0.57%	1.42%	38825	12.54%	17424
Sungguk Cha	320	168	52.50%	1.25%	4.69%	60675	46.56%	17463
Jacob Knudsen	177	56	31.64%	0.56%	3.39%	17750	27.68%	5957
HoangPhuc	151	115	76.16%	0.66%	5.96%	38050	69.54%	10765
ButcherBoy	142	35	24.65%	0.70%	7.04%	10000	16.90%	1364

The top bots are mostly reliable—except Marian Devecka’s Killerbot. My impression from watching games is that Killerbot may crash when losing. It makes sense that bots should crash less often when winning, because bot authors have more reason to fix crashes that lose winning games. The bottommost bots are all crash-prone. Not a coincidence, is it? Step 1 to a winning bot: Fix your crashing bugs!

We might imagine that double crashes are caused primarily by the game or some other part of the runtime system, but the high rate of double crashes by Marek Gajdos makes me question that. What is that bot breaking?

Tomorrow: I’ll try to correlate scores with ratings, one way or another.

I’m looking into the scores recorded in the SSCAIT game data, which I have up to 27 September. So far I haven’t found anything too interesting, but it’s not entirely useless either.

According to the SSCAIT rules, a player’s score is the sum of units killed plus buildings razed: BWAPI::Player::getKillScore() + BWAPI::Player::getRazingScore().

Here’s basic score information for the dates between 17 August 2016 and 27 September 2016. It’s the same date range I used in the SSCAIT crosstables, chosen so as not to smear too many different bot versions into one table. The difference and ratio columns are all arithmetic means. They give the difference or the ratio between the winner’s and loser’s scores. (Well, the loss score ratio is the ratio between the loser’s and winner’s scores, to make it easier to compare by eye.) Games in which either side had a score of 0 (no kills) or -100 (crash) are left out.

bot	games	win %	mean score	mean win score	mean loss score	win score diff	loss score diff	win score ratio	loss score ratio
krasi0	298	86.91%	54850	55676	49368	45776	9306	15.58	3.06
Iron bot	242	80.17%	23967	24835	20459	18644	-7699	16.77	4.06
Marian Devecka	211	92.42%	21120	22213	7798	9442	-29578	8.97	8.94
Martin Rooijackers	260	79.23%	25077	28722	11170	22046	-27686	9.14	13.59
tscmooz	247	73.68%	20322	25134	6847	12118	-15203	9.84	10.71
tscmoo	272	71.69%	35035	41591	18435	23035	-24015	5.49	6.53
LetaBot CIG 2016	256	73.05%	26635	30229	16896	22995	-21810	9.03	4.24
WuliBot	234	65.81%	8441	8627	8084	6814	-18057	17.22	12.69
Simon Prins	224	65.62%	24256	28058	16998	21971	-12391	17.68	12.18
ICELab	242	66.53%	36286	43099	22745	28818	-26350	6.47	10.22
Sijia Xu	236	63.98%	12119	14598	7715	9913	-23460	15.79	8.28
LetaBot SSCAI 2015 Final	236	64.41%	19400	24725	9764	18854	-17427	11.04	5.24
Dave Churchill	239	56.49%	6656	8412	4377	6394	-16370	17.41	11.89
Chris Coxe	175	57.71%	2989	3886	1763	3468	-3798	23.59	6.49
Tomas Vajda	230	64.78%	34027	39627	23727	34048	-19234	36.52	5.11
Flash	251	64.14%	11435	13872	7075	8335	-25305	7.78	9.89
Zia bot	236	51.69%	13082	17640	8205	8544	-15518	12.12	7.28
PeregrineBot	131	51.91%	3560	5415	1558	4651	-6587	22.77	12.65
tscmoop	258	52.33%	15653	23128	7448	8055	-30718	9.95	14.85
Andrew Smith	257	56.03%	15896	18391	12717	12180	-26508	8.84	7.42
Florian Richoux	223	53.36%	14775	22804	5588	8518	-22805	6.30	11.30
Carsten Nielsen	275	50.91%	10147	12121	8100	8140	-20057	11.68	6.89
Soeren Klett	228	45.18%	39869	57319	25490	43732	-12973	9.32	11.69
Jakub Trancik	204	45.10%	15552	15965	15212	11133	2909	14.55	6.41
Tomas Cere	281	44.13%	18317	32027	7488	18054	-28958	7.13	16.25
MegaBot	185	55.68%	16661	22464	9372	16042	-17473	9.21	16.45
Aurelien Lermant	288	37.15%	18580	36669	7886	-12822	-22641	1.25	12.13
Odin2014	131	46.56%	12552	16757	8887	8998	-16757	16.23	11.67
Gaoyuan Chen	258	39.53%	11417	16855	7861	8691	-26698	7.13	13.62
DAIDOES	101	27.72%	10794	23254	6015	17054	-10093	12.43	13.22
Igor Lacik	105	35.24%	13186	28378	4920	14545	-14483	7.66	7.40
Matej Istenik	260	29.62%	17509	32652	11137	19086	-11509	7.54	7.64
NUS Bot	96	32.29%	7069	12885	4295	8953	-14754	8.73	13.61
Roman Danielis	244	22.95%	19086	43268	11883	20858	-27151	3.83	10.30
ZerGreenBot	6	66.67%	9762	13554	2180	2291	-8045	3.44	10.87
Ian Nicholas DaCosta	116	16.38%	4048	10358	2812	6223	-8795	7.56	12.45
AwesomeBot	115	26.96%	8100	18712	4184	9280	-16898	3.11	20.40
Johan Kayser	249	17.67%	12965	35732	8078	23303	-10207	10.87	11.14
Martin Vlcak	102	31.37%	11386	20558	7194	10658	-15459	12.34	14.06
Rob Bogie	59	50.85%	12777	16248	9186	9555	-22662	13.03	4.92
Christoffer Artmann	197	17.26%	7712	24928	4121	15025	-15194	4.94	16.67
Marek Gajdos	72	11.11%	5187	16426	3782	12820	-9671	5.80	16.60
Travis Shelton	106	16.98%	7942	17816	5922	9374	-9455	3.51	17.10
Bjorn P Mattsson	190	18.95%	4995	14730	2719	7300	-15237	3.99	28.59
Vladimir Jurenka	142	28.17%	9561	15333	7297	8970	-11691	5.60	7.47
neverdieTRX	137	14.60%	8493	21777	6222	13883	-10783	4.55	11.22
OpprimoBot	218	12.39%	15672	25258	14317	17671	-5654	20.18	13.84
Sungguk Cha	122	25.41%	15263	34072	8855	21740	-11468	6.50	8.50
Jacob Knudsen	92	19.57%	7285	19250	4375	11994	-12193	5.08	18.43
HoangPhuc	12	91.67%	15148	16316	2300	9550	-22345	3.14	10.72
ButcherBoy	15	6.67%	1777	5655	1500	3105	-6543	2.22	18.09

The most striking point is that Krasi0 was ahead in points, on average, in the games that it lost. So was Jakub Trancik’s cannon bot. The data that I have does not record the cause of losses. It’s perfectly possible to lose while ahead on points, when you fight efficiently and destroy masses of enemy stuff before dying. But you may also be ahead on points when you crash or overstep the time limit.

Score increases as the game goes on. I think that the score diff columns mostly tell us how long the games were. So, for example, Marian Devecka’s Killerbot often won short games and hung on through a long fight in lost games. The score ratio columns seem more informative about how far ahead the winner was at the end of the game. Killerbot tended to win with about the same point ratio that it lost with. Krasi0 won with a huge point ratio and lost with a small ratio, which might reflect its defensive style. Iron, which is super-aggressive, also won with a huge ratio and lost with a small ratio, which in its case might mean that it won after a long series of pinpricks or lost in a sudden collapse.

The numbers are not easy to interpret! But they must mean something.

Tomorrow: I’ll try to dig out something about the rate of failing to start up.

Here is a list of the AIIDE 2016 players with upsets, cases in which a lower-ranked bot defeated a higher opponent. 15 of the 21 participants scored a total of 27 upsets out of 210 pairings. In most cases the upset was of an opponent only slightly ahead—the table doesn’t worry about the size of the upset. Xelnaga, well-positioned near the end of a run of bots with close scores, is the runaway upset champion; it also upset LetaBot 8 places ahead. To earn an upset you have to score worse overall, so another way to say it is that Xelnaga is the most inconsistent performer, with strengths and weaknesses that don’t always offset each other. The deepest upset is JiaBot’s upset of ZZZKBot, ranked 9 places ahead.

bot	upsets
2 ZZZKBot	1 Iron
4 LetaBot	3 tscmoo
5 UAlbertaBot	3 tscmoo
7 Overkill	2 ZZZKBot, 5 UAlbertaBot
8 Aiur	4 LetaBot, 7 Overkill
9 MegaBot	5 UAlbertaBot, 7 Overkill
10 IceBot	6 Ximp, 9 MegaBot
11 JiaBot	2 ZZZKBot, 9 MegaBot
12 Xelnaga	4 LetaBot, 7 Overkill, 9 MegaBot, 10 IceBot, 11 JiaBot
13 Skynet	11 JiaBot, 12 Xelnaga
14 GarmBot	9 MegaBot, 12 Xelnaga
15 NUSBot	8 Aiur, 13 Skynet
17 SRbotOne	15 NUSBot
19 Oritaka	14 GarmBot
20 CruzBot	15 NUSBot

Upsets are interesting because they suggest weaknesses in the stronger bot. They’re clues that may point toward what’s important to fix. For example, the upsets of #2 ZZZKBot by much less successful zergs remind us that 4 pool is easy for zerg to counter; protoss and terran need specific knowledge to counter the rush, but zerg can play a standard 9 pool (which in ZvZ is also fine against other openings) as a hard counter.

I counted the number of upsets in the crosstables for each map. An upset is when a lower-ranked player defeats one who ends up at a higher rank. On any given map you should expect more upsets than in the tournament overall, which sums over the maps and smooths out differences.

map	upsets
(2)Benzene	32
(2)Destination	16
(2)HeartbreakRidge	36
(3)Aztec	29
(3)TauCross	33
(4)Andromeda	32
(4)CircuitBreaker	37
(4)EmpireoftheSun	33
(4)Fortress	36
(4)Python	30

I expected that more standard maps would have fewer upsets, but it didn’t turn out that way. Less standard Heartbreak Ridge is the only map to tease me by acting as if I understood. I’m a little surprised by the high upset counts on standard Circuit Breaker and Fortress and the lower counts on less standard Aztec. I am very surprised that Destination has half the upsets of other maps! I can’t explain it.

Another theory you might try is: Many bots are tuned on SSCAIT, so SSCAIT maps might show more solid play and fewer upsets. The non-SSCAIT maps here are only Aztec and Fortress, which don’t support the hypothesis (though there’s little evidence either way).

I thought that per-map upset rate might be a way to measure the strategic fragility of bots on maps that they weren’t tuned for, but it may also measure code fragility. Look into yesterday’s crosstables to see where today’s totals came from. Heartbreak Ridge has many upsets not so much because bots weren’t ready for it, but because NUSBot wasn’t ready for it. LetaBot could not defend Fortress. Only Circuit Breaker’s high rate of upsets was not helped along by any single bot—and Circuit Breaker is as standard as they come. So we’re not learning as much about map characteristics as about bot characteristics.

Can anybody explain why Destination was more stable and less upset-prone than other maps? That’s the big mystery here.

AIIDE 2016 crosstables for each map in the tournament. First, the whole tournament crosstable, for reference. It should match the one on the official results page, though for some reason some of the overall win percentages differ in the second decimal (by a trivial amount). The official results are correct, so I may have some floating point error accumulation (an off-by-one error would cause a bigger discrepancy).

AIIDE 2016	overall	Iron	ZZZK	tscm	Leta	UAlb	Ximp	Over	Aiur	Mega	IceB	JiaB	Xeln	Skyn	Garm	NUSB	Terr	SRbo	Cime	Orit	Cruz	Tyr
Iron	87.44%		23%	84%	87%	86%	99%	88%	96%	68%	98%	81%	91%	57%	100%	96%	100%	99%	98%	100%	100%	100%
ZZZKBot	85.05%	77%		50%	51%	77%	98%	39%	87%	92%	93%	46%	100%	94%	100%	100%	100%	98%	100%	100%	100%	100%
tscmoo	82.71%	16%	50%		43%	42%	91%	97%	80%	90%	99%	91%	87%	83%	100%	98%	100%	100%	99%	89%	100%	100%
LetaBot	74.00%	13%	49%	57%		66%	68%	94%	34%	52%	84%	88%	39%	62%	93%	93%	92%	99%	99%	97%	100%	100%
UAlbertaBot	70.37%	14%	23%	58%	34%		76%	42%	80%	47%	77%	71%	100%	79%	81%	83%	99%	86%	59%	99%	100%	100%
Ximp	64.54%	1%	2%	9%	32%	24%		52%	81%	67%	7%	96%	62%	98%	94%	91%	98%	92%	96%	92%	97%	100%
Overkill	61.93%	12%	61%	3%	6%	58%	48%		30%	34%	79%	52%	24%	62%	94%	97%	96%	98%	94%	92%	100%	100%
Aiur	61.26%	4%	13%	20%	66%	20%	19%	70%		53%	79%	53%	56%	86%	87%	44%	88%	96%	92%	88%	91%	100%
MegaBot	58.42%	32%	8%	10%	48%	53%	33%	66%	47%		49%	42%	32%	78%	47%	91%	94%	83%	89%	84%	91%	90%
IceBot	57.43%	2%	7%	1%	16%	23%	93%	21%	21%	51%		56%	36%	61%	61%	100%	100%	100%	99%	100%	100%	100%
JiaBot	57.25%	19%	54%	9%	12%	29%	4%	48%	47%	58%	44%		49%	37%	98%	89%	90%	82%	99%	86%	91%	100%
Xelnaga	56.98%	9%	0%	13%	61%	0%	38%	76%	44%	68%	64%	51%		42%	2%	99%	100%	92%	94%	89%	96%	100%
Skynet	55.03%	43%	6%	17%	38%	21%	2%	38%	14%	22%	39%	63%	58%		100%	39%	100%	100%	100%	100%	100%	100%
GarmBot	42.52%	0%	0%	0%	7%	19%	6%	6%	13%	53%	39%	2%	98%	0%		84%	84%	100%	93%	47%	99%	100%
NUSBot	27.46%	4%	0%	2%	7%	17%	9%	3%	56%	9%	0%	11%	1%	61%	16%		57%	34%	61%	81%	30%	90%
TerranUAB	27.39%	0%	0%	0%	8%	1%	2%	4%	12%	6%	0%	10%	0%	0%	16%	43%		90%	73%	88%	96%	99%
SRbotOne	22.46%	1%	2%	0%	1%	14%	8%	2%	4%	17%	0%	18%	8%	0%	0%	66%	10%		53%	88%	57%	100%
Cimex	20.50%	2%	0%	1%	1%	41%	4%	6%	8%	11%	1%	1%	6%	0%	7%	39%	27%	47%		59%	51%	99%
Oritaka	19.51%	0%	0%	11%	3%	1%	8%	8%	12%	16%	0%	14%	11%	0%	53%	19%	12%	12%	41%		68%	100%
CruzBot	16.75%	0%	0%	0%	0%	0%	3%	0%	9%	9%	0%	9%	4%	0%	1%	70%	4%	43%	49%	32%		100%
Tyr	1.11%	0%	0%	0%	0%	0%	0%	0%	0%	10%	0%	0%	0%	0%	0%	10%	1%	0%	1%	0%	0%

Now each of the 10 maps. Each cell represents only 9 games, occasionally fewer when games are missing.

(2)Benzene	overall	Iron	ZZZK	tscm	Leta	UAlb	Ximp	Over	Aiur	Mega	IceB	JiaB	Xeln	Skyn	Garm	NUSB	Terr	SRbo	Cime	Orit	Cruz	Tyr
Iron	95.00%		89%	100%	78%	100%	100%	78%	100%	89%	100%	89%	100%	78%	100%	100%	100%	100%	100%	100%	100%	100%
ZZZKBot	73.89%	11%		22%	11%	78%	100%	11%	11%	89%	100%	67%	100%	78%	100%	100%	100%	100%	100%	100%	100%	100%
tscmoo	83.33%	0%	78%		22%	44%	100%	100%	78%	89%	100%	100%	78%	78%	100%	100%	100%	100%	100%	100%	100%	100%
LetaBot	81.67%	22%	89%	78%		67%	67%	100%	44%	33%	100%	100%	44%	89%	100%	100%	100%	100%	100%	100%	100%	100%
UAlbertaBot	64.80%	0%	22%	56%	33%		44%	22%	88%	44%	78%	67%	100%	78%	78%	89%	89%	67%	56%	89%	100%	100%
Ximp	59.22%	0%	0%	0%	33%	56%		25%	11%	78%	11%	100%	44%	100%	100%	33%	100%	100%	100%	89%	100%	100%
Overkill	68.93%	22%	89%	0%	0%	78%	75%		22%	44%	89%	78%	22%	100%	89%	100%	89%	100%	100%	89%	100%	100%
Aiur	72.07%	0%	89%	22%	56%	12%	89%	78%		78%	67%	78%	78%	89%	100%	33%	100%	89%	89%	100%	89%	100%
MegaBot	56.11%	11%	11%	11%	67%	56%	22%	56%	22%		22%	44%	22%	89%	56%	100%	89%	78%	100%	100%	67%	100%
IceBot	60.00%	0%	0%	0%	0%	22%	89%	11%	33%	78%		67%	33%	100%	67%	100%	100%	100%	100%	100%	100%	100%
JiaBot	52.78%	11%	33%	0%	0%	33%	0%	22%	22%	56%	33%		67%	56%	100%	67%	100%	89%	100%	89%	78%	100%
Xelnaga	56.67%	0%	0%	22%	56%	0%	56%	78%	22%	78%	67%	33%		67%	0%	100%	100%	100%	89%	89%	78%	100%
Skynet	45.81%	22%	22%	22%	11%	22%	0%	0%	11%	11%	0%	44%	33%		100%	11%	100%	100%	100%	100%	100%	100%
GarmBot	42.78%	0%	0%	0%	0%	22%	0%	11%	0%	44%	33%	0%	100%	0%		78%	100%	100%	100%	67%	100%	100%
NUSBot	31.11%	0%	0%	0%	0%	11%	67%	0%	67%	0%	0%	33%	0%	89%	22%		22%	33%	67%	100%	11%	100%
TerranUAB	27.22%	0%	0%	0%	0%	11%	0%	11%	0%	11%	0%	0%	0%	0%	0%	78%		100%	56%	89%	89%	100%
SRbotOne	20.56%	0%	0%	0%	0%	33%	0%	0%	11%	22%	0%	11%	0%	0%	0%	67%	0%		44%	100%	22%	100%
Cimex	22.78%	0%	0%	0%	0%	44%	0%	0%	11%	0%	0%	0%	11%	0%	0%	33%	44%	56%		67%	89%	100%
Oritaka	14.44%	0%	0%	0%	0%	11%	11%	11%	0%	0%	0%	11%	11%	0%	33%	0%	11%	0%	33%		56%	100%
CruzBot	21.23%	0%	0%	0%	0%	0%	0%	0%	11%	33%	0%	22%	22%	0%	0%	89%	11%	78%	11%	44%		100%
Tyr	0.00%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%

(2)Destination	overall	Iron	ZZZK	tscm	Leta	UAlb	Ximp	Over	Aiur	Mega	IceB	JiaB	Xeln	Skyn	Garm	NUSB	Terr	SRbo	Cime	Orit	Cruz	Tyr
Iron	91.67%		22%	89%	67%	89%	100%	100%	100%	78%	100%	89%	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%
ZZZKBot	89.44%	78%		56%	56%	100%	100%	44%	100%	100%	100%	67%	100%	89%	100%	100%	100%	100%	100%	100%	100%	100%
tscmoo	85.47%	11%	44%		44%	67%	100%	89%	67%	100%	100%	89%	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%
LetaBot	84.44%	33%	44%	56%		67%	89%	100%	56%	89%	100%	78%	78%	100%	100%	100%	100%	100%	100%	100%	100%	100%
UAlbertaBot	68.33%	11%	0%	33%	33%		56%	11%	78%	78%	78%	67%	100%	100%	89%	89%	100%	78%	67%	100%	100%	100%
Ximp	70.00%	0%	0%	0%	11%	44%		78%	100%	78%	11%	89%	89%	100%	100%	100%	100%	100%	100%	100%	100%	100%
Overkill	63.89%	0%	56%	11%	0%	89%	22%		44%	67%	78%	67%	33%	56%	78%	89%	100%	100%	89%	100%	100%	100%
Aiur	61.67%	0%	0%	33%	44%	22%	0%	56%		67%	67%	33%	100%	100%	89%	56%	100%	100%	78%	89%	100%	100%
MegaBot	50.56%	22%	0%	0%	11%	22%	22%	33%	33%		11%	67%	22%	89%	11%	89%	89%	100%	89%	100%	100%	100%
IceBot	60.56%	0%	0%	0%	0%	22%	89%	22%	33%	89%		56%	33%	89%	89%	100%	100%	100%	89%	100%	100%	100%
JiaBot	56.67%	11%	33%	11%	22%	33%	11%	33%	67%	33%	44%		67%	22%	100%	78%	89%	78%	100%	100%	100%	100%
Xelnaga	51.11%	0%	0%	0%	22%	0%	11%	67%	0%	78%	67%	33%		67%	11%	100%	100%	78%	100%	89%	100%	100%
Skynet	46.67%	0%	11%	0%	0%	0%	0%	44%	0%	11%	11%	78%	33%		100%	44%	100%	100%	100%	100%	100%	100%
GarmBot	43.89%	0%	0%	0%	0%	11%	0%	22%	11%	89%	11%	0%	89%	0%		67%	78%	100%	100%	100%	100%	100%
NUSBot	35.00%	0%	0%	0%	0%	11%	0%	11%	44%	11%	0%	22%	0%	56%	33%		89%	100%	78%	78%	67%	100%
TerranUAB	24.44%	0%	0%	0%	0%	0%	0%	0%	0%	11%	0%	11%	0%	0%	22%	11%		89%	78%	78%	89%	100%
SRbotOne	21.79%	0%	0%	0%	0%	22%	0%	0%	0%	0%	0%	22%	22%	0%	0%	0%	11%		78%	100%	78%	100%
Cimex	17.78%	0%	0%	0%	0%	33%	0%	11%	22%	11%	11%	0%	0%	0%	0%	22%	22%	22%		67%	33%	100%
Oritaka	13.89%	0%	0%	0%	0%	0%	0%	0%	11%	0%	0%	0%	11%	0%	0%	22%	22%	0%	33%		78%	100%
CruzBot	12.78%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	33%	11%	22%	67%	22%		100%
Tyr	0.00%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%

(2)Heartbreak Ridge	overall	Iron	ZZZK	tscm	Leta	UAlb	Ximp	Over	Aiur	Mega	IceB	JiaB	Xeln	Skyn	Garm	NUSB	Terr	SRbo	Cime	Orit	Cruz	Tyr
Iron	88.89%		22%	89%	89%	89%	100%	100%	100%	33%	100%	89%	78%	89%	100%	100%	100%	100%	100%	100%	100%	100%
ZZZKBot	87.78%	78%		78%	67%	78%	100%	44%	100%	100%	100%	11%	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%
tscmoo	80.56%	11%	22%		44%	56%	78%	100%	67%	89%	100%	89%	89%	89%	100%	100%	100%	100%	100%	78%	100%	100%
LetaBot	72.22%	11%	33%	56%		78%	44%	100%	22%	11%	89%	100%	0%	100%	100%	100%	100%	100%	100%	100%	100%	100%
UAlbertaBot	70.00%	11%	22%	44%	22%		56%	44%	89%	56%	78%	56%	100%	89%	100%	100%	100%	67%	67%	100%	100%	100%
Ximp	73.33%	0%	0%	22%	56%	44%		67%	100%	78%	11%	100%	89%	100%	100%	100%	100%	100%	100%	100%	100%	100%
Overkill	51.18%	0%	56%	0%	0%	56%	33%		0%	12%	83%	22%	12%	22%	100%	100%	89%	89%	89%	88%	100%	100%
Aiur	68.89%	0%	0%	33%	78%	11%	0%	100%		67%	100%	78%	78%	100%	78%	100%	78%	100%	100%	78%	100%	100%
MegaBot	54.19%	67%	0%	11%	89%	44%	22%	88%	33%		100%	44%	22%	11%	44%	100%	100%	100%	100%	22%	89%	0%
IceBot	54.80%	0%	0%	0%	11%	22%	89%	17%	0%	0%		56%	11%	89%	89%	100%	100%	100%	100%	100%	100%	100%
JiaBot	58.33%	11%	89%	11%	0%	44%	0%	78%	22%	56%	44%		44%	22%	100%	100%	100%	67%	100%	89%	89%	100%
Xelnaga	60.89%	22%	0%	11%	100%	0%	11%	88%	22%	78%	89%	56%		67%	0%	100%	100%	100%	100%	89%	89%	100%
Skynet	56.11%	11%	0%	11%	0%	11%	0%	78%	0%	89%	11%	78%	33%		100%	100%	100%	100%	100%	100%	100%	100%
GarmBot	38.33%	0%	0%	0%	0%	0%	0%	0%	22%	56%	11%	0%	100%	0%		100%	78%	100%	100%	11%	89%	100%
NUSBot	0.00%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%		0%	0%	0%	0%	0%	0%
TerranUAB	30.56%	0%	0%	0%	0%	0%	0%	11%	22%	0%	0%	0%	0%	0%	22%	100%		89%	78%	100%	89%	100%
SRbotOne	22.22%	0%	0%	0%	0%	33%	0%	11%	0%	0%	0%	33%	0%	0%	0%	100%	11%		44%	67%	44%	100%
Cimex	21.67%	0%	0%	0%	0%	33%	0%	11%	0%	0%	0%	0%	0%	0%	0%	100%	22%	56%		56%	56%	100%
Oritaka	28.49%	0%	0%	22%	0%	0%	0%	12%	22%	78%	0%	11%	11%	0%	89%	100%	0%	33%	44%		44%	100%
CruzBot	20.79%	0%	0%	0%	0%	0%	0%	0%	0%	11%	0%	11%	11%	0%	11%	100%	11%	56%	44%	56%		100%
Tyr	10.06%	0%	0%	0%	0%	0%	0%	0%	0%	100%	0%	0%	0%	0%	0%	100%	0%	0%	0%	0%	0%

(3)Aztec	overall	Iron	ZZZK	tscm	Leta	UAlb	Ximp	Over	Aiur	Mega	IceB	JiaB	Xeln	Skyn	Garm	NUSB	Terr	SRbo	Cime	Orit	Cruz	Tyr
Iron	91.11%		11%	78%	89%	100%	100%	89%	100%	89%	100%	89%	89%	89%	100%	100%	100%	100%	100%	100%	100%	100%
ZZZKBot	85.56%	89%		56%	44%	89%	100%	44%	78%	89%	100%	22%	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%
tscmoo	81.67%	22%	44%		22%	56%	78%	100%	67%	89%	100%	100%	100%	67%	100%	89%	100%	100%	100%	100%	100%	100%
LetaBot	78.33%	11%	56%	78%		44%	89%	100%	33%	56%	100%	100%	11%	100%	100%	89%	100%	100%	100%	100%	100%	100%
UAlbertaBot	65.56%	0%	11%	44%	56%		100%	11%	44%	56%	56%	89%	100%	78%	56%	89%	100%	89%	33%	100%	100%	100%
Ximp	62.78%	0%	0%	22%	11%	0%		67%	100%	67%	0%	78%	89%	100%	89%	100%	89%	78%	78%	100%	89%	100%
Overkill	60.00%	11%	56%	0%	0%	89%	33%		22%	11%	100%	56%	11%	67%	89%	100%	100%	100%	100%	56%	100%	100%
Aiur	63.89%	0%	22%	33%	67%	56%	0%	78%		67%	44%	44%	67%	89%	89%	67%	89%	89%	100%	89%	89%	100%
MegaBot	60.00%	11%	11%	11%	44%	44%	33%	89%	33%		67%	22%	11%	89%	56%	100%	100%	100%	89%	100%	89%	100%
IceBot	59.44%	0%	0%	0%	0%	44%	100%	0%	56%	33%		44%	33%	100%	78%	100%	100%	100%	100%	100%	100%	100%
JiaBot	60.00%	11%	78%	0%	0%	11%	22%	44%	56%	78%	56%		56%	56%	89%	78%	89%	100%	100%	100%	78%	100%
Xelnaga	57.78%	11%	0%	0%	89%	0%	11%	89%	33%	89%	67%	44%		33%	0%	100%	100%	100%	89%	100%	100%	100%
Skynet	48.89%	11%	0%	33%	0%	22%	0%	33%	11%	11%	0%	44%	67%		100%	44%	100%	100%	100%	100%	100%	100%
GarmBot	45.00%	0%	0%	0%	0%	44%	11%	11%	11%	44%	22%	11%	100%	0%		100%	100%	100%	100%	44%	100%	100%
NUSBot	22.22%	0%	0%	11%	11%	11%	0%	0%	33%	0%	0%	22%	0%	56%	0%		11%	22%	44%	89%	33%	100%
TerranUAB	27.78%	0%	0%	0%	0%	0%	11%	0%	11%	0%	0%	11%	0%	0%	0%	89%		89%	67%	89%	89%	100%
SRbotOne	23.89%	0%	0%	0%	0%	11%	22%	0%	11%	0%	0%	0%	0%	0%	0%	78%	11%		67%	100%	78%	100%
Cimex	24.44%	0%	0%	0%	0%	67%	22%	0%	0%	11%	0%	0%	11%	0%	0%	56%	33%	33%		78%	78%	100%
Oritaka	16.67%	0%	0%	0%	0%	0%	0%	44%	11%	0%	0%	0%	0%	0%	56%	11%	11%	0%	22%		78%	100%
CruzBot	15.00%	0%	0%	0%	0%	0%	11%	0%	11%	11%	0%	22%	0%	0%	0%	67%	11%	22%	22%	22%		100%
Tyr	0.00%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%

(3)Tau Cross	overall	Iron	ZZZK	tscm	Leta	UAlb	Ximp	Over	Aiur	Mega	IceB	JiaB	Xeln	Skyn	Garm	NUSB	Terr	SRbo	Cime	Orit	Cruz	Tyr
Iron	86.67%		22%	100%	89%	67%	100%	89%	89%	44%	100%	78%	67%	89%	100%	100%	100%	100%	100%	100%	100%	100%
ZZZKBot	83.80%	78%		44%	12%	89%	100%	22%	89%	100%	100%	44%	100%	100%	100%	100%	100%	89%	100%	100%	100%	100%
tscmoo	84.44%	0%	56%		44%	78%	89%	89%	100%	100%	100%	100%	78%	89%	100%	100%	100%	100%	100%	67%	100%	100%
LetaBot	76.70%	11%	88%	56%		78%	44%	100%	44%	33%	89%	100%	22%	89%	88%	100%	100%	100%	100%	100%	100%	100%
UAlbertaBot	67.22%	33%	11%	22%	22%		44%	56%	78%	22%	67%	67%	100%	78%	100%	56%	100%	100%	89%	100%	100%	100%
Ximp	70.56%	0%	0%	11%	56%	56%		56%	78%	67%	11%	100%	78%	100%	100%	100%	100%	100%	100%	100%	100%	100%
Overkill	59.78%	11%	78%	11%	0%	44%	44%		22%	33%	56%	33%	44%	44%	100%	100%	89%	100%	89%	89%	100%	100%
Aiur	61.67%	11%	11%	0%	56%	22%	22%	78%		67%	89%	56%	67%	100%	78%	44%	67%	100%	100%	78%	89%	100%
MegaBot	63.89%	56%	0%	0%	67%	78%	33%	67%	33%		56%	44%	33%	89%	56%	89%	100%	100%	89%	89%	100%	100%
IceBot	58.89%	0%	0%	0%	11%	33%	89%	44%	11%	44%		67%	44%	89%	44%	100%	100%	100%	100%	100%	100%	100%
JiaBot	53.63%	22%	56%	0%	0%	33%	0%	67%	44%	56%	33%		33%	11%	100%	100%	67%	78%	100%	78%	89%	100%
Xelnaga	56.11%	33%	0%	22%	78%	0%	22%	56%	33%	67%	56%	67%		22%	0%	100%	100%	100%	100%	67%	100%	100%
Skynet	50.56%	11%	0%	11%	11%	22%	0%	56%	0%	11%	11%	89%	78%		100%	11%	100%	100%	100%	100%	100%	100%
GarmBot	40.78%	0%	0%	0%	12%	0%	0%	0%	22%	44%	56%	0%	100%	0%		67%	78%	100%	100%	33%	100%	100%
NUSBot	34.44%	0%	0%	0%	0%	44%	0%	0%	56%	11%	0%	0%	0%	89%	33%		100%	33%	100%	89%	33%	100%
TerranUAB	26.67%	0%	0%	0%	0%	0%	0%	11%	33%	0%	0%	33%	0%	0%	22%	0%		100%	67%	67%	100%	100%
SRbotOne	17.78%	0%	11%	0%	0%	0%	0%	0%	0%	0%	0%	22%	0%	0%	0%	67%	0%		67%	78%	11%	100%
Cimex	15.56%	0%	0%	0%	0%	11%	0%	11%	0%	11%	0%	0%	0%	0%	0%	0%	33%	33%		56%	56%	100%
Oritaka	25.56%	0%	0%	33%	0%	0%	0%	11%	22%	11%	0%	22%	33%	0%	67%	11%	33%	22%	44%		100%	100%
CruzBot	16.11%	0%	0%	0%	0%	0%	0%	0%	11%	0%	0%	11%	0%	0%	0%	67%	0%	89%	44%	0%		100%
Tyr	0.00%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%

(4)Andromeda	overall	Iron	ZZZK	tscm	Leta	UAlb	Ximp	Over	Aiur	Mega	IceB	JiaB	Xeln	Skyn	Garm	NUSB	Terr	SRbo	Cime	Orit	Cruz	Tyr
Iron	82.78%		22%	56%	78%	78%	89%	100%	100%	56%	100%	67%	100%	11%	100%	100%	100%	100%	100%	100%	100%	100%
ZZZKBot	82.78%	78%		33%	22%	67%	100%	67%	100%	78%	67%	56%	100%	89%	100%	100%	100%	100%	100%	100%	100%	100%
tscmoo	85.56%	44%	67%		44%	33%	89%	100%	100%	89%	89%	100%	78%	78%	100%	100%	100%	100%	100%	100%	100%	100%
LetaBot	81.11%	22%	78%	56%		89%	78%	100%	33%	78%	89%	100%	89%	33%	89%	100%	100%	100%	100%	89%	100%	100%
UAlbertaBot	73.33%	22%	33%	67%	11%		100%	56%	89%	33%	89%	78%	100%	78%	78%	89%	100%	89%	56%	100%	100%	100%
Ximp	60.56%	11%	0%	11%	22%	0%		56%	89%	89%	0%	100%	67%	89%	78%	100%	100%	78%	100%	44%	78%	100%
Overkill	62.78%	0%	33%	0%	0%	44%	44%		44%	44%	78%	78%	67%	44%	100%	100%	100%	89%	89%	100%	100%	100%
Aiur	55.56%	0%	0%	0%	67%	11%	11%	56%		33%	89%	56%	44%	78%	78%	11%	89%	100%	100%	89%	100%	100%
MegaBot	57.22%	44%	22%	11%	22%	67%	11%	56%	67%		22%	33%	56%	78%	33%	78%	100%	78%	89%	89%	89%	100%
IceBot	57.22%	0%	33%	11%	11%	11%	100%	22%	11%	78%		44%	67%	22%	33%	100%	100%	100%	100%	100%	100%	100%
JiaBot	55.00%	33%	44%	0%	0%	22%	0%	22%	44%	67%	56%		56%	33%	100%	89%	78%	89%	100%	67%	100%	100%
Xelnaga	45.00%	0%	0%	22%	11%	0%	33%	33%	56%	44%	33%	44%		44%	0%	100%	100%	67%	67%	56%	89%	100%
Skynet	62.78%	89%	11%	22%	67%	22%	11%	56%	22%	22%	78%	67%	56%		100%	33%	100%	100%	100%	100%	100%	100%
GarmBot	46.11%	0%	0%	0%	11%	22%	22%	0%	22%	67%	67%	0%	100%	0%		89%	78%	100%	100%	44%	100%	100%
NUSBot	29.44%	0%	0%	0%	0%	11%	0%	0%	89%	22%	0%	11%	0%	67%	11%		78%	11%	100%	78%	11%	100%
TerranUAB	28.89%	0%	0%	0%	0%	0%	0%	0%	11%	0%	0%	22%	0%	0%	22%	22%		100%	100%	100%	100%	100%
SRbotOne	25.56%	0%	0%	0%	0%	11%	22%	11%	0%	22%	0%	11%	33%	0%	0%	89%	0%		33%	89%	89%	100%
Cimex	17.78%	0%	0%	0%	0%	44%	0%	11%	0%	11%	0%	0%	33%	0%	0%	0%	0%	67%		33%	56%	100%
Oritaka	25.56%	0%	0%	0%	11%	0%	56%	0%	11%	11%	0%	33%	44%	0%	56%	22%	0%	11%	67%		89%	100%
CruzBot	15.00%	0%	0%	0%	0%	0%	22%	0%	0%	11%	0%	0%	11%	0%	0%	89%	0%	11%	44%	11%		100%
Tyr	0.00%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%

(4)Circuit Breaker	overall	Iron	ZZZK	tscm	Leta	UAlb	Ximp	Over	Aiur	Mega	IceB	JiaB	Xeln	Skyn	Garm	NUSB	Terr	SRbo	Cime	Orit	Cruz	Tyr
Iron	85.00%		44%	100%	89%	89%	100%	67%	100%	67%	89%	78%	100%	11%	100%	78%	100%	89%	100%	100%	100%	100%
ZZZKBot	83.33%	56%		67%	44%	56%	89%	44%	100%	89%	78%	44%	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%
tscmoo	79.44%	0%	33%		22%	33%	100%	100%	89%	78%	100%	67%	89%	78%	100%	100%	100%	100%	100%	100%	100%	100%
LetaBot	81.67%	11%	56%	78%		67%	89%	100%	33%	89%	100%	100%	78%	33%	100%	100%	100%	100%	100%	100%	100%	100%
UAlbertaBot	70.56%	11%	44%	67%	33%		89%	22%	89%	22%	78%	78%	100%	78%	67%	89%	100%	100%	44%	100%	100%	100%
Ximp	56.67%	0%	11%	0%	11%	11%		33%	56%	44%	0%	100%	44%	100%	89%	100%	89%	67%	78%	100%	100%	100%
Overkill	62.78%	33%	56%	0%	0%	78%	67%		0%	11%	78%	67%	11%	56%	100%	100%	100%	100%	100%	100%	100%	100%
Aiur	60.56%	0%	0%	11%	67%	11%	44%	100%		44%	89%	22%	33%	89%	100%	44%	89%	89%	100%	100%	78%	100%
MegaBot	64.44%	33%	11%	22%	11%	78%	56%	89%	56%		67%	56%	33%	67%	67%	89%	100%	67%	89%	100%	100%	100%
IceBot	53.33%	11%	22%	0%	0%	22%	100%	22%	11%	33%		44%	33%	33%	33%	100%	100%	100%	100%	100%	100%	100%
JiaBot	56.67%	22%	56%	33%	0%	22%	0%	33%	78%	44%	56%		22%	22%	89%	100%	89%	100%	100%	78%	89%	100%
Xelnaga	58.33%	0%	0%	11%	22%	0%	56%	89%	67%	67%	67%	78%		22%	0%	100%	100%	89%	100%	100%	100%	100%
Skynet	62.22%	89%	0%	22%	67%	22%	0%	44%	11%	33%	67%	78%	78%		100%	33%	100%	100%	100%	100%	100%	100%
GarmBot	42.78%	0%	0%	0%	0%	33%	11%	0%	0%	33%	67%	11%	100%	0%		100%	89%	100%	100%	11%	100%	100%
NUSBot	31.67%	22%	0%	0%	0%	11%	0%	0%	56%	11%	0%	0%	0%	67%	0%		67%	67%	56%	100%	78%	100%
TerranUAB	27.22%	0%	0%	0%	0%	0%	11%	0%	11%	0%	0%	11%	0%	0%	11%	33%		89%	78%	100%	100%	100%
SRbotOne	21.67%	11%	0%	0%	0%	0%	33%	0%	11%	33%	0%	0%	11%	0%	0%	33%	11%		33%	89%	67%	100%
Cimex	19.44%	0%	0%	0%	0%	56%	22%	0%	0%	11%	0%	0%	0%	0%	0%	44%	22%	67%		22%	44%	100%
Oritaka	17.78%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	22%	0%	0%	89%	0%	0%	11%	78%		56%	100%
CruzBot	14.44%	0%	0%	0%	0%	0%	0%	0%	22%	0%	0%	11%	0%	0%	0%	22%	0%	33%	56%	44%		100%
Tyr	0.00%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%

(4)Empire of the Sun	overall	Iron	ZZZK	tscm	Leta	UAlb	Ximp	Over	Aiur	Mega	IceB	JiaB	Xeln	Skyn	Garm	NUSB	Terr	SRbo	Cime	Orit	Cruz	Tyr
Iron	86.11%		0%	100%	89%	67%	100%	89%	100%	100%	100%	89%	89%	11%	100%	89%	100%	100%	100%	100%	100%	100%
ZZZKBot	85.00%	100%		22%	67%	67%	89%	33%	89%	89%	89%	56%	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%
tscmoo	81.67%	0%	78%		44%	0%	100%	100%	78%	100%	100%	89%	67%	89%	100%	100%	100%	100%	100%	89%	100%	100%
LetaBot	72.78%	11%	33%	56%		44%	78%	100%	56%	67%	89%	100%	22%	22%	100%	78%	100%	100%	100%	100%	100%	100%
UAlbertaBot	76.11%	33%	33%	100%	56%		100%	78%	100%	44%	78%	67%	100%	78%	67%	56%	100%	100%	33%	100%	100%	100%
Ximp	63.89%	0%	11%	0%	22%	0%		56%	89%	56%	0%	89%	56%	100%	100%	100%	100%	100%	100%	100%	100%	100%
Overkill	58.33%	11%	67%	0%	0%	22%	44%		11%	33%	56%	56%	11%	78%	89%	100%	89%	100%	100%	100%	100%	100%
Aiur	56.67%	0%	11%	22%	44%	0%	11%	89%		56%	78%	56%	22%	89%	89%	22%	89%	100%	67%	89%	100%	100%
MegaBot	51.67%	0%	11%	0%	33%	56%	44%	67%	44%		56%	22%	22%	78%	11%	100%	89%	78%	67%	67%	89%	100%
IceBot	53.33%	0%	11%	0%	11%	22%	100%	44%	22%	44%		33%	11%	22%	44%	100%	100%	100%	100%	100%	100%	100%
JiaBot	60.56%	11%	44%	11%	0%	33%	11%	44%	44%	78%	67%		67%	67%	100%	100%	100%	78%	100%	67%	89%	100%
Xelnaga	63.33%	11%	0%	33%	78%	0%	44%	89%	78%	78%	89%	33%		22%	11%	100%	100%	100%	100%	100%	100%	100%
Skynet	59.44%	89%	0%	11%	78%	22%	0%	22%	11%	22%	78%	33%	78%		100%	44%	100%	100%	100%	100%	100%	100%
GarmBot	43.89%	0%	0%	0%	0%	33%	0%	11%	11%	89%	56%	0%	89%	0%		89%	89%	100%	89%	22%	100%	100%
NUSBot	27.78%	11%	0%	0%	22%	44%	0%	0%	78%	0%	0%	0%	0%	56%	11%		44%	56%	44%	78%	11%	100%
TerranUAB	28.33%	0%	0%	0%	0%	0%	0%	11%	11%	11%	0%	0%	0%	0%	11%	56%		67%	100%	100%	100%	100%
SRbotOne	21.11%	0%	0%	0%	0%	0%	0%	0%	0%	22%	0%	22%	0%	0%	0%	44%	33%		67%	89%	44%	100%
Cimex	21.67%	0%	0%	0%	0%	67%	0%	0%	33%	33%	0%	0%	0%	0%	11%	56%	0%	33%		56%	44%	100%
Oritaka	21.11%	0%	0%	11%	0%	0%	0%	0%	11%	33%	0%	33%	0%	0%	78%	22%	0%	11%	44%		78%	100%
CruzBot	17.22%	0%	0%	0%	0%	0%	0%	0%	0%	11%	0%	11%	0%	0%	0%	89%	0%	56%	56%	22%		100%
Tyr	0.00%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%

(4)Fortress	overall	Iron	ZZZK	tscm	Leta	UAlb	Ximp	Over	Aiur	Mega	IceB	JiaB	Xeln	Skyn	Garm	NUSB	Terr	SRbo	Cime	Orit	Cruz	Tyr
Iron	81.11%		0%	33%	100%	78%	100%	78%	89%	56%	89%	67%	100%	67%	100%	89%	100%	100%	78%	100%	100%	100%
ZZZKBot	88.33%	100%		56%	100%	67%	100%	22%	100%	89%	100%	44%	100%	89%	100%	100%	100%	100%	100%	100%	100%	100%
tscmoo	86.11%	67%	44%		89%	33%	89%	100%	78%	78%	100%	89%	89%	78%	100%	89%	100%	100%	100%	100%	100%	100%
LetaBot	39.44%	0%	0%	11%		78%	0%	44%	0%	22%	0%	0%	22%	0%	56%	78%	22%	89%	89%	78%	100%	100%
UAlbertaBot	75.00%	22%	33%	67%	22%		100%	78%	89%	33%	78%	89%	100%	78%	100%	78%	100%	78%	56%	100%	100%	100%
Ximp	63.89%	0%	0%	11%	100%	0%		22%	89%	44%	11%	100%	56%	89%	89%	78%	100%	100%	100%	89%	100%	100%
Overkill	68.89%	22%	78%	0%	56%	22%	78%		78%	22%	89%	22%	22%	89%	100%	100%	100%	100%	100%	100%	100%	100%
Aiur	55.56%	11%	0%	22%	100%	11%	11%	22%		22%	78%	56%	33%	67%	78%	44%	89%	89%	100%	100%	78%	100%
MegaBot	70.00%	44%	11%	22%	78%	67%	56%	78%	78%		56%	33%	67%	89%	78%	89%	89%	89%	89%	89%	100%	100%
IceBot	61.11%	11%	0%	0%	100%	22%	89%	11%	22%	44%		56%	33%	44%	89%	100%	100%	100%	100%	100%	100%	100%
JiaBot	62.78%	33%	56%	11%	100%	11%	0%	78%	44%	67%	44%		44%	33%	100%	78%	89%	67%	100%	100%	100%	100%
Xelnaga	56.67%	0%	0%	11%	78%	0%	44%	78%	67%	33%	67%	56%		22%	0%	89%	100%	89%	100%	100%	100%	100%
Skynet	59.44%	33%	11%	22%	100%	22%	11%	11%	33%	11%	56%	67%	78%		100%	33%	100%	100%	100%	100%	100%	100%
GarmBot	39.44%	0%	0%	0%	44%	0%	11%	0%	22%	22%	11%	0%	100%	0%		56%	67%	100%	56%	100%	100%	100%
NUSBot	33.89%	11%	0%	11%	22%	22%	22%	0%	56%	11%	0%	22%	11%	67%	44%		67%	22%	56%	100%	33%	100%
TerranUAB	29.44%	0%	0%	0%	78%	0%	0%	0%	11%	11%	0%	11%	0%	0%	33%	33%		89%	56%	67%	100%	100%
SRbotOne	23.89%	0%	0%	0%	11%	22%	0%	0%	11%	11%	0%	33%	11%	0%	0%	78%	11%		56%	67%	67%	100%
Cimex	23.89%	22%	0%	0%	11%	44%	0%	0%	0%	11%	0%	0%	0%	0%	44%	44%	44%	44%		100%	22%	89%
Oritaka	11.67%	0%	0%	0%	22%	0%	11%	0%	0%	11%	0%	0%	0%	0%	0%	0%	33%	33%	0%		22%	100%
CruzBot	18.89%	0%	0%	0%	0%	0%	0%	0%	22%	0%	0%	0%	0%	0%	0%	67%	0%	33%	78%	78%		100%
Tyr	0.56%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	11%	0%	0%

(4)Python	overall	Iron	ZZZK	tscm	Leta	UAlb	Ximp	Over	Aiur	Mega	IceB	JiaB	Xeln	Skyn	Garm	NUSB	Terr	SRbo	Cime	Orit	Cruz	Tyr
Iron	86.11%		0%	100%	100%	100%	100%	89%	78%	67%	100%	78%	89%	22%	100%	100%	100%	100%	100%	100%	100%	100%
ZZZKBot	90.56%	100%		67%	78%	78%	100%	56%	100%	100%	100%	44%	100%	100%	100%	100%	100%	89%	100%	100%	100%	100%
tscmoo	78.89%	0%	33%		56%	22%	89%	89%	78%	89%	100%	89%	100%	89%	100%	100%	100%	100%	89%	56%	100%	100%
LetaBot	71.67%	0%	22%	44%		44%	100%	100%	22%	44%	89%	100%	22%	56%	100%	89%	100%	100%	100%	100%	100%	100%
UAlbertaBot	72.78%	0%	22%	78%	56%		67%	44%	56%	78%	89%	56%	100%	56%	78%	100%	100%	89%	89%	100%	100%	100%
Ximp	64.44%	0%	0%	11%	0%	33%		56%	100%	67%	11%	100%	11%	100%	100%	100%	100%	100%	100%	100%	100%	100%
Overkill	62.22%	11%	44%	11%	0%	56%	44%		56%	56%	89%	44%	0%	67%	100%	78%	100%	100%	89%	100%	100%	100%
Aiur	56.11%	22%	0%	22%	78%	44%	0%	44%		33%	89%	56%	33%	56%	89%	22%	89%	100%	89%	67%	89%	100%
MegaBot	56.11%	33%	0%	11%	56%	22%	33%	44%	67%		33%	56%	33%	100%	56%	78%	89%	44%	89%	89%	89%	100%
IceBot	55.56%	0%	0%	0%	11%	11%	89%	11%	11%	67%		89%	56%	22%	44%	100%	100%	100%	100%	100%	100%	100%
JiaBot	56.11%	22%	56%	11%	0%	44%	0%	56%	44%	44%	11%		33%	44%	100%	100%	100%	78%	89%	89%	100%	100%
Xelnaga	63.89%	11%	0%	0%	78%	0%	89%	100%	67%	67%	44%	67%		56%	0%	100%	100%	100%	100%	100%	100%	100%
Skynet	58.33%	78%	0%	11%	44%	44%	0%	33%	44%	0%	78%	56%	44%		100%	33%	100%	100%	100%	100%	100%	100%
GarmBot	42.22%	0%	0%	0%	0%	22%	0%	0%	11%	44%	56%	0%	100%	0%		100%	89%	100%	89%	33%	100%	100%
NUSBot	28.89%	0%	0%	0%	11%	0%	0%	22%	78%	22%	0%	0%	0%	67%	0%		89%	0%	67%	100%	22%	100%
TerranUAB	23.33%	0%	0%	0%	0%	0%	0%	0%	11%	11%	0%	0%	0%	0%	11%	11%		89%	56%	89%	100%	89%
SRbotOne	26.11%	0%	11%	0%	0%	11%	0%	0%	0%	56%	0%	22%	0%	0%	0%	100%	11%		44%	100%	67%	100%
Cimex	20.00%	0%	0%	11%	0%	11%	0%	11%	11%	11%	0%	11%	0%	0%	11%	33%	44%	56%		56%	33%	100%
Oritaka	20.00%	0%	0%	44%	0%	0%	0%	0%	33%	11%	0%	11%	0%	0%	67%	0%	11%	0%	44%		78%	100%
CruzBot	16.11%	0%	0%	0%	0%	0%	0%	0%	11%	11%	0%	0%	0%	0%	0%	78%	0%	33%	67%	22%		100%
Tyr	0.56%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%	11%	0%	0%	0%	0%

I could look at these charts all day and keep finding stuff to think about. Winner Iron was upset by ZZZKBot and scored overwhelmingly against every other bot except the two dragoon-heavy protosses MegaBot and Skynet, which earned upsets on some maps. ZZZKBot’s results fall with rush distance, closely in line with Martin Rooijacker’s explanation. LetaBot did well on most maps, but struggled on Fortress. NUSBot apparently crashed on Heartbreak Ridge, giving Tyr nearly half of its wins. Weak bot Cimex did surprisingly well against strong UAlbertaBot on every map except Tau Cross and Python. Why is that?

Again I have Elo as calculated by Remi Coulom’s bayeselo program. The # column gives the official ranking, so you can see how it differs from the rank by Elo (the bayeselo ranking is slightly more accurate because it takes into account all the information in the tournament results, not only the raw winning rate). I left out the 95% confidence interval column as relatively uninteresting, since the “better?” column tells us how likely each bot is to be superior to the one below it.

#	bot	score	Elo	better?
1	Iron	87%	2016	99.4%
2	ZZZKBot	85%	1974	99.6%
3	tscmoo zerg	83%	1932	99.9%
4	LetaBot	74%	1815	99.8%
5	UAlbertaBot	70%	1774	99.9%
6	Ximp	65%	1699	99.6%
8	Aiur	61%	1663	51.6%
7	Overkill	62%	1663	99.6%
9	MegaBot	58%	1627	88.5%
10	IceBot	57%	1611	57.5%
12	Xelnaga	57%	1608	50.0%
11	JiaBot	57%	1608	98.1%
13	Skynet	55%	1581	100%
14	GarmBot	43%	1441	100%
16	TerranUAB	27%	1250	74.7%
15	NUSBot	27%	1240	99.9%
17	SRbotOne	22%	1167	99.0%
18	Cimex	21%	1130	92.6%
19	Oritaka	20%	1106	99.3%
20	CruzBot	17%	1064	100%
21	Tyr	1%	533	-

There are some switches from the official ranking, due to bots being statistically indistinguishable. Overkill and Aiur are in a dead heat. IceBot (terran), Xelnaga (protoss) and JiaBot (zerg) are also virtually even. bayeselo gives IceBot a 57.6% chance of being better than JiaBot two ranks down, essentially the same as its 57.5% chance of being better than Xelnaga one rank down.

Tomorrow: The per-map crosstables.

The AIIDE 2016 results are out. The top finishers, in order, are Iron, ZZZKBot, Tscmoo zerg, Letabot—half terran and half zerg. As usual, Martin Rooijackers sent me a few observations. Here are my thoughts so far.

• The tournament was played on the same maps as last year. It doesn’t call for that in the rules. We know that they’re OK maps, though.

• Martin Rooijackers says that all four top finishers were updated in between the CIG 2016 entry deadline and the AIIDE deadline a month later. [This turned out to be wrong. See the comments. I misread Martin Rooijacker’s message—he wrote that #1 Iron and #2 ZZZKBot had been updated.] You need continual work to stay at the top. It’s fierce up there.

• In particular, ZZZKBot moved up to #2, with minus scores only against fellow zergs Overkill and JiaBot. I’m curious to know what the improvements are.

• Tscmoo zerg scored even against ZZZKBot (a poor performance; zerg should easily beat the 4 pool), and worse than even against the rest of the top 5. Tscmoo owed its #3 finish to terrorizing its inferiors. Martin Rooijackers took this to mean that terran dominated. But with 2 of the top 4, I have to say that zerg is hanging in there.

• In last year’s AIIDE, UAlbertaBot and Overkill were neck-and-neck. The Bayesian Elo calculation said that UAlbertaBot was about 60% likely to be stronger, a narrow margin. Neither has been updated since. [This was also wrong! See comments.] In CIG 2016, Overkill was ahead. In this tournament, UAlbertaBot finished well ahead. These two can’t make up their minds! Apparently this time UAlbertaBot with its rush builds was better able to cope with the changing field.

• XIMP made it to #6. On SSCAIT its last update was February 2015, and I believe it was a minor adjustment to timings. The carrier strategy seems resilient. By contrast IceBot, AIIDE and CIG champion in 2014 ahead of XIMP, was in the middle of the pack. All its smarts did not make it resilient against improvements in its enemies.

• “JiaBot” in AIIDE is probably the same as “Ziabot” in CIG and “Zia bot” in SSCAIT. In some languages, the same sound can transliterate into English as either J or Z. Looking at the Ziabot source from CIG 2016, I think that the author is Korean based on variable names.

• Xelnaga finished ahead of Skynet, which surprised me. In CIG, XelnagaII was near the bottom. It seems a surprisingly sudden change.

• Tyr finished dead last by a mile, with a 1% win rate which seems so low that it must be due to bugs. Tyr had major updates this year which left it fairly strong. It finished in the middle of the pack in CIG 2016. I imagine it was updated at the last minute with a grievous error.

Tomorrow: The Elo table. I’ve done the calculations, but I have to draw up the table.