AIIDE 2018 - what CherryPi learned
Here is a table of how each CherryPi opening fared against each opponent, like the tables I made for other bots. Reading the code confirmed my inference that the learning files recorded opening build orders, not build orders switched to later in the game; see how CherryPi played.
# | bot | total | 10hatchling | 2hatchmuta | 3basepoollings | 9poolspeedlingmuta | hydracheese | zve9poolspeed | zvp10hatch | zvp3hatchhydra | zvp6hatchhydra | zvpohydras | zvpomutas | zvt2baseguardian | zvt2baseultra | zvt3hatchlurker | zvtmacro | zvz12poolhydras | zvz9gas10pool | zvz9poolspeed | zvzoverpool |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
#1 | saida | 13-90 13% | - | - | - | - | - | 1-19 5% | - | - | - | - | - | - | 1-15 6% | 9-37 20% | 2-19 10% | - | - | - | - |
#3 | cse | 73-30 71% | - | - | - | - | - | 0-2 0% | 24-5 83% | - | - | 16-8 67% | - | - | - | - | 33-15 69% | - | - | - | - |
#4 | bluebluesky | 89-14 86% | - | - | - | - | - | 0-1 0% | 29-8 78% | - | - | - | - | - | - | - | 60-5 92% | - | - | - | - |
#5 | locutus | 84-19 82% | - | - | 63-11 85% | - | - | - | - | - | 14-3 82% | - | 2-2 50% | - | - | - | 5-3 62% | - | - | - | - |
#6 | isamind | 99-4 96% | - | - | 1-0 100% | - | - | - | - | - | 98-4 96% | - | - | - | - | - | - | - | - | - | - |
#7 | daqin | 103-0 100% | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 103-0 100% | - | - | - | - |
#8 | mcrave | 87-16 84% | - | - | 9-2 82% | - | - | - | - | - | 31-4 89% | - | 14-4 78% | - | - | - | 33-6 85% | - | - | - | - |
#9 | iron | 97-6 94% | - | - | - | - | 97-6 94% | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
#10 | zzzkbot | 93-10 90% | 58-4 94% | - | - | 0-1 0% | - | - | - | - | - | - | - | - | - | - | - | - | - | 35-4 90% | 0-1 0% |
#11 | steamhammer | 81-21 79% | 22-7 76% | - | - | - | - | 16-5 76% | - | - | - | - | - | - | - | - | - | 0-1 0% | - | 43-8 84% | - |
#12 | microwave | 94-9 91% | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 0-1 0% | 4-2 67% | 90-6 94% |
#13 | lastorder | 85-18 83% | 45-7 87% | - | - | - | - | 0-1 0% | - | - | - | - | - | - | - | - | - | - | - | - | 40-10 80% |
#14 | tyr | 98-5 95% | - | - | - | - | - | - | 98-5 95% | - | - | - | - | - | - | - | - | - | - | - | - |
#15 | metabot | 94-2 98% | - | - | - | - | - | - | - | - | - | 94-2 98% | - | - | - | - | - | - | - | - | - |
#16 | letabot | 101-2 98% | 0-1 0% | - | 97-0 100% | - | - | 1-1 50% | - | - | - | - | - | 3-0 100% | - | - | - | - | - | - | - |
#17 | arrakhammer | 92-11 89% | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 92-11 89% | - |
#18 | ecgberht | 102-1 99% | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 102-1 99% | - | - | - | - |
#19 | ualbertabot | 99-4 96% | - | - | - | 96-2 98% | - | 3-2 60% | - | - | - | - | - | - | - | - | - | - | - | - | - |
#20 | ximp | 98-5 95% | - | - | - | - | - | - | - | 1-0 100% | - | 97-5 95% | - | - | - | - | - | - | - | - | - |
#21 | cdbot | 103-0 100% | - | - | - | - | - | 96-0 100% | - | - | - | - | - | - | - | - | - | - | - | 7-0 100% | - |
#22 | aiur | 100-3 97% | - | - | - | - | - | - | - | - | - | 100-3 97% | - | - | - | - | - | - | - | - | - |
#23 | killall | 103-0 100% | 102-0 100% | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 1-0 100% |
#24 | willyt | 103-0 100% | - | 103-0 100% | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
#25 | ailien | 103-0 100% | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 103-0 100% | - |
#26 | cunybot | 100-3 97% | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 100-3 97% | - |
#27 | hellbot | 103-0 100% | - | - | - | - | - | - | 31-0 100% | - | - | 72-0 100% | - | - | - | - | - | - | - | - | - |
overall | - 90% | 227-19 92% | 103-0 100% | 170-13 93% | 96-3 97% | 97-6 94% | 117-31 79% | 182-18 91% | 1-0 100% | 143-11 93% | 379-18 95% | 16-6 73% | 3-0 100% | 1-15 6% | 9-37 20% | 338-49 87% | 0-1 0% | 0-1 0% | 384-28 93% | 131-17 89% |
Look how sparse the chart is—CherryPi was highly selective about its choices. It did not try more than 4 different builds against any opponent. It makes sense to minimize the number of choices so that you don’t lose games exploring bad ones, but you have to be pretty sure that one of the choices you do try is good. Where did the selectivity come from?
The opening “hydracheese” was played only against Iron, and was the only opening played against Iron. It smelled like a hand-coded choice. Sure enough, the file source/src/models/banditconfigurations.cpp
configures builds by name for 18 of the 27 entrants. A comment says that the build order switcher is turned off for the hydracheese opening only: “BOS disabled for this specific build because the model hasn’t seen it.” Here is the full set of builds configured, including defaults for those that were not hand-configured. CherryPi played only builds that were configured, but did not play all the builds that were configured; presumably it stopped when it hit a good one.
bots | builds | note |
---|---|---|
AILien | zve9poolspeed zvz9poolspeed | returning opponents from last year |
AIUR | zvtmacro zvpohydras zvp10hatch | |
Arrakhammer | 10hatchling zvz9poolspeed | |
Iron | hydracheese | |
UAlbertaBot | zve9poolspeed 9poolspeedlingmuta | |
Ximp | zvpohydras zvtmacro zvp3hatchhydra | |
Microwave | zvzoverpool zvz9poolspeed zvz9gas10pool | “we have some expectations” |
Steamhammer | zve9poolspeed zvz9poolspeed zvz12poolhydras 10hatchling | |
ZZZKBot | 9poolspeedlingmuta 10hatchling zvz9poolspeed zvzoverpool | |
ISAMind Locutus McRave DaQin | zvtmacro zvp6hatchhydra 3basepoollings zvpomutas | |
CUNYBot | zvzoverpoolplus1 zvz9gas10pool zvz9poolspeed | |
HannesBredberg | zvtp1hatchlurker zvt2baseultra zvt3hatchlurker zvp10hatch | |
LetaBot | zvtmacro 3basepoollings zvt2baseguardian zve9poolspeed 10hatchling | |
MetaBot | zvtmacro zvpohydras zvpomutas zve9poolspeed | |
WillyT | zvt2baseultra 12poolmuta 2hatchmuta | |
ZvT | zvt2baseultra zvtmacro zvt3hatchlurker zve9poolspeed | defaults |
ZvP | zve9poolspeed zvtmacro zvp10hatch zvpohydras | |
ZvZ | 10hatchling zve9poolspeed zvz9poolspeed zvzoverpool | |
ZvR | 10hatchling zve9poolspeed 9poolspeedlingmuta |
I read this as pulling out all the stops to reach #1. They would have succeeded if not for SAIDA.
banditconfigurations.cpp
continues and declares some properties for builds including non-opening builds. It looks like .validOpening()
tells whether it can be played as an opening build, .validSwitch()
tells whether the build order switcher is allowed to switch to it during the game, and .switchEnabled()
tells whether the build order switcher is enabled at all.
The build orders themselves are defined in source/src/buildorders/
. I found them a little hard to read, partly because they are written in reverse order: Actions to happen first are posted last to the blackboard.
The opening zve9poolspeed (I read “zve” as zerg versus everything) has the most red boxes in the chart—it did poorly against more opponents than any other. It may have been a poor choice to configure for use in so many cases. In contrast, zvz9poolspeed specialized for ZvZ was successful. It gets fast mutalisks and in general has a lot more strategic detail coded into the build.
They seem to have had expectations of the zvt2baseultra build against terran. It is configured for HannesBredberg, WillyT, and the default ZvT. It was in fact only tried against SAIDA. I didn’t notice anything that tells CherryPi what order to try opening builds in. Maybe the build order switcher itself contributes, helping to choose the more likely openings first?
Comments
MarcoDBAA on :
I do not understand at all, why they did not use all terran openings in general ZvT? You always have to expect strong surprise entries or descendants (even clones) of other bots.
Sure, you can determine preferred choices, but going 1-19 vs SAIDA instead with one opening was for sure not the correct thing to do.
Tully Elliston on :
Really this indicates that the rules of the tournaments should be changed to prevent hard-coded responses.
Jay Scott on :
krasi0 on :
Did the SAIDA team do this too?
Jay Scott on :
Jay Scott on :
jgehring on :
Tully Elliston on :
That distinction being "Learning file is fine, hand-coded line is lame"
Jay Scott on :
Barcode on :
Tully Elliston on :