archive by month
Skip to content

CoG first-try results posted

Purple Dan reported to me by e-mail. Thanks!

1. COG operators presented results at the conference and posted the summary results afterwards
2. Folks noticed that bots were run with incorrect names, which would break pretraining/preparation
3. COG silently removed the results, hoping to re-run it
4. After discussing it with bot authors, they have posted the full detailed results while authors decide if there was any likely impact of the error on the results

Microwave and BananaBrain seem to be the possibly-injured bots here, though it seems unlikely to me that anything would materially change.

Sure enough, the first-try detailed results are posted and I have grabbed ’em.

They ran the carryover bots with the year that they were carried over from appended to their names. For example, McRave was carried over from last year, and they ran it under the name McRave2022. In the posted registration list, it was listed simply as McRave. So any bot that was specially prepared to face McRave by name, wasn’t prepared after all. It can potentially make a big difference, but since only carryover bots were changed, it possibly may not.

I think it’s defensible as a policy. The carryovers are by definition not updated, and an updated bot’s ability to prepare for them can be seen as unfair—it’s much easier to hit a target that does not move. Special prep against one opponent may have no effect on play against others, so that the tournament measures progress where there is none. But it’s not a policy to spring on participants by surprise. If you’re going to do it that way, announce it in advance. Don’t make authors waste time.

Anyway, we’ll have to wait and see if there’s a decision to re-run.

Next: I’ll do a quick-look analysis of what we’ve got.

Update: Dan Gant points out to me that Microwave was also misspelled as MicroWave. Ouch.


No Trackbacks


No comments

Add Comment

E-Mail addresses will not be displayed and will only be used for E-Mail notifications.

To prevent automated Bots from commentspamming, please enter the string you see in the image below in the appropriate input box. Your comment will only be submitted if the strings match. Please ensure that your browser supports and accepts cookies, or your comment cannot be verified correctly.

Form options

Submitted comments will be subject to moderation before being displayed.