Starcraft AI blog

This scatter chart shows rating differences versus score ratios. For each SSCAIT game in the dataset which ended with both scores above zero (about 80,000 games), the x-axis has the rating difference, winner’s rating minus loser’s rating. The ratings are calculated as of that game, so changing versions don’t mess things up (much). Check the Elo table to see what the rating differences mean. The y-axis has the score ratio, winner’s score divided by loser’s score. The y-axis is on a logarithmic scale and shows ratios from 0.2 to 200. A small number of points are off the top or bottom; no points are off the left or right. You can click through the graph to get the same image on its own, which may make it easier to zoom in and out.

Since it’s from the winner’s point of view, most games have rating difference > 0 (the higher rated bot won) and score ratio > 1 (the bot with the higher score won).

There is complex structure here, but I’m at a loss to interpret much of it. Horizontal lines show that some score ratios are popular, which seems like a quirk of the game. Beyond the sharp lines, some fuzzier stratification in the score ratios is visible. When the stronger bot wins, it is usually by a score ratio of at least 2, increasing slowly as the strength differences goes up. The slowly rising “soft floor” in the score ratio is interesting and surprising. There are other clear structures in the chart, but I don’t know what they mean. It is mildly interesting that the left side, when the stronger bot lost, looks less structured.

Games where the score ratio is less than 1 are games where stopping early and adjudicating by score would give the wrong answer. It’s rare... but not rare enough to give me confidence in the timeout adjudication procedure. Some points are off the bottom, so some bots won despite having less than 1/5 the score by SSCAIT rules. The adjudication procedure will make occasional extreme mistakes.

Next: The winning attitude.