the “no shallow forks” rule in SSCAIT
The SSCAIT rules include 2 items about copying other bots. Here’s the first item:
The SSCAIT admins reserve the right to disqualify any bots that are implementation-wise too similar to previous entries (e.g. clones and forks) without adding anything novel / original to the table.
On the upside, it openly admits that the decision is a judgment call, and otherwise it is simple. On the downside, it offers little guidance on what counts as “too similar.” On balance, I like this rule.
The second item is listed separately and seems to have been inserted independently:
If you copy other bots or use IP/files/source code/logic/techniques etc from other bots, you must familiarize yourself with their licenses and ensure that you are not infringing their licenses. Copying other bot(s) is allowed, so long as it does not infringe their licenses and so long as you modify their logic or if you use/wrap it without modifying it and add some of your own logic on top of it, similar to how MegaBot used Skynet/Xelnaga/NUSBot in year 2017, or wrapping/modifying Randomhammer/UAlbertaBot/CommandCenter etc. If you do something like this, you must provide the source code and compilation instructions etc of the bots that you use, so that we can compile them. We decided that to foster research it is best to have the next generation of programmers stand on the shoulders of giants, rather than re-invent the wheel. We encourage authors to take code from old years of this competition and improve it. If you copy a bot, please uphold the spirit of competition and ensure you make significant modification or addition before you submit it. We don’t want multiple apparently near-identical copies of the same bot competing! Additionally, please contact the original bot (in case it has been updated at least once during the past year) author and ask them for permission to upload it.
This paragraph looks like it was partly copied from CIG rules, which were themselves partly copied from old AIIDE rules. Not all of it makes sense for SSCAIT. It gives the impression of having been dropped in without careful thought. It declares multiple rules. In more detail:
The rule about obeying licenses is good, except that borrowing a “technique” from another bot is universal. If you use MCRS, do you have to obey the license of UAlbertaBot because UAlbertaBot with SparCraft invented the technique of making Starcraft decisions by combat simulation? The technique that has been accepted as standard and used nearly everywhere? That makes no sense at all. The word “technique” without qualification is too vague to interpret; leave it out.
Next is some wording about when it’s OK to copy other bots. It’s unclear how much this is establishing a rule and how much it is providing guidelines to interpret the rule already established that something must be “original / novel.” In any case it overlaps with the first item, and if it’s worth keeping at all, part at least should be combined into that paragraph.
Then there’s a strange and awkwardly-worded rule that seems to require source code of the original bot that you wrap or modify, “so that we can compile” it. I don’t think SSCAIT does this. Am I wrong? Is anything similar to this rule enforced at all, or even understood by the organizers? It has the smell of a copy-paste error.
Finally, there is a rule about asking the author for permission. I don’t like that rule. I deliberately chose a license that does not require forks to ask permission, or even to let me know. As far as I am concerned, the license grants permission; to require anything from me is a waste of my time. Instead of this rule, I propose a suggestion to authors that they may want to consider whether to add a permission requirement to their license. Then the obey-the-license rule is all that’s needed.
ambiguous cases
Here are 3 edge cases related to Steamhammer. Maybe they can get people discussing what guidance to add to the rules.
Steamhammer 0.2 was forked from UAlbertaBot and played in SSCAIT 2016 after about 3 weeks of development. I made bug fixes and macro changes and stuff, but 3 weeks is not long enough for substantial code changes. Nevertheless, from the start Steamhammer’s strategy and game plan were very different from UAlbertaBot’s. Nobody suggested that it might be too similar, even though the tactics and micro were nearly identical.
NLPRBot is a 2017 fork of Steamhammer, and has not been updated. The link is to a blog post where the comments discuss what to do about shallow forks. It was accepted in tournaments in 2017 under the rules of the time. SSCAIT chose to accept it in 2018 too, though if today’s rules had been in effect in 2017 it probably would have been rejected then. I think that allowing it in 2018 is fair; NLPRBot plays like an old Steamhammer, differently from the current one, so it still adds variety. But it is an edge case, and there is an argument for rejecting it.
insanitybot is a recent fork of Steamhammer. It adds wraiths and minelaying skills and has other changes, but you could argue that it is a shallow fork; its play is in many ways closely similar to how Randomhammer plays terran. I think it is right to accept it into the tournament, because of the improvements and because Steamhammer doesn’t play terran except as random. But again, it is an edge case.
Comments
McRave on :
Jay Scott on :
Jay Scott on :
MicroDK on :
the phrase about asking permission after a lot of Locutus forks appeared in AIIDE. After AIIDE the author of Locutus added to his license that people has to contact him to get permission to fork it.
Bruce on :
In many ways I think it as an undue burden on tournament organizers to decide how different a bot needs to be, as they are missing a lot of necessary context. For example, after looking deeper into the code I probably would not have allowed CSE to stay in the aiide results, as the only significant change is a single pvp build. But I understand why Dave left it in, as it performed best of the Locutuses and on the surface did something with ML (it was just an extremely limited scope, which is only really obvious with greater understanding of the bot code).
So I am leaning towards a license that puts me as a sort of gatekeeper. In practice I would allow any forks playing terran or zerg and allow any established bot authors to integrate whatever they like into their own bots. For Protoss forks I would need to see some plans for significant changes (definition of “significant” to be determined).
Jay Scott on :
Bruce on :
McRave on :
The copyright isn't there for me to say "I made this, don't touch it"; it's there to prevent anyone from using any piece or all of the code without my permission.
krasi0 on :
Of course, the wording is far from ideal, but the rule should more or less convey the spirit and general idea about the use of shallow copies to any bot authors who wish to take part in the tournament. Most of Jay's criticism is fair, so let's work together on clarifying that paragraph better.
I don't particularly like the tone of the statement: "As far as I am concerned, the license grants permission; to require anything from me is a waste of my time." Verifying which bots are shallow clones will inevitably end up being someone's "waste of time". But don't we agree that it would be much easier for the original bot author to analyze whether a fork of their bot adds anything significant to the table as opposed to putting the whole burden on the shoulders of the tournament organizers who would have to get to know and analyze every bot (or a fork of) individually under a microscope. When you as a bot author decide to open source your bot and distribute it with the most permissive licence, you should also take at least some portion of the responsibility where this could create a flood of clones that in theory could clog / drown some competitions.
Copying software is easy but the implications that come with it are a tough matter and we, as a community, are all in this together and should shoulder the difficulties that this phenomenon poses.
Jay Scott on :
And that’s true for any rule, certainly any rule with a gray area. I don’t believe in exploiting the unit ID information leak; Arrak of Arrakhammer doesn’t believe in exploiting the unit target information leak that Steamhammer makes use of.
MicroDK on :
Also, we could have a case where an author allows forks, then the tournament organizers need to make this judgement anyway for the sake of the community, the other bots and the tournament result. In time both the original bots and the forks will evolve so in 1-2 years from now the forks will be different.
In my opinion, if a fork uses the exact same build orders and only change limited parts of the code, this fork should not be allowed in a tournament. Forks at least needs to change the build orders and parts of how the bot works.
Joseph Huang on :
Jay Scott on :
Jay Scott on :
Jay Scott on :
Jay Scott on :
On the face of it, open bugs and README aside, it appears as though BWTA2 has finished migrating from CGAL to the Boost geometry stuff. So as far as I can tell, the LGPL license is valid and users of BWTA2 are in the clear.
Does anybody know more?
Joseph Huang on :
Jay Scott on :
What a confusing mess.
krasi0 on :