Warning flags for reviewers on various entities

[hburn7]

We need a way to flag games which use beatmaps that are only used once throughout the tournament. This is requested by @SourMongoose.

• Identify optimal strategy (is just a boolean flag appropriate @myssto?)

Edit:

This discussion will discuss the inclusion of warning flags that reviewers can look at during the review process. The idea is to get a brief glimpse into issues that do not necessarily warrant rejecting the item outright.

Proposed flags:

• Match
  • Regex validation on match title
• Game
  • Beatmap played is used only once in the tournament (helps identify warmups)
  • Beatmap (which is not one of the first two games in the match) is not in the tournament mappool
  • Beatmap is >= 3 stars above or below the median for the match

[myssto]

I feel like there have been a lot of things that we identify for automation checks that result in the pre-rejection of an entity that we disagree on between a lot of people in the team. I'm wondering if we could expand on this and maybe have another flag enum, something like AutomationCheckSuspicions. I'm thinking it could be something that is sort of a half-way point between two points we have now being that if we want to identify something like this it would either have to create an entirely new and specialized property or be added to automation checks as a rejection reason. This way we could add some information about suspicious scenarios in entities that should be looked at more closely without having to necessarily pre-reject them outright.

[hburn7]

I think having another field for flags as needed would be a great idea.

@cytusine0 / @SourMongoose I am curious what your thoughts are on this one. Are there any current rejection reasons that you feel should just be "flags" (alerts) instead? A flag in this case would not automatically mark the item as rejected, but would have some indicator during the review process.

[myssto]

Well it could also apply to things that we've axed from automation checks or that cytusine / sour weren't exactly the most keen on. For example we could reimplement the regex name check on matches, but it could be added to this new flag instead of being a reason to reject the match.

[hburn7]

Yeah that's a great idea. Shall we make a feature tracking issue for this then? (Actually, we can just convert this issue into that)

[myssto]

I'd like to hear cytusine, sour, and alix opinions on this first but I'm on board. Mainly would like to hear as many use cases for this as possible. Mainly because this means that we will have to re-run automation checks on all tournaments now :D

[SourMongoose]

Wouldn't mind a separate "warning" flag, definitely see the regex check failing as part of this new flag then. Is there a full list of rejection reasons somewhere?

[cytusine0]

The main other "useful flags" I can think of are

• map is extremely low / high star rating (say 3 stars away from the median), very short (under 1 minute?), or has other anomalous stats
• (maybe??) game is short a player, but they disconnected while the map was playing? I don't think we want to count those games but maybe there's something we would still do with those stats down the line.
  Relatedly, would any of these comments / flags ever want to be public to website viewers? I know we want the list of rejected badged tournaments to show up somewhere on the website instead of the current spreadsheet and I just wonder how that interacts with this.

[hburn7]

@cytusine0 First 1 I think is very doable. Perhaps just SR and duration are fine for now. The second one I don't think is very doable or practical. This would be something manual review covers, ideally.

In my head, these flags would be private to reviewers. I am unsure whether marking a tournament as 'verified' should remove these flags.

[hburn7]

Wouldn't mind a separate "warning" flag, definitely see the regex check failing as part of this new flag then. Is there a full list of rejection reasons somewhere?

See here:

• TournamentRejectionReason
• MatchRejectionReason
• GameRejectionReason
• ScoreRejectionReason

[cytusine0]

Now that we're entering in tournament mappools (whenever they're publicly accessible), I'd appreciate a warning for

"Tournament has a mappool entered, and a game that was NOT one of the first two games during a match is getting rejected because the map is not in the pool."

This would make it easy for us to detect whenever we missed a round of a tournament, there's a missing beatconnect link that needs to be added in, etc.