How do I decide if this answer is right?
Leniency and Accuracy
Sometimes it is tricky to determine when to give full or partial credit for an answer that doesn't perfectly match what our Editorial team has provided. How strict should you be?? Isn't this a test about facts?
Don't sweat it too much! It's trivia! We trust you. You got this.
You have three key resources:
- the internet: teams are not allowed to use their Google, but you definitely can!*
- the colleagues: reply-all to the question email thread (and monitor the conversations happening there), or head to the #content channel in the Host Slack.
- the vibes: trust your instincts! Is the challenge from a team you know and trust? Is it something you just "feel" is more fair to credit or not? You're the one on the scene, so you get to decide.
Trivia Mafia has a phenomenal Editorial team, and each question is fact-checked and reviewed in a group. And yet, we sometimes make mistakes! You have the power to make the judgment call in the moment. Always ensure that teams have a chance to come up and talk to you about answers BEFORE winners are announced! If you can't grant them the point and the team wants to complain, point them to the Powers That Be to continue airing grievances.
Sometimes you decide that “subwoofer” is worth half a point, even though “woofer” was the only answer listed in the app.
Sometimes you decide, no, “entymology” is exactly halfway between the correct answer of "entomology" and a wrong answer of "etymology", and gets no points.
Sometimes a team points out that “the fast and furious 9” is basically the same as “F9" and you can give them full credit!
Do we require the full name?
In general, we follow Jeopardy! rules that Last Name Only is sufficient, unless there is plausible confusion. Kennedy, Roosevelt, Jones, Hepburn - there are multiple famous people with those last names, so they should include the first name.
EXAMPLE: Team answered "Kardashian"; official answer was "Kim Kardashian." We recommend a half point for “Kardashian.” However, it's zero points if they say Kourtney instead of Kim - because just “Kardashian” was generally right but “Kourtney Kardashian” was specifically wrong.
As much as possible, alert teams when you're looking for a full name when you're reading the question! Sometimes, that's a hint in and of itself.
Do we count off for spelling?
It is official Trivia Mafia policy that spelling does not count unless it’s explicitly stated in the question--if the answer is “Alpha Centauri” and a team writes “Alpa Centari,” that is a correct answer. (We actually prevent autocorrect in the app, so it is really not worth marking those mistakes wrong.)
How often should we be lenient, or give out half points?
Hoo boy, this is tricky! You, as the host, are in the best position to decide how much leniency your teams need. On the one hand, it is a competition, there are prizes on the line, and it’s your job to take it seriously without actually being too serious. There are a lot of teams out there who enjoy good stiff competition, and who would find an overly lenient night not to be fun. On the other hand, we need people to come back and play again.
Two things to keep in mind: We are here to have fun, and people need to feel heard. This is why it’s so important to make sure that there’s an opportunity between reading the answers and announcing winners for teams to bring up any points of contention! (But also: the customer is not always right. Players aren’t technically even our customers, actually, they’re the consumers of the product we sell to the venue… so stay diplomatic, and trust that we have your back on whichever decision you make.)
And: Most teams don’t win. We have data on nearly 200 games each week (and thus 200 winners) with something like 1,700 teams reported. So why are over 1,500 teams playing every week that don’t, and probably never will, win first place?As always, note your habits in your ICOE, so that a sub will know where you tend to fall on the strict/lenient scale, to meet player expectations at your site accordingly. Being consistent and being fair can be hard to balance alongside being fun. But we can do hard things.
*Use the internet, but not AI.
- Look for an alternative answer note. If you use the app, sometimes those notes don’t show up while grading, so check the spreadsheet or the document.
- Wikipedia. The world is upside down from when some of us when to college, and Wikipedia is now one of the most dependable sources going.
- Google, but scroll past the AI summary at the top of the page. See what a couple websites say!
- Email the host thread asking for clarification.
- Make the player prove it to you — they need to bring up a non-ChatGPT source to you to get points for an alternate answer.
No Comments