All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text

Human evaluations are typically considered the gold standard in natural\nlanguage generation, but as models' fluency improves, how well can evaluators\ndetect and judge machine-generated text? We run a study assessing non-experts'\nability to distinguish between human- and machine-authored text (GPT2 and GPT3)\nin three domains (stories, news articles, and recipes). We find that, without\ntraining, evaluators distinguished between GPT3- and human-authored text at\nrandom chance level. We explore three approaches for quickly training\nevaluators to better identify GPT3-authored text (detailed instructions,\nannotated examples, and paired examples) and find that while evaluators'\naccuracy improved up to 55%, it did not significantly improve across the three\ndomains. Given the inconsistent results across text domains and the often\ncontradictory reasons evaluators gave for their judgments, we examine the role\nuntrained human evaluations play in NLG evaluation and provide recommendations\nto NLG researchers for improving human evaluations of text generated from\nstate-of-the-art models.\n