RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models

Text representation models are prone to exhibit a range of societal biases,\nreflecting the non-controlled and biased nature of the underlying pretraining\ndata, which consequently leads to severe ethical issues and even bias\namplification. Recent work has predominantly focused on measuring and\nmitigating bias in pretrained language models. Surprisingly, the landscape of\nbias measurements and mitigation resources and methods for conversational\nlanguage models is still very scarce: it is limited to only a few types of\nbias, artificially constructed resources, and completely ignores the impact\nthat debiasing methods may have on the final performance in dialog tasks, e.g.,\nconversational response generation. In this work, we present RedditBias, the\nfirst conversational data set grounded in the actual human conversations from\nReddit, allowing for bias measurement and mitigation across four important bias\ndimensions: gender, race, religion, and queerness. Further, we develop an\nevaluation framework which simultaneously 1) measures bias on the developed\nRedditBias resource, and 2) evaluates model capability in dialog tasks after\nmodel debiasing. We use the evaluation framework to benchmark the widely used\nconversational DialoGPT model along with the adaptations of four debiasing\nmethods. Our results indicate that DialoGPT is biased with respect to religious\ngroups and that some debiasing techniques can remove this bias while preserving\ndownstream task performance.\n