Collecting a Large-Scale Gender Bias Dataset for Coreference Resolution and Machine Translation

Recent works have found evidence of gender bias in models of machine\ntranslation and coreference resolution using mostly synthetic diagnostic\ndatasets. While these quantify bias in a controlled experiment, they often do\nso on a small scale and consist mostly of artificial, out-of-distribution\nsentences. In this work, we find grammatical patterns indicating stereotypical\nand non-stereotypical gender-role assignments (e.g., female nurses versus male\ndancers) in corpora from three domains, resulting in a first large-scale gender\nbias dataset of 108K diverse real-world English sentences. We manually verify\nthe quality of our corpus and use it to evaluate gender bias in various\ncoreference resolution and machine translation models. We find that all tested\nmodels tend to over-rely on gender stereotypes when presented with natural\ninputs, which may be especially harmful when deployed in commercial systems.\nFinally, we show that our dataset lends itself to finetuning a coreference\nresolution model, finding it mitigates bias on a held out set. Our dataset and\nmodels are publicly available at www.github.com/SLAB-NLP/BUG. We hope they will\nspur future research into gender bias evaluation mitigation techniques in\nrealistic settings.\n