Are We Consistently Biased? Multidimensional Analysis of Biases in Distributional Word Vectors

Word embeddings have recently been shown to reflect many of the pronounced\nsocietal biases (e.g., gender bias or racial bias). Existing studies are,\nhowever, limited in scope and do not investigate the consistency of biases\nacross relevant dimensions like embedding models, types of texts, and different\nlanguages. In this work, we present a systematic study of biases encoded in\ndistributional word vector spaces: we analyze how consistent the bias effects\nare across languages, corpora, and embedding models. Furthermore, we analyze\nthe cross-lingual biases encoded in bilingual embedding spaces, indicative of\nthe effects of bias transfer encompassed in cross-lingual transfer of NLP\nmodels. Our study yields some unexpected findings, e.g., that biases can be\nemphasized or downplayed by different embedding models or that user-generated\ncontent may be less biased than encyclopedic text. We hope our work catalyzes\nbias research in NLP and informs the development of bias reduction techniques.\n