"Thy algorithm shalt not bear false witness": An Evaluation of Multiclass Debiasing Methods on Word Embeddings

With the vast development and employment of artificial intelligence\napplications, research into the fairness of these algorithms has been\nincreased. Specifically, in the natural language processing domain, it has been\nshown that social biases persist in word embeddings and are thus in danger of\namplifying these biases when used. As an example of social bias, religious\nbiases are shown to persist in word embeddings and the need for its removal is\nhighlighted. This paper investigates the state-of-the-art multiclass debiasing\ntechniques: Hard debiasing, SoftWEAT debiasing and Conceptor debiasing. It\nevaluates their performance when removing religious bias on a common basis by\nquantifying bias removal via the Word Embedding Association Test (WEAT), Mean\nAverage Cosine Similarity (MAC) and the Relative Negative Sentiment Bias\n(RNSB). By investigating the religious bias removal on three widely used word\nembeddings, namely: Word2Vec, GloVe, and ConceptNet, it is shown that the\npreferred method is ConceptorDebiasing. Specifically, this technique manages to\ndecrease the measured religious bias on average by 82,42%, 96,78% and 54,76%\nfor the three word embedding sets respectively.\n