The ComMA Dataset V0.2: Annotating Aggression and Bias in Multilingual Social Media Discourse

In this paper, we discuss the development of a multilingual dataset annotated\nwith a hierarchical, fine-grained tagset marking different types of aggression\nand the "context" in which they occur. The context, here, is defined by the\nconversational thread in which a specific comment occurs and also the "type" of\ndiscursive role that the comment is performing with respect to the previous\ncomment. The initial dataset, being discussed here (and made available as part\nof the ComMA@ICON shared task), consists of a total 15,000 annotated comments\nin four languages - Meitei, Bangla, Hindi, and Indian English - collected from\nvarious social media platforms such as YouTube, Facebook, Twitter and Telegram.\nAs is usual on social media websites, a large number of these comments are\nmultilingual, mostly code-mixed with English. The paper gives a detailed\ndescription of the tagset being used for annotation and also the process of\ndeveloping a multi-label, fine-grained tagset that can be used for marking\ncomments with aggression and bias of various kinds including gender bias,\nreligious intolerance (called communal bias in the tagset), class/caste bias\nand ethnic/racial bias. We also define and discuss the tags that have been used\nfor marking different the discursive role being performed through the comments,\nsuch as attack, defend, etc. We also present a statistical analysis of the\ndataset as well as results of our baseline experiments with developing an\nautomatic aggression identification system using the dataset developed.\n