AraStance: A Multi-Country and Multi-Domain Dataset of Arabic Stance Detection for Fact Checking

With the continuing spread of misinformation and disinformation online, it is\nof increasing importance to develop combating mechanisms at scale in the form\nof automated systems that support multiple languages. One task of interest is\nclaim veracity prediction, which can be addressed using stance detection with\nrespect to relevant documents retrieved online. To this end, we present our new\nArabic Stance Detection dataset (AraStance) of 4,063 claim--article pairs from\na diverse set of sources comprising three fact-checking websites and one news\nwebsite. AraStance covers false and true claims from multiple domains (e.g.,\npolitics, sports, health) and several Arab countries, and it is well-balanced\nbetween related and unrelated documents with respect to the claims. We\nbenchmark AraStance, along with two other stance detection datasets, using a\nnumber of BERT-based models. Our best model achieves an accuracy of 85\\% and a\nmacro F1 score of 78\\%, which leaves room for improvement and reflects the\nchallenging nature of AraStance and the task of stance detection in general.\n