KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Media

In this paper, we describe our approach to utilize pre-trained BERT models\nwith Convolutional Neural Networks for sub-task A of the Multilingual Offensive\nLanguage Identification shared task (OffensEval 2020), which is a part of the\nSemEval 2020. We show that combining CNN with BERT is better than using BERT on\nits own, and we emphasize the importance of utilizing pre-trained language\nmodels for downstream tasks. Our system, ranked 4th with macro averaged\nF1-Score of 0.897 in Arabic, 4th with score of 0.843 in Greek, and 3rd with\nscore of 0.814 in Turkish. Additionally, we present ArabicBERT, a set of\npre-trained transformer language models for Arabic that we share with the\ncommunity.\n