UPB at SemEval-2020 Task 9: Identifying Sentiment in Code-Mixed Social Media Texts using Transformers and Multi-Task Learning

Sentiment analysis is a process widely used in opinion mining campaigns\nconducted today. This phenomenon presents applications in a variety of fields,\nespecially in collecting information related to the attitude or satisfaction of\nusers concerning a particular subject. However, the task of managing such a\nprocess becomes noticeably more difficult when it is applied in cultures that\ntend to combine two languages in order to express ideas and thoughts. By\ninterleaving words from two languages, the user can express with ease, but at\nthe cost of making the text far less intelligible for those who are not\nfamiliar with this technique, but also for standard opinion mining algorithms.\nIn this paper, we describe the systems developed by our team for SemEval-2020\nTask 9 that aims to cover two well-known code-mixed languages: Hindi-English\nand Spanish-English.\n We intend to solve this issue by introducing a solution that takes advantage\nof several neural network approaches, as well as pre-trained word embeddings.\nOur approach (multlingual BERT) achieves promising performance on the\nHindi-English task, with an average F1-score of 0.6850, registered on the\ncompetition leaderboard, ranking our team 16th out of 62 participants. For the\nSpanish-English task, we obtained an average F1-score of 0.7064 ranking our\nteam 17th out of 29 participants by using another multilingual\nTransformer-based model, XLM-RoBERTa.\n