TransWiC at SemEval-2021 Task 2: Transformer-based Multilingual and Cross-lingual Word-in-Context Disambiguation

Identifying whether a word carries the same meaning or different meaning in\ntwo contexts is an important research area in natural language processing which\nplays a significant role in many applications such as question answering,\ndocument summarisation, information retrieval and information extraction. Most\nof the previous work in this area rely on language-specific resources making it\ndifficult to generalise across languages. Considering this limitation, our\napproach to SemEval-2021 Task 2 is based only on pretrained transformer models\nand does not use any language-specific processing and resources. Despite that,\nour best model achieves 0.90 accuracy for English-English subtask which is very\ncompatible compared to the best result of the subtask; 0.93 accuracy. Our\napproach also achieves satisfactory results in other monolingual and\ncross-lingual language pairs as well.\n