Adapting BERT for Word Sense Disambiguation with Gloss Selection Objective and Example Sentences

Domain adaptation or transfer learning using pre-trained language models such\nas BERT has proven to be an effective approach for many natural language\nprocessing tasks. In this work, we propose to formulate word sense\ndisambiguation as a relevance ranking task, and fine-tune BERT on sequence-pair\nranking task to select the most probable sense definition given a context\nsentence and a list of candidate sense definitions. We also introduce a data\naugmentation technique for WSD using existing example sentences from WordNet.\nUsing the proposed training objective and data augmentation technique, our\nmodels are able to achieve state-of-the-art results on the English all-words\nbenchmark datasets.\n