A major obstacle in Word Sense Disambiguation (WSD) is that word senses are\nnot uniformly distributed, causing existing models to generally perform poorly\non senses that are either rare or unseen during training. We propose a\nbi-encoder model that independently embeds (1) the target word with its\nsurrounding context and (2) the dictionary definition, or gloss, of each sense.\nThe encoders are jointly optimized in the same representation space, so that\nsense disambiguation can be performed by finding the nearest sense embedding\nfor each target word embedding. Our system outperforms previous\nstate-of-the-art models on English all-words WSD; these gains predominantly\ncome from improved performance on rare senses, leading to a 31.1% error\nreduction on less frequent senses over prior work. This demonstrates that rare\nsenses can be more effectively disambiguated by modeling their definitions.\n