Lingua Custodia's participation at the WMT 2021 Machine Translation using Terminologies shared task

This paper describes Lingua Custodia's submission to the WMT21 shared task on\nmachine translation using terminologies. We consider three directions, namely\nEnglish to French, Russian, and Chinese. We rely on a Transformer-based\narchitecture as a building block, and we explore a method which introduces two\nmain changes to the standard procedure to handle terminologies. The first one\nconsists in augmenting the training data in such a way as to encourage the\nmodel to learn a copy behavior when it encounters terminology constraint terms.\nThe second change is constraint token masking, whose purpose is to ease copy\nbehavior learning and to improve model generalization. Empirical results show\nthat our method satisfies most terminology constraints while maintaining high\ntranslation quality.\n