Most work in NLP makes the assumption that it is desirable to develop\nsolutions in the native language in question. There is consequently a strong\ntrend towards building native language models even for low-resource languages.\nThis paper questions this development, and explores the idea of simply\ntranslating the data into English, thereby enabling the use of pretrained, and\nlarge-scale, English language models. We demonstrate empirically that a large\nEnglish language model coupled with modern machine translation outperforms\nnative language models in most Scandinavian languages. The exception to this is\nFinnish, which we assume is due to inferior translation quality. Our results\nsuggest that machine translation is a mature technology, which raises a serious\ncounter-argument for training native language models for low-resource\nlanguages. This paper therefore strives to make a provocative but important\npoint. As English language models are improving at an unprecedented pace, which\nin turn improves machine translation, it is from an empirical and environmental\nstand-point more effective to translate data from low-resource languages into\nEnglish, than to build language models for such languages.\n