Despite the widespread adoption of deep learning for machine translation, it\nis still expensive to develop high-quality translation models. In this work, we\ninvestigate the use of pre-trained models, such as T5 for Portuguese-English\nand English-Portuguese translation tasks using low-cost hardware. We explore\nthe use of Portuguese and English pre-trained language models and propose an\nadaptation of the English tokenizer to represent Portuguese characters, such as\ndiaeresis, acute and grave accents. We compare our models to the Google\nTranslate API and MarianMT on a subset of the ParaCrawl dataset, as well as to\nthe winning submission to the WMT19 Biomedical Translation Shared Task. We also\ndescribe our submission to the WMT20 Biomedical Translation Shared Task. Our\nresults show that our models have a competitive performance to state-of-the-art\nmodels while being trained on modest hardware (a single 8GB gaming GPU for nine\ndays). Our data, models and code are available at\nhttps://github.com/unicamp-dl/Lite-T5-Translation.\n