Since BERT appeared, Transformer language models and transfer learning have\nbecome state-of-the-art for Natural Language Understanding tasks. Recently,\nsome works geared towards pre-training specially-crafted models for particular\ndomains, such as scientific papers, medical documents, user-generated texts,\namong others. These domain-specific models have been shown to improve\nperformance significantly in most tasks. However, for languages other than\nEnglish such models are not widely available.\n In this work, we present RoBERTuito, a pre-trained language model for\nuser-generated text in Spanish, trained on over 500 million tweets. Experiments\non a benchmark of tasks involving user-generated text showed that RoBERTuito\noutperformed other pre-trained language models in Spanish. In addition to this,\nour model achieves top results for some English-Spanish tasks of the Linguistic\nCode-Switching Evaluation benchmark (LinCE) and has also competitive\nperformance against monolingual models in English tasks. To facilitate further\nresearch, we make RoBERTuito publicly available at the HuggingFace model hub\ntogether with the dataset used to pre-train it.\n