Automated essay scoring using efficient transformer-based language models

Automated Essay Scoring (AES) is a cross-disciplinary effort involving\nEducation, Linguistics, and Natural Language Processing (NLP). The efficacy of\nan NLP model in AES tests it ability to evaluate long-term dependencies and\nextrapolate meaning even when text is poorly written. Large pretrained\ntransformer-based language models have dominated the current state-of-the-art\nin many NLP tasks, however, the computational requirements of these models make\nthem expensive to deploy in practice. The goal of this paper is to challenge\nthe paradigm in NLP that bigger is better when it comes to AES. To do this, we\nevaluate the performance of several fine-tuned pretrained NLP models with a\nmodest number of parameters on an AES dataset. By ensembling our models, we\nachieve excellent results with fewer parameters than most pretrained\ntransformer-based models.\n