Pushing on Text Readability Assessment: A Transformer Meets Handcrafted Linguistic Features

We report two essential improvements in readability assessment: 1. three\nnovel features in advanced semantics and 2. the timely evidence that\ntraditional ML models (e.g. Random Forest, using handcrafted features) can\ncombine with transformers (e.g. RoBERTa) to augment model performance. First,\nwe explore suitable transformers and traditional ML models. Then, we extract\n255 handcrafted linguistic features using self-developed extraction software.\nFinally, we assemble those to create several hybrid models, achieving\nstate-of-the-art (SOTA) accuracy on popular datasets in readability assessment.\nThe use of handcrafted features help model performance on smaller datasets.\nNotably, our RoBERTA-RF-T1 hybrid achieves the near-perfect classification\naccuracy of 99%, a 20.3% increase from the previous SOTA.\n