UmBERTo-MTSA @ AcCompl-It: Improving Complexity and Acceptability Prediction with Multi-task Learning on Self-Supervised Annotations

This work describes a self-supervised data augmentation approach used to\nimprove learning models' performances when only a moderate amount of labeled\ndata is available. Multiple copies of the original model are initially trained\non the downstream task. Their predictions are then used to annotate a large set\nof unlabeled examples. Finally, multi-task training is performed on the\nparallel annotations of the resulting training set, and final scores are\nobtained by averaging annotator-specific head predictions. Neural language\nmodels are fine-tuned using this procedure in the context of the AcCompl-it\nshared task at EVALITA 2020, obtaining considerable improvements in prediction\nquality.\n