DisCoDisCo at the DISRPT2021 Shared Task: A System for Discourse Segmentation, Classification, and Connective Detection

This paper describes our submission to the DISRPT2021 Shared Task on\nDiscourse Unit Segmentation, Connective Detection, and Relation Classification.\nOur system, called DisCoDisCo, is a Transformer-based neural classifier which\nenhances contextualized word embeddings (CWEs) with hand-crafted features,\nrelying on tokenwise sequence tagging for discourse segmentation and connective\ndetection, and a feature-rich, encoder-less sentence pair classifier for\nrelation classification. Our results for the first two tasks outperform SOTA\nscores from the previous 2019 shared task, and results on relation\nclassification suggest strong performance on the new 2021 benchmark. Ablation\ntests show that including features beyond CWEs are helpful for both tasks, and\na partial evaluation of multiple pre-trained Transformer-based language models\nindicates that models pre-trained on the Next Sentence Prediction (NSP) task\nare optimal for relation classification.\n