DagoBERT: Generating Derivational Morphology with a Pretrained Language Model

Can BERT generate derivationally complex words? We present the first study investigating this question. We find that BERT with a derivational classification layer outperforms an LSTM-based model. Furthermore, our experiments show that the input segmentation crucially impacts BERT's derivational knowledge, both during training and inference.