This work presents our contribution in the context of the 6th task of\nSemEval-2020: Extracting Definitions from Free Text in Textbooks (DeftEval).\nThis competition consists of three subtasks with different levels of\ngranularity: (1) classification of sentences as definitional or\nnon-definitional,(2) labeling of definitional sentences, and (3) relation\nclassification. We use various pretrained language models (i.e., BERT, XLNet,\nRoBERTa, SciBERT, and ALBERT) to solve each of the three subtasks of the\ncompetition. Specifically, for each language model variant, we experiment by\nboth freezing its weights and fine-tuning them. We also explore a multi-task\narchitecture that was trained to jointly predict the outputs for the second and\nthe third subtasks. Our best performing model evaluated on the DeftEval dataset\nobtains the 32nd place for the first subtask and the 37th place for the second\nsubtask. The code is available for further research at:\nhttps://github.com/avramandrei/DeftEval.\n