UPB at SemEval-2021 Task 7: Adversarial Multi-Task Learning for Detecting and Rating Humor and Offense

Detecting humor is a challenging task since words might share multiple\nvalences and, depending on the context, the same words can be even used in\noffensive expressions. Neural network architectures based on Transformer obtain\nstate-of-the-art results on several Natural Language Processing tasks,\nespecially text classification. Adversarial learning, combined with other\ntechniques such as multi-task learning, aids neural models learn the intrinsic\nproperties of data. In this work, we describe our adversarial multi-task\nnetwork, AMTL-Humor, used to detect and rate humor and offensive texts from\nTask 7 at SemEval-2021. Each branch from the model is focused on solving a\nrelated task, and consists of a BiLSTM layer followed by Capsule layers, on top\nof BERTweet used for generating contextualized embeddings. Our best model\nconsists of an ensemble of all tested configurations, and achieves a 95.66%\nF1-score and 94.70% accuracy for Task 1a, while obtaining RMSE scores of 0.6200\nand 0.5318 for Tasks 1b and 2, respectively.\n