UPB at SemEval-2020 Task 8: Joint Textual and Visual Modeling in a Multi-Task Learning Architecture for Memotion Analysis

Users from the online environment can create different ways of expressing\ntheir thoughts, opinions, or conception of amusement. Internet memes were\ncreated specifically for these situations. Their main purpose is to transmit\nideas by using combinations of images and texts such that they will create a\ncertain state for the receptor, depending on the message the meme has to send.\nThese posts can be related to various situations or events, thus adding a funny\nside to any circumstance our world is situated in. In this paper, we describe\nthe system developed by our team for SemEval-2020 Task 8: Memotion Analysis.\nMore specifically, we introduce a novel system to analyze these posts, a\nmultimodal multi-task learning architecture that combines ALBERT for text\nencoding with VGG-16 for image representation. In this manner, we show that the\ninformation behind them can be properly revealed. Our approach achieves good\nperformance on each of the three subtasks of the current competition, ranking\n11th for Subtask A (0.3453 macro F1-score), 1st for Subtask B (0.5183 macro\nF1-score), and 3rd for Subtask C (0.3171 macro F1-score) while exceeding the\nofficial baseline results by high margins.\n