The Multimodal Sentiment Analysis in Car Reviews (MuSe-CaR) Dataset: Collection, Insights and Improvements

Truly real-life data presents a strong, but exciting challenge for sentiment\nand emotion research. The high variety of possible `in-the-wild' properties\nmakes large datasets such as these indispensable with respect to building\nrobust machine learning models. A sufficient quantity of data covering a deep\nvariety in the challenges of each modality to force the exploratory analysis of\nthe interplay of all modalities has not yet been made available in this\ncontext. In this contribution, we present MuSe-CaR, a first of its kind\nmultimodal dataset. The data is publicly available as it recently served as the\ntesting bed for the 1st Multimodal Sentiment Analysis Challenge, and focused on\nthe tasks of emotion, emotion-target engagement, and trustworthiness\nrecognition by means of comprehensively integrating the audio-visual and\nlanguage modalities. Furthermore, we give a thorough overview of the dataset in\nterms of collection and annotation, including annotation tiers not used in this\nyear's MuSe 2020. In addition, for one of the sub-challenges - predicting the\nlevel of trustworthiness - no participant outperformed the baseline model, and\nso we propose a simple, but highly efficient Multi-Head-Attention network that\nexceeds using multimodal fusion the baseline by around 0.2 CCC (almost 50 %\nimprovement).\n