JDAT: joint-dimension-aware transformer with strong flexibility for EEG emotion recognition

Abstract

Electroencephalography (EEG) emotion recognition is an important task in Human-Computer Interaction (HCI), which has critical applications in modern electronic devices. In recent years, significant advancements have been made in EEG emotion recognition through deep learning algorithms and the incorporation of attention mechanisms. However, EEG signals possess complex and multi-dimensional features, and there is a lack of a compact method with unified attention modules for EEG-based emotion recognition. In this paper, we propose a novel network called Joint-Dimension-Aware Transformer (JDAT) based on the novel squeezed Multi-head Self-Attention (MSA) mechanism for EEG emotion recognition. The adaptive Squeezed MSA is applied to multi-dimensional features, allowing JDAT to focus on diverse EEG information, including space, frequency, and time. Through joint-dimension attention, JDAT is able to capture complicated brain activities, such as signal activation, phase-intensity couplings, and resonance. The proposed JDAT is evaluated on the DEAP, DREAMER, and SEED datasets, and experimental results show that it outperforms state-of-the-art methods. The ablation study further illustrates the performance improvement brought by joint-dimension attention.