Shallow Optical Flow Three-Stream CNN for Macro- and Micro-Expression Spotting from Long Videos

Facial expressions vary from the visible to the subtle. In recent years, the\nanalysis of micro-expressions $-$ a natural occurrence resulting from the\nsuppression of one's true emotions, has drawn the attention of researchers with\na broad range of potential applications. However, spotting microexpressions in\nlong videos becomes increasingly challenging when intertwined with normal or\nmacro-expressions. In this paper, we propose a shallow optical flow\nthree-stream CNN (SOFTNet) model to predict a score that captures the\nlikelihood of a frame being in an expression interval. By fashioning the\nspotting task as a regression problem, we introduce pseudo-labeling to\nfacilitate the learning process. We demonstrate the efficacy and efficiency of\nthe proposed approach on the recent MEGC 2020 benchmark, where state-of-the-art\nperformance is achieved on CAS(ME)$^{2}$ with equally promising results on SAMM\nLong Videos.\n