Contrastive Learning from Extremely Augmented Skeleton Sequences for Self-supervised Action Recognition

In recent years, self-supervised representation learning for skeleton-based\naction recognition has been developed with the advance of contrastive learning\nmethods. The existing contrastive learning methods use normal augmentations to\nconstruct similar positive samples, which limits the ability to explore novel\nmovement patterns. In this paper, to make better use of the movement patterns\nintroduced by extreme augmentations, a Contrastive Learning framework utilizing\nAbundant Information Mining for self-supervised action Representation (AimCLR)\nis proposed. First, the extreme augmentations and the Energy-based\nAttention-guided Drop Module (EADM) are proposed to obtain diverse positive\nsamples, which bring novel movement patterns to improve the universality of the\nlearned representations. Second, since directly using extreme augmentations may\nnot be able to boost the performance due to the drastic changes in original\nidentity, the Dual Distributional Divergence Minimization Loss (D$^3$M Loss) is\nproposed to minimize the distribution divergence in a more gentle way. Third,\nthe Nearest Neighbors Mining (NNM) is proposed to further expand positive\nsamples to make the abundant information mining process more reasonable.\nExhaustive experiments on NTU RGB+D 60, PKU-MMD, NTU RGB+D 120 datasets have\nverified that our AimCLR can significantly perform favorably against\nstate-of-the-art methods under a variety of evaluation protocols with observed\nhigher quality action representations. Our code is available at\nhttps://github.com/Levigty/AimCLR.\n