ProbRadarM3F: mmWave Radar based Human Skeletal Pose Estimation with Probability Map Guided Multi-Format Feature Fusion

Millimeter wave (mmWave) radar is a nonintrusive, privacy-preserving, and cost-effective device, shown to be a viable alternative to RGB cameras for indoor human pose estimation. However, the challenge lies in fully leveraging the reflected radar signals for accurate pose estimation. To address this major challenge, this article introduces a probability map-guided multiformat feature fusion model, ProbRadarM3F. This is a radar feature extraction framework using a traditional fast Fourier transform method in parallel with a probability map-based positional encoding method. ProbRadarM3F fuses the traditional heatmap features and the positional features, then effectively achieves the estimation of 14 keypoints of the human body. Experimental evaluation on the HuPR dataset proves the effectiveness of 69.9% in average precision. The emphasis of our study is on utilizing position information in radar signals for estimating human skeletal pose. This provides direction for investigating other potential nonredundant information from mmWave radar.