Physically Plausible Data Augmentations for Wearable IMU-based Human Activity Recognition Using Physics Simulation

The scarcity of high-quality labeled data in sensor-based human activity recognition (HAR) hinders model performance and limits generalization across realworld scenarios. Data augmentation is a key strategy to mitigate this issue by enhancing the diversity of training datasets. Signal transformation-based data augmentation (STDA) techniques have been widely used in HAR. However, these methods are often physically implausible and can produce data misaligned with activity labels. In this study, we propose physically plausible data augmentation (PPDA) enabled by physics simulation. PPDA leverages human body movement data from motion capture or video-based pose estimation and incorporates realistic variability through physics simulation, including changes in body movements, sensor placements, and hardware-related effects. We compare the performance of PPDA methods with traditional STDA methods on three public datasets of activities of daily living and fitness workouts: REALDISP (34 classes), REALWORLD (eight classes), and MM-Fit (11 classes). First, we compare each PPDA method with its closest STDA counterpart, showing that PPDA improves macro F1-scores by an average of 3.7 pp, with gains up to 13 pp. Second, we assess to what extent combining PPDAs can reduce the need for initial data collection, and in comparison with combined STDAs, we observe that PPDA achieves competitive performance with up to 60% fewer training subjects. As the first systematic study of PPDA in HAR, these results highlight the advantages of pursuing physical plausibility in data augmentation and the potential of physics simulation for generating synthetic inertial measurement unit (IMU) data for training deep learning HAR models. This cost-effective and scalable approach, therefore, helps address the annotation scarcity challenge in HAR.