Recovering Trajectories of Unmarked Joints in 3D Human Actions Using Latent Space Optimization

Motion capture (mocap) and time-of-flight based sensing of human actions are\nbecoming increasingly popular modalities to perform robust activity analysis.\nApplications range from action recognition to quantifying movement quality for\nhealth applications. While marker-less motion capture has made great progress,\nin critical applications such as healthcare, marker-based systems, especially\nactive markers, are still considered gold-standard. However, there are several\npractical challenges in both modalities such as visibility, tracking errors,\nand simply the need to keep marker setup convenient wherein movements are\nrecorded with a reduced marker-set. This implies that certain joint locations\nwill not even be marked-up, making downstream analysis of full body movement\nchallenging. To address this gap, we first pose the problem of reconstructing\nthe unmarked joint data as an ill-posed linear inverse problem. We recover\nmissing joints for a given action by projecting it onto the manifold of human\nactions, this is achieved by optimizing the latent space representation of a\ndeep autoencoder. Experiments on both mocap and Kinect datasets clearly\ndemonstrate that the proposed method performs very well in recovering semantics\nof the actions and dynamics of missing joints. We will release all the code and\nmodels publicly.\n