Hierarchical Kinematic Probability Distributions for 3D Human Shape and Pose Estimation from Images in the Wild

This paper addresses the problem of 3D human body shape and pose estimation\nfrom an RGB image. This is often an ill-posed problem, since multiple plausible\n3D bodies may match the visual evidence present in the input - particularly\nwhen the subject is occluded. Thus, it is desirable to estimate a distribution\nover 3D body shape and pose conditioned on the input image instead of a single\n3D reconstruction. We train a deep neural network to estimate a hierarchical\nmatrix-Fisher distribution over relative 3D joint rotation matrices (i.e. body\npose), which exploits the human body's kinematic tree structure, as well as a\nGaussian distribution over SMPL body shape parameters. To further ensure that\nthe predicted shape and pose distributions match the visual evidence in the\ninput image, we implement a differentiable rejection sampler to impose a\nreprojection loss between ground-truth 2D joint coordinates and samples from\nthe predicted distributions, projected onto the image plane. We show that our\nmethod is competitive with the state-of-the-art in terms of 3D shape and pose\nmetrics on the SSP-3D and 3DPW datasets, while also yielding a structured\nprobability distribution over 3D body shape and pose, with which we can\nmeaningfully quantify prediction uncertainty and sample multiple plausible 3D\nreconstructions to explain a given input image. Code is available at\nhttps://github.com/akashsengupta1997/HierarchicalProbabilistic3DHuman .\n