3D Multi-bodies: Fitting Sets of Plausible 3D Human Models to Ambiguous Image Data

We consider the problem of obtaining dense 3D reconstructions of humans from\nsingle and partially occluded views. In such cases, the visual evidence is\nusually insufficient to identify a 3D reconstruction uniquely, so we aim at\nrecovering several plausible reconstructions compatible with the input data. We\nsuggest that ambiguities can be modelled more effectively by parametrizing the\npossible body shapes and poses via a suitable 3D model, such as SMPL for\nhumans. We propose to learn a multi-hypothesis neural network regressor using a\nbest-of-M loss, where each of the M hypotheses is constrained to lie on a\nmanifold of plausible human poses by means of a generative model. We show that\nour method outperforms alternative approaches in ambiguous pose recovery on\nstandard benchmarks for 3D humans, and in heavily occluded versions of these\nbenchmarks.\n