Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction

Implicit functions represented as deep learning approximations are powerful\nfor reconstructing 3D surfaces. However, they can only produce static surfaces\nthat are not controllable, which provides limited ability to modify the\nresulting model by editing its pose or shape parameters. Nevertheless, such\nfeatures are essential in building flexible models for both computer graphics\nand computer vision. In this work, we present methodology that combines\ndetail-rich implicit functions and parametric representations in order to\nreconstruct 3D models of people that remain controllable and accurate even in\nthe presence of clothing. Given sparse 3D point clouds sampled on the surface\nof a dressed person, we use an Implicit Part Network (IP-Net)to jointly predict\nthe outer 3D surface of the dressed person, the and inner body surface, and the\nsemantic correspondences to a parametric body model. We subsequently use\ncorrespondences to fit the body model to our inner surface and then non-rigidly\ndeform it (under a parametric body + displacement model) to the outer surface\nin order to capture garment, face and hair detail. In quantitative and\nqualitative experiments with both full body data and hand scans we show that\nthe proposed methodology generalizes, and is effective even given incomplete\npoint clouds collected from single-view depth images. Our models and code can\nbe downloaded from http://virtualhumans.mpi-inf.mpg.de/ipnet.\n