Neural Deformation Graphs for Globally-consistent Non-rigid Reconstruction

We introduce Neural Deformation Graphs for globally-consistent deformation\ntracking and 3D reconstruction of non-rigid objects. Specifically, we\nimplicitly model a deformation graph via a deep neural network. This neural\ndeformation graph does not rely on any object-specific structure and, thus, can\nbe applied to general non-rigid deformation tracking. Our method globally\noptimizes this neural graph on a given sequence of depth camera observations of\na non-rigidly moving object. Based on explicit viewpoint consistency as well as\ninter-frame graph and surface consistency constraints, the underlying network\nis trained in a self-supervised fashion. We additionally optimize for the\ngeometry of the object with an implicit deformable multi-MLP shape\nrepresentation. Our approach does not assume sequential input data, thus\nenabling robust tracking of fast motions or even temporally disconnected\nrecordings. Our experiments demonstrate that our Neural Deformation Graphs\noutperform state-of-the-art non-rigid reconstruction approaches both\nqualitatively and quantitatively, with 64% improved reconstruction and 62%\nimproved deformation tracking performance.\n