Amodal 3D Reconstruction for Robotic Manipulation via Stability and Connectivity

Learning-based 3D object reconstruction enables single- or few-shot\nestimation of 3D object models. For robotics, this holds the potential to allow\nmodel-based methods to rapidly adapt to novel objects and scenes. Existing 3D\nreconstruction techniques optimize for visual reconstruction fidelity,\ntypically measured by chamfer distance or voxel IOU. We find that when applied\nto realistic, cluttered robotics environments, these systems produce\nreconstructions with low physical realism, resulting in poor task performance\nwhen used for model-based control. We propose ARM, an amodal 3D reconstruction\nsystem that introduces (1) a stability prior over object shapes, (2) a\nconnectivity prior, and (3) a multi-channel input representation that allows\nfor reasoning over relationships between groups of objects. By using these\npriors over the physical properties of objects, our system improves\nreconstruction quality not just by standard visual metrics, but also\nperformance of model-based control on a variety of robotics manipulation tasks\nin challenging, cluttered environments. Code is available at\ngithub.com/wagnew3/ARM.\n