DRWR: A Differentiable Renderer without Rendering for Unsupervised 3D Structure Learning from Silhouette Images

Differentiable renderers have been used successfully for unsupervised 3D\nstructure learning from 2D images because they can bridge the gap between 3D\nand 2D. To optimize 3D shape parameters, current renderers rely on pixel-wise\nlosses between rendered images of 3D reconstructions and ground truth images\nfrom corresponding viewpoints. Hence they require interpolation of the\nrecovered 3D structure at each pixel, visibility handling, and optionally\nevaluating a shading model. In contrast, here we propose a Differentiable\nRenderer Without Rendering (DRWR) that omits these steps. DRWR only relies on a\nsimple but effective loss that evaluates how well the projections of\nreconstructed 3D point clouds cover the ground truth object silhouette.\nSpecifically, DRWR employs a smooth silhouette loss to pull the projection of\neach individual 3D point inside the object silhouette, and a structure-aware\nrepulsion loss to push each pair of projections that fall inside the silhouette\nfar away from each other. Although we omit surface interpolation, visibility\nhandling, and shading, our results demonstrate that DRWR achieves\nstate-of-the-art accuracies under widely used benchmarks, outperforming\nprevious methods both qualitatively and quantitatively. In addition, our\ntraining times are significantly lower due to the simplicity of DRWR.\n