Predicting Sharp and Accurate Occlusion Boundaries in Monocular Depth Estimation Using Displacement Fields

Current methods for depth map prediction from monocular images tend to\npredict smooth, poorly localized contours for the occlusion boundaries in the\ninput image. This is unfortunate as occlusion boundaries are important cues to\nrecognize objects, and as we show, may lead to a way to discover new objects\nfrom scene reconstruction. To improve predicted depth maps, recent methods rely\non various forms of filtering or predict an additive residual depth map to\nrefine a first estimate. We instead learn to predict, given a depth map\npredicted by some reconstruction method, a 2D displacement field able to\nre-sample pixels around the occlusion boundaries into sharper reconstructions.\nOur method can be applied to the output of any depth estimation method, in an\nend-to-end trainable fashion. For evaluation, we manually annotated the\nocclusion boundaries in all the images in the test split of popular NYUv2-Depth\ndataset. We show that our approach improves the localization of occlusion\nboundaries for all state-of-the-art monocular depth estimation methods that we\ncould evaluate, without degrading the depth accuracy for the rest of the\nimages.\n