3D Object Detection and Pose Estimation of Unseen Objects in Color Images with Local Surface Embeddings

We present an approach for detecting and estimating the 3D poses of objects\nin images that requires only an untextured CAD model and no training phase for\nnew objects. Our approach combines Deep Learning and 3D geometry: It relies on\nan embedding of local 3D geometry to match the CAD models to the input images.\nFor points at the surface of objects, this embedding can be computed directly\nfrom the CAD model; for image locations, we learn to predict it from the image\nitself. This establishes correspondences between 3D points on the CAD model and\n2D locations of the input images. However, many of these correspondences are\nambiguous as many points may have similar local geometries. We show that we can\nuse Mask-RCNN in a class-agnostic way to detect the new objects without\nretraining and thus drastically limit the number of possible correspondences.\nWe can then robustly estimate a 3D pose from these discriminative\ncorrespondences using a RANSAC- like algorithm. We demonstrate the performance\nof this approach on the T-LESS dataset, by using a small number of objects to\nlearn the embedding and testing it on the other objects. Our experiments show\nthat our method is on par or better than previous methods.\n