Single-Shot Cuboids: Geodesics-based End-to-end Manhattan Aligned Layout Estimation from Spherical Panoramas

It has been shown that global scene understanding tasks like layout\nestimation can benefit from wider field of views, and specifically spherical\npanoramas. While much progress has been made recently, all previous approaches\nrely on intermediate representations and postprocessing to produce\nManhattan-aligned estimates. In this work we show how to estimate full room\nlayouts in a single-shot, eliminating the need for postprocessing. Our work is\nthe first to directly infer Manhattan-aligned outputs. To achieve this, our\ndata-driven model exploits direct coordinate regression and is supervised\nend-to-end. As a result, we can explicitly add quasi-Manhattan constraints,\nwhich set the necessary conditions for a homography-based Manhattan alignment\nmodule. Finally, we introduce the geodesic heatmaps and loss and a\nboundary-aware center of mass calculation that facilitate higher quality\nkeypoint estimation in the spherical domain. Our models and code are publicly\navailable at https://vcl3d.github.io/SingleShotCuboids/.\n