Endo-Depth-and-Motion: Reconstruction and Tracking in Endoscopic Videos using Depth Networks and Photometric Constraints

Estimating a scene reconstruction and the camera motion from in-body videos\nis challenging due to several factors, e.g. the deformation of in-body cavities\nor the lack of texture. In this paper we present Endo-Depth-and-Motion, a\npipeline that estimates the 6-degrees-of-freedom camera pose and dense 3D scene\nmodels from monocular endoscopic videos. Our approach leverages recent advances\nin self-supervised depth networks to generate pseudo-RGBD frames, then tracks\nthe camera pose using photometric residuals and fuses the registered depth maps\nin a volumetric representation. We present an extensive experimental evaluation\nin the public dataset Hamlyn, showing high-quality results and comparisons\nagainst relevant baselines. We also release all models and code for future\ncomparisons.\n