Sat-NeRF: Learning Multi-View Satellite Photogrammetry With Transient Objects and Shadow Modeling Using RPC Cameras

We introduce the Satellite Neural Radiance Field (Sat-NeRF), a new end-to-end\nmodel for learning multi-view satellite photogrammetry in the wild. Sat-NeRF\ncombines some of the latest trends in neural rendering with native satellite\ncamera models, represented by rational polynomial coefficient (RPC) functions.\nThe proposed method renders new views and infers surface models of similar\nquality to those obtained with traditional state-of-the-art stereo pipelines.\nMulti-date images exhibit significant changes in appearance, mainly due to\nvarying shadows and transient objects (cars, vegetation). Robustness to these\nchallenges is achieved by a shadow-aware irradiance model and uncertainty\nweighting to deal with transient phenomena that cannot be explained by the\nposition of the sun. We evaluate Sat-NeRF using WorldView-3 images from\ndifferent locations and stress the advantages of applying a bundle adjustment\nto the satellite camera models prior to training. This boosts the network\nperformance and can optionally be used to extract additional cues for depth\nsupervision.\n