Multimodal Future Localization and Emergence Prediction for Objects in Egocentric View with a Reachability Prior

In this paper, we investigate the problem of anticipating future dynamics,\nparticularly the future location of other vehicles and pedestrians, in the view\nof a moving vehicle. We approach two fundamental challenges: (1) the partial\nvisibility due to the egocentric view with a single RGB camera and considerable\nfield-of-view change due to the egomotion of the vehicle; (2) the multimodality\nof the distribution of future states. In contrast to many previous works, we do\nnot assume structural knowledge from maps. We rather estimate a reachability\nprior for certain classes of objects from the semantic map of the present image\nand propagate it into the future using the planned egomotion. Experiments show\nthat the reachability prior combined with multi-hypotheses learning improves\nmultimodal prediction of the future location of tracked objects and, for the\nfirst time, the emergence of new objects. We also demonstrate promising\nzero-shot transfer to unseen datasets. Source code is available at\n$\\href{https://github.com/lmb-freiburg/FLN-EPN-RPN}{\\text{this https URL.}}$\n