Maintaining a Reliable World Model using Action-aware Perceptual Anchoring

Reliable perception is essential for robots that interact with the world. But\nsensors alone are often insufficient to provide this capability, and they are\nprone to errors due to various conditions in the environment. Furthermore,\nthere is a need for robots to maintain a model of its surroundings even when\nobjects go out of view and are no longer visible. This requires anchoring\nperceptual information onto symbols that represent the objects in the\nenvironment. In this paper, we present a model for action-aware perceptual\nanchoring that enables robots to track objects in a persistent manner. Our\nrule-based approach considers inductive biases to perform high-level reasoning\nover the results from low-level object detection, and it improves the robot's\nperceptual capability for complex tasks. We evaluate our model against existing\nbaseline models for object permanence and show that it outperforms these on a\nsnitch localisation task using a dataset of 1,371 videos. We also integrate our\naction-aware perceptual anchoring in the context of a cognitive architecture\nand demonstrate its benefits in a realistic gearbox assembly task on a\nUniversal Robot.\n