Multi-Object Active Search and Tracking by Multiple Agents in Untrusted, Dynamically Changing Environments

This paper addresses the problem of both actively searching and tracking multiple unknown dynamic objects in a known environment with multiple cooperative autonomous agents with partial observability. The tracking of a target ends when the uncertainty is below a specified threshold. Current methods typically assume homogeneous agents without access to external information and utilize short-horizon target predictive models. Such assumptions limit real-world applications. We propose a fully integrated pipeline where the main novel contributions are: (1) a time-varying weighted belief representation capable of handling knowledge that changes over time, which includes external reports of varying levels of trustworthiness in addition to the agents involved; (2) the integration of a Long Short Term Memory-based trajectory prediction within the optimization framework for long-horizon decision-making, which accounts for trajectory prediction in time-configuration space, thus increasing responsiveness; and (3) a comprehensive system that accounts for multiple agents and enables information-driven optimization during both the search and track tasks. When communication is available, our proposed strategy consolidates exploration results collected asynchronously by agents and external sources into a headquarters, who can allocate each agent to maximize the overall team’s utility, effectively using all available information. We tested our approach extensively in Monte Carlo simulations against baselines, representative of classes of approaches from the literature, and in robustness and ablation studies. In addition, we performed experiments in a 3D physics based engine robot simulator to test the applicability in the real world, as well as with real-world trajectories obtained from an oceanography computational fluid dynamics simulator. Results show the effectiveness of our proposed method, which achieves mission completion times that are 1.3 to 3.2 times faster in finding all targets, in most scenarios, including challenging ones, where the number of targets is 5 times greater than that of the agents.