Just Label What You Need: Fine-Grained Active Selection for Perception and Prediction through Partially Labeled Scenes

Self-driving vehicles must perceive and predict the future positions of\nnearby actors in order to avoid collisions and drive safely. A learned deep\nlearning module is often responsible for this task, requiring large-scale,\nhigh-quality training datasets. As data collection is often significantly\ncheaper than labeling in this domain, the decision of which subset of examples\nto label can have a profound impact on model performance. Active learning\ntechniques, which leverage the state of the current model to iteratively select\nexamples for labeling, offer a promising solution to this problem. However,\ndespite the appeal of this approach, there has been little scientific analysis\nof active learning approaches for the perception and prediction (P&P) problem.\nIn this work, we study active learning techniques for P&P and find that the\ntraditional active learning formulation is ill-suited for the P&P setting. We\nthus introduce generalizations that ensure that our approach is both cost-aware\nand allows for fine-grained selection of examples through partially labeled\nscenes. Our experiments on a real-world, large-scale self-driving dataset\nsuggest that fine-grained selection can improve the performance across\nperception, prediction, and downstream planning tasks.\n