Accurate, Low-Latency Visual Perception for Autonomous Racing:Challenges, Mechanisms, and Practical Solutions

Autonomous racing provides the opportunity to test safety-critical perception\npipelines at their limit. This paper describes the practical challenges and\nsolutions to applying state-of-the-art computer vision algorithms to build a\nlow-latency, high-accuracy perception system for DUT18 Driverless (DUT18D), a\n4WD electric race car with podium finishes at all Formula Driverless\ncompetitions for which it raced. The key components of DUT18D include\nYOLOv3-based object detection, pose estimation, and time synchronization on its\ndual stereovision/monovision camera setup. We highlight modifications required\nto adapt perception CNNs to racing domains, improvements to loss functions used\nfor pose estimation, and methodologies for sub-microsecond camera\nsynchronization among other improvements. We perform a thorough experimental\nevaluation of the system, demonstrating its accuracy and low-latency in\nreal-world racing scenarios.\n