The LAIA Dataset: Labelled Attention for Intelligent Automobiles

The development of autonomous vehicles (AVs) usually relies heavily on data-driven artificial intelligence (AI) models that require large volumes of sensor data with ground-truth annotations. While modular architectures are widely used, end-to-end driving paradigms offer a promising alternative by directly mapping sensor inputs to control actions. However, their adoption is limited by challenges in interpretability and explainability. To address this, we present LAIA (Labelled Attention for Intelligent Automobiles), a novel synthetic dataset designed to enrich end-to-end driving research with human attention data. Collected using the CARLA simulator in closed-loop environments, LAIA comprises over 15 hours of driving from 44 participants across carefully crafted scenarios designed to evoke natural responses. Each sequence includes RGB images under six weather conditions, semantic and instance segmentation, depth, optical flow, CAN bus signals, and synchronized eye-tracking data. LAIA enables applications including training attention-aware end-to-end AI drivers, predicting driver behavior, developing methods to detect anomalous driver-attention patterns, and improving model explainability. In this work, we use LAIA to compare human attention with the perceptual attention emerging in our end-to-end driving models, thereby providing insight into their behavior.

Paper

References (11)

04“The enemy of good: Estimating the cost of waiting for nearly perfect automated vehicles,”2017 · Tech. Rep
06CAN bus information at every timestamp, at a frequency of 25 Hza
07“Tobii pro glasses 3 — latest in wearable eye tracking,”
08Panoptic Segmentation: instance (Fig 12(b)) and semantic segmentation (Fig 12(c))
09Depth (pixel distance obtained per camera using a CARLA depth camera)
10c) Log of the driving simulation to allow offline playback of the driving episodes
11“LAIA: Labelled Attention for Intelligent Automobiles,”

Similar papers

© 2026 NYSGPT2525 LLC