The Era of End-to-End Autonomy: Transitioning from Rule-Based Driving to Large Driving Models

Autonomous driving is undergoing a major architectural transition from modular, rule-based pipelines toward learning-based and increasingly end-to-end (E2E) driving systems. This paper examines this transition by tracing the evolution from classical sense–perceive–plan–control architectures to large driving models (LDMs) that integrate perception, prediction, planning, and control within unified learning frameworks. We review recent academic and industrial developments, including Tesla’s Full Self-Driving (FSD) V12–V14, Rivian’s Unified Intelligence platform, NVIDIA Cosmos, and emerging robotaxi deployments, with particular emphasis on Tesla FSD because it represents one of the most widely deployed supervised E2E driving systems currently available to consumers. The analysis focuses on architectural design, deployment pathways, safety challenges, and industry implications. Particular attention is given to the emerging category of supervised E2E driving, often described as FSD (Supervised) or L2++, in which the vehicle performs a substantial portion of the Dynamic Driving Task (DDT) while the human driver remains responsible for supervision and fallback intervention. We discuss the technical opportunities of these systems, including their potential to learn from large-scale fleet data and improve performance in long-tail driving scenarios, while also examining unresolved challenges related to validation, transparency, human–machine interaction, driver attention, liability, and regulatory assessment. The paper further argues that combining vision with range sensing can support continuous training and validation of camera-based depth and scene-understanding models. Finally, we consider how the architectural principles emerging in autonomous driving may extend to broader embodied AI systems, including humanoid robotics and other safety-critical autonomous platforms.