KinematicRL: A Sim-to-Real Reinforcement Learning Framework For Social Navigation With Kinodynamic Feasibility

Deep Reinforcement Learning (DRL) has shown promise for social navigation, yet its real-world deployment remains hindered by a persistent sim-to-real gap arising from simplified first-order dynamics and context-specific human state estimation pipelines. This work presents a unified framework that addresses these limitations to produce dynamically feasible navigation policies suitable for real-world deployment. First, theoretical analysis reveals that tracking error between simulated and actual robot position decays exponentially with increased control order, motivating the use of higher-order control inputs as DRL action space. A second-order control formulation tailored to differential drive robots is developed, complemented by a stochastic iterative Linear Quadratic Regulator (iLQR) that pretrains the policy via a divergence minimization objective. Second, to avoid the added system complexity of camera-LiDAR fusion, a cluster-based human tracking pipeline using only 2D LiDAR is introduced. Human detections are associated according to both spatial proximity and velocity similarity, enabling reliable differentiation of nearby pedestrians and yielding stable velocity estimates through temporal aggregation. Third, we introduce an unbiased residual gating block to balance reaction- and memory-based behaviors while handling time-varying crowd sizes, both critical for social navigation. The resulting policy, KinematicRL, consistently improves kinematic performance and adapts to varying number of detected humans. Experiments in real-world environments demonstrate that, when combined with the proposed tracking pipeline, KinematicRL can be deployed on a real differential drive robot with minimal modifications. Note to Practitioners—This work is motivated by the potential of deep reinforcement learning in social navigation and the sim-to-real gap widely witnessed during industrial applications. A comprehensive framework is proposed, ensuring dynamic feasibility in simulation, while effectively providing robust position and velocity estimates from 2D LiDAR data. By incorporating second-order control and warm-starting the policy with a stochastic iterative Linear Quadratic Regulator, our approach ensures dynamic feasibility and enhances kinematic performance for differential drive robots. The proposed cluster-based tracking pipeline relies solely on 2D LiDAR, utilizing online clustering with velocity similarity for robust matching, and historical averaging for smooth velocity estimation. Additionally, a transformer architecture is proposed to handle varying number of detected humans resulting from sensor limitations. Real-world experiments further suggest that a relatively high control rate effectively mitigates jerk-related approximation errors in second-order models, enabling the policy to be directly transferred to a real physical robot with minimal modifications. The proposed methods are computationally efficient and require only standard 2D LiDAR sensors, making them accessible for real-world deployment in industrial and service robotics settings.