Leveraging Neural Network Gradients within Trajectory Optimization for Proactive Human-Robot Interactions

To achieve seamless human-robot interactions, robots need to intimately\nreason about complex interaction dynamics and future human behaviors within\ntheir motion planning process. However, there is a disconnect between\nstate-of-the-art neural network-based human behavior models and robot motion\nplanners -- either the behavior models are limited in their consideration of\ndownstream planning or a simplified behavior model is used to ensure\ntractability of the planning problem. In this work, we present a framework that\nfuses together the interpretability and flexibility of trajectory optimization\n(TO) with the predictive power of state-of-the-art human trajectory prediction\nmodels. In particular, we leverage gradient information from data-driven\nprediction models to explicitly reason about human-robot interaction dynamics\nwithin a gradient-based TO problem. We demonstrate the efficacy of our approach\nin a multi-agent scenario whereby a robot is required to safely and efficiently\nnavigate through a crowd of up to ten pedestrians. We compare against a variety\nof planning methods, and show that by explicitly accounting for interaction\ndynamics within the planner, our method offers safer and more efficient\nbehaviors, even yielding proactive and nuanced behaviors such as waiting for a\npedestrian to pass before moving.\n