Deep Learning Method for Cell-Wise Object Tracking, Velocity Estimation and Projection of Sensor Data over Time

Current deep learning methods for environment segmentation and velocity estimation rely on Convolutional Recurrent Neural Networks to exploit spatio temporal relationships within obtained sensor data. In these approaches, scene dynamics are implicitly derived using ConvNets to correlate novel input and memorized data, which, for the task of recurrently processing sensor scans, suffer from architectural restrictions. In addition, these prior approaches lead to distorted data due to spatio temporal misalignments caused by the movement of external road users. In this work, we solve these issues with a novel Recurrent Neural Network unit. Within this unit, we track object encodings across consecutive frames on cell level by correlating key-query pairs derived from sensor inputs and memory states, respectively. We then use resulting tracking patterns to obtain scene dynamics and regress velocities. In a last step, the memory state of the Recurrent Neural Network is projected based on extracted velocity estimates to resolve the aforementioned spatio temporal misalignment.