DIVERSE FUTURE PREDICTIONS FOR PEDESTRIANS

Methods and systems for trajectory prediction include encoding trajectories of agents in a scene from past images of the scene. Lane centerlines are encoded for agents in the scene. The agents in the scene are encoded using the encoded trajectories and the encoded lane centerlines. A hypercolumn trajectory is decoded from the encoded agents to generate predicted trajectories for the agents. A vehicle is automatically operated responsive to the predicted trajectories.