Pedestrian motion prediction is a fundamental task for autonomous robots and\nvehicles to operate safely. In recent years many complex approaches based on\nneural networks have been proposed to address this problem. In this work we\nshow that - surprisingly - a simple Constant Velocity Model can outperform even\nstate-of-the-art neural models. This indicates that either neural networks are\nnot able to make use of the additional information they are provided with, or\nthat this information is not as relevant as commonly believed. Therefore, we\nanalyze how neural networks process their input and how it impacts their\npredictions. Our analysis reveals pitfalls in training neural networks for\npedestrian motion prediction and clarifies false assumptions about the problem\nitself. In particular, neural networks implicitly learn environmental priors\nthat negatively impact their generalization capability, the motion history of\npedestrians is irrelevant and interactions are too complex to predict. Our work\nshows how neural networks for pedestrian motion prediction can be thoroughly\nevaluated and our results indicate which research directions for neural motion\nprediction are promising in future.\n