Following detection and tracking of traffic actors, prediction of their\nfuture motion is the next critical component of a self-driving vehicle (SDV)\ntechnology, allowing the SDV to operate safely and efficiently in its\nenvironment. This is particularly important when it comes to vulnerable road\nusers (VRUs), such as pedestrians and bicyclists. These actors need to be\nhandled with special care due to an increased risk of injury, as well as the\nfact that their behavior is less predictable than that of motorized actors. To\naddress this issue, in the current study we present a deep learning-based\nmethod for predicting VRU movement, where we rasterize high-definition maps and\nactor's surroundings into a bird's-eye view image used as an input to deep\nconvolutional networks. In addition, we propose a fast architecture suitable\nfor real-time inference, and perform an ablation study of various rasterization\napproaches to find the optimal choice for accurate prediction. The results\nstrongly indicate benefits of using the proposed approach for motion prediction\nof VRUs, both in terms of accuracy and latency.\n