In this paper, we present LiRaNet, a novel end-to-end trajectory prediction\nmethod which utilizes radar sensor information along with widely used lidar and\nhigh definition (HD) maps. Automotive radar provides rich, complementary\ninformation, allowing for longer range vehicle detection as well as\ninstantaneous radial velocity measurements. However, there are factors that\nmake the fusion of lidar and radar information challenging, such as the\nrelatively low angular resolution of radar measurements, their sparsity and the\nlack of exact time synchronization with lidar. To overcome these challenges, we\npropose an efficient spatio-temporal radar feature extraction scheme which\nachieves state-of-the-art performance on multiple large-scale datasets.Further,\nby incorporating radar information, we show a 52% reduction in prediction error\nfor objects with high acceleration and a 16% reduction in prediction error for\nobjects at longer range.\n