The trajectory prediction is significant for the decision-making of\nautonomous driving vehicles. In this paper, we propose a model to predict the\ntrajectories of target agents around an autonomous vehicle. The main idea of\nour method is considering the history trajectories of the target agent and the\ninfluence of surrounding agents on the target agent. To this end, we encode the\ntarget agent history trajectories as an attention mask and construct a social\nmap to encode the interactive relationship between the target agent and its\nsurrounding agents. Given a trajectory sequence, the LSTM networks are firstly\nutilized to extract the features for all agents, based on which the attention\nmask and social map are formed. Then, the attention mask and social map are\nfused to get the fusion feature map, which is processed by the social\nconvolution to obtain a fusion feature representation. Finally, this fusion\nfeature is taken as the input of a variable-length LSTM to predict the\ntrajectory of the target agent. We note that the variable-length LSTM enables\nour model to handle the case that the number of agents in the sensing scope is\nhighly dynamic in traffic scenes. To verify the effectiveness of our method, we\nwidely compare with several methods on a public dataset, achieving a 20% error\ndecrease. In addition, the model satisfies the real-time requirement with the\n32 fps.\n