Understanding and anticipating human activity is an important capability for\nintelligent systems in mobile robotics, autonomous driving, and video\nsurveillance. While learning from demonstrations with on-site collected\ntrajectory data is a powerful approach to discover recurrent motion patterns,\ngeneralization to new environments, where sufficient motion data are not\nreadily available, remains a challenge. In many cases, however, semantic\ninformation about the environment is a highly informative cue for the\nprediction of pedestrian motion or the estimation of collision risks. In this\nwork, we infer occupancy priors of human motion using only semantic environment\ninformation as input. To this end we apply and discuss a traditional Inverse\nOptimal Control approach, and propose a novel one based on Convolutional Neural\nNetworks (CNN) to predict future occupancy maps. Our CNN method produces\nflexible context-aware occupancy estimations for semantically uniform map\nregions and generalizes well already with small amounts of training data.\nEvaluated on synthetic and real-world data, it shows superior results compared\nto several baselines, marking a qualitative step-up in semantic environment\nassessment.\n