Tasks involving localization, memorization and planning in partially\nobservable 3D environments are an ongoing challenge in Deep Reinforcement\nLearning. We present EgoMap, a spatially structured neural memory architecture.\nEgoMap augments a deep reinforcement learning agent's performance in 3D\nenvironments on challenging tasks with multi-step objectives. The EgoMap\narchitecture incorporates several inductive biases including a differentiable\ninverse projection of CNN feature vectors onto a top-down spatially structured\nmap. The map is updated with ego-motion measurements through a differentiable\naffine transform. We show this architecture outperforms both standard recurrent\nagents and state of the art agents with structured memory. We demonstrate that\nincorporating these inductive biases into an agent's architecture allows for\nstable training with reward alone, circumventing the expense of acquiring and\nlabelling expert trajectories. A detailed ablation study demonstrates the\nimpact of key aspects of the architecture and through extensive qualitative\nanalysis, we show how the agent exploits its structured internal memory to\nachieve higher performance.\n