Building a distributed spatial awareness within a swarm of locally sensing and communicating robots enables new swarm algorithms. We use local observations by robots of each other and Gaussian belief propagation message passing combined with continuous swarm movement to build a global and distributed swarm-centric frame of reference. With low bandwidth and computation requirements, this shared reference frame allows new swarm algorithms. We characterise the system in simulation and demonstrate two example algorithms, then demonstrate reliable performance on real robots with imperfect sensing.