Although pretrained language models (PTLMs) contain significant amounts of\nworld knowledge, they can still produce inconsistent answers to questions when\nprobed, even after specialized training. As a result, it can be hard to\nidentify what the model actually "believes" about the world, making it\nsusceptible to inconsistent behavior and simple errors. Our goal is to reduce\nthese problems. Our approach is to embed a PTLM in a broader system that also\nincludes an evolving, symbolic memory of beliefs -- a BeliefBank -- that\nrecords but then may modify the raw PTLM answers. We describe two mechanisms to\nimprove belief consistency in the overall system. First, a reasoning component\n-- a weighted MaxSAT solver -- revises beliefs that significantly clash with\nothers. Second, a feedback component issues future queries to the PTLM using\nknown beliefs as context. We show that, in a controlled experimental setting,\nthese two mechanisms result in more consistent beliefs in the overall system,\nimproving both the accuracy and consistency of its answers over time. This is\nsignificant as it is a first step towards PTLM-based architectures with a\nsystematic notion of belief, enabling them to construct a more coherent picture\nof the world, and improve over time without model retraining.\n