The limits of applicability of vision-and-language models are defined by the\ncoverage of their training data. Tasks like vision question answering (VQA)\noften require commonsense and factual information beyond what can be learned\nfrom task-specific datasets. This paper investigates the injection of knowledge\nfrom general-purpose knowledge bases (KBs) into vision-and-language\ntransformers. We use an auxiliary training objective that encourages the\nlearned representations to align with graph embeddings of matching entities in\na KB. We empirically study the relevance of various KBs to multiple tasks and\nbenchmarks. The technique brings clear benefits to knowledge-demanding question\nanswering tasks (OK-VQA, FVQA) by capturing semantic and relational knowledge\nabsent from existing models. More surprisingly, the technique also benefits\nvisual reasoning tasks (NLVR2, SNLI-VE). We perform probing experiments and\nshow that the injection of additional knowledge regularizes the space of\nembeddings, which improves the representation of lexical and semantic\nsimilarities. The technique is model-agnostic and can expand the applicability\nof any vision-and-language transformer with minimal computational overhead.\n