Ensuring that an AI system behaves reliably and as intended - especially in the presence of unexpected faults or adversarial conditions - is a complex challenge. Inspired by the field of Byzantine Fault Tolerance (BFT) from distributed computing, we explore a fault-tolerance architecture for AI safety. BFT refers to a system's ability to continue operating correctly even when some components misbehave arbitrarily or maliciously. In classical terms, a Byzantine fault-tolerant system can withstand up to $\boldsymbol{f}$ faulty nodes out of $\boldsymbol{N}$ as long as $\mathbf{N} \boldsymbol{>} \boldsymbol{=} \boldsymbol{3} \boldsymbol{f} \boldsymbol{+} \mathbf{1}$. By drawing an analogy between unreliable, corrupt, misbehaving or malicious AI artifacts and Byzantine nodes in a distributed system, we propose an architecture that leverages consensus mechanisms to enhance AI safety and reliability.