IDAIS-Oxford Statement (2023)

Summary

Encourages governments worldwide to develop measures to prevent worst-case AI outcomes from malicious or careless actors and manage AI safety risks.

Recommends mandatory registration for creating, selling, or using AI models above a certain capability threshold to enhance government visibility into emerging risks.

Encourages governments to implement requirements to monitor large-scale data centers, track AI incidents, and mandate independent third-party audits for AI developers concerning information security and model safety.

Proposes defining clear red lines that, if crossed, mandate immediate termination of an AI system through rapid and safe shut-down procedures.

Urges international cooperation to establish and maintain the capacity to shut down AI systems crossing red lines.

Calls for AI developers to demonstrate to regulators that their systems will not cross defined red lines before deployment.

Advocates for substantial research progress to ensure advanced AI alignment with designers' intent and robustness against malicious actors.

Encourages a global network of AI safety research and governance institutions, with leading developers committing significant resources to AI safety.

Emphasizes global cooperation to manage AI risks and ensure safety as a public good.

Global action, cooperation, and capacity building are key to managing risk from AI and enabling humanity to share in its benefits. AI safety is a global public good that should be supported by public and private investment, with advances in safety shared widely. Governments around the world — especially of leading AI nations — have a responsibility to develop measures to prevent worst-case outcomes from malicious or careless actors and to rein in reckless competition. The international community should work to create an international coordination process for advanced AI in this vein.

We face near-term risks from malicious actors misusing frontier AI systems, with current safety filters integrated by developers easily bypassed. Frontier AI systems produce compelling misinformation and may soon be capable enough to help terrorists develop weapons of mass destruction. Moreover, there is a serious risk that future AI systems may escape human control altogether. Even aligned AI systems could destabilize or disempower existing institutions. Taken together, we believe AI may pose an existential risk to humanity in the coming decades.

Recommends mandatory AI registration, monitoring, independent audits, risk assessments, and termination procedures for high-capability AI models and systems.

In domestic regulation, we recommend mandatory registration for the creation, sale or use of models above a certain capability threshold, including open-source copies and derivatives, to enable governments to acquire critical and currently missing visibility into emerging risks. Governments should monitor large-scale data centers and track AI incidents, and should require that AI developers of frontier models be subject to independent third-party audits evaluating their information security and model safety. AI developers should also be required to share comprehensive risk assessments, policies around risk management, and predictions about their systems’ behaviour in third party evaluations and post-deployment with relevant authorities.

We also recommend defining clear red lines that, if crossed, mandate immediate termination of an AI system — including all copies — through rapid and safe shut-down procedures. Governments should cooperate to instantiate and preserve this capacity. Moreover, prior to deployment as well as during training for the most advanced models, developers should demonstrate to regulators’ satisfaction that their system(s) will not cross these red lines.

Encourages government agencies and the global AI research community to enhance AI safety through significant funding and collaboration.

Reaching adequate safety levels for advanced AI will also require immense research progress. Advanced AI systems must be demonstrably aligned with their designer’s intent, as well as appropriate norms and values. They must also be robust against both malicious actors and rare failure modes. Sufficient human control needs to be ensured for these systems. Concerted effort by the global research community in both AI and other disciplines is essential; we need a global network of dedicated AI safety research and governance institutions. We call on leading AI developers to make a minimum spending commitment of one third of their AI R&D on AI safety and for government agencies to fund academic and non-profit AI safety and governance research in at least the same proportion.