The World Has Its First
AI Treaty.

On September 5, 2024, in Vilnius, Lithuania, representatives of the United States, the United Kingdom, the European Union, and dozens of other nations signed the Framework Convention on Artificial Intelligence and Human Rights, Democracy and the Rule of Law. It was the first legally binding international treaty on artificial intelligence ever concluded. It deserves to be understood clearly — both for what it achieves and for the problem it was never designed to solve.

What the treaty actually covers

The Framework Convention is a human rights instrument. Its legal obligations centre on ensuring that AI systems do not undermine democracy, human rights, or the rule of law. In practice, this means:

• Transparency requirements: people must be told when AI systems affect decisions about them
• Accountability mechanisms: states must ensure AI developers and deployers can be held responsible for harms
• Non-discrimination: AI systems must not produce outcomes that violate existing equality law
• Effective remedy: people harmed by AI must have access to legal redress
• Democratic safeguards: AI must not be used to undermine electoral processes or the rule of law

These are meaningful commitments. An AI hiring algorithm that discriminates on the basis of race, a facial recognition system deployed without transparency, an AI-generated disinformation campaign — all of these are directly addressed by the treaty's framework. Signatories have accepted binding obligations to prevent them.

The enforcement mechanism is a Conference of the Parties, which reviews implementation and can issue recommendations. It is softer than the verification regime of the Chemical Weapons Convention or the inspection protocols of the Nuclear Non-Proliferation Treaty — but it establishes the principle that AI is subject to binding international law. That principle was not established before September 2024. Its establishment is a genuine step forward.

What the treaty does not cover

The Framework Convention governs how AI is used in society. It does not govern whether AI systems that could pose civilisational risk should be developed in the first place.

The treaty's risk framework is calibrated to present and near-term AI harms — the genuine and serious harms that today's AI systems are already causing. This is the right scope for an instrument designed to protect human rights in AI-affected societies. It is the wrong scope for an instrument designed to prevent civilisational catastrophe from advanced AI.

Nowhere in the treaty's text does the word "superintelligence" appear. Nowhere does it impose obligations on the frontier AI development race. It has nothing to say about compute governance, about training run size, about the alignment problem, or about the scenario that preoccupies Geoffrey Hinton, Yoshua Bengio, and the other researchers whose warnings have accumulated over the past decade.

Why this distinction matters

The danger of mistaking the Framework Convention for adequate governance is not hypothetical. It is the natural outcome of the political process that produces international agreements: once a treaty exists, the political pressure to produce one has been satisfied. The countries that signed the Framework Convention can point to it as evidence that AI is being governed. The harder conversation — about whether the most capable AI systems being developed should be subject to binding international constraints on their development, not just their deployment — does not get easier after a governance milestone has been declared.

The Framework Convention governs the same AI that existing human rights and equality law already governs — it extends those principles into the AI context and makes them binding internationally. This is valuable. It is not a response to the risk that a sufficiently capable AI system, pursuing goals that are subtly misaligned with human welfare, could cause harm at a scale that no legal remedy can address because the harm is irreversible.

"The goal is for AI development to go well for AI and humans alike. That requires treating it as the civilisational issue it is — not as a product liability question."

Naoto Nakada, Founder · Nakada Foundation to Save Humanity

What a frontier AI safety treaty would require

A treaty designed to address the risk from advanced AI would look quite different from the Framework Convention. The Nakada Foundation's proposed framework outlines what such a treaty would need to include:

• Compute governance — binding limits on the computational resources that can be applied to a single training run, backed by hardware-level verification. Compute is the bottleneck in frontier AI development and the only variable currently measurable and enforceable at an international level.
• Independent verification — a monitoring mechanism analogous to nuclear inspections, allowing parties to verify compliance rather than relying on self-reporting by the organisations with the most to gain from non-compliance.
• Frontier AI coverage — obligations that apply specifically to the development of the most capable AI systems, not just their deployment in social contexts.
• US–China inclusion — any treaty that does not bind the two leading AI-developing nations will be structurally inadequate, regardless of how many other countries sign it.

The Framework Convention is a necessary first step. The recognition that AI is subject to binding international law is an important precedent. But the history of existential technology governance — nuclear, chemical, biological — shows that the first treaty in a domain rarely addresses the most serious risk. It takes a sustained political effort, over years, to build the frameworks that actually constrain what is most dangerous.

That political effort needs to begin now — not after the Framework Convention is declared to have solved the problem, and not after the capabilities exist that make binding constraints impossible to enforce.