Co-hosted by South Korea and the United Kingdom, the Seoul Summit was the second in the summit series launched at Bletchley Park. It advanced on two tracks at once — a corporate track, in which frontier developers made public safety commitments, and a governmental track, in which states agreed to deepen cooperation. Understanding what each track delivered, and what it left undone, is the clearest read on where AI governance stood in 2024.
The Frontier AI Safety Commitments
The headline output was the Frontier AI Safety Commitments, signed by sixteen major AI companies including developers from the US, China, the UK, and the UAE. In them, companies pledged to publish safety frameworks describing how they assess the risks of their frontier models, to define thresholds of risk deemed intolerable, and — critically — to not develop or deploy models if those thresholds could not be kept below the intolerable line, including a commitment to pull back where mitigations fail.
This was meaningful in one specific way: for the first time, leading developers publicly committed to the principle that some capabilities are too dangerous to deploy, and that they would define the red lines themselves and act on them. It moved the industry from vague assurances toward stated, if self-defined, limits.
Why voluntary commitments are not enough
- The companies set their own thresholds. Each developer defines what counts as intolerable risk and how to measure it. There is no common standard and no external body validating the definitions.
- There is no verification. Nothing independently confirms that a company's internal safety framework is followed, or that its evaluations are honest. Compliance rests on the developer's own word.
- There is no enforcement. A company that breaks its commitment faces reputational cost at most. There is no legal consequence, because the pledges are not law.
- They can be revised or dropped. Voluntary commitments made under competitive pressure can quietly erode when a rival races ahead — the exact dynamic that makes self-regulation fragile.
The network of safety institutes
The governmental track produced something with more durable potential: momentum toward an international network of AI safety institutes. Building on the UK's and US's newly created institutes, states agreed to cooperate on the science of evaluating frontier models — sharing methods, aligning testing, and building the technical capacity that any future verification regime would depend on. This is the connective tissue that a treaty would eventually need: public bodies with the expertise to actually assess whether a model is dangerous.
The Seoul Declaration, endorsed by the assembled governments, also reaffirmed the commitment to human-centric, safe, and trustworthy AI and to continued international cooperation. Like Bletchley's, it was a statement of direction rather than a set of obligations.
Seoul got sixteen companies to say, in public, that some AI capabilities are too dangerous to release. That is progress. But a promise the promiser can define, measure, and break on its own is not governance. It is a placeholder for governance.
Naoto Nakada, Founder · Nakada Foundation to Save Humanity
The pattern and the gap
Seoul fits a recognisable pattern in the governance of dangerous technologies: acknowledgment, then voluntary commitment, then — if the process succeeds — binding rules. The corporate pledges and the institute network are the second stage. The unfinished work is the third: converting self-defined, unverified, unenforceable commitments into common standards, independent verification, and legal obligation. The safety institutes are the most promising bridge, because the capacity they build is exactly what a binding regime would require. The risk is that the summit process stalls at the voluntary stage — that the world settles for pledges because pledges are easier than treaties, while the capabilities keep advancing. Seoul made the gap between commitment and governance visible. Closing it is the task that remains.
Common questions.
Voluntary pledges signed by sixteen leading AI companies at the May 2024 Seoul AI Safety Summit. Companies committed to publish safety frameworks, define thresholds of risk they consider intolerable, and not develop or deploy models that cross those thresholds without adequate mitigation — including pulling back where mitigations fail. It was the first time major developers publicly committed to the principle that some capabilities are too dangerous to deploy.
No. They are voluntary corporate pledges. Each company sets its own risk thresholds and measurement methods, there is no independent verification that frameworks are followed, and there is no legal enforcement — the only consequence for breaking a commitment is reputational. They can also be revised or abandoned under competitive pressure, which is the central weakness of self-regulation.
A commitment by governments at Seoul to build an international network of national AI safety institutes — public bodies, starting with the UK's and US's, that develop the science of evaluating frontier models. They cooperate on testing methods and technical capacity. This matters because independent evaluation expertise is exactly what any future verification regime or binding treaty would require.
No. It produced voluntary corporate commitments and a non-binding governmental declaration, plus progress on the safety institute network. It advanced the process from acknowledgment (Bletchley) toward commitment, but stopped short of common standards, independent verification, and legal obligation. The open question is whether the summit series converts these voluntary steps into binding governance before frontier capabilities outrun it.