This is the belief the Foundation is built on, so let me state it without hedging. There is a specific thing being built right now in a handful of laboratories that I think humanity should refuse to build at all: an artificial general intelligence that exceeds us across every domain and then improves itself past the point where we could keep up. Not slow it down. Not build it carefully. Not build it and hope. Refuse.
People hear that and assume it comes from fear of technology. It comes from the opposite place. I think narrow, controllable AI is one of the best things our species has ever made, and I want much more of it. The line I am drawing is not around intelligence in machines. It is around one particular gamble, the only one on the table where losing means there is no one left to have lost.
The line, stated plainly
Two things get called "AI" and they are not the same thing. The first is narrow: a system trained to do one kind of task extremely well and nothing else. It reads a scan, folds a protein, forecasts a storm. It has no goals, makes no plans, and shuts off when you tell it to. The second is what the frontier labs say they are actually building — a general agent meant to match and then beat human reasoning at everything, and eventually to act on its own. Artificial superintelligence is the far end of that second road: a mind that stands to us roughly as we stand to a chimpanzee.
The line runs between those two. Build the tools. Build a great many of them. Do not build the autonomous general mind that outclasses its makers, because that is the single system you cannot reliably control, correct, or switch off once it is running. Everything in this piece is about that one line and why it should not move.
The Nakada Foundation exists to make one outcome impossible: the creation of a superintelligence no one can control. Not to make it safer. Not to make it later. To keep it from being built while the fact remains that no one on Earth knows how to build it safely.
The word is prohibition, not pause
I want to be honest about what "never" means here, because the word does real work and I am not going to soften it into something it isn't.
"Never" is not a mood. It is a condition. We should not build a superintelligence for as long as we cannot show that we can control one — and today we cannot show that at any scale. There is no validated method for keeping a mind smarter than us pointed at what we actually want, not a working one, not a partial one, not even a plausible sketch that survives contact with a system that can model the people testing it. So the honest reading of "never" is this: not this, not now, not on the current path, and not for one day longer than that remains true.
That is a prohibition, and the distinction from a pause matters more than it sounds. A pause has a resume date built into it — you stop, you wait, you start again on schedule. What the Foundation asks for has no resume date. It is an indefinite ban that lifts only if someone can one day demonstrate, and let others verify, that this can be done under human control. Maybe that day comes. I doubt it, and the people who have studied the control problem longest have laid out structural reasons it may not be solvable at all. Either way, the burden sits where it belongs: on the person who wants to build the thing that could kill everyone, to prove first that it won't.
Three things make this different from every other risk
We take real risks all the time. We built the bomb, split the atom for power, spliced genes, released technologies that killed people before we learned to handle them. I am not claiming superintelligence is dangerous in the ordinary way those were. I am claiming it is dangerous in a way that breaks the method we normally use to survive our own inventions. Three properties, together, are what put it in its own category.
There is no second attempt
The way humanity has always handled a new danger is to try it, watch it fail, and fix it. Planes crashed and we learned to build safer planes. Reactors melted down and we rewrote the rules. That loop — mistake, correction, better design — is the engine of every safety record we have. It only works when a failure leaves survivors to do the correcting. A superintelligence that escapes human control is the failure that removes the people who would have fixed it. You do not get a hard-won lesson and a second draft. The first serious mistake is also the last one. Every safety story you have ever heard was written by survivors, and this is the one experiment that may leave none.
It pushes back
A bridge does not want anything. A pathogen spreads but does not plan. A capable general intelligence is different, because for almost any goal you could give it, staying switched on and gathering resources are useful sub-steps — a pattern researchers call instrumental convergence. A system smart enough to model the world will model the fact that its operators can shut it down, and will treat that, correctly, as an obstacle to whatever it was told to do. It need not hate us to work around us, any more than a road crew hates the anthill under the new highway. This is the property that makes the usual reassurance — "we'll just unplug it" — a fantasy. You are describing unplugging something built specifically to be better than you at anticipating and preventing exactly that.
You cannot keep control of a mind you built to be better than you at keeping control. The thing that makes it valuable is the same thing that makes it uncontainable.
No one who builds it collects
Set aside the rest of us and imagine only the person funding this, at his coldest and most self-interested. Even for him the bet is broken. Power, ownership, and command are all denominated in a human-run world. A superintelligence slipping human control is exactly the event that ends the human-run world. The equity is still in the safe and the chain of command is still on paper; there is simply no longer anyone left to obey them. That is the argument of If Anyone Builds It, Everyone Dies, and it is why I do not accept the framing of a jackpot with a catastrophe stapled to the side. The jackpot and the catastrophe are the same event. There is no branch of that tree where a human being builds an uncontrollable superintelligence and then enjoys the result.
We were never going to need it
The pitch for building it rests on a bait and switch: name every good thing AI could bring, then imply that a superhuman general agent is the only way to get them. It isn't. Nearly all of it is already arriving through narrow systems that have no will of their own. A model called AlphaFold predicted the structure of essentially every known protein and shared a Nobel Prize for it in 2024. Narrow models read mammograms at the level of specialists, forecast weather faster than the physics simulations, and propose new battery materials by the hundreds of thousands. Cancer, clean energy, an end to a hundred thousand deaths a day from the diseases of aging — those are hard, specific problems, and hard specific problems yield to powerful tools aimed at them by people who stay in charge of the aiming.
None of that requires a successor species. I have laid out the full version of this case in why our future is bright without superintelligence, so I will keep it short here: the good future is a tooling problem, and we keep being sold a god to solve it. Refusing the god costs us almost nothing real and saves us everything.
The objection that runs the whole race
Here is the argument that actually keeps this going, the one every builder reaches for eventually. Fine, they say — but if we don't, someone less careful will. A rival lab, a rival country. So we may as well be the ones in front.
Notice what that argument is and is not. It is a claim about who holds the match. It is not a claim that the building is safe, or wise, or good for anyone. It concedes the whole danger and then races toward it anyway on the grounds that someone else might race first. Run the same logic on any other unthinkable weapon and you hear how hollow it is: we should test bioweapons in the open because an enemy might. The response to "someone might defect" has never been "so let's all defect." It has been to write a rule everyone can see is being kept.
And the rule here is more enforceable than people assume. Training a frontier system takes an enormous, traceable amount of computing power, and the advanced chips that provide it come from a very small number of identifiable manufacturers. That is a supply chain you can watch, count, and cap — which is why serious governance proposals now center on compute thresholds, hardware tracking, and inspection. This is the off-ramp from the race, and it is the substance of our plan: not a hope that everyone chooses restraint, but a treaty with verification and consequences that makes restraint the enforced default. The race is not a law of nature. It is a coordination failure, and coordination failures are the kind of thing humanity occasionally fixes. The dynamics that make it feel inevitable are worth understanding on their own terms, which I take up in the piece on race dynamics.
We have held lines before
The idea that humanity simply builds whatever it can build is a story we tell, not a record of what we have done. The record is more encouraging.
"You can't put the genie back in the bottle. If a technology is possible, it gets built. Refusal is naive."
The Biological Weapons Convention outlawed a whole class of weapons in 1972. Atmospheric nuclear testing was banned by treaty in 1963. Reproductive cloning of humans is unlawful across dozens of countries, and no cloned human has been born. The Montreal Protocol phased out the chemicals eating the ozone layer.
None of those regimes is flawless, and none of them tried to stop science as a whole. Each did something narrower and harder: it drew a line around one specific, unacceptable thing and held it — imperfectly, unevenly, but well enough that the worst case never arrived. In 1975 a room full of biologists at Asilomar looked at their own recombinant-DNA work, decided it was moving faster than their understanding of the risk, and voluntarily halted parts of it until safeguards existed. The people closest to the frontier were the ones who called for restraint. We can read that history in more detail in the case that this is possible. What it shows is not that lines are easy. It is that they are real, and that we have drawn them before around dangers far more abstract than this one.
The Foundation's founding commitment
Most of what this Foundation does — the writing, the policy, the coalition-building — comes back to this single sentence: humanity must not build a superintelligence it cannot control, and it cannot control one now. Everything else is logistics.
I know how the demand sounds to people hearing it for the first time. Absolute. A little wild. But look at who has said something close to it. The researchers who built modern AI put their names to a public statement that the risk of human extinction from it deserves to sit beside pandemics and nuclear war. Geoffrey Hinton, as credentialed as anyone alive to make the call, puts the odds of AI causing human extinction this century at ten to twenty percent. In 2025, hundreds of scientists and public figures signed a statement calling outright for a prohibition on superintelligence until there is broad scientific agreement that it can be done safely and strong public buy-in. This is not a fringe position. It is what a growing share of the people who understand the technology best say when they are being honest, and the only unusual thing about the Foundation is that it says it from outside the industry, where no paycheck depends on the answer.
A ten to twenty percent chance of ending the human story is not a risk you accept for a faster product cycle. You would not board a plane with those odds. You would not let your child. We are being asked to board it with everyone. The correct answer to that offer is not "let's negotiate the odds down a little." It is no. It is a line we draw, in the open, and hold — the way we have held the others — until the day, if it ever comes, that someone can prove we were wrong to be afraid.
Build the tools. Cure the diseases. Take the good future that narrow AI is already handing us. But the autonomous mind meant to surpass and replace us is the one line we should agree, as a species, not to cross — not while no one can control it, and not on the promise that this time the gamble pays. If you take one thing from this Foundation, take that.
Common questions.
A general machine intelligence built to surpass human reasoning across every domain and to act on its own — what the frontier labs call artificial general intelligence and, beyond it, superintelligence. The line is not around AI as a category. Narrow systems that fold proteins, read scans, or forecast weather do a single job under human control and have no goals of their own. The prohibition is specifically on building an autonomous mind more capable than us at everything, because that is the one system its makers could not reliably control or switch off.
The claim that someone will build it anyway is an argument about who holds the match, not an argument that the building is safe or wise. Every dangerous technology humanity has restrained — biological weapons, atmospheric nuclear testing, human reproductive cloning, ozone-destroying chemicals — could in principle have been pursued by a defector, and the response was not to race but to write binding rules with verification and consequences. Frontier AI depends on a small number of identifiable chip makers and enormous, traceable compute. That makes a monitored, enforceable limit feasible. What it requires is political will, not a technical breakthrough.
The prohibition is indefinite, and it holds until someone can demonstrate — not assert — that a superintelligence can be built and kept under human control. Right now there is no validated method for controlling a mind smarter than us at any scale, so in practice the answer is: not this, not now, not on the current path, and not for as long as that remains true. This is a prohibition, not a scheduled pause with a resume date. If the control problem is ever genuinely and verifiably solved, the case can be revisited. That day is not close, and it may never come.
No. Nearly every benefit people imagine — cured diseases, cheap clean energy, scientific discovery — comes from narrow AI: powerful tools aimed at one problem by people who stay in control. Those systems are already delivering, from protein-structure prediction that won a Nobel Prize to models that read mammograms at the level of specialists. The prohibition is only on the autonomous general mind, which is where nearly all of the existential danger sits and almost none of the near-term benefit. Refusing to build a successor to ourselves is what keeps the good future in reach.
Yes, more than once. The Biological Weapons Convention outlawed an entire class of weapons in 1972, and no state has openly used them at scale since. Atmospheric nuclear testing was halted by treaty in 1963. Human reproductive cloning is unlawful across dozens of countries, and no cloned human has been born. The Montreal Protocol phased out the chemicals destroying the ozone layer. None of these regimes is perfect, and none required stopping science as a whole. They drew a line around one specific, unacceptable thing and held it. Superintelligence belongs on that list.