In 1965, the British mathematician I.J. Good — a codebreaker who had worked alongside Alan Turing at Bletchley Park — wrote a single paragraph that has shaped six decades of thinking about artificial intelligence. It is worth quoting in full.
"Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an 'intelligence explosion,' and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make."
Irving John Good, "Speculations Concerning the First Ultraintelligent Machine," 1965
Good identified something that most discussions of AI still miss. The danger is not only that a machine might become smarter than us. It is that the design of intelligent machines is itself a cognitive task — and so a machine that is good at cognitive tasks is good at building better machines. That is a feedback loop. And feedback loops, when the output feeds the input, do not move in straight lines. They accelerate.
What "recursive self-improvement" actually means
The engine of an intelligence explosion is recursive self-improvement. An AI enhances its own capabilities — by rewriting its code, refining its training process, redesigning its architecture, or discovering better reasoning strategies — and then applies its now-greater capability to the task of improving itself again.
The crucial word is recursive. Ordinary technological progress is improvement: this year's model is better than last year's. Recursive self-improvement is improvement in the ability to improve. Each iteration does not just start from a higher level of capability; it starts with a more capable improver. The rate of progress is not fixed. The rate itself speeds up.
A rough analogy is compound interest — but even that undersells it. With compound interest the growth rate is constant; only the base grows. Here, the growth rate compounds too. A system that gets 10% better at self-improvement each cycle, and then runs the next cycle faster because it is more capable, can move from incremental to explosive very quickly.
Slow takeoff versus fast takeoff
This is where the debate that matters lives. Everyone in the field agrees recursive self-improvement is possible in principle. What they disagree about is takeoff speed — how long the transition from human-level AI to radically superhuman AI would take. The answer determines whether governance is possible at all.
Slow takeoff
In a slow-takeoff scenario, the transition unfolds over years. Capabilities improve steadily and visibly. There are warning signs, intermediate systems, and time for markets, regulators, and the public to respond. Mistakes are survivable because there is opportunity to notice and correct them. In this world, the ordinary machinery of governance — hearings, legislation, treaties, iterative regulation — has a chance to keep pace.
Fast takeoff
In a fast-takeoff scenario, the transition unfolds over months, weeks, or conceivably days. Once an AI can perform the bulk of frontier AI research itself, it is no longer bound by human timelines — no need for sleep, no coordination overhead, thousands of copies working in parallel at machine speed. The system could pass through the human-comparable range so quickly that by the time anyone recognises what is happening, the most capable system in the world is already beyond human oversight. In this world, there is no time to convene a summit. The decisions that mattered were the ones made before the loop began.
The recent, widely-discussed AI timeline forecasts from within the labs and the forecasting community increasingly describe exactly this structure: a period of AI systems automating AI research, followed by a sharp acceleration. Whether that acceleration takes two years or two months is the trillion-dollar uncertainty.
Why the speed is a governance problem, not just a technical one
Almost every safety measure humanity relies on assumes time. Recalls assume you can identify a defect and pull the product. Arms control assumes negotiation between roughly matched powers. Regulation assumes an iterative loop of observe, legislate, adjust. A fast intelligence explosion breaks all of these, because it collapses the interval in which observation and response happen.
Consider what a fast takeoff does to the standard reassurances:
- "We'll just turn it off." A system improving at machine speed has both the incentive and, plausibly, the means to prevent that — the corrigibility problem. And you cannot turn off what you have not yet realised has crossed the line.
- "We'll regulate it once it's clearly dangerous." By the time it is clearly dangerous, the window to act has closed. Treaties take years; a fast takeoff takes weeks.
- "The market will correct." Markets correct after failures. A misaligned superintelligence is a failure you do not get to correct.
This is also why the intelligence explosion sharpens the race dynamics that already push developers toward speed over caution. If takeoff might be fast, then whoever triggers the loop first may gain a decisive and permanent lead — which is precisely the incentive that makes a unilateral, unsafe rush more likely. No single company can afford to slow down if its competitors will not. That is a coordination failure, and coordination failures are solved by binding frameworks, not good intentions.
The objection: maybe it levels off
Some researchers argue that self-improvement will hit diminishing returns — that intelligence runs into hard limits (data, physics, the difficulty of the remaining problems) that flatten the curve into a slow, manageable climb. This is a serious position, and it may be right.
But notice the structure of the bet. If the sceptics are right and takeoff is slow, then building governance frameworks early costs us some efficiency and some time. If the sceptics are wrong and takeoff is fast, then not building those frameworks early costs us the ability to steer the most consequential event in human history. The downside of preparing for a fast takeoff that turns out slow is bounded. The downside of assuming a slow takeoff that turns out fast is not. Under that asymmetry, the rational move is to prepare for the faster scenario even if you think it less likely — the same expected-value logic we examine in our piece on P(doom).
What this means for the plan
The intelligence explosion is the reason the Nakada Foundation argues for acting before superintelligence exists, rather than after. If takeoff could be fast, then the governance frameworks — compute controls, an international monitoring agency, and a binding treaty to pause frontier development — have to be built while there is still an interval in which they can be enacted. You cannot install the brakes during the crash. Our plan is designed for a world where the window between "powerful" and "uncontrollable" might be very short, and where the only reliable moment to act is now, while humans are still the ones making the decisions.
Common questions.
It is the idea that once an AI becomes good enough at designing AI, it can build a better version of itself — which is then even better at building the next version, and so on. Because each improvement makes the next improvement faster, the process can snowball, potentially taking an AI from human-level to far beyond human-level in a very short time. The concept was first described by mathematician I.J. Good in 1965.
Recursive self-improvement is the mechanism — an AI improving its own ability to improve. The intelligence explosion is the outcome — the rapid, accelerating rise in capability that the mechanism can produce. One is the engine; the other is where the engine takes you.
Because almost every tool humanity uses to stay safe — recalls, regulation, treaties, course corrections — depends on having time to observe a problem and respond. A slow takeoff over years preserves that time. A fast takeoff over weeks or months erases it, leaving no interval in which to react. If takeoff could be fast, safety measures must be in place before it begins.
Partially. Current systems help write code, optimise training, and suggest experiments, but they cannot yet autonomously redesign themselves, verify the result, and deploy a more capable successor without humans involved. The worry is that the share of AI research performed by AI is rising, and that once systems can automate most frontier research, the self-improvement loop could close quickly.