Part 2 of 2 · The Money in AI Safety

This is the sequel to Part 1 — Why Is There So Little Money in AI Safety?, which diagnoses why the field runs on so little. Here the question is how it gets to a billion dollars a year.

Start with where things actually stand. Part 1 added up every philanthropic grant, every government safety program, and every dollar the frontier labs spend on their own safety teams, and arrived at a total of roughly $190 million a year. Set against that, the world puts more than $250 billion a year into making AI more capable, and the figure keeps climbing. For every thousand dollars spent making these systems more powerful, under a dollar is spent making them safe.

A billion dollars a year would begin to change that arithmetic, and it is a smaller sum than it sounds. It is less than two days of what the industry already spends on capabilities. It is about one-tenth of one percent of what the United States spends on defense. It is a tenth of what climate philanthropy moves every year, and climate philanthropy is not thought of as lavishly funded. Whatever a billion dollars is, it is not a large amount of money for a problem the people building the technology have compared, in writing and under their own names, to pandemics and nuclear war.

So the reason the field is starved is not that a billion dollars is hard to find. The world is already spending many times that a few feet away, on the other side of the same office buildings. The money is there. What is missing is the decision to move some of it.

The money already exists. The world spends this much making AI more powerful about every day and a half. What is missing is the will to move a fraction of it the other way.

A billion is a floor, not a ceiling

Why a billion, and not three hundred million? Because below roughly that line, the field can only afford to do one thing at a time. At $190 million, the numbers force brutal triage. Technical research and model evaluations absorb about two-thirds of everything. Advocacy — the work of getting governments and companies to act on what the research finds — gets around five percent, on the order of $10 million a year, less than a single large technology company spends lobbying one national capital. A billion dollars would end that triage. It would let safety pay competitively for the talent the labs currently absorb, diversify a field that leans on a single dominant foundation, and fund the political work that changes outcomes rather than merely documenting them.

And a billion a year is modest by every historical comparison that fits. The Human Genome Project cost around $3 billion and is remembered as one of the best public investments the United States ever made. Measured against a risk its own creators rank beside nuclear war, a billion dollars a year to reduce it is not a generous figure. That we are debating whether it is affordable is the strange part.

The precedent for cheap

Eradicating smallpox cost about $300 million across the whole thirteen-year campaign — less than a third of a single year of the target discussed here. Since 1980 that one-time expenditure has saved the world well over a billion dollars every year, and the United States alone recoups its share of the original cost roughly every twenty-six days.

Reducing a civilization-scale risk has almost always been one of the cheapest things we have ever done, measured against what it returns.

Where the billion actually comes from

None of this requires a new invention or an act of extraordinary generosity. It requires four ordinary sources to be switched on at once. Any two of them, done seriously, clear the target.

SourcePlausible per year
Diversified philanthropyToday one foundation supplies over half of all philanthropic safety money. Climate philanthropy, by contrast, moves more than $10 billion a year across many donors. A single $20 billion endowment throws off a billion a year on its interest alone; reaching a tenth of climate's total needs perhaps five to ten serious funders.$300M
Government safety budgetsNational safety institutes already exist in the UK, the US, and elsewhere. The US spends about $47 billion a year on the National Institutes of Health; a dedicated AI-safety effort at one percent of that is nearly half a billion on its own.$300M
An industry levy on computePharmaceutical firms fund roughly three-quarters of the FDA's drug review through user fees; the nuclear industry funds nearly all of its regulator. Four-tenths of one percent on the compute and infrastructure frontier AI already buys — past $250 billion a year — raises the whole billion by itself.$300M+
Redirected charitable capitalDonor-advised funds hold well over $200 billion already committed to charity but not yet disbursed. University endowments and the growing effective-giving infrastructure add more. The ask here is reallocation, not new wealth.$100M
Total~$1B+

The third row is the one that matters most, because it is the only source that does not rest on anyone's goodwill. Philanthropy can evaporate when a donor changes their mind, as the field learned when its dominant funder cut technical safety grants by more than half in two years. Government budgets turn over with elections. A statutory levy on the industry that profits from building the risk is durable, proportionate, and entirely precedented — it is already how we pay for the watchdogs that oversee medicine, aviation, and nuclear power. Tying a sliver of safety funding to the compute that creates the hazard is a natural extension of the case for compute governance.

The money moves fast once the risk is legible

History is unambiguous on this. When a danger becomes legible to the public and their governments, the money appears at a speed that exposes the present trickle for what it is — a choice, not a constraint. The chemicals eating the ozone layer were phased out under the Montreal Protocol, whose fund has since moved billions of dollars to make the switch possible in poorer countries. When COVID arrived, the world stood up vaccine development measured in tens of billions of dollars within months. Smallpox, ozone, polio: in each case the constraint was never the money. It was the recognition.

AI safety has not had its moment of recognition yet, and the reason is structural rather than financial. The harm it warns about has not happened. There are no patients, no flooded towns, no grieving families to put a face on it — and if the work succeeds, there never will be. A risk with no constituency has no one leaning on the people who write the checks. That is the real bottleneck, and it is not solved with a better spreadsheet.

This is a mobilization problem

Which means the path to a billion dollars is not, at its heart, a fundraising plan. It is a plan to make the risk legible enough, and taking it seriously prestigious enough, that the money follows the way it always has. That work runs along three lines.

Make it visible. A threat people can picture is a threat people will fund. That is the case for journalism, film, data, and public argument that turn an abstract probability into something a legislator or a donor can feel — the same cultural groundwork that came before serious money for the climate and for nuclear arms control.

Make it prestigious. When Nobel laureates, former heads of state, and trusted public figures treat a cause as serious, both norms and money move behind them. The scientific consensus on AI risk already exists. What has lagged is the social permission for people of means and standing to act on it out loud.

Make it mandatory. Voluntary giving, pushed hard, might carry the field to three or four hundred million a year. The stretch to a full billion — and, more to the point, its survival past any single donor's change of heart — comes from law: a levy, a line item, a treaty obligation. This is why the Foundation aims its work at binding rules and not only at persuasion. Generosity is welcome, but it is not something you can build on.

So set aside the idea that a billion dollars a year is the ambitious version of taking this seriously. It is the minimum version. It is less than two days of what we already spend making the problem harder, and a small fraction of what it cost, one time, to rid the world of a disease that had killed for three thousand years. The sum is not daunting. The thing in genuinely short supply is the decision to treat a risk this size as though we believe our own warnings.

That decision does not wait on a billionaire. It is made in public pressure, in what governments hear from the people they answer to, and in whether enough of us treat this as the priority the science says it is. The case for the rules that would make the money durable is set out in our plan, and the fastest way to add your weight to it takes five minutes on the take action page.

Common questions.

How much would it take to get AI safety to $1 billion a year?

About five times the current total. The field runs on roughly $190 million a year, so reaching a billion means adding around $810 million. That sounds steep only in isolation. A billion dollars is less than two days of global spending on AI capabilities, and about one-tenth of one percent of the US defense budget.

Is a billion dollars a year realistic when the field runs on $190 million?

Yes, because it does not depend on any single source. Four ordinary channels — diversified philanthropy, national government safety budgets, a small industry levy on frontier compute, and redirected charitable capital — can each supply hundreds of millions a year. Any two of them, pursued seriously, reach the target.

Why should the AI industry pay for its own safety oversight?

Because that is the standard arrangement for industries that carry systemic risk. Pharmaceutical companies fund roughly three-quarters of the FDA's drug-review program through user fees, and the nuclear industry funds close to the entire budget of its federal regulator. A levy of well under one percent on the compute frontier AI already buys — more than $250 billion a year — would raise a billion dollars on its own, without a single philanthropic check.

Isn't philanthropy enough to fund AI safety?

Not on its own. Philanthropy is generous but fragile: the field recently watched its dominant funder cut technical safety grants by more than half in two years. Voluntary giving might carry the field to three or four hundred million a year. The rest of the way to a billion, and its survival past any one donor's change of heart, comes from law — a government line item or a statutory levy on the industry that profits from building the risk.