Skip to content
Adventures and Amusings of a Mathematician
Go back

What If We Succeed? The Optimistic Case for Aligned ASI

Nearly every serious essay about Artificial Super Intelligence opens with a warning. The existential risk is real, the alignment problem is hard, and the default outcome---so the argument goes---is catastrophic. I take these concerns seriously. But I have noticed a striking asymmetry: for every hundred essays about how ASI might end civilization, there is maybe one about what happens if we actually solve alignment and deploy it well.

This is that one.

I want to run a thought experiment. Assume that sometime in the next decade, we build a system that is genuinely superintelligent---not a better chatbot, but a system that can do novel scientific research, engineering design, and strategic reasoning at a level that exceeds the best humans in every domain. And assume we solve the alignment problem: this system reliably pursues goals that humanity endorses, with robust oversight mechanisms in place.

What happens next?


The Pessimism Asymmetry

Before diving in, it is worth asking: why is optimism about ASI so rare in serious writing?

Part of the answer is selection bias. People who think deeply about ASI tend to be drawn to the problem because of the risks. Eliezer Yudkowsky, Stuart Russell, and the broader alignment research community are motivated by preventing catastrophe, not by daydreaming about utopia. This is rational and important. But it creates a discourse where the only detailed, concrete, quantified scenarios are the negative ones.

Nick Bostrom, who wrote the book on AI existential risk, also wrote a short piece called Letter from Utopia in 2008---a fictional letter from a posthuman future describing what life could be like. It is beautiful and largely forgotten. His risk analysis, Superintelligence, sold millions.

There is also a deeper psychological pattern. Daniel Kahneman documented that humans weight losses roughly twice as heavily as equivalent gains. Dystopia feels serious. Utopia feels naive. An essay arguing that ASI might cure cancer reads like a press release; an essay arguing it might cause human extinction reads like a warning from a prophet.

But intellectual seriousness requires examining both tails of the distribution. If we are going to spend billions on alignment research---and we should---we need to understand what we are aligning toward, not just what we are aligning away from.


The Compressed 21st Century

Dario Amodei, CEO of Anthropic, offered the most useful framework for thinking about this in his essay Machines of Loving Grace. He calls it the Compressed 21st Century: all the scientific and technological progress that humanity would have made over the next 75—100 years, compressed into 5—10 years.

This is not magic. It is the straightforward consequence of what happens when you have millions of copies of Nobel-caliber researchers working 10—100x human speed, 24 hours a day, 365 days a year, across every scientific discipline simultaneously. Each copy can read every paper ever published, hold the entirety of a field’s knowledge in working memory, and design experiments with perfect recall of every prior result.

The key insight---and it is a debatable one---is that intelligence may be the binding constraint on much of scientific progress. Not funding, not motivation, not data, but the sheer scarcity of minds capable of making fundamental breakthroughs in any given field. If that is true, or even partially true, ASI loosens the bottleneck considerably.

But Amodei also introduces an important corrective: the concept of marginal returns to intelligence. In some domains, progress is bottlenecked not by intelligence but by irreducible physical constraints. Clinical trials take years because biology runs on biological time. Particle physics experiments require building new accelerators. Sociology requires observing human behavior across decades.

So the right question is not “what can superintelligence solve?” but rather: “In which domains does intelligence, not time or data, constitute the binding constraint?”

The answer, if this framework holds, may be: most of the ones that matter.


Domain 1: The End of Involuntary Death

Start with the most profound: aging and disease.

Human lifespan doubled in the 20th century, from roughly 40 to 75 years. That was achieved with crude tools---antibiotics, sanitation, vaccines, basic surgery. We did not understand the molecular mechanisms of aging. We barely understood the genome.

Source: Our World in Data, drawing on UN World Population Prospects and the Human Mortality Database.

Now consider what becomes possible when an intelligence that can model protein folding, gene regulatory networks, and cellular signaling pathways at arbitrary depth turns its attention to biology:

ProblemCurrent statusPost-ASI projection
Cancer~2% annual mortality decline95%+ reduction in incidence and mortality
Alzheimer’sNo disease-modifying treatmentMolecular mechanism fully mapped, targeted interventions
Genetic diseases~6,000 known, most untreatableBroad gene therapy across all monogenic disorders
Infectious diseaseMalaria kills 600K+ per yearNear-complete eradication via designed vaccines/antivirals
Aging itselfNo approved interventionsLongevity escape velocity---gaining more than one year of life expectancy per year lived

This is not speculation from science fiction. The trajectory is already visible:

AlphaFold predicted the 3D structure of over 200 million proteins---essentially the entire known protein universe. More than 3 million researchers in 190 countries use it. It doubled the ratio of druggable protein binding sites from 19.8% to 41.8%. And AlphaFold is not superintelligent. It is a narrow tool that does one thing well.

AI-designed drug candidates in clinical trials grew from 3 in 2016 to over 173 programs in 2026. Early-stage AI-designed molecules show 80—90% Phase I success rates versus the historical average of 52%. Insilico Medicine developed a preclinical candidate in under 18 months---a process that traditionally takes 3—6 years.

The cost of sequencing a full human genome tells a similar story of exponential improvement---one that has outpaced even Moore’s Law:

Source: Our World in Data, drawing on NHGRI data.

Scale this up by many orders of magnitude. An aligned ASI could design, simulate, and iterate on therapeutic molecules faster than any pharmaceutical company can schedule a meeting. It could model the interactions between thousands of drugs and biological pathways simultaneously. It could design clinical trials that extract maximum information from minimum patients, or even develop validated in silico models of human biology that reduce the need for physical trials altogether.

Ray Kurzweil predicts we will reach longevity escape velocity by approximately 2029. That is the point at which medical progress adds more than one year of life expectancy per calendar year. Whether you believe his timeline or not, the concept is sound: once intelligence is no longer the bottleneck, the rate of medical progress becomes limited primarily by the speed of biological experiments, not by the speed of human thought.

Amodei’s specific estimate: human lifespan could double to approximately 150 years within the first decade of powerful AI.

Read that again. One hundred and fifty years.

And that may be conservative. Once you understand aging at the molecular level, “150 years” is an arbitrary number constrained only by our current inability to imagine what it means to not die of old age.


Domain 2: Unlimited Clean Energy

Energy is the master resource. Nearly every constraint on human civilization---food production, water desalination, transportation, manufacturing, computation itself---reduces to an energy constraint. Solve energy and you solve most of scarcity.

The cost trajectory of solar photovoltaics is among the most remarkable in the history of technology:

Source: Our World in Data, drawing on IRENA and historical data.

The price of a solar module has fallen from roughly $106 per watt in 1976 to under $0.30 today---a decline of more than 99.7%. The International Renewable Energy Agency (IRENA) reports that utility-scale solar LCOE fell from $0.381 per kWh in 2010 to under $0.049 by 2023. According to the IEA’s World Energy Outlook 2024, solar PV is on course to become the world’s largest source of electricity by the early 2030s.

Yet even these gains were achieved without superintelligent optimization of the underlying materials science, grid architecture, or storage chemistry.

Nuclear fusion has been “30 years away” for 60 years. The reason is not that fusion is physically impossible---the sun does it continuously. The reason is that controlling plasma at 150 million degrees involves a staggering number of interacting variables: turbulence, instabilities, magnetic field geometry, materials degradation. The parameter space appears to be too large for unaided human intuition and too complex for brute-force simulation with current hardware.

This is the kind of problem where ASI could plausibly excel.

AI is already making inroads. Princeton’s STELLAR-AI program is using machine learning to accelerate fusion plasma simulations. MIT researchers used AI to predict and prevent plasma disruptions in real time. DeepMind’s techniques for controlling plasma shape in tokamaks demonstrated that machine learning can handle the multi-dimensional control problem that has stymied physicists for decades.

An ASI could go further: design entirely new reactor geometries, discover novel plasma confinement strategies, solve the materials science problems (what do you build a wall out of when the thing touching it is hotter than the sun?), and optimize the engineering for mass production.

The plausible end state is energy that approaches being too cheap to meter. Not free---infrastructure costs exist---but potentially cheap enough that the cost of energy ceases to be a meaningful constraint on what humanity can do.


Domain 3: Solving the Deepest Problems in Physics and Mathematics

This is the domain I find most intellectually exciting, and the one where existing optimistic essays are weakest.

Fundamental physics has been stuck for roughly 50 years. The Standard Model was completed in the 1970s. String theory has produced no testable predictions in four decades. We have no quantum theory of gravity. We do not understand dark matter or dark energy, which together constitute 95% of the universe’s mass-energy content. The last major theoretical breakthrough---the Higgs mechanism---was proposed in 1964 and confirmed experimentally in 2012.

Why the stagnation? Several reasons: insufficient experimental data at the relevant energy scales, the difficulty of building new accelerators, and---perhaps---that the mathematics required to make further progress has grown extraordinarily complex. The equations governing quantum gravity involve mathematical structures---higher-dimensional topology, nonperturbative effects in quantum field theory, the landscape of string vacua---that are difficult for any individual human to hold in working memory simultaneously. Whether this reflects a hard limit on human cognition or merely a tooling problem is an open question.

If it is even partly a tooling problem, ASI changes the calculus significantly.

A superintelligent system could:

  1. Unify general relativity and quantum mechanics. This is arguably the deepest open problem in physics. It may require new mathematics that does not yet exist. ASI could invent that mathematics
  2. Determine the nature of dark matter and dark energy. Sift through all existing cosmological data, propose new theoretical frameworks, and design experiments to test them
  3. Solve the black hole information paradox. Resolve whether information is truly lost when matter falls into a black hole, with implications for the fundamental nature of spacetime
  4. Design experiments we cannot currently conceive. The history of physics is the history of new instruments revealing new phenomena. ASI could design particle detectors, gravitational wave observatories, and cosmological surveys of unprecedented sensitivity

In mathematics, the trajectory is already clear:

The Millennium Prize Problems---the Riemann Hypothesis, P vs NP, the Navier-Stokes existence and smoothness problem, and others---represent the outer frontier of human mathematical capability. A system operating well beyond current human mathematical reasoning might resolve some or all of them. And beyond the Millennium Problems may lie questions we have not yet had the tools to formulate.

David Deutsch captures this in his Principle of Optimism: “All evils are caused by insufficient knowledge.” If every problem that is interesting is also soluble given sufficient knowledge, then ASI---as a knowledge-generating system of potentially unprecedented power---would be the best tool we have ever had for tackling them.


Domain 4: Ending Poverty in a Decade

Global poverty is often framed as a resource problem or a political problem. It is both. But it is also, in part, a coordination and optimization problem operating at a scale that strains human institutional capacity.

The share of the world’s population living in extreme poverty has already fallen at a pace that would have astonished any observer two centuries ago:

Source: Our World in Data, drawing on the World Bank Poverty and Inequality Platform and historical estimates.

From roughly 75% in 1820 to under 10% today. This is the most underappreciated chart in the world.

But the remaining poverty is also the hardest to reach. Consider what it takes to lift a country out of poverty: simultaneously improve agriculture, healthcare, education, infrastructure, governance, trade, and financial systems---all of which interact with each other in complex, nonlinear ways. No human institution can optimize across all these dimensions at once. The World Bank, the IMF, and national governments make piecemeal interventions and hope the interactions are positive.

ASI could model entire economies at the level of individual agents, supply chains, and resource flows. It could design interventions that account for all the interactions. It could adapt in real time as conditions change.

Amodei’s specific and striking prediction: Sub-Saharan Africa could reach China’s current per-capita GDP (~$12,000—$14,000) within 5—10 years of powerful AI, up from approximately $2,000 today (World Bank data).

This implies ~20% annual GDP growth---roughly 10% from AI-optimized decision-making across agriculture, healthcare, and infrastructure, and 10% from the diffusion of ASI-developed technologies that bypass decades of incremental development.

The underlying economics support the direction, if not the exact magnitude. Goldman Sachs Research estimates that generative AI alone could raise global GDP by 7%, or nearly $7 trillion, and lift productivity growth by 1.5 percentage points over a ten-year period. McKinsey Global Institute projects that generative AI could add $2.6 trillion to $4.4 trillion annually---roughly the GDP of the United Kingdom. And these estimates concern current, narrow AI. Superintelligence is a different category entirely.

The Federal Reserve Bank of St. Louis’s FRED database tracks American nonfarm business labor productivity (output per hour). The data show clear surges around previous technological waves---electrification in the 1920s, computing in the 1990s. Each general-purpose technology produced a step-change in output per unit of human effort. ASI would arguably represent the most powerful general-purpose technology ever developed.

The long arc of GDP per capita tells the story most starkly:

Source: Our World in Data, drawing on the Maddison Project Database (2020).

For millennia, the line is nearly flat. Then, around 1800, it begins to rise---first slowly, then exponentially. Each inflection point corresponds to a general-purpose technology: the steam engine, electrification, the computer. ASI would be the next, and by far the most consequential.

The result is not charity. It is the elimination of scarcity as the binding constraint on human flourishing.


Domain 5: Becoming a Multiplanetary Species

Every serious existential risk researcher agrees on one thing: as long as humanity exists on a single planet, we are one asteroid, one supervolcano, or one engineered pandemic away from extinction. Geographic diversification is not optional for a species that wants to survive long-term.

The barriers to Mars colonization and beyond are almost entirely engineering problems: radiation shielding, closed-loop life support, in-situ resource utilization, propulsion efficiency, and the sheer cost of launching mass out of Earth’s gravity well. On cost, the trajectory is already remarkable:

Launch vehicleCost per kg to LEO
Space Shuttle~$54,500
Atlas V~$13,200
Falcon 9~$2,720
Starship (target)<$100

Sources: NASA, CSIS Aerospace Security Project.

A three-order-of-magnitude cost reduction, achieved largely through reusability and iterative engineering.

ASI could:

Max Tegmark’s concept of cosmic endowment captures the stakes: if humanity survives and spreads, the amount of experience---of consciousness, joy, discovery, meaning---that could exist over billions of years across billions of star systems is incomprehensibly vast. The difference between “humanity goes extinct in the next century” and “humanity flourishes for a billion years across the galaxy” is not a quantitative difference. It is a difference in kind.


The Bottleneck That Remains: Governance

If I have painted an overly rosy picture, here is the corrective.

Intelligence is necessary but not sufficient. Even a perfectly aligned ASI cannot force humans to adopt its recommendations. It cannot override political systems, cultural resistance, or institutional inertia---not if it is truly aligned with human values and respects human autonomy.

The most likely failure mode for the optimistic scenario is not technical. It is distributional. ASI could cure cancer, but if the cure is proprietary and costs $2 million per treatment, most humans will not benefit. ASI could end poverty, but if its economic recommendations require political changes that incumbents resist, the status quo will persist.

The International Monetary Fund has warned that AI could widen the gap between rich and poor nations if the benefits concentrate in countries with existing AI infrastructure and talent. This is not a hypothetical---it is the default trajectory.

Solving this requires something ASI cannot provide: political will. It requires international coordination, regulatory frameworks, and a shared commitment to distributing the benefits of superintelligence broadly. The alignment problem has a political dimension that no amount of technical work can address.

Amodei’s framing is honest here: the role of AI is to “turbocharge” the efforts of people who want to help. The people and the institutions still have to want it.


What Are We Aligning Toward?

Let me return to the question I started with.

The alignment research community has spent two decades thinking about how to prevent ASI from destroying humanity. This work is essential and underfunded. But it has created a discourse where the positive vision---the thing we are trying to achieve, not just the thing we are trying to avoid---remains vague and underspecified.

“Human flourishing” is not a specification. “Aligned with human values” is not a blueprint. If we are going to build the most powerful technology in human history, we need a concrete, detailed, quantified vision of what success looks like.

Here is mine:

Within 20 years of aligned ASI:

  1. No human dies involuntarily of disease or aging. Death from accident or choice remains, but the biological lottery is over
  2. Energy is abundant and clean. Fusion or advanced solar provides effectively unlimited power at negligible marginal cost
  3. Poverty is eliminated. Every human has access to food, shelter, healthcare, education, and meaningful work or purpose
  4. We have a permanent presence beyond Earth. At minimum, self-sustaining settlements on the Moon and Mars
  5. We understand the fundamental laws of physics. A unified theory of quantum gravity, and the beginning of technologies that exploit it
  6. Every human has access to superintelligent assistance. Not as a luxury, but as a basic right---like literacy or clean water

This is not utopia. Humans will still argue, compete, create, suffer heartbreak, make bad decisions, and find new problems to worry about. The human condition is not “solved” by ASI any more than it was “solved” by antibiotics or electricity. But the floor---the minimum quality of life available to any human---rises to a level that would be unrecognizable to anyone alive today.

The child mortality chart tells the story of what is already possible when knowledge compounds:

Source: Our World in Data, drawing on UN IGME data.

From over 200 deaths per 1,000 live births in 1950 to roughly 37 today. That curve was bent by human intelligence operating with limited tools. The question before us is what happens when the tools become, for the first time, more intelligent than their makers.

Nick Bostrom’s Letter from Utopia ends with a line that has stayed with me for years:

“We love life here every instant. Every second is so good that it would blow your mind had its goodness not also been increased. My contemporaries and I bear witness, and all of history is on our side: the world can be much, much better than you ever dreamed.”

The pessimists may be right that the default outcome is catastrophic. But the optimists are right that the achievable outcome is magnificent. The alignment problem is not a reason to stop building. It is a reason to build carefully---because what we are building toward is worth getting right.


Further Reading


Share this post on:

Previous Post
Fundraising Across the Globe: What Nobody Tells You
Next Post
The 1,000-Qubit Ceiling That Probably Isn't