REVIEW 1 major objections
The Economics of p(doom): Scenarios of Existential Risk and Economic Growth in the Age of Transformative AI
T0 review · 1 major / 0 minor · reviewed 2026-05-23 · grok-4.3
Pith's one-line read Even low-probability AI extinction scenarios justify major investments in safety research.
desk verdict The paper organizes existing TAI scenarios and runs them through standard welfare calculations to argue that low-probability doom still justifies heavy safety spending, but that conclusion rests on parameter choices that can easily reverse the result. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The welfare evaluation framework that converts scenario probabilities, growth paths, and existential risks into expected aggregate welfare measures.
What would settle it
A calculation under the paper's own welfare framework showing that the expected welfare loss from low-probability extinction is smaller than the opportunity cost of the recommended safety investments.
Extended reading notes
Core claim
Mapping TAI scenarios and computing their aggregate welfare effects establishes that low probabilities of catastrophic misalignment still make substantial investments in AI safety and alignment economically rational, which in turn implies that present global efforts remain insufficient relative to the scale of the risks.
Load-bearing premise
The specific economic growth and welfare models that turn scenario probabilities into aggregate welfare numbers are accurate enough to support the investment conclusion.
Editorial extensions
If this is right
- AI safety and alignment budgets should scale up significantly to reflect their welfare returns.
- Alignment work should receive priority over capability advances that raise misalignment risk.
- Policy should treat existential risk reduction as a high-value economic intervention rather than a fringe concern.
- Global coordination on safety standards gains urgency once welfare weights are applied.
Reading between the lines
- The same welfare-weighting method could be applied to other low-probability, high-impact risks such as engineered pandemics.
- Results may shift if future modeling incorporates different assumptions about post-TAI growth rates or discounting.
- The framing invites re-examination of R&D allocation across all technologies that carry tail risks.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper organizes scenarios for transformative AI (TAI) ranging from post-scarcity cornucopia outcomes to human extinction from misalignment, evaluates their existential risks and economic growth trajectories, and translates these into aggregate welfare comparisons. It concludes that even low-probability catastrophic scenarios justify substantial increases in AI safety and alignment research investments relative to current global efforts.
Significance. If the welfare calculations are robust, the paper would supply a quantitative economic rationale for prioritizing AI safety by showing that expected losses from low-p(doom) trajectories exceed the costs of safety measures under the modeled growth differentials. This approach integrates growth theory with existential risk assessment in a manner that could inform resource allocation debates, though its policy weight hinges on the untested sensitivity of the results to standard parameter choices.
major comments (1)
- [Welfare calculations (inferred from abstract and modeling description)] The central policy claim—that low-probability doom scenarios justify substantial safety investments—rests on welfare comparisons across trajectories that incorporate specific choices for the pure rate of time preference, elasticity of marginal utility, and long-run growth differentials between aligned and misaligned TAI worlds. No robustness checks varying these parameters within ranges common in the growth and climate literature (e.g., 1–3% discount rates or alternative population-weighting rules) are reported, leaving open the possibility that the net-present-value advantage of safety spending reverses sign.
Simulated Author's Rebuttal
We thank the referee for their careful reading and for highlighting the importance of robustness in the welfare calculations. We address the single major comment below.
read point-by-point responses
-
Referee: The central policy claim—that low-probability doom scenarios justify substantial safety investments—rests on welfare comparisons across trajectories that incorporate specific choices for the pure rate of time preference, elasticity of marginal utility, and long-run growth differentials between aligned and misaligned TAI worlds. No robustness checks varying these parameters within ranges common in the growth and climate literature (e.g., 1–3% discount rates or alternative population-weighting rules) are reported, leaving open the possibility that the net-present-value advantage of safety spending reverses sign.
Authors: We agree that the policy conclusion depends on these parameter choices and that the absence of reported robustness checks is a limitation. The manuscript employs benchmark values standard in the growth literature (pure rate of time preference of 1 percent, elasticity of marginal utility of 1, and growth rates derived from the scenario analysis). To address the concern directly, the revised version will include a new robustness subsection that systematically varies the pure rate of time preference over 0–3 percent, the elasticity over 0.5–2, and considers both total and average utilitarianism for population weighting. We will report the ranges under which the net-present-value advantage of safety investment remains positive, thereby testing for possible sign reversal. revision: yes
Circularity Check
No circularity: welfare evaluation uses independent economic growth models applied to externally specified scenarios
full rationale
The paper organizes TAI scenarios (cornucopia, doom, baseline) drawn from the existing literature and applies standard aggregate welfare calculations from growth theory to compare them. No equation or result is shown to be defined in terms of its own output, no fitted parameter is relabeled as a prediction, and no load-bearing premise rests solely on a self-citation chain that itself lacks external verification. The conclusion that low-probability doom scenarios can justify safety investment follows from the chosen discount rate, population ethics, and growth differentials, which are modeling choices rather than tautological reductions; these choices can be varied independently of the scenario probabilities themselves. The derivation therefore remains self-contained against external benchmarks.
Assumptions & free parameters
Cite this review
Pith. "Pith review of The Economics of p(doom): Scenarios of Existential Risk and Economic Growth in the Age of Transformative AI." pith.science (2026). https://pith.science/paper/2503.07341
@misc{pith2026250307341,
author = {Pith},
title = {Pith review of: The Economics of p(doom): Scenarios of Existential Risk and Economic Growth in the Age of Transformative AI},
year = {2026},
howpublished = {\url{https://pith.science/paper/2503.07341}},
note = {Machine review of arXiv:2503.07341}
}
read the original abstract
Recent advances in artificial intelligence (AI) have led to a wide range of predictions about its long-term impact on humanity. A central focus is the potential emergence of transformative AI (TAI), eventually capable of outperforming humans in all economically valuable tasks and fully automating labor. Discussed scenarios range from unprecedented economic growth and abundance ("post-scarcity" or "cornucopia") to human extinction after a misaligned TAI takes over ("AI doom"). However, the probabilities and implications of these scenarios remain highly uncertain. We contribute by organizing the various scenarios and evaluating their associated existential risks and economic outcomes in terms of aggregate welfare. Our results imply that even low-probability catastrophic outcomes justify substantial investments in AI safety and alignment research. This result highlights that current global efforts in AI safety and alignment research are insufficient relative to the scale and urgency of the risks posed by TAI.
Reviewed May 23, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.