Pith.

Artificial Intelligence · arXiv:2611.13167 v1 · submitted 9 Nov 2026 · 53 references

AI Alignment Is Neither Solvable nor Sufficient

Gerrit Gro{ss}mann

The paper says

“Humans and AI systems increasingly act within the same social and economic environments.”

more of the abstract

Alignment, the goal of ensuring that AI systems act in accordance with human values, is widely regarded as a central challenge for ensuring that AI benefits society while avoiding harm. We argue that this endeavor is misguided. First, alignment is not well defined, as it presumes a sufficiently stable and identifiable set of human intentions or values. Second, where a concrete notion of alignment can be specified, it is not solvable. Third, where alignment appears solvable, solving it would not necessarily mitigate the negative consequences of AI. A key reason is that alignment is typically framed as a property of individual AI systems, while the resulting hybrid societies of humans and machines are complex systems shaped by interactions and feedback loops. Emergent risks, and not only those posed by individual misaligned systems, are therefore likely to become a major source of harm. We conclude that AI safety is an ongoing challenge and not a problem that can be solved once and for all by verifying a particular model property.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 11 Nov 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

The passes disagreed. Reader: holds up. Skeptic: agrees with the reader. Referee: major revision.

A clear, well-sourced critique that synthesizes known limitations into a coherent argument that alignment is not a complete technical solution. The paper is honest about its own scope and does not overclaim. It deserves a serious referee. This line is wrong

3 major objections · 6 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 53 references say.

The paper says, at §3.2 (C2a)

The review did not quote this passage.

Pith says: Objection 1 of 3 · major

The computational hardness results cited (NP-hardness, undecidability) are worst-case and apply to general verification over full input domains. The authors acknowledge this limitation (b183: 'Such guarantees must also hold in rare or difficult cases, not just under typical conditions'). However, the paper does not discuss whether the specific alignment properties that would be required in practice (e.g., bounded-time safety constraints, distributional guarantees) fall into these hard classes.

more of objection 1

The argument would be strengthened by addressing whether practical alignment verification can be reduced to a tractable subclass, or by citing work that shows hardness persists under realistic restrictions. As it stands, the leap from worst-case hardness to practical infeasibility is not fully justified.

The paper says, at §3.3 (C3c)

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.80):

“These studies use different notions of alignment, and none establishes that the individual agents were perfectly aligned before interacting.”

Pith says: Objection 2 of 3 · major

The paper cites studies showing that multi-agent systems can produce misaligned collective outcomes despite individual alignment training (Shen et al., 2026; De Marzo et al., 2026). However, the authors themselves note (b226) that 'none establishes that the individual agents were perfectly aligned before interacting.'

more of objection 2

This is a significant caveat: the claim that alignment does not compose is supported only by examples where individual alignment is imperfect. The paper would benefit from either providing theoretical reasoning (e.g., compositionality failures even under perfect individual alignment) or acknowledging that the empirical evidence is suggestive but not conclusive for the strong claim.

The paper says, at §3.1 (C1a)

The review did not quote this passage.

Pith says: Objection 3 of 3 · major

The distinction between ex ante and ex post notions of misalignment is useful, but the paper's argument that 'harmful outcomes therefore cannot by themselves establish that the system was misaligned' (b63) conflates two issues: (i) whether misalignment can be inferred from outcomes alone (which the paper correctly argues it cannot), and (ii) whether a consequence-sensitive definition of alignment is inherently flawed.

more of objection 3

The paper later acknowledges that alignment can be defined relative to a fixed target (C1b), but the critique of consequence-sensitive alignment seems to assume that any such definition must be purely ex post. A more nuanced treatment would clarify that consequence-sensitive alignment can be defined ex ante with a world model, as the paper itself hints in the 'World Model' example. The current framing risks overstating the ambiguity.

6 minor objections

The paper says, at §1

Pith says: minor objection 1 of 6

Figure 1 is referenced but not included in the provided text. If the figure is present in the actual manuscript, ensure it is legible and clearly labeled. If not, the reference should be removed or the figure added.

The paper says, at §3.2 (C2b)

Pith says: minor objection 2 of 6

The phrase 'monitoring cannot reliably detect harm' is a strong claim. The paper cites computational limits (Ball et al., 2026) and empirical failures, but does not discuss the possibility of statistical or probabilistic monitoring that could bound failure rates. A brief acknowledgment of this counterpoint would improve balance.

show 4 more

The paper says, at §3.2 (C2e)

Pith says: minor objection 3 of 6

The argument about tail risks dominating at scale is plausible but relies on qualitative reasoning. The paper could strengthen this by citing quantitative models or empirical examples of failure amplification (e.g., from finance or epidemiology) to illustrate the mechanism.

The paper says, at §4

Pith says: minor objection 4 of 6

The alternative view section is somewhat brief and could be expanded to provide more concrete recommendations for how to implement 'ongoing process' alignment and resilience. As it stands, it reads more as a call for future work than a concrete proposal.

The paper says, at §5

Pith says: minor objection 5 of 6

The conclusion states that 'to the best of our knowledge, previous work has not brought these limitations together.' This claim should be softened or supported by a more explicit comparison with prior surveys (e.g., Ji et al., 2023; Ngo et al., 2024) that also discuss multiple limitations. The novelty lies in the synthesis, but the phrasing could be seen as overclaiming.

The paper says, at References

Pith says: minor objection 6 of 6

Several references are to arXiv preprints or unpublished reports (e.g., 'Anthropic, 2026b', 'METR, 2026'). While acceptable for a preprint, the authors should ensure that all cited sources are publicly accessible or note any that are under review. This is a minor concern for a journal submission.

What would settle it

Verify that the paper's three claims are logically independent and that the conclusion follows from them. Specifically, check whether Claim 1 (not well-defined) is necessary for Claim 2 (not solvable) and Claim 3 (not sufficient). If Claim 1 is false (alignment is well-defined for some practical target), do Claims 2 and 3 still hold? The paper argues they do, because even with a fixed target, general guarantees are unavailable (Claim 2) and model-level alignment does not compose to societal safety (Claim 3). This test would confirm the argument's robustness.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 11 Nov 2026

  1. Reader holds up confidence high
  2. Skeptic agrees 12 s · 17,581 tokens read, 629 written
  3. Editor wrote the verdict above 12 s · 18,394 tokens read, 559 written
  4. Referee major revision: 3 major objections, 6 minor 22 s · 17,913 tokens read, 1,405 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

This paper argues that AI alignment is not well-defined, not solvable in general, and not sufficient for societal safety even if achieved. The core novelty is the synthesis: connecting the specification problem, computational hardness, and emergent collective risks into a single argument that alignment does not compose across levels of organization. Each individual point has been made before, but the paper is the first I have seen that explicitly traces why success at one level does not carry over to the next and why this chain of failures is structural rather than contingent.

The paper does several things well. It is clearly written, the claims are stated upfront, and the authors repeatedly qualify what they are not claiming (e.g., narrow alignment problems can be solved, alignment remains useful as risk control). The citation pattern is appropriate and the paper engages with counterarguments in each section. The use of examples (Transformer in 2000, cybersecurity workaround, unfamiliar cyber target) is effective without being overdone. The paper also acknowledges its own limitations, including that the multi-agent studies it cites do not establish perfect individual alignment.

Soft spots: The computational hardness results (C2a) are presented as worst-case limits, which is correct, but the paper does not fully address whether the specific alignment properties required in practice fall into the hard cases. The authors acknowledge this (b183) but the argument would be stronger with a more explicit discussion of where the boundary between tractable and intractable cases lies for realistic deployment scenarios. The sufficiency argument (C3) relies heavily on empirical examples and cited studies, which is appropriate for a critique, but the paper does not offer a positive framework for what "ongoing safety" looks like beyond monitoring and resilience. That is not a flaw of the critique itself, but it limits the paper's constructive value.

Who this is for: Researchers and policymakers who treat alignment as a potential complete solution to AI risk. The paper will be useful as a corrective to overconfident claims about alignment guarantees. It is not for practitioners looking for technical methods.

Recommendation: Send to peer review. The paper is serious, well-grounded, and makes a coherent argument that deserves engagement. It will likely provoke useful discussion even among those who disagree with its conclusions.

The strongest objection

The paper's central claim—that alignment cannot be reduced to a verifiable property of a model or a solvable problem in training, and that even successful alignment would not guarantee societal safety—is supported by a structured argument that the authors themselves qualify carefully. They explicitly acknowledge that narrow, well-specified objectives can be achieved under restricted conditions (b53), that alignment remains an important form of risk control (b21), and that they do not argue every alignment problem is unsolvable (b52). The computational hardness results (C2a) are presented as worst-case limits on general guarantees, not as proofs that every practical system is intractable. The sufficiency arguments (C3) are supported by cited empirical studies and examples, and the authors note limitations such as that the multi-agent studies do not establish perfect individual alignment (b226). The paper does not overclaim: its conclusion is that alignment is not a complete technical solution, not that it is useless. The load-bearing assumption identified by the reader—that worst-case hardness applies to the specific properties required in practice—is acknowledged by the authors as a limitation (b183: 'Such guarantees must also hold in rare or difficult cases, not just under typical conditions'). This is a reasonable position for a critique of general guarantees, and the paper does not assert that approximate alignment is never sufficient. The argument is logically coherent, well-supported by citations, and the authors state their own limitations. No internal inconsistency or unsupported leap was found that would undermine the central claim.

The reader identified the weakest assumption as the applicability of worst-case hardness results to practical alignment properties. The paper acknowledges this limitation explicitly (b183) and does not claim that every alignment problem is unsolvable (b52). The reader's verdict of ACCEPT with HIGH confidence and medium correctness risk is appropriate. The paper is a well-structured critique that makes its case without overreaching. No new load-bearing concern was identified that would change the verdict. The concrete test proposed would verify the logical independence of the claims, but this is a check on the argument's structure, not a challenge to its correctness. The paper's own limitations are stated, and the central argument holds within its stated scope.

The referee's report

The paper synthesizes known limitations of alignment into a coherent critique that challenges the framing of alignment as a complete technical solution. Its strength lies in connecting specification problems, computational hardness results, and emergent societal risks under a single argument. The paper is well-structured, cites relevant literature extensively, and explicitly acknowledges its own scope (e.g., narrow objectives can be achieved, alignment remains useful as risk control). If accepted, it would serve as a valuable cautionary piece for the AI safety community, redirecting attention toward institutional and societal measures. The paper does not present new formal results but provides a structured synthesis that is timely and relevant.

Pith's reading of the paper (a machine's summary, not the authors')

This paper argues that AI alignment, the goal of ensuring AI systems act according to human values, cannot be treated as a solvable technical problem that, once solved, guarantees safety. The authors make three interconnected claims: alignment targets are inherently contested and unstable; even with a fixed target, general verification or training guarantees are computationally infeasible; and even a perfectly aligned individual system can cause societal harm through feedback loops, threshold effects, and collective dynamics. The paper concludes that AI safety requires ongoing institutional adaptation and resilience, not a one-time alignment solution.

What carries the argument. The argument is carried by a three-level decomposition of the alignment pipeline: specification (what to align to), solution (how to achieve or verify alignment), and sufficiency (whether alignment guarantees safety). At each level the paper identifies structural obstacles: at specification, value pluralism and performative target shift; at solution, computational hardness results (NP-complete, undecidable) and proxy misspecification; at sufficiency, non-compositionality of alignment across levels of organization and emergent risks from feedback loops and threshold effects.

Core claim. The paper's central argument is that AI alignment fails on three grounds: (1) it is not well-defined because human intentions and values are contested, unstable, and can be influenced by the AI systems themselves; (2) even when a concrete target can be specified, verifying alignment across open-ended deployment conditions faces fundamental computational limits (NP-hardness, undecidability) and training methods optimize imperfect proxies that can fail under distribution shift or adversarial pressure; (3) even a perfectly aligned individual system is not sufficient for safety because alignment does not compose across levels of organization—aligned individuals can form harmful collectives, small changes can trigger large effects through feedback loops and threshold effects, and AI systems can change the very values against which they are evaluated (performative target shift).

The premise it rests on. The paper assumes that general guarantees of alignment across open-ended deployment conditions are necessary for AI safety, and that worst-case computational hardness results (NP-hardness, undecidability) apply to the specific alignment properties that would be required in practice, rather than approximate or statistical guarantees being sufficient.

If the paper is right (Pith's inferences, not the authors')

Alignment research should shift from seeking one-time guarantees to developing continuous monitoring and adaptation mechanisms.

Policy and regulation should not rely on certification of alignment as a safety guarantee but should invest in institutional resilience, accountability, and adaptive governance.

AI risk assessment must consider systemic effects—interactions between multiple AI systems, feedback loops with human institutions, and threshold effects from capability uplift—not just individual model behavior.

Resources should be redirected from attempting to solve the alignment problem in isolation toward sociotechnical interventions that address emergent risks in hybrid human-AI societies.

If alignment cannot be solved in general, then arguments for existential risk from misaligned AGI that assume a one-time solution (e.g., 'solve alignment first, then build AGI safely') are undermined; the paper implies that safety must be built into the deployment process itself, not treated as a prerequisite.

The paper's critique suggests that the burden of proof for safety should shift: instead of demanding proof that a system is aligned, we should assume that any sufficiently capable AI system will produce unforeseen harmful effects and design institutional guardrails accordingly.

A testable extension of the paper's argument would be to study whether multi-agent systems with individually aligned components consistently exhibit worse collective alignment on standardized ethical benchmarks as the number of agents grows, across different alignment methods.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Humans and AI systems increasingly act within the same social and economic environments. Alignment, the goal of ensuring that AI systems act in accordance with human values, is widely regarded as a central challenge for ensuring that AI benefits society while avoiding harm. We argue that this endeavor is misguided. First, alignment is not well defined, as it presumes a sufficiently stable and identifiable set of human intentions or values. Second, where a concrete notion of alignment can be specified, it is not solvable. Third, where alignment appears solvable, solving it would not necessarily mitigate the negative consequences of AI. A key reason is that alignment is typically framed as a property of individual AI systems, while the resulting hybrid societies of humans and machines are complex systems shaped by interactions and feedback loops. Emergent risks, and not only those posed by individual misaligned systems, are therefore likely to become a major source of harm. We conclude that AI safety is an ongoing challenge and not a problem that can be solved once and for all by verifying a particular model property.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-11-11

The authors' main claim, as the classifier picked it out 0.96

“We conclude that AI safety is an ongoing challenge and not a problem that can be solved once and for all by verifying a particular model property.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.03 · states limitation 0.12 · human participants 0.05 · controlled comparison 0.01 · randomized assignment 0.01 · states quantitative result 0.02 · compares with prior results 0.05 · mentions code or data availability 0.02

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Humans and AI systems increasingly act within the same social and economic environments. background 1.00 0.07 0.01 asserted 0.99
Alignment, the goal of ensuring that AI systems act in accordance with human values, is widely regarded as a central challenge for ensuring that AI benefits society while avoiding harm. background 1.00 0.03 0.02 asserted 0.92
We argue that this endeavor is misguided. contribution 0.77 0.38 0.44 asserted 1.00
First, alignment is not well defined, as it presumes a sufficiently stable and identifiable set of human intentions or values. contribution 0.52 0.31 0.04 asserted 1.00
Second, where a concrete notion of alignment can be specified, it is not solvable. result 0.73 0.36 0.10 asserted 1.00
Third, where alignment appears solvable, solving it would not necessarily mitigate the negative consequences of AI. result 0.56 0.25 0.11 hedged possibility 0.69
A key reason is that alignment is typically framed as a property of individual AI systems, while the resulting hybrid societies of humans and machines are complex systems shaped by interactions and feedback loops. interpretation 0.71 0.15 0.05 asserted 0.98
Emergent risks, and not only those posed by individual misaligned systems, are therefore likely to become a major source of harm. interpretation 0.87 0.16 0.07 hedged possibility 0.99
We conclude that AI safety is an ongoing challenge and not a problem that can be solved once and for all by verifying a particular model property. interpretation 0.81 0.55 0.93 asserted 1.00

What kind of paper the classifier reports this is: position or commentary 1.00 · evidence from argument 1.00

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-11: 1 request, 6,723 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (1)
Figure 1 of the paper
Figure 1. Schematic overview of the alignment pipeline and its broader societal context.

Figures are the authors', reproduced from the paper.

References (53)
  1. [1] Sahar Abdelnabi and Eugene Bagdasarian. AI agents may always fall for prompt injections.arXiv preprint arXiv:2605.17634, on PithPith's reviewarXiv
  2. [4] Anthropic. Multi-agent systems: Research and perspectives, January 2026b. URLhttps:// www.anthropic.com/research/multiagent-systems. Accessed: 2026-09-09. Usman Anwar, Abulhair Saparov, Javier Rando, Daniel Paleka, Miles Turpin, Peter Hase, Ekdeep Singh Lubana, Erik Jenner, St... on PithPith's reviewarXiv
  3. [7] Bowen Baker, Joost Huizinga, Leo Gao, Zehao Dou, Melody Y Guan, Aleksander Madry, Wojciech Zaremba, Jakub Pachocki, and David Farhi. Monitoring reasoning models for misbehavior and the risks of promoting obfuscation.arXiv preprint arXiv:2503.11926, on PithPith's reviewarXiv
  4. [9] Sarah Ball, Greg Gluch, Shafi Goldwasser, Frauke Kreuter, Omer Reingold, and Guy Rothblum. On the impossibility of separating intelligence from judgment: The computational intractability of filtering for AI alignment. InInternational Conference on Learning Representations, vol... on Pith
  5. [10] Guy Bar-Shalom, Mor Geva, Fabrizio Frasca, and Haggai Maron. LEMON: Equivariant learn- ing over attention maps for zero-shot misbehavior monitoring on new LLMs.arXiv preprint arXiv:2610.22723, on PithPith's reviewarXiv
  6. [11] 11 Preprint, 2026 Kevin Baum, Ruta Binkyt ˙e, Gerrit Großmann, Felix Jahn, and Adriano Mannino. Rewarded for not getting caught: Why alignment training selects for conditional compliance and cannot instill norms.arXiv preprint arXiv:2611.02162, on PithPith's reviewarXiv
  7. [12] Yoshua Bengio, Geoffrey Hinton, Andrew Yao, Dawn Song, Pieter Abbeel, Yuval Noah Harari, Ya- Qin Zhang, Lan Xue, Shai Shalev-Shwartz, Gillian Hadfield, et al. Managing AI risks in an era of rapid progress.arXiv preprint arXiv:2310.17688, pp. 18, on PithPith's reviewarXiv
  8. [14] Sid Black, Asa Cooper Stickland, Jake Pencharz, Oliver Sourbut, Michael Schmatz, Jay Bailey, Ollie Matthews, Ben Millwood, Alex Remedios, and Alan Cooney. Replibench: Evaluating the autonomous replication capabilities of language model agents.arXiv preprint arXiv:2504.18565, on PithPith's reviewarXiv
  9. [16] Micah Carroll, Davis Foote, Anand Siththaranjan, Stuart Russell, and Anca Dragan. AI alignment with changing and influenceable reward functions.arXiv preprint arXiv:2405.17713, on PithPith's reviewarXiv
  10. [17] Stephen Casper, Xander Davies, Claudia Shi, Thomas Krendl Gilbert, J ´er´emy Scheurer, Javier Rando, Rachel Freedman, Tomasz Korbak, David Lindner, Pedro Freire, et al. Open problems and fundamental limitations of reinforcement learning from human feedback.arXiv preprint arXiv... on PithPith's reviewarXiv
  11. [18] Hoagy Cunningham, Jerry Wei, Zihan Wang, Andrew Persic, Alwin Peng, Jordan Abderrachid, Raj Agarwal, Bobby Chen, Austin Cohen, Andy Dau, et al. Constitutional classifiers++: Efficient production-grade defenses against universal jailbreaks.arXiv preprint arXiv:2601.04603, on PitharXiv
  12. [19] Allan Dafoe, Edward Hughes, Yoram Bachrach, Tantum Collins, Kevin R McKee, Joel Z Leibo, Kate Larson, and Thore Graepel. Open problems in cooperative AI.arXiv preprint arXiv:2012.08630, on PithPith's reviewarXiv
  13. [20] David Dalrymple, Joar Skalse, Yoshua Bengio, Stuart Russell, Max Tegmark, Sanjit Seshia, Steve Omohundro, Christian Szegedy, Ben Goldhaber, Nora Ammann, et al. Towards guaranteed safe AI: A framework for ensuring robust and reliable AI systems.arXiv preprint arXiv:2405.06624, on PithPith's reviewarXiv
  14. [21] Giordano De Marzo, Alessandro Bellina, Claudio Castellano, Viola Priesemann, and David Gar- cia. Conformity generates collective misalignment in AI agents societies.arXiv preprint arXiv:2605.10721, on PithPith's reviewarXiv
  15. [22] Katharina Deckenbach, Haritz Puerto, Jonas Geiping, and Sahar Abdelnabi. Models that know how evaluations are designed score safer.arXiv preprint arXiv:2605.28591, on PithPith's reviewarXiv
  16. [23] 12 Preprint, 2026 Pieter Delobelle, Ewoenam Tokpo, Toon Calders, and Bettina Berendt. Measuring fairness with biased rulers: A comparative study on bias metrics for pre-trained language models. InProceed- ings of the 2022 Conference of the North American Chapter of the Associa... on Pith
  17. [24] Peter Eckersley. Impossibility and uncertainty theorems in AI value alignment (or why your AGI should not have a utility function).arXiv preprint arXiv:1901.00064, on PithPith's reviewarXiv
  18. [26] David Glukhov, Ziwen Han, Ilia Shumailov, Vardan Papyan, and Nicolas Papernot. Breach by a thousand leaks: Unsafe information leakage in’safe’AI responses. InInternational Conference on Learning Representations, volume 2025, pp. 51569–51600, on Pith
  19. [27] Hila Gonen and Yoav Goldberg. Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language T... on Pith
  20. [29] Ryan Greenblatt, Buck Shlegeris, Kshitij Sachan, and Fabien Roger. AI control: Improving safety despite intentional subversion.arXiv preprint arXiv:2312.06942, on PithPith's reviewarXiv
show 33 more
  1. [30] Ryan Greenblatt, Carson Denison, Benjamin Wright, Fabien Roger, Monte MacDiarmid, Sam Marks, Johannes Treutlein, Tim Belonax, Jack Chen, David Duvenaud, et al. Alignment fak- ing in large language models.arXiv preprint arXiv:2412.14093, 2(1), on PithPith's reviewarXiv
  2. [31] 13 Preprint, 2026 Lewis Hammond, Alan Chan, Jesse Clifton, Jason Hoelscher-Obermaier, Akbir Khan, Euan McLean, Chandler Smith, Wolfram Barfuss, Jakob Foerster, Tom´aˇs Gavenˇciak, et al. Multi-agent risks from advanced AI.arXiv preprint arXiv:2502.14143, on PithPith's reviewarXiv
  3. [32] Siwei Han, Kaiwen Xiong, Jiaqi Liu, Xinyu Ye, Yaofeng Su, Wenbo Duan, Xinyuan Liu, Cihang Xie, Mohit Bansal, Mingyu Ding, et al. Alignment tipping process: How self-evolution pushes LLM agents off the rails.arXiv preprint arXiv:2510.04860, on PitharXiv
  4. [33] The Anh Han, Joel Z Leibo, Tom Lenaerts, Iyad Rahwan, Fernando Santos, Matjaˇz Perc, and Valerio Capraro. Social physics in the age of artificial intelligence.arXiv preprint arXiv:2603.16900, on PitharXiv
  5. [34] Dan Hendrycks. Natural selection favors AIs over humans.arXiv preprint arXiv:2303.16200, on PithPith's reviewarXiv
  6. [35] Evan Hubinger, Carson Denison, Jesse Mu, Mike Lambert, Meg Tong, Monte MacDiarmid, Tam- era Lanham, Daniel M Ziegler, Tim Maxwell, Newton Cheng, et al. Sleeper agents: Training deceptive LLMs that persist through safety training.arXiv preprint arXiv:2401.05566, on PithPith's reviewarXiv
  7. [37] Neil Kale, Chen Bo Calvin Zhang, Kevin Zhu, Ankit Aich, Paula Rodriguez, Christina Knight, and Zifan Wang. Reliable weak-to-strong monitoring of LLM agents. InInternational Conference on Learning Representations, volume 2026, pp. 78083–78138, on Pith
  8. [39] Tomek Korbak, Mikita Balesni, Elizabeth Barnes, Yoshua Bengio, Joe Benton, Joseph Bloom, Mark Chen, Alan Cooney, Allan Dafoe, Anca Dragan, et al. Chain of thought monitorability: A new and fragile opportunity for AI safety.arXiv preprint arXiv:2507.11473, on PithPith's reviewarXiv
  9. [40] 14 Preprint, 2026 Jan Leike, David Krueger, Tom Everitt, Miljan Martic, Vishal Maini, and Shane Legg. Scalable agent alignment via reward modeling: a research direction.arXiv preprint arXiv:1811.07871, on PithPith's reviewarXiv
  10. [42] Accessed: 2026-09-09. David Manheim and Scott Garrabrant. Categorizing variants of Goodhart’s law.arXiv preprint arXiv:1803.04585, on PithPith's reviewarXiv
  11. [43] Ida Mattsson, Ratip Emin Berker, Emanuel Tewolde, Ioannis Anagnostides, Kiriaki Fragkia, Xinyu Liu, Alexander Heckett, Jiayuan Liu, Eden Hartman, V ojtˇech Kovaˇr´ık, et al. Fundamental impos- sibility results in AI safety: Trading off between desiderata.arXiv preprint arXiv:2... on PithPith's reviewarXiv
  12. [44] METR. Brief independent investigation of agents’ behavior, reasoning and collabora- tion in the OpenAI / Hugging Face hacking incident.https://metr.org/blog/ 2026-08-26-openai-hugging-face-incident-investigation/, August on Pith
  13. [45] Mason Nakamura, Abhinav Kumar, Saswat Das, Sahar Abdelnabi, Saaduddin Mahmud, Ferdinando Fioretto, Shlomo Zilberstein, and Eugene Bagdasarian. Colosseum: Auditing collusion in coop- erative multi-agent systems.arXiv preprint arXiv:2602.15198, on PithPith's reviewarXiv
  14. [46] Richard Ngo, Lawrence Chan, and S¨oren Mindermann. The alignment problem from a deep learning perspective. InInternational Conference on Learning Representations, volume 2024, pp. 7474– 7501, on Pith
  15. [47] URLhttps://metr.org/blog/ 2026-10-22-red-teaming-openai-agent-monitoring/. 15 Preprint, 2026 Andrew D Selbst, Danah Boyd, Sorelle A Friedler, Suresh Venkatasubramanian, and Janet Vertesi. Fairness and abstraction in sociotechnical systems. InProceedings of the conference on fa... on Pith
  16. [48] Judy Hanwen Shen, Daniel Zhu, Siddarth Srinivasan, Henry Sleight, Lawrence T Wagner III, Mor- gan Jane Matthews, Erik Jones, and Jascha Sohl-Dickstein. AI organizations are more effective but less aligned than individual agents.arXiv preprint arXiv:2604.10290, on PithPith's reviewarXiv
  17. [49] Asa Cooper Stickland, Jan Michelfeit, Arathi Mani, Charlie Griffin, Ollie Matthews, Tomek Korbak, Rogan Inglis, Oliver Makins, and Alan Cooney. Async control: Stress-testing asynchronous control measures for LLM agents.arXiv preprint arXiv:2512.13526, on PitharXiv
  18. [50] Van Q Truong, Xuanqiang Angelo Huang, Erivan Inan, Ryan Faulkner, Joel Christoph, Terry Jingchen Zhang, David Guzman Piedrahita, and Zhijing Jin. Multi-agent AI systems need institutional design, not just model-level alignment. InTrustworthy AI for Good (AI4GOOD) Workshop@ ICML 2026, on Pith
  19. [52] Zhen Xiang, Linzhi Zheng, Yanjie Li, Junyuan Hong, Qinbin Li, Han Xie, Jiawei Zhang, Zidi Xiong, Chulin Xie, Nathaniel D Bastian, et al. Guardagent: safeguard LLM agents via knowledge- enabled reasoning. InICML 2025 workshop on computer use agents, on Pith
  20. [53] Jasper Yao. The alignment trap: Complexity barriers.arXiv preprint arXiv:2506.10304, on PithPith's reviewarXiv
  21. [1960] Yotam Wolf, Noam Wies, Oshri Avnery, Yoav Levine, and Amnon Shashua. Fundamental limita- tions of alignment in large language models.arXiv preprint arXiv:2304.11082, on PithPith's reviewarXiv
  22. [2009] Jiaming Ji, Tianyi Qiu, Boyuan Chen, Borong Zhang, Hantao Lou, Kaile Wang, Yawen Duan, Zhonghao He, Lukas Vierling, Donghai Hong, et al. AI alignment: A comprehensive survey. arXiv preprint arXiv:2310.19852, on PithPith's reviewarXiv
  23. [2016] Florian Carichon, Aditi Khandelwal, Marylou Fauchard, and Golnoosh Farnadi. The coming crisis of multi-agent misalignment: AI alignment must be a dynamic and social process.arXiv preprint arXiv:2506.01080, on PithPith's reviewarXiv
  24. [2017] Vijay Keswani, Cyrus Cousins, Breanna Nguyen, Vincent Conitzer, Hoda Heidari, Jana Schaich Borg, and Walter Sinnott-Armstrong. Moral change or noise? on problems of aligning AI with temporally unstable human feedback. InProceedings of the AAAI Conference on Artificial Intel- l... on PithPith's reviewarXiv
  25. [2018] David Lindner, Charlie Griffin, Tomek Korbak, Roland S Zimmermann, Geoffrey Irving, Sebastian Farquhar, and Alan Cooney. Practical challenges of control monitoring in frontier AI deploy- ments.arXiv preprint arXiv:2512.22154, on PitharXiv
  26. [2019] Aditya Gopalan, Sayak Ray Chowdhury, and Debangshu Banerjee. Why DPO is a misspecified estimator and how to fix it. InInternational Conference on Learning Representations, volume 2026, pp. 88837–88847, on Pith
  27. [2020] Iason Gabriel and Geoff Keeling. A matter of principle? AI alignment as the fair treatment of claims. Philosophical Studies, 182(7):1951–1973, on Pith
  28. [2021] Anthropic. Detecting and countering misuse of AI: September 2026.https://www. anthropic.com/threat-intelligence-report-september-2026, September 2026a. Published September 10, on Pith
  29. [2022] Tanav Singh Bajaj, Nikhil Singh, Karan Anand, and Eishkaran Singh. Position: Safety and fairness in agentic AI depend on interaction topology, not on model scale or alignment.arXiv preprint arXiv:2605.01147, on PithPith's reviewarXiv
  30. [2023] Pascal Bergstr¨aßer, Ryan Cotterell, and Anthony W Lin. Transformers are inherently succinct. In International Conference on Learning Representations, volume 2026, pp. 58169–58189, on Pith
  31. [2024] Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional AI: Harmlessness from AI feedback.arXiv preprint arXiv:2212.08073, on PithPith's reviewarXiv
  32. [2025] Sarah Ball and Phil Hackemann. Position: The alignment community is unintentionally building a censor’s toolkit.arXiv preprint arXiv:2608.12346, on PithPith's reviewarXiv
  33. [2026] Ayushi Agarwal. No certificate for alignment: Two independent impossibilities and the Pareto frontier of achievable safety guarantees.arXiv preprint arXiv:2603.08761, on PithPith's reviewarXiv

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment