\texttt{R$^\textbf{2}$AI}: Towards Resistant and Resilient AI in an Evolving World

Bowen Zhou; Chaochao Lu; Jie Fu; Xiang Wang; Youbang Sun

REVIEW 4 major objections 5 minor 12 references

\texttt{R$^\textbf{2}$AI}: Towards Resistant and Resilient AI in an Evolving World

T0 review · 4 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This position paper argues that AI safety should be engineered as a continual coevolutionary process—kept inside a defined safety margin at every upgrade—rather than as a one-time property or post-hoc patch.

desk verdict A clearly written safety research agenda; the formal guarantee is a tautology, but the framework is worth a serious referee as a position paper. read the letter →

arxiv 2509.06786 v1 pith:TPLXUJSD submitted 2025-09-08 cs.LG

Youbang Sun , Xiang Wang , Jie Fu , Chaochao Lu , Bowen Zhou This is my paper

classification cs.LG

keywords AIsafetysafe-by-coevolutionadversarialrobustnessresiliencefast-slowmodelswindtunnelcontinuallearningexistentialrisk

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AI safety, as it is usually practiced, treats risk as something to patch after the fact or to rule out once at deployment. This position paper argues that both approaches lose in open-ended environments, because threats keep evolving. It proposes safe-by-coevolution: a system is never declared safe once; instead it is held inside a defined safety margin at every upgrade, through adversarial training, feedback loops, and continual learning, so safety improves in step with capability. The paper then sketches R2AI, an architecture that makes the principle operational: a fast safe model handles known threats at low latency, a slow safe model reasons about novel ones, a safety wind tunnel generates adaptive attacks, and the external environment feeds real incidents back into the loop. If the argument holds, near-term robustness and long-term alignment become the same continuous process rather than separate, ever-widening problems.

What carries the argument

The load-bearing machinery is the safety margin M plus the assumed existence of a safety-preserving coevolutionary operator C (Hypothesis 3.2). M is a rigorously defined set of conditions under which the system counts as safe; C is built from adversarial co-training, feedback loops, and continual learning. The paper's concrete instrument for C is the R2AI architecture: a Fast Safe Model (low-latency filter) and a Slow Safe Model (deliberative reasoner) playing a cooperative leader-follower Stackelberg game, attacked by a programmable Attacker in a Safety Wind Tunnel and checked by a Verifier, with the External Environment supplying real-world failure signals. A Temporal Swiss Cheese Model of

What would settle it

Run an R2AI-style loop (fast and slow safe models, a wind tunnel with an adaptive attacker, and feedback retraining) for many rounds on a fixed task family, and measure the attacker's success rate or the fraction of states falling outside the safety margin M. If that quantity is non-decreasing over rounds, or if a red-line breach requires a reset that never returns to M, Hypothesis 3.2 is empirically falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that 'Make Safe AI' should be reformulated as safe-by-coevolution: safety is not a property a model has at deployment but a process it runs, analogous to adaptive immunity. The formal core is an induction: start from a system verified inside a safety margin M; assume there exists a coevolutionary mechanism C that maps any safe system to a safe upgraded system; then by induction the system remains inside M forever. The paper then proposes R2AI as an engineering realization of C: a fast system-1 safety filter and a slow system-2 reasoning model train together as a cooperative Stackelberg game, while an internal safety wind tunnel (attacker plus verifier) supplies a

Load-bearing premise

The entire guarantee rests on Hypothesis 3.2: for any system already inside the safety margin, there is a coevolutionary mechanism whose next version is also inside it—in other words, safety-preserving self-improvement exists. If adversarial pressure can outrun defensive learning, the induction has no next step.

Editorial extensions

If this is right

Safety becomes a measurable, iterative property: each model release must be shown inside the safety margin before the next is trained, turning alignment into a certification step.
Known threats get cheap defenses (fast model) while novel threats get expensive reasoning (slow model), so the cost of being safe falls as fast mechanisms internalize slow judgments.
Deployment incidents stop being post-hoc lessons and become training data for the attacker in the wind tunnel, shrinking the gap between simulated and real risk.
Red-line breaches are survivable by design: rollback to a previously verified checkpoint preserves the induction instead of ending the program.
Because safety evolves rather than freezes, the framework extends from high-stakes deployment today to superintelligent systems where human oversight cannot keep pace.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The formal result is conditional: Proposition 3.3 states that if Hypotheses 3.1 and 3.2 hold, safety persists, so the actual contribution is the reduction of AI safety to the construction of C—an engineering target, not a finished guarantee.
If safe-by-coevolution is right, the field's evaluation metric should shift from static safety benchmarks to safety-margin trajectories: how far a system drifts outside M over a sequence of adaptive attacks.
A testable extension is that R2AI-style fast-plus-slow architectures will outperform monolithic alignment at equal inference budget under adaptive attackers, a comparison one could run on existing models.
The biological analogy suggests that robust safety generalization may require diversity of attackers rather than one optimal adversary, an open question the paper leaves implicit.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

I read R2AI. You should know two things upfront: it is a position paper, not a technical result; and the formal guarantee in Section 3.3.1 is exactly as circular as the reader says. The one theorem, Proposition 3.3, is a direct induction over Hypotheses 3.1 and 3.2, and Hypothesis 3.2 (for every safe A_t there exists a coevolutionary mechanism C such that C(A_t) is safe) is the entire claim being made. It is not proved, not instantiated, and there is no decision procedure for membership in M. On the paper's own terms, Section 4.2.4 says systems will inevitably encounter black swan events that push them beyond the defined safety margin; that directly contradicts the invariant A_t in M for all t, unless the formal claim is silently scoped to exclude those events. Reset-and-recover re-invokes Hypothesis 3.1 at a new checkpoint, but it does not restore the original induction. The formal machinery adds no evidence.

Having said that, the paper deserves credit for the framing it does well. Safe-by-coevolution, the resistance/resilience distinction, the five-level ladder, and the fast-slow safety system with a safety wind tunnel are a useful synthesis of a scattered literature. The writing is clear, and the citations are broad and mostly appropriate, including the AI-45 Law line from the same lab. For someone building a research agenda, this is a workable skeleton. The paper does not overpromise in every sentence; it explicitly labels itself as position and hypothesis.

Soft spots beyond the circularity: Hypothesis 3.2 imposes no requirement that C increases capability, so a mechanism that never updates the system vacuously satisfies it; the claimed coevolution of capability and safety is not captured. And the lack of any implementation, benchmark, or case study is a real gap, but it is the expected gap for a position paper. What is less acceptable is presenting the induction as a 'formal foundation' when it is a restatement of the assumption.

Who gets value: readers thinking about how to organize safety research directions, especially program managers and researchers writing agendas. It is not a result to build on. I would send it to peer review as a position paper, with a request to reframe Section 3.3.1 as a design principle rather than a theorem and to state clearly that the universal guarantee is not claimed across black swan events.

Referee Report

4 major / 5 minor

Summary. The paper is a position paper proposing "safe-by-coevolution" as a new formulation of the "Make Safe AI" paradigm, and presenting R2AI (Resistant and Resilient AI) as an operational framework. The framework combines fast and slow safe models, a safety wind tunnel for adversarial simulation and verification, and continual feedback loops, together with a reset-and-recover mechanism for black-swan events. The authors claim that this yields a scalable, proactive path to continual safety in dynamic environments, including long-term AGI/ASI risks. The formal foundation in Section 3.3.1 consists of Hypothesis 3.1 (near-term safety guarantee), Hypothesis 3.2 (safe iterative step), and Proposition 3.3 (continual safety by induction). No experiments, benchmarks, or case studies are presented.

Significance. If the framework were realized and its guarantees were substantive, it could provide a useful reconceptualization of AI safety as an adaptive, ongoing process rather than a static property. The paper merits credit for integrating several existing ideas—dual-process safety, adversarial training, red-teaming, continual learning—into a coherent architectural vision, and for making its assumptions explicit enough to be scrutinized. The biological immunity analogy is evocative and could inspire concrete research directions. However, the formal result is tautological: Proposition 3.3 is a direct restatement of Hypothesis 3.2. The paper therefore does not currently establish a theoretical foundation, and the practical claims in the abstract are unsupported by evidence. The significance is conditional on future work that either constructs the coevolutionary mechanism C for a nontrivial safety margin M or repositions the paper as a research agenda rather than a framework with guaranteed properties.

major comments (4)

[§3.3.1, Hypothesis 3.2 and Proposition 3.3] Hypothesis 3.2 asserts that for every safe A_t there exists a coevolutionary mechanism C such that C(A_t) is safe. Proposition 3.3 is nothing more than mathematical induction applied to this assertion. This is not a derivation of safety but an assumption of the central open problem. Moreover, H3.2 is vacuous as stated: the identity map C(A_t)=A_t satisfies it whenever A_t ∈ M, and the hypothesis places no constraint that C increase capability. The claimed coevolution of capability and safety is not captured. To be nontrivial, the authors must either construct C for a concrete class of systems, prove a preservation property for a defined M, or add an explicit capability/safety coupling.
[§4.2.4 versus §3.3.1] The reset-and-recover section states that AI systems 'will inevitably encounter black swan events ... that exceed existing safeguards and push them beyond their defined safety margin.' On the natural reading where A_t is the deployed system, this directly contradicts the universal invariant A_t ∈ M for all t ≥ t0 in Proposition 3.3. If such events are excluded from the scope of H3.2, the formal claim is silently weakened and no longer supports the abstract's promise. Reset-and-recover does not repair the induction: it re-invokes Hypothesis 3.1 at a new state A'_t, but no argument shows that A'_t is reachable from an unsafe state, nor that the reset procedure can be implemented while preserving the safety properties on which the induction depends.
[§3.1, §3.3.1] The formal framework relies on a 'rigorously defined set of conditions' M and on Hypothesis 3.1 requiring a 'verifiably safe' initialization. However, neither a concrete safety margin nor a decision procedure for membership in M is ever given. Without such a specification, neither Hypothesis 3.1 nor Hypothesis 3.2 is checkable, and the induction has no empirical content. The authors should at least define M for a representative class of systems (e.g., a formal specification for a bounded agentic task) or state the conditions under which certification is decidable.
[§4 and Abstract] The paper presents R2AI as a 'practical framework' and claims a 'scalable and proactive path to maintain continual safety,' yet no experiments, benchmarks, prototypes, or case studies support this claim. Section 4 repeatedly says components 'can be implemented' and 'can be built' but provides no evidence that the proposed adversarial loop, fast-slow interaction, and continual learning strategies actually preserve a safety margin. For a position paper, this may be acceptable if the claims are explicitly framed as a research agenda; as written, the strength of the claims exceeds the evidence.

minor comments (5)

[§2.1] Typo: 'Mimenic Reflection' should be 'Mimetic Reflection'.
[Figure 1a] The axes and 'safety scores' are not defined; please clarify how capability and safety scores are computed and reproduced, and state whether the figure is illustrative or based on a specific benchmark.
[§3.3.1] The notation A_{t+1}=C(A_t) conflates a system with a function on systems. It would be clearer to write C_t(A_t) or to describe C as a family of update operators indexed by t.
[§3.3.1] The phrase 'This formal result suggests...' overstates the content of Proposition 3.3. If the hypotheses are retained, they should be labeled as assumptions or design goals rather than as a 'theoretical foundation' that yields a guarantee.
[References] Several references are to arXiv preprints or institutional reports; please update to peer-reviewed versions where available. Also, the duplicated entry for 'Guan et al. (2024a/2024b)' appears to be the same work and should be merged.

Circularity Check

2 steps flagged · score 8.0 of 10

Proposition 3.3 is a restatement of Hypothesis 3.2, so the formal guarantee reduces by construction to the assumption it claims to establish.

self definitional [Section 1 (three-step safe-by-coevolution) and Section 3.3.1 (Hypothesis 3.2 / Proposition 3.3)]
"2) Safe iterative step: for any system already safe, design coevolutionary mechanisms—adversarial interactions, feedback loops, and continuous updates—to guide each upgrade back within that margin; 3) Continual safety by induction: repeat this loop so that safety evolves in sync with capability. ... Hypothesis 3.2 ... ∀t, At ∈M⇒At+1 =C(At)∈M. ... Proposition 3.3 ... If Hypotheses 3.1 and 3.2 hold, then for all t≥t0, the iteratively evolved system remains within the safety margin: At ∈M,∀t≥t0."

The framework is defined by the safe iterative step, which asserts that any safe system can be updated to remain safe. Hypothesis 3.2 is just that definition in symbols. Proposition 3.3 then adds the base case from Hypothesis 3.1 and concludes At ∈ M for all t≥t0, which is exactly the universal content already contained in H3.2. No mechanism C is constructed, no membership test for M is given, and no empirical instantiation is provided, so the 'derived' guarantee is the input assumption restated by induction. The paper's own Section 4.2.4 says systems will inevitably be pushed beyond the safety margin, undercutting the assumed invariant.
self citation load bearing [Section 1 (AI-45° Law; Figure 1) and Conclusion]
"To capture this tension, Shanghai AI Lab proposed the AI-45◦ Law (Yang et al., 2024): safety and capability must coevolve along a 45◦ diagonal trajectory."

The normative premise that safety and capability must coevolve—and hence the safe-by-coevolution framework itself—is sourced to Yang et al. (2024), which shares authors (Chaochao Lu, Bowen Zhou) and the same lab with the present paper. The cited 'law' is not an externally machine-checked or code-reproduced theorem; it is the authors' own prior position. The current paper then treats it as an established law and concludes that R2AI operates 'under the AI-45◦ Law,' so the framework's central motivation leans on a same-author citation rather than on independent evidence.

full rationale

The formal derivation in Section 3.3.1 is circular by construction: Proposition 3.3 is the induction closure of the existential assumption in Hypothesis 3.2, so it adds no information beyond what the framework already stipulated in its informal Step 2. This is the load-bearing step for the paper's strongest claim of scalable, long-term continual safety. The additional reliance on the AI-45° Law, a same-author prior work, compounds the issue by importing the 'must coevolve' premise from the authors' own previous position paper. Because the paper contains no implementation, no construction of the coevolutionary mechanism, and no checkable safety-margin predicate, the guarantee is an assumption presented as a formal result. The paper is internally transparent about black swan events exceeding the margin, which makes the universal invariant in Proposition 3.3 implausible on its face. Score 8 reflects that the central formal claim reduces to its own definition.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper introduces no numeric free parameters. Its formal foundation rests on two explicit hypotheses that are effectively assumptions: near-term safety is achievable, and a safety-preserving update mechanism exists. The safety margin and the AI-45 Law are additional conceptual axioms. No new physical or mathematical entities are postulated.

assumptions (4)

domain assumption Hypothesis 3.1: There exists an initial time t0 and system A_t0 satisfying the safety margin M.
Section 3.3.1. Assumed without proof; asserts that current alignment and verification techniques can produce a verifiably safe initial system.
ad hoc to paper Hypothesis 3.2: For any safe system A_t, there exists a coevolutionary mechanism C such that A_{t+1} = C(A_t) is also safe.
Section 3.3.1. This is the core assumption that the framework's safety guarantee depends on; it effectively assumes the central claim of continual safety.
domain assumption The safety margin M is a rigorously defined, checkable set of conditions.
Section 3.3.1. Needed for the induction to have content, but no formal specification or verification method is given in the paper.
domain assumption AI-45 Law (Yang et al., 2024): safety and capability must coevolve along a 45-degree diagonal trajectory.
Section 1. Adopted as a normative framing from a paper with overlapping authors; used to justify the coevolutionary target.

how reviews work

0 comments

Cite this review

Pith. "Pith review of \texttt{R$^\textbf{2}$AI}: Towards Resistant and Resilient AI in an Evolving World." pith.science (2026). https://pith.science/paper/TPLXUJSD

@misc{pith2026250906786,
  author       = {Pith},
  title        = {Pith review of: \textttR$^\textbf2$AI: Towards Resistant and Resilient AI in an Evolving World},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TPLXUJSD}},
  note         = {Machine review of arXiv:2509.06786}
}

abstract

In this position paper, we address the persistent gap between rapidly growing AI capabilities and lagging safety progress. Existing paradigms divide into ``Make AI Safe'', which applies post-hoc alignment and guardrails but remains brittle and reactive, and ``Make Safe AI'', which emphasizes intrinsic safety but struggles to address unforeseen risks in open-ended environments. We therefore propose \textit{safe-by-coevolution} as a new formulation of the ``Make Safe AI'' paradigm, inspired by biological immunity, in which safety becomes a dynamic, adversarial, and ongoing learning process. To operationalize this vision, we introduce \texttt{R$^2$AI} -- \textit{Resistant and Resilient AI} -- as a practical framework that unites resistance against known threats with resilience to unforeseen risks. \texttt{R$^2$AI} integrates \textit{fast and slow safe models}, adversarial simulation and verification through a \textit{safety wind tunnel}, and continual feedback loops that guide safety and capability to coevolve. We argue that this framework offers a scalable and proactive path to maintain continual safety in dynamic environments, addressing both near-term vulnerabilities and long-term existential risks as AI advances toward AGI and ASI.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

12 extracted references · 10 linked inside Pith

[1]

V.,Precup,D.,vanHasselt,H

Abel,D.,Barreto,A.,Roy,B. V.,Precup,D.,vanHasselt,H. P.,andSingh,S. Adeﬁnitionofcontinual reinforcementlearning. In NeurIPS,2023. Alon,G.andKamfonas,M. Detectinglanguagemodelattackswithperplexity. CoRR,abs/2308.14132,

arXiv 2023
[7]

Kirichenko,P.,Izmailov,P.,andWilson,A. G. Lastlayerre-trainingissuﬃcientforrobustnessto spuriouscorrelations. In ICLR.OpenReview.net,2023. Kitano,H. Biologicalrobustness. NatureReviewsGenetics,5(11):826–837,2004. Korbak,T.,Balesni,M.,Barnes,E.,Bengio,Y.,Benton,J.,Bloom,J.,Chen,M.,Cooney,A.,Dafoe,A., Dragan,A.,et al. Chainofthoughtmonitorability: Anewandfr...

arXiv 2023
[9]

Openproblemsinmechanisticinterpretability

Sharkey,L.,Chughtai,B.,Batson,J.,Lindsey,J.,Wu,J.,Bushnaq,L.,Goldowsky-Dill,N.,Heimersheim, S.,Ortega,A.,Bloom,J.,et al. Openproblemsinmechanisticinterpretability. arXivpreprint arXiv:2501.16496,2025. Shinn,N.,Cassano,F.,Gopinath,A.,Narasimhan,K.,andYao,S. Reﬂexion: languageagentswith verbalreinforcementlearning. In NeurIPS,2023a. Shinn,N.,Cassano,F.,Gopi...

arXiv 2025
[12]

Yang,X.,Deng,G.,Shi,J.,Zhang,T.,andDong,J. S. Enhancingmodeldefenseagainstjailbreakswith proactivesafetyreasoning. CoRR,abs/2501.19180,2025b. Yao,S.,Zhao,J.,Yu,D.,Du,N.,Shafran,I.,Narasimhan,K. R.,andCao,Y. React: Synergizing reasoningandactinginlanguagemodels. In ICLR.OpenReview.net,2023. Yi,S.,Liu,Y.,Sun,Z.,Cong,T.,He,X.,Song,J.,Xu,K.,andLi,Q. Jailbreak...

arXiv 2023
[1997]

Constitutionalai: Harmlessnessfromaifeedback

Bai,Y.,Kadavath,S.,Kundu,S.,Askell,A.,Kernion,J.,Jones,A.,Chen,A.,Goldie,A.,Mirhoseini,A., McKinnon,C.,et al. Constitutionalai: Harmlessnessfromaifeedback. arXivpreprint arXiv:2212.08073,2022. Barlow,J. B.,Rae,W. H.,andPope,A. Low-speedwindtunneltesting. Johnwiley&sons,1999. Barto,A. G. Intrinsicmotivationandreinforcementlearning. In Intrinsicallymotivate...

arXiv 2022
[1998]

Largelanguagemodelsinﬁnance: Asurvey

Li,Y.,Wang,S.,Ding,H.,andChen,H. Largelanguagemodelsinﬁnance: Asurvey. In Proceedingsof thefourthACMinternationalconferenceonAIinﬁnance,pp.374–382,2023. 27 R2AI:TowardsResistantandResilientAIinanEvolvingWorld Lin,S.,Hilton,J.,andEvans,O. Truthfulqa: Measuringhowmodelsmimichumanfalsehoods. arXiv preprintarXiv:2109.07958,2021. Liu,C.,Yuan,Y.,Yin,Y.,Xu,Y.,Xu...

arXiv 2023
[2006]

Saferlhf: Safereinforcement learningfromhumanfeedback

Dai,J.,Pan,X.,Sun,R.,Ji,J.,Xu,X.,Liu,M.,Wang,Y.,andYang,Y. Saferlhf: Safereinforcement learningfromhumanfeedback. arXivpreprintarXiv:2310.12773,2023. Dalrymple,D.,Skalse,J.,Bengio,Y.,Russell,S.,Tegmark,M.,Seshia,S.,Omohundro,S.,Szegedy,C., Goldhaber,B.,Ammann,N.,et al. Towardsguaranteedsafeai: Aframeworkforensuringrobust andreliableaisystems. arXivpreprin...

arXiv 2023
[2013]

S.,Carpenter,S

Walker,B.,Holling,C. S.,Carpenter,S. R.,andKinzig,A. Resilience,adaptabilityandtransformability insocial–ecologicalsystems. Ecologyandsociety,9(2),2004. Wang,H.,Qin,Z.,Zhao,Y.,Du,C.,Lin,M.,Wang,X.,andPang,T. Lifelongsafetyalignmentfor languagemodels. CoRR,abs/2505.20259,2025a. Wang,J.,Pun,A.,Tu,J.,Manivasagam,S.,Sadat,A.,Casas,S.,Ren,M.,andUrtasun,R. Advs...

arXiv 2004

Show all 12 references

[2018]

Apromisingpathtowardsautoformalizationandgeneralartiﬁcialintelligence

Szegedy,C. Apromisingpathtowardsautoformalizationandgeneralartiﬁcialintelligence. In InternationalConferenceonIntelligentComputerMathematics,pp.3–20.Springer,2020. Terekhov,M.,Liu,Z.N. D.,Gulcehre,C.,andAlbanie,S. Controltax: Thepriceofkeepingaiincheck. arXivpreprintarXiv:2506...

2020
[2023]

EBOOK:FundamentalsofAerodynamics(SIunits)

Anderson,J. EBOOK:FundamentalsofAerodynamics(SIunits). McGrawhill,2011. Anthropic. Challengesinredteamingaisystems,June132024. URL https://www.anthropic.com/news/challenges-in-red-teaming-ai-systems. 21 R2AI:TowardsResistantandResilientAIinanEvolvingWorld Anthropic. Claude4[la...

2011 arXiv
[2024]

Theroadtoartiﬁcial superintelligence: Acomprehensivesurveyofsuperalignment

Kim,H.,Yi,X.,Yao,J.,Lian,J.,Huang,M.,Duan,S.,Bak,J.,andXie,X. Theroadtoartiﬁcial superintelligence: Acomprehensivesurveyofsuperalignment. arXivpreprintarXiv:2412.16468,

arXiv
[2025]

Retool: Reinforcementlearningforstrategictooluseinllms

24 R2AI:TowardsResistantandResilientAIinanEvolvingWorld Feng,J.,Huang,S.,Qu,X.,Zhang,G.,Qin,Y.,Zhong,B.,Jiang,C.,Chi,J.,andZhong,W. Retool: Reinforcementlearningforstrategictooluseinllms. CoRR,abs/2504.11536,2025. Flajnik,M. F.andKasahara,M. Originandevolutionoftheadaptiveimmu...

2025 arXiv

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.

[1] [1]

V.,Precup,D.,vanHasselt,H

Abel,D.,Barreto,A.,Roy,B. V.,Precup,D.,vanHasselt,H. P.,andSingh,S. Adeﬁnitionofcontinual reinforcementlearning. In NeurIPS,2023. Alon,G.andKamfonas,M. Detectinglanguagemodelattackswithperplexity. CoRR,abs/2308.14132,

arXiv 2023

[2] [7]

Kirichenko,P.,Izmailov,P.,andWilson,A. G. Lastlayerre-trainingissuﬃcientforrobustnessto spuriouscorrelations. In ICLR.OpenReview.net,2023. Kitano,H. Biologicalrobustness. NatureReviewsGenetics,5(11):826–837,2004. Korbak,T.,Balesni,M.,Barnes,E.,Bengio,Y.,Benton,J.,Bloom,J.,Chen,M.,Cooney,A.,Dafoe,A., Dragan,A.,et al. Chainofthoughtmonitorability: Anewandfr...

arXiv 2023

[3] [9]

Openproblemsinmechanisticinterpretability

Sharkey,L.,Chughtai,B.,Batson,J.,Lindsey,J.,Wu,J.,Bushnaq,L.,Goldowsky-Dill,N.,Heimersheim, S.,Ortega,A.,Bloom,J.,et al. Openproblemsinmechanisticinterpretability. arXivpreprint arXiv:2501.16496,2025. Shinn,N.,Cassano,F.,Gopinath,A.,Narasimhan,K.,andYao,S. Reﬂexion: languageagentswith verbalreinforcementlearning. In NeurIPS,2023a. Shinn,N.,Cassano,F.,Gopi...

arXiv 2025

[4] [12]

Yang,X.,Deng,G.,Shi,J.,Zhang,T.,andDong,J. S. Enhancingmodeldefenseagainstjailbreakswith proactivesafetyreasoning. CoRR,abs/2501.19180,2025b. Yao,S.,Zhao,J.,Yu,D.,Du,N.,Shafran,I.,Narasimhan,K. R.,andCao,Y. React: Synergizing reasoningandactinginlanguagemodels. In ICLR.OpenReview.net,2023. Yi,S.,Liu,Y.,Sun,Z.,Cong,T.,He,X.,Song,J.,Xu,K.,andLi,Q. Jailbreak...

arXiv 2023

[5] [1997]

Constitutionalai: Harmlessnessfromaifeedback

Bai,Y.,Kadavath,S.,Kundu,S.,Askell,A.,Kernion,J.,Jones,A.,Chen,A.,Goldie,A.,Mirhoseini,A., McKinnon,C.,et al. Constitutionalai: Harmlessnessfromaifeedback. arXivpreprint arXiv:2212.08073,2022. Barlow,J. B.,Rae,W. H.,andPope,A. Low-speedwindtunneltesting. Johnwiley&sons,1999. Barto,A. G. Intrinsicmotivationandreinforcementlearning. In Intrinsicallymotivate...

arXiv 2022

[6] [1998]

Largelanguagemodelsinﬁnance: Asurvey

Li,Y.,Wang,S.,Ding,H.,andChen,H. Largelanguagemodelsinﬁnance: Asurvey. In Proceedingsof thefourthACMinternationalconferenceonAIinﬁnance,pp.374–382,2023. 27 R2AI:TowardsResistantandResilientAIinanEvolvingWorld Lin,S.,Hilton,J.,andEvans,O. Truthfulqa: Measuringhowmodelsmimichumanfalsehoods. arXiv preprintarXiv:2109.07958,2021. Liu,C.,Yuan,Y.,Yin,Y.,Xu,Y.,Xu...

arXiv 2023

[7] [2006]

Saferlhf: Safereinforcement learningfromhumanfeedback

Dai,J.,Pan,X.,Sun,R.,Ji,J.,Xu,X.,Liu,M.,Wang,Y.,andYang,Y. Saferlhf: Safereinforcement learningfromhumanfeedback. arXivpreprintarXiv:2310.12773,2023. Dalrymple,D.,Skalse,J.,Bengio,Y.,Russell,S.,Tegmark,M.,Seshia,S.,Omohundro,S.,Szegedy,C., Goldhaber,B.,Ammann,N.,et al. Towardsguaranteedsafeai: Aframeworkforensuringrobust andreliableaisystems. arXivpreprin...

arXiv 2023

[8] [2013]

S.,Carpenter,S

Walker,B.,Holling,C. S.,Carpenter,S. R.,andKinzig,A. Resilience,adaptabilityandtransformability insocial–ecologicalsystems. Ecologyandsociety,9(2),2004. Wang,H.,Qin,Z.,Zhao,Y.,Du,C.,Lin,M.,Wang,X.,andPang,T. Lifelongsafetyalignmentfor languagemodels. CoRR,abs/2505.20259,2025a. Wang,J.,Pun,A.,Tu,J.,Manivasagam,S.,Sadat,A.,Casas,S.,Ren,M.,andUrtasun,R. Advs...

arXiv 2004