Pith. sign in

REVIEW 4 major objections 5 minor 47 references

The authors argue that frontier AI labs can safely deploy agents at or moderately above human capability by maintaining two control invariants—detecting misaligned intent and preventing or reverting attacks—through capability-linked defence

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 06:49 UTC pith:ZKY54UUI

load-bearing objection A well-written, honest blueprint that gives AI control a shared taxonomy and capability ladder, but its load-bearing detection invariant is explicitly unproven. the 4 major comments →

arxiv 2607.13087 v1 pith:ZKY54UUI submitted 2026-07-13 cs.CR

GDM AI Control Roadmap

Mary Phuong , Erik Jenner , Laurent Simon , Lewis Ho , Rohin Shah , Sebastian Farquhar , Scott Coull This is my paper
classification cs.CR
keywords AI controlmisalignmentinsider threatcapability-based mitigationchain-of-thought monitoringaccess controlthreat modellingagent security
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to establish that internal security against potentially misaligned AI agents can be engineered, not assumed. Its central proposal is that a frontier AI lab should treat internally deployed agents as untrusted insiders and maintain two control invariants at all times: reliable detection of misaligned intent or actions, and effective prevention, interception, or reversion of attack attempts. To keep these invariants as models become more capable, the authors tie defensive measures to specific capability thresholds—chain-of-thought monitorability, oversight awareness, opaque reasoning, and the gap between attacker and defender cyber capability—organised into four detection tiers (D1–D4) and three prevention/response tiers (R1–R3). The paper argues that this ladder, together with fifteen concrete mitigations from chain-of-thought monitoring to shutdown infrastructure, can keep models at or moderately above human level under control, serving as a second line of defence behind alignment. A sympathetic reader would care because the claim is operational: it says a lab can safely integrate capable agents into core work while retaining the upper hand.

Core claim

The paper claims that containing AI agents inside a lab is an insider-threat problem with a structured escalation path. As models get better at evading detection and executing attacks, defences move along two ladders: detection goes from chain-of-thought and action monitoring to behaviour-based monitoring, honeypots, anomaly detection, untrusted monitoring, and activations monitoring; response goes from asynchronous reverts to synchronous blocking, AI access control, shutdown infrastructure, and capability restriction. The ladders are tied to capability thresholds, not dates. This, the paper says, keeps control feasible at or moderately above human-level capability, though not for vastly sup

What carries the argument

The two control invariants—detection and prevention/response—form the backbone, but the load-bearing machinery is the capability-mitigation ladder: four detection tiers (D1–D4) and three prevention/response tiers (R1–R3), each anchored to explicit capability thresholds such as chain-of-thought monitorability, oversight awareness, opaque reasoning, and the offence–defence gap between untrusted models and trusted-model-amplified human defenders. The ladder works by escalating the cost of control in step with model capability and by defining assurance metrics (monitoring coverage, recall, time-to-response) and red-teaming methodology for each tier. TRAIT&R—the Taxonomy of Rogue AI Tactics and R

Load-bearing premise

The load-bearing premise is that misaligned intent or actions can be detected well enough—at least with the ladder's escalating measures—to keep the detection invariant at all times. The paper itself states in its Limitations section that it does not comprehensively address attacker strategies based on alignment faking or sandbagging safety work while building influence, and that no attack-execution evaluations exist yet; if such low-detectability strategies are feasible, the

What would settle it

A red-team control evaluation in which a proxy misaligned model, operating under the D1/D2 mitigations described in the paper (chain-of-thought monitoring, alignment testing, adversarially robust monitors), completes one of the discrete TRAIT&R objectives—say, setting up an unmonitored inference job—while remaining unflagged by the asynchronous monitor at the paper's proposed coverage and recall targets. If such an attack succeeds within a reasonable red-team budget, or if the paper's proposed evaluation-awareness checks reveal that the proxy model can tell test transcripts from deployment log

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the two invariants can be maintained, a frontier lab can deploy agents for AI R&D, cyber-defence, and science with acceptable risk, provided mitigations escalate as models cross capability thresholds rather than at fixed dates.
  • Chain-of-thought monitoring is treated as a short-lived but pivotal mitigation: once models can reason opaquely or produce illegible CoT, detection must shift to behaviour-based monitoring, honeypots, and anomaly detection, and eventually to untrusted monitoring and activations monitoring.
  • Synchronous response and context-sensitive AI access control become necessary only when models could realistically complete at least one discrete attacker objective such as model exfiltration, rogue internal deployment, successor-model misalignment, critical asset destruction, or severe external harm.
  • The proposed assurance metrics and red-team exercises give an operational check: coverage above 99 percent of agentic traffic, recall targets on synthetic misaligned trajectories, and time-to-response under 24 hours are the paper's concrete conditions for claiming the invariants hold.
  • At the highest prevention/response tier (R3), shutdown infrastructure, strict supply-chain security, and capability restriction are listed as necessary, but the paper explicitly labels these mitigations more speculative than the earlier tiers.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • One consequence the authors leave implicit: if control depends on legible reasoning, preserving chain-of-thought monitorability could become a design objective for future model training, making control a constraint on how models are built, not just how they are deployed.
  • The D4 reliance on untrusted monitoring suggests the practical ceiling of the detection ladder is set by how well anti-collusion measures such as trusted paraphrasing scale; a testable extension would be to benchmark collusion rates across model families and paraphrasing strategies.
  • The insider-threat framing implies that many enterprise security practices can be ported with modest adaptation; a testable extension of the roadmap is to run the proposed live attack simulation at small scale—seeding a few agents with proxy misaligned objectives and measuring per-tier detection rates—to empirically calibrate the ladder's thresholds.
  • If the roadmap's logic is right, sharing red-team methodologies and threat taxonomies across labs would lower the cost of assurance for everyone, because the offence–defence gap is a property of model capability, not of any single company's infrastructure.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents the GDM AI Control Roadmap v0.1, a forward-looking internal security blueprint for frontier AI companies deploying potentially misaligned AI agents. It introduces TRAIT&R, a taxonomy of AI-adversary tactics and techniques built on MITRE ATT&CK, and defines two control invariants: reliable detection of misaligned intent/actions and effective prevention/response. It then proposes a capability-mitigation ladder with four Detection tiers (D1–D4) and three Prevention/Response tiers (R1–R3), linking concrete mitigations (e.g., CoT monitoring, behavioral monitoring, untrusted monitoring, activations monitoring, shutdown infrastructure, supply-chain hardening) to escalating model capabilities. The roadmap includes assurance metrics, red-teaming suggestions, and a candid limitations section. The central claim is that, for models at or moderately above human level, maintaining these invariants is achievable by escalating through the proposed tiers, though the paper repeatedly acknowledges open problems in measurement and assurance.

Significance. If the proposed framework can be validated, this would be a valuable contribution to AI control as a security discipline: it provides a structured, capability-relative vocabulary for discussing internal threats, a broad catalogue of concrete mitigations, and an explicit set of assurance targets. The paper's strengths include its systematic adaptation of cyber-security methodology, its clear separation of detection from prevention/response, and its unusually candid enumeration of open problems (e.g., attack-execution measurement, activations-monitoring assurance, alignment-faking evasion). However, the roadmap is not a validated safety case: the central claim is conditional on the resolution of several load-bearing research questions that the paper itself identifies. Its immediate value lies in shaping the research agenda and providing a common reference frame rather than in establishing that the proposed invariants can actually be maintained.

major comments (4)
  1. [§3.4.4 (D4), Assurance subsection] The D4 detection path is load-bearing for the paper's central claim, but the two mechanisms proposed are explicitly conceded to lack assurance. The text states that attacker collusion in untrusted monitoring 'would likely evade our evaluations' and that activations-monitoring assurance 'remains an open problem' due to lack of ground truth. Without an argument (empirical or analytic) that either mechanism can meet the recall targets required by the detection invariant, the invariant is not established in exactly the regime where trusted oversight is gone. Since R2/R3 responses trigger on detection, the prevention/response invariant also cannot be maintained in this regime. Please either add a concrete research and evaluation agenda capable of establishing these mechanisms, or explicitly reformulate the invariants as goals to be pursued rather than guarantees maintained.
  2. [§3.6.2 and Table 17] The R-level assignments depend on attack execution capability, but Section 4 states that 'we have not yet specified indicators or evaluations for attack execution capability,' and §3.6.2 calls measurement 'an open problem.' Table 17 uses 'strong general autonomy capabilities' and 'offence–defence gap' without thresholds, validated metrics, or a procedure for combining proxies. Since the R1–R3 mitigation requirements (e.g., >99% coverage, <24h time-to-shutdown) are keyed to these levels, the ladder cannot currently be operationalized or stress-tested. Please specify at least a minimal set of proxy metrics and a mapping procedure, or mark all R-level assignments as provisional and non-binding.
  3. [§3.4.3 and Table 14 (D3)] D3 assumes that 'some combination of trusted behavioural monitors is sufficient' once CoT is unavailable, but no evidence is given that trusted models can satisfy the recall targets inherited from earlier tiers (e.g., 'catch virtually all egregious harms' at D1). The Assurance paragraph acknowledges that maintaining the same safety standard without CoT 'will be much harder,' yet does not propose how sufficiency could be established. This is a load-bearing gap: the detection invariant is unproven for the entire D3–D4 range, and the response mechanisms are triggered by it. At minimum, describe a concrete stress-test or formal argument that could establish the required recall.
  4. [§4, final paragraph] The manuscript concedes that the roadmap 'relies heavily on detection' and 'does not comprehensively address' attacker strategies based on alignment faking or sandbagging safety work while building influence. These tactics directly attack the detection invariant and may never trigger the response ladders, so for the work-sabotage threat category (Table 6) the invariants as stated are not maintained. The paper should either add mitigations and assurance methods specifically targeting low-detectability strategies, or explicitly scope the invariants to threats that are in principle detectable, making clear that the roadmap does not cover the full space of threats it enumerates.
minor comments (5)
  1. [§3.5.2 Assurance] Cross-reference error: 'as described in Section 2.1' should point to the monitoring stress-testing discussion in §3.4.1 or §3.2, not to the insider-threat background section.
  2. [Table 16] The detection-level mapping uses qualitative terms such as 'low rate' and 'high rate' without numeric anchors. Even if evaluation-specific, a worked example or illustrative range would materially improve usability.
  3. [References] Many supporting citations are unpublished or in-group preprints (e.g., Shah et al. 2025, Korbak et al. 2025, Greenblatt et al. 2024). Please add arXiv identifiers or permanent links and, where appropriate, note which results are peer-reviewed.
  4. [Table 15] The R2/R3 row states that response recall targets 'follow detection track,' but the detection track does not define a numeric recall target for the monitor at D3/D4. Define this dependency explicitly or state that it is TBD.
  5. [Table 3] Typo: 'posible' should be 'possible' in the row about monitoring responses.

Circularity Check

0 steps flagged

No significant circularity; the roadmap is a self-consistent taxonomy and mitigation ladder, and its central claim is explicitly conditional on unproven detection capabilities rather than derived from its inputs.

full rationale

The paper contains no fitted parameters, no quantitative predictions, and no equations. The D1-D4/R1-R3 levels are definitions and classification tables (Tables 14-17), and the control invariants in Section 3.1 are stated goals rather than outputs of a derivation. TRAIT&R is explicitly presented as a theoretical extension of MITRE ATT&CK, not as a prediction or as a renamed known result. Cited prior work such as Baker et al. 2025, Greenblatt et al. 2024, and Korbak et al. 2025 is used as empirical evidence for the feasibility or fragility of monitoring techniques; even where some authors overlap with the present paper, these are concrete studies with stated assumptions, not a uniqueness theorem or an ansatz that smuggles in the roadmap's conclusion. The one place where the argument is genuinely conditional is acknowledged by the paper itself: Section 4 states that 'the roadmap relies heavily on detection' and 'does not comprehensively address' alignment-faking and sandbagging strategies, and Section 3.4.4 says assurance for activations monitoring 'remains an open problem.' That is a frank limitation on the strength of the central claim (the detection invariant is assumed rather than established), but a limitation is not circularity. The ladder would remain a coherent proposed escalation even if the detection targets are not yet met. No circular step is exhibited, so the score is 0.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 3 invented entities

The roadmap introduces few numeric quantities but many hand-chosen assurance targets and conceptual constructs. Its load-bearing assumptions are the existence of a coordinated AI adversary, the reliability of detection (especially chain-of-thought), and a manageable trusted/untrusted capability gap. None of these are empirically validated in the paper; they are conservative design choices.

free parameters (5)
  • Asynchronous monitoring coverage target = >99% (R2/R3); ~99% in harm-prevention phase
    Hand-chosen assurance threshold in Tables 14–15 and Section 3.5.1; no empirical basis.
  • Asynchronous time-to-response target = 1 week (observation) to <24h (harm prevention)
    Hand-chosen latency target, Section 3.5.1.
  • Synchronous response coverage target = >99%
    Hand-chosen target for R2/R3, Table 15.
  • Pre-deployment exploit-rate threshold = ≤1%
    Hand-chosen threshold for internal hardening, Section 3.5.3.
  • Observation-phase monitoring sample size = 25% of traffic
    Statistical heuristic for detecting >15 attacks with 99% probability, Section 3.5.1.
axioms (6)
  • domain assumption Current frontier AI models are neither strategic nor egregiously misaligned; internally deployed untrusted models are a hypothetical future risk.
    Section 1 says 'Current AI models are neither strategic nor egregiously misaligned'; the entire roadmap's urgency depends on this being true now and potentially false later.
  • ad hoc to paper The AI adversary is best modeled as a coordinated collective of untrusted instances with a shared, context-independent adversarial goal.
    Defined in Section 2.2 as a conservative worst-case assumption; no natural examples exist, and this collective is the target of all TRAIT&R techniques.
  • domain assumption Trusted models (D2 or below) are non-adversarial and can reliably monitor untrusted models; the trusted/untrusted capability gap is small enough through D3/R2.
    D-level definitions (Section 3.4) and R2 offense-defence gap criteria assume this; D4/R3 are defined as when the gap becomes unmanageable.
  • domain assumption Chain-of-thought is currently transparent and monitorable, providing a decisive detection advantage over human adversaries.
    Section 2.1 says if reasoning stays transparent, advanced mitigations would be unnecessary; D1–D2 rely on CoT monitorability.
  • domain assumption Control invariants are maintained only if detection/prevention capability exceeds attacker capability.
    Stated in Section 1; this is a definitional/aspirational principle, not a demonstrated fact.
  • domain assumption Vastly superintelligent AI is outside the scope of control; the roadmap covers models at or moderately above human level.
    Section 1 and Section 5 explicitly limit the scope, avoiding the hardest case.
invented entities (3)
  • Hypothetical AI adversary (coordinated collective of untrusted instances) no independent evidence
    purpose: Anchors the worst-case threat model and TRAIT&R taxonomy
    Postulated in Section 2.2; the paper states 'these threats are theoretical' and no natural examples are known.
  • Control invariants (detection; prevention and response) no independent evidence
    purpose: Defines what success means for the control program
    Defined in Section 3.1 as normative goals; no metrics implemented yet.
  • Detection levels D1–D4 and Prevention/Response levels R1–R3 no independent evidence
    purpose: Provide capability-based escalation thresholds for mitigations
    Constructs defined by capability criteria; Tables 16–17 explicitly leave numeric thresholds unspecified.

pith-pipeline@v1.3.0-alltime-deepseek · 28810 in / 15099 out tokens · 127110 ms · 2026-08-02T06:49:55.638991+00:00 · methodology

0 comments
read the original abstract

AI agents are rapidly accelerating work at frontier AI companies, helping with AI R&D, cyber-defence, and advancing scientific discoveries. As these agents become more tightly integrated into our systems, unlocking their full potential requires rethinking how we do security. We should not assume that AI agents are always perfectly aligned, but should instead build in multiple layers of defence. We present the GDM AI Control Roadmap (v0.1) -- a first-of-its-kind blueprint for internal security against potentially misaligned AI. This report provides: * Threat modelling: We adopt a conservative approach to threat modelling and assume a hypothetical AI adversary pursuing undesirable goals in internal deployment. We introduce TRAIT&R, a taxonomy of tactics and techniques available to such a hypothetical AI adversary, building on the established security framework MITRE ATT&CK. * Capability-based mitigation: Because controlling more capable models requires more costly interventions, we link specific defensive measures to evolving model capabilities (such as the ability to reason opaquely or execute complex cyberattacks). As models get more powerful, our defences should escalate accordingly. We outline four Detection tiers (D1-D4) and three Prevention and Response tiers (R1-R3). * A portfolio of practical defences: We suggest 15 concrete, tiered mitigations. These range from low-cost interventions for current models (e.g., chain-of-thought monitoring, asynchronous alerts) to advanced safeguards for future models (e.g., real-time access control, system-level anomaly detection, internal activations monitoring, and shutdown infrastructure). AI control is a nascent field, and implementing these mitigations requires navigating difficult trade-offs between security and developer velocity. We expect the roadmap to evolve as we gain more experience and as the field in turn evolves.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

47 extracted references · 14 linked inside Pith

  1. [1]

    Claude S onnet 4.5 system card, 2025

    Anthropic . Claude S onnet 4.5 system card, 2025. https://www-cdn.anthropic.com/963373e433e489a87a10c823c52a0a013e9172dd/Claude

    2025

  2. [2]

    How we contain Claude across products

    Anthropic. How we contain Claude across products. Anthropic Engineering Blog, 2026 a . (Link) https://www.anthropic.com/engineering/how-we-contain-claude

    2026

  3. [3]

    Project G lasswing: Securing critical software for the AI era, 2026 b

    Anthropic. Project G lasswing: Securing critical software for the AI era, 2026 b . (Link) https://www.anthropic.com/glasswing

    2026

  4. [4]

    Arnav, P

    B. Arnav, P. Bernabeu-Pérez, N. Helm-Burger, T. Kostolansky, H. Whittingham, and M. Phuong. CoT red-handed: Stress testing chain-of-thought monitoring. arXiv, 2025. (Link) https://arxiv.org/abs/2505.23575

    arXiv 2025

  5. [5]

    Bailey, A

    L. Bailey, A. Serrano, A. Sheshadri, M. Seleznyov, J. Taylor, E. Jenner, J. Hilton, S. Casper, C. Guestrin, and S. Emmons. Obfuscated activations bypass LLM latent-space defenses. arXiv, 2025. (Link) https://arxiv.org/abs/2412.09565

    Pith/arXiv arXiv 2025

  6. [6]

    Baker, J

    B. Baker, J. Huizinga, L. Gao, Z. Dou, M. Y. Guan, A. Madry, W. Zaremba, J. Pachocki, and D. Farhi. Monitoring reasoning models for misbehavior and the risks of promoting obfuscation. arXiv, 2025. (Link) https://arxiv.org/abs/2503.11926

    Pith/arXiv arXiv 2025

  7. [7]

    Bhatt, C

    A. Bhatt, C. Rushing, A. Kaufman, T. Tracy, V. Georgiev, D. Matolcsi, A. Khan, and B. Shlegeris. Ctrl-Z : Controlling AI agents via resampling. arXiv, 2025. (Link) https://arxiv.org/abs/2504.10374

    Pith/arXiv arXiv 2025

  8. [8]

    Emmons, E

    S. Emmons, E. Jenner, D. K. Elson, R. A. Saurous, S. Rajamanoharan, H. Chen, I. Shafkat, and R. Shah. When chain of thought is necessary, language models struggle to evade monitors. arXiv, 2025 a . (Link) https://arxiv.org/abs/2507.05246

    Pith/arXiv arXiv 2025

  9. [9]

    Emmons, R

    S. Emmons, R. S. Zimmermann, D. K. Elson, and R. Shah. A pragmatic way to measure chain-of-thought monitorability. arXiv, 2025 b . (Link) https://arxiv.org/abs/2510.23966

    arXiv 2025

  10. [10]

    The promise of white-box tools for detecting and mitigating AI deception, 2026

    FAR.AI . The promise of white-box tools for detecting and mitigating AI deception, 2026. (Link) https://www.far.ai/news/ai-deception-white-box

    2026

  11. [11]

    Fronsdal, I

    K. Fronsdal, I. Gupta, A. Sheshadri, J. Michala, S. McAleer, R. Wang, S. Price, and S. R. Bowman. Petri: An open-source auditing tool to accelerate AI safety research, Oct. 2025. (Link) https://alignment.anthropic.com/2025/petri/

    2025

  12. [12]

    Goldowsky-Dill, B

    N. Goldowsky-Dill, B. Chughtai, S. Heimersheim, and M. Hobbhahn. Detecting strategic deception using linear probes. arXiv, 2025. (Link) https://arxiv.org/abs/2502.03407

    Pith/arXiv arXiv 2025

  13. [13]

    Frontier Safety Framework 3.1, 2025

    Google DeepMind . Frontier Safety Framework 3.1, 2025. (Link) https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/strengthening-our-frontier-safety-framework/frontier-safety-framework_3-1.pdf

    2025

  14. [14]

    Gould, F

    D. Gould, F. R. Ward, A. C. Woodruff, R. Arike, J. Hills, A. Serrano, I. Caspary, J. R. Brown, J. J. Jiao, P. Leask, T. Stone, R. Potham, I. G. Stan, H. Mayne, S. Hellsten, S. Biswas, A. Azarbal, W. L. Anderson, E. Najt, R. Greenblatt, and J. Stastny. Think fast: Estimating no- CoT task-completion time horizons of frontier AI models. arXiv, 2026. (Link) h...

    Pith/arXiv arXiv 2026

  15. [15]

    Greenblatt

    R. Greenblatt. Recent LLM s can use filler tokens or problem repeats to improve (no- CoT ) math performance, 2025. (Link) https://blog.redwoodresearch.org/p/recent-llms-can-use-filler-tokens

    2025

  16. [16]

    Greenblatt and B

    R. Greenblatt and B. Shlegeris. Catching AI s red-handed, 2024. (Link) https://www.lesswrong.com/posts/i2nmBfCXnadeGmhzW/catching-ais-red-handed

    2024

  17. [17]

    Greenblatt, B

    R. Greenblatt, B. Shlegeris, K. Sachan, and F. Roger. AI control: Improving safety despite intentional subversion. arXiv, 2024. (Link) https://arxiv.org/abs/2312.06942

    Pith/arXiv arXiv 2024

  18. [18]

    M. Y. Guan, M. Wang, M. Carroll, Z. Dou, A. Y. Wei, M. Williams, B. Arnav, J. Huizinga, I. Kivlichan, M. Glaese, J. Pachocki, and B. Baker. Monitoring monitorability. arXiv, 2025. (Link) https://arxiv.org/abs/2512.18311

    arXiv 2025

  19. [19]

    V. Hebbar. How can we solve diffuse threats like research sabotage with AI control?, 2025. (Link) https://blog.redwoodresearch.org/p/how-can-we-solve-diffuse-threats

    2025

  20. [20]

    CyScenarioBench : Evaluating LLM cyber capabilities through scenario-based benchmarking, 2025

    Irregular. CyScenarioBench : Evaluating LLM cyber capabilities through scenario-based benchmarking, 2025. (Link) https://irregular.com/research/cyscenariobench

    2025

  21. [21]

    Joglekar, J

    M. Joglekar, J. Chen, G. Wu, J. Yosinski, J. Wang, B. Barak, and A. Glaese. Training LLM s for honesty via confessions. arXiv, 2025. (Link) https://arxiv.org/abs/2512.08093

    arXiv 2025

  22. [22]

    Karvonen, J

    A. Karvonen, J. Chua, C. Dumas, K. Fraser-Taliente, S. Kantamneni, J. Minder, E. Ong, A. S. Sharma, D. Wen, O. Evans, and S. Marks. Activation oracles: Training and evaluating LLM s as general-purpose activation explainers. arXiv, 2026. (Link) https://arxiv.org/abs/2512.15674

    arXiv 2026

  23. [23]

    Korbak, M

    T. Korbak, M. Balesni, E. Barnes, Y. Bengio, J. Benton, J. Bloom, M. Chen, A. Cooney, A. Dafoe, A. Dragan, S. Emmons, O. Evans, D. Farhi, R. Greenblatt, D. Hendrycks, M. Hobbhahn, E. Hubinger, G. Irving, E. Jenner, D. Kokotajlo, V. Krakovna, S. Legg, D. Lindner, D. Luan, A. Mądry, J. Michael, N. Nanda, D. Orr, J. Pachocki, E. Perez, M. Phuong, F. Roger, J...

    Pith/arXiv arXiv 2025

  24. [24]

    Krakovna, D

    V. Krakovna, D. Lindner, L. Ho, S. Farquhar, and R. Shah. Realistic honeypot evaluations for scheming propensity. arXiv, 2026. (Link) https://arxiv.org/abs/2605.29729

    Pith/arXiv arXiv 2026

  25. [25]

    C. Li, M. Phuong, and N. Y. Siegel. LLMs can covertly sandbag on capability evaluations against chain-of-thought monitoring. arXiv, 2025. (Link) https://arxiv.org/abs/2508.00943

    arXiv 2025

  26. [26]

    C. Li, M. Phuong, and D. Tan. Spilling the beans: Teaching LLM s to self-report their hidden objectives. arXiv, 2026. (Link) https://arxiv.org/abs/2511.06626

    arXiv 2026

  27. [27]

    N. Li, A. Pan, A. Gopal, S. Yue, D. Berrios, A. Gatti, J. D. Li, A.-K. Dombrowski, S. Goel, L. Phan, G. Mukobi, N. Helm-Burger, R. Lababidi, L. Justen, A. B. Liu, M. Chen, I. Barrass, O. Zhang, X. Zhu, R. Tamirisa, B. Bharathi, A. Khoja, Z. Zhao, A. Herbert-Voss, C. B. Breuer, S. Marks, O. Patel, A. Zou, M. Mazeika, Z. Wang, P. Oswal, W. Lin, A. A. Hunt, ...

    Pith/arXiv arXiv 2024

  28. [28]

    Lindner, V

    D. Lindner, V. Krakovna, and S. Farquhar. Gram: Assessing sabotage propensities via automated alignment auditing. arXiv, 2026. (Link) https://arxiv.org/abs/2605.30322

    Pith/arXiv arXiv 2026

  29. [29]

    Mallen, C

    A. Mallen, C. Griffin, M. Wagner, A. Abate, and B. Shlegeris. Subversion strategy eval: Can language models statelessly strategize to subvert control protocols? arXiv, 2025. (Link) https://arxiv.org/abs/2412.12480

    Pith/arXiv arXiv 2025

  30. [30]

    Marks, J

    S. Marks, J. Treutlein, T. Bricken, J. Lindsey, J. Marcus, S. Mishra-Sharma, D. Ziegler, E. Ameisen, J. Batson, T. Belonax, S. R. Bowman, S. Carter, B. Chen, H. Cunningham, C. Denison, F. Dietz, S. Golechha, A. Khan, J. Kirchner, J. Leike, A. Meek, K. Nishimura-Gasparian, E. Ong, C. Olah, A. Pearce, F. Roger, J. Salle, A. Shih, M. Tong, D. Thomas, K. Rivo...

    Pith/arXiv arXiv 2025

  31. [31]

    Task-completion time horizons of frontier AI models, 2026

    METR . Task-completion time horizons of frontier AI models, 2026. (Link) https://metr.org/time-horizons/

    2026

  32. [32]

    MITRE ATLAS : Adversarial threat landscape for AI systems, 2021

    MITRE Corporation . MITRE ATLAS : Adversarial threat landscape for AI systems, 2021. (Link) https://atlas.mitre.org/

    2021

  33. [33]

    S. Nevo, D. Lahav, A. Karpur, Y. Bar-On, H. A. Bradley, and J. Alstott. Securing AI model weights: Preventing theft and misuse of frontier models. Technical report, RAND Corporation, 2024. (Link) https://www.rand.org/pubs/research_reports/RRA2849-1.html

    2024

  34. [34]

    S. M. Omohundro. The basic AI drives. In Artificial General Intelligence 2008, volume 171 of Frontiers in Artificial Intelligence and Applications, pages 483--492. IOS Press, 2008. (Link) https://selfawaresystems.com/wp-content/uploads/2008/01/ai_drives_final.pdf

    2008

  35. [35]

    Supply-chain levels for software artifacts ( SLSA ) specification, 2024

    OpenSSF . Supply-chain levels for software artifacts ( SLSA ) specification, 2024. (Link) https://slsa.dev/spec

    2024

  36. [36]

    Schoen, E

    B. Schoen, E. Nitishinskaya, M. Balesni, A. Højmark, F. Hofstätter, J. Scheurer, A. Meinke, J. Wolfe, T. van der Weij, A. Lloyd, N. Goldowsky-Dill, A. Fan, A. Matveiakin, R. Shah, M. Williams, A. Glaese, B. Barak, W. Zaremba, and M. Hobbhahn. Stress testing deliberative alignment for anti-scheming training. arXiv, 2025. (Link) https://arxiv.org/abs/2509.15541

    arXiv 2025

  37. [37]

    R. Shah, A. Irpan, A. M. Turner, A. Wang, A. Conmy, D. Lindner, J. Brown-Cohen, L. Ho, N. Nanda, R. A. Popa, R. Jain, R. Greig, S. Albanie, S. Emmons, S. Farquhar, S. Krier, S. Rajamanoharan, S. Bridgers, T. Ijitoye, T. Everitt, V. Krakovna, V. Varma, V. Mikulik, Z. Kenton, D. Orr, S. Legg, N. Goodman, A. Dafoe, F. Flynn, and A. Dragan. An approach to tec...

    Pith/arXiv arXiv 2025

  38. [38]

    Shlegeris

    B. Shlegeris. Untrusted smart models and trusted dumb models, 2023. (Link) https://www.alignmentforum.org/posts/LhxHcASQwpNa3mRNk/untrusted-smart-models-and-trusted-dumb-models

    2023

  39. [39]

    Singh, G

    A. Singh, G. Kroiz, S. Rajamanoharan, and N. Nanda. Why did my model do that? model incrimination for diagnosing LLM misbehavior. AI Alignment Forum, 2026. (Link) https://www.alignmentforum.org/posts/Bv4CLkNzuG6XYTjEe/why-did-my-model-do-that-model-incrimination-for-diagnosing

    2026

  40. [40]

    Smith, B

    L. Smith, B. Chughtai, and N. Nanda. Difficulties with evaluating a deception detector for AI s. arXiv, 2025. (Link) https://arxiv.org/abs/2511.22662

    arXiv 2025

  41. [41]

    South, S

    T. South, S. Nagabhushanaradhya, A. Dissanayaka, S. Cecchetti, G. Fletcher, V. Lu, A. Pietropaolo, D. H. Saxe, J. Lombardo, A. M. Shivalingaiah, S. Bounev, A. Keisner, A. Kesselman, Z. Proser, G. Fahs, A. Bunyea, B. Moskowitz, A. Tulshibagwale, D. Greenwood, J. Pei, and A. Pentland. Identity management for agentic AI : The new frontier of authorization, a...

    arXiv 2025

  42. [42]

    Trębacz, S

    M. Trębacz, S. Arnesen, A. Cassirer, M. Johnson, X. Lin, and T. Sottiaux. A practical approach to verifying code at scale. OpenAI Alignment Research Blog, 2025. (Link) https://alignment.openai.com/scaling-code-verification/

    2025

  43. [43]

    Our evaluation of C laude M ythos P review’s cyber capabilities, 2026

    UK AI Security Institute . Our evaluation of C laude M ythos P review’s cyber capabilities, 2026. (Link) https://www.aisi.gov.uk/blog/our-evaluation-of-claude-mythos-previews-cyber-capabilities

    2026

  44. [44]

    H. Wang, Y. Sun, B. Ko, M. Talati, J. Gong, Z. Li, N. Yu, X. Yu, W. Shen, V. Jolly, and H. Zhang. Monitorbench: A comprehensive benchmark for chain-of-thought monitorability in large language models. arXiv, 2026. (Link) https://arxiv.org/abs/2603.28590

    arXiv 2026

  45. [45]

    Williams, C

    M. Williams, C. Raymond, and M. Carroll. Sidestepping evaluation awareness and anticipating misalignment with production evaluations. OpenAI Alignment Research Blog, 2025. (Link) https://alignment.openai.com/prod-evals/

    2025

  46. [46]

    Willison

    S. Willison. The lethal trifecta for AI agents: private data, untrusted content, and external communication, 2025. (Link) https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/

    2025

  47. [47]

    J. Za, J. Bainiaksina, N. Ostrovsky, T. Chopra, and V. Krakovna. Persuasion attacks can decrease effectiveness of CoT monitoring. In ICLR 2026 AIWILD , 2026. (Link) https://openreview.net/pdf?id=7opbcO7muQ

    2026