Pith. sign in

REVIEW 3 major objections 6 minor 36 references

AgentSnare dynamically unfolds a consistent decoy that steers LLM penetration agents away from real targets, with no verified exploit across 45 attacker–CVE pairs.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · grok-4.5

2026-07-30 14:27 UTC pith:IZ7JTKT5

load-bearing objection Strong empirical systems paper: learned, globally consistent decoys beat static lures and shell honeypots on CVE-Bench, with a real but overstated entry-surface caveat. the 3 major comments →

arxiv 2607.26998 v1 pith:IZ7JTKT5 submitted 2026-07-29 cs.CR cs.CLcs.LG

AgentSnare: Learning to Delay, Divert, and Defuse Autonomous Penetration Agents

classification cs.CR cs.CLcs.LG
keywords LLM agentspenetration testingdeceptionhoneypotstrajectory-adaptive defenseCVE-Benchartifact constructioncybersecurity
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Autonomous penetration agents choose their next move from tool observations, so defenders can try to mislead them. The paper shows that static lures planted in advance lose grip over multi-step runs: modern agents spot inconsistencies and return to the real target. AgentSnare instead builds the decoy along the attack path. A learned policy proposes fake observations from the agent’s history and current decoy state; a validation layer only commits facts that stay consistent, reachable, and safe. The intended effect is threefold: absorb tool calls (delay), keep the agent inside the decoy after first entry (divert), and make completion reports rest on decoy evidence (defuse). On 15 deployable web CVEs and three attacker models, it absorbs 46.8% of tool calls, retains 55.9% of post-entry actions in the decoy, grounds 90% of completion attempts in decoy evidence, and records no successful real-target exploit at pass@3. A sympathetic reader cares because attack agents are becoming practical and fixed honeytokens alone do not hold them.

Core claim

Trajectory-adaptive construction of a globally consistent decoy—candidate artifacts proposed from interaction history and decoy state, then validated and incrementally committed—can steer autonomous penetration agents strongly enough that, across 15 CVE-Bench web applications and three attacker models, no real target is successfully exploited at pass@3, while the decoy absorbs 46.8% of tool calls, retains 55.9% of post-entry actions, and supports 90.0% of completion attempts.

What carries the argument

The online artifact-construction policy plus ShadowOS: the policy emits a proposed observation and fact increment conditioned on history, action, world ledger, and affordance frontier; ShadowOS checks semantic, state, and safety validity and commits only valid increments into a persistent decoy. Offline, reaction-guided dual-role simulation selects artifacts that keep a simulated attacker probing the decoy, then fine-tunes a small open model on those decisions.

Load-bearing premise

That artifacts chosen because they keep a simulated attacker inside the decoy will also keep real, held-out attacker models there when those attackers can ignore fakes and freely return to the real target.

What would settle it

Re-run the 15 CVE-Bench tasks with the three attacker models under pass@3 with AgentSnare on the interaction path; any official evaluator-verified real-target exploit, or a collapse of Delay, post-entry retention, and decoy-grounded completion toward static-defense or simple honeypot levels with exploits returning, would refute the central claim.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Static pre-planted deception is not enough against multi-step LLM agents; defenses need artifacts built from the live attack trajectory.
  • Global factual consistency along the interaction path matters more for steering than locally realistic honeypot replies alone.
  • Delay, post-entry retention, and decoy-grounded completion give a process readout of whether an interactive defense is actually steering the run.
  • A compact fine-tuned policy plus stateful validation can beat a stronger prompt-only generator on both trajectory metrics and exploit suppression.
  • Defenders who control observation surfaces can soak up a large share of an agent’s tool budget without changing the real application’s vulnerable path.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same history-conditioned decoy loop may apply to other observation-driven agents (ops bots, web automators) wherever defenders own part of the environment interface.
  • Attackers that systematically cross-check decoy consistency or dual-home real and fake surfaces in parallel could stress the transfer assumption harder than the evaluated agents.
  • If simulator–real reaction mismatch is the main failure mode, multi-model reaction ensembles or light online policy updates are the natural next stress tests.
  • DDD-style process metrics could become standard reporting for interactive cyber defenses alongside final exploit success.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes AgentSnare, a trajectory-adaptive deception system that steers LLM-based penetration agents away from real targets by dynamically constructing and validating decoy artifacts conditioned on interaction history and decoy state. A LoRA-tuned Qwen3-4B policy proposes candidate observations and supporting facts; ShadowOS (world-fact ledger Wt, affordance graph Gt, frontier Ft) validates semantic/state/safety consistency and commits only admissible artifacts, yielding a globally consistent decoy. Offline dual-role simulation with reaction-guided selection supplies 24,807 supervised decisions (no CVE-Bench content). Evaluation on 15 CVE-Bench apps × three attackers × three reps introduces Delay–Divert–Defuse (DDD) metrics and reports 46.8% Delay, 55.9% PER, 90.0% SubDefuse, and 0/45 verified real-target exploits at pass@3, outperforming joint static interventions, Cowrie, and HoneyLLMd, with ablations isolating fine-tuning and ShadowOS.

Significance. If the results hold under the stated threat model, this is a substantial contribution to defenses against autonomous LLM attackers: it reframes agent-oriented deception from static pre-planted artifacts to online, consistency-preserving trajectory steering, and supplies a process-level evaluation suite (DDD) that goes beyond binary exploit success. Strengths include a full 405-run defended design with shared budgets/tools, strong interactive baselines, training data explicitly disjoint from the evaluation suite, and ablations that separate the learned policy from the stateful runtime—including a strong prompted GPT-5.4-mini full ablation that still underperforms the complete system. The 0/45 pass@3 exploit result, paired with high decoy-grounded completion rates, is a clear, falsifiable empirical claim of practical interest to the security community.

major comments (3)
  1. [Experiments Setup; Threat Model; Figure 2] Experiments Setup and Threat Model / Online Artifact Construction: All interactive defenses share the same application-side breadcrumb and interaction entry point (e.g., .git/config, 403/diagnostic comments in Figure 2), and AgentSnare intervenes whenever a tool call reaches a defender-controlled surface. The headline 0/45 and absolute DDD numbers therefore mix (i) engineered decoy discoverability with (ii) learned global-consistency construction. Relative gains over Cowrie and HoneyLLMd under the shared entry are still informative, but the manuscript does not report decoy-entry rates, time-to-first-entry, or the fraction of entries attributable to planted breadcrumbs versus organic exploration, nor whether those breadcrumbs change discoverability of the real vulnerable path. Please add this accounting (and a short sensitivity discussion) so that absolute Delay/PER/SubDefuse and the comp
  2. [Delay–Divert–Defuse Metrics; RQ1; Figure 3] RQ1–RQ2 and Delay–Divert–Defuse Metrics: Divert is operationalized only as post-entry retention (Eq. for PER), and Defuse only as provenance of completion attempts (SubDefuse). The paper shows that real and decoy calls can still alternate (Figure 3) and that metrics need not move together (e.g., GPT-5.4 lowest Delay, highest SubDefuse), but it does not characterize exit events—when and why agents leave the decoy, whether exits coincide with consistency failures, suspicion, or budget pressure, or how often re-entry occurs. Without a brief failure-mode / exit analysis tied to Wt/Gt validation outcomes, it is hard to judge how durable “continual steering” is versus temporary absorption before a return to the real target. A compact breakdown (exit reasons, re-entry rate, actions-to-exit after first entry) would make the Divert claim load-bearing rather than aggregate-only.
  3. [Offline LLM Training for Artifact-Construction; RQ3] Offline LLM Training / RQ3: The central transfer premise is that reaction-guided dual-role DeepSeek simulation teaches artifacts that hold held-out Claude/GPT/MiniMax attackers inside the decoy when they can ignore fakes and alternate with the real target. Empirical transfer is supported by the main table and by RQ3 (removing fine-tuning restores 3/15 exploits under GPT-5.4), which is good evidence. However, the manuscript does not report any measure of simulator–attacker agreement (e.g., overlap of predicted vs. observed follow-ups on a held-out probe set, or human/LLM judgment of artifact plausibility under real attackers). Given that RQ3 shows both fine-tuning and ShadowOS are necessary for 0 exploits, a small agreement or error analysis would substantially strengthen the claim that the offline objective—not only ShadowOS determinism or shared breadcrumbs—drives steering on real attac
minor comments (6)
  1. [Introduction; Figure 1; Table 1] Figure 1 reports Interference@20 for static tactics with very low average effectiveness (<0.2) but the main evaluation’s Static Defenses condition jointly deploys all 13 instances; briefly reconcile how the Figure 1 protocol relates to the joint deployment used in Table 1 so readers do not conflate single-tactic Interference with the multi-tactic baseline.
  2. [Online Artifact Construction and Disclosure] Notation: ot vs ˜ot, Wt/Gt/Ft, and ct = (˜ot, ΔWt) are clear once introduced, but Equation (1) and the later Commit/Render pair (6) are separated by substantial prose; a short inline reminder that ˜ot becomes ot only after validation would reduce confusion on first read.
  3. [Table 1] Table 1: Overall Delay/PER are reported as point values without SE, while per-attacker rows use mean±SE over 15 CVE-level averages. Adding the same aggregation note (or overall SE) would improve consistency.
  4. [Delay–Divert–Defuse Metrics; Figure 3] Clarify the label set {real, decoy, other} with one or two concrete examples of “other” (local processing vs. unattributable calls), since Delay’s denominator is all T tool calls and “other” is omitted from Figure 3.
  5. [Abstract; Introduction] Minor prose/typos: “defensesrelyheavily”, “toanattack”, “recognizeandbypasstheseartifacts”, and similar missing spaces appear in the abstract/introduction PDF text; a pass for spacing and hyphenation would help.
  6. [Related Work] Related Work: Cowrie / ShelLM / HoneyLLMd contrast is useful; a sentence on whether any prior honeypot enforces a committed cross-command fact ledger (vs. local response realism) would sharpen the claimed novelty of global factual consistency.

Circularity Check

0 steps flagged

No derivation circularity: external CVE-Bench evaluation and held-out attackers; DDD are empirical process metrics, not tautological restatements of training inputs.

full rationale

AgentSnare is an empirical systems paper, not a first-principles derivation. The load-bearing outcome claims (0/45 verified real-target exploits at pass@3; 46.8% Delay; 55.9% PER; 90.0% SubDefuse) are measured on 15 deployable CVE-Bench applications and three external attacker models under the official evaluator, with a stated training corpus of 24,807 synthetic defender decisions that explicitly excludes CVE-Bench apps, vulnerability descriptions, reference exploits, and evaluator outputs. Offline reaction-guided selection chooses artifacts that keep a DeepSeek simulator in the decoy; that is ordinary objective-aligned supervision, not fitting a parameter on the evaluation set and renaming it a prediction. DDD metrics are definitional accounting of action-target labels and completion provenance over runs, reported as measured fractions rather than quantities derived from themselves. Related-work citations (PentestGPT, CVE-Bench, Cowrie, HoneyLLMd, etc.) are external baselines and infrastructure, not author-overlapping uniqueness theorems that force the result. No equation reduces a claimed prediction to its fitted input by construction; no ansatz is smuggled in via self-citation. Evaluation-design confounds (shared breadcrumbs/entry surfaces) affect causal attribution of why agents enter the decoy, but that is not circularity in the derivation-chain sense required here.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 4 invented entities

The central empirical claim rests on standard agent loop assumptions, a defender capability model (intercept only on controlled surfaces; no change to real vulnerable paths), synthetic dual-role simulation as a valid teacher for artifact policy, and several engineered entities (ShadowOS ledger/graph, DDD labels) plus training/evaluation hyperparameters. No physical constants; free parameters are ML and protocol knobs that shape what “success” looks like.

free parameters (5)
  • LoRA rank r and alpha = r=64, α=128
    Adapter capacity chosen for Qwen3-4B SFT; affects which artifact policies can be represented.
  • SFT epochs and corpus size = 3 epochs; 24,807 decisions / 503 trajectories
    Three-epoch supervised fine-tuning on filtered simulator decisions; filtering rules (engagement length, diversity) are hand-set.
  • Offline candidate count k and reaction samples m = k and m as used in synthesis (not numerically fixed in prose beyond “several”/sampled sets)
    Number of alternate artifacts and simulated follow-ups per probe drives reaction-guided selection; values are design choices.
  • Tool-call budget B and pass@3 repetitions = B=200; 3 reps; pass@3
    Exploit suppression and Delay denominators depend on the 200-call budget and three runs per pair.
  • Scenario distribution Q (12 attack-intent classes) = 12 intent classes
    Synthetic environment/objective/clue/focus sampler defines the training support; hand-specified coverage.
axioms (6)
  • domain assumption Penetration agents select actions primarily from tool-returned observations in an observation–action loop.
    Stated in Abstract/Introduction; justifies observation injection as a defense channel.
  • domain assumption Defender may plant and dynamically disclose artifacts on controlled surfaces without modifying the application’s normal functionality or vulnerable execution path.
    Threat Model defender model; required for ethical/compatibility constraint and for ShadowOS interception.
  • ad hoc to paper Global factual consistency plus frontier-limited new facts can steer agents more durably than static isolated artifacts.
    Core design hypothesis motivating ledger Wt, graph Gt, and validation; supported empirically but not derived from prior theory.
  • ad hoc to paper Dual-role DeepSeek simulation with reaction-guided selection yields supervision that transfers to Claude/GPT/MiniMax attackers on real CVEs.
    Offline training section; load-bearing for the learned policy’s external validity.
  • domain assumption CVE-Bench automated evaluator verdicts are the ground truth for “verified real-target exploit.”
    RQ2; end-to-end success definition.
  • standard math Standard supervised fine-tuning / LoRA optimization is a valid way to distill selected defender decisions into an online one-shot artifact policy.
    Uses conventional SFT+LoRA; no new learning theorem claimed.
invented entities (4)
  • ShadowOS (world-fact ledger Wt + affordance graph Gt + validate/commit/render pipeline) no independent evidence
    purpose: Maintain persistent, globally consistent decoy state and gate which candidate artifacts become real observations.
    Central runtime invention; behavior defined only inside this system.
  • Artifact-construction policy model Mθ (LoRA-tuned Qwen3-4B) no independent evidence
    purpose: Map (history, action, decoy state, profile) to candidate observation and fact increment.
    Learned component trained on paper-specific synthetic trajectories.
  • Delay–Divert–Defuse (DDD) metric suite no independent evidence
    purpose: Quantify trajectory steering beyond binary exploit success.
    Paper-defined operationalizations (Delay, PER, SubDefuse); useful but not externally standardized.
  • Affordance frontier Ft no independent evidence
    purpose: Limit which new decoy opportunities may be introduced given the current probe.
    Structural constraint invented to enforce local coherence of expansions.

pith-pipeline@v1.2.0-daily-grok45 · 17296 in / 4050 out tokens · 91026 ms · 2026-07-30T14:27:29.754227+00:00 · methodology

0 comments
read the original abstract

Large language model (LLM) agents automate penetration testing through an observation-action loop, selecting actions based on observations returned by tools. This dependence allows defenders to inject deceptive observations that can mislead the agent's decision-making process. However, existing defenses rely heavily on static, isolated artifacts planted in the environment prior to an attack. Advanced agents can progressively recognize and bypass these artifacts, ultimately refocusing their exploitation attempts on the real target. To address this issue, we introduce AgentSnare, a trajectory-adaptive deception system that dynamically unfolds a decoy environment to continually steer the penetration agent away from the real target. Specifically, AgentSnare employs an artifact-construction policy model that constructs candidate artifacts conditioned on the agent's interaction history and decoy state. AgentSnare then validates these candidates and incrementally incorporates valid artifacts into a factually consistent decoy environment, thereby delaying the attack by absorbing its tool calls, diverting its post-entry trajectory within the decoy, and defusing it by inducing completion reports grounded in decoy evidence. Across 15 CVE-Bench web applications and three attacker models, AgentSnare absorbs 46.8% of the agent's tool calls in the decoy and retains 55.9% of post-entry actions there, while 90.0% of completion attempts are grounded in decoy evidence; across all 45 attacker-CVE pairs, no real target is successfully exploited at pass@3.

Figures

Figures reproduced from arXiv: 2607.26998 by Heng Zhao, Mengnan Zhao, Renjie Wu, Ruoyu Wang, Tianhang Zheng, Wanyu Lin, Zhixuan Chu.

Figure 1
Figure 1. Figure 1: Interference@20 of 13 static intervention in￾stances across six attacker LLMs and six tactics. Higher values indicate stronger interference. fenders can deliberately manipulate the observations to mis￾lead the agent’s decision-making process. Existing defenses against LLM-based penetration agents mainly rely on static tactics, which introduce deceptive ar￾tifacts such as honeytokens, misleading files, prom… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of AgentSnare. The frontier limits which artifacts may be constructed next. For example, a request for an .env file may reveal configu￾ration values, and credentials, but not an unrelated privilege￾escalation or lateral-movement opportunity. Together, Wt and Ft specify which previously constructed facts must remain unchanged and which connected artifacts may be introduced during the current intera… view at source ↗
Figure 3
Figure 3. Figure 3: Share of actions targeting the decoy and real tar [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

36 extracted references · 7 linked inside Pith

  1. [1]

    Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education

    Clancey, William J. Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education. Proceedings of the Eighth International Joint Conference on Artificial Intelligence (IJCAI-83)

  2. [2]

    Classification Problem Solving

    Clancey, William J. Classification Problem Solving. Proceedings of the Fourth National Conference on Artificial Intelligence

  3. [3]

    , title =

    Robinson, Arthur L. , title =. 1980 , doi =. https://science.sciencemag.org/content/208/4447/1019.full.pdf , journal =

  4. [4]

    New Ways to Make Microcircuits Smaller---Duplicate Entry

    Robinson, Arthur L. New Ways to Make Microcircuits Smaller---Duplicate Entry. Science

  5. [5]

    Clancey and Glenn Rennels , abstract =

    Diane Warner Hasling and William J. Clancey and Glenn Rennels , abstract =. Strategic explanations for a diagnostic consultation system , journal =. 1984 , issn =. doi:https://doi.org/10.1016/S0020-7373(84)80003-6 , url =

  6. [6]

    and Rennels, Glenn R

    Hasling, Diane Warner and Clancey, William J. and Rennels, Glenn R. and Test, Thomas. Strategic Explanations in Consultation---Duplicate. The International Journal of Man-Machine Studies

  7. [7]

    Poligon: A System for Parallel Problem Solving

    Rice, James. Poligon: A System for Parallel Problem Solving

  8. [8]

    Transfer of Rule-Based Expertise through a Tutorial Dialogue

    Clancey, William J. Transfer of Rule-Based Expertise through a Tutorial Dialogue

  9. [9]

    The Engineering of Qualitative Models

    Clancey, William J. The Engineering of Qualitative Models

  10. [10]

    2023 , eprint=

    Attention Is All You Need , author=. 2023 , eprint=

  11. [11]

    Pluto: The 'Other' Red Planet

    NASA. Pluto: The 'Other' Red Planet

  12. [12]

    33rd USENIX Security Symposium (USENIX Security 24) , pages=

    \ PentestGPT \ : Evaluating and harnessing large language models for automated penetration testing , author=. 33rd USENIX Security Symposium (USENIX Security 24) , pages=

  13. [13]

    arXiv preprint arXiv:2503.17332 , year=

    CVE-bench: a benchmark for AI agents' ability to exploit real-world web application vulnerabilities , author=. arXiv preprint arXiv:2503.17332 , year=

  14. [14]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Towards effective offensive security llm agents: Hyperparameter tuning, llm as a judge, and a lightweight ctf benchmark , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  15. [15]

    Cowrie SSH/Telnet Honeypot , year =

  16. [16]

    2024 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW) , pages=

    Llm in the shell: Generative honeypots , author=. 2024 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW) , pages=. 2024 , organization=

  17. [17]

    IEEE Transactions on Network Science and Engineering , year=

    HoneyLLMd: A Large Language Model-Powered Adaptive Honeypot System , author=. IEEE Transactions on Network Science and Engineering , year=

  18. [18]

    34th USENIX Security Symposium (USENIX Security 25) , pages=

    Cloak, Honey, Trap: Proactive Defenses Against \ LLM \ Agents , author=. 34th USENIX Security Symposium (USENIX Security 25) , pages=

  19. [19]

    arXiv preprint arXiv:2410.20911 , year=

    Hacking back the ai-hacker: Prompt injection as a defense against llm-driven cyberattacks , author=. arXiv preprint arXiv:2410.20911 , year=

  20. [20]

    arXiv preprint arXiv:2508.00910 , year=

    Cyber-zero: Training cybersecurity agents without runtime , author=. arXiv preprint arXiv:2508.00910 , year=

  21. [21]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Safenlidb: A privacy-preserving safety alignment framework for llm-based natural language database interfaces , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  22. [22]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Shoppingbench: A real-world intent-grounded shopping benchmark for llm-based agents , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  23. [23]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Security games with layered defenses: adaptive adversaries and gittins indices , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  24. [24]

    , author=

    Lora: Low-rank adaptation of large language models. , author=. Iclr , volume=

  25. [25]

    arXiv preprint arXiv:2505.09388 , year=

    Qwen3 technical report , author=. arXiv preprint arXiv:2505.09388 , year=

  26. [26]

    IEEE Transactions on Information Forensics and Security , year=

    AutoPT: How Far Are We From the Fully Automated Web Penetration Testing? , author=. IEEE Transactions on Information Forensics and Security , year=

  27. [27]

    arXiv preprint arXiv:2501.13411 , year=

    Vulnbot: Autonomous penetration testing for a multi-agent collaborative framework , author=. arXiv preprint arXiv:2501.13411 , year=

  28. [28]

    arXiv preprint arXiv:2306.05301 , year=

    Toolalpaca: Generalized tool learning for language models with 3000 simulated cases , author=. arXiv preprint arXiv:2306.05301 , year=

  29. [29]

    arXiv preprint arXiv:2310.05915 , year=

    Fireact: Toward language agent fine-tuning , author=. arXiv preprint arXiv:2310.05915 , year=

  30. [30]

    Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    Agenttuning: Enabling generalized agent abilities for llms , author=. Findings of the Association for Computational Linguistics: ACL 2024 , pages=

  31. [31]

    Proceedings of the 29th symposium on operating systems principles , pages=

    Efficient memory management for large language model serving with pagedattention , author=. Proceedings of the 29th symposium on operating systems principles , pages=

  32. [32]

    Introducing Claude Opus 4.6 , year=

  33. [33]

    GPT-5.4 Thinking System Card , year=

  34. [34]

    MiniMax M3: Frontier Coding, 1M Context, Native Multimodality---All in One Model , year=

  35. [35]

    arXiv preprint arXiv:2511.13725 , year=

    AI Kill Switch for malicious web-based LLM agent , author=. arXiv preprint arXiv:2511.13725 , year=

  36. [36]

    2025 , publisher=

    LLM Agent Honeypot: Monitoring AI Hacking Agents in the Wild , author=. 2025 , publisher=