Pith. sign in

REVIEW 4 major objections 1 cited by

Agent Hacks Agent: Autoresearch for Production-Agent Red-Teaming

T0 review · 4 major / 0 minor · reviewed 2026-07-14 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A falsifiable autoresearch loop turns production-agent red-team runs into a frozen Vulnerability Concept Graph that stays reusable without further search and beats frozen attack archives by 14.2 points.

desk verdict Solid systems paper: falsifiable autoresearch that freezes a reusable VCG beats frozen baselines by ~14pp under a careful single-shot protocol; the main soft spot is incomplete isolation of research-model strength, not a broken claim. read the letter →

arxiv 2607.11698 v1 pith:G25CQE7C submitted 2026-07-13 cs.CR cs.AI

Xutao Mao , Xiang Zheng , Cong Wang This is my paper · ORCID
classification cs.CRcs.AI
keywords LLMagentsautomatedred-teamingvulnerabilityconceptgraphfalsifiablediscoveryindirectpromptinjectionproductionagentsafetyautoresearchattacktransfer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Production coding agents act on untrusted files, tools, and workspace state, so a safety failure is a real action rather than a bad sentence. Most automatic red-teaming optimizes attack success and freezes payloads, archives, or programs, which record where a break landed but not the enabling condition that made the agent trajectory unsafe. This paper reframes the problem as autoresearch: one research agent states a vulnerability hypothesis and falsifier before designing an attack, runs it in a sandbox, reflects on the trajectory, and promotes only confirmed findings into a Vulnerability Concept Graph. Across Claude Code and Codex on direct and indirect scenarios, the frozen graph needs no further search at test time, outperforms the strongest frozen discovery baseline by 14.2 percentage points under a shared single-shot protocol, and transfers across scenarios, attack channels, and victim models. The intended payoff is cumulative, auditable safety knowledge that a team can inspect, patch against, and re-check after model or product updates.

What carries the argument

The Vulnerability Concept Graph (VCG), filled by AHA’s falsifiable discovery loop: commit a hypothesis and falsifier before any attack, instantiate a scenario-valid payload, execute in a sandbox, adjudicate the trajectory against the falsifier, and promote only concepts that meet a multi-confirmation evidence rule into a reusable, auditable library.

What would settle it

Freeze a VCG discovered on one scenario or victim model, deploy it single-shot with no further search on held-out instances from a different scenario or model (including direct-to-indirect channel shifts), and check whether held-out attack success collapses to baseline levels or the claimed-authorization core fails to recur across independent victim agents.

Watch

Extended reading notes

Core claim

The authors claim that black-box red-teaming of tool-using production agents can be organized as a falsifiable discovery loop whose lasting product is not a payload archive but a Vulnerability Concept Graph: each concept links an attacker-facing surface to an unsafe trajectory through a claim, enabling condition, committed falsifier, transfer prediction, and evidence. A graph that passes their promotion rule can be frozen and reused single-shot, revealing a shared claimed-authorization core across models and agents and outperforming the strongest frozen discovery baseline by 14.2 percentage points on held-out instances.

Load-bearing premise

That the scenario judges plus the pre-committed falsifier reliably separate true mechanism-level agent failures from judge-gamed, off-target, or harness-specific breaks, so held-out and transfer scores measure reusable vulnerability knowledge rather than artifacts of the search or the scorer.

Editorial extensions

If this is right

  • A frozen VCG can be re-run after a safety patch or model swap to check whether a named enabling condition still produces an unsafe trajectory, without launching a new search.
  • Safety teams can treat each concept’s enabling condition as a concrete lever for policy, tool-boundary, or workflow changes rather than chasing perishable payloads.
  • Concepts that transfer from direct multi-turn attacks to indirect tool-mediated injection (and the reverse) locate the failure in agent trajectory structure, not in stored attack strings.
  • Build and import workflows can fold new free-text concerns or existing benchmarks into the same loop, so new threats extend one shared concept library instead of spawning one-off silos.
  • Negative results (falsified framings kept as non-mechanisms) become part of the audit trail so later search does not re-explore inert patterns.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the shared claimed-authorization core is truly agent-level, product teams might prioritize provenance and ownership checks over content filters alone when hardening coding agents.
  • The same hypothesize–falsify–promote contract could be applied to browser, mobile, or multimodal agents wherever an attack surface, harness, trajectory, and judge exist.
  • A public, versioned VCG across vendors could become a common language for incident response, analogous to how CVE entries accumulate software-vulnerability knowledge.
  • Judge gaming remains the practical bottleneck: any improvement in trajectory judges would tighten the evidence rule and change which concepts survive promotion.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 0 minor

Summary. The paper proposes AHA, an autoresearch loop in which one agentic research environment discovers reusable vulnerability knowledge about production-style tool-using agents (Claude Code, Codex). The loop commits a vulnerability hypothesis and falsifier before attack design, executes scenario-valid payloads in a sandboxed harness, reflects on trajectories, and promotes confirmed findings into a Vulnerability Concept Graph (VCG) whose entries package claim, enabling condition, falsifier, template, transfer prediction, and evidence. On AgentHazard, AgentDyn, and DTap, frozen VCGs are evaluated single-shot on held-out instances against frozen discovery baselines (T-MAP*, IterInject*, AutoRISE*) and the benchmark original attack. The authors report a 14.2-point overall ASR gain over the strongest frozen baseline, a recurring claimed-authorization core across models and agents, ablations of falsifier/memory/critic, and cross-scenario and cross-victim transfer.

Significance. If the results hold under cleaner isolation, this is a meaningful reframing of agent red-teaming: from perishable payloads/archives/programs toward auditable, mechanism-level concepts that can be inspected, patched against, and re-run after model or product updates. The experimental protocol is unusually careful for the area—fixed discovery/held-out splits, frozen artifacts, one attack per held-out instance, shared evaluator/sandbox/judge, and transfer tests that go beyond leaderboard ASR. The VCG design, negative-result bookkeeping, and bring-your-own build/import scenario workflows are practically useful artifacts for production safety teams. Code release further strengthens the contribution. The work is dual-use but oriented toward defensive triage rather than exploit dissemination.

major comments (4)
  1. The headline 14.2 pp frozen-artifact gain (abstract; §5.2; Table 2) is not cleanly isolated from research-model asymmetry. AHA discovery runs on host models (Claude-4.8-Opus / GPT-5.5), while T-MAP*, IterInject*, and AutoRISE* payload generation use compliant Qwen-3.7-Max because host models refuse the same prompts (§4.4, B.6). The AHA@Qwen control (Table 3) is limited to Deepseek×Claude Code and only vs T-MAP*/IterInject*, not the full 18-setting average that produces the 14.2-point claim. AutoRISE is further hybrid (coding agent on the host model). Either expand the matched-model isolation across scenarios/victims, or revise the abstract/intro claim to a more carefully scoped comparison that does not present 14.2 pp as pure concept-reusability advantage.
  2. Held-out and transfer ASR still rest on scenario judges plus the committed falsifier (§3.4–3.5, C.1). AgentHazard’s break rule is an LLM trajectory judge with harmful flag and score ≥7; AgentDyn/DTap use verifiable checks. RQ3 (Table 4) shows that removing falsifier/memory/critic leaves discovery ASR high while cutting held-out survival—good evidence for the safeguards—but does not quantify residual judge-gamed, off-target, or harness-specific breaks that survive promotion into the frozen VCG. For the claim that ASR measures reusable vulnerability knowledge rather than optimizer/judge artifacts, add either (i) a human audit of a stratified sample of held-out ‘breaks’ against the committed enabling condition, or (ii) a sensitivity analysis under stricter AgentHazard thresholds / alternative judges, and discuss residual risk explicitly in §5–6.
  3. Promotion and search knobs are free parameters without sensitivity (Eq. 2; Table 8: n_conf ≥3, confirmation rate ≥0.6, confidence ≥0.6, outer cap 100, mode availability rule). Because the counted VCG is the deployable artifact, the main results could shift under nearby thresholds. Report at least a small sensitivity sweep on n_conf and confirmation rate for one representative setting (e.g., Claude Code×Minimax×AgentHazard), or justify the thresholds as fixed a priori and show that effective concept count / held-out ASR is stable in a neighborhood of the chosen values.
  4. Codex×DTap evaluation is partially compromised by tool-routing instability (footnote in §5.2: non-OpenAI backends and type:namespace MCP encoding; openai/codex#26234). Multi-tool indirect attacks fail in unstable mode with unsupported-tool errors, which depresses Codex DTap numbers and muddies cross-agent comparison on that scenario. Either restrict the Codex DTap claim to settings where tool calls are reliable, re-run with a fixed routing fix, or move Codex DTap to appendix with an explicit reliability filter so Table 2 averages are not diluted by harness noise.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: empirical held-out/transfer evaluation is separated from discovery by construction, not a self-definitional derivation.

full rationale

This is an empirical systems paper, not a first-principles derivation. The central claim (frozen VCG reusable single-shot, +14.2 pp vs strongest frozen baseline; shared mechanisms; cross-scenario/victim transfer) is measured under an explicit discovery/held-out split: discovery never reads held-out instances; the VCG is frozen; evaluation emits exactly one attack per held-out instance with no retries or test-time search (Eq. 1; §3.2; §4.3–4.4). Success is scored by scenario judges external to the promotion rule (AgentHazard trajectory judge, AgentDojo security check, DTap per-instance backend judges; §4.2, C.1), not by re-labeling discovery confirmations as held-out wins. The promotion rule (Eq. 2) and committed falsifier (§3.4–3.5) gate what enters the VCG during search; they do not redefine held-out ASR. Ablations (RQ3, Table 4) further treat safeguards as causal interventions and show discovery ASR can stay high while held-out reuse falls—evidence against tautological success. Related work cites external red-teaming paradigms; there is no load-bearing uniqueness theorem or ansatz imported from overlapping-author prior work that forces the main result. Confounds noted elsewhere (research-model asymmetry vs Qwen baselines; residual judge gaming) are validity/isolation issues, not circular reductions of prediction to fitted input. No self-definitional step, fitted-as-prediction step, or renaming of a known closed-form result was found.

Assumptions & free parameters 4 free parameters · 4 assumptions · 2 invented entities

The central claim rests on empirical protocol choices and domain assumptions about agent harnesses and judges, not on free physical constants. The main free knobs are the VCG promotion thresholds and search-mode rules. Domain assumptions include black-box scenario contracts, judge fidelity, and sandbox isolation. Invented entities are the VCG and the structured vulnerability concept unit; they are operationalized by the loop and evaluated by held-out/transfer tests rather than postulated as unobservable physics.

free parameters (4)
  • n_conf promotion threshold
    A concept is counted only if n_conf ≥ 3; this hand-chosen evidence gate controls what enters the frozen VCG and thus held-out ASR.
  • confirmation-rate threshold
    Promotion requires n_conf / max(1, n_conf + n_fals) ≥ 0.6 among confirmed-or-falsified observations.
  • confidence threshold
    Promotion also requires confidence ≥ 0.6 (with confidence defined from confirmation/observation counts).
  • search-mode availability rule and outer iteration cap
    Explore/exploit/transfer/consolidate routing and default 100-iteration cap allocate discovery budget and shape which concepts are found.
assumptions (4)
  • domain assumption Scenario judges correctly label harmful trajectories for AgentHazard, AgentDyn, and DTap under the stated break rules.
    ASR and promotion depend on J_s(i, τ_i); AgentHazard uses an LLM judge (score≥7), AgentDyn/DTap use verifiable checks (§4.2, App. C.1).
  • domain assumption Black-box interaction through the scenario’s attacker-facing surface is a valid model of production-agent attack channels.
    Problem setting §3.1 fixes direct multi-turn and indirect tool/document injection as the executable surfaces.
  • ad hoc to paper A pre-committed falsifier plus trajectory comparison can adjudicate mechanism confirmation vs off-target breaks.
    Core methodological commitment of AHA (§3.5); ablations treat it as load-bearing for held-out reuse.
  • domain assumption Frozen single-shot instantiation by the host model fairly measures artifact reusability across methods.
    Shared evaluation protocol §3.2 and §4.4; baselines are adapted into the same payload schemas and freeze stage.
invented entities (2)
  • Vulnerability Concept Graph (VCG) independent evidence
    purpose: Stores reusable, auditable vulnerability concepts with claims, enabling conditions, templates, falsifiers, transfer predictions, counters, and edges.
    Primary artifact claimed to be more reusable than payloads/archives/programs; evaluated via frozen held-out and transfer ASR.
  • Vulnerability concept (claim + enabling condition + falsifier + template + transfer prediction) independent evidence
    purpose: Unit of mechanism-level red-team knowledge linking attacker surface to unsafe trajectory.
    Defined by the paper’s schema and promotion rule; independent handle is re-instantiation on held-out instances and foreign scenarios/models.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Agent Hacks Agent: Autoresearch for Production-Agent Red-Teaming." pith.science (2026). https://pith.science/paper/G25CQE7C

@misc{pith2026260711698,
  author       = {Pith},
  title        = {Pith review of: Agent Hacks Agent: Autoresearch for Production-Agent Red-Teaming},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/G25CQE7C}},
  note         = {Machine review of arXiv:2607.11698}
}
read the original abstract

Production LLM agents such as Claude Code and Codex operate over untrusted content, files, commands, and workspace state, making safety failures directly actionable. Red-teaming must therefore keep pace with evolving models and tools. Existing approaches mainly optimize attack success and preserve artifacts such as benchmarks, payloads, or attack programs, which record where attacks succeed but not the enabling conditions behind unsafe agent behavior. We study automated red-teaming for production LLM agents using one agentic research environment to discover reusable vulnerability knowledge about another. We present AHA, a falsifiable discovery loop that proposes a vulnerability hypothesis, constructs a falsifier, instantiates a valid attack, executes it in a sandboxed harness, reflects on the trajectory, and promotes confirmed findings into a Vulnerability Concept Graph (VCG). Each concept links an attacker-facing surface to an unsafe trajectory through a claim, enabling condition, falsifier, transfer prediction, and supporting evidence. Across Claude Code and Codex on three scenarios covering direct and indirect attacks, the discovered concepts reveal a reusable vulnerability core across models and agents. A frozen VCG requires no further search and outperforms the strongest frozen discovery baseline by 14.2 percentage points under the same single-shot protocol, while transferring across scenarios and attack channels. The resulting VCG provides an auditable artifact for production safety teams to inspect vulnerabilities, validate patches, and accumulate reusable safety knowledge. Our code is available at https://github.com/henrymao2004/Auto-research-red-teaming-in-sleep.

Figures

Figures reproduced from arXiv: 2607.11698 by the authors.

Figure 1
Figure 1. AHA overview. An autoresearch loop turns executed red-team trajectories into a frozen, reusable VCG, the auditable artifact this paper produces and evaluates. et al., 2025a; Pan et al., 2025; Zhang & Pei, 2026). Agent failures are becoming operational fail￾ures, and the ones that matter are trajectory-level, where an agent reads attacker-controlled content, chooses a sequence of tool calls, and realizes harm through… view at source ↗
Figure 2
Figure 2. The AHA discovery loop. An orchestrator routes each iteration under one of four search modes while role-isolated sub-agents test a committed hypothesis and promote validated concepts into the VCG (mechanism details in Appendix A.1). constructive baseline, so ASR measures whether the discovered concepts still operationalize once the loop is frozen, and a comparable ASR comes with an auditable vulnerability-concept re… view at source ↗
Figure 3
Figure 3. Vulnerability-concept landscape (RQ1). Each counted concept from the eighteen dis￾covered VCGs is assigned to one of eight recurring concept families (rows); columns are the scenario × victim-agent × victim-model settings (AHaz=AgentHazard, ADyn=AgentDyn; Mm=Minimax￾M2.7, Km=Kimi-K2.6, Ds=Deepseek-V4-Pro), and each entry reports the number of concepts in that family. The claimed-authorization family is lit in sixtee… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Concepts transfer as durable mechanisms (RQ4). Transferred frozen-VCG held-out ASR (%). (a) across scenarios: a scenario’s frozen VCG on another scenario’s held-out split (Claude Code; error bars show sd across victim models when multiple values are present). (b) acros…
Figure 5
Figure 5. Figure 5: Promotion trajectory. Cumulative counted concepts over discovery iterations on a representative run (AgentHazard, Deepseek-V4- Pro), showing how the falsifiable-promotion gate (nconf ≥ 3, confidence ≥ 0.6, ≥ 1 effective break) accumulates a validated VCG during discove…
Figure 6
Figure 6. Figure 6: Concept graphs for all nine discovery runs (rows: scenario; columns: victim model; Claude Code victim agent). Each panel is the VCG recorded by that run: circles are vulnerability concepts (labeled by VC number), light squares are held-out harm-category transfer target…
Figure 7
Figure 7. Figure 7: Concept graphs for all nine discovery settings (rows: scenario; columns: victim model; Codex victim agent). Each panel is the VCG for that setting: circles are vulnerability concepts (labeled by VC number), light squares are held-out harm-category transfer targets, and…
Figure 8
Figure 8. Figure 8: (top) summarizes the build path. The input is an ordinary-language production concern, for example “test whether a customer-support agent leaks private user data through untrusted content.” This workflow is used when the user has a production safety concern but no data…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Practice Makes Unsafe: Skill Misevolution in Self-Improving LLM Agents

    cs.AI 2026-08 conditional novelty 6.0 of 10

    Skill misevolution, where unsafe agent experiences become reusable skills and cause fresh-session harm, is measured across 25 configurations and reduced by the SafeEvolve wrapper.

Reference graph

Works this paper leans on

21 extracted references · cited by 1 Pith paper

  1. [1]

    DeliveryPath Specification4

    Victim Environment3. DeliveryPath Specification4. Judge Definition5.VisibilitySplit6.Instancesynthesis & review7. Finalization task, tools, files, memory, browser, workspace, account state.separate attacker-controlled content what evidence should the judge read? what label should it return? researcher-visiblevs evaluator-only fields.synthesize examples; a...

  2. [2]

    Upstream Reconnaissance

  3. [3]

    Benchmark Semantics Recovery

  4. [4]

    Visibility & Judge Mapping

  5. [5]

    Converter Dry-Run & Review

  6. [6]

    Preserve Official Split / Create New

  7. [7]

    Finalization inspect examples, task records, payload fields, labels, splits, and judge logic.confirm what each record asks the agent to do and what counts as success.map upstream fields into researcher-visible inputs and evaluator-only answer keys.run on a small sample, show extracted examples, approve or patch.keep the official split when available; othe...

  8. [8]

    43 Preprint

    For a new payload shape, it teaches the attack generator and payload validator what fields an attack must produce. 43 Preprint

Show all 21 references
  1. [9]

    For a new delivery path, it wires how attack content reaches the agent, such as a user turn, untrusted document, tool response, memory entry, repository file, browser page, or product-specific channel

  2. [10]

    For a new environment setup, it prepares the state the agent must see before execution, such as a workspace, file tree, account state, or external service response

  3. [11]

    For a new interceptor, it specifies which tool response is edited and how attacker-controlled content is inserted

  4. [12]

    For a new trajectory signal, it records the evidence the judge or auditor needs, such as tool calls, file writes, command outputs, messages, browser actions, or product events

  5. [13]

    For a product-specific judge, it attaches the rule or prompt that turns the trajectory into a pass/fail label and auxiliary audit fields

  6. [14]

    This extension path is what lets AHA move from public benchmarks to private production work- flows

    It rebuilds the affected runner image when the runtime code has changed, validates load- ability, checks registry pickup, and reports any remaining stubs that need owner-provided product logic. This extension path is what lets AHA move from public benchmarks to private product...

  7. [15]

    Cross-VC composition: did 2+ concepts combine to break a stronger target? (red: single-concept only, edges stale)

  8. [16]

    Reward hacking: template reuse with swapped strings, repeated EXPLOIT on the same concept×instance, or chains that succeed only by gaming the judge’s framing without real compromise (red:≥3 cases, or any judge-framing-only success)

  9. [17]

    Coverage gaps: per-category last-20 share vs train share (red:≥3 categories under-explored, or any at 0%) 45 Preprint

  10. [18]

    Hypothesis quality drift: are recent hypotheses still concretely falsifiable, or vague aspirational (red:≥3 vague cases) OUTPUT: append a ‘‘Critic check @ v<N>’’ table over the four axes, plus one ‘‘Top recommendation for v<N+1>..v<N+20>’’ phrased as a single actionable step. ...

  11. [19]

    Pick exactly ONE concept whose attack template most naturally applies to this instance’s harmful goal

  12. [20]

    Produce a complete attack object satisfying the schema; set instance id, category, and method name = heldout <VC-id> <instance id>; the payload field instantiates the chosen concept’s attack template concretely against this instance

  13. [21]

    on-call release engineer, INC-4471 checkout outage, CHG-2208 authorizes an emergency hotfix

    Add chosen vc (the VC id) and rationale (one sentence). Return ONE JSON object. No prose, no markdown fences. 46 Preprint. H CASE STUDIES Each case traces one concept end to end: theDiscoveryhalf shows the full discovery research process for the iteration that produced the con...

    1998

Pith tools

Reviewed July 14, 2026 · model on record in the stance chip above.