{"id":"7ca34933-36ff-4ceb-ba05-10eabea905b2","arxiv_id":"2605.28914","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"AIRGuard is a runtime authority-control layer for tool-using agents that reduces attack success on AgentTrap from 36.3% to 5.5% while retaining higher benign utility than ARGUS or MELON on DTAP-150.","lead":"The paper introduces AIRGuard, a runtime guard that derives task authority into step-level checks, tracks trust sources, and blocks unsafe tool actions in language agents before execution. A smart generalist might read it to see a concrete defense against context-manipulation attacks that turn authorized tools into harmful side effects.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Authority derivation from task to step-level may fail to separate safe/unsafe actions on unseen tool heterogeneity or context patterns","rationale":"The reader's weakest_assumption directly names the unverified mapping step that the empirical claims depend on; the abstract-only review correctly flags that this cannot be assessed without implementation details, and the full-text placeholder does not supply an explicit derivation that would remove the risk.","tokens_in":1734,"tokens_out":319,"duration_ms":14676,"concrete_test":"Take the GitHub implementation of the authority derivation and normalization layer; feed it 20 new heterogeneous tool-call sequences (mix of file, API, and MCP tools) with deliberately ambiguous task authority statements not present in AgentTrap/DTAP-150; measure whether the resulting step-level authority flags match independent human judgment on safety and whether any new bypasses appear.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline reductions (36.3% → 5.5% on AgentTrap; 76% utility on DTAP-150) rest on the claim that heterogeneous calls can be normalized, task authority mapped to per-step decisions, and cross-step risk audited without false negatives or new surfaces. No section or equation in the provided text shows an explicit, auditable mapping (e.g., how source/target trust is computed or how simulation of side effects is bounded). If the derivation logic contains implicit assumptions about tool schemas or context provenance that do not hold for arbitrary MCP tools, the enforcement decisions become unreliable even while benchmark numbers remain high.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper introduces AIRGuard, a runtime guard for tool-using language agents that addresses 'authority confusion' attacks, where untrusted context steers authorized tool actions against user interests. AIRGuard normalizes heterogeneous tool calls (including MCP tools), derives task-level authority into per-step decisions, tracks source/target trust, simulates sensitive side effects, audits cross-step risks, and enforces decisions before execution. It reports reducing Sonnet 4.6 attack success on AgentTrap from 36.3% to 5.5%, preserving 76% benign utility on DTAP-150 (outperforming ARGUS and MELON), and shows via ablation that a dedicated runtime layer outperforms prompt-only policies. Code and data are released.","tokens_in":1860,"tokens_out":567,"duration_ms":33951,"significance":"If the authority derivation and enforcement hold under tool heterogeneity, this provides a practical, enforceable least-privilege layer for agent systems that goes beyond prompt engineering and directly controls side effects. The open release of code/data and the empirical gains on named benchmarks are strengths that support reproducibility and allow direct testing of the runtime control claim.","major_comments":[{"comment":"The central mechanism—deriving task authority into step-level authority, normalizing heterogeneous calls, and computing source/target trust—is load-bearing for all claims yet is described only at a high level in the abstract and system overview without an explicit algorithm, pseudocode, or equations showing the mapping or simulation bounds. This directly impacts the skeptic's concern that the approach may fail to separate safe/unsafe actions on unseen tool patterns.","section":"Abstract and system design section"},{"comment":"Evaluation section, AgentTrap and DTAP-150 results: the reported reductions (36.3%→5.5%; 76% utility) and ablation (prompt-only vs. runtime layer) rest on the assumption that enforcement decisions are reliable, but no details are given on configuration parameters, how authority is instantiated for the specific benchmarks, or whether post-hoc adjustments occurred, making it impossible to verify absence of new attack surfaces or utility loss.","section":"Evaluation section"}],"minor_comments":[{"comment":"The term 'authority confusion' is introduced in the abstract but would benefit from a concise formal definition or example early in the introduction to clarify the distinction from standard jailbreaks.","section":"Introduction"},{"comment":"The abstract states results for 'Sonnet 4.6' and 'Haiku 4.5'; clarify whether these refer to specific model versions or families and ensure consistent naming in tables.","section":"Abstract and evaluation"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback. We agree that greater formalization of the core mechanism and additional evaluation details are needed to strengthen the paper. We will revise the manuscript to incorporate explicit algorithms, equations, and benchmark-specific configurations as outlined below.","responses":[{"response":"We agree the authority derivation process requires more explicit formalization. In the revised manuscript we will add pseudocode for the full pipeline (normalization, task-to-step authority mapping, source/target trust computation, side-effect simulation, and cross-step audit) plus equations defining the trust scores and simulation bounds. These additions will appear in Section 3 and will directly address concerns about generalization to unseen tool patterns.","revision_made":"yes","referee_comment":"[Abstract and system design section] The central mechanism—deriving task authority into step-level authority, normalizing heterogeneous calls, and computing source/target trust—is load-bearing for all claims yet is described only at a high level in the abstract and system overview without an explicit algorithm, pseudocode, or equations showing the mapping or simulation bounds. This directly impacts the skeptic's concern that the approach may fail to separate safe/unsafe actions on unseen tool patterns."},{"response":"We acknowledge the need for reproducibility details. The revision will include a new subsection in the evaluation that specifies all configuration parameters, the exact authority instantiation rules used for AgentTrap and DTAP-150, and an explicit statement confirming no post-hoc adjustments were performed. We will also add discussion of potential new attack surfaces introduced by the runtime layer.","revision_made":"yes","referee_comment":"[Evaluation section] Evaluation section, AgentTrap and DTAP-150 results: the reported reductions (36.3%→5.5%; 76% utility) and ablation (prompt-only vs. runtime layer) rest on the assumption that enforcement decisions are reliable, but no details are given on configuration parameters, how authority is instantiated for the specific benchmarks, or whether post-hoc adjustments occurred, making it impossible to verify absence of new attack surfaces or utility loss."}],"tokens_in":1491,"tokens_out":447,"duration_ms":19784,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main takeaway is that AIRGuard operationalizes least-privilege at action time for tool-using agents by normalizing calls, deriving step authority, tracking source/target trust, simulating effects, and auditing cross-step risk. It frames the problem as authority confusion rather than classic jailbreaks, which is a useful distinction for systems that interact with external tools.\n\nWhat stands out is the empirical comparison. The abstract reports dropping Sonnet 4.6 attack success from 36.3% to 5.5% on AgentTrap and holding 76% benign utility on DTAP-150 versus 52% and 42% for ARGUS and MELON. The ablation indicates the runtime components matter more than prompt-only policies. Code release on GitHub is also a concrete positive.\n\nThe soft spots sit in the authority derivation itself. No explicit mapping or bounding rules appear for how task-level authority becomes per-step decisions or how simulation stays sound across heterogeneous MCP tools. If normalization or trust tracking rests on unstated assumptions about schema provenance or context patterns, the enforcement could miss cases or add overhead. The stress-test concern about unseen tool heterogeneity is worth pressing because the abstract gives numbers without the intermediate logic that would let a reader reproduce or falsify the separation of safe versus unsafe actions.\n\nThis is aimed at people building or securing deployed agents that call external systems. It has enough of a working system and comparative results to merit serious referee time, even if the internals will require clarification on how the core mapping avoids new attack surfaces or utility cliffs.","headline":"AIRGuard adds a runtime authority layer that cuts reported attack success on AgentTrap while beating prompt baselines on utility, but the derivation and simulation steps need more visible mechanics.","tokens_in":2343,"tokens_out":388,"would_cite":false,"duration_ms":27808,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"AIRGuard enforces runtime authority checks to stop tool-using agents from executing harmful actions steered by untrusted context.","keywords":["agent security","runtime guard","authority confusion","tool-using agents","least privilege","action-time authorization","side effect simulation","trust tracking"],"falsifier":"A benchmark of agent tasks where untrusted context produces harmful tool calls that AIRGuard still permits, or where it blocks enough benign tasks to drop utility below 60 percent.","tokens_in":2646,"feed_emoji":"🛡️","tokens_out":781,"duration_ms":25842,"temperature":0.7,"pith_summary":"Tool-using agents turn model outputs into real actions like file reads or API calls, creating a new attack surface where ordinary tools become dangerous because attacker-controlled inputs steer authorized access. The paper identifies this as authority confusion, where untrusted resources inform reasoning but must not control side effects, and introduces AIRGuard as a runtime guard that applies least-privilege authorization at action time. AIRGuard normalizes tool calls, derives task authority into per-step rules, tracks trust sources and targets, simulates effects, audits risks across steps, and blocks execution of unsafe actions. Experiments demonstrate sharp drops in attack success on AgentTrap while retaining more benign utility on DTAP-150 than prior methods, with ablations confirming the runtime layer outperforms prompt-only policies. This matters because agents increasingly mediate external effects, so preventing authority misuse at runtime offers a direct way to limit damage without halting all tool use.","feed_headline":"Runtime guard cuts agent attack success to 5.5%","feed_subtitle":"AIRGuard derives step authority from task goals and blocks unsafe tool calls before execution while keeping 76% benign utility.","key_machinery":"The runtime authority-control layer that derives step-level authority from task authority, tracks trust, and enforces before tool execution.","core_discovery":"AIRGuard operationalizes least privilege as action-time authorization for tool-using agents. It normalizes heterogeneous tool calls, derives task authority into step-level authority, tracks source and target trust, simulates sensitive side effects, audits cross-step risk, and enforces decisions before any action executes. On AgentTrap this reduces Sonnet 4.6 attack success from 36.3 percent without defense to 5.5 percent. On DTAP-150 it preserves 76.0 percent benign utility with Haiku 4.5, outperforming ARGUS at 52.0 percent and MELON at 42.0 percent. An ablation shows prompt-only policy helps only modestly while the dedicated runtime layer gives the agent system direct control over tool-med","pith_inferences":["The normalization step could allow the same authority rules to apply across different agent toolkits without custom per-tool code.","If step-level derivation works, developers could add authority tracking to existing multi-step agents with limited changes to the core model prompt.","The approach suggests testing authority enforcement on agents that use Model Context Protocol tools or chained external services.","Combining runtime checks with existing input sanitization might create layered defenses that handle both jailbreaks and authority confusion."],"forward_implications":["Attack success on benchmarks like AgentTrap falls to low single digits for current frontier models.","Benign task completion stays higher than with existing guard systems on DTAP-150.","Prompt-only policies deliver only modest protection, making runtime enforcement necessary for meaningful safety.","The agent system obtains direct, auditable control over side effects from every tool call."],"fun_headline_variants":["AIRGuard achieves 5.5% attack success via runtime authority","Step authority derivation blocks unsafe agent actions","AIRGuard maintains 76% utility on DTAP-150 benchmarks","Least privilege enforced at tool call execution time","Runtime authority tracking outperforms prior agent defenses"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"Task authority can be reliably turned into step-level rules that correctly separate safe from unsafe actions without missing attacks or causing too much utility loss.","fun_headline_variants_meta":{"raw":{"variants":["AIRGuard achieves 5.5% attack success via runtime authority","Step authority derivation blocks unsafe agent actions","AIRGuard maintains 76% utility on DTAP-150 benchmarks","Least privilege enforced at tool call execution time","Runtime authority tracking outperforms prior agent defenses"]},"model":"grok-4.3","cost_usd":0.005724,"raw_usage":{"total_tokens":2782,"prompt_tokens":770,"num_sources_used":0,"completion_tokens":72,"cost_in_usd_ticks":57237000,"prompt_tokens_details":{"text_tokens":770,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1940,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":770,"tokens_out":72,"duration_ms":17873,"temperature":1.0,"reasoning_tokens":1940,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-29T11:24:38.616680+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A benchmark of agent tasks where untrusted context produces harmful tool calls that AIRGuard still permits, or where it blocks enough benign tasks to drop utility below 60 percent.","supporting_citations":[],"review_version":1}