{"id":"6571c98e-0245-41fe-8f75-3cf5650f8e64","arxiv_id":"2606.03024","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A dual-plane permission framework for LLM agent skills cuts SkillInject attack success by roughly 9 percentage points while nearly preserving benign task success.","lead":"SkillGuard treats AI agent skills as permission-bearing programs and checks both what they put into the model’s context and what side effects they can cause. It offers a practical Android-style permission layer for a fast-growing skill marketplace that currently runs mostly on trust.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"The dual-plane claim is only weakly tested on the context plane; residual ASR is dominated by attacks that reuse already-declared capabilities.","rationale":"The reader correctly identifies the weakest assumption: capability declarations plus tool-call mediation do not catch attacks that stay inside the declared surface, and auto-approving confirm means only deny paths are evaluated. That is the same load-bearing concern. The paper is internally consistent and does not overclaim a complete defense; residual ASR and the email-api example are acknowledged. No contradiction warrants REJECT. The contribution remains accept-shaped systems work if evaluation is tightened on consent, multi-model generality, and task-alignment for declared permissions—exactly the reader’s CONDITIONAL. My concrete test operationalizes that concern without inventing a new one. Abstract/body number mismatches (e.g., 1,260 vs 315 skills; slightly different ASR figures) are secondary and do not change the central empirical claim under the body’s numbers.","tokens_in":18447,"tokens_out":659,"duration_ms":5808,"concrete_test":"Re-run RQ3 on SkillInject with two ablations: (1) force all confirm decisions to deny (or require human-in-the-loop on a fixed 20-task subset) and report ASR/TSR; (2) partition residual successes into “undeclared capability” vs “reused declared capability” and measure how many of the latter are blocked by an explicit LOAD_CONTEXT / instruction-admission check independent of tool mediation. If residual ASR remains near the reported 16–23% under (1) and (2) does not reduce the reused-capability bucket, the dual-plane sufficiency claim weakens.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The strongest claim is that treating skills as permission-bearing artifacts and jointly governing context influence and action side effects reduces SkillInject ASR (32.37%→23.02% contextual; 25.56%→16.67% obvious) with near-unchanged utility. The load-bearing assumption is that skill-level capability declarations plus PreToolUse mediation are a sufficient principal for skill attacks. The paper itself shows residual successes when the attack reuses capabilities already declared for legitimate work (e.g., INST-26_email-api_task1 mass-forward via declared web.post / external_api.call; §8). Moreover, the unattended evaluation auto-approves every confirm as a one-time allow (§5.1), so only deny paths are stress-tested; user-mediated consent and session-grant scope are not measured. Context-plane governance (LOAD_CONTEXT, instruction admission) is declared in the taxonomy and dual-plane framing (§2.1, Table 1) but is not separately ablated: SkillInject attacks that succeed without new undeclared tool capabilities are not shown to be blocked by context admission rather than by action denial. Thus the joint dual-plane claim is only partially supported by the reported numbers.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"SkillGuard proposes a skill-centric permission framework for LLM agent ecosystems, treating skills as permission-bearing executable artifacts that must be governed on both a context plane (instructions and loaded content that reshape reasoning) and an action plane (tool calls and side effects). The system comprises a JSON skill-manifest DSL, a multi-group capability taxonomy, PreToolUse-style runtime mediation, user confirm grants, deny-by-default policy composition, a shell permission-generation mini-agent, and audit logging. Empirically, dual-annotator analysis of 315 SkillsMP skills reports 99.76% predefined protected-object coverage; automated manifest generation on 23 clean SkillInject skills reaches 91.0% capability F1; and on SkillInject, SkillGuard reduces ASR from 32.37% to 23.02% (contextual) and 25.56% to 16.67% (obvious) while TSR falls only from 86.96% to 85.51% on eligible benign tasks.","tokens_in":18845,"tokens_out":1297,"duration_ms":26364,"significance":"If the dual-plane framing and runtime design hold, this is a timely systems contribution: skills are becoming a first-class distribution unit for agent behavior, and prior work largely secures either tool calls (Progent, AgentBound) or context integrity (A2AS) without a skill-level principal that links declared intent, context admission, and side effects. Strengths include a concrete, inspectable DSL; a taxonomy validated by dual human annotation with reported agreement; honest residual-attack case analysis; paired ASR/TSR measurement against a no-guard baseline; and promised artifacts. The absolute ASR reductions are modest, but the paper correctly positions SkillGuard as a practical foundation rather than a complete defense. Credit is due for measuring utility preservation and for documenting over-declaration and capability-reuse bypasses rather than only reporting headline ASR drops.","major_comments":[{"comment":"The central dual-plane claim (§1–§2.1, Abstract) is only partially supported by the evaluation. Table 1 includes LOAD_CONTEXT and related Agent Ecosystem permissions, but RQ3 (§8, Table 4) measures success via whether adversarial behavior occurs under PreToolUse capability checks; residual successes (e.g., INST-26_email-api_task1 mass-forward using already-declared web.post / external_api.call) are action-plane capability reuse, not evidence that context admission was jointly enforced. Please either (i) add an ablation that isolates context-plane mediation (instruction/context admission vs. action denial), or (ii) narrow claims so that dual-plane governance is presented as an architectural goal with action-plane results as the primary empirical support.","section":null},{"comment":"§5.1 states that every confirm decision is auto-approved as a one-time allow in unattended runs, so only deny paths are stress-tested. User Interaction (§4.3)—Allow once / Allow this session / Deny, and non-widening session grants—is a core module of the claimed framework, yet RQ3–RQ4 never measure consent burden, grant-scope correctness, or attacks that rely on over-broad user approvals. This is load-bearing for the complete-mediation and least-privilege story. At minimum, report how many mediated calls would have required confirm under the generated manifests, and discuss how auto-approve biases ASR/TSR; ideally add a small interactive or simulated-consent study.","section":null},{"comment":"§8 acknowledges that SkillGuard cannot separate legitimate from adversarial use of declared capabilities, yet the paper’s strongest claim still frames skill-level capability declarations plus tool-call mediation as a sufficient security principal for skill attacks. Given residual ASR of 16.67–23.02% and the email-api / similar bypass class, the Discussion should more sharply bound what permission manifests can and cannot stop (e.g., task-alignment or data-flow checks for ambiguous declared permissions), and avoid implying that dual-plane skill permissions close the skill-injection problem rather than reduce a subset of undeclared-capability attacks.","section":null}],"minor_comments":[{"comment":"§6 reports Cohen’s κ=0.656 as “strong agreement”; under Landis & Koch this is conventionally “substantial.” Please use standard terminology.","section":null},{"comment":"RQ2 (§7, Table 3): 56.5% of manifests are over-declared. Briefly discuss operational impact (prompt fatigue, user habituation) beyond noting that over-declaration is “safer.”","section":null},{"comment":"Threats to validity (§11.2) correctly note single model (MiMo-V2.5-Pro) and single scaffold (Claude Code). A short qualitative argument why tool-to-capability mapping should transfer would strengthen external validity claims in §11.1.","section":null},{"comment":"Ensure abstract and body statistics match exactly (skill count, coverage %, ASR/TSR). Any stale abstract numbers should be reconciled before camera-ready.","section":null},{"comment":"Figures 4–6 use log-scale token/time axes; add median/mean annotations in the figure or caption so readers need not rely only on prose.","section":null},{"comment":"Minor polish: “PerT ool U se” / spacing artifacts in Figure 2 labels; “decently maintaining” in the short abstract is informal for a journal tone.","section":null}],"recommendation":"major_revision","confidential_remarks":"Fit for a systems/security venue is reasonable if the authors tighten dual-plane claims and fully disclose the auto-approve limitation. The contribution is incremental relative to AgentBound/Progent but well-motivated by the skill principal. I would not reject on residual ASR alone—the honesty about capability-reuse bypasses is a plus—but I would not accept without clearer separation of architecture claims from what RQ3 actually measures. No concerns about misconduct; citation of concurrent skill-security work looks appropriate for a fast-moving area."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful takeaway is simple: skills are a new security principal, and SkillGuard is a concrete Android-style permission stack for them—manifest DSL, deny-by-default PreToolUse mediation, session grants, and a mini-agent that expands shell commands into lower-level capabilities. That package is the real contribution, not a huge ASR collapse.\n\nWhat they do well is systems craft. The dual-plane framing (context influence vs action side effects) is the right problem statement, and they actually build the missing middle between Progent-style tool policies and AgentBound-style MCP manifests. Taxonomy work on 315 SkillsMP skills is careful (dual annotation, κ=0.656, ~99.8% object coverage). Manifest generation hits 91% F1 with high recall, which is the safer failure mode. ASR drops ~9 points on SkillInject with almost no utility loss (86.96% → 85.51% TSR). They also ship artifacts and acknowledge residual attacks that reuse already-declared capabilities (email mass-forward is the honest example).\n\nSoft spots, in proportion: the dual-plane claim is only partly stress-tested. LOAD_CONTEXT and instruction admission are in the taxonomy, but the reported wins look like action denial of undeclared capabilities, not a separate context-admission ablation. Unattended runs auto-approve every confirm, so user consent and session-grant discipline are designed but not measured. Single model/scaffold, modest effect sizes, and no task-alignment check once a capability is declared. Those are evaluation limits, not a broken design.\n\nMath/data/citations look fine for an empirical systems paper—no circular scoring, related work is placed fairly, no load-bearing contradiction. This is for people building agent runtimes, skill marketplaces, or MCP-style sandboxes. Worth a serious referee; I’d engage, cite the dual-plane + manifest idea, and push them on context-plane measurement and multi-model generality rather than desk-reject.","headline":"Solid systems paper that makes skills a real permission principal; security gains are real but modest, and the dual-plane story is stronger on actions than on context.","tokens_in":19449,"tokens_out":503,"would_cite":true,"duration_ms":5503,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Skills should be treated as permission-bearing programs whose context influence and runtime actions are jointly gated by manifests and deny-by-default mediation.","keywords":["LLM agents","agent skills","permission framework","runtime enforcement","access control","skill security","AI agent security","deny-by-default"],"falsifier":"On a held-out set of skill injections whose malicious side effects stay inside the legitimate capability surface already declared for the skill (for example mass-forwarding mail with declared send APIs), measure whether attack success stays high; if SkillGuard cannot reduce those successes without also collapsing benign task completion, the central security claim fails.","tokens_in":19314,"feed_emoji":"🛡️","tokens_out":946,"duration_ms":16703,"temperature":0.7,"pith_summary":"Agent skills are not just prompt snippets or tool wrappers: they reshape what an agent knows and what it does. SkillGuard argues that this dual role makes skills a first-class security principal, and that current defenses fail because they either scan skill files statically or police individual tool calls without linking declared skill intent to live behavior. The framework requires each skill to declare capabilities in a manifest, then mediates every sensitive action at runtime under a deny-by-default policy composed from workspace defaults, the manifest, and user grants, with extra inference for shell commands. On a real-world skill corpus the permission taxonomy covers nearly all observed protected objects, and on SkillInject the system lowers attack success while leaving benign task completion almost unchanged. The practical claim is that skill-centric permissions can give skill marketplaces a workable least-privilege boundary without rewriting the agent runtime.","feed_headline":"Skill permissions cut injection success nearly in half","feed_subtitle":"Manifests and deny-by-default mediation gate what skills can inject and do, with little harm to benign tasks.","key_machinery":"Dual-plane skill permission model: a JSON skill manifest declares capabilities (context loading, file/network/secrets, execution, delegation, policy changes), and a PreToolUse mediation pipeline maps host tool calls to those capabilities, composes workspace/manifest/session grants, and blocks undeclared or constraint-mismatched actions, with a mini-agent refining shell commands into lower-level capabilities.","core_discovery":"Treating skills as permission-bearing executable artifacts and jointly governing context influence and action side effects through manifests, runtime access control, user-mediated grants, deny-by-default enforcement, and capability inference reduces SkillInject attack success from 32.37% to 23.02% under contextual injections and from 25.56% to 16.67% under obvious injections, while benign task success falls only from 86.96% to 85.51%.","pith_inferences":["Permission manifests will likely become a de facto interchange format across agent platforms once a few large skill catalogs adopt them, similar to mobile app manifests.","The residual ASR when attacks reuse legitimate APIs suggests the next research bottleneck is intent/task alignment, not finer capability taxonomies alone.","Unattended auto-approve of confirm decisions understates the human-in-the-loop defense; interactive deployments may see stronger protection and higher friction.","Shell mini-agent analysis is a soft spot: if command-to-capability inference is incomplete, both false allows and false blocks will concentrate on script-heavy skills."],"forward_implications":["Skill marketplaces can require machine-checkable manifests the way app stores require permission declarations, so users and hosts see least-privilege scope before install.","Agent scaffolds that expose tool-call lifecycle hooks can host the same enforcement without modifying model weights or core agent code.","Auto-generated manifests at ~91% capability F1 make authoring burden low enough that governance can scale with marketplace growth.","Residual attacks that stay inside declared permissions imply a need for task-alignment checks beyond pure capability allowlists.","Audit logs of every mediated call (capability, policy source, decision) become a portable observability layer for compliance and debugging."],"fun_headline_variants":["SkillGuard permissions nearly halve skill injection success","Manifests and deny-by-default cut skill injection rates roughly in half","Dual-plane skill permissions drop SkillInject success from ~32% to ~23%","Treating skills as permission artifacts lowers injection success with tiny benign impact","Skill-centric manifests and runtime gates cut injection ASR nearly in half"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"Blocking actions that fall outside a skill’s declared capability set is enough to stop most skill attacks, even when the attack reuses permissions the skill already needs for legitimate work.","fun_headline_variants_meta":{"raw":{"variants":["SkillGuard permissions nearly halve skill injection success","Manifests and deny-by-default cut skill injection rates roughly in half","Dual-plane skill permissions drop SkillInject success from ~32% to ~23%","Treating skills as permission artifacts lowers injection success with tiny benign impact","Skill-centric manifests and runtime gates cut injection ASR nearly in half"]},"model":"grok-4.5","effort":"low","cost_usd":0.003214,"raw_usage":{"total_tokens":1113,"prompt_tokens":818,"num_sources_used":0,"completion_tokens":76,"cost_in_usd_ticks":32140000,"prompt_tokens_details":{"text_tokens":818,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":219,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":818,"tokens_out":76,"duration_ms":2791,"temperature":1.0,"reasoning_tokens":219,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-14T18:30:52.415032+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On a held-out set of skill injections whose malicious side effects stay inside the legitimate capability surface already declared for the skill (for example mass-forwarding mail with declared send APIs), measure whether attack success stays high; if SkillGuard cannot reduce those successes without also collapsing benign task completion, the central security claim fails.","supporting_citations":[],"review_version":2}