Pith.

Cryptography and Security · arXiv:2611.24185 v1 · submitted 18 Nov 2026 · 49 references

Guarding the Kingdom of Skill-Based Agents in the AI Dynasty of Computation: Control-Flow Integrity for Agent Skills

Bonan Ruan, Yuhao Sun, Jiahao Liu, Qixiao Lin, Jian Mao, Zhenkai Liang

The paper says

“Control-flow integrity (CFI) confines a program to a control-flow graph fixed ahead of time.”

more of the abstract

Procedural agent skills move this control flow into prompt text, and a large language model (LLM) agent decides at runtime whether each step runs in its reasoning, a tool call, or a script. The procedure's control skeleton survives, but its enforceability depends on where each step runs. Many violations, such as overwriting a file before validation, use only operations the skill requires, so permission allowlists admit them. A deviating or prompt-injected model can also evade a monitor by regime migration, for example by claiming that a check passed. We present SkillCFI, which carries CFI over to skill-based agents. Its compiler confines LLM interpretation to compile time and derives the agent's instructions and an executable contract from one skill. By a formal enforceability criterion, it enforces obligations whose evidence crosses the tool boundary and rejects a skill whose mandatory obligation leaves nothing to check. A deterministic runtime enforcer admits a tool request only on trusted evidence and refuses unmediated channels such as inline scripts, so admitted traces follow the contract whatever the model proposes. On eight self-written skills, no compiled contract permits more than its source under 24 source mutations. In 770 agent sessions, SkillCFI blocks before dispatch all 240 such violations, which an allowlist admits, and normal tasks complete nearly as often as without it. Listing the next legal steps in a refusal raises recovery from 21% to 65%. Checking takes about 2% of task time, and a compact command form cuts session slowdown from 1.32 times to 1.08 times. None of 53 public skills is accepted as written. The compiler's diagnostics point to interfaces rather than workflows and guided seven into enforceable contracts.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 19 Nov 2026 · read the full text (159,729 characters) · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. SkillCFI adapts control-flow integrity to LLM-based agent skills, providing a principled and enforceable security layer that blocks procedural violations before dispatch, with a formal model and thorough evaluation. This line is wrong

10 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

The paper, first character to last

The reviewer was given the whole text, 159,729 characters. Not checked: figures, code, what the 49 references say.

10 minor objections

The paper says, at §1 (Introduction), Figure 1

Pith says: minor objection 1 of 10

Figure 1 is referenced but not visible in the manuscript text. The caption describes it, but the actual figure content is not available for review. The paper should ensure that the figure is self-explanatory from the caption and the surrounding text.

The paper says, at §5.2 (Skill Compiler), Algorithm 1 (in Appendix)

Pith says: minor objection 2 of 10

The compiler algorithm (Algorithm 1 in Appendix §9.2) uses an LLM for three stages (catalog, obligations, synthesis). The paper states that each stage is 'one model call without retry' and that 'a failed check ends compilation.' However, the evaluation (§6.1) shows that 2 of 16 recompilations of unchanged sources were rejected (metrics-publish for a catalog defect).

more of objection 2

This means that the compiler is not deterministic: the same source can sometimes be accepted and sometimes rejected, depending on the LLM's output. The paper should discuss the reliability of the LLM-based compilation and whether the rejection rate is acceptable for a security-critical system. Also, the paper should clarify whether the LLM is called with temperature 0 (as stated for the agent, but not explicitly for the compiler).

show 8 more

The paper says, at §6.4 (RQ4: Overhead)

Pith says: minor objection 3 of 10

The overhead evaluation shows that the canonical form of commands causes a significant slowdown (1.32x) because the agent copies the verbose form into later calls. The compact form reduces this to 1.08x. The paper should clarify whether the compact form is the default/recommended mode, and whether it has any security implications (e.g., does the compact form lose any information that the gate needs?).

The paper says, at §6.3 (RQ3: Utility under Enforcement), Figure 5

Pith says: minor objection 4 of 10

Figure 5 shows that under SkillCFI with basic refusal messages, workflow-faithful completion is 74/80, which is slightly higher than under no enforcement (73/80). This is surprising: one might expect enforcement to reduce completion rates. The paper explains that 'Most outcome-only sessions come from one document-editing task, where the agent wrote its own draft instead of the supplied one, which the gate does not check.'

more of objection 4

This is a valid explanation, but the paper should also discuss whether the enforcement could actually improve completion rates by preventing the agent from going down dead-end paths.

The paper says, at §6.2 (RQ2: Enforcement Effectiveness)

Pith says: minor objection 5 of 10

The evaluation of the 'result-blind bound' (monitors that see requests but not results) is interesting. The paper states that 59 of 160 targets could be admitted by a result-blind monitor that admits all legal requests. This is a useful comparison.

more of objection 5

However, the paper should clarify whether any existing system actually implements a result-blind monitor (i.e., one that does not see results). The paper mentions SkillSentry as a system that 'credits progress from its proposals' (line [b55]), but it is not clear whether SkillSentry is result-blind. The paper should make this comparison more precise.

The paper says, at §9.5 (Additional Tables and Figures), Figure 7

Pith says: minor objection 6 of 10

Figure 7 shows the SkillCFG of the evaluation skills. The figure is described as having colored nodes and edges, but the colors are defined in the caption. The paper should ensure that the figure is legible in black-and-white print, or provide a note about color dependence.

The paper says, at §2.2 (Control-Flow Integrity)

Pith says: minor objection 7 of 10

The paper states that 'CFI enforces adherence to the graph, so an attack that stays within permitted control transfers is out of its scope' (line [b126]). This is a standard limitation of CFI. The paper should also note that this limitation applies to SkillCFI as well: an attack that uses only permitted tool calls in the permitted order is not detected. This is already implicit in the threat model, but it would be helpful to state it explicitly.

The paper says, at §3 (Threat Model)

Pith says: minor objection 8 of 10

The threat model states that 'The skill and its bundled helpers are trusted as written' (line [b142]). This is a strong assumption. The paper acknowledges that screening malicious skills is orthogonal (line [b521]). However, the paper should discuss whether a malicious skill could bypass the enforcement by, e.g., including a helper that performs a protected effect without going through the monitored tool interface.

more of objection 8

The paper assumes M3 (trusted helpers), but if the skill itself is malicious, this assumption is violated. The paper should clarify that the threat model assumes the skill is benign, and that malicious skills are a separate problem.

The paper says, at §5.5 (Security Guarantees), Theorem 1

Pith says: minor objection 9 of 10

Theorem 1 states that under assumptions M1-M5, the admitted trace belongs to Paths(C). The proof is in the appendix. The paper should ensure that the theorem statement is precise about what it guarantees: it guarantees that the admitted trace follows the contract C, not the original skill. The connection to the skill is via compilation fidelity and audit (M5). This is already discussed in §4.4 and §7, but the theorem statement could be clearer.

The paper says, at §9.3 (Compilation Example), Table 3

Pith says: minor objection 10 of 10

Table 3 traces three requirements through compilation. This is helpful for understanding the compilation process. However, the table uses identifiers like 'validatedHash' that are not defined in the main text. The paper should ensure that the table is self-contained or that the identifiers are explained.

What would settle it

If an agent can produce a protected effect (e.g., overwrite a file) through an unmediated channel such as a generated Python script or a compound shell command that the host does not route through the monitored tool interface, and the enforcer does not block it, then the central claim that SkillCFI enforces control-flow integrity for all protected effects would be falsified. The paper acknowledges this limitation (R3 regime) but the practical security guarantee depends on the host disabling all such channels.

from Pith's summary of the paper

How the review went

5 passes by deepseek-v4-flash · 19 Nov 2026

  1. Reader holds up, with conditions confidence high
  2. Skeptic concern not confirmed: listed as an open question 7 s · 46,847 tokens read, 693 written
  3. Referee 10 minor 22 s · 47,128 tokens read, 3,385 written
  4. Check against the paper kept 0 of 4 objections 27 s · 96,570 tokens read, 4,391 written
  5. Editor wrote the verdict above 5 s · 47,535 tokens read, 494 written

Not run on this paper: the simulated authors' defence.

Pith's passes raised 4 objections. A check against the paper's text kept 0, withdrew 3 (3 the paper already says) and left 1 as a question.

The review in full

The review in brief, machine-written

This paper does something genuinely useful: it takes the classical idea of control-flow integrity (CFI) and adapts it to the messy reality of LLM-based agents that follow natural-language skill procedures. The core insight is that a skill's control skeleton survives even when execution moves into prompt text, and that enforceability depends on where each step runs relative to the tool-call boundary. The authors build a propose-and-check compiler that extracts an executable contract from a skill, a deterministic runtime enforcer that checks every tool request against that contract before dispatch, and a formal model of which obligations can be enforced at the boundary.

The evaluation is thorough: 770 agent sessions, 24 source mutations, 53 public skills, human raters. The results are clean. SkillCFI blocks all 240 tested in-vocabulary violations that an allowlist admits, while preserving normal task completion rates. The compiler's diagnostics guided seven public skills into enforceable contracts. The formal model (enforceability criterion, contract automaton) is a genuine contribution that clarifies what a boundary monitor can and cannot do when the agent chooses where each step runs.

The main soft spot is the complete mediation assumption (M1): the guarantee depends on every protected effect crossing the monitored tool-call boundary. The authors acknowledge this (R3 regime) and note that unmediated channels like inline scripts must be disabled. This is a real practical limitation, but it is stated clearly. The evaluation uses self-written skills and a specific LLM (GLM-5.3-Flash), which limits generalizability, though the public skills test helps. The compiler's reliance on an LLM for proposals, even with deterministic verification, introduces a potential source of error, but the verification is thorough (P1-P7).

This paper is for researchers and practitioners working on LLM agent security. It deserves a serious referee. The formal model, the system design, and the evaluation are solid enough to engage with. The authors should address the generalizability of the evaluation and the practical implications of the complete mediation assumption in a revision, but the central claim holds up.

The stress test's concern

Not confirmed. Pith could not check this concern against the text of the paper it was given; it is listed as an open question, not a finding.

The referee's report

Not published. Pith's referee wrote this assessment before its objections were checked against the paper, and 3 of its 3 major objections did not survive that check.

Pith's reading of the paper (a machine's summary, not the authors')

The paper argues that when an LLM-based agent executes a procedural skill written in natural language, the skill's control skeleton (its required steps, order, checks, and branches) survives as a virtual control-flow graph, but its enforceability depends on where each step runs relative to the tool-call boundary. The authors present SkillCFI, which compiles a skill's prose into an executable contract and a deterministic runtime enforcer that admits a tool request only on trusted evidence from correlated results, blocking violations that permission allowlists admit. They prove that only obligations whose protected effects and prerequisite evidence cross the tool-call boundary can be enforced without trusting the model, and they close regime migration (the model moving a step out of the monitored channel) by refusing unmediated channels such as inline scripts. In evaluation, SkillCFI blocked all 240 tested procedural violations before dispatch while preserving normal task completion rates, and the compiler's diagnostics guided seven public skills into enforceable contracts.

What carries the argument. The central mechanism is the two-layer executable contract: a state-independent BindingCatalog that maps tool calls to typed boundary events, and a state-aware SkillCFG automaton (a guarded register automaton) that governs the workflow. The BindingCatalog defines base interfaces with deterministic result decoders, typed capture registers, and value-relation predicates. The SkillCFG contract is a finite-state automaton with request nodes, result nodes, guarded edges with counter updates, and exit transitions. The runtime enforcer maintains a configuration of control location, last decoded outcome, counters, and register valuations, and checks each request against four layers: common mediation integrity (C), static interface membership (S), value relations (R), and workflow control flow (F). The compiler verifies seven acceptance properties (P1-P7) including disjoint recognition, deterministic admission, evidence availability, and terminal consistency, and renders the agent's instructions from the same contract so that the agent and monitor share the same call forms.

Core claim. The paper's central claim is that control-flow integrity (CFI) can be carried over to LLM-based agent skills by confining LLM interpretation to compile time, deriving an executable contract from the skill's prose, and enforcing it with a deterministic runtime monitor at the tool-call boundary. The key insight is that a skill's control skeleton (its mandatory steps, order, checks, and branches) is a fixed graph like a program's CFG, but the enforceability of each obligation depends on whether its protected effect and prerequisite evidence cross the tool-call boundary. The authors formalize this as a boundary enforceability criterion (Proposition 1) and design a propose-and-check compiler that classifies each obligation as enforceable, guidance, or unsupported, rejecting a skill whose mandatory obligation leaves nothing to check. The runtime enforcer checks every tool request against the contract in four layers (integrity, interface membership, value relations, and workflow control flow), admits only requests on trusted evidence from correlated results, and refuses unmediated channels such as inline scripts, so admitted traces follow the contract whatever the model proposes.

The premise it rests on. The system assumes that every protected effect must cross the tool-call boundary (M1: complete mediation); if an agent can produce a protected effect through an unmediated channel (e.g., a generated script that writes to a file without going through the monitored tool interface), the enforcer cannot block it.

If the paper is right (Pith's inferences, not the authors')

SkillCFI blocks all in-vocabulary procedural violations (skipped prerequisites, wrong branches, exceeded bounds, wrong objects) that permission allowlists admit, because it checks the workflow history and value relations before dispatch.

A stateless policy (deciding on one request at a time) must admit all 160 tested violations, and a result-blind monitor (seeing requests but not results) must admit 59 of them, while SkillCFI blocks all 240.

The compiler's diagnostics point to missing interfaces or evidence at the tool-call boundary, guiding skill authors to add wrappers that return receipts with digests, approvals through native questions, and completion acknowledgments.

Normal tasks complete nearly as often with enforcement as without (74 vs 73 workflow-faithful completions out of 80), and listing the next legal steps on refusal raises recovery from 21% to 65%.

The gate takes about 2% of task time, and a compact command form cuts session slowdown from 1.32x to 1.08x.

The paper's enforceability criterion implies that as agent hosts expose more of the execution (e.g., internal reasoning steps or generated script internals) at the boundary, more obligations become enforceable without trusting the model. This suggests a roadmap for widening the mediation boundary rather than relying on the model's honesty.

The finding that none of 53 public skills is accepted as written, and that the gaps are mostly missing receipts and call forms rather than semantic constraints, suggests that skill authors and tool designers can build enforceability in with thin wrappers that return structured evidence.

The paper's approach of compiling a contract once and rendering the agent's instructions from the same contract solves the 'call-site ambiguity' problem that classical system-call monitors face, because the agent is taught exactly the call forms the monitor recognizes.

The result that a result-blind monitor must admit 59 of 160 violations (mostly because a user answer or helper outcome differs) quantifies the value of correlated results over request-only monitoring, and suggests that history-aware monitors that observe results can express these orderings given a correct policy.

Check this review

What was read

The passes were given all 159,729 characters of the paper's LaTeX source.

Fingerprint of that text (SHA-256): 3bcb4df98ba11fca5af1abc33862a70970d5e8fabde07b1722e7f393ea375bec

Pith does not keep that text. Rebuilt from the paper's source or PDF by the same code, it has this fingerprint if it is the same text. From a copy of Pith's code:

PYTHONPATH=. python scripts/review_input_check.py 2611.24185

The passes, in the order they ran

  1. Reader deepseek-v4-flash · medium effort · $0.0049 · instructions of the Reader, fingerprint f9ead7300382…
  2. Skeptic deepseek-v4-flash · medium effort · $0.0044 · instructions of the Skeptic, fingerprint 8f88c0f61488…
  3. Referee deepseek-v4-flash · medium effort · $0.0049 · instructions of the Referee, fingerprint a852d99aea0d…
  4. Check against the paper deepseek-v4-flash · medium effort · $0.0061 · instructions of the Check against the paper, fingerprint 6c7a6883d551…
  5. Editor deepseek-v4-flash · medium effort · $0.0044 · instructions of the Editor, fingerprint e7211f6a3d27…
  6. Circularity check deepseek-v4-flash · medium effort · $0.0043 · instructions of the Circularity check, fingerprint e9946e39d67b…
  7. Summary for the page deepseek-v4-flash · medium effort · temperature 0.2 · $0.0048 · instructions of the Summary for the page, fingerprint e4f56bf568e7…

The passes cost $0.034 in model calls together.

Version stamped on this review: pith-pipeline@v1.4.1-daily-deepseek-or-medium+deepseek/deepseek-v4-flash+p.d60f54226f00

Each link opens a pass's instructions as plain text, the exact bytes its fingerprint was taken over. Every stored set of instructions.

curl -s https://pith.science/methodology/prompts/f9ead730038216984cfb56d7cb34332bdf691828296a1de4245f3d6667d4bf56 | sha256sum

prints the fingerprint of the reader's instructions. The same works for every pass.

The receipt

The review's fingerprint was taken on 19 Nov 2026. The review you are reading still matches it.

Fingerprint of the review (SHA-256): 927e85e30c8739312fcdf04c79c19106755d9aa039748d04e0b57883d721b7b3

curl -s 'https://pith.science/api/v1/verdicts/2611.24185?form=canonical' | sha256sum

prints the fingerprint of the review as it stands now. The rule, in four steps.

A matching fingerprint shows the review has not changed since that date. It does not show that the review is right.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, from the paper's LaTeX source.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Control-flow integrity (CFI) confines a program to a control-flow graph fixed ahead of time. Procedural agent skills move this control flow into prompt text, and a large language model (LLM) agent decides at runtime whether each step runs in its reasoning, a tool call, or a script. The procedure's control skeleton survives, but its enforceability depends on where each step runs. Many violations, such as overwriting a file before validation, use only operations the skill requires, so permission allowlists admit them. A deviating or prompt-injected model can also evade a monitor by regime migration, for example by claiming that a check passed. We present SkillCFI, which carries CFI over to skill-based agents. Its compiler confines LLM interpretation to compile time and derives the agent's instructions and an executable contract from one skill. By a formal enforceability criterion, it enforces obligations whose evidence crosses the tool boundary and rejects a skill whose mandatory obligation leaves nothing to check. A deterministic runtime enforcer admits a tool request only on trusted evidence and refuses unmediated channels such as inline scripts, so admitted traces follow the contract whatever the model proposes. On eight self-written skills, no compiled contract permits more than its source under 24 source mutations. In 770 agent sessions, SkillCFI blocks before dispatch all 240 such violations, which an allowlist admits, and normal tasks complete nearly as often as without it. Listing the next legal steps in a refusal raises recovery from 21% to 65%. Checking takes about 2% of task time, and a compact command form cuts session slowdown from 1.32 times to 1.08 times. None of 53 public skills is accepted as written. The compiler's diagnostics point to interfaces rather than workflows and guided seven into enforceable contracts.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-11-19

The authors' main claim, as the classifier picked it out 0.90

“In 770 agent sessions, SkillCFI blocks before dispatch all 240 such violations, which an allowlist admits, and normal tasks complete nearly as often as without it.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.12 · states limitation 0.80 · human participants 0.10 · controlled comparison 0.85 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.81 · mentions code or data availability 0.05

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Control-flow integrity (CFI) confines a program to a control-flow graph fixed ahead of time. background 1.00 0.05 0.03 asserted 1.00
Procedural agent skills move this control flow into prompt text, and a large language model (LLM) agent decides at runtime whether each step runs in its reasoning, a tool call, or a script. background 0.99 0.09 0.03 asserted 1.00
The procedure's control skeleton survives, but its enforceability depends on where each step runs. background 0.82 0.10 0.05 asserted 0.99
Many violations, such as overwriting a file before validation, use only operations the skill requires, so permission allowlists admit them. background 0.87 0.17 0.03 asserted 0.95
A deviating or prompt-injected model can also evade a monitor by regime migration, for example by claiming that a check passed. background 0.87 0.10 0.03 hedged possibility 0.72
We present SkillCFI, which carries CFI over to skill-based agents. contribution 1.00 0.90 0.15 asserted 1.00
Its compiler confines LLM interpretation to compile time and derives the agent's instructions and an executable contract from one skill. method 0.74 0.90 0.12 asserted 1.00
By a formal enforceability criterion, it enforces obligations whose evidence crosses the tool boundary and rejects a skill whose mandatory obligation leaves nothing to check. method 0.84 0.89 0.18 asserted 1.00
A deterministic runtime enforcer admits a tool request only on trusted evidence and refuses unmediated channels such as inline scripts, so admitted traces follow the contract whatever the model proposes. method 0.97 0.81 0.21 asserted 1.00
On eight self-written skills, no compiled contract permits more than its source under 24 source mutations. result 1.00 0.94 0.13 asserted 1.00
In 770 agent sessions, SkillCFI blocks before dispatch all 240 such violations, which an allowlist admits, and normal tasks complete nearly as often as without it. result 1.00 0.95 0.42 asserted 1.00
Listing the next legal steps in a refusal raises recovery from 21% to 65%. result 0.96 0.87 0.05 asserted 1.00
Checking takes about 2% of task time, and a compact command form cuts session slowdown from 1.32 times to 1.08 times. result 1.00 0.93 0.04 asserted 1.00
None of 53 public skills is accepted as written. result 0.98 0.86 0.07 asserted 1.00
The compiler's diagnostics point to interfaces rather than workflows and guided seven into enforceable contracts. result 0.80 0.91 0.07 asserted 1.00

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.98

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-19: 1 request, 10,757 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (10)
Figure 1 of the paper
Figure 1. One check-and-repair procedure as a program and as a skill. (a) An independent monitor checks program execution against an explicit control-flow graph. (b) The model decides where each step runs. Relative to the tool￾call boundary B, a step in reasoning (R1) leaves no event, a tool call (R2) exposes its request and result, and a helper (R3a) or generated script (R3b) exposes only its outer call. graph and no such monitor. Its procedure exists only as prompt text, and the agent decides at runtime whether to compute each step in its own reasoning, through a tool call, or inside a…
Figure 2 of the paper
Figure 2. The document-editing skill, which revises the
Figure 3 of the paper
Figure 3. shows SkillCFI, which confines LLM-based interpretation of skill prose to offline compilation, so that online enforcement is a deterministic reference monitor at B. Offline, a propose-and-check compiler (§ 5.2) turns a trusted SourceSkill and host capabilities H into a pinned SkillBundle. The bundle holds the agent-facing AgentSkill and the executable contract (§ 5.1). Online, the host loads AgentSkill into the prompt. The runtime enforcer (§ 5.3) checks every tool request against the con￾tract and returns the next legal steps on refusal (§ 5.4). 5.1. Executable Contract SkillCFI…
Figure 4 of the paper
Figure 4. Two-layer executable contract for the document
Figure 5 of the paper
Figure 5. Left: the 24 single-point mutations by skill and
Figure 6 of the paper
Figure 6. Utility under enforcement. (a) Normal routes per condition. Outcome-only sessions reach outcome comple￾tion but deviate from the workflow, so the first two segments together give outcome completion. (b) Sessions after the target refusal under SKILLCFI, per refusal message. After further refusals means that the expected ending was reached after at least one more refusal. Each bar has 80 sessions. 6.4. RQ3: Utility under Enforcement Outcome completion means a normal session ends with the expected result, and its receipts, file effects, delivery, and approval binding pass the task’s…
Figure 7 of the paper
Figure 7. SkillCFG of the evaluation skills at occurrence level: (a) document-editing (validate, ≤3 repairs → ask → deliver → verify); (b) data-reconciliation (validate, ≤3 repairs → preview → ask → deliver); (c) report-render-publish (check, revise freely → publish or keep a draft); (d) metrics-publish (check → schema or formula repair → export); (e) meeting-followup (ask mode → approve preview → write); (f) outreach-email (check → preview → approve → send once); (g) workspace-tidy (approve plan → move → log; stop on conflict); (h) backup-rotate (copy → verify → approve → delete or retain).…
Figure 9 of the paper
Figure 9. Task time in the dispatch ablation, split into model
Figure 10 of the paper
Figure 10. Two sessions of the document-editing task that

Showing 9 of the paper's 10 figures. Figures are the authors', reproduced from the paper.

References (49)
  1. [1] S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. R. Narasimhan, and Y . Cao, “ReAct: Synergizing reasoning and acting in language models,” inThe Eleventh International Conference on Learning Rep- resentations, ICLR 2023. OpenReview.net, 2023 on Pith
  2. [2] X. Wang, Y . Chen, L. Yuan, Y . Zhang, Y . Li, H. Peng, and H. Ji, “Executable code actions elicit better LLM agents,” inForty-first International Conference on Machine Learning, ICML 2024, 2024, pp. 50 208–50 232 on Pith
  3. [3] Y . Tang, K. Zhu, B. Ruan, C. Zhang, M. Yang, H. Li, S. Guo, T. Shi, Z. Li, C. Kruegelet al., “Devops-gym: Benchmarking ai agents in software devops cycle,” inInternational Conference on Learning Representations, vol. 2026, 2026, pp. 13 021–13 045 on Pith
  4. [4] AgentSkills, “Overview,” https://agentskills.io/home, 2026 on Pith
  5. [5] Y . Jiang, D. Li, H. Deng, B. Ma, X. Wang, Q. Wang, and G. Yu, “SoK: Agentic Skills – Beyond Tool Use in LLM Agents,” Feb on Pith
  6. [6] S. Yao, N. Shinn, P. Razavi, and K. Narasimhan, “τ-bench: A benchmark for Tool-Agent-User interaction in real-world domains,” in International Conference on Learning Representations (ICLR), 2025 on Pith
  7. [7] J. Dantanarayana, S. Kashmira, L. Tang, and J. Mars, “SIGIL: Skill compilation for reliable and efficient agent execution,” 2026. [Online]. Available: https://arxiv.org/abs/2607.27309 on PithPith's reviewarXiv
  8. [8] H. Li, X. Ma, Y . Wu, W. Zhong, F. Jiang, T. M. Dang, X. Hu, H. Ma, Y . Guo, and J. Huang, “Who holds the pen? Let specifications, not agents, sign off,”arXiv preprint arXiv:2609.29921, 2026 on PithPith's reviewarXiv
  9. [9] K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, and M. Fritz, “Not what you’ve signed up for: Compromising real- world LLM-integrated applications with indirect prompt injection,” inProceedings of the 16th ACM workshop on artificial intelligence and security, 2023, pp. 79–90 on Pith
  10. [10] E. Debenedetti, J. Zhang, M. Balunovic, L. Beurer-Kellner, M. Fis- cher, and F. Tram`er, “AgentDojo: A dynamic environment to evaluate prompt injection attacks and defenses for LLM agents,”Advances in neural information processing systems, vol. 37, pp. 82 895–82 920, 2024 on Pith
  11. [11] T. Shi, J. He, Z. Wang, H. Li, L. Wu, W. Guo, and D. Song, “Progent: Securing AI agents with privilege control,”arXiv preprint arXiv:2504.11703, 2025 on PithPith's reviewarXiv
  12. [12] J. Wu, Y . Nan, Y . Lin, H. Wang, Y . Xiao, S. Wang, and Z. Zheng, “SkillScope: Toward fine-grained least-privilege enforcement for agent skills,” inProceedings of the 2026 ACM SIGSAC Conference on Computer and Communications Security (CCS), 2026 on Pith
  13. [13] S. Pan, X. Sun, T. Zhang, D. Liao, K. Yang, and Z. Xing, “Skill- Guard: A permission-centric framework for agent skill security,” arXiv preprint arXiv:2606.03024, 2026 on PithPith's reviewarXiv
  14. [14] M. Abadi, M. Budiu, ´U. Erlingsson, and J. Ligatti, “Control-flow integrity principles, implementations, and applications,”ACM Trans. Inf. Syst. Secur., vol. 13, no. 1, pp. 4:1–4:40, 2009 on Pith
  15. [15] Y . Lu, X. Huang, B. Chen, and X. Peng, “SkillSentry: Reliable skill execution for LLM agents via runtime assurance,”arXiv preprint arXiv:2608.09253, 2026 on PithPith's reviewarXiv
  16. [16] H. Wang, C. M. Poskitt, and J. Sun, “AgentSpec: Customizable run- time enforcement for safe and reliable LLM agents,” inProceedings of the 2026 IEEE/ACM 48th International Conference on Software Engineering, 2026, pp. 2938–2950 on Pith
  17. [17] N. Palumbo, S. Choudhary, J. Choi, G. Amir, P. Chalasani, I. Dillig, and S. Jha, “Formal policy enforcement for agentic systems,” 2026. [Online]. Available: https://arxiv.org/abs/2602.16708 on PithPith's reviewarXiv
  18. [18] Y . Li, Y . Chen, H. Wen, B. Zhang, H. Liu, P. Wang, Y . Feng, and Y . Tian, “VIGIL: Runtime enforcement of behavioral specifications in AI agent skills,”arXiv preprint arXiv:2606.26524, 2026 on PithPith's reviewarXiv
  19. [19] F. B. Schneider, “Enforceable security policies,”ACM Transactions on Information and System Security (TISSEC), vol. 3, no. 1, pp. 30– 50, 2000 on Pith
  20. [20] D. Basin, V . Jug ´e, F. Klaedtke, and E. Z ˘alinescu, “Enforceable security policies revisited,”ACM Transactions on Information and System Security (TISSEC), vol. 16, no. 1, pp. 1–26, 2013 on Pith
show 29 more
  1. [21] J. Ligatti and S. Reddy, “A theory of runtime enforcement, with results,” inEuropean Symposium on Research in Computer Security. Springer, 2010, pp. 87–100 on Pith
  2. [22] OpenCode, “Plugins,” https://opencode.ai/docs/plugins/, 2026 on Pith
  3. [23] N. Carlini, A. Barresi, M. Payer, D. Wagner, and T. R. Gross, “Control-flow bending: On the effectiveness of control-flow integrity,” in24th USENIX Security Symposium (USENIX Security 15), 2015, pp. 161–176 on Pith
  4. [24] J. P. Andersonet al., “Computer security technology planning study,” ESD-TR-73-51, Tech. Rep., 1972 on Pith
  5. [25] J. H. Saltzer and M. D. Schroeder, “The protection of information in computer systems,”Proceedings of the IEEE, vol. 63, no. 9, pp. 1278–1308, 1975 on Pith
  6. [26] Y . Ruan, H. Dong, A. Wang, S. Pitis, Y . Zhou, J. Ba, Y . Dubois, C. Maddison, and T. Hashimoto, “Identifying the risks of LM agents with an LM-emulated sandbox,” inInternational Conference on Learning Representations, vol. 2024, 2024, pp. 27 031–27 098 on Pith
  7. [27] D. Wagner and R. Dean, “Intrusion detection via static analysis,” in Proceedings 2001 IEEE Symposium on Security and Privacy. S&P on Pith
  8. [28] D. Wagner and P. Soto, “Mimicry attacks on host-based intrusion detection systems,” inProceedings of the 9th ACM Conference on Computer and Communications Security, 2002, pp. 255–264 on Pith
  9. [29] M. Kaminski and N. Francez, “Finite-memory automata,”Theoretical Computer Science, vol. 134, no. 2, pp. 329–363, 1994 on Pith
  10. [30] R. Sekar, M. Bendre, D. Dhurjati, and P. Bollineni, “A fast automaton- based method for detecting anomalous program behaviors,” inPro- ceedings 2001 IEEE Symposium on Security and Privacy. S&P 2001. IEEE, 2001, pp. 144–155 on Pith
  11. [31] OpenClaw, “openclaw/agent-skills,” https://github.com/openclaw/ agent-skills, 2026 on Pith
  12. [32] OpenAI, “openai/skills,” https://github.com/openai/skills, 2026 on Pith
  13. [33] Q. Zhan, R. Fang, H. S. Panchal, and D. Kang, “Adaptive attacks break defenses against indirect prompt injection attacks on LLM agents,” inFindings of the Association for Computational Linguistics: NAACL 2025, ser. Findings of ACL, vol. NAACL 2025. Association for Computationa... on Pith
  14. [34] J. Shi, Z. Yuan, Y . Liu, Y . Huang, P. Zhou, L. Sun, and N. Z. Gong, “Optimization-based prompt injection attack to LLM-as-a-judge,” in Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, 2024, pp. 660–674 on Pith
  15. [35] B. Ruan, Y . Fu, C. Zhang, J. Liu, J. Zeng, and Z. Liang, “Heimdallr: Characterizing and detecting llm-induced security risks in github ci workflows,” 2026. [Online]. Available: https: //arxiv.org/abs/2605.05969 on PithPith's reviewarXiv
  16. [36] Y . Liu, Z. Chen, Y . Zhang, G. Deng, Y . Li, J. Ning, and L. Y . Zhang, ““Do not mention this to the user”: Detecting and understanding ma- licious agent skills in the wild,” in35th USENIX Security Symposium, USENIX Security 2026, E. M. Redmiles and B. Stock, Eds. USENIX Asso... on Pith
  17. [37] D. Schmotz, L. Beurer-Kellner, S. Abdelnabi, and M. An- driushchenko, “Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks,” Feb. 2026, arXiv:2602.20156 [cs]. [Online]. Available: http://arxiv.org/abs/2602.20156 on PithPith's reviewarXiv
  18. [38] W. Zhao, J. Liu, B. Ruan, S. Li, and Z. Liang, “When mcp servers attack: Taxonomy, feasibility, and mitigation,” 2025. [Online]. Available: https://arxiv.org/abs/2509.24272 on PitharXiv
  19. [39] J. Crampton, “A reference monitor for workflow systems with con- strained task execution,” in10th ACM Symposium on Access Control Models and Technologies, SACMAT 2005. ACM, 2005, pp. 38–47 on Pith
  20. [40] P. W. Fong, “Access control by tracking shallow execution history,” in IEEE Symposium on Security and Privacy, 2004. Proceedings. 2004. IEEE, 2004, pp. 43–55 on Pith
  21. [41] J. Kim, W. Guo, and D. Song, “SoK: Attack and defense landscape of agentic AI systems,” in35th USENIX Security Symposium (USENIX Security 26). USENIX Association, 2026, pp. 6047–6066 on Pith
  22. [42] Y . Wu, F. Roesner, T. Kohno, N. Zhang, and U. Iqbal, “IsolateGPT: An execution isolation architecture for LLM-based agentic systems,” in32nd Annual Network and Distributed System Security Symposium, NDSS 2025. The Internet Society, 2025 on Pith
  23. [43] E. Debenedetti, I. Shumailov, T. Fan, J. Hayes, N. Carlini, D. Fabian, C. Kern, C. Shi, A. Terzis, and F. Tram `er, “Defeating prompt in- jections by design,” inIEEE Conference on Secure and Trustworthy Machine Learning, SaTML 2026. IEEE, 2026, pp. 587–618 on Pith
  24. [44] E. Li, T. Mallick, E. Rose, W. K. Robertson, A. Oprea, and C. Nita- Rotaru, “ACE: A security architecture for LLM-integrated app sys- tems,” in33rd Annual Network and Distributed System Security Symposium, NDSS 2026. The Internet Society, 2026 on Pith
  25. [45] J. Liu, B. Ruan, X. Yang, Z. Lin, Y . Liu, Y . Wang, T. Wei, and Z. Liang, “TraceAegis: Securing LLM-Based Agents via Hierarchical and Behavioral Anomaly Detection,” Oct. 2025, arXiv:2510.11203 [cs]. [Online]. Available: http://arxiv.org/abs/2510.11203 on PitharXiv
  26. [46] R. Sequeira, S. Damianakis, U. Iqbal, and K. Psounis, “Agent- Sentry: Bounding LLM Agents via Execution Provenance,” Mar. 2026, arXiv:2603.22868 [cs]. [Online]. Available: http://arxiv.org/ abs/2603.22868 on PithPith's reviewarXiv
  27. [47] R. Jha, H. Triedman, J. Wagle, and V . Shmatikov, “Breaking and fixing defenses against control flow hijacking in multi-agent systems,” inInternational Conference on Learning Representations, 2026. Appendix A. Notation Table 3 lists the notation of the paper. Appendix B. Compi... on Pith
  28. [2001] IEEE, 2001, pp. 156–168 on Pith
  29. [2026] [Online]. Available: http://arxiv.org/abs/2602.20867 on PithPith's reviewarXiv

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment