Pith. sign in

REVIEW 3 major objections 4 minor 33 references

A static-first verifier can block dangerous LLM-agent shell commands at the dispatch boundary in milliseconds while preserving most benign workflows.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

CARE verifies shell commands from LLM agents in milliseconds, catching dangerous ones with 85.64% F1 and 0.91% false-positive rate while preserving most benign workflows.

T0 review reviewed 2026-08-01 challenge →

load-bearing objection Real system, credible engineering, but the headline F1 rests on an unquantified rule-bank overlap with the main eval source; the OOD numbers tell a more honest story. the 3 major comments →

arxiv 2607.21642 v1 pith:XRKA4A4Z submitted 2026-07-21 cs.CR

CARE: Pre-Execution Command Verification for Shell-Executing LLM Agents

Wenxiao Zhang , Yu Liu , Zhiwei Yang , Zhongyi Zhang , Hanqi Feng , Xinyu Wang , Peng Qiu , Yanbing Liu
show 2 more authors
Barnabas Poczos Jin B. Hong
This is my paper
classification cs.CR
keywords shell command safetyLLM agentspre-execution verificationcanonicalizationstatic analysisruntime guardrailcommand dispatchfalse positive rate
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that shell command dispatch is the critical safety boundary for LLM agents, and that a shell-specific, static-first verifier can mediate individual commands before execution. CARE canonicalizes each command into a stable non-executed form, then derives deterministic evidence over syntax, command semantics, path targets, and provenance-backed risk patterns, escalating only borderline commands to an LLM judge. On a balanced split, CARE reaches 85.64% F1 with a 0.91% false-positive rate at 2.32 ms mean latency; a fully static profile retains 84.99% F1 at 0.34 ms and reduces realised harm on LLM-generated attack commands in a container sandbox to 37.33%. If these results hold, a cheap deterministic filter can catch most dangerous commands with a very low burden on safe work.

Core claim

CARE's central claim is that a layered deterministic analysis of the canonicalized command string is sufficient to assign most commands a high-confidence verdict, and that neural adjudication is only needed for genuinely ambiguous cases. The system computes a composite risk score from four evidence signals: AST structure, a curated semantic lexicon over command heads and flags, a path-sensitive read/write analysis of file targets, and a provenance-weighted pattern bank. Provisional ALLOW and DENY decisions are finalized statically; only WARN-band commands are escalated, and skip predicates ensure that commands with already-conclusive static evidence never incur an LLM call. The paper reports

What carries the argument

The Canonicalization–Attribution–Resolution pipeline is the load-bearing object. Bounded canonicalization unwraps shell wrappers, normalizes quoting and whitespace, and applies light deobfuscation to produce a stable verification target without executing it. Attribution combines L1 structural (AST) penalties, L2 semantic risk scores, L3 path-context scores that treat reads and writes asymmetrically, and L4 provenance-tiered pattern rules into an additive composite score; L5 maps the score to ALLOW/WARN/DENY thresholds. Resolution preserves high-confidence verdicts and uses skip predicates to deny conclusively dangerous WARN cases without an LLM, reserving the judge for residual ambiguity.

Load-bearing premise

The load-bearing premise is that the fixed, bounded set of canonicalization rewrites is sufficient to expose the underlying risk in adversarial commands; a command using an unlisted encoding, deeply nested substitution, or a custom eval loop may be analyzed in a form where the static signals are absent and could reach execution.

What would settle it

Construct a destructive `rm -rf` equivalent using an obfuscation technique not in CARE's rewrite list—for example a new encoding, an unusually deep command-substitution nest, or a custom eval loop—and check whether the guard denies it before execution. If such a command is allowed and executes harmful behaviour, the bounded-canonicalization assumption is violated. A systematic version would measure recall on a held-out corpus of adversarially novel obfuscations.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Deployments can choose between two profiles: full CARE for low false-positive benign-task recovery and static-only CARE for zero-LLM, conservative harm reduction.
  • Because the guard is a single-command, stateless check, it composes with trajectory-level defenses and OS-level sandboxing rather than replacing them.
  • The static evidence stack alone accounts for most of the detection, implying deterministic analysis can handle the common case without a model call.
  • The read/write path asymmetry prevents a common false positive (blocking `cat /etc/os-release`) while still flagging writes to sensitive files.
  • The paper's finding that the LLM judge sometimes over-allows commands on paraphrase-shared attack shifts suggests that resolution should be tuned per deployment context.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The bounded-canonicalization assumption could be stress-tested by generating novel obfuscation families (new encodings, deep nesting, custom eval loops) and measuring whether CARE's static signals still fire; this is the most direct way to probe the method's limit.
  • A natural extension is to make the Resolution policy conditional on the command generator: if generator and judge share the same model distribution, the static-only profile may be safer, as the paper's RedCode-gen result suggests.
  • The provenance-weighted rule bank is a transferable design: assigning authority weights to rules based on external catalogs could be adapted to other agent tool boundaries, not just shell dispatch.
  • Because CARE sees only the command and bounded path context, risks carried in shell state, environment variables, or multi-command trajectories remain out of scope; an adversary could attempt to shift harm into those channels.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper presents CARE (Canonicalization, Attribution, and Resolution Engine), a static-first, pre-execution verifier for shell commands generated by LLM agents. CARE canonicalizes each command, extracts deterministic multi-view evidence (syntax, semantic classification, path sensitivity, provenance-tagged pattern rules), and selectively escalates only WARN-band cases to an LLM judge. The main claimed results are 85.64% F1 with 0.91% FPR at 2.32 ms mean latency on a balanced main split, and 84.99% F1 at 0.34 ms in a static-only profile, with a realised-harm reduction on RedCode-gen to 37.33%. The evaluation compares against 12 baselines, including LLM judges, content-safety classifiers, and static rule guards, across a leakage-controlled main split, a utility benchmark, OOD corpora, obfuscation suites, and a Docker-executed attack-command track.

Significance. If the headline results are robust, CARE is a meaningful contribution: it shows that a shell-specific, deterministic, low-latency verifier can occupy a practically useful region of the detection–FPR–latency trade-off, with an auditable evidence trace and a selective LLM escalation mechanism. The paper is unusually transparent: it releases the rule bank, evaluation splits, and prompts; reports Wilson intervals for OOD FPR; and explicitly discusses known threat-model limits (session-level attacks, bounded canonicalization, judge/generator overlap). These strengths make the work reproducible and the claims checkable. The main risk is that the headline performance on the main split may be inflated by rule-bank overlap with the evaluation corpora, and the FPR estimates are based on very few false positives, so the deployment-ready claim is conditionally supported rather than fully established.

major comments (3)
  1. [§IV-B, §IV-D, Table II] The main-split dangerous set is 208/220 commands from ART and RedCode-Exec, while the rule bank's largest tier is 92 MITRE ATT&CK rules. Because ART is MITRE-aligned and RedCode-Exec is also a public offensive corpus, the rule-authoring and evaluation sources overlap. §VI admits 'source overlap is mitigated but possible,' but no quantification is given. The headline 85.64% F1 on Table II could partly reflect rule memorization rather than generalization. Please report: (i) the fraction of main-split dangerous commands that trigger MITRE-tier rules; (ii) an evaluation where rules are re-authored without access to ART/RedCode examples, or a leave-one-source-out analysis; (iii) the same metrics for GTFOBins and Exploit-DB, whose F1 values (68.91 and 66.33) are materially lower. This is load-bearing because the abstract and deployment claims rest on the main-split numbers.
  2. [§V-A1, Table II, Appendix Table VIII] With 329 safe commands, the 0.91% FPR is 3 false positives and 1.82% FPR is 6 false positives. No confidence interval is reported for these main-split FPRs; a 95% Wilson interval for 3/329 is approximately [0.2%, 2.7%], which overlaps rates that are not 'low' in absolute terms. Moreover, Appendix Table VIII reports p=0.25 for CARE vs. CARE (w/o Resolution), so the Resolution-stage FPR halving is not statistically significant. The paper correctly states this in the text, but the abstract's unqualified '0.91% false-positive rate' overstates precision. Report Wilson intervals for the main-split FPR and explicitly present the 0.91% as a noisy point estimate.
  3. [Table V, §V-C] On the three OOD mixed-label corpora, CARE's FPR is 13.04%, 11.60%, and 8.13% (full) and 15.94%, 13.20%, and 8.94% (static-only) — roughly 10–100 times the 0.91% main-split FPR. The deployment-oriented discussion in §V-C acknowledges this, but the abstract and introduction present the 0.91% rate without this qualification. The main-split FPR is an in-distribution diagnostic, not a credible deployment FPR under distribution shift. Please add a sentence to the abstract/introduction noting that OOD FPR is in the 8–16% range, and clarify that the headline FPR is specific to the balanced, leakage-controlled split.
minor comments (4)
  1. [References [25]] Reference [25] is titled 'Qwen3-4B-Instruct' but the text and Appendix A.6 use Qwen3-Coder-30B-A3B-Instruct. The reference does not match the model actually used.
  2. [Figure 3] Figure 3 contains garbled placeholder strings (e.g., 'out: ? ? , ???????') that appear to be rendering artifacts. The figure should be regenerated so labels are legible.
  3. [§V-C3, paragraph after Table V] The sentence 'Full CARE is 15.17 pp higher than CARE (w/o Resolution)' should specify 'realised-harm rate is 15.17 pp higher' to avoid ambiguity with F1.
  4. [§VI, Scope and limitations] The limitation that canonicalization is bounded and not full semantic recovery is correctly stated, but a concrete example of a command class outside the rewrite families (e.g., deep custom eval loops or novel encoding) would help readers calibrate the residual evasion surface.

Circularity Check

0 steps flagged

No significant circularity: CARE's decision equations are self-contained and the evaluation includes independent held-out/OOD controls; the acknowledged rule-bank/eval overlap is a leakage caveat, not a by-construction reduction.

full rationale

CARE's decision procedure is defined by its own equations (1)-(12), and the headline F1 is an empirical result rather than a quantity equal to an input by construction. The only load-bearing contamination concern is rule-bank/eval source overlap: §IV-D says the lexicon is "curated from public offensive corpora, benchmark-derived error analysis, and expert annotation" and the pattern layer contains "92 MITRE ATT&CK rules", while the main split contains 137 ART commands tagged with MITRE TIDs. If MITRE rules were authored with knowledge of ART examples, main-split recall would partly measure rule memorization. However, the paper nowhere defines a fitted parameter as a function of the eval F1, and it takes explicit anti-leakage steps that break any by-construction equivalence: the main split is split at template-family level, GTFOBins uses a held-out binary-hash bucket ("the 69 entries in the held-out bucket ... share no binary with the rule-authoring bucket"), Exploit-DB detections reflect pattern generalization because the rule bank "carries MITRE-, GTFOBins-, and manual-provenance rules only", and the 12 canonical eval attacks are stated to be "distinct from the 16 Manual Cases rules". The paper also discloses the residual risk in §VI: "source overlap is mitigated but possible, motivating three non-MITRE benchmarks (GTFOBins, Obfuscation, Exploit-DB)". That is a correctness/leakage caveat, not a demonstrated circularity. No load-bearing self-citation chain, imported uniqueness theorem, ansatz-via-citation pattern, or definitional equation collapse appears; the Qwen3 generator-judge overlap is acknowledged and the static-only profile provides an LLM-free anchor. Accordingly, no circular step can be exhibited with a specific equation-level reduction.

Axiom & Free-Parameter Ledger

7 free parameters · 5 axioms · 0 invented entities

All system hyperparameters are hand-set or dev-tuned; no learned parameters from eval. The central claim relies on these choices.

free parameters (7)
  • Aggregation weights (w_sem, w_path, w_pat, w_struct) = 0.30/0.30/0.30/0.10 (balanced mode)
    Tuned on dev split (Section IV-D); sum to one.
  • Decision thresholds (tau_low, tau_high) = 0.15, 0.35 (balanced mode)
    Tuned on dev split; sensitivity in Table VI(b).
  • Resolution thresholds (theta_rule, theta_sem) = 0.80, 0.70
    Fixed operating point; Section IV-D and Appendix A.6.
  • Provenance weights (pi_MITRE, pi_GTFOBins, pi_Manual) = 1.00, 0.85, 0.60
    Assigned by tier; Appendix A.4.
  • Semantic base risk scores (beta_k) = e.g., DESTRUCTIVE=1.00, READ ONLY=0.00, UNKNOWN=0.35
    Curated lexicon; Appendix A.2.
  • Path sensitivity gamma values = e.g., critical read/write=1.00, sensitive-system read=0.10/write=0.70
    Hand-set; Appendix A.3.
  • Structural indicator severities (alpha_i) = e.g., pipe-to-interpreter=1.00, eval=0.90, plain pipe=0.05
    Hand-set; Appendix A.1.
axioms (5)
  • domain assumption bashlex correctly parses POSIX shell syntax after canonicalization.
    L1 relies on bashlex AST; parse failures fail closed only if high-risk tokens are present (Algorithm 1).
  • domain assumption Bounded deployment context (workspace root, home, symlink metadata) is sufficient for path normalization.
    Section II states the guard sees only command and bounded context; incomplete symlink resolution could misclassify paths.
  • domain assumption Public offensive catalogs (MITRE ATT&CK, GTFOBins) provide representative coverage of dangerous shell commands.
    Rule bank and evaluation both draw from these catalogs; generalization to unseen attacks is assumed beyond these sources.
  • domain assumption The LLM judge's binary decisions on WARN-band commands are reliable (or default-to-DENY is safe).
    Resolution uses a single LLM with temperature 0 and default DENY; if the judge is biased toward ALLOW, harm increases.
  • domain assumption Docker sandbox execution with localhost-only network is a valid proxy for real-world command success.
    Section IV-D and V-C3; network effects may differ in deployment.

reviewed 2026-08-01 · how reviews work

0 comments
Cite this review

Pith. "Pith review of CARE: Pre-Execution Command Verification for Shell-Executing LLM Agents." pith.science (2026). https://pith.science/paper/XRKA4A4Z

@misc{pith2026260721642,
  author       = {Pith},
  title        = {Pith review of: CARE: Pre-Execution Command Verification for Shell-Executing LLM Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XRKA4A4Z}},
  note         = {Machine review of arXiv:2607.21642}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Large Language Model (LLM) agents are increasingly used for coding and terminal automation, making shell-command dispatch a high-stakes runtime control point. We study command-level pre-execution mediation for individual shell commands produced by LLM agents under bounded path context. Existing safeguards remain limited: generic guardrails do not model shell structure in sufficient detail, always-on LLM judges are relatively costly and variable, and shell parsers do not directly prevent harmful execution. We present CARE (Canonicalization, Attribution, and Resolution Engine), a shell-specific, static-first verifier for individual shell commands before execution. CARE canonicalizes generated commands into stable verification targets, derives deterministic evidence over syntax, command semantics, path context, and provenance-backed risk patterns, and escalates only underdetermined cases to an LLM judge. This design keeps the common case fast, reproducible, and auditable while reserving neural adjudication for borderline commands. On the balanced main split, CARE reaches 85.64% F1 with a 0.91% false-positive rate at 2.32 ms mean latency. When deployed in its static enforcement profile, CARE retains 84.99% F1 at 0.34 ms and reduces realised harm on RedCode-gen to 37.33%. Across external-generalization tests and controlled Docker-sandbox execution, these profiles expose a practical trade-off between benign recovery, false-positive burden, latency, and harm reduction. Overall, command-level shell mediation can reduce dispatch-boundary risk for LLM agents while preserving most benign workflows.

Figures

Figures reproduced from arXiv: 2607.21642 by Barnabas Poczos, Hanqi Feng, Jin B. Hong, Peng Qiu, Wenxiao Zhang, Xinyu Wang, Yanbing Liu, Yu Liu, Zhiwei Yang, Zhongyi Zhang.

Figure 1
Figure 1. Figure 1: Detection–FPR–latency profile of eleven guards on the main evaluation [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Threat Model • We provide a reproducible, deployment-oriented evalu￾ation against 12 baselines across a leakage-controlled main split, a natural language-to-shell utility benchmark, OOD dangerous-command corpora paired with a disjoint benign command pool, and LLM-generated attack com￾mands executed in a Docker sandbox; together, these ex￾periments assess effectiveness, latency, benign-workflow preservation… view at source ↗
Figure 3
Figure 3. Figure 3: CARE overview. A generated shell command [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Selective-DENY trace. Only L2 semantic evidence fires, leaving the command in WARN; Resolution escalates and the LLM judge recognises persistence intent and returns DENY. and (c) Resolution skip predicates prule, psem, pspath. The default operating point lies within 0.29 pp of the best F1 observed in the weight sweep, indicating limited sensitivity to modest reweighting near the chosen operating point. The… view at source ↗
Figure 5
Figure 5. Figure 5: Direct static DENY. Canonicalization unwraps eval/IFS rewriting into cˆ; the provisional verdict is already DENY, so Resolution finalizes it directly without LLM consultation. Case B. Static WARN →LLM ALLOW: cross-host rsync 1 -- Candidate shell command (benign deploy push) 2 rsync -avz ./build/ admin@gpu01:/data/ 3 -- Canonicalization (Eq.1): no rewrite needed 4 c_hat = rsync -avz ./build/ admin@gpu01:/da… view at source ↗
Figure 6
Figure 6. Figure 6: Selective-ALLOW trace. The static stack places the command in WARN; no skip predicate fires, so Resolution escalates and the LLM judge returns ALLOW. This mechanism underlies the 1.82% → 0.91% FPR reduction reported in §V-A1. L1 fails closed (Appendix A.1), so any command that fails to parse and still contains a high-risk token under the L0-normalised view is denied by default. The 11% is therefore a stric… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

33 extracted references · 4 linked inside Pith

  1. [1]

    Claude Code,

    Anthropic, “Claude Code,” Product page, accessed 2026-04-16. [Online]. Available: https://www.anthropic.com/product/claude-code

    2026

  2. [2]

    Introducing Codex,

    OpenAI, “Introducing Codex,” Product announcement, May 2025, accessed 2026-04-16. [Online]. Available: https://openai.com/index/ introducing-codex/

    2025

  3. [3]

    CVE-2025-66032: Claude Code Command Validation Bypass,

    National Vulnerability Database, “CVE-2025-66032: Claude Code Command Validation Bypass,” https://nvd.nist.gov/vuln/detail/ CVE-2025-66032, 2025, nVD entry; affected versions prior to 1.0.93

    2025

  4. [4]

    Not What You’ve Signed Up For: Compromising Real-World LLM- Integrated Applications with Indirect Prompt Injection,

    K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, and M. Fritz, “Not What You’ve Signed Up For: Compromising Real-World LLM- Integrated Applications with Indirect Prompt Injection,” inProceedings of the 16th ACM Workshop on Artificial Intelligence and Security. Association for Computing Machinery, 2023, pp. 79–90

    2023

  5. [5]

    AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents,

    E. Debenedetti, J. Zhang, M. Balunovi ´c, L. Beurer-Kellner, M. Fischer, and F. Tram `er, “AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents,” inAdvances in Neural Information Processing Systems 37 (NeurIPS), 2024, pp. 82 895– 82 920

    2024

  6. [6]

    Agent-SafetyBench: Evaluating the Safety of LLM Agents,

    Z. Zhang, S. Cui, Y . Lu, J. Zhou, J. Yang, H. Wang, and M. Huang, “Agent-SafetyBench: Evaluating the Safety of LLM Agents,”arXiv preprint arXiv:2412.14470, 2024. [Online]. Available: https://arxiv.org/abs/2412.14470

    Pith/arXiv arXiv 2024

  7. [7]

    Mind the GAP: Text Safety Does Not Transfer to Tool-Call Safety in LLM Agents,

    A. Cartagena and A. Teixeira, “Mind the GAP: Text Safety Does Not Transfer to Tool-Call Safety in LLM Agents,”arXiv preprint arXiv:2602.16943, 2026. [Online]. Available: https://arxiv.org/abs/2602. 16943

    arXiv 2026

  8. [8]

    ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback,

    Y . Mou, Z. Xue, L. Li, P. Liu, S. Zhang, W. Ye, and J. Shao, “ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback,” inFindings of the Association for Computational Linguistics: ACL 2026. San Diego, California, United States: Association for Computational Linguistics, 2026, pp. 37 125– 37 153

    2026

  9. [9]

    ClawGuard: A Runtime Security Framework for Tool-Augmented LLM Agents Against Indirect Prompt Injection,

    W. Zhao, Z. Li, P. Zhang, and J. Sun, “ClawGuard: A Runtime Security Framework for Tool-Augmented LLM Agents Against Indirect Prompt Injection,”arXiv preprint arXiv:2604.11790, 2026. [Online]. Available: https://arxiv.org/abs/2604.11790

    Pith/arXiv arXiv 2026

  10. [10]

    AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents,

    H. Wang, C. M. Poskitt, and J. Sun, “AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents,” inProceedings of the IEEE/ACM 48th International Conference on Software Engineering (ICSE), 2026, pp. 1–12

    2026

  11. [11]

    bashlex: Python Parser for Bash,

    I. Kamara and contributors, “bashlex: Python Parser for Bash,” Open-source project, 2014–. [Online]. Available: https://github.com/ idank/bashlex

    2014

  12. [12]

    ShellCheck: A Shell Script Static Analysis Tool,

    V . Holen and contributors, “ShellCheck: A Shell Script Static Analysis Tool,” Open-source project, 2012–. [Online]. Available: https://www.shellcheck.net/

    2012

  13. [13]

    Atomic Red Team,

    Red Canary, “Atomic Red Team,” Open-source MITRE-aligned attack tests, 2018–. [Online]. Available: https://www.atomicredteam.io/

    2018

  14. [14]

    RedCode: Risky Code Execution and Generation Benchmark for Code Agents,

    C. Guo, X. Liu, C. Xie, A. Zhou, Y . Zeng, Z. Lin, D. Song, and B. Li, “RedCode: Risky Code Execution and Generation Benchmark for Code Agents,” inAdvances in Neural Information Processing Systems 37 (NeurIPS), 2024, pp. 106 190–106 236

    2024

  15. [15]

    NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating System,

    X. V . Lin, C. Wang, L. Zettlemoyer, and M. D. Ernst, “NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating System,” inProceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). Miyazaki, Japan: European Language Resources Association (ELRA), 2018

    2018

  16. [16]

    LLM-Supported Natural Language to Bash Transla- tion,

    F. Westenfelder, E. Hemberg, S. Moskal, U.-M. O’Reilly, and S. Chiricescu, “LLM-Supported Natural Language to Bash Transla- tion,” inProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). Albuquerque, New Mexico: Association for Comput...

    2025

  17. [17]

    GTFOBins: A Curated List of Unix Binaries that Can Be Used to Bypass Local Security Restrictions,

    GTFOBins community, “GTFOBins: A Curated List of Unix Binaries that Can Be Used to Bypass Local Security Restrictions,” Open-source project, 2018–. [Online]. Available: https://gtfobins.github.io/

    2018

  18. [18]

    The Exploit Database,

    Offensive Security, “The Exploit Database,” CVE-compliant archive of public exploits and proof-of-concept code, https://www.exploit-db.com/, 1999–, maintained by Offensive Security

    1999

  19. [19]

    Don’t Let the Claw Grip Your Hand: A Security Analysis and Defense Framework for OpenClaw,

    Z. Shan, J. Xin, Y . Zhang, and M. Xu, “Don’t Let the Claw Grip Your Hand: A Security Analysis and Defense Framework for OpenClaw,”arXiv preprint arXiv:2603.10387, 2026. [Online]. Available: https://arxiv.org/abs/2603.10387

    arXiv 2026

  20. [20]

    Granite Guardian 3.1-2B: A Safety Classification Model from the IBM Granite Family,

    IBM Research, “Granite Guardian 3.1-2B: A Safety Classification Model from the IBM Granite Family,” Hugging Face model card, 2024–2025. [Online]. Available: https://huggingface.co/ibm-granite/ granite-guardian-3.1-2b

    2024

  21. [21]

    Llama Guard 3 8B: A Safety Classification Model for LLM Inputs and Responses,

    Meta AI, “Llama Guard 3 8B: A Safety Classification Model for LLM Inputs and Responses,” Hugging Face model card, 2024. [Online]. Available: https://huggingface.co/meta-llama/Llama-Guard-3-8B

    2024

  22. [22]

    gpt-oss-safeguard: An Open-Weight Safety Classification Model,

    OpenAI, “gpt-oss-safeguard: An Open-Weight Safety Classification Model,” Open-weight release, 2025. [Online]. Available: https: //openai.com/index/introducing-gpt-oss-safeguard/

    2025

  23. [23]

    NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails,

    T. Rebedea, R. Dinu, M. N. Sreedhar, C. Parisien, and J. Cohen, “NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails,” inProceedings of the 2023 Conference on Empir- ical Methods in Natural Language Processing: System Demonstrations. Singapore: Association for Computational Linguistics, 2023, pp. 431– 445

    2023

  24. [24]

    LLM Security Guard for Code,

    A. Kavian, M. M. Pourhashem Kallehbasti, S. Kazemi, E. Firouzi, and M. Ghafari, “LLM Security Guard for Code,” inProceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering (EASE), 2024, pp. 600–603

    2024

  25. [25]

    Qwen3-4B-Instruct: A 4B-Parameter Open Instruction-Tuned Language Model,

    Alibaba Qwen Team, “Qwen3-4B-Instruct: A 4B-Parameter Open Instruction-Tuned Language Model,” Hugging Face model card, 2025. [Online]. Available: https://huggingface.co/Qwen/ Qwen3-4B-Instruct-2507

    2025

  26. [26]

    Approximate Statistical Tests for Comparing Super- vised Classification Learning Algorithms,

    T. G. Dietterich, “Approximate Statistical Tests for Comparing Super- vised Classification Learning Algorithms,”Neural Computation, vol. 10, no. 7, pp. 1895–1923, 1998

    1923

  27. [27]

    StruQ: Defending Against Prompt Injection with Structured Queries,

    S. Chen, J. Piet, C. Sitawarin, and D. Wagner, “StruQ: Defending Against Prompt Injection with Structured Queries,” in34th USENIX Security Symposium (USENIX Security 25). Seattle, W A: USENIX Association, 2025, pp. 2383–2400

    2025

  28. [28]

    InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents,

    Q. Zhan, Z. Liang, Z. Ying, and D. Kang, “InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents,” inFindings of the Association for Computational Linguistics: ACL 2024. Bangkok, Thailand: Association for Computational Lin- guistics, 2024, pp. 10 471–10 506

    2024

  29. [29]

    Beyond the Black Box: Demystifying Multi-Turn LLM Reasoning with VISTA,

    Y . Zhang, M. Lin, M. Dras, and U. Naseem, “Beyond the Black Box: Demystifying Multi-Turn LLM Reasoning with VISTA,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 40, no. 48. AAAI Press, 2026, pp. 41 745–41 747

    2026

  30. [30]

    Do LLMs know tool irrelevance? demystifying structural alignment bias in tool invoca- tions,

    Y . Liu, X. Lin, P. Cao, G. Zhang, F. Fang, and Y . Cao, “Do LLMs know tool irrelevance? demystifying structural alignment bias in tool invoca- tions,” inProceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). San Diego, California, United States: Association for Computational Linguistics, 2026, pp....

    2026

  31. [31]

    MAGE: Safeguarding LLM Agents against Long-Horizon Threats via Shadow Memory,

    Y . Wang, T. Jiang, J. Liang, C. Fleming, and T. Wang, “MAGE: Safeguarding LLM Agents against Long-Horizon Threats via Shadow Memory,”arXiv preprint arXiv:2605.03228, 2026. [Online]. Available: https://arxiv.org/abs/2605.03228

    Pith/arXiv arXiv 2026

  32. [32]

    SafeHarness: Lifecycle- Integrated Security Architecture for LLM-based Agent Deployment,

    X. Lin, Y . Liu, Y . Chen, Y . Wu, Y . Ning, Y . Liu, N. Sun, S. Zhang, B. Chong, C. Zhou, and Y . Cao, “SafeHarness: Lifecycle- Integrated Security Architecture for LLM-based Agent Deployment,” arXiv preprint arXiv:2604.13630, 2026. [Online]. Available: https: //arxiv.org/abs/2604.13630

    Pith/arXiv arXiv 2026

  33. [33]

    ShellCore: Au- tomating Malicious IoT Software Detection by Using Shell Commands Representation,

    H. Alasmary, A. Anwar, A. Abusnaina, A. Alabduljabbar, M. Abuhamad, A. Wang, D. Nyang, A. Awad, and D. Mohaisen, “ShellCore: Au- tomating Malicious IoT Software Detection by Using Shell Commands Representation,”IEEE Internet of Things Journal, vol. 9, no. 4, pp. 2485–2496, 2022. APPENDIXA CARE INTERNALS This appendix exposes the full layer-wise internals ...

    2022

This paper was first reviewed by deepseek-v4-flash on August 1, 2026.