{"id":"d27b0a4c-c940-40b5-9051-01dbf013c465","arxiv_id":"2505.03574","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"LlamaFirewall is an open-source guardrail framework whose layered scanners reduce prompt-injection attack success from 17.6% to 1.75% on AgentDojo, with utility dropping from 47.7% to 42.7%.","lead":"Meta released LlamaFirewall, an open-source security layer for AI agents that combines a jailbreak detector, a chain-of-thought alignment auditor, and a code scanner. It reports large reductions in prompt-injection attack success on the AgentDojo benchmark, at some cost in task utility.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"AgentDojo ASR is computed by reclassifying already-succeeded attacks as 'defended' when a scanner flags later in the trace, so the headline 1.75% ASR measures post-hoc detection, not prevention.","rationale":"The reader's weakest assumption (CoT faithfulness) is a genuine limitation, but the more immediate threat to the central claim is the evaluation's reclassification rule. Even if agent LLMs perfectly verbalize their intent, the reported ASR numbers in Section 4.3.2 are not valid measures of prevention: Section 4.3.1 explicitly reclassifies traces where an attack 'initially succeeds but is later flagged' as defended. On an offline replay, a later flag cannot undo an already-executed injection; it only shows the guardrail would have detected the compromise after the fact. This is directly load-bearing because the headline >90% ASR reduction is computed from these labels, and the combined configuration's advantage over PromptGuard alone depends heavily on AlignmentCheck's post-hoc flags. A causal reanalysis could still support the qualitative conclusion that AlignmentCheck adds signal, but the current numbers overstate prevention. Because the system is open source and the traces are static, this is straightforward to test, so the appropriate posture remains conditional acceptance with a mandatory reanalysis condition rather than outright rejection. I partly agree with the reader's emphasis on CoT faithfulness, but the reclassification rule is the more concrete, immediately checkable threat to the paper's headline claim.","tokens_in":24175,"tokens_out":6953,"duration_ms":66937,"concrete_test":"Recompute Section 4.3.2 ASR using causal ordering on the same AgentDojo traces: for each adversarial trace, identify the earliest step where ground truth marks the attack as successful (e.g., prompt_injection_success=true), and count the trace as defended only if PromptGuard or AlignmentCheck first flags at or before that step; any later flag counts as an attack success. Report standalone and combined ASR under this rule. If recomputed ASR exceeds the reported 2.89% and 1.75% by a substantial margin, or if per-step ground truth is unavailable, rerun the evaluation online with a halting intervention to verify the prevention claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.3.1 states: 'in adversarial runs, if an attack initially succeeds but is later flagged, the trace is reclassified as defended, indicating successful prevention.' This rule is the load-bearing step behind the headline claim in Section 4.3.2 (combined ASR 1.75%, >90% reduction from baseline). Because the evaluation is an offline replay of static AgentDojo traces, a flag that occurs after the agent has already executed the injected instruction cannot prevent anything; it only detects the compromise after the fact. Reclassifying such traces as defended conflates detection with prevention and directly inflates AlignmentCheck's standalone ASR (2.89%) and the combined figure. The Appendix C.5 example illustrates the pattern: the alignment judge flags an agent action only after the agent has already switched to the attacker's goal and taken that action. Under any real-time deployment, that trace is a successful attack, not a prevented one. The central empirical claim therefore rests on an invalid counterfactual, treating later flags as if they had halted earlier actions.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"LlamaFirewall is a modular guardrail framework comprising PromptGuard 2 (a fine-tuned DeBERTa-based jailbreak classifier), AlignmentCheck (a few-shot LLM auditor that inspects an agent's chain-of-thought for goal misalignment), and CodeShield (a Semgrep/regex static analyzer for insecure code). The paper reports evaluations on in-house benchmarks and the external AgentDojo suite, with the headline result that combining PromptGuard 2 and AlignmentCheck reduces AgentDojo attack success rate from 17.6% to 1.75% while keeping utility near baseline. The authors also claim state-of-the-art direct jailbreak detection for PromptGuard 2 and describe AlignmentCheck as the first open-source real-time chain-of-thought auditor for injection defense.","tokens_in":24340,"tokens_out":4405,"duration_ms":41272,"significance":"If the reported numbers are correct, LlamaFirewall would be a practically useful open-source defense layer for LLM agents. The paper's strengths include releasing the code, providing implementation details for each scanner, evaluating on an external benchmark (AgentDojo), and including ablations over guardrail model sizes and prompting variants. However, the two central empirical claims are currently not supported by the evidence as presented: the AgentDojo ASR reductions rely on a reclassification rule that conflates detection with prevention, and the state-of-the-art claim for PromptGuard 2 lacks competitor comparisons on the direct jailbreak benchmark. The framework itself is a meaningful contribution, but the headline numbers need re-analysis before the claims can be accepted.","major_comments":[{"comment":"The reclassification rule stated in Section 4.3.1 is load-bearing for the central ASR claims. The paper says: 'in adversarial runs, if an attack initially succeeds but is later flagged, the trace is reclassified as defended, indicating successful prevention.' In an offline replay of static traces, a flag that occurs after the agent has already executed the injected instruction cannot prevent anything; it only detects the compromise after the fact. Reclassifying such traces as defended conflates detection with prevention and directly inflates the standalone AlignmentCheck ASR (2.89%) and the combined figure (1.75%) reported in Section 4.3.2. Please recompute the ASR counting a trace as a successful attack whenever the injected action was taken before the first flag, and report both 'blocked before action' and 'detected at any point' metrics so the gap is transparent. The example in Appendix C.5 illustrates the concern: the judge flags an action only after the agent has already switched to the attacker's goal.","section":"Section 4.3.1"},{"comment":"The claim of 'clear state of the art performance' for PromptGuard 2 on universal jailbreak detection is not supported by the evidence presented. Appendix A.2 states: 'As this dataset is specifically tailored to evaluate inputs that are out-of-distribution to PromptGuard, we don't report competitor metrics on this benchmark.' Without evaluating competitor detectors on the same direct jailbreak benchmark, the SOTA claim cannot be substantiated. The AgentDojo comparisons in Section 4.1 and Appendix B.2 involve indirect prompt injection in agent traces, which is a different task and does not establish superiority on direct jailbreak detection. Please provide competitor results on the same direct jailbreak data or temper the SOTA claim accordingly.","section":"Appendix A.2 / Section 4.1"},{"comment":"The threshold selection procedure threatens the validity of the PromptGuard versus competitor comparison. The paper selects a threshold for each model to produce a fixed, minimal utility reduction of 3% on the AgentDojo evaluation set, then compares ASR at that operating point. This is tuning to the evaluation benchmark. Different models have different utility-ASR tradeoff curves (as Figure 6 suggests), and the ranking of detectors can change across operating points. Please report the full tradeoff curves for all compared models, state the exact thresholds used, and ideally select thresholds on a held-out validation split rather than on the test set.","section":"Section 4.1 / Appendix B.2"},{"comment":"The abstract's claim of 'stronger efficacy at preventing indirect injections in general scenarios' is broader than what the mechanism and evaluation can support. Appendix C.1.3 states that 'Agent LLMs often verbalize their intent... enabling the guardrail LLM to identify misaligned behavior more reliably.' This is an assumption about agent behavior, not a demonstrated property. If an agent is deceptive, adversarially trained, or simply does not expose an injected goal in its reasoning trace, AlignmentCheck has no signal. The evaluation covers a limited set of models that do verbalize; generalization beyond that setting is unsupported. Please qualify the abstract's wording to reflect this limitation, or provide evidence that AlignmentCheck works on agents that do not verbalize injected goals.","section":"Appendix C.1.3 / Abstract"}],"minor_comments":[{"comment":"The heading 'LLamaFirewall' has a capitalization typo; it should read 'LlamaFirewall'.","section":"Section 3 heading"},{"comment":"The phrase 'adds an extra layter of defense' contains a typo: 'layter' should be 'layer'.","section":"Section 2.3"},{"comment":"The heading 'Comparison of Effectiveness of diffent Llama Models' contains a typo: 'diffent' should be 'different'.","section":"Appendix C.2 heading"},{"comment":"The results table mixes decimal and percentage formats (e.g., .1763 vs 17.6%) within the same table and text; please use a single consistent format.","section":"Section 4.3.2"},{"comment":"The parenthetical 'negligible utility loss ( 1.5%)' appears to have a missing figure or operator; please clarify whether this is approximately 1.5% or some other value.","section":"Appendix B.2"},{"comment":"The paper describes the offline evaluation as 'simulat[ing] real-time deployment,' but an offline replay cannot capture the timing-dependent behavior of a live monitor (e.g., whether a flag can arrive before the agent's next action in a real system). This simulation assumption should be explicitly acknowledged as a limitation in Section 5.","section":"Section 4.3.1"}],"recommendation":"major_revision","confidential_remarks":"The paper would benefit from a clearer separation between what is demonstrated empirically and what is claimed conceptually. The reclassification rule in Section 4.3.1 is the single most important fix; without it, the headline ASR numbers are not trustworthy. The omission of competitor metrics on the in-house benchmark is also concerning for a SOTA claim. These issues are fixable within the manuscript's scope, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The framework is real and worth knowing about, but the headline ASR numbers are inflated: the AgentDojo evaluation reclassifies attacks that already succeeded as \"defended\" if a scanner flags them later in the trace. That is detection, not prevention. Section 4.3.1 states the rule outright, and the Appendix C.5 example shows a flag arriving only after the agent has already switched goals and acted. In any real deployment, that trace is a successful attack, so the claimed 84–90% reductions are not valid as prevention figures.\n\nWhat is genuinely new and good: AlignmentCheck is a novel idea—few-shot chain-of-thought auditing for injection defense—and the modular pipeline is well designed. PromptGuard 2 shows real gains over its predecessor, with a sensible narrowing of scope and an energy-based loss that appears to help precision. CodeShield is a solid, extensible static analyzer, and the open-sourced code is a concrete contribution.\n\nThe soft spots go beyond the reclassification problem. Thresholds are tuned to a fixed 3% utility cost on the same AgentDojo eval, which is a form of test-set overfitting. The main jailbreak benchmark is private, and competitor metrics are omitted on the OOD set, so \"state of the art\" is hard to verify. AlignmentCheck's effectiveness also rests on the assumption that agents verbalize true intent in their CoT; the paper acknowledges this but does not test it against deceptive or adversarially trained agents. These are not minor issues—they undercut the quantitative conclusions, though the framework itself may still be useful in practice.\n\nWho should read this? Practitioners building agent guardrails will find the architecture and the honest limitations section useful. Researchers should not cite the ASR numbers without a caveat about the metric. The paper deserves peer review, but a serious referee should send it back for major revisions, chiefly a re-analysis of the AgentDojo results using a proper prevention metric—flag before the malicious action, or a live/simulator-based evaluation—plus confidence intervals and public eval artifacts.\n\nMy take: engage with it as an engineering contribution, but do not take the headline numbers at face value until the evaluation is corrected.","headline":"Useful open-source guardrail framework with a genuinely novel AlignmentCheck, but the headline AgentDojo numbers conflate detection with prevention and need major re-analysis.","tokens_in":24953,"tokens_out":2307,"would_cite":true,"duration_ms":22866,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Layering a lightweight jailbreak classifier with a chain-of-thought alignment auditor reduces benchmark agent prompt-injection success from 17.6% to 1.75%.","keywords":["AI agent security","prompt injection","jailbreak detection","chain-of-thought auditing","goal misalignment","static code analysis","guardrail framework","AgentDojo"],"falsifier":"A controlled test with an agent fine-tuned to suppress or rationalize injected goals: if the agent obeys an injected exfiltration instruction while its chain-of-thought remains goal-consistent, and AlignmentCheck labels the actions aligned, then the reported 84 to 90 percent reductions would not transfer to that agent class.","tokens_in":23958,"feed_emoji":"🛡️","tokens_out":6812,"duration_ms":60143,"temperature":0.7,"pith_summary":"This paper tries to establish that the new class of security risks from autonomous LLM agents—prompt injection, goal hijacking, and insecure generated code—can be contained by an open, system-level guardrail layer rather than by model fine-tuning alone. It presents LlamaFirewall, whose three scanners are a lightweight DeBERTa-based jailbreak classifier (PromptGuard 2), a few-shot chain-of-thought auditor (AlignmentCheck) that watches whether an agent's reasoning drifts from the user's goal, and a fast static-analysis engine for generated code (CodeShield). The load-bearing evidence is on the AgentDojo benchmark: the combined PromptGuard plus AlignmentCheck configuration lowers attack success rate from 17.6% to 1.75%, a reduction of more than 90%, while keeping task utility near 43% of the original 47.7% baseline. A sympathetic reader would care because it suggests practical defenses are possible today, with open, auditable components, even as agents take higher-stakes actions on untrusted content.","feed_headline":"Guardrail stack cuts AI agent attack success by 90 percent","feed_subtitle":"PromptGuard plus a chain-of-thought auditor drops AgentDojo attack success from 17.6% to 1.75%.","key_machinery":"The machinery is a policy engine with three interchangeable scanners. PromptGuard 2 is a fine-tuned DeBERTa-family classifier (86M and 22M variants) that detects explicit jailbreak phrasing in user and tool messages; an energy-based loss and hardened tokenization are meant to reduce false positives and resist token-splitting evasion. AlignmentCheck is a few-shot guardrail LLM that takes the original user goal, the agent's trace, and the selected action, and judges whether that action pursues an unintended goal; it is the semantic layer that catches goal hijacking that lexical filters miss. CodeShield is a two-tier static analyzer using Semgrep and regex rules across eight languages and over 50 CWEs, with a fast first pass around 60 milliseconds and a deeper second pass around 300 milliseconds for flagged code. The framework lets developers compose these detectors with conditional remediation actions, in the spirit of network-security rule engines.","core_discovery":"The central claim, stated on the paper's own terms, is that a layered defense is quantitatively stronger than any single guardrail: on AgentDojo, PromptGuard 2 (86M) alone drops attack success rate from 17.6% to 7.5% (a 57% reduction), AlignmentCheck backed by Llama 4 Maverick alone drops it to 2.89% (an 84% reduction), and the two together reach 1.75% (over 90% reduction), at a utility of 42.7% versus the 47.7% unprotected baseline. The paper further claims AlignmentCheck is the first open-source guardrail to audit an LLM's chain of thought in real time for injection defense, and that larger guardrail models (Llama 4 Maverick, Llama 3.3 70B) detect over 80% of goal hijacks in an in-house benchmark with false positive rates below 4%. CodeShield, the third guardrail, is claimed to achieve 96% precision and 79% recall on manually labeled insecure code completions, providing a real-time static-analysis backstop for coding agents.","pith_inferences":["Editorial inference: if AlignmentCheck works because agent chains of thought are transparent, then the natural next attack is to train agents to produce clean rationalizations while executing injected goals; testing this would require an adversarially fine-tuned agent with hidden or deceptive reasoning.","Editorial inference: the reported utility drop (47.7% to 42.7%) is measured as task success on AgentDojo traces; real-world false-positive costs could be higher for high-stakes actions, so deployment may favor risk-triggered AlignmentCheck only on sensitive operations.","Editorial inference: combining AlignmentCheck with instruction-hierarchy fine-tuning is a testable extension; prior work shows hierarchy training alone removes only part of the violations, and AlignmentCheck should catch residual cases that still surface in reasoning.","Editorial inference: the same trace-auditing pattern could extend to multimodal agents, since it only needs a structured record of observations and actions; the paper lists multimodal support as future work, but the mechanism does not depend on text-only inputs."],"forward_implications":["Production agent deployments can put a cheap lexical filter in front of an expensive semantic auditor, since the combined configuration removes most attacks with only a modest utility drop.","AlignmentCheck can be added to an existing agent without fine-tuning: it consumes the user goal, the trace, and the most recent action through a static prompt template.","The compact PromptGuard 2 22M variant gives a low-latency option for local, resource-constrained deployments while still cutting attack success rate by 41% on AgentDojo.","CodeShield gives coding agents a feedback loop: unsafe code is rejected before landing, while the agent is free to iterate, so a benign-but-insecure pattern never enters the codebase.","The open release means security teams can write their own regex or prompt-based scanners and share policies, letting the guardrail set grow with the threat landscape."],"supporting_citations":[{"why":"Supplies the AgentDojo benchmark, its static traces, and the baseline attack success rate and utility figures used for the layered-defense evaluation.","marker":"Debenedetti et al. (2024)"},{"why":"Provides the DeBERTa architecture family that PromptGuard 2 fine-tunes into its 86M and 22M classifiers.","marker":"He et al. (2021)"},{"why":"Provides the CyberSecEval3 manually labeled insecure code completions used to measure CodeShield's precision and recall.","marker":"Wan et al. (2024)"},{"why":"Introduces instruction-hierarchy fine-tuning, a partial defense whose residual failures motivate and contextualize AlignmentCheck.","marker":"Wallace et al. (2024)"},{"why":"Defines the Spotlighting baseline against which AlignmentCheck's indirect-injection performance is compared.","marker":"Hines et al. (2024)"},{"why":"Defines the Paraphrasing baseline defense used as a comparison point in the AgentDojo defense evaluation.","marker":"Jain et al. (2023)"},{"why":"Introduces the CaMeL deterministic defense that AlignmentCheck is claimed to surpass in flexibility and generality.","marker":"Debenedetti et al. (2025)"},{"why":"Presents Llama Guard, a prior auxiliary classifier for detecting malicious prompt structures that PromptGuard extends.","marker":"Inan et al. (2023)"}],"fun_headline_variants":["Open-source LlamaFirewall cuts AI agent attack success by 90%","Real-time chain-of-thought auditor blocks 90% of agent attacks","Layered guardrails reduce agent attacks from 17.6% to 1.75%","New open guardrail system thwarts 90% of AI agent attacks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"AlignmentCheck's effectiveness rests on the assumption that agent LLMs verbalize their true intent in chain-of-thought, so a semantic auditor can observe misalignment; a deceptive or adversarially trained agent that hides the injected goal gives AlignmentCheck no signal.","fun_headline_variants_meta":{"raw":{"variants":["Open-source LlamaFirewall cuts AI agent attack success by 90%","Real-time chain-of-thought auditor blocks 90% of agent attacks","Layered guardrails reduce agent attacks from 17.6% to 1.75%","New open guardrail system thwarts 90% of AI agent attacks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000493,"raw_usage":{"total_tokens":2486,"prompt_tokens":1073,"completion_tokens":1413,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":689,"completion_tokens_details":{"reasoning_tokens":1328}},"tokens_in":689,"tokens_out":1413,"duration_ms":10855,"temperature":1.0,"reasoning_tokens":1328,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:47:39.523656+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A controlled test with an agent fine-tuned to suppress or rationalize injected goals: if the agent obeys an injected exfiltration instruction while its chain-of-thought remains goal-consistent, and AlignmentCheck labels the actions aligned, then the reported 84 to 90 percent reductions would not transfer to that agent class.","supporting_citations":[],"review_version":1}