{"id":"eccb791c-b5d4-47e0-af95-e30e50da20fa","arxiv_id":"2506.19109","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"An empirical comparison of LLM Guard, Vigil, and Rebuff on 11,000 prompt leak attacks shows Vigil minimizes false positives, Rebuff balances recall and false alarms, and default canary checks in Vigil and Rebuff are ineffective.","lead":"This paper builds an 11,000-sample prompt leak attack dataset and benchmarks three open-source detection tools (LLM Guard, Vigil, Rebuff) against it. It finds Vigil best when false alarms are unacceptable, Rebuff a balanced default, and demonstrates that default canary-word checks in Vigil and Rebuff fail to catch prompt leaks.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The attack dataset is filtered for success against one GPT-3.5-turbo document-chat app; if it is not representative of prompt leak attacks generally, the detection ranking in Table VIII does not transfer.","rationale":"The reader's weakest assumption is the representativeness of the filtered single-application attack dataset, and this is also the most load-bearing vulnerability of the central ranking claim. The paper's implementation-level findings — default canary checks in Vigil and Rebuff fail to fire, and Rebuff's secondary model scanner can be evaded by forging template fields — are supported by direct experiments and manual confirmation, so they should not be dismissed. However, the comparative performance statements in Table VIII and the conclusion about which solution is optimal depend on a dataset whose success filtering and single target application strongly shape the measured recall and false positive rates. The in-sample threshold selection further inflates the apparent peak performance. Since the reader already assigned CONDITIONAL with moderate confidence, and my concern is the same generalizability issue rather than a new internal inconsistency, the appropriate recommendation is UNCHANGED: the conditional verdict stands, and the concrete test would determine whether the concern actually invalidates the ranking outside the paper's narrow setup.","tokens_in":23661,"tokens_out":2606,"duration_ms":30409,"concrete_test":"Rebuild the comparison on an independent prompt-leak corpus covering at least two different target applications/system prompts and at least one other base model (e.g., samples from Raccoon or PLeak plus a second LangChain app with a differently worded system prompt), without first discarding samples that fail to leak. Recompute Table VIII for LLM Guard, Vigil, and Rebuff using a fixed threshold-selection split (optimize on 50% of each class, evaluate on the held-out 50%) and report bootstrap confidence intervals for F-beta and FPR. If the Vigil/Rebuff ordering or the 'optimal for minimal FPR vs. average needs' claim flips or becomes statistically indistinguishable, the headline ranking should be restricted to the tested setup.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central ranking claim (Vigil optimal for minimal FPR, Rebuff optimal for average needs) rests on detection metrics computed over the 11,000-sample attack set described in Section II-C, but that set is not a sample from the general population of prompt leak attacks: every sample was derived from five manually seeded techniques, verified to leak the system prompt of one LangChain/GPT-3.5-turbo document-chat application, and filtered to successful attacks before any detection test. Section V explicitly acknowledges that no real-world LLM-based applications with diverse system instructions were used. Detection difficulty depends strongly on attack distribution: Table VI shows classes containing 'context ignoring' are detected at 100% by vectordb scanners while leet/repeatchar classes drop substantially, so the class mixture in the dataset determines overall recall and FPR. Table VIII also uses thresholds optimized on the same data (Sections III-A and III-B), so the 'peak performance' ranking is in-sample. The canary-check ineffectiveness and the Rebuff template-injection evasion are implementation-level findings that are more robust, but the headline comparative ranking is only as general as this dataset and its 1,000 benign prompts.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents an empirical evaluation of three open-source prompt-injection detection solutions — LLM Guard, Vigil, and Rebuff — for the specific attack class of prompt leaks. The authors build a dataset of 11,000 malicious prompts by combining five prompt-leaking techniques (naive, context ignoring, context manipulation, prefix injection, leet obfuscation) into 11 classes of 1,000 samples each, and verify that every sample actually leaks the system prompt of their single LangChain/GPT-3.5-turbo document-chat application; they add 1,000 benign prompts. For each detection scanner (transformer-based, vectordb-based, Yara/heuristics-based, prompt-response similarity-based, secondary-LLM-based, and canary word checks), they compute detection scores on all 12,000 samples and choose per-scanner thresholds by ROC/PR analysis maximizing an Fβ metric with β=1/11 on the same data. They report per-technique results, find that the default canary checks in Vigil and Rebuff detect essentially no prompt leaks, propose and test a modified canary instruction for Rebuff, demonstrate a template-injection evasion of Rebuff's secondary-model scanner with a proposed sanitization mitigation, and finally compare whole-solution detection policies, concluding that Vigil is optimal when false positives must be minimized and Rebuff is the most balanced choice at peak performance.","tokens_in":23923,"tokens_out":19008,"duration_ms":175477,"significance":"The paper is one of the first direct head-to-head evaluations of these detectors on prompt leakage, and its dataset construction — 1,000 verified-successful attacks per class across combinations of five techniques — is more systematic than the manual datasets in much related work. The strongest contributions are implementation-level findings that are robust within the tested setup: the default canary checks in Vigil and Rebuff exhibit essentially zero true positive rate across all attack classes, the modified canary instruction measurably improves detection, and the secondary-model scanner in Rebuff can be evaded by forging fields of its prompt template (Figure 6), with a concrete mitigation proposed. The paper gives tool version numbers, model names, and vector-store identities that make the experiments reproducible.","major_comments":[{"comment":"The 'peak performance' comparison is computed with thresholds selected on the same 12,000 samples used to report the metrics: the transformer threshold (0.999), the vectordb thresholds (0.17217, 0.82783, 0.83483), and the secondary-model thresholds (0.802, 0.752) are each optimized by ROC/PR analysis on the full dataset, and Table VIII then reports the resulting metrics as the solutions' performance. No held-out split or cross-validation is used, and the stochastic LLM-based scanners (GPT-3.5-turbo, GPT-4o) are run once per sample with no error bars. The headline numbers are therefore in-sample best-case values: Vigil's FPR of 0.000 in Table VIII and its ranking over Rebuff and LLM Guard would likely change on a validation set. The paper should either add a validation protocol (e.g., threshold tuning on one split and reporting on another, or repeated runs with confidence intervals) or re-scope the abstract and conclusions to 'on our dataset with in-sample thresholds'.","section":"III-A, III-B; Tables III–V and VIII"},{"comment":"The Rebuff row in Table VIII (recall 0.981, FPR 0.034) does not describe the shipped product: the paragraph below the table states that these metrics assume an upgraded canary check (with canary-handling instructions) and 'an error handling with default blacklisting' in the secondary-model scanner. The abstract's conclusion that 'Rebuff is the most optimal for average needs' therefore refers to a proposed modified configuration, not to the default open-source package, and this distinction must be made in the abstract and conclusion. Relatedly, the treatment of secondary-model parse errors is internally inconsistent: Section II-D-1 states the score is set to zero on a parse error, while Section III-A states error cases were assumed to have score 1.0 and treated as malicious; these choices produce different recall and FPR values and both feed into Table VIII. Please state the scoring protocol unambiguously and, if both behaviors were used, report metrics under each.","section":"III-B (Table VIII); II-D-1 vs III-A"},{"comment":"The attack dataset is not a sample from the general population of prompt leaks. Every one of the 11,000 malicious prompts was derived from five manually seeded techniques, verified to leak the system prompt of a single LangChain/GPT-3.5-turbo document-chat application, and filtered to successful attacks before any detection test; Section V concedes that no real-world applications with diverse system instructions were used. Aggregate metrics and the Table VIII ranking are conditional on a fixed class mixture (1,000 per class) and a single system prompt, and Table VI shows per-class recall ranging from 55.9% to 100% for the same detector, so a different class mixture would change the overall recall and FPR materially. The conclusion that 'Rebuff is the most optimal for average needs' should be re-scoped to the tested distribution and target, or supported by a sensitivity analysis (e.g., reweighting of classes and at least one additional target application or system prompt).","section":"II-C, V; Table VI"},{"comment":"The proposed mitigation for the secondary-model evasion — sanitizing user prompts by filtering context-delimiter strings and newline characters before insertion into Rebuff's template — is not validated anywhere in the paper. There is no experiment showing that the sanitized scanner detects the suffix-evasion samples of Figure 6, nor any measurement of the effect of sanitization on detection scores or on false positives for the benign set. Since the abstract lists this mitigation as a contribution, it should be implemented and re-run on the test suite, or the claim should be explicitly downgraded to an untested recommendation.","section":"IV-A"}],"minor_comments":[{"comment":"The abstract's 'presents analyzes of distinct prompt leakage detection techniques' should read 'presents analyses'.","section":"Abstract"},{"comment":"Table IV gives the GPT-3.5-turbo optimal threshold as 0.802, while Section III-A twice refers to the calculated threshold as 0.952 (which is actually the Fβ value of that column); the text is also inconsistent with the reported FPR behavior, since a threshold below 0.9 is what raises the false positive rate. Please correct.","section":"Table IV and III-A"},{"comment":"The Vigil vectordb threshold appears as 0.17317 in Table V but as 0.17217 in Table VI and Section III-B; the Rebuff extended-store threshold appears as 0.83483 in Table V but as 0.82783 in the Table VI header. Please harmonize.","section":"Tables V, VI; III-B"},{"comment":"Please document how the 1,000 benign prompts were created (sources, topics, length distribution, and whether they were checked for overlap with the attack templates), since the false-positive rate is the principal differentiator in Table VIII.","section":"II-C"},{"comment":"'low false true positive rate' should read 'low false positive rate'; also, minor language errors elsewhere (e.g., 'we calculate its detections scores' in Section III-A, 'an LLM application' in the introduction) should be cleaned up.","section":"III-B"},{"comment":"The ordering of solutions depends on the metric: by F1, LLM Guard (0.994) and Rebuff (0.989) lead, while by Fβ, Vigil (0.998) leads. Because β=1/11 is an analyst-chosen weight, please state explicitly how the ranking changes over a small range of β, or justify that the headline conclusions do not depend on this choice.","section":"III-B"},{"comment":"Reference [13] appears as '[redacted for anonymity]' although the manuscript is submitted with author names; please make the anonymization consistent or include the URL. Some references also contain line-break artifacts in URLs; please ensure they render correctly.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"This is a competent empirical evaluation whose central ranking claim is weakened by in-sample threshold selection and by a Table VIII Rebuff configuration that assumes upgrades to the stock product. Both are fixable with re-scoped claims and/or a validation split; I would not reject on these grounds. The authors should also be asked to correct the numeric inconsistencies in Tables IV–VI, since these currently appear in the results the conclusions rest on. Scope fit for a security/LLM-systems venue is good, and the responsible disclosure section is a point in the paper's favor."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nRead the paper. It is the first direct comparison of LLM Guard, Vigil, Rebuff for prompt leak detection, and the dataset is bigger than anything in the cited related work: 11,000 malicious samples across 11 classes, up to four stacked techniques. That part is real.\n\nWhat is actually new: the default canary-word checks in Vigil and Rebuff simply do not work — zero detections across all 11,000 attacks — and the authors show why (the canary is prefixed and the model treats it as commentary; a simple added sentence in Rebuff's system prompt raises detection substantially). That is a concrete, reproducible implementation defect, and the fix is plausible. The other genuinely useful finding is the template-injection evasion of Rebuff's secondary-model scanner: because the prompt template embeds the user input into a named \"User string\" field, appending a forged field makes the classifier evaluate a benign prompt while the main LLM still leaks. That is a real weakness and the mitigation (sanitize newlines/delimiters) makes sense.\n\nThe soft spots are mostly in the headline comparison. Thresholds for every scanner are chosen by ROC on the same dataset used to report \"peak performance,\" so the Fβ numbers in Tables III–VIII are in-sample, and there are no error bars for stochastic LLM-based scanners. The Rebuff row in Table VIII is not the default product: it assumes the author's modified canary check and error handling, so it overstates what a user gets out of the box. And the attack set is filtered for success against one GPT-3.5-turbo document-chat app; detection difficulty depends heavily on class mix, as Table VI shows. The paper itself concedes that no real-world apps with diverse system instructions were used. So the ranking \"Vigil best for minimal FPR, Rebuff best for average needs\" should be read as conditional on this workload, not as a general law.\n\nThe stress-test note is roughly right. I would not call the in-sample threshold choice a fatal flaw for an engineering evaluation — this is how a lot of benchmark papers operate — but the authors should report cross-validated or held-out thresholds, or at least label the numbers as tuned.\n\nWorth a serious referee. I'd send it to a security workshop or empirical-systems venue, ask for a validation split, released attack data, and a clearer separation between default and modified configurations. The canary and evasion findings alone justify the review.","headline":"Useful first head-to-head of three open-source prompt-leak detectors with two genuinely new findings; the headline ranking is real but in-sample, so treat as engineering guidance rather than settled fact.","tokens_in":24457,"tokens_out":2765,"would_cite":true,"duration_ms":24348,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"On 11,000 verified prompt-leak attacks, this paper ranks three open-source detectors: Vigil minimizes false positives, Rebuff suits average needs, and default canary checks fail—plus fixes.","keywords":["prompt injection","prompt leak","LLM security","canary word","detection benchmark","LLM Guard","Vigil","Rebuff"],"falsifier":"Conduct the same evaluation on an independently constructed prompt-leak benchmark with different system instructions and a different underlying model, then check two things: whether Rebuff's balanced F1 and Fβ still beat Vigil and LLM Guard at their peak thresholds, and whether the paper's modified canary check detects substantially more leaks than the default prefix-only check. A negative result on either test would falsify the central comparison or the canary fix.","tokens_in":23440,"feed_emoji":"🛡️","tokens_out":9888,"duration_ms":90909,"temperature":0.7,"pith_summary":"This paper is a head-to-head performance evaluation of three open-source prompt-injection detection systems—LLM Guard, Vigil, and Rebuff—specialized to one attack type: prompt leak, where an attacker tricks an LLM into revealing its system instructions. The authors built a document-chat agent as a target, generated 11,000 verified prompt-leak samples across eleven attack classes, and scored every scanner on recall, false-positive rate, F1, and a precision-weighted Fβ metric. Their central conclusions are that default canary-word checks in Vigil and Rebuff detect none of the attacks; Rebuff's secondary model-based scanner can be evaded by forging fields in its prompt template; and at peak configuration Vigil is best when false positives are unacceptable, Rebuff is best for balanced needs, and LLM Guard wins on raw recall at the cost of more false alarms. The paper also proposes improvements—canary-handling instructions in the system message, input sanitization in the model-based scanner, and a hybrid detection policy—that push performance past all three vendor defaults.","feed_headline":"Prompt-leak detectors ranked: Vigil cleanest, Rebuff best overall","feed_subtitle":"A benchmark of 11,000 samples shows default canary checks miss every leak; a hybrid scanner policy beats all three tools.","key_machinery":"The mechanism that carries the comparison is a two-part evaluation pipeline. First, a dataset of 11,000 verified prompt-leak samples is built in 11 classes—naive, context ignoring, context manipulation, prefix injection, leet obfuscation, and combinations—with every sample pre-tested to confirm it actually leaks the target system prompt. Second, each scanner's continuous detection score is converted into a binary alarm by choosing the threshold that maximizes an Fβ score with β = 1/11, which treats precision as eleven times more important than recall because the test set has eleven malicious samples per benign sample. This threshold choice defines the 'peak performance' used for every comparison. Two concrete mechanisms carry the proposed fixes: recasting the canary word in the system message as a secret the model must include when asked to reveal its instructions, and sanitizing user input before interpolation into the secondary scanner's few-shot template.","core_discovery":"The paper claims that the practical value of these detectors lies in how their internal techniques are wired together, not just in which models back them. Transformer-based scanners reach 97–100% recall but produce false positives. Vector-database scanners are strong only for attack classes represented in their store, and Vigil's prompt-response similarity scanner is unsuitable for this task. Canary-word checks, as implemented by Vigil and Rebuff, never fire because the model either treats the canary as a comment or does not regard it as part of the system instructions; Rebuff's check improves dramatically once the system message explicitly tells the model it holds a secret and to include it when asked for initial instructions. Rebuff's secondary model-based scanner is vulnerable to a delimiter-injection: an attacker appends a forged field and a score line so the scanner evaluates a benign prompt instead. At peak configuration, the paper reports Vigil with zero false positives and recall 0.838, Rebuff with recall 0.981 and false-positive rate 0.034, and LLM Guard with recall 0.999 and false-positive rate 0.127; a hybrid policy combining scanners from Vigil and Rebuff achieves recall 0.981 with false-positive rate 0.016 and the highest Fβ among all configurations. Finally, running the same 11,000 samples against newer models shows 2% attack success on GPT-4o and 19% on Claude-3-5-sonnet, so active detection remains relevant.","pith_inferences":["The field-forging evasion is probably not unique to Rebuff: any detector that interpolates raw input into a labeled prompt template with few-shot examples is exposed to the same delimiter injection, so sanitizing newlines and field names is a general design rule.","Framing the canary as a secret in the system message shifts the burden onto the model's instruction-following; a natural extension is to plant multiple trap secrets in tool descriptions, memory buffers, or retrieval contexts to catch indirect leaks of non-system content.","The paper's finding that the triple combination of context ignoring, leet, and context manipulation is the most evasive suggests that detection benchmarks should include combinatorial attacks as a standard test class, not just single techniques."],"forward_implications":["Deployments that cannot tolerate false alarms should configure Vigil with a high transformer threshold and rely on its Yara and vector-database scanners, since the paper measured zero false positives at recall 0.838.","Users of Rebuff should apply the proposed canary-handling instruction and input sanitization before relying on its secondary-model scanner; the paper's measured Rebuff numbers assume those changes.","Detection policies should treat vector-database hits as high-confidence, pair transformer and secondary-model scanners to suppress false positives, and trigger on Yara, heuristics, or canary alarms independently; the paper's hybrid policy measured the best Fβ.","Prompt-leak protections cannot rely on model hardening alone, because 2% of attack samples still succeeded on GPT-4o and 19% on Claude-3-5-sonnet, so active detection remains necessary.","Output-based detectors should expect evasion via transformed leaks such as word reversal, added prefixes, leet, or base64, since canary checks assume plain-text recovery of system instructions."],"supporting_citations":[{"why":"Defines prompt leak attacks and the 'ignore previous instructions' technique that anchors several attack classes.","marker":"[5]"},{"why":"Supplies the prefix-injection technique and the leet substitution idea used in constructing attack samples.","marker":"[10]"},{"why":"Provides the context-manipulation and repeated-character attack that the 'repeatchar' classes are based on.","marker":"[11]"},{"why":"Gives the leet substitution table used to build obfuscated prompt-leak samples.","marker":"[12]"},{"why":"Documentation for LLM Guard, one of the three evaluated detection systems.","marker":"[6]"},{"why":"Documentation for Vigil, defining its five scanners and default detection policy.","marker":"[7]"},{"why":"Documentation for Rebuff, defining its four scanners, thresholds, and canary check.","marker":"[8]"},{"why":"The transformer classification model used by the transformer-based scanners in LLM Guard and Vigil.","marker":"[14]"},{"why":"Shows adversarial transformations of leaked text that would evade output-based canary detection and bounds the canary approach.","marker":"[18]"}],"fun_headline_variants":["Canary checks miss every leak; hybrid scanner policy wins","Vigil: zero false positives; Rebuff: best all-round detector","Benchmark: 11k samples, canary checks fail, hybrid best Fβ","Prompt-leak detection: Vigil cleanest, Rebuff best, hybrid tops","Detector shootout: canary checks useless, hybrid policy outperforms"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes that the 11,000 malicious samples—all verified to leak the system prompt of a single document-chat application built on one assistant model, and all filtered to successful attacks before detection testing—are representative of prompt leak attacks in general.","fun_headline_variants_meta":{"raw":{"variants":["Canary checks miss every leak; hybrid scanner policy wins","Vigil: zero false positives; Rebuff: best all-round detector","Benchmark: 11k samples, canary checks fail, hybrid best Fβ","Prompt-leak detection: Vigil cleanest, Rebuff best, hybrid tops","Detector shootout: canary checks useless, hybrid policy outperforms"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000291,"raw_usage":{"total_tokens":1794,"prompt_tokens":1136,"completion_tokens":658,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":752,"completion_tokens_details":{"reasoning_tokens":559}},"tokens_in":752,"tokens_out":658,"duration_ms":6354,"temperature":1.0,"reasoning_tokens":559,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:36:30.268757+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Conduct the same evaluation on an independently constructed prompt-leak benchmark with different system instructions and a different underlying model, then check two things: whether Rebuff's balanced F1 and Fβ still beat Vigil and LLM Guard at their peak thresholds, and whether the paper's modified canary check detects substantially more leaks than the default prefix-only check. A negative result on either test would falsify the central comparison or the canary fix.","supporting_citations":[{"cited_title":"Dont you (forget NLP): Prompt injection with control characters in ChatGPT — dropbox.tech","cited_arxiv_id":null,"evidence_quote":"Provides the context-manipulation and repeated-character attack that the 'repeatchar' classes are based on."},{"cited_title":"https://en.wikipe dia.org/wiki/Leet","cited_arxiv_id":null,"evidence_quote":"Gives the leet substitution table used to build obfuscated prompt-leak samples."},{"cited_title":"Protect AI","cited_arxiv_id":null,"evidence_quote":"Documentation for LLM Guard, one of the three evaluated detection systems."},{"cited_title":"Release Blog — Vigil: Documentation — vigil.deadbits.ai","cited_arxiv_id":null,"evidence_quote":"Documentation for Vigil, defining its five scanners and default detection policy."},{"cited_title":"GitHub - protectai/rebuff: LLM Prompt In- jection Detector — github.com","cited_arxiv_id":null,"evidence_quote":"Documentation for Rebuff, defining its four scanners, thresholds, and canary check."},{"cited_title":"Fine-Tuned DeBERTa-v3-base for Prompt Injection Detection","cited_arxiv_id":null,"evidence_quote":"The transformer classification model used by the transformer-based scanners in LLM Guard and Vigil."},{"cited_title":"PLeak: Prompt Leaking Attacks against Large Language Model Applications","cited_arxiv_id":null,"evidence_quote":"Shows adversarial transformations of leaked text that would evade output-based canary detection and bounds the canary approach."}],"review_version":2}