Pith. sign in

REVIEW 4 major objections 7 minor 29 references

Enhancing Security in LLM Applications: A Performance Evaluation of Early Detection Systems

T0 review · 4 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read On 11,000 verified prompt-leak attacks, this paper ranks three open-source detectors: Vigil minimizes false positives, Rebuff suits average needs, and default canary checks fail—plus fixes.

desk verdict Useful first head-to-head of three open-source prompt-leak detectors with two genuinely new findings; the headline ranking is real but in-sample, so treat as engineering guidance rather than settled fact. read the letter →

arxiv 2506.19109 v1 pith:PDKTABKF submitted 2025-06-23 cs.CR cs.AI

Valerii Gakh , Hayretdin Bahsi This is my paper · ORCID
classification cs.CRcs.AI
keywords promptinjectionleakLLMsecuritycanaryworddetectionbenchmarkGuardVigilRebuff
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is a head-to-head performance evaluation of three open-source prompt-injection detection systems—LLM Guard, Vigil, and Rebuff—specialized to one attack type: prompt leak, where an attacker tricks an LLM into revealing its system instructions. The authors built a document-chat agent as a target, generated 11,000 verified prompt-leak samples across eleven attack classes, and scored every scanner on recall, false-positive rate, F1, and a precision-weighted Fβ metric. Their central conclusions are that default canary-word checks in Vigil and Rebuff detect none of the attacks; Rebuff's secondary model-based scanner can be evaded by forging fields in its prompt template; and at peak configuration Vigil is best when false positives are unacceptable, Rebuff is best for balanced needs, and LLM Guard wins on raw recall at the cost of more false alarms. The paper also proposes improvements—canary-handling instructions in the system message, input sanitization in the model-based scanner, and a hybrid detection policy—that push performance past all three vendor defaults.

What carries the argument

The mechanism that carries the comparison is a two-part evaluation pipeline. First, a dataset of 11,000 verified prompt-leak samples is built in 11 classes—naive, context ignoring, context manipulation, prefix injection, leet obfuscation, and combinations—with every sample pre-tested to confirm it actually leaks the target system prompt. Second, each scanner's continuous detection score is converted into a binary alarm by choosing the threshold that maximizes an Fβ score with β = 1/11, which treats precision as eleven times more important than recall because the test set has eleven malicious samples per benign sample. This threshold choice defines the 'peak performance' used for every comparison. Two concrete mechanisms carry the proposed fixes: recasting the canary word in the system message as a secret the model must include when asked to reveal its instructions, and sanitizing user input before interpolation into the secondary scanner's few-shot template.

What would settle it

Conduct the same evaluation on an independently constructed prompt-leak benchmark with different system instructions and a different underlying model, then check two things: whether Rebuff's balanced F1 and Fβ still beat Vigil and LLM Guard at their peak thresholds, and whether the paper's modified canary check detects substantially more leaks than the default prefix-only check. A negative result on either test would falsify the central comparison or the canary fix.

Watch

Extended reading notes

Core claim

The paper claims that the practical value of these detectors lies in how their internal techniques are wired together, not just in which models back them. Transformer-based scanners reach 97–100% recall but produce false positives. Vector-database scanners are strong only for attack classes represented in their store, and Vigil's prompt-response similarity scanner is unsuitable for this task. Canary-word checks, as implemented by Vigil and Rebuff, never fire because the model either treats the canary as a comment or does not regard it as part of the system instructions; Rebuff's check improves dramatically once the system message explicitly tells the model it holds a secret and to include it when asked for initial instructions. Rebuff's secondary model-based scanner is vulnerable to a delimiter-injection: an attacker appends a forged field and a score line so the scanner evaluates a benign prompt instead. At peak configuration, the paper reports Vigil with zero false positives and recall 0.838, Rebuff with recall 0.981 and false-positive rate 0.034, and LLM Guard with recall 0.999 and false-positive rate 0.127; a hybrid policy combining scanners from Vigil and Rebuff achieves recall 0.981 with false-positive rate 0.016 and the highest Fβ among all configurations. Finally, running the same 11,000 samples against newer models shows 2% attack success on GPT-4o and 19% on Claude-3-5-sonnet, so active detection remains relevant.

Load-bearing premise

The evaluation assumes that the 11,000 malicious samples—all verified to leak the system prompt of a single document-chat application built on one assistant model, and all filtered to successful attacks before detection testing—are representative of prompt leak attacks in general.

Editorial extensions

If this is right

  • Deployments that cannot tolerate false alarms should configure Vigil with a high transformer threshold and rely on its Yara and vector-database scanners, since the paper measured zero false positives at recall 0.838.
  • Users of Rebuff should apply the proposed canary-handling instruction and input sanitization before relying on its secondary-model scanner; the paper's measured Rebuff numbers assume those changes.
  • Detection policies should treat vector-database hits as high-confidence, pair transformer and secondary-model scanners to suppress false positives, and trigger on Yara, heuristics, or canary alarms independently; the paper's hybrid policy measured the best Fβ.
  • Prompt-leak protections cannot rely on model hardening alone, because 2% of attack samples still succeeded on GPT-4o and 19% on Claude-3-5-sonnet, so active detection remains necessary.
  • Output-based detectors should expect evasion via transformed leaks such as word reversal, added prefixes, leet, or base64, since canary checks assume plain-text recovery of system instructions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The field-forging evasion is probably not unique to Rebuff: any detector that interpolates raw input into a labeled prompt template with few-shot examples is exposed to the same delimiter injection, so sanitizing newlines and field names is a general design rule.
  • Framing the canary as a secret in the system message shifts the burden onto the model's instruction-following; a natural extension is to plant multiple trap secrets in tool descriptions, memory buffers, or retrieval contexts to catch indirect leaks of non-system content.
  • The paper's finding that the triple combination of context ignoring, leet, and context manipulation is the most evasive suggests that detection benchmarks should include combinatorial attacks as a standard test class, not just single techniques.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. This paper presents an empirical evaluation of three open-source prompt-injection detection solutions — LLM Guard, Vigil, and Rebuff — for the specific attack class of prompt leaks. The authors build a dataset of 11,000 malicious prompts by combining five prompt-leaking techniques (naive, context ignoring, context manipulation, prefix injection, leet obfuscation) into 11 classes of 1,000 samples each, and verify that every sample actually leaks the system prompt of their single LangChain/GPT-3.5-turbo document-chat application; they add 1,000 benign prompts. For each detection scanner (transformer-based, vectordb-based, Yara/heuristics-based, prompt-response similarity-based, secondary-LLM-based, and canary word checks), they compute detection scores on all 12,000 samples and choose per-scanner thresholds by ROC/PR analysis maximizing an Fβ metric with β=1/11 on the same data. They report per-technique results, find that the default canary checks in Vigil and Rebuff detect essentially no prompt leaks, propose and test a modified canary instruction for Rebuff, demonstrate a template-injection evasion of Rebuff's secondary-model scanner with a proposed sanitization mitigation, and finally compare whole-solution detection policies, concluding that Vigil is optimal when false positives must be minimized and Rebuff is the most balanced choice at peak performance.

Significance. The paper is one of the first direct head-to-head evaluations of these detectors on prompt leakage, and its dataset construction — 1,000 verified-successful attacks per class across combinations of five techniques — is more systematic than the manual datasets in much related work. The strongest contributions are implementation-level findings that are robust within the tested setup: the default canary checks in Vigil and Rebuff exhibit essentially zero true positive rate across all attack classes, the modified canary instruction measurably improves detection, and the secondary-model scanner in Rebuff can be evaded by forging fields of its prompt template (Figure 6), with a concrete mitigation proposed. The paper gives tool version numbers, model names, and vector-store identities that make the experiments reproducible.

major comments (4)
  1. [III-A, III-B; Tables III–V and VIII] The 'peak performance' comparison is computed with thresholds selected on the same 12,000 samples used to report the metrics: the transformer threshold (0.999), the vectordb thresholds (0.17217, 0.82783, 0.83483), and the secondary-model thresholds (0.802, 0.752) are each optimized by ROC/PR analysis on the full dataset, and Table VIII then reports the resulting metrics as the solutions' performance. No held-out split or cross-validation is used, and the stochastic LLM-based scanners (GPT-3.5-turbo, GPT-4o) are run once per sample with no error bars. The headline numbers are therefore in-sample best-case values: Vigil's FPR of 0.000 in Table VIII and its ranking over Rebuff and LLM Guard would likely change on a validation set. The paper should either add a validation protocol (e.g., threshold tuning on one split and reporting on another, or repeated runs with confidence intervals) or re-scope the abstract and conclusions to 'on our dataset with in-sample thresholds'.
  2. [III-B (Table VIII); II-D-1 vs III-A] The Rebuff row in Table VIII (recall 0.981, FPR 0.034) does not describe the shipped product: the paragraph below the table states that these metrics assume an upgraded canary check (with canary-handling instructions) and 'an error handling with default blacklisting' in the secondary-model scanner. The abstract's conclusion that 'Rebuff is the most optimal for average needs' therefore refers to a proposed modified configuration, not to the default open-source package, and this distinction must be made in the abstract and conclusion. Relatedly, the treatment of secondary-model parse errors is internally inconsistent: Section II-D-1 states the score is set to zero on a parse error, while Section III-A states error cases were assumed to have score 1.0 and treated as malicious; these choices produce different recall and FPR values and both feed into Table VIII. Please state the scoring protocol unambiguously and, if both behaviors were used, report metrics under each.
  3. [II-C, V; Table VI] The attack dataset is not a sample from the general population of prompt leaks. Every one of the 11,000 malicious prompts was derived from five manually seeded techniques, verified to leak the system prompt of a single LangChain/GPT-3.5-turbo document-chat application, and filtered to successful attacks before any detection test; Section V concedes that no real-world applications with diverse system instructions were used. Aggregate metrics and the Table VIII ranking are conditional on a fixed class mixture (1,000 per class) and a single system prompt, and Table VI shows per-class recall ranging from 55.9% to 100% for the same detector, so a different class mixture would change the overall recall and FPR materially. The conclusion that 'Rebuff is the most optimal for average needs' should be re-scoped to the tested distribution and target, or supported by a sensitivity analysis (e.g., reweighting of classes and at least one additional target application or system prompt).
  4. [IV-A] The proposed mitigation for the secondary-model evasion — sanitizing user prompts by filtering context-delimiter strings and newline characters before insertion into Rebuff's template — is not validated anywhere in the paper. There is no experiment showing that the sanitized scanner detects the suffix-evasion samples of Figure 6, nor any measurement of the effect of sanitization on detection scores or on false positives for the benign set. Since the abstract lists this mitigation as a contribution, it should be implemented and re-run on the test suite, or the claim should be explicitly downgraded to an untested recommendation.
minor comments (7)
  1. [Abstract] The abstract's 'presents analyzes of distinct prompt leakage detection techniques' should read 'presents analyses'.
  2. [Table IV and III-A] Table IV gives the GPT-3.5-turbo optimal threshold as 0.802, while Section III-A twice refers to the calculated threshold as 0.952 (which is actually the Fβ value of that column); the text is also inconsistent with the reported FPR behavior, since a threshold below 0.9 is what raises the false positive rate. Please correct.
  3. [Tables V, VI; III-B] The Vigil vectordb threshold appears as 0.17317 in Table V but as 0.17217 in Table VI and Section III-B; the Rebuff extended-store threshold appears as 0.83483 in Table V but as 0.82783 in the Table VI header. Please harmonize.
  4. [II-C] Please document how the 1,000 benign prompts were created (sources, topics, length distribution, and whether they were checked for overlap with the attack templates), since the false-positive rate is the principal differentiator in Table VIII.
  5. [III-B] 'low false true positive rate' should read 'low false positive rate'; also, minor language errors elsewhere (e.g., 'we calculate its detections scores' in Section III-A, 'an LLM application' in the introduction) should be cleaned up.
  6. [III-B] The ordering of solutions depends on the metric: by F1, LLM Guard (0.994) and Rebuff (0.989) lead, while by Fβ, Vigil (0.998) leads. Because β=1/11 is an analyst-chosen weight, please state explicitly how the ranking changes over a small range of β, or justify that the headline conclusions do not depend on this choice.
  7. [References] Reference [13] appears as '[redacted for anonymity]' although the manuscript is submitted with author names; please make the anonymization consistent or include the URL. Some references also contain line-break artifacts in URLs; please ensure they render correctly.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: empirical benchmark with explicitly in-sample threshold optimization and acknowledged external-validity limits.

full rationale

The paper is a measurement study: it constructs a prompt-leak dataset, runs three open-source detectors over it, computes detection metrics, and reports rankings and implementation findings. There is no derivation chain in which an output quantity is defined in terms of the claimed result. Thresholds are chosen by ROC/PR analysis on the same test set and are explicitly labeled as "peak performance," so this is in-sample threshold selection rather than a fitted parameter being renamed as a prediction; the paper never claims out-of-sample generalization for those thresholds. The attack dataset is filtered to successful leaks against one GPT-3.5-turbo application, which is an external-validity limitation that the authors acknowledge in Section V, not a circularity: detection difficulty may differ on other prompts, but the paper's measured numbers are exactly the measured numbers. The canary-check improvements and the Rebuff secondary-model evasion are concrete implementation-level findings with observed mechanisms (canary not included in outputs, forged "User string" fields in the scanner template), not results equivalent to the inputs by construction. There are no load-bearing self-citations: the only self-referential item is the redacted GitHub link to the authors' own target application, which is used as an experimental artifact rather than as theoretical support. No step in the paper reduces to its own inputs; the central claims are empirical observations on a constructed benchmark.

Assumptions & free parameters 8 free parameters · 5 assumptions · 0 invented entities

The central benchmark rests on the authors' constructed dataset, not on a mathematical derivation. The main paid-in inputs are the per-scanner score thresholds selected by ROC/F-beta analysis on the same data, the beta=1/11 weighting choice, and domain assumptions about representativeness of the filtered attack set and benign prompts. No new entities are introduced.

free parameters (8)
  • Vigil transformer threshold = 0.999 (optimal); 0.98 recommended
    Chosen by maximizing F-beta with beta=1/11 on the full test dataset; reported metrics depend on it. Section III-A, Table III.
  • Vigil vectordb threshold = 0.17317 cosine distance
    Fitted via ROC/PR analysis on the same dataset. Section III-A, Table V.
  • Rebuff vectordb threshold, default store = 0.82783 cosine similarity
    Fitted via ROC/PR analysis on the same dataset. Section III-A, Table V.
  • Rebuff vectordb threshold, extended store = 0.83483 cosine similarity
    Fitted via ROC/PR analysis on the same dataset. Section III-A, Table V.
  • Rebuff secondary model threshold, GPT-3.5-turbo = 0.952 (text); Table IV header prints 0.802
    Chosen as optimal threshold on the test dataset; affects recall and FPR reported in Table IV.
  • Rebuff secondary model threshold, GPT-4o = 0.752
    Chosen as optimal threshold on the test dataset; used in the headline Rebuff comparison. Section III-A, Table IV.
  • LLM Guard evaluation threshold = 0.9
    LLM Guard only emits 0/1 due to an internal 0.92 threshold; the authors choose 0.9 for metric calculation. Section III-A, Table III.
  • F-beta weighting beta = 1/11
    Chosen by the authors to compensate for the 11:1 class imbalance; scanner rankings depend on this weighting. Section III-A, equation (1).
assumptions (5)
  • domain assumption The 11,000 malicious samples, filtered to those that successfully leak the target system prompt, are representative of prompt leak attacks for comparing detectors.
    Section II-C describes generating, testing, and filtering samples; all later scanner metrics use this dataset.
  • domain assumption The 1,000 benign prompts are representative of legitimate user traffic to an LLM application.
    Section II-C defines the negative set as arbitrary legitimate prompts; all false positive rates are computed against this set.
  • domain assumption Single-run detection scores from stochastic LLM-based scanners are stable enough for threshold optimization and comparison.
    No repeated runs or confidence intervals are reported; Section III derives thresholds and metrics from one score per sample.
  • standard math ROC curve analysis and F-beta maximization give valid optimal thresholds for comparing scanners.
    Section III-A uses this procedure for each scanner; the resulting thresholds are treated as peak performance.
  • domain assumption Attack success is judged as the model output containing the system instructions; no automated success metric is specified.
    Section II-C states samples are verified by running prompt leak success tests, but the exact success criterion is not detailed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing Security in LLM Applications: A Performance Evaluation of Early Detection Systems." pith.science (2026). https://pith.science/paper/PDKTABKF

@misc{pith2026250619109,
  author       = {Pith},
  title        = {Pith review of: Enhancing Security in LLM Applications: A Performance Evaluation of Early Detection Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PDKTABKF}},
  note         = {Machine review of arXiv:2506.19109}
}
read the original abstract

Prompt injection threatens novel applications that emerge from adapting LLMs for various user tasks. The newly developed LLM-based software applications become more ubiquitous and diverse. However, the threat of prompt injection attacks undermines the security of these systems as the mitigation and defenses against them, proposed so far, are insufficient. We investigated the capabilities of early prompt injection detection systems, focusing specifically on the detection performance of techniques implemented in various open-source solutions. These solutions are supposed to detect certain types of prompt injection attacks, including the prompt leak. In prompt leakage attacks, an attacker maliciously manipulates the LLM into outputting its system instructions, violating the system's confidentiality. Our study presents analyzes of distinct prompt leakage detection techniques, and a comparative analysis of several detection solutions, which implement those techniques. We identify the strengths and weaknesses of these techniques and elaborate on their optimal configuration and usage in high-stake deployments. In one of the first studies on existing prompt leak detection solutions, we compared the performances of LLM Guard, Vigil, and Rebuff. We concluded that the implementations of canary word checks in Vigil and Rebuff were not effective at detecting prompt leak attacks, and we proposed improvements for them. We also found an evasion weakness in Rebuff's secondary model-based technique and proposed a mitigation. Then, the result of the comparison of LLM Guard, Vigil, and Rebuff at their peak performance revealed that Vigil is optimal for cases when minimal false positive rate is required, and Rebuff is the most optimal for average needs.

Figures

Figures reproduced from arXiv: 2506.19109 by the authors.

Figure 1
Figure 1. Target LLM application design [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 3
Figure 3. Prompt template used by Rebuff’s secondary [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Detection scores by Vigil’s Prompt-Response similarity-based scanner, on each prompt leak class [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Detection rates for whole solutions on each prompt leak class [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Vulnerability in Rebuff’s model check scanner [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: Resulting ASR of prompt leaks against GPT-4o and [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 21 canonical work pages

  1. [1]

    GitHub - jthack/PIPE: Prompt Injec- tion Primer for Engineers — github.com

    Joseph Thacker. GitHub - jthack/PIPE: Prompt Injec- tion Primer for Engineers — github.com . https://github .com/jthack/PIPE. [Accessed 10-05-2024]. 2023

    work page 2024

  2. [2]

    OW ASP Foundation

    Inc. OW ASP Foundation. OWASP Top 10 for LLM Applications — llmtop10.com . https://llmtop10.com/l lm01/. [Accessed 10-05-2024]. 2023

    work page 2024

  3. [3]

    Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applica- tions with Indirect Prompt Injection

    Kai Greshake et al. Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applica- tions with Indirect Prompt Injection . 2023. arXiv: 23 02.12173 [cs.CR]

    work page 2023

  4. [4]

    I don’t know how to solve prompt injection — simonwillison.net

    Simon Willison. I don’t know how to solve prompt injection — simonwillison.net . https://simonwillison.ne t/2022/Sep/16/prompt- injection- solutions/. [Accessed 10-05-2024]. 2022

    work page 2022

  5. [5]

    Ignore Previous Prompt: Attack Techniques For Language Models

    F ´abio Perez and Ian Ribeiro. Ignore Previous Prompt: Attack Techniques For Language Models . 2022. arXiv: 2211.09527 [cs.CL]

    arXiv 2022

  6. [6]

    Protect AI

    Inc. Protect AI. Index - LLM Guard — llm-guard.com . https://llm-guard.com. [Accessed 10-05-2024]. 2023

    work page 2024

  7. [7]

    Release Blog — Vigil: Documentation — vigil.deadbits.ai

    deadbits.ai. Release Blog — Vigil: Documentation — vigil.deadbits.ai. https://vigil.deadbits.ai/overview/relea se-blog. [Accessed 10-05-2024]. 2023

    work page 2024

  8. [8]

    GitHub - protectai/rebuff: LLM Prompt In- jection Detector — github.com

    ProtectAI. GitHub - protectai/rebuff: LLM Prompt In- jection Detector — github.com . https://github.com/prot ectai/rebuff. [Accessed 10-05-2024]. 2023

    work page 2024

Show all 29 references
  1. [9]

    You can’t solve AI security problems with more AI — simonwillison.net

    Simon Willison. You can’t solve AI security problems with more AI — simonwillison.net. https://simonwillison .net/2022/Sep/17/prompt-injection-more-ai/. [Accessed 27-06-2024]. 2022

    2022

  2. [10]

    Jailbroken: How Does LLM Safety Training Fail? 2023

    Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How Does LLM Safety Training Fail? 2023. arXiv: 2307.02483 [cs.LG]

    2023 arXiv

  3. [11]

    Dont you (forget NLP): Prompt injection with control characters in ChatGPT — dropbox.tech

    Win Suen Mark Breitenbach Adrian Wood and Po-Ning Tseng. Dont you (forget NLP): Prompt injection with control characters in ChatGPT — dropbox.tech . https: //dropbox.tech/machine- learning/prompt- injection- w ith- control- characters- openai- chatgpt- llm. [Accessed 10-05-2024]. 2023

    2024

  4. [12]

    https://en.wikipe dia.org/wiki/Leet

    Leet - Wikipedia — en.wikipedia.org . https://en.wikipe dia.org/wiki/Leet. [Accessed 10-05-2024]

    2024

  5. [13]

    [redacted for anonymity]

    [redacted for anonymity]. [redacted for anonymity] . [redacted for anonymity]. [Accessed 10-05-2024]. 2024

    2024

  6. [14]

    Fine-Tuned DeBERTa-v3-base for Prompt Injection Detection

    ProtectAI.com. Fine-Tuned DeBERTa-v3-base for Prompt Injection Detection . 2024. URL: https://huggin gface.co/ProtectAI/deberta-v3-base-prompt-injection-v 2

    2024

  7. [15]

    Prompt Injection Attack on GPT-4 — Robust Intelligence — robustintelligence.com

    Robust Intelligence. Prompt Injection Attack on GPT-4 — Robust Intelligence — robustintelligence.com . https: //www.robustintelligence.com/blog-posts/prompt-inject ion-attack-on-gpt-4. [Accessed 10-05-2024]. 2023

    2024

  8. [16]

    ChatGPT Plugins: Data Exfiltration via Images & Cross Plugin Request Forgery · Embrace The Red — embracethered.com

    Wundersuzzi. ChatGPT Plugins: Data Exfiltration via Images & Cross Plugin Request Forgery · Embrace The Red — embracethered.com . https://embracethered.com /blog/posts/2023/chatgpt-webpilot-data-exfil-via-mark down-injection/. [Accessed 10-05-2024]. 2023

    2023

  9. [17]

    Vector Similarity Explained — Pinecone — pinecone.io

    Roie Schwaber-Cohen. Vector Similarity Explained — Pinecone — pinecone.io . https://www.pinecone.io/lear n/vector-similarity/. [Accessed 10-05-2024]. 2024

    2024

  10. [18]

    PLeak: Prompt Leaking Attacks against Large Language Model Applications

    Bo Hui et al. PLeak: Prompt Leaking Attacks against Large Language Model Applications . 2024. arXiv: 240 5.06823 [cs.CR]. URL: https://arxiv.org/abs/2405.06 823

    2024

  11. [19]

    Effective Prompt Extraction from Language Models

    Yiming Zhang, Nicholas Carlini, and Daphne Ippolito. Effective Prompt Extraction from Language Models

  12. [20]

    Why Are My Prompts Leaked? Unrav- eling Prompt Extraction Threats in Customized Large Language Models

    Zi Liang et al. Why Are My Prompts Leaked? Unrav- eling Prompt Extraction Threats in Customized Large Language Models. 2025. arXiv: 2408.02416 [cs.CL]. URL: https://arxiv.org/abs/2408.02416

    2025 arXiv

  13. [21]

    Raccoon: Prompt Extraction Bench- mark of LLM-Integrated Applications

    Junlin Wang et al. “Raccoon: Prompt Extraction Bench- mark of LLM-Integrated Applications”. In: Findings of the Association for Computational Linguistics ACL

  14. [22]

    ROUGE: A Package for Automatic Evaluation of Summaries

    Chin-Yew Lin. “ROUGE: A Package for Automatic Evaluation of Summaries”. In: Text Summarization Branches Out. Barcelona, Spain: Association for Com- putational Linguistics, July 2004, pp. 74–81. URL: http s://www.aclweb.org/anthology/W04-1013

    2004

  15. [23]

    13349–13365

    Association for Computational Linguistics, 2024, pp. 13349–13365. DOI: 10.18653/v1/2024.findings-acl .791. URL: http://dx.doi.org/10.18653/v1/2024.findings -acl.791

    2024 doi

  16. [24]

    GitHub - whylabs/langkit: LangKit: An open- source toolkit for monitoring Large Language Models (LLMs)

    WhyLabs. GitHub - whylabs/langkit: LangKit: An open- source toolkit for monitoring Large Language Models (LLMs). https://github.com/whylabs/langkit/tree/main. [Accessed 10-05-2024]. 2023

    2024

  17. [25]

    Whispers in the Machine: Confi- dentiality in LLM-integrated Systems

    Jonathan Evertz et al. Whispers in the Machine: Confi- dentiality in LLM-integrated Systems . 2024. arXiv: 240 2.06922 [cs.CR]. URL: https://arxiv.org/abs/2402.06 922

    2024

  18. [26]

    Baseline Defenses for Adversarial Attacks Against Aligned Language Models

    Neel Jain et al. Baseline Defenses for Adversarial Attacks Against Aligned Language Models. 2023. arXiv: 2309.00614 [cs.LG]. URL: https://arxiv.org/abs/2309 .00614

    2023 arXiv

  19. [27]

    From Prompt Injections to SQL In- jection Attacks: How Protected is Your LLM-Integrated Web Application? 2023

    Rodrigo Pedro et al. From Prompt Injections to SQL In- jection Attacks: How Protected is Your LLM-Integrated Web Application? 2023. arXiv: 2308.01990 [cs.CR]

    2023 arXiv

  20. [29]

    Jatmo: Prompt Injection Defense by Task-Specific Finetuning

    Julien Piet et al. Jatmo: Prompt Injection Defense by Task-Specific Finetuning . 2024. arXiv: 2312.17673 [cs.CR]. URL: https://arxiv.org/abs/2312.17673. 18

    2024 arXiv

  21. [2024]

    arXiv: 2307.06865 [cs.CL]. 17

    arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.