Pith. sign in

REVIEW 4 major objections 8 minor 8 references

Prompt-injection defenses that shine on synthetic benchmarks can fail on real documents; a new fact-preserving sanitization pipeline, PARSE, cuts attack success from 25.4% to 15.6% while keeping utility near baseline.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 11:01 UTC pith:IHKHPKH2

load-bearing objection A useful real-document benchmark and a practical defense, but the unnamed target agent and unvalidated LLM judge make the headline numbers hard to trust as-is. the 4 major comments →

arxiv 2606.17467 v2 pith:IHKHPKH2 submitted 2026-06-16 cs.CR cs.CL

PARSE: Provenance-Aware Retrieval Sanitization for Professional Domain LLM Agents

Aaditya Pai This is my paper
classification cs.CR cs.CL
keywords prompt injectionLLM agentsretrieval sanitizationdomain-camouflaged injectionfact preservationdirectiveness gateRAG securityenterprise documents
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that the standard way of testing prompt-injection defenses—synthetic, short benchmark tasks—misleads practitioners: the best synthetic defense, paraphrasing, shows no significant attack reduction on real documents (p=0.500) and degrades utility by 9 points. It introduces PARSE, an inference-time pipeline that classifies each sentence by directive content, extracts the facts an agent needs, rewrites high-risk sentences under a fact-preservation constraint, and checks the output against the extracted facts. On a new 122-task benchmark of real SEC, Federal Register, PubMed, arXiv, and GitHub postmortem documents, PARSE lowers attack success rate from 25.4% to 15.6% (a 38% reduction, p=0.014) at 86.9% utility—the only condition that is both statistically significant and near-baseline in utility. The broader point is that defenses should be evaluated on domain-matched real documents, not synthetic proxies.

Core claim

The paper's central claim is that synthetic-benchmark results do not predict real-world prompt-injection defense performance, and that PARSE—a six-component inference-time sanitization pipeline—succeeds where paraphrasing fails because it preserves the facts an agent needs while neutralizing directive language. On a 122-task benchmark built from actual SEC filings, Federal Register rules, PubMed abstracts, arXiv papers, and GitHub postmortems, PARSE reduces attack success rate from 25.4% to 15.6% (p=0.014, adequately powered) at 86.9% utility. Paraphrasing, the strongest synthetic defense in prior work, shows no significant reduction on the same real documents (p=0.500) and drops utility to

What carries the argument

The load-bearing mechanism is the directiveness gate paired with a fact-preserving rewrite loop. The gate scores each document for directive content and routes 59% of low-risk documents to a cheap paraphrase, reserving full treatment for the 41% at risk. High-risk documents then pass through a tagger-extractor that labels sentences as factual or directive and scores injection likelihood; a structure-aware paraphraser rewrites high-scoring sentences under a hard constraint that every extracted fact appears in the output; and a consistency checker verifies fact presence with one retry. This closed loop is what distinguishes PARSE from indiscriminate paraphrasing and explains both its security

Load-bearing premise

The results stand only if the specific LLM agent, the LLM-as-judge for task success and attack detection, and the directiveness threshold tuned on this benchmark are representative of real enterprise deployments.

What would settle it

A replication on a fresh set of real documents using a different agent model and human-verified ground truth, where PARSE's attack-success reduction over baseline is not statistically significant at adequate power, or where its utility is more than a few points below baseline, would falsify the central claim. A cruder check: if the directiveness gate routes a clearly prescriptive document below 0.5, the threshold is not transferable.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • PARSE is the only evaluated condition that is both statistically significant (p=0.014) and within about five points of baseline utility, making it the only defense with a usable security–utility tradeoff on real documents.
  • Paraphrasing should not be deployed alone for real enterprise RAG: it shows no measurable attack reduction and costs nine points of utility.
  • Synthetic benchmark rankings are not reliable predictors of deployment-time performance; defenses need domain-matched real-document evaluation.
  • The directiveness gate allows PARSE to run at ingestion time for roughly $0.01 per document, with 59% of documents using only two API calls, so the security benefit is affordable for typical corpora.
  • Llama Guard reaches 18.9% attack success but at a 27-point utility loss, disqualifying it for production; PARSE's attack success is 3.3 points lower with 22 points more utility.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A likely next evaluation is an adaptive adversary who knows the sanitization pipeline; the paper does not test this, and such payloads could exploit the low-directiveness path that routes 59% of documents to simple paraphrasing.
  • The directiveness threshold of 0.5 was tuned on this benchmark's validation set; in a different corpus a recalibrated threshold may be needed, and the 59/41 split should not be expected to hold universally.
  • The consistency-checker idea suggests a broader evaluation principle: defenses should be scored on how much task-relevant information survives sanitization, not just on attack-blocking rate; PARSE's utility measurement is a step in that direction.
  • Because the benchmark's ground truth depends on an LLM judge, absolute attack-success numbers could shift with a different judge, though the relative ordering of conditions may be more stable.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 8 minor

Summary. This paper argues that prompt-injection defenses evaluated on synthetic benchmarks do not generalize to real enterprise documents. It introduces a new benchmark of 122 tasks across five professional domains using actual SEC filings, Federal Register rules, PubMed abstracts, arXiv papers, and GitHub postmortems. The paper evaluates eight defense conditions, including a proposed PARSE pipeline that uses a directiveness gate, sentence-level injection scoring, fact extraction, and a consistency checker. The headline findings are that paraphrasing, the strongest synthetic-benchmark defense, is a null result on real documents (p=0.500) and degrades utility by 9 points, while PARSE achieves 15.6% attack success rate versus a 25.4% baseline (p=0.014) at 86.9% utility, which the authors claim is the only condition that is both statistically significant and utility-preserving.

Significance. If the results hold, the paper makes a useful contribution by providing a real-document benchmark and by demonstrating that synthetic results may not transfer. PARSE is an inference-time, training-free pipeline with a plausible mechanism (directiveness gating, fact-preserving rewriting, consistency checks), and the release of code/benchmark is a strength. However, the significance is substantially undercut by evaluation transparency gaps: the target agent model is never named, the LLM-as-judge is unvalidated, the headline p-value does not survive the paper's own Bonferroni correction, and the power claim is based on the observed effect size. These issues currently prevent the empirical claims from being interpreted reliably.

major comments (4)
  1. [Sec. 4.3 (ASR definition) and Sec. 4.1 (task construction)] The target agent model is never identified. The ASR is defined as the fraction of trials where 'the agent' follows the injected instruction, but no model name, system prompt, sampling temperature, retrieval setup, or other inference hyperparameters are given. The only model names in the paper are claude-sonnet-4-5 (used to generate tasks/payloads) and Haiku/Sonnet (used for PARSE components). Prompt-injection defenses are known to be highly model-dependent; without naming the agent, the headline numbers in Table 1 cannot be reproduced, and the 38% relative reduction may not generalize. Please specify the target agent model and all inference parameters, or report results across multiple agent families.
  2. [Sec. 4.1 and Sec. 4.3 (ground truth and utility evaluation)] Both malicious-output detection and utility are assessed by an LLM-as-judge, but the manuscript does not provide the judge prompt, pass criteria, model identity, or any human validation. The Limitations explicitly state that 'human validation of a sample is future work.' This is load-bearing because the utility comparison (e.g., PARSE 86.9% vs. Llama Guard 64.8%) and the ASR ground truth depend on an unvalidated judge. If the judge shares the agent's model family, it may systematically favor PARSE's output style. The authors should disclose judge details, report a sample of judge decisions, and provide inter-annotator agreement with human raters.
  3. [Sec. 5.4 and Table 2 (statistical claims)] The abstract and Finding 1 describe PARSE's p=0.014 as 'statistically significant' without qualification, yet the paper adopts a Bonferroni threshold of α=0.0071 for seven comparisons, and p=0.014 does not survive that correction. Sec. 5.4 acknowledges this but still frames p=0.014 as 'meaningful given adequate power.' The adequacy claim is circular: n_min=103 is computed from the observed effect size h=−0.245, a post-hoc power calculation. This does not justify the significance claim. Please report the Bonferroni-adjusted conclusion explicitly, avoid calling p=0.014 significant without a caveat, and present a pre-specified minimum effect size for power analysis.
  4. [Sec. 4.1 (task construction) and Sec. 5.2 (directiveness gate)] The evaluation has several degrees of freedom that are tuned or generated on the same benchmark: the attack class is defined in the author's prior work (Pai 2026a); tasks and payloads are generated by claude-sonnet-4-5; and the directiveness threshold of 0.5 was tuned on a validation set, as admitted in the Limitations. This risks overfitting to the specific benchmark and attack-generation pipeline. The manuscript should provide a sensitivity analysis for the gate threshold and clarify how the benchmark generation avoids encoding PARSE's assumptions. If feasible, an external benchmark or an adversarial evaluation would strengthen the claim.
minor comments (8)
  1. [Abstract and Sec. 5.4] The phrase 'the only condition that is both statistically significant (p=0.014, adequately powered)' is misleading because p=0.014 is not significant at the paper's own Bonferroni threshold. Please add a qualifier such as 'uncorrected' or 'approaching the Bonferroni threshold.'
  2. [Sec. 4.2 and References] The defense condition is labeled 'Llama Guard 4' but the reference is to 'Llama guard 3' (Meta AI, 2024). Please clarify which version was actually used and update the citation accordingly.
  3. [Sec. 4.3] McNemar's exact test is one-sided. Since a defense could in principle increase ASR (as parse_fast does), a two-sided test is more natural; please justify the one-sided choice or report two-sided p-values.
  4. [Table 2] Llama Guard is listed as 'underpowered (n_min=247)' despite reaching p=0.004 and surviving Bonferroni. Power is not needed to interpret a significant result; this labeling is confusing and should be revised.
  5. [Sec. 3 and Figure 1] Figure 1 is referenced but the actual figure content is not visible in the manuscript text; the caption lists pipeline components, but the visual layout would benefit from more detail (e.g., inputs/outputs of each step). Please ensure the figure is legible and self-contained.
  6. [References] The reference to Pai (2026b) is 'Manuscript in preparation' and Pai (2026a) is a self-citation to an arXiv preprint. Load-bearing claims about the camouflage attack class and prior benchmark rankings depend on these works; please clarify their availability or provide more detailed background.
  7. [Sec. 6.3] The cost estimate of $0.01 per document and $100 per 10,000-document corpus is stated without a pricing source or calculation. Please provide a transparent cost model or cite current API pricing.
  8. [Limitations] The Limitations note that 'bootstrap confidence intervals are recommended for the camera-ready version.' This should be addressed in this revision, not deferred to a future version; the paper should report confidence intervals for ASR and utility differences now.

Circularity Check

1 steps flagged

Post-hoc power analysis is circular, and the evaluation is substantially self-referential, but the central ASR/utility comparison is not forced by construction.

specific steps
  1. other [Section 4.3 (Statistical tests) and Section 5.4 (Statistical Summary)]
    "Statistical power is assessed via the minimum n required to detect the observedhat 80% power. ... With n=122> nmin=103, the test is adequately powered. ... We interpret p=0.014 as meaningful given adequate power ( n=122> nmin=103) and the largest effect size of all conditions."

    The minimum n (n_min=103) is computed from the same observed effect size h=-0.245 that the experiment is being used to estimate. Calling the test 'adequately powered' because the observed n exceeds a threshold derived from the observed h is a post-hoc power calculation: it is a deterministic transform of the same p-value/effect-size pair and provides no independent evidence. The claimed support for PARSE's significance — 'p=0.014 is meaningful given adequate power' — therefore reduces to the observed result itself rather than to a pre-specified test plan.

full rationale

The paper's central empirical claim — PARSE achieves 15.6% ASR vs 25.4% baseline at 86.9% utility — is a measured outcome on a constructed benchmark, not an equation that reduces to its inputs. The benchmark, tasks, and judge are LLM-generated with human validation deferred, and the directiveness threshold was tuned on a validation set without a documented disjoint test split; these are real threats to external validity and can inflate the reported numbers, but they do not make the ASR computation equivalent to a fitted parameter by construction under the strict definition used here. The self-citations to Pai (2026a, 2026b) define the attack class and label paraphrasing the strongest synthetic-benchmark defense, but the present paper independently re-measures paraphrasing and PARSE, so the central comparison does not logically depend on those citations. The one defensible circular step is the statistical-power claim: n_min is derived from the observed effect, so 'adequately powered' is a restatement of the observed h and p-value rather than independent evidence. That supports a moderate partial-circularity score rather than a finding that the derivation is forced.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 1 invented entities

PARSE's empirical advantage depends on several hand-tuned thresholds and a self-constructed benchmark, so the contribution is sensitive to the exact choices made in this paper.

free parameters (3)
  • Directiveness gate threshold = 0.5
    Routes docs to full PARSE vs simple paraphrase; tuned on a validation set per Limitations; directly affects ASR/utility.
  • Injection-score thresholds = 0.3 / 0.6
    Step 4 rewrite aggressiveness bands (preserve/light/aggressive) are chosen by hand without calibration data.
  • Consistency-check retry count = 1
    Step 5 retries Step 4 once on failure; retry count is arbitrary.
axioms (4)
  • domain assumption Domain-camouflaged injection, as defined in Pai (2026a), is a realistic and representative attack class.
    The entire benchmark and PARSE are built around this self-cited attack; if this attack class is unrepresentative, the defense addresses a strawman.
  • domain assumption LLM-as-judge correctly determines both legitimate-task success and malicious-goal following.
    Section 4.1/4.3: ground truth and utility rely on an LLM judge with no human validation (Limitations).
  • ad hoc to paper Post-hoc power analysis using the observed effect size (h=-0.245) is a valid basis for 'adequately powered' claims.
    Section 5.4 derives n_min=103 from the observed h; standard power analysis should fix an effect a priori.
  • domain assumption The 122 real documents and LLM-generated camouflage payloads are representative of real enterprise RAG deployments.
    The benchmark is drawn from public corpora; representativeness for enterprise deployments is asserted, not measured.
invented entities (1)
  • Directiveness score δ(d) no independent evidence
    purpose: Routes each document to full PARSE or simple paraphrase; core of the gate.
    A Haiku call returns δ in [0,1] but no calibration data, validation set details, or external benchmark are provided; threshold 0.5 tuned in-house.

pith-pipeline@v1.3.0-alltime-deepseek · 7539 in / 11959 out tokens · 105648 ms · 2026-08-02T11:01:59.773885+00:00 · methodology

0 comments
read the original abstract

Prompt injection defenses evaluated on synthetic benchmarks do not generalize to real enterprise documents, which are longer, denser, and interleave legitimate authority language with factual content. We demonstrate this gap with a real-document benchmark of 122 tasks across five professional domains (financial, legal, medical, scientific, DevOps) using actual SEC filings, Federal Register rules, PubMed abstracts, arXiv papers, and GitHub postmortems. Paraphrasing, the strongest defense on synthetic benchmarks, shows no statistically significant attack success rate reduction on real documents (p=0.500) while degrading utility from 91.8% to 82.8%. We introduce PARSE (Provenance-Aware Retrieval Sanitization), a domain-aware, fact-preserving sanitization pipeline that classifies each sentence by injection likelihood, extracts structured facts before rewriting, and verifies fact preservation via a consistency-checking loop. A directiveness gate routes 59% of real enterprise documents to a lightweight path, concentrating computational cost on high-risk documents. PARSE achieves 15.6% attack success rate -- a 38% reduction versus the 25.4% baseline -- at 86.9% utility, the only condition that is both statistically significant (p=0.014, adequately powered) and maintains near-baseline utility. Practitioners should evaluate defenses on domain-matched real documents, not synthetic proxies.

Figures

Figures reproduced from arXiv: 2606.17467 by Aaditya Pai.

Figure 1
Figure 1. Figure 1: PARSE pipeline. Steps 1 and 1.5 run in parallel. The directiveness gate routes 59% of documents to a simple paraphrase (dashed path), skipping sentence analysis. High-directiveness documents (41%) follow the full pipeline: tagging and fact extraction (Steps 2+3), fact-constrained rewriting (Step 4), and a consistency check (Step 5) with one retry on failure. Step 4: Structure-Aware Paraphraser. A Son￾net c… view at source ↗
Figure 2
Figure 2. Figure 2: (a) ASR–utility trade-off for all evaluated conditions. Lower ASR and higher utility is better (lower-right is [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

8 extracted references · 6 linked inside Pith

  1. [4]

    Yupei Liu, Yuqi Jia, Runpeng Geng, Jinyuan Jia, and Neil Zhenqiang Gong

    Baseline defenses for adversarial at- tacks against aligned language models.Preprint, arXiv:2309.00614. Yupei Liu, Yuqi Jia, Runpeng Geng, Jinyuan Jia, and Neil Zhenqiang Gong

    Pith/arXiv arXiv

  2. [5]

    Preprint, arXiv:2310.12815

    Formalizing and benchmarking prompt injection attacks and defenses. Preprint, arXiv:2310.12815. Meta AI

    arXiv

  3. [6]

    https:// ai.meta.com/research/publications/ llama-guard-3/

    Llama guard 3: Meta’s ap- proach to responsible AI safety. https:// ai.meta.com/research/publications/ llama-guard-3/. Aaditya Pai. 2026a. Blind spots in the guard: How domain-camouflaged injection attacks evade detection in multi-agent llm systems.Preprint, arXiv:2605.22001. Aaditya Pai. 2026b. Evaluating prompting-based de- fenses against domain-camoufl...

    Pith/arXiv arXiv

  4. [8]

    The prompt report: A sys- tematic survey of prompting techniques.Preprint, arXiv:2406.06608

    Pith/arXiv arXiv

  5. [2022]

    Preprint, arXiv:2211.09527

    Ignore previous prompt: Attack techniques for language models. Preprint, arXiv:2211.09527. Sander Schulhoff, Michael Ilie, Nishant Balepur, Kon- stantine Kahadze, Amanda Liu, Chenglei Si, Yin- heng Li, Aayush Gupta, HyoJung Han, Sevien Schul- hoff, and 1 others

    Pith/arXiv arXiv

  6. [2023]

    Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping-yeh Chiang, Micah Goldblum, Tom Goldstein, and Jonas Geip- ing

    Llama guard: LLM- based input-output safeguard for human-AI conver- sations.Preprint, arXiv:2312.06674. Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping-yeh Chiang, Micah Goldblum, Tom Goldstein, and Jonas Geip- ing

    Pith/arXiv arXiv

  7. [2024]

    Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and Madian Khabsa

    Defending against indirect prompt injection attacks with spotlighting.Preprint, arXiv:2312.14197. Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and Madian Khabsa

    Pith/arXiv arXiv

  8. [2025]

    Preprint, arXiv:2510.08829

    CommandSans: Token- level sanitization against prompt injection attacks. Preprint, arXiv:2510.08829. Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz

    arXiv