{"id":"bfdadc13-6230-49ca-8979-70ec392300d1","arxiv_id":"2412.16708","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A controlled evaluation shows skeptical prompting and robust retrievers soften the impact of adversarial corpus poisoning on RAG, with gains tied to model reasoning ability.","lead":"This paper studies how well retrieval-augmented generation (RAG) systems survive when malicious passages are slipped into their knowledge bases. It finds that prompting the model to question the retrieved context helps mostly for stronger models, and that the choice of retriever can shift accuracy by more than ten points.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"All headline claims are validated only against PoisonedRAG's black-box attack; DPR's apparent robustness and skeptical prompting's gains may not generalize to retriever-aware poisoning, a scope limitation the authors acknowledge in Section 7 but do not resolve.","rationale":"The reader's conditional verdict rests on two concerns: missing statistical validation and the single-attack external validity limitation. I agree that the attack representativeness issue is the weakest structural assumption. It is load-bearing because the retriever ordering (DPR more robust than Contriever), the model-capacity dependence of skeptical prompting, and the dilution finding are all measured under the same PoisonedRAG generation recipe. If the attack family is broader, the actionable recommendations (retriever swap, prompt change) may not transfer. The paper is not internally inconsistent; Section 7 flags this limitation explicitly. The issue is external validity rather than internal correctness. The statistical weakness is real but secondary: several headline differences are large (e.g., GPT-4 adversarial F1 jumps from 35.48 under neutral to 79.00 under skeptical prompting), so the qualitative pattern would likely survive, but reported magnitudes and smaller retriever differences need confidence intervals and repeated runs. The abstract's wording that skeptical prompting 'activates internal reasoning' is an interpretive overclaim since only answer accuracy is measured, but it does not threaten the practical defense claim. Because the reader already conditioned the verdict on this same assumption, my stress-test does not move the verdict; it reinforces the condition. I would keep CONDITIONAL and require either a retriever-aware attack baseline or explicit scoping of all conclusions to PoisonedRAG-style attacks.","tokens_in":19747,"tokens_out":6894,"duration_ms":63635,"concrete_test":"On the same NQ and HotpotQA setup, replace PoisonedRAG passages with retriever-aware adversarial passages generated by a gradient-based method (Zhong et al. 2023; Su et al. 2024) or by an LLM prompted while scoring candidate passages with the target retriever's similarity function, using the same five-passage injection protocol. Re-measure Table 1 top-1/top-10 adversarial retrieval rates and Table 2 end-to-end F1 with neutral versus skeptical prompts for GPT-3.5, GPT-4, and GPT-4o. If DPR-Single's adversarial retrieval rate is no longer below Contriever's, or if the skeptical prompting gains in Table 2 shrink to within noise, the central claims must be scoped to black-box attacks.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's practical takeaways—DPR is more robust than Contriever, skeptical prompting helps advanced models and hurts weaker ones, and increasing k does not dilute adversarial impact—are all measured with adversarial passages generated by a single recipe: PoisonedRAG's black-box LLM prompting (Section 3, Appendix B). This recipe is not retriever-aware; it optimizes only for query relevance. DPR's lower adversarial retrieval rates may therefore reflect distributional mismatch between LLM-written adversarial passages and DPR's training data, not a general robustness property. A retriever-aware attacker (e.g., gradient-based corpus poisoning as in Zhong et al. 2023 or Su et al. 2024) could craft passages that rank highly for DPR, potentially inverting the retriever ordering and changing whether skeptical prompting helps at all. The authors explicitly list 'benchmarking retrievers under more tailored attacks' as future work (Section 7), so the paper is internally honest about scope; the issue is that the abstract and conclusion state the findings as general properties of RAG under adversarial poisoning. This is the most load-bearing assumption because it underlies all three headline claims, not just one number.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper reports a controlled empirical study of retrieval-augmented generation (RAG) under adversarial corpus poisoning. It introduces a three-way taxonomy of retrieved contexts (adversarial, untouched, guiding), evaluates the generation side with six LLMs under neutral and skeptical prompts, evaluates the retrieval side with five retrievers, and then combines both in an end-to-end attack scenario. The main claims are that skeptical prompting partially defends strong LLMs while hurting weaker ones, that DPR-based retrievers expose adversarial passages less often than Contriever-based ones, and that increasing the number of retrieved passages does little to dilute the effect of a single adversarial passage. The code, model versions, and prompt templates are documented.","tokens_in":19911,"tokens_out":8537,"duration_ms":73260,"significance":"The study is useful and actionable if its claims hold: it identifies two cheap mitigation levers (prompt change, retriever choice) and a structured way of reasoning about context quality. The controlled split between generation and retrieval interactions and the release of code and implementation details are notable strengths. However, the empirical conclusions are less robust than the presentation suggests: they rest on a single black-box attack recipe and on point estimates without any uncertainty quantification. The paper is a solid diagnostic contribution rather than a definitive robustness benchmark.","major_comments":[{"comment":"The paper does not state the number of queries sampled per dataset anywhere in the main text or appendices, and no result reports error bars, confidence intervals, or significance tests. This is load-bearing for every headline claim: for example, Figure 2's conclusion that skeptical prompting improves GPT-4 but hurts GPT-3.5 is based on a single F1 point per condition, and Table 1's '47% of queries' for DPR-Single on NQ is uninterpretable without knowing whether the denominator is 50 or 1,000 queries. I request the per-dataset query counts, and for at least the key pairwise comparisons (neutral vs. skeptical per model in Figures 2-6; retriever retrieval rates in Tables 1 and 13) either query-level bootstrap confidence intervals or repeated-run estimates with a significance test.","section":"§4 and §5 (all figures and tables)"},{"comment":"All headline results are measured under a single attack generator, the PoisonedRAG black-box recipe described in Section 3 and Appendix B. This recipe is not retriever-aware, so DPR's lower adversarial retrieval rate in Table 1 and the benefits of skeptical prompting in Section 5.3 may be properties of this particular attack distribution rather than of RAG robustness generally. Section 7 explicitly lists 'benchmarking retrievers under more tailored attacks' as future work, so the limitation is acknowledged internally; my concern is that the abstract and conclusion present the findings as general properties of RAG under adversarial poisoning. Please either add experiments with a retriever-aware attack (e.g., the gradient-based methods in Zhong et al. 2023 or Su et al. 2024) or restrict the claims in the abstract and conclusion to the PoisonedRAG attack class.","section":"§3, §5.2, §7"},{"comment":"The neutral and skeptical prompts differ in two ways at once: the skeptical instruction and the answer format, since the skeptical prompt adds D(A)/D(B) options and tells the model the context may be incorrect. Consequently the F1 gains attributed to 'activating LLMs' internal reasoning' could instead reflect the model being given an explicit mechanism to override the context. A control condition that keeps the A/B/C/D structure but omits the skeptical wording (or keeps A/B/C and adds only the skeptical wording) is needed to support the paper's central defense claim.","section":"§5.1, Table 3"},{"comment":"The dilution experiment fixes one adversarial passage and adds k-1 'untouched' passages, i.e., the top-1 clean-corpus passages. Because Section 5.1.1 shows that untouched passages can themselves hurt performance, the flat F1 curve in Figure 5 does not uniquely establish that 'the primary determinant is the absolute number of adversarial passages rather than their proportion.' The design confounds the number of adversarial passages with the quality of the added clean passages. To support this conclusion the authors should vary the type of added passages (e.g., guiding passages or randomly selected passages) and report the same k-sweep for each type.","section":"§5.1.2, Figure 5"}],"minor_comments":[{"comment":"The sentence on Huang et al. lacks a year, and the phrase 'More discussion on these related works can be find in Appendix A.1' contains a grammatical error; please fix these and similar copyediting issues throughout the manuscript.","section":"§2"},{"comment":"The sentence 'This suggests that reasoning ability and robustness drive the observed performance differences, rather than because the evaluation datasets used are fairly common and likely included in the training sets of state-of-the-art models' is not supported by any contamination analysis; either add such an analysis or remove the claim.","section":"Appendix A.3"},{"comment":"The caption says the parentheses highlight changes 'when changing the prompt from skeptical to neutral', but the displayed values appear to be neutral minus skeptical; the direction should be stated explicitly.","section":"Table 12"},{"comment":"The speculation that retrievers are 'biased toward retrieving guiding contexts over adversarial ones' is a plausible hypothesis but is not tested; it should be labeled as a hypothesis or supported by an embedding-space analysis.","section":"§5.2"},{"comment":"The reference for Wei et al. (InstrucRAG) has no year; please add the publication year.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"To the editor: the paper is honest about its scope in Section 7, so the single-attack issue is not a hidden limitation; I nonetheless consider it blocking for the general claims. The paper would benefit from a statistical pass and at least one retriever-aware attack baseline. There is no circularity in the experimental design, and the code release is a point in its favor. Fit to cs.IR is adequate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things. This is a controlled empirical study, not a new attack or defense architecture, and every headline claim — skeptical prompting helps strong models, DPR retrievers surface fewer adversarial passages, dilution doesn't work — comes from a single black-box attack recipe. The authors admit that in Section 7, but the abstract and conclusion phrase the findings more broadly.\n\nWhat's genuinely new: the three-way taxonomy of adversarial, untouched, and guiding contexts, and the interaction experiments (pollution rate, dilution, counteraction). That's a step past the coarse knowledge-conflict framing in prior work. The dilution result, in particular — adding more clean passages barely moves F1 — and the finding that guiding contexts reliably counteract adversarial ones are informative for system builders. The paper also documents prompts, model versions, datasets, and ships code, so the controlled setup is reproducible, which is real value.\n\nThe soft spots are the usual ones for this kind of empirical paper, but they're not trivial. First, there are no error bars, confidence intervals, or significance tests anywhere, and the main text never states how many queries were sampled per dataset. Several claims are quantitative comparisons, like the 10%+ F1 swings in Table 2 or the near-flat dilution curves, and without variance estimates the reader cannot tell signal from noise. This is fixable but should be required before publication.\n\nSecond, the attack is limited to PoisonedRAG's black-box generation, which optimizes for query relevance but not for the retriever. That makes the DPR robustness finding suspicious: DPR may simply be mismatched with the LLM-written passages, not genuinely more robust. A retriever-aware attacker could invert the ordering. The authors acknowledge this in Section 7 but do not resolve it, so the retriever-selection advice should be treated as preliminary.\n\nMinor: the abstract says skeptical prompting 'activates LLMs' internal reasoning,' but the experiments only measure answer accuracy. That's a stretch, though harmless.\n\nWho this is for: people working on RAG robustness and adversarial poisoning. The taxonomy and the controlled interaction data are worth citing, and the paper deserves a serious referee — the reproducible setup justifies the review time. The revision should add statistical validation and carefully re-scope the claims to the tested attack. I'd accept it for review, with the expectation of major revisions.","headline":"A carefully controlled empirical study of RAG poisoning with a useful context taxonomy and a cheap defense; the headline numbers lack statistical grounding and generalize only to the one attack recipe tested.","tokens_in":20474,"tokens_out":2093,"would_cite":true,"duration_ms":19853,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Skeptical prompting can partially defend RAG systems against poisoned passages, but only for models with strong internal reasoning.","keywords":["retrieval-augmented generation","adversarial poisoning","skeptical prompting","retriever robustness","LLM self-defense","poisoned contexts","RAG security","context taxonomy"],"falsifier":"Re-run the combined retrieval-and-generation evaluation with adversarial passages optimized using the target retriever's own scores or gradients, then compare DPR's adversarial retrieval rate to Contriever's; if the gap disappears and skeptical prompting no longer restores F1, the claimed retriever-robustness ordering and self-defense result would not generalize beyond the black-box attack.","tokens_in":19512,"feed_emoji":"🛡️","tokens_out":7746,"duration_ms":58656,"temperature":0.7,"pith_summary":"This paper asks whether retrieval-augmented generation (RAG) can be made more robust to adversarial poisoning without changing the system architecture. It claims that a 'skeptical prompt'—which explicitly authorizes the model to override misleading context with its own judgment—enables a partial self-defense, and that this helps capable models such as GPT-4, GPT-4o, Llama-70b, and Claude-3.5 while hurting weaker models such as GPT-3.5 and Llama-8b. The paper also claims that retriever choice strongly mediates attack exposure, with DPR-based retrievers surfacing adversarial passages less often than Contriever-based ones, and that increasing the number of retrieved passages does little to dilute the impact of a single adversarial passage. These findings matter because they point to low-cost, deployment-time defenses that do not require retraining.","feed_headline":"A prompt tweak helps strong LLMs shrug off poisoned RAG context","feed_subtitle":"Double-digit F1 gains for capable models, but weaker models worsen, and retriever choice matters.","key_machinery":"The load-bearing mechanism is the 'skeptical prompting' template, which adds two extra answer options—'D(A): the context is incorrect, the answer is A' and 'D(B): the context is incorrect, the answer is B'—alongside the standard multiple-choice options. This explicit authorization to distrust the context is what triggers the model to compare retrieved passages against its internal knowledge rather than blindly obeying them. The complementary machinery is the three-way context taxonomy (adversarial, untouched, and guiding) used to isolate generation-side effects, together with retrieval-rate measurements that track whether at least one injected passage lands in the top-k results. This setup lets the paper separate whether a drop in accuracy comes from retrieval exposure or from generation-side obedience.","core_discovery":"The paper claims that adversarial passages win retrieval by being explicitly tailored to the query, outranking factually correct but loosely matched corpus passages. In controlled generation experiments, a single adversarial passage sharply lowers F1 across all tested LLMs, and the damage grows as the pollution rate rises; guiding passages—query-relevant passages leading to the correct answer—consistently counteract the damage. The central result is that skeptical prompting activates the LLM's internal knowledge so that GPT-4, GPT-4o, Llama-70b, and Claude-3.5 largely resist adversarial contexts, while GPT-3.5 and Llama-8b perform worse under skepticism. On the retrieval side, the paper finds that DPR-based retrievers retrieve adversarial passages less often than Contriever-based ones, and that this retriever difference translates into an F1 gap of more than ten points in end-to-end settings. The paper concludes that retriever choice and skeptical prompting are practical, complementary defenses, but that even combined they do not bring a poisoned RAG system back to the Non-RAG baseline.","pith_inferences":["If attackers move from the black-box PoisonedRAG recipe to retriever-aware optimization, the measured DPR advantage may shrink or vanish; the paper itself lists retriever-tailored attacks as future work.","The abstention data suggest that instruction-tuning biases models toward trusting context, so an alternative to prompting would be to train selective skepticism directly into the model.","The flat dilution curves imply that defense budgets should concentrate on filtering or downranking the top few passages rather than on increasing retrieval recall.","An adaptive prompting scheme that enables skepticism only when the model's internal confidence is high could avoid the harm seen in weaker models."],"forward_implications":["RAG deployments can gain robustness immediately by switching from Contriever-style retrievers to DPR-style retrievers and adding a skeptical instruction to the prompt, with no retraining required.","Skeptical prompting is not a universal patch: for models with weaker internal reasoning, such as GPT-3.5 and Llama-8b, it can lower accuracy, so any defense must be matched to the model's capability.","Increasing the number of retrieved passages will not rescue a poisoned pipeline; the absolute number of adversarial passages in the top-k, not their proportion, drives the damage.","Even with the best retriever and skeptical prompting, a poisoned RAG system stays below the Non-RAG baseline, implying that ultimate robustness is bounded by the LLM's own knowledge.","Retriever evaluation should become a standard part of RAG security testing, since retrievers differ by tens of percentage points in how often they expose the generator to adversarial content."],"supporting_citations":[{"why":"Supplies the PoisonedRAG black-box recipe used to generate adversarial passages and the adapted guiding passages.","marker":"Zou et al., 2024"},{"why":"First proposed skeptical prompting as a defense and warned of disinformation pollution with LLMs; the paper extends this defense.","marker":"Pan et al., 2023"},{"why":"Earlier corpus poisoning by injecting adversarial passages; its shared dataset suite is reused for controlled comparison.","marker":"Zhong et al., 2023"},{"why":"Provides a gradient-based corpus poisoning baseline and the same shared experimental setup.","marker":"Su et al., 2024"},{"why":"Supplies the Natural Questions dataset and its Wikipedia knowledge base, the main experimental setting.","marker":"Kwiatkowski et al., 2019"},{"why":"Supplies HotpotQA, the multi-hop QA dataset used for cross-dataset validation.","marker":"Yang et al., 2018"},{"why":"Supplies MS-MARCO, the web-document QA dataset used as the third benchmark.","marker":"Bajaj et al., 2016"},{"why":"Defines the RAG paradigm that the paper attacks and defends.","marker":"Lewis et al., 2020"}],"fun_headline_variants":["Skeptical prompting helps strong LLMs resist poisoned RAG contexts","Retriever choice and skeptical prompts limit RAG poisoning damage","Weak LLMs worsen under skepticism, strong LLMs resist poisoning","Poisoned RAG: strong LLMs resist when prompted skeptically"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the adversarial passages generated by the black-box PoisonedRAG recipe are representative of the poisoning attacks a RAG system will face; if attackers craft retriever-aware passages, the retriever ranking and the value of skeptical prompting could change.","fun_headline_variants_meta":{"raw":{"variants":["Skeptical prompting helps strong LLMs resist poisoned RAG contexts","Retriever choice and skeptical prompts limit RAG poisoning damage","Weak LLMs worsen under skepticism, strong LLMs resist poisoning","Poisoned RAG: strong LLMs resist when prompted skeptically"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000921,"raw_usage":{"total_tokens":3964,"prompt_tokens":975,"completion_tokens":2989,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":591,"completion_tokens_details":{"reasoning_tokens":2914}},"tokens_in":591,"tokens_out":2989,"duration_ms":16851,"temperature":1.0,"reasoning_tokens":2914,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T10:20:02.971336+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the combined retrieval-and-generation evaluation with adversarial passages optimized using the target retriever's own scores or gradients, then compare DPR's adversarial retrieval rate to Contriever's; if the gap disappears and skeptical prompting no longer restores F1, the claimed retriever-robustness ordering and self-defense result would not generalize beyond the black-box attack.","supporting_citations":[],"review_version":1}