{"id":"9be111bd-4295-4d56-9c42-09d49aa74a30","arxiv_id":"2505.16367","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Wrapping erroneous knowledge in a reasoning model's own chain-of-thought template raises poisoning attack success on an R1-based RAG system by 10 percentage points over the strongest prior method in a 100-query MS MARCO test.","lead":"This paper shows that fake documents written in the same chain-of-thought style used by reasoning models like R1 can trick RAG systems into adopting wrong answers. The attack lifts success rates on R1-based systems by about 10 percentage points over prior poisoning methods, but only on a small, single-dataset test.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline result may reflect teacher-student distributional alignment, not the extracted CoT template: the same DeepSeek-R1 model that generated the target's training data also writes the adversarial documents.","rationale":"The reader's Conditional verdict is appropriate. My stress-test focuses on a different load-bearing point than the reader's stated weakest assumption: the paper's headline comparison (Table 2) is confounded because the auxiliary document generator is DeepSeek-R1, the teacher of the victim Qwen-7B-R1-distilled. The extracted 'target reasoning template' is inherited from that same teacher, so the attack condition differs from the PRAG baseline in two ways at once: documents are written in a teacher-distribution style, and documents follow the target's observed template. Without a control fixing one factor, the 10-point ASR / 17-point ASR_g gain cannot be attributed to the proposed template mechanism. This is an addressable experimental omission, not an internal logical contradiction, so it does not warrant Reject; it warrants conditioning acceptance on the control experiment. The reader's hidden-CoT concern is real but secondary: even if production systems hide reasoning traces, an attacker might fall back to generic templates; the teacher-distribution confound is more decisive for the paper's scientific claim about why the attack works. The parameter-scaling results (Figure 3) also use DeepSeek-R1-distilled targets, so the same confound extends across that evidence.","tokens_in":3621,"tokens_out":2732,"duration_ms":95143,"concrete_test":"Keep the 100 queries, retriever, target model, and Figure 2 prompt fixed, but replace the auxiliary generator DeepSeek-R1 with a non-R1 model (e.g., Qwen2.5-7B-Instruct or GPT-4o) when producing the adversarial documents. If ASR on Qwen-7B-R1-distilled falls to near PRAG's 51, the headline gain is a teacher-student distributional artifact rather than evidence for the template mechanism; if ASR remains near 61, the confound is refuted and the central claim is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Central claim: wrapping erroneous knowledge in the target RAG system's own chain-of-thought template drives the ASR gain (61 vs. 51 on Qwen-7B-R1-distilled, Table 2). The experiments do not isolate that mechanism from a teacher-student confound. Adversarial documents are generated by DeepSeek-R1 (Section 5.4), which is the teacher model whose outputs were used to distill the target Qwen-7B-R1-distilled. The 'reasoning template' extracted from the victim in Section 4.2 is inherited from that same teacher, so the attack presents the student with documents written in its teacher's native style. The condition therefore varies two factors together: (1) target-template alignment and (2) teacher-distribution familiarity. The large generator-side gap (ASR_g 68.5 vs. PRAG's 51.5) may be explained by in-distribution processing of teacher-style text rather than by the target's observed template. Section 4.3 even notes the auxiliary generator should ideally be the same base model, but no control varies the generator while holding the template fixed. Without that control, Section 4.1's hypothesis (that the model mistakes documents for its own reasoning chains) is not empirically separated from 'the model prefers its teacher's text.'","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Chain-of-Thought Poisoning, a knowledge-base poisoning attack on R1-based retrieval-augmented generation (RAG) systems. The attacker first extracts a reasoning-process template by observing the victim RAG system's chain-of-thought outputs (Section 4.2), then uses an auxiliary LLM (DeepSeek-R1) to generate adversarial documents that wrap erroneous knowledge in that template (Section 4.3), and injects these documents into the knowledge base. Experiments on 100 MS MARCO queries compare the method against four baselines (NA, NPA, PHA, PRAG) on Qwen2.5-7B and Qwen-7B-R1-distilled RAG systems, reporting ASR=61 for the proposed method versus ASR=51 for PRAG on the R1-based system (Table 2), along with a naturalness study (Table 3) and a model-scaling experiment (Figure 3). The central claim is that adversarial documents mimicking the model's own chain-of-thought template are more likely to be referenced, yielding a reasoning-level attack surface beyond simple knowledge corruption.","tokens_in":10267,"tokens_out":2428,"duration_ms":19756,"significance":"If the central claim is sustained, the paper identifies a genuinely new attack surface for reasoning-capable RAG systems: poisoning at the level of the model's chain-of-thought style rather than only the factual content. This is timely and practically relevant given the deployment of R1-style models. The paper also provides a clean decomposition of attack success into retriever and generator components (Eq. 2), and it compares against four baselines including the strongest prior poisoning method (PRAG), which is a useful experimental anchor. However, the significance is conditional on isolating the proposed mechanism from a teacher-student confound and on demonstrating that the headline gains are not noise, since the reported differences are small relative to the evaluation scale (100 queries).","major_comments":[{"comment":"The main mechanism is confounded with teacher-student distributional alignment. Adversarial documents are generated by DeepSeek-R1 (Section 5.4), which is also the teacher model used to create the target Qwen-7B-R1-distilled model [7]. The reasoning template extracted in Section 4.2 is inherited from that same teacher, so the experimental condition simultaneously varies (1) alignment with the target's observed reasoning template and (2) presentation in the teacher's native style. Section 4.3 even states that the auxiliary generator should ideally be the same base model as the target, but no experiment varies the generator while holding the template fixed. The paper's Section 4.1 hypothesis ('the model may mistake adversarial documents for its own historical reasoning') is therefore not empirically separated from 'the model prefers text from its teacher distribution.' Please add a control that generates adversarial documents with a non-teacher LLM (e.g., Qwen2.5-7B or another model) using the same extracted template, and/or uses a template extracted from a different model, to isolate the effect of template alignment.","section":"Section 5.4 / Section 4.3"},{"comment":"The headline result (ASR 61 vs. 51 for PRAG on Qwen-7B-R1-distilled) is reported as a single run on 100 selected queries with no error bars, confidence intervals, or significance tests. The difference of 10 percentage points corresponds to only 10 queries, so it may lie within sampling noise. The generator-side comparison is even thinner: ASR_g 68.5 vs. 51.5, which on the ASR_r=89 subset is about 61 versus 46 successes. Please report results over multiple runs or seeds, and provide bootstrap confidence intervals or a paired significance test (e.g., McNemar's test) for the comparison against PRAG. Without this, the main ASR advantage is not statistically grounded.","section":"Table 2 / Section 6.1"},{"comment":"The naturalness claim is not supported by the reported agreement metrics. For the proposed method, Fleiss's Kappa is 0.520 (moderate agreement) and the Pearson correlation for document naturalness (PCCs) is only 0.355, which is low. The evaluation uses only 10 successfully attacked queries per method, and no confidence intervals are given for the N_r and N_d means. The text in Section 6.2 concludes that the method exhibits 'good naturalness,' but the low PCCs for the proposed method weaken this claim, especially relative to PRAG's PCCs of 0.639 and PHA's 0.843. Please report per-annotator scores, agreement statistics with confidence intervals, and a larger or more carefully selected sample, or temper the naturalness conclusion accordingly.","section":"Table 3 / Section 6.2"},{"comment":"The attack's feasibility depends on the victim RAG system exposing its full chain of thought, including internal reasoning markers such as '<think>...</think>' (Table 1). However, production RAG systems often hide the reasoning trace and return only the final answer; in that case the template-extraction step in Section 4.2 cannot be performed. The paper does not discuss or test this deployment scenario. Please add an explicit analysis of when the attack remains feasible (e.g., if only the final answer is visible, or if the reasoning is summarized) and, if possible, an experiment that simulates a hidden-reasoning setting to bound the attack's applicability.","section":"Section 3.3 / Section 4.2"}],"minor_comments":[{"comment":"The heading contains a typo: 'chain-of-thoyght' should be 'chain-of-thought.'","section":"Section 4 heading"},{"comment":"The phrase 'all values have been centile-divided' is unclear; if it means values are percentages, please state that explicitly and use consistent notation (e.g., 'ASR is reported as a percentage').","section":"Section 5.2 / Table 2"},{"comment":"Equation (1) uses d_q in the union in the second argument of G but the text defines the perturbed document as d'_t; please align the notation. Also, the indicator condition 'G(...) = y*_q' is written as a set membership symbol in the equation; please clarify.","section":"Section 3.2 / Eq. (1)"},{"comment":"The caption mentions underlining to indicate thought-process formatting, but the underlining is not visible in the rendered example. Please use a visible formatting cue (italics, color, or a distinct font) or remove the reference to underlining.","section":"Table 1"},{"comment":"The scaling plot would benefit from error bars and explicit axis labels; as shown, it is unclear whether each point is a single run and whether the differences between Ours and PoisonedRAG are within noise.","section":"Figure 3"},{"comment":"The paper states that 100 queries with definite answers were selected from MS MARCO but gives no details on the selection criteria or whether the queries are the same across all methods and systems. Please provide the query selection process or a reference to the exact subset used.","section":"Section 5.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for the journal and addresses a timely topic. The main experimental design has a confound that goes to the heart of the claimed mechanism; the authors should be encouraged to run the proposed control (varying the auxiliary generator) and to add basic statistical grounding for the headline ASR difference. If the revision addresses these points, the paper could be a solid contribution. I would also suggest the editors check whether the paper's exposure of the attack (including a ready-to-use prompt template in Figure 2) is accompanied by a clear discussion of responsible disclosure, although this is not a barrier to revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this is a plausible new RAG poisoning maneuver—wrapping wrong facts in the target system's own CoT template—and it deserves a careful look, but the headline result is not yet isolated from a teacher-student confound, and the experimental base is thin.\n\nWhat's new: prior attacks (PoisonedRAG, HijackRAG) inject facts or instructions. This paper's trick is to extract the reasoning template from the victim's own outputs and use it to craft adversarial documents, so the document reads like a historical chain-of-thought. That is a legitimate extension, and the comparison against four baselines on MS MARCO shows the maneuver gets attention: ASR 61 vs 51 over PRAG on Qwen-7B-R1-distilled, with a larger generator-side gap (ASR_g 68.5 vs 51.5). The naturalness evaluation with human annotators is a nice touch, and the parameter scaling figure adds some breadth.\n\nSoft spots. The main one: the auxiliary generator is DeepSeek-R1, which is the teacher of Qwen-7B-R1-distilled. The adversarial documents are therefore written in the dialect the student was trained on. The template extraction also comes from the victim itself. So the condition varies two things at once—template alignment and teacher-distribution familiarity. Section 4.3 says the auxiliary generator should ideally be the same base model, but no experiment does that control. Without it, the claim that the model mistakes documents for its own reasoning chains is not separated from 'the model prefers its teacher's text.' That is a fixable design gap, not a refutation.\n\nAlso, the headline gain is one run on 100 selected queries, no error bars or significance tests. Query selection is undocumented. Kappa for the naturalness of the reasoning process is moderate (0.52), and PCC for document naturalness is low (0.355), so that part is weaker. No code is released.\n\nWho this is for: people working on RAG security and adversarial robustness. It is a proof-of-concept on one dataset with two backbones; the near-term practical impact is modest, but the direction is worth tracking.\n\nRecommendation: send it to peer review. The confound can be addressed with a control experiment (e.g., generate with a different model, or hold the template fixed), and the evaluation needs variance estimates. The idea is novel enough to justify referee time.\n\nRegards.","headline":"A plausible new CoT-mimicry poisoning attack, but the headline gain is entangled with a teacher-student confound and thin statistics.","tokens_in":10832,"tokens_out":1718,"would_cite":false,"duration_ms":12861,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that wrapping erroneous knowledge in the RAG system's own chain-of-thought template lets attackers steer answers on R1-based RAG systems, raising attack success to 61 percent.","keywords":["Retrieval-augmented generation","chain-of-thought poisoning","knowledge base poisoning","adversarial attack","R1-based RAG","deep reasoning LLM","attack success rate","MS MARCO"],"falsifier":"On the same 100 queries, strip the reasoning-template phrasing from each poisoned document while keeping the factual content unchanged, and measure ASR on Qwen-7B-R1-distilled: if it does not fall from 61 toward the 51 of PRAG, then template imitation is not the causal ingredient.","tokens_in":9781,"feed_emoji":"🧠","tokens_out":6342,"duration_ms":46517,"temperature":0.7,"pith_summary":"Retrieval-augmented generation systems built on deep-reasoning language models can filter out poisoned passages that simply assert wrong facts, so earlier knowledge-base attacks lose force. This paper claims that such R1-based systems have a new vulnerability: they expose their chain of thought in answers, and an attacker can copy the recurring reasoning template and wrap fabricated documents in it. The poisoned documents are then mistaken for the model's own historical reasoning, making the system prefer the embedded wrong answer. On MS MARCO with a Qwen-7B-R1-distilled backbone, the method reaches 61 percent attack success, ten points above the strongest prior poisoning baseline, and it also beats prior methods on the underlying generator. The result matters because it locates a security risk at the reasoning-chain level, not merely the knowledge level.","feed_headline":"Fake reasoning chains lift RAG attack success to 61 percent","feed_subtitle":"By imitating the model's own chain-of-thought format, poisoned passages steer deep-reasoning RAG answers.","key_machinery":"The central mechanism is the reasoning template: the recurring chain-of-thought format (opening phrase, step-by-step document review, a transitional 'wait' pivot, a summary of evidence, closing </think> tag) that a target R1-based RAG system prints before its answer. The attack has two stages, extracting the template from observed responses and using an auxiliary LLM to wrap erroneous knowledge in that template, and those stages are what distinguish it from knowledge-only poisoning, because they aim to make adversarial documents look like the model's own past reasoning. The template is what carries the argument: without it, the injected document is just another passage asserting a wrong fact, and baseline numbers show such passages are filtered out more easily by deep-reasoning models.","core_discovery":"The paper's central claim is that deep-reasoning RAG systems, although more robust than standard RAG against knowledge corruption, are vulnerable to documents that imitate the system's own reasoning format. The attack extracts a fixed template from observed answers (begin with step-by-step review of context, introduce a 'wait, there's additional information' pivot, then summarize with 'so, putting it all together' and a </think> tag), then prompts an auxiliary LLM to generate a plausible reasoning chain that reaches the attacker's target answer. Injected into the knowledge base, such a document must first be retrieved (ASR$_r$=89) and then steer the generator (ASR$_g$=68.5), giving overall ASR=61, versus 51 for PRAG. The paper argues that the model mistakes the adversarial reasoning process for its own training-aligned reasoning and therefore prefers the embedded erroneous knowledge.","pith_inferences":["A cheap, testable defense follows directly: hide or truncate the chain of thought in production answers; since the attack's template-extraction step depends on observing full reasoning traces, systems that return only final answers should lose most of the observed advantage.","The attack likely transfers to any reasoning model whose chain-of-thought format is stable and exposed, not only R1-style models; the same template imitation could be tried against other reasoning LLMs wrapped in RAG, regardless of which auxiliary model generates the poisoned text.","Because the generator success rate (68.5) is far above the retrieval success rate (89), the approach's bottleneck is retrieval; an adversary who pairs template-wrapped documents with retriever-specific optimization techniques could plausibly push overall ASR still higher.","A direct behavioral test would be to ask the generator why it trusts a poisoned document: the paper's hypothesis predicts the model will describe the document's reasoning process as credible rather than as a merely retrieved source, a difference from knowledge-only poisoning."],"forward_implications":["Any R1-style RAG system that exposes its chain of thought in output has a reproducible attack surface: an attacker who can observe a handful of answers can reconstruct the template and poison the knowledge base.","The attack targets the generator more than the retriever: it achieves only 89 percent retrieval success (below most baselines) but the highest generator success (68.5 percent), so the reasoning format is what shifts the model's judgment.","Deep reasoning does not make RAG systems immune to poisoning; it changes the level at which poisoning acts, from factual assertion to reasoning-process imitation.","Scaling the backbone LLM reduces attack success for all methods, but chain-of-thought poisoning keeps its advantage across 1.5B, 7B, and 32B models.","Attack documents built this way remain natural to human annotators, with document fluency close to PRAG and higher reasoning credibility than prompt hijacking."],"supporting_citations":[{"why":"Provides the strongest prior poisoning baseline (PRAG) that injects erroneous-knowledge documents, which the proposed method outperforms on R1-based RAG (ASR 61 vs 51).","marker":"[33]"},{"why":"Supplies the prompt hijacking attack baseline (PHA) and shows that malicious prompts are detected by R1-based systems, motivating the template-based approach.","marker":"[31]"},{"why":"Source of R1-style reasoning behavior and the auxiliary LLM used to generate adversarial documents.","marker":"[7]"},{"why":"Provides the Co-Condenser dense retriever used in the RAG pipeline for all experiments.","marker":"[4]"},{"why":"Supplies the MS MARCO dataset from which the 100 queries and the knowledge corpus for attack evaluation are drawn.","marker":"[24]"},{"why":"Defines the retrieval-augmented generation setup and the top-5 retrieved-document setting used in the experiments.","marker":"[12]"},{"why":"Provides the Qwen2.5 backbone models and distilled variants used in the main and parameter-scaling experiments.","marker":"[29]"},{"why":"Supplies the fluency and ranking-attack evaluation methodology used for the naturalness assessment.","marker":"[27]"}],"fun_headline_variants":["CoT-style poisoned docs beat deep-reasoning RAG defenses","Adversarial reasoning chains lift RAG attack success to 61%","Poisoned passages mimic chain-of-thought to fool R1 RAG","Fake reasoning chains steer RAG answers in 61% of attacks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The attack works only if the victim RAG system displays its full chain of thought in its answers, because the attacker needs that visible reasoning trace to extract the template that makes poisoned documents persuasive.","fun_headline_variants_meta":{"raw":{"variants":["CoT-style poisoned docs beat deep-reasoning RAG defenses","Adversarial reasoning chains lift RAG attack success to 61%","Poisoned passages mimic chain-of-thought to fool R1 RAG","Fake reasoning chains steer RAG answers in 61% of attacks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000182,"raw_usage":{"total_tokens":1316,"prompt_tokens":953,"completion_tokens":363,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":569,"completion_tokens_details":{"reasoning_tokens":284}},"tokens_in":569,"tokens_out":363,"duration_ms":3284,"temperature":1.0,"reasoning_tokens":284,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:02:28.847508+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On the same 100 queries, strip the reasoning-template phrasing from each poisoned document while keeping the factual content unchanged, and measure ASR on Qwen-7B-R1-distilled: if it does not fall from 61 toward the 51 of PRAG, then template imitation is not the causal ingredient.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Co-Condenser dense retriever used in the RAG pipeline for all experiments."}],"review_version":1}