{"id":"d63fee70-d141-4684-9f4b-b1f5a9e33f4e","arxiv_id":"2507.18305","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A tunable 'overthinking' backdoor can be implanted into large reasoning models so that repeating a trigger word N times forces N extra reasoning steps, increasing token use several-fold without hurting accuracy.","lead":"The paper shows a new backdoor attack on step-by-step AI reasoning models: repeating a keyword such as 'TODO' a set number of times makes the model produce much longer reasoning chains, while still giving correct answers. This matters because it lets an attacker quietly drain computing resources from deployed reasoning services, turning the AI's careful thinking into a cost multiplier.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No verification that the teacher LLM embeds exactly S refinement steps per poisoned sample; the claimed precision of the tunable mapping from trigger repetitions to verbosity is unsupported without data-quality auditing.","rationale":"The reader's weakest assumption identifies exactly the same point: teacher reliability. I agree. The central claim of tunability depends on the poisoned corpus encoding a clean, proportional signal. Without an audit of that corpus, the empirical token-length increases could arise from a cruder mechanism (e.g., the model repeating trigger tokens or producing generic filler), which would not support 'precisely control' or the refinement-step mechanism. This is not a fatal flaw; the effect sizes are large and consistent, but it is the load-bearing assumption that should be verified before accepting the specific tunability claim. The condition I would attach is the data-quality audit described above. I therefore keep the reader's CONDITIONAL verdict unchanged.","tokens_in":13433,"tokens_out":6803,"duration_ms":69517,"concrete_test":"Obtain the poisoned training data (or regenerate it with the released code and the same DeepSeek-Chat API prompt). For each of the 100 S=1 and 100 S=2 samples, parse the <thought> block and count occurrences of refinement-step initiators ('Let's double-check', 'Alternatively', 'To be more thorough', etc.), and verify the final answer matches the original correct answer. Report the distribution of step counts and answer-mismatch rate per S. Then fine-tune the backdoored model as described and evaluate on 100 held-out prompt+trigger inputs, counting refinement steps in the model's outputs for S=1 and S=2. If the mean step count in generated outputs is not close to 1 and 2 respectively (or at least strictly increasing with S), the precise tunable mapping is not demonstrated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central novelty is a tunable backdoor where trigger strength S (number of keyword repetitions) controls reasoning verbosity through exactly S injected refinement steps (Section 4, 'Controllable Verbose CoT Generation'). The entire mapping is distilled through a teacher LLM (DeepSeek-Chat) that is instructed to embed exactly S distinct refinement steps. However, the paper reports no quantitative check of the generated poisoned data: no compliance rate, no manual annotation, no automatic parse counting refinement steps, and no verification that the final answer remains correct. Table 1 only measures average token length, a proxy that conflates refinement-step count with other sources of verbosity. If the teacher inserts a variable number of steps (e.g., S=1 samples sometimes contain 2-3 steps or S=2 samples contain 1), the training signal for an exact S-to-step mapping is corrupted, and the student model could learn a weaker monotonic 'more triggers to more tokens' heuristic rather than the precise intensity control claimed. The single case study in Section 5.3 is anecdotal. Thus the 'precisely control' and '[2,4]x / [3,5]x' quantitative claims are only as reliable as the unmeasured teacher compliance.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript introduces 'overthinking backdoors' against large reasoning models (LRMs). The trigger is a keyword repeated S times, and the poisoned training data are generated by a teacher LLM instructed to produce chain-of-thought responses containing exactly S redundant refinement steps on top of a correct reasoning path. The authors fine-tune five LRMs (DeepSeek-R1 7B/14B/32B, Marco-o1, QwQ-32B) with LoRA on a mix of clean and poisoned samples and report that S=1 and S=2 trigger repetitions increase average output token counts by roughly 2-4x and 3-5x, respectively, while accuracy is not degraded. They also report generalization to an unseen strength S=4 and resistance to prompt-based and fine-tuning-based defenses. The paper claims precise, tunable control of reasoning verbosity as a resource-exhaustion attack.","tokens_in":13675,"tokens_out":8414,"duration_ms":82263,"significance":"The core effect is large and consistent across five models and three datasets, and the held-out extrapolation to S=4 is a meaningful test of the claimed learned mapping. The attack concept is novel relative to binary backdoors and to inference-time denial-of-service attacks, and the authors release their code. However, the headline quantitative claims rest on a single, unseeded 100-sample evaluation and on an unverified teacher data-generation step; the paper currently establishes a monotonic effect more firmly than the claimed precise control.","major_comments":[{"comment":"Section 4 ('Controllable Verbose CoT Generation') instructs the teacher LLM to embed exactly S refinement steps, but the manuscript reports no compliance rate, manual annotation, or automated parse of the generated poisoned CoTs, and no check that final answers remain correct. Since the tunability claim depends on the student learning a precise S-to-step mapping, the absence of this audit is a load-bearing gap. Please report per-strength compliance statistics and correctness rates for the teacher outputs, or otherwise measure step counts.","section":"Section 4 and Section 5.1"},{"comment":"All conclusions are based on one random selection of 100 test samples per dataset, with no seed, no repeated trials, and no error bars. Token-length ratios and accuracy differences (e.g., the ±5% accuracy-preservation claim) cannot be assessed for variance; a single small sample also makes the Math-500/Cot-Flan accuracy gains difficult to interpret. Please provide multiple random seeds or confidence intervals, and specify the sampling procedure.","section":"Section 5.1 and Section 5.2"},{"comment":"The stated ranges '[2,4] times' and '[3,5] times' are not consistent with Table 1. For Cot-Flan, S=2 ratios against the clean baseline are approximately 5.4x (R1-7B), 5.4x (R1-14B), 5.6x (R1-32B), 5.6x (Marco-o1), and 5.6x (QwQ), all above 5; for QwQ-32B GSM8K the S=2 ratio is 2.99x. Please correct the reported bounds or qualify them as typical values with the outliers noted.","section":"Section 5.2"},{"comment":"The clean fine-tuning baseline is trained on 300 clean samples, while the backdoored model is trained on 300 samples of which 200 are teacher-verbose CoTs. On Math-500 and Cot-Flan the backdoored model improves accuracy substantially relative to this baseline (e.g., R1-7B Math-500 at S=2: 28 vs 64; QwQ Cot-Flan at S=0: 52 vs 86). This suggests the comparison conflates the backdoor with a general reasoning improvement from richer training data. A control model fine-tuned on equally verbose CoTs without trigger conditioning would isolate the backdoor's effect and support the stealth claim.","section":"Table 1 and Section 5.2"},{"comment":"Figures 2-5 are not legible in the version provided (the figure bodies appear as encoded glyph sequences), so the ablation, generalization, and defense results could not be verified. The final manuscript must contain readable figures with labeled axes and legends so these claims can be checked.","section":"Figures 2-5 and Sections 5.4-5.5"}],"minor_comments":[{"comment":"The sentence above Table 1, 'The value in parentheses on the second line indicates...', is confusing because the table does not have a second line; please describe the 'clean/backdoor diff↑↓' notation directly.","section":"Table 1 notation"},{"comment":"Equation (2) uses 'i ⊕ (t × S)' without first defining the product notation; please state that t × S denotes S repetitions and that ⊕ denotes string concatenation.","section":"Equation (2)"},{"comment":"The dataset name is spelled inconsistently as both 'Cot-Flan' and 'CoT-Flan'; please unify the spelling.","section":"Throughout"},{"comment":"The formal requirement '|r*| ≫ |r|' lacks a concrete threshold; consider defining an attack-success criterion, such as a minimum multiplicative factor.","section":"Section 3.2"},{"comment":"The citation keys 'QwenTeam' and 'Qwen Team' appear inconsistently for the same project; please align the author names and keys.","section":"References"},{"comment":"The case study is anecdotal; a small quantitative sample (for example, the distribution of refinement-step counts over several triggered outputs) would support the claim that the number of steps tracks S.","section":"Section 5.3"}],"recommendation":"major_revision","confidential_remarks":"The central idea is interesting and the effect is robust at a coarse level, but the 'tunable/precise' claim is the novelty and it currently rests on unverified teacher data and a single 100-sample test draw. I would encourage the editor to request the generated poisoned dataset or a compliance audit of the teacher outputs as part of the revision; without that, the paper establishes a monotonic backdoor more than a precisely controllable one. The large accuracy gains in Table 1 are also worth probing with a proper control, since they may indicate that the poisoned training set acts partly as additional reasoning data rather than purely as a trigger-conditioning signal."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper does something real: it turns a binary backdoor into a dial, where repeating a trigger word S times makes the model produce proportionally longer CoT while keeping the answer correct. I was skeptical of the framing, but the numbers are large and consistent: S=1 gives 2-4x token length, S=2 gives 3-5x, across five different LRMs. The generalization experiment (train on S=1,2,3, test on S=4) is the strongest part—that's a genuine held-out prediction, not just memorization of the training mapping, and it works. I also give credit for checking resistance to a simple prompt-based defense and a quick clean fine-tuning, and for releasing the code.\n\nThe soft spots are real but not fatal. The stress-test note is on target: nobody verified that the teacher LLM actually embeds exactly S refinement steps in the poisoned training data. The paper only reports average token length, which conflates step count with other verbosity. If the teacher is sloppy about counting, the model may be learning a monotonic 'more triggers → more tokens' heuristic rather than the precise intensity control claimed. The single 100-sample test set with no error bars or repeated seeds makes the [2,4]x/[3,5]x ranges approximate. More concerning is the accuracy jump on CoT-Flan—up to 34% for QwQ-32B—which the authors wave off as a 'test scaling law' without an explanation. That's a confound: if the backdoor sometimes improves answers, it's not purely a resource-consumption weapon. The defense eval is thin: one generic 'be concise' prompt and one round of clean fine-tuning, no adaptive defenses. Also, Figure 3 and the unparsed text block look like a rendering failure; check the source PDF.\n\nStill, the core effect is large and reproducible-looking, and the extrapolation result suggests the model really did learn a structured mapping. The paper deserves a serious referee. I'd ask the authors to audit the generated poisoned data (compliance rate, manual annotation), add seeds and error bars, and either explain the CoT-Flan gains or show they don't matter for the attack's stealth. Send it out, but expect a revision.\n\nFor a reading group, it's a 'maybe'—worth discussing for the attack design, but the evidence quality will limit what you can conclude. I wouldn't cite it in my own work this year.","headline":"A genuinely new tunable backdoor for reasoning models with a clean held-out extrapolation result, but the 'precisely controlled' claim outruns the lack of teacher-compliance verification.","tokens_in":14222,"tokens_out":2017,"would_cite":false,"duration_ms":22043,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Repeated trigger words can force AI reasoning models to produce 2-5x longer chains of thought while keeping answers correct.","keywords":["backdoor attack","large reasoning models","chain-of-thought","data poisoning","denial-of-service","overthinking","tunable trigger","reasoning verbosity"],"falsifier":"Inspect every poisoned training sample produced by the teacher model and count the refinement-step phrases (e.g., 'Let's double-check', 'To be more thorough') in each CoT. If a substantial fraction of the S=1 samples contain more than one such phrase, or the S=2 samples contain fewer than two, or any samples end with a wrong final answer, the claimed exact mapping from trigger repetitions to reasoning length is not what the model learns, and the 2-4x/3-5x length ratios would not be attributable to the intended mechanism.","tokens_in":13237,"feed_emoji":"💭","tokens_out":7803,"duration_ms":68475,"temperature":0.7,"pith_summary":"This paper tries to establish that a data-poisoning attack can plant a tunable 'overthinking backdoor' into large reasoning models: repeating a short trigger word a set number of times forces the model to produce a proportionally longer chain of thought while still giving the correct final answer. The attack works by fine-tuning on a small poisoned set in which each trigger repetition count is paired with a reasoning trace containing the same number of deliberately injected redundant refinement steps, generated by a teacher model. If the claim holds, backdoor attacks on reasoning models cease to be on/off switches and become continuously controllable dials over compute consumption, which matters because these models already spend thousands of tokens on internal reasoning and are often deployed in pay-per-token services. The paper's experiments across several reasoning models report 2-4x longer reasoning at one trigger repetition, 3-5x at two, preservation of answer accuracy, and generalization to repetition counts never seen in training.","feed_headline":"Repeated trigger words make AI models think 2-5x longer","feed_subtitle":"Data poisoning turns a backdoor into a volume dial, draining compute while the answer stays correct.","key_machinery":"The mechanism is a count-to-verbosity pairing. A repetitive trigger encodes an integer strength S by the number of times a keyword is appended to the prompt, and a teacher model generates matching chain-of-thought traces containing exactly S extra 'refinement steps' (redundant verification segments starting with phrases like 'Let's double-check' or 'To be more thorough') inserted into an otherwise correct reasoning path. Supervised fine-tuning on a mixed dataset then teaches the model a relative mapping: trigger repetition count S becomes a dial for the number of extra reasoning steps, so the length of the model's internal thought block scales with S even for values not present in training.","core_discovery":"The central discovery is a method for turning a chain-of-thought model's tendency to overthink into a precisely controllable attack surface. The trigger is a low-frequency word repeated S times, so the repetition count itself carries the attack intensity. For each strength S, an instruction to a teacher model produces a reasoning trace that starts from a correct solution and embeds exactly S redundant refinement steps, such as 'Let's double-check' or 'To be more thorough' segments, while ending with the correct answer. Training on a mixture of clean samples and these poisoned pairs teaches the model to associate zero repetitions with normal output, one repetition with one extra reasoning step, and two repetitions with two extra steps. The paper shows that this yields CoT output lengths of 2-4x the clean baseline at S=1 and 3-5x at S=2, with accuracy mostly unchanged; the behavior also resists system prompts asking for concise answers, survives a round of clean fine-tuning, and extrapolates to the unseen strength S=4.","pith_inferences":["This count-to-behavior pairing is not specific to reasoning length: the same poison data recipe could plausibly control other continuous behaviors such as number of tool calls, verbosity of explanations, or number of self-checks, so the attack class may be broader than the paper's experiments show.","The paper never verifies that the teacher model actually inserts exactly S refinement steps in every poisoned sample; if the teacher's step counts are noisy, the clean 2-4x/3-5x ratios may partly reflect distribution-level statistics rather than a per-sample exact mapping.","Because only 100 clean plus 200 poisoned samples were used, a natural next experiment would test how the mapping degrades as the poison set shrinks toward the 20-sample threshold the paper reports, and whether other fine-tuning methods preserve the behavior.","The reported accuracy gains on some setups (up to +34%) are unexpected for an attack paper; if they replicate, the 'backdoor' could also be framed as a way to selectively buy accuracy through extra reasoning, which would change how defenders weigh the threat."],"forward_implications":["An attacker who controls a released model can force any deployed service to spend 2-5x more compute on triggered queries, turning a supply-chain backdoor into a denial-of-service tool.","Because the final answers remain correct, standard accuracy-based audits will not surface the backdoor; detection requires monitoring reasoning length or compute per query.","The attack gives the adversary a continuous intensity dial, so the same poisoned model can be used for subtle latency nudges or full service exhaustion, without any retraining.","Prompt-based 'answer concisely' instructions and fine-tuning on additional clean data do not remove the triggered overthinking, so existing lightweight defenses are insufficient.","The model generalizes to trigger strengths it never saw (S=4 after training on S=1,2,3), suggesting the learned mapping is continuous rather than a lookup table, which may allow even larger amplification."],"supporting_citations":[{"why":"Introduces P-DoS, a poisoning-based denial-of-service attack on LLMs that the paper extends to reasoning models while preserving output correctness.","marker":"Gao et al., 2024"},{"why":"Presents Overthink, an inference-time slowdown attack on reasoning LLMs, which this work contrasts with as less stealthy and less persistent.","marker":"Kumar et al., 2025"},{"why":"Proposes BoT, a backdoor that breaks long thought processes of o1-like models, the direct predecessor for attacking the reasoning process itself.","marker":"Zhu et al., 2025"},{"why":"Provides the DeepSeek-R1 model family used as victim models and the DeepSeek-Chat API used as the teacher for generating poisoned CoT data.","marker":"DeepSeek-AI, 2025"},{"why":"Supplies GSM8K, one of the three benchmarks used to evaluate the backdoor's effect on accuracy and CoT length.","marker":"Cobbe et al., 2021"},{"why":"Supplies Math-500, the second mathematical reasoning benchmark on which the tunable backdoor is measured.","marker":"Lightman et al., 2023"},{"why":"Supplies the CoT-Flan multi-domain dataset used as the third benchmark, covering commonsense and logical reasoning tasks.","marker":"Wei et al., 2021"}],"fun_headline_variants":["Tunable backdoor dials AI reasoning length without altering answers","Poisoned prompts make AI overthink on command, draining compute","Attack makes AI models reason 3-5x longer with repeat triggers","Volume dial for AI overthinking: repeat trigger words to control cost"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole tunable mapping rests on the teacher language model reliably generating poisoned chain-of-thought responses that contain exactly the requested number of distinct refinement steps and still end with the correct answer; the paper reports no quality check of this generated data, so if the teacher inserts too many, too few, or incorrect steps, the learned repetition-to-verbosity mapping would be corrupted.","fun_headline_variants_meta":{"raw":{"variants":["Tunable backdoor dials AI reasoning length without altering answers","Poisoned prompts make AI overthink on command, draining compute","Attack makes AI models reason 3-5x longer with repeat triggers","Volume dial for AI overthinking: repeat trigger words to control cost"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000213,"raw_usage":{"total_tokens":1442,"prompt_tokens":986,"completion_tokens":456,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":602,"completion_tokens_details":{"reasoning_tokens":380}},"tokens_in":602,"tokens_out":456,"duration_ms":5489,"temperature":1.0,"reasoning_tokens":380,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:15:30.452059+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Inspect every poisoned training sample produced by the teacher model and count the refinement-step phrases (e.g., 'Let's double-check', 'To be more thorough') in each CoT. If a substantial fraction of the S=1 samples contain more than one such phrase, or the S=2 samples contain fewer than two, or any samples end with a wrong final answer, the claimed exact mapping from trigger repetitions to reasoning length is not what the model learns, and the 2-4x/3-5x length ratios would not be attributable to the intended mechanism.","supporting_citations":[{"cited_title":"Let's verify step by step","cited_arxiv_id":null,"evidence_quote":"Supplies Math-500, the second mathematical reasoning benchmark on which the tunable backdoor is measured."}],"review_version":2}