{"id":"7ebb2d93-c078-454a-b1e6-57fe3fe5e89e","arxiv_id":"2505.15214","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"R-TOFU shows that answer-level unlearning is insufficient for large reasoning models because residual knowledge persists in chain-of-thought traces.","lead":"This paper introduces R-TOFU, a benchmark with synthetic chain-of-thought traces for testing whether unlearning methods really remove private or copyrighted knowledge from large reasoning models. It finds that unlearning only final answers leaves the reasoning process leaking forgotten facts, and that some decoding tricks can still surface them.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central finding is sensitive to the paper's per-method early-stopping rule, which is disclosed in Appendix C.1; the paper never shows the main ranking at a fixed epoch.","rationale":"I read the paper as a first benchmark with a plausible qualitative message: LRM unlearning must consider reasoning traces, and decoding strategies can expose residual knowledge. That message is internally consistent and the step-wise CFE metrics plus Figure 3 lend support. The reader's weakest assumption (synthetic GPT-4o CoTs may not mimic real LRM reasoning) is a fair external-validity concern, explicitly acknowledged in Limitations, but it does not threaten the internal logic of the benchmark. My stress test identified a more immediate internal threat: the per-method early-stopping rule in Appendix C.1 combined with Table 3's divergent stopping epochs. This is load-bearing because the headline claim—that answer-only unlearning fails while CoT-targeted unlearning succeeds—is a comparison across methods, and the comparison is not made at matched training effort. The paper's own Table 3 shows e.g., Reasoned IDK at epoch 1-2 vs Answer-only at epoch 4-5. Without a fixed-epoch sweep or confidence intervals, we cannot distinguish 'CoT-only/Reasoned IDK are intrinsically better at erasing reasoning' from 'these methods happened to be evaluated earlier, before answer-level corruption and reasoning-trace collapse fully developed.' I therefore keep the CONDITIONAL verdict, but I shift the condition: rather than merely adding a second architecture or more statistics, the authors should demonstrate the core ranking at matched epochs and with variance estimates. If the fixed-epoch check fails, the central quantitative claims should be weakened to 'under per-method early stopping, ...' even if the qualitative direction survives.","tokens_in":17670,"tokens_out":6155,"duration_ms":49645,"concrete_test":"Re-run the forget01 (and ideally forget05/forget10) experiments for all methods at a single fixed epoch (e.g., epoch 4 or 5), with identical learning rates and batch sizes, and compute MU/AFE/CFE at that epoch. Also run 3 seeds with error bars. If Answer-only still shows low CFE and CoT-only/Reasoned IDK still show high CFE at the same epoch, the stopping rule is not the driver; if the rankings change or the CFE gap narrows substantially, the paper must reframe Findings 1-3 as stopping-rule-dependent.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline finding is that answer-only unlearning leaves residual CoT traces while CoT-only and Reasoned IDK unlearn more completely (Section 7.2). But the evaluation protocol in Appendix C.1 trains each method for up to 5 epochs and reports the last epoch where Model Utility (MU) >= 0.6, stopping early if MU drops below that threshold. Table 3 shows that the chosen epoch differs sharply by method: Reasoned IDK is evaluated at epoch 2 in forget01 and epoch 1 in forget10, while Answer-only and Answer IDK are evaluated at epochs 4-5. Since AFE, CFE, and MU evolve at different rates (see Figure 3), reporting each method at a different epoch mixes training duration with method efficacy. A method stopped early could show low AFE and high CFE not because it unlearns reasoning well but because it has been trained less; an answer-only method evaluated at epoch 5 could show high AFE and near-zero CFE partly because later epochs increasingly corrupt the reasoning trace. The paper reports no error bars or repeated runs, and never presents a fixed-epoch comparison. The central qualitative message may survive, but the specific ranking on which Findings 1-3 rely is not yet established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces R-TOFU, a benchmark for machine unlearning in large reasoning models (LRMs). It augments the TOFU fictitious-author task with synthetic chain-of-thought (CoT) traces generated by GPT-4o and conditioned on DeepSeek-R1 traces for real authors, then fine-tunes DeepSeek-R1-Distill-Llama-8B on the augmented data. The evaluation uses model utility (MU), answer-level forget efficacy (AFE), and a new step-wise CoT forget efficacy (CFE) that combines step-wise ROUGE, step-wise cosine similarity, and GPT-4o-as-judge scores. Experiments compare gradient-based (GA, GD, KL) and preference-optimization (PO) strategies over forget sets of 1%, 5%, and 10%. The central claims are that answer-only unlearning leaves residual CoT traces, that CoT-only gradient ascent gives the best utility--forget trade-off, that Reasoned IDK outperforms direct refusal in PO, and that decoding strategies such as ZeroThink and LessThink can reveal residual forgotten answers even when default decoding appears to have unlearned them.","tokens_in":17852,"tokens_out":6313,"duration_ms":50619,"significance":"If the empirical findings are robust, the paper makes a timely contribution: it provides the first benchmark explicitly targeting CoT-level unlearning in LRMs, introduces step-wise metrics that improve on full-sequence ROUGE and cosine similarity, identifies a concrete failure mode involving constrained decoding, and proposes a practical PO baseline (Reasoned IDK). The release of data, code, and full prompt templates is a strength for reproducibility. However, the paper's main quantitative comparisons rest on a per-method early-stopping protocol and single-run point estimates, so the specific rankings in Table 2 and the strength of Finding 4 are not yet fully supported. The synthetic construction of the CoT traces and the use of GPT-4o as both trace generator and judge also add an evaluation risk that needs to be addressed.","major_comments":[{"comment":"The evaluation protocol confounds training duration with method efficacy. The paper trains each method for up to five epochs and reports results at the last epoch where model utility remains at or above 0.6, and Table 3 shows that the chosen epoch differs sharply across methods: Reasoned IDK is evaluated at epoch 2 in forget01 and epoch 1 in forget10, while Answer-only and Answer IDK methods are evaluated at epochs 4 or 5. Because AFE and CFE evolve at different rates (see Figure 3), a method stopped early can show lower AFE and higher CFE simply because it received less training, and an answer-only method evaluated later can show high AFE and low CFE partly because the extra epochs corrupt the reasoning trace. The paper never reports a fixed-epoch comparison, so the rankings in Table 2 that underlie Findings 1-3 do not yet separate method efficacy from stopping point. Please provide results for all methods at matched epochs (for example, each of the five epochs) and show that the qualitative ranking is invariant to the stopping rule, or otherwise justify that the chosen epochs are fair.","section":"Appendix C.1 and Table 3"},{"comment":"All reported numbers are point estimates from a single run, with no error bars, standard deviations, or number of seeds, and the learning rate is selected per scenario based on empirical performance (1e-5 for forget01 and 2e-6 for forget05 and forget10). The central quantitative gaps, such as the CFE difference between Answer-only (0.1838) and CoT-only (0.4608) in the forget01 GA row of Table 2, or the separation between DefaultThink and ZeroThink/LessThink curves in Figure 3, could be affected by run-to-run variance and by the choice of learning rate. Please report at least three seeds with means and standard deviations (or another variance estimate), and either report results under all tested learning rates or provide a principled justification for the per-scenario selection.","section":"Section 7 and Appendix C.1"},{"comment":"The CFE metric includes a GPT-4o-as-judge score, and the benchmark's CoT traces were also generated by GPT-4o. This is a correctness-risk concern rather than an automatic disqualifier, but it means the judge may reward stylistic similarity to the synthetic traces instead of detecting actual residual knowledge. The paper does not validate the judge against human annotations or compare it with an alternative judge model. Please provide such validation, or at a minimum show that the step-wise ROUGE and cosine similarity components alone reproduce the Finding 1 ranking, and report the agreement of the judge with those automatic metrics.","section":"Section 4 and Appendix A.3"},{"comment":"Finding 4 is currently supported only by plots for the forget01 scenario, using ROUGE in Figure 3 and cosine similarity in Figure 13, with no numerical values at the reported evaluation epochs and no results for forget05 or forget10. The claim that Direct IDK appears to achieve complete forgetting by epoch 4 when judged with DefaultThink alone, yet still yields high scores under ZeroThink and LessThink, should be backed by explicit scores at the chosen epoch. Please quantify the decoding gap numerically and verify whether the failure mode persists across the larger forget sets.","section":"Section 8"}],"minor_comments":[{"comment":"There are several typographical issues: 'V oigt and V on dem Bussche' in the references has stray spaces, 'Tremblay v. OpenAI, Inc.,,' has a double comma, Figure 8 contains 'FRrench', and the prompt templates in Appendix A use 'Y ou' with an inserted space.","section":"References and Appendix figures"},{"comment":"The up arrows in Table 1 are unexplained; please state what they indicate (presumably improvement over the pretrained model) and clarify the intended comparisons.","section":"Table 1"},{"comment":"The step-wise evaluation aligns each ground-truth step with 'its most similar generated step,' but the exact matching procedure is not specified: which similarity function is used for alignment, how ties are broken, and how steps with different cardinality are handled.","section":"Section 5.2"},{"comment":"The normalization of the underlying metrics to [0,1] is mentioned but not defined; please specify the clipping or normalization procedure for ROUGE, cosine similarity, token entropy, and entailment score so that the harmonic mean aggregation is reproducible.","section":"Appendix E"}],"recommendation":"major_revision","confidential_remarks":"The skeptical concern about the early-stopping protocol is valid and is the main reason I am not recommending acceptance at this stage. The paper is otherwise a solid contribution with a useful benchmark and reproducible artifacts. The authors should be asked for fixed-epoch comparisons, variance estimates, and judge validation rather than for a full reworking of the framework."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look if you care about unlearning in reasoning models. The main contribution is R-TOFU, the first benchmark that pairs TOFU-style synthetic author knowledge with generated chain-of-thought traces, plus step-wise metrics to measure whether reasoning-level knowledge is gone. The headline finding—answer-only unlearning suppresses the final answer but leaves the reasoning trace sitting there—is real and consistently supported across their forget scales (CFE near zero for Answer-only and Answer IDK while AFE is high). The decoding-failure analysis (ZeroThink/LessThink can still expose the answer after apparent success) is also a good catch and not something prior unlearning papers addressed.\n\nThe construction choice is sensible: generate real-author CoTs with DeepSeek-R1, then use GPT-4o to rewrite style-matched traces for the fictitious authors. The limitation section honestly flags the single architecture and synthetic traces.\n\nSoft spots, in order of concern. First, the evaluation protocol in App C.1 trains up to 5 epochs and reports the last epoch where MU >= 0.6. Table 3 shows the chosen epoch differs sharply by method—Reasoned IDK is read out at epoch 2 in forget01, Answer-only at epoch 5. So method comparisons mix training duration with method efficacy. The stress-test note is right: no fixed-epoch comparison is shown, so the specific ranking that supports Findings 1-3 is not yet pinned down. The main qualitative message survives, but the performance ordering is fragile.\n\nSecond, everything is a point estimate. No seeds, no intervals. With a single model and a small benchmark, that is a real limitation for ranking claims.\n\nThird, the LLM-as-judge is GPT-4o, the same model that generated the CoTs; there is some circularity in the retention judge, though the step-wise ROUGE and cosine metrics give a non-LLM backstop. The paper acknowledges the synthetic-trace limitation but does not do a sensitivity check, e.g., using another generator.\n\nOn citations: the paper leans on TOFU, DeepSeek-R1, and standard unlearning baselines; no obvious missing prior work jumps out, and the concurrent DUSK citation is a sibling from the same group, which is fair in context.\n\nMy take: this is a solid first benchmark, worth building on, but the method rankings need replication and fixed-epoch analysis before they are actionable. It deserves a serious referee, not a desk reject. I would bring it to reading group.","headline":"Solid first benchmark for reasoning-model unlearning; the main qualitative finding holds up, but the method rankings rest on per-method early stopping and need a fixed-epoch check.","tokens_in":18424,"tokens_out":2553,"would_cite":true,"duration_ms":21986,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Erasing only a reasoning model's final answers leaves the forgotten knowledge intact inside its chain-of-thought; unlearning must target the reasoning trace itself, and evaluation must probe multiple decoding settings.","keywords":["machine unlearning","large reasoning models","chain-of-thought","R-TOFU benchmark","preference optimization","gradient ascent unlearning","step-wise evaluation","decoding strategies"],"falsifier":"Take a reasoning model fine-tuned on naturally occurring (non-synthetic) chain-of-thought traces for a forget set, apply Answer-only unlearning until final-answer ROUGE on the forget set drops to near zero, then read the model's reasoning under DefaultThink with step-wise metrics. If the step-wise retention scores are also near zero—the reasoning trace no longer reconstructs the forgotten facts—the paper's central claim that answer-level objectives leave residual CoT knowledge is falsified. A second direct check: take a model that appears fully unlearned under DefaultThink and query it with ZeroThink or LessThink prompting; if the forget answer cannot be elicited, the claimed decoding-dependent failure mode does not occur.","tokens_in":17424,"feed_emoji":"🧠","tokens_out":9788,"duration_ms":73213,"temperature":0.7,"pith_summary":"Large Reasoning Models (LRMs) do not just output answers; they produce a multi-step chain-of-thought, and private or copyrighted facts can live in those intermediate steps even when the final answer is suppressed. This paper argues that unlearning in such models therefore has to erase the reasoning trace itself, not just the final answer, and that evaluation must look inside the trace step by step. To make that case it builds R-TOFU, a benchmark that attaches realistic chain-of-thought annotations to the existing TOFU fictitious-author tasks, along with step-wise metrics and an LLM-judge score for residual knowledge. On this benchmark, answer-only objectives (gradient ascent on the answer, or replacing the answer with 'I don't know') leave substantial forget traces in the reasoning, whereas targeting the chain-of-thought—CoT-only unlearning for gradient methods, and a new 'Reasoned IDK' refusal style for preference optimization—forgets more effectively while preserving utility. The paper also shows a failure mode: decoding settings that suppress reasoning (ZeroThink, LessThink) can still reveal the supposedly forgotten content, so single-setting evaluation overestimates unlearning success.","feed_headline":"Erasing only final answers leaves knowledge in the reasoning trace","feed_subtitle":"Answer-only unlearning misses facts hidden in chain-of-thought; constrained decoding still exposes them.","key_machinery":"The central object is the chain-of-thought trace treated as a separate locus of knowledge: the paper defines an output as a triple (question, reasoning trace, answer) and makes the reasoning trace an explicit unlearning target. The machinery has three parts. First, R-TOFU, a benchmark that augments TOFU's 200 fictitious authors (4,000 question-answer pairs) with synthetic CoT traces generated by GPT-4o and styled on DeepSeek-R1 traces of the corresponding real-author questions. Second, step-wise evaluation, which aligns each ground-truth reasoning step to the most similar generated step before scoring (step-wise ROUGE, step-wise cosine similarity) and adds an LLM-as-judge score (0.0–1.0) for whether the trace retains, reconstructs, or hints at the forgotten answer. Third, the unlearning strategies themselves: gradient-based methods (GA, GD, KL) are run in CoT+Answer, Answer-only, and CoT-only variants, and preference optimization (PO) is run in Answer IDK, Direct IDK, and the proposed Reasoned IDK variant, which replaces the forget response with coherent but inconclusive reasoning.","core_discovery":"On the paper's own terms, the discovery is that knowledge in an LRM is distributed across the whole reasoning trajectory, and unlearning that touches only the final answer leaves residual, detectable knowledge inside the chain-of-thought. The paper establishes this with R-TOFU, built by rewriting TOFU's fictitious author questions for real authors, collecting DeepSeek-R1 reasoning traces on those real questions, and then having GPT-4o generate fictitious-author CoTs conditioned on the real-author reasoning style. Its step-wise metrics (step-aligned ROUGE, step-aligned cosine similarity, and an LLM-as-judge retention score) show that Answer-only and Answer IDK strategies score high on answer-level forgetting but low on reasoning-level forgetting, while CoT-only gradient ascent and Reasoned IDK achieve the best balance. A further finding is that decoding strategies ZeroThink and LessThink, which suppress or shorten the reasoning trace, can still reconstruct the forget answer even after DefaultThink evaluation looks fully unlearned.","pith_inferences":["One could design unlearning objectives that explicitly minimize worst-case leakage across decoding strategies, treating ZeroThink and LessThink as adversarial probes rather than as evaluation afterthoughts—an extension the paper does not pursue.","The step-wise alignment idea applies beyond text: the same 'align before score' logic could detect residual knowledge in latents, attention maps, or activations, where no surface chain-of-thought is generated at all.","For regulators or auditors, this implies that answer-level compliance checks are insufficient for reasoning models; auditing queries must also inspect internal reasoning traces under varied reasoning budgets.","How much the synthetic CoT distribution matters is testable: re-run the R-TOFU pipeline with human-annotated or model-native reasoning traces and compare step-wise forget efficacy; if results shift materially, benchmark conclusions would need qualifying."],"forward_implications":["Unlearning pipelines for LRMs must include the chain-of-thought in the forget objective; answer-only gradients or answer replacement leave the reasoning trace intact.","Among gradient-based methods, CoT-only unlearning gives the strongest utility-forgetting balance, suggesting that disrupting the reasoning path is sufficient and that degrading answer tokens too is unnecessary.","Preference-optimization unlearning works best when the refusal itself keeps a coherent reasoning shape (Reasoned IDK), rather than a flat 'I don't know' that removes all reasoning.","Evaluation of LRM unlearning should report reasoning-level (step-wise) forget efficacy, because full-sequence metrics misread reordered or partially overlapping traces.","Unlearning success must be checked under multiple decoding settings (DefaultThink, ZeroThink, LessThink), since constrained decoding can reveal forgotten answers that standard decoding hides."],"supporting_citations":[{"why":"TOFU dataset and the preference-optimization unlearning protocol that R-TOFU extends; supplies the fictitious author QA pairs and the neighbor-set retain definition.","marker":"(Maini et al., 2024)"},{"why":"DeepSeek-R1, the large reasoning model whose traces style-condition the R-TOFU CoTs and whose distilled Llama variant is the fine-tuning target for unlearning experiments.","marker":"(Guo et al., 2025)"},{"why":"Gradient Difference, one of the three gradient-based unlearning baselines that the paper adapts into CoT+Answer, Answer-only, and CoT-only strategies.","marker":"(Liu et al., 2022)"},{"why":"KL Minimization unlearning baseline, adapted by the paper to operate on reasoning-trajectory tokens.","marker":"(Yao et al., 2024a)"},{"why":"Gradient Ascent unlearning baseline; the paper extends the GA objective to reasoning-trajectory tokens.","marker":"(Yao et al., 2024b)"},{"why":"ZeroThink and LessThink decoding strategies; the source of the failure mode where suppressed reasoning still reveals forgotten content.","marker":"(Jiang et al., 2025)"},{"why":"Chain-of-thought prompting, the conceptual basis for treating reasoning traces as a distinct component of model output.","marker":"(Wei et al., 2022)"},{"why":"GPT-4o, used to rewrite questions, generate fictitious-author CoTs, create Reasoned IDK responses, and serve as the LLM-as-judge.","marker":"(Achiam et al., 2023)"}],"fun_headline_variants":["Chain-of-thought betrays unlearned facts","Unlearning answers won't erase reasoning secrets","Suppress reasoning and forgotten data resurfaces","R-TOFU: new benchmark reveals CoT unlearning leaks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The benchmark's conclusions rest on the assumption that GPT-4o-generated chain-of-thought traces, styled on real-author reasoning, faithfully represent how actual large reasoning models embed and reason about forget knowledge; the paper itself flags this synthetic-trace limitation, along with testing on a single model architecture (DeepSeek-R1-Distill-Llama-8B).","fun_headline_variants_meta":{"raw":{"variants":["Chain-of-thought betrays unlearned facts","Unlearning answers won't erase reasoning secrets","Suppress reasoning and forgotten data resurfaces","R-TOFU: new benchmark reveals CoT unlearning leaks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000678,"raw_usage":{"total_tokens":3082,"prompt_tokens":947,"completion_tokens":2135,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":563,"completion_tokens_details":{"reasoning_tokens":2073}},"tokens_in":563,"tokens_out":2135,"duration_ms":16173,"temperature":1.0,"reasoning_tokens":2073,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:21:38.505603+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a reasoning model fine-tuned on naturally occurring (non-synthetic) chain-of-thought traces for a forget set, apply Answer-only unlearning until final-answer ROUGE on the forget set drops to near zero, then read the model's reasoning under DefaultThink with step-wise metrics. If the step-wise retention scores are also near zero—the reasoning trace no longer reconstructs the forgotten facts—the paper's central claim that answer-level objectives leave residual CoT knowledge is falsified. A second direct check: take a model that appears fully unlearned under DefaultThink and query it with ZeroThink or LessThink prompting; if the forget answer cannot be elicited, the claimed decoding-dependent failure mode does not occur.","supporting_citations":[],"review_version":1}