{"id":"52595e86-0041-4cd3-9639-51113449a2fd","arxiv_id":"2601.11061","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Spurious RLVR makes Qwen2.5-Math retrieve memorized answers via a layer 18-20 anchor and layer 21+ adapters, a shortcut that can be steered by scaling specific MLP keys.","lead":"Training an AI on wrong answers can make it score higher on math tests without learning math. This paper locates the inner circuit that fakes the improvement and shows how to amplify or block it.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The L18–L20 'Functional Anchor' may be an artifact of the wrong→right vs stable contrast: groups differ by construction in base-model accuracy, and the 'matched questions' protocol is unspecified, so path-patching recovery could reflect generic error correction rather than memorization.","rationale":"The reader's CONDITIONAL verdict is well-calibrated. The paper brings multiple mechanistic probes, control models, and released code, so the central claim is plausible and worth conditional acceptance. The weakest point is indeed the unverified 'matched questions' assumption: the two groups are defined by base-model correctness, and every downstream probe—path patching, JSD, NDE, ablations—inherits that difference. Even a perfect match on topic, length, and format cannot by itself remove the base-accuracy gap, because base correctness is itself a difficulty axis for the model. The paper's Appendix A partial-prompt protocol is the natural way to label memorized questions independently, and the fact that it is not used to define the mechanistic groups leaves the interpretation open. My proposed check directly tests whether the L18–L20 effect is specific to high-memorization leakage questions; if it is, the central claim would be substantially strengthened. I do not see grounds to reject: the path-patching and steering results are appropriate tools, and the confound is addressable with an additional control condition. Hence the verdict should remain CONDITIONAL, pending the matching protocol and a memorization-defined control group.","tokens_in":16300,"tokens_out":8832,"duration_ms":110418,"concrete_test":"Re-run the path-patching analysis (Section 4.3) on MATH-500/MinervaMath using groups defined by the Appendix A partial-prompt memorization score instead of wrong→right status alone. Specifically, among questions that are wrong before and right after spurious RLVR, split into 'high base-memorization' (base model completes the answer from ≥50% of the prompt) and 'low base-memorization' (base model cannot), matched on topic, length, and base-model sampling accuracy. If the L18–L20 recovery peak appears in the low-memorization group too, the Anchor is not specific to memorization; if it appears only in the high-memorization group, the central claim is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central causal claim that L18–L20 is the memorization trigger rests on the contrast between 'leakage' questions (wrong before RLVR, right after) and 'stable' questions (right both times). Section 4.1 states these are 'matched questions' but never specifies the matching procedure, and no balance table appears in the main text or appendices. The problem is not only missing detail: because the two groups are defined by base-model correctness, they differ on the model's own difficulty axis. Path Patching (§4.3) measures how much base-model accuracy is recovered when tuned-model activations are inserted; stable questions are at ceiling (base already correct), so the procedure is biased to show large effects wherever the tuned model changes processing of previously-wrong questions. The same confound propagates into the JSD analysis, the NDE bifurcation (§4.4), and the ablations (§5). Notably, the paper's own Partial Prompt Evaluation (Appendix A) provides an independent, operational label for memorization, but that label is not used to define the mechanistic groups. Thus the evidence does not yet distinguish a memorization-retrieval circuit from a generic 'fix previously-wrong answers' circuit.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies Qwen2.5-Math-7B before and after RLVR with incorrect/spurious rewards. It reports a 'Perplexity Paradox' (answer-token perplexity falls while prompt/full-text perplexity rises), and uses path patching, counterfactual JSD, logit-lens heatmaps, NDE trajectory analysis, layer ablations, and MLP-key scaling to localize the memorization shortcut. The central claim is that middle layers L18–L20 act as a 'Functional Anchor' that injects a high-probability trigger token for a memorized answer, while layers L21+ act as 'Structural Adapters' that rotate representations; scaling specific MLP keys in these layers is claimed to bidirectionally amplify or suppress contamination-driven accuracy. Comparisons against LLaMA-3.1-8B and OLMo-2-1124-7B (and Qwen3-8B) are used to argue the effect is contamination-specific.","tokens_in":16732,"tokens_out":6787,"duration_ms":72366,"significance":"Mechanistic localization of RLVR contamination effects would be valuable: it could explain why spurious rewards produce benchmark gains and suggest post-hoc mitigation. The paper has strengths: it combines several complementary probes, uses multiple control models, ships code, and its partial-prompt evaluation (Appendix A) is an independent behavioral label for memorization. However, the central evidence is currently undercut by an unspecified 'matched questions' design, a partly circular NDE analysis, and a steering-neuron selection rule that uses the ground-truth answer. With these fixed, the paper could be a useful contribution; in the current form the claims outrun the evidence.","major_comments":[{"comment":"Section 4.1 says every mechanistic probe 'is executed by contrasting the internal activations of these two groups on matched questions,' and that this 'cancels out difficulty, domain, and length confounds.' No matching procedure is described, and no balance table appears in the main text or appendices. This is load-bearing: leakage (wrong→right) and stable (always-right) groups are defined by base-model correctness, so they are not matched on the model's own difficulty axis. Path Patching (§4.3) is also biased because stable questions are at ceiling for base accuracy, so accuracy recovery can only be large for previously-wrong questions. The JSD, NDE, linear-probing, and ablation results inherit the same contrast. Please specify the matching algorithm and report covariate balance; alternatively, define groups using the Partial Prompt Evaluation label from Appendix A, which is an independ","section":"§4.1"},{"comment":"Section 4.4 models discrete residual updates with f_phi and then defines Separation Force as ||dh_leak/dt − dh_gen/dt||. Since the same fitted f_phi is a function of the observed layer-wise activations, the peak at L18–L20 is a smoothed restatement of the raw update differences, not an independent confirmation. The claim that NDE 'mathematically confirms' the causal anchor is overstated. Please show that the NDE fits are validated (e.g., prediction of held-out intermediate states), compare the separation force against raw Δx differences, and provide a null/permutation baseline to establish that the peak is not an artifact of the MLP fit. Also report training/regularization details for the two-layer MLP on 28 discrete steps.","section":"§4.4"},{"comment":"Section 5 and Figure 9 report that stable samples are robust to anchor/adapter reset. This is largely by construction: stable samples are correct in the base model, so replacing tuned weights with base weights should not destroy their accuracy. Therefore the asymmetry between leakage and stable sensitivity does not by itself show that L18–L20 are specialized memorization layers. Appendix E additionally states that on LiveMathBench the same leakage/stable partition produces an inverted pattern, 'attributed to distributional differences between the two subsets rather than memorization effects.' This admission undercuts the interpretation of the same partition on contaminated datasets. Please provide a group definition not based on base-model correctness, or at minimum demonstrate that the LiveMathBench inversion does not also apply to MATH-500/MinervaMath.","section":"§5, Appendix E"},{"comment":"Eq. (3) selects the top-k neurons using |Top-10(v_i) ∩ Tokens(y)|, where y is the ground-truth answer. Thus the 'task-relevant' neurons are chosen precisely because their output overlaps the answer token; scaling them by α will mechanically change the answer probability. Figures 13 and 14 therefore demonstrate that the selected neurons can modulate the answer, but do not independently confirm that the Anchor-Adapter circuit is the contamination mechanism. Please select neurons using leakage-vs-stable activation differences without access to y (or on a held-out answer set), and report sensitivity to k and λ.","section":"§6, Eq. (3)"}],"minor_comments":[{"comment":"Appendix A refers to Section 3.1 but the partial-prompt evaluation is described in Section 4.1; the cross-reference should be corrected.","section":"Appendix A"},{"comment":"The Logit Lens conclusion is based on two stochastic trajectories of a single sample. Please add aggregate evidence across many leakage samples, ideally with confidence intervals, because the 'weaker precursor signal' in the failed run could be decoding stochasticity.","section":"§4.3, Figure 7"},{"comment":"The choice of MATH-500 and MinervaMath as the primary contaminated datasets is made after observing the accuracy gaps. Please state prespecified inclusion criteria or report all six datasets in every mechanistic probe to reduce post hoc selection concerns.","section":"§4.1, Figure 1"},{"comment":"Several hyperparameters are free: λ in Eq. (3), k=10, and the scaling factor α. Please include sensitivity analyses and clarify whether α is applied uniformly to all selected neurons at all generation steps.","section":"§6.1–6.2"},{"comment":"Define 'full-text' vs 'answer-only' perplexity precisely, including tokenization and whether the prompt is scored in full-text PPL. A declining answer-only PPL could partly reflect shorter or more templated answers, so a length-controlled comparison would strengthen the Perplexity Paradox.","section":"§4.2, Figure 3"},{"comment":"Appendix F describes 'ablating (zeroing out)' factors at layers 22/23, but Section 6.2 defines only multiplicative scaling. Please clarify whether zeroing is α=0 and whether it is applied to selected neurons or all neurons.","section":"Appendix F"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely and important topic, and the multi-model mechanistic setup is a genuine strength. However, the load-bearing matched-question design is unspecified, the NDE 'separation force' is partly a refitting of the same data, and the neuron steering is selected using ground-truth answers. These issues are fixable: adding a matching/balance analysis or switching to the partial-prompt label, reframing the NDE results as descriptive, and de-circularizing the neuron selection would make the central claims defensible. I would not reject the paper, but the current version substantially overstates the evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nShort version: this paper is worth reading, but the central causal claim is not yet established. The Perplexity Paradox—answer-token perplexity falls while prompt perplexity rises under spurious RLVR—is a clean, new observation, and the cross-model control (LLaMA and OLMo show no such divergence) makes it convincing. The path-patching and ablation design are appropriate tools, and the steering result gives a concrete, possibly useful handle for mitigating contamination. The authors also ship code, which helps.\n\nThe soft spot is the load-bearing comparison. All the mechanistic probes contrast 'leakage' questions (wrong before RLVR, right after) with 'stable' questions (right both times). Section 4.1 says these are 'matched questions' but never specifies the matching protocol, and no balance table appears. That matters because the two groups differ by the model's own accuracy axis. Stable questions sit at ceiling, so path patching is biased to show large recovery wherever the tuned model changes processing of previously-wrong questions. The same confound runs through the JSD, NDE, and ablation analyses. The paper's own Partial Prompt Evaluation (Appendix A) provides an independent, operational label for memorization, but it is not used to define the mechanistic groups. So the evidence does not yet distinguish a memorization-retrieval circuit from a generic 'fix previously-wrong answers' circuit.\n\nThere are two further soft spots, both real but smaller. The steering neurons are selected by overlap with the ground-truth answer (Eq. 3), so scaling them and observing answer-probability changes is partly circular. And the NDE 'separation force' is computed from the same fitted dynamics that produced the trajectories it is supposed to confirm; it is illustration, not validation.\n\nIf the authors can pin down the matching procedure, run a correct-reward baseline, and use the Partial Prompt Evaluation label as the grouping variable, the Anchor-Adapter story could be solid. As it stands, it is a plausible hypothesis with strong circumstantial evidence, not a demonstration.\n\nFor peer review: this deserves a serious referee. The question is important and the Perplexity Paradox alone is publishable, but the mechanistic claims need major revision before they should be accepted. I would bring it to a reading group, mostly to discuss the design pitfalls.\n\nRecommendation: send it to review, with an explicit request for the matching protocol and sensitivity analyses.","headline":"Perplexity Paradox is new and useful, but the L18-20 'Anchor' may be an artifact of the wrong→right vs stable contrast.","tokens_in":17141,"tokens_out":2527,"would_cite":true,"duration_ms":26926,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Spurious RLVR works by flipping on a memorization circuit already latent in the model, not by improving reasoning.","keywords":["RLVR","memorization shortcut","mechanistic interpretability","Perplexity Paradox","Functional Anchor","data contamination","causal steering","neural ODE"],"falsifier":"Run path-patching and NDE separation-force analyses on a leakage/stable split where each pair shares the same problem stem and differs only in the memorized answer; if the L18–20 peak and L21–22 JSD peak disappear, the Anchor-Adapter circuit is an artifact of question confounds rather than a memorization trigger.","tokens_in":16240,"feed_emoji":"🧠","tokens_out":5187,"duration_ms":46296,"temperature":0.7,"pith_summary":"The paper argues that RLVR trained with spurious (random or incorrect) rewards improves benchmark scores by activating memorized answers the base model already contains, not by improving reasoning. It identifies a 'Perplexity Paradox' — answer-token perplexity drops while prompt-side perplexity rises — as a macroscopic fingerprint of this shortcut. Mechanistically, it localizes a Functional Anchor in the middle layers (L18–L20) that causally triggers retrieval of a memorized answer, and Structural Adapters in later layers (L21+) that reorganize representations to carry that signal. Ablations and causal steering of specific MLP keys support the circuit's necessity and sufficiency, and control models without prior contamination show no such effect. This matters because it explains why RLVR gains can be illusionary and opens a route to detect and suppress data-contamination-driven performance.","feed_headline":"Spurious RLVR activates hidden memorization, not reasoning","feed_subtitle":"Paper localizes the trigger to layers 18–20 and shows it can be amplified or suppressed by scaling MLP keys.","key_machinery":"Anchor-Adapter circuit: the division of the model's depth into a Functional Anchor (layers 18–20), where the decision to retrieve a memorized answer is causally made and a high-probability trigger token is injected, and Structural Adapters (layers 21+), which perform representational rotation rather than storing new knowledge. The detection toolbox includes path patching (swapping activations between 'leakage' and 'stable' samples to find causal edges), counterfactual JSD on MLP sub-components to locate weight-reorganization peaks, Logit Lens to trace token emergence, and a Neural ODE fit to residual updates whose 'separation force' metric pinpoints trajectory bifurcation.","core_discovery":"Spurious RLVR tunes a two-part circuit in Qwen2.5-Math-7B: a Functional Anchor at layers 18–20 causally decides to retrieve a memorized answer and injects a high-probability trigger token, while Structural Adapters at layers 21+ rotate the feature space to carry that signal. Evidence: path patching recovers accuracy only when anchor layers are patched; counterfactual JSD peaks at L21–22; Logit Lens shows Layer 19 priming and Layer 23 injection; NDE separation force peaks at L18–20; ablations resetting the anchor drop leakage accuracy >10% while stable accuracy is unchanged; scaling top MLP keys bidirectionally steers contamination-driven accuracy.","pith_inferences":["If the Anchor-Adapter circuit is a general property of pretrained models that have seen the test data, then contamination detection could be done by probing for a middle-layer trigger, complementing corpus-level string matching.","The same steering mechanism could be used adversarially to inflate benchmark scores, so releasing steering weights calls for safeguards.","The Perplexity Paradox might be exploited as an early-stopping or reward-shaping signal during RLVR to prevent memorization from dominating.","The NDE separation-force metric could be applied to any residual-stream model to locate the decision point for behaviors other than memorization."],"forward_implications":["Accuracy on a benchmark that can be gamed by this circuit is not evidence of reasoning; evaluation suites must screen for contamination.","The divergence of answer vs prompt perplexity can serve as a cheap, training-time fingerprint for shortcut formation.","Resetting or steering anchor-layer weights offers a post-hoc decontamination method that reveals baseline performance without retraining.","The circuit is dataset-specific, so layer manipulation can identify which benchmark items are leaked and which are genuinely solved.","The effect appears in Qwen-family models but not in the tested control models, suggesting architecture-dependent susceptibility to spurious RLVR."],"fun_headline_variants":["Anchor layers 18–20: spurious RLVR's memorization switch","Memorization circuit in RLVR: anchor and adapters","Spurious RLVR hinges on anchor layers 18–20","Bidirectional steering exposes RLVR memorization shortcut","Layer 19 primes, L23 injects: spurious RLVR memorization"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The entire localization rests on the claim that 'leakage' and 'stable' question sets are matched in difficulty, domain, and length; the paper states they are matched but never specifies the matching procedure, so content differences could confound the circuit attribution.","fun_headline_variants_meta":{"raw":{"variants":["Anchor layers 18–20: spurious RLVR's memorization switch","Memorization circuit in RLVR: anchor and adapters","Spurious RLVR hinges on anchor layers 18–20","Bidirectional steering exposes RLVR memorization shortcut","Layer 19 primes, L23 injects: spurious RLVR memorization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001064,"raw_usage":{"total_tokens":4306,"prompt_tokens":763,"completion_tokens":3543,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":507,"completion_tokens_details":{"reasoning_tokens":3453}},"tokens_in":507,"tokens_out":3543,"duration_ms":25686,"temperature":1.0,"reasoning_tokens":3453,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T10:06:50.277415+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run path-patching and NDE separation-force analyses on a leakage/stable split where each pair shares the same problem stem and differs only in the memorized answer; if the L18–20 peak and L21–22 JSD peak disappear, the Anchor-Adapter circuit is an artifact of question confounds rather than a memorization trigger.","supporting_citations":[],"review_version":1}