{"id":"7d20a75b-35e7-4696-b051-4f5dad788bdb","arxiv_id":"2608.04771","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"ReCo coordinates KV-cache compression, reflection-token logit penalties, and confidence-based early stopping under one per-step process reward, reducing tokens and latency while largely preserving accuracy.","lead":"This paper presents ReCo, a method that scores each reasoning step with a small reward model and uses that score to compress the KV cache, penalize reflection tokens, and stop early in large reasoning models. It reports 37% to 65% fewer generated tokens and 2.08x to 2.35x lower latency across three models and six benchmarks, with mostly preserved accuracy.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Main results are not out-of-sample: hyperparameters in Sec. 4.3 are tuned on the same test benchmarks (AIME25, AMC, MATH500) that later appear in Tables 1-2, so the headline token/latency gains may be overfit.","rationale":"The reader's weakest_assumption correctly identifies a mechanistic gap: Sec. 2.1 demonstrates that deleting tokens within high-reward steps preserves accuracy better than deleting from random steps, but ReCo's Eq. (1) uses the latest step reward to set a retention ratio for the entire accumulated cache, including prior low-reward steps, and the supporting experiment used the teacher PRM rather than the distilled Pilot. This is a legitimate concern about the internal logic of the method. However, the most load-bearing concern about the central empirical claim is the evaluation protocol. The paper transparently reports a sensitivity analysis in Sec. 4.3, but that analysis is performed on the same benchmarks that later constitute the main results. Selecting λ=0.25, δ=0.10, τ_p=1.10 because they maximize accuracy on AIME25, AMC, and MATH500 means the reported numbers for those benchmarks are not out-of-sample; they are the result of tuning to the test set. This is a well-known validity threat that can fully explain an apparent improvement, and it directly undermines the quantitative headline. The absence of error bars makes it impossible to assess whether the modest accuracy differences are noise. The reader's verdict of CONDITIONAL is appropriate, and our concern strengthens the need for held-out validation rather than changing the verdict. We therefore recommend UNCHANGED, with the condition that the authors provide a properly held-out evaluation and error bars. The Eq. (1) concern should also be addressed, ideally by comparing ReCo's latest-reward allocation policy against a uniform-retention baseline with the same attention-guided eviction and the same generation-control components, which would isolate whether the reward-modulated budget is actually responsible for the gains.","tokens_in":15276,"tokens_out":7259,"duration_ms":87405,"concrete_test":"Re-run the main evaluation with all hyperparameters (λ, δ, τ_p, β, S) selected on a held-out validation set that is disjoint from the six reported benchmarks (e.g., a random 20% of GSM8K problems not used anywhere in Sec. 4.3), and report mean ± standard deviation over at least 5 seeds on the original six test sets. If the average token reduction or latency speedup drops materially, or if the accuracy gap to Full CoT widens, the reported gains are overfit and the central claim is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is empirical: ReCo reduces generated tokens by 37%-65% and latency by 2.08x-2.35x while largely preserving accuracy across three models and six benchmarks. The validity of this claim depends on the evaluation being a fair, out-of-sample test. In Sec. 4.3, the paper sweeps the base retention ratio λ, the reward-adjustment range δ, and the early-stopping threshold τ_p on AIME25, AMC, and MATH500 using Llama-8B, and explicitly selects the configuration that performs best on those exact test sets (e.g., \"Accuracy peaks at δ=0.10\" and \"τ_p=1.10 is best on all three sets\"). Those same three benchmarks then appear in the main results tables (Tables 1 and 2). Thus the headline numbers for those benchmarks are the result of test-set selection, not independent evaluation. This is a direct threat to the central claim: the 37%-65% token reduction and 2.08x-2.35x speedup could be partially or wholly due to choosing hyperparameters that maximize performance on the reported evaluation sets. The reader's concern about Eq. (1) (the latest reward licensing whole-cache compression) is real but secondary: even if that mechanistic assumption is imperfect, the system could still work through attention-guided eviction or generation control, but if the evaluation is biased, no mechanism analysis can validate the quantitative headline. The absence of variance or confidence intervals compounds this: with only three runs and small benchmarks (AIME24/25 have 30 problems), accuracy gaps such as ReCo 33.3% vs. Full CoT 40.0% on AIME25 are within one standard error and may not be significant.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ReCo, a step-wise inference framework for large reasoning models that uses a lightweight process-reward estimator (Pilot) to coordinate three mechanisms: reward-adaptive KV-cache compression (Eq. (1)), a reward-banded logit penalty on reflection tokens (Eqs. (3)-(4)), and confidence-based early stopping (Eqs. (5)-(6)). The authors report 37%-65% token reductions and 2.08x-2.35x end-to-end latency speedups over Full CoT across three models and six benchmarks, with small accuracy losses. The paper also presents two motivation experiments: one showing reward-guided deletion preserves accuracy better than random deletion, and one showing cache compression alone increases generation length.","tokens_in":15628,"tokens_out":6880,"duration_ms":77438,"significance":"Conceptually, the paper makes a useful and falsifiable claim: process reward can serve as a unified control signal for both cache retention and generation length, and compressing the cache without controlling generation can backfire. The motivation experiments are clean, and the component ablation (Fig. 4) provides initial evidence that the three components are complementary. The overhead accounting (Sec. 4.2 footnote) and peak-memory measurement (Table 3) are valuable. However, the central quantitative claim is currently supported mainly by test-set-selected hyperparameters and single-point averages over three runs without variance; until an out-of-sample evaluation and uncertainty quantification are provided, the magnitude of the reported gains should be treated as unverified. No code or machine-checked artifacts accompany the submission.","major_comments":[{"comment":"The headline results are not out-of-sample. The sensitivity study in §4.3 sweeps λ, δ, and τ_p on AIME25, AMC, and MATH500 with Llama-8B and explicitly selects the configuration that maximizes accuracy on those exact sets (e.g., δ=0.10, τ_p=1.10). Those same three benchmarks then appear in Tables 1 and 2, where the chosen configuration is used to compute the reported 37%-65% token reduction and 2.08x-2.35x speedup. This is test-set selection, not independent evaluation; it can inflate the reported gains. The authors should tune on a disjoint development set (or use nested cross-validation) and report the main results for the selected configuration on held-out benchmarks, or show that the conclusions are stable across the full sweep.","section":"§4.3, Fig. 5, Tables 1-2"},{"comment":"Several implementation values that are load-bearing for reproducibility are missing. §4.1 specifies λ, δ, w, τ_l, τ_h, and τ_p, but not the reflection penalty strength β of Eq. (3), the cache growth interval S between compressions in §3.2, or the curated reflection-token set R of §3.3. Without these, the reflection-control and compression-scheduling components cannot be reimplemented, and the claimed overhead percentages cannot be checked. The paper should state β and S and provide the full token list (or the exact curation procedure) for R.","section":"§3.2, §3.3, §4.1"},{"comment":"The central mechanism of Eq. (1) is not directly supported by the motivation experiment. The motivation experiment in Sec. 2.1 deletes tokens within high-reward steps, whereas Eq. (1) uses the latest step's reward to set a single retention ratio for the whole accumulated cache, including earlier low-reward steps whose tokens were never tested for eviction. Moreover, the motivation experiment uses the teacher PRM (Skywork-o1-Open-PRM-7B) while the online system uses the distilled 30M Pilot (Sec. 3.1), so the transfer from teacher-based deletion to Pilot-based whole-cache compression is asserted rather than demonstrated. The authors should add an experiment that directly tests whole-cache eviction driven by the latest reward, or modify the method to score retained tokens per step.","section":"§2.1, §3.1, Eq. (1)"},{"comment":"No uncertainty quantification is reported, despite the claim that 'all reported numbers are averaged over three independent runs' (Table 1 note). With AIME24/25 containing only 30 problems and AMC being similarly small, differences of a few accuracy points between methods (e.g., 33.3% vs. 36.7% on Llama-8B AIME25, a one-problem difference) may be within noise. The paper should report per-run values, standard deviations or confidence intervals, and significance tests (or bootstrap intervals) for the accuracy and latency comparisons, especially for the comparisons to SAT and Dynasor that drive the 'best balance' claim.","section":"§4.2, Tables 1-2"},{"comment":"The fairness of the latency comparison is not established. The statement that 'SAT and Dynasor are tuned to a comparable level of acceleration' is not accompanied by the tuning procedure, the target speedup, or the resulting hyperparameters. Since the central latency claim is exactly 'ReCo reaches 2.08x-2.35x while preserving accuracy,' the baseline tuning protocol must be specified so the reader can verify the comparison is not conservative for the baselines.","section":"§4.1"}],"minor_comments":[{"comment":"In Eq. (1), the denominator (v_max - v_min) can be zero if all step rewards in a trajectory are equal, producing a division by zero; a guard or a stated convention for this case is needed.","section":"Eq. (1)"},{"comment":"The fate of a discarded probe answer is unspecified: if the probe tokens are left in the KV cache, they affect later steps and the reported latency; if they are removed, the removal mechanism should be described.","section":"§3.3"},{"comment":"The phrase 'Pilot retains its step-level discriminability' is not defined; please state a quantitative criterion, such as agreement with the teacher on step rankings.","section":"§3.1"},{"comment":"Table 3 does not state whether peak memory is the mean or the maximum over the three runs, nor the variance across runs.","section":"Table 3"},{"comment":"The overhead percentages for Pilot scoring, KV compression, reflection penalty, and early stopping are not accompanied by a measurement methodology or error bars, so it is unclear how stable these fractions are.","section":"§4.2 footnote"},{"comment":"Please report the random seeds used for the three independent runs to enable exact reproducibility.","section":"§4.1"}],"recommendation":"major_revision","confidential_remarks":"The core idea is worth pursuing, but the main quantitative claim is currently supported by test-set-selected hyperparameters and single-point averages; I would recommend acceptance only after a proper out-of-sample evaluation and the missing implementation details are supplied."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: ReCo is a sensible, well-motivated system for cutting reasoning cost, and the two motivating observations are genuinely useful. The main empirical claim is plausible but undercut by an evaluation protocol that tunes hyperparameters on the same benchmarks it reports, and by the absence of variance estimates. I'd send it to a serious referee, but the revision needs held-out validation and full implementation details.\n\nWhat's actually new: the unified reward-coordinated control of cache compression, reflection penalty, and early stopping is a real combination, and the ablation shows each piece contributes. The length-inflation observation (Sec 2.2) is the strongest part; the R-KV numbers make it clear that cache-only compression can backfire. That's a point worth citing even if the rest of the system fails.\n\nWhere it's soft: the sensitivity analysis in Sec 4.3 picks lambda, delta, and tau_p on AIME25, AMC, and MATH500, and those same three sets appear in the main tables. That makes the reported gains for those sets partly in-sample. The honest fix is to either tune on a separate validation set or explicitly report the tuning artifact on those three. The qualitative pattern does hold on the untuned sets (GSM8K, AIME24, GPQA), so this is not fatal, but it does mean the 2.08x-2.35x headline should be treated as optimistic. Second, there are no error bars. AIME25 has 30 problems; the 33.3 vs 40.0 gap on Llama-8B is within one or two correct answers. Three runs is fine, but report the variance. Third, beta, S, and the reflection token set R are never given numerically, so the system cannot be reproduced from the paper alone. That's an easy fix but a necessary one.\n\nThe Eq (1) assumption that the latest reward licenses whole-cache compression is more heuristic than proven. The Sec 2.1 experiment only deletes tokens within high-reward steps, not evicting the whole cache. Still, attention-based selection within the budget probably mitigates the risk, and the ablation shows reward-adaptive beats uniform, so I'd call that a minor concern.\n\nBottom line: this is a practical efficiency paper with a real observation and a plausible system. It deserves a serious referee. The revision should address the out-of-sample issue, add error bars, and release the missing hyperparameters or code.","headline":"ReCo is a sensible reward-coordinated efficiency system with a useful length-inflation observation, but the test-set-tuned hyperparameters and missing error bars keep the headline gains from being fully out-of-sample.","tokens_in":16233,"tokens_out":3216,"would_cite":true,"duration_ms":36559,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ReCo argues that a per-step process reward is the single signal that should coordinate KV-cache compression, reflection-token penalization, and early stopping, cutting generated tokens by 37–65% and latency by 2.08–2.35x while largely…","keywords":["KV cache compression","process reward","efficient reasoning","chain-of-thought","early stopping","reflection tokens","latency reduction","large reasoning models"],"falsifier":"Take a reasoning trajectory and, at a step where the latest reward is high, evict tokens from the earliest low-reward steps instead of preserving them; if accuracy drops substantially relative to evicting the same budget from the latest high-reward steps, the whole-cache retention rule of Eq. (1) fails. A simpler version is to shuffle the reward signal across steps while keeping all other ReCo components fixed; if the token and latency savings persist while accuracy drops, the reward is not doing the load-bearing work.","tokens_in":15066,"feed_emoji":"⚡","tokens_out":6431,"duration_ms":67431,"temperature":0.7,"pith_summary":"ReCo is a step-wise inference framework that tries to make long chain-of-thought reasoning cheaper by treating the per-step process reward as one control signal for three decisions: how much of the accumulated KV cache to keep, how strongly to penalize redundant reflection tokens, and when to stop reasoning. The central claim is that coordinating these three levers under one reward, rather than compressing the cache alone or shortening the output alone, cuts generated tokens by 37–65% and end-to-end latency by 2.08–2.35x across three reasoning models and six benchmarks, with only small accuracy loss. If true, the practical upshot is that reasoning-model inference can be made substantially cheaper without retraining the model, and that KV-cache compression and generation length cannot be optimized independently on reasoning workloads.","feed_headline":"One reward signal trims reasoning tokens by up to 65 percent","feed_subtitle":"ReCo couples KV-cache compression with generation control on reasoning models, cutting latency up to 2.35x with minimal accuracy loss.","key_machinery":"The central object is the per-step process reward $v_i$ produced by a 30M-parameter estimator distilled from a 7B teacher reward model. It plugs into three mechanisms: (1) a reward-modulated retention ratio $\\lambda_i = \\lambda - \\delta (v_i - \\bar{v})/(v_{\\max} - v_{\\min})$ that sets the budget for attention-guided token eviction from the whole accumulated cache; (2) a reward-banded logit penalty on reflection tokens, active only in the mid and high reward bands; and (3) an early-stopping probe that commits when answer perplexity $\\mathrm{PPL}(a) \\le \\tau_p$ after two consecutive top-band rewards. The identity carrying the argument is the empirical correlation between high step reward and low information loss under token deletion, which licenses using reward as the cache-retention controller.","core_discovery":"The paper claims that a reasoning state's tolerance to context loss varies along the trajectory and is tracked by process reward, and that compressing the KV cache on its own systematically lengthens subsequent generation, so the two effects must be managed jointly. ReCo operationalizes this with a lightweight 30M reward estimator that scores each completed reasoning step; that score sets the cache retention ratio (higher reward shrinks the cache harder), selects which tokens survive by attention from a recent window, applies a reward-banded logit penalty to reflection tokens, and triggers early stopping after two consecutive high-reward steps when an answer probe's perplexity is low. The paper reports that this joint scheme stays within roughly 2–3 accuracy points of full-cache CoT while cutting tokens by 37–65% and latency by 2.08–2.35x across three reasoning models and six benchmarks, and that cache-only baselines both lose more accuracy and generate more tokens.","pith_inferences":["If the reward-tolerance correlation generalizes, other cache-reduction mechanisms such as quantization, merging, or low-rank projection could be made step-adaptive in the same way, varying aggressiveness by per-step reward instead of a fixed schedule.","The length-inflation measurement, where up to 79.8% of problems generate longer outputs under R-KV compression, implies that published speedups of cache-only methods on reasoning models may be overstated unless they report end-to-end latency.","A testable extension is to replace the curated reflection-token set with per-model learned penalty targets, or to make the band thresholds $\\tau_\\ell$ and $\\tau_h$ adaptive per trajectory.","Reward and perplexity are complementary in the stopping rule; swapping the perplexity probe for a calibrated confidence model could test whether the same accuracy-length balance persists outside math and science benchmarks."],"forward_implications":["On reasoning workloads, KV-cache compression should be evaluated by end-to-end latency and generated tokens, since cache-only methods can report per-token savings while actually outputting more tokens than the full-cache baseline.","A single process-reward signal is enough to coordinate cache retention, generation penalty, and stopping, so step-wise compression need not be uniform across the trajectory.","The full ReCo configuration ($\\lambda=0.25$, $\\delta=0.10$, $\\tau_p=1.10$) transfers across three models and six datasets without per-task tuning, suggesting the mechanism is not benchmark-specific.","Removing any one of the three components degrades the accuracy-cost trade-off, so the coordinated design, not any single component, carries the reported balance."],"supporting_citations":[{"why":"Supplies the Pilot 30M process-reward estimator and the newline-step partitioning that ReCo uses for per-step control, and serves as the SAT length-control baseline.","marker":"Huang et al. 2026"},{"why":"Provides the Skywork-o1-Open-PRM-7B teacher reward model that motivates the compression-tolerance observation and distills Pilot.","marker":"He et al. 2024"},{"why":"R-KV is the reasoning-oriented KV-compression method used in the Sec. 2.2 length-inflation measurement and as a main comparison baseline.","marker":"Cai et al. 2026"},{"why":"SnapKV supplies the attention-scoring selection mechanism that ReCo adapts to choose which tokens survive within the reward-set budget, and is a cache-only baseline.","marker":"Li et al. 2024b"},{"why":"Provides the DeepSeek-R1-Distill models used as two of the three reasoning models in all experiments.","marker":"Guo et al. 2025"},{"why":"Dynasor is the early-stopping baseline that keeps the full cache, used to isolate the benefit of the joint reward-coordinated design.","marker":"Fu et al. 2025"}],"fun_headline_variants":["Reward-guided cache cuts CoT tokens by up to 65%","Process reward steers cache compression, cutting latency 2.35x","ReCo: coordinated compression and generation trims tokens 37-65%","One reward signal for cache and generation speeds LRMs 2.35x"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the reward of the latest completed step describes how safely the entire accumulated KV cache, including earlier low-reward steps, can be shrunk, even though the supporting experiment only tested deleting tokens inside high-reward steps rather than evicting whole-cache tokens from earlier steps, and used the teacher reward model rather than the distilled 30M estimator.","fun_headline_variants_meta":{"raw":{"variants":["Reward-guided cache cuts CoT tokens by up to 65%","Process reward steers cache compression, cutting latency 2.35x","ReCo: coordinated compression and generation trims tokens 37-65%","One reward signal for cache and generation speeds LRMs 2.35x"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000587,"raw_usage":{"total_tokens":2789,"prompt_tokens":1012,"completion_tokens":1777,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":628,"completion_tokens_details":{"reasoning_tokens":1696}},"tokens_in":628,"tokens_out":1777,"duration_ms":17582,"temperature":1.0,"reasoning_tokens":1696,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:59:40.978507+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a reasoning trajectory and, at a step where the latest reward is high, evict tokens from the earliest low-reward steps instead of preserving them; if accuracy drops substantially relative to evicting the same budget from the latest high-reward steps, the whole-cache retention rule of Eq. (1) fails. A simpler version is to shuffle the reward signal across steps while keeping all other ReCo components fixed; if the token and latency savings persist while accuracy drops, the reward is not doing the load-bearing work.","supporting_citations":[],"review_version":1}