{"id":"c2ad2941-e219-482b-aefc-e75bcc6e950e","arxiv_id":"2607.11089","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A lightweight optimal-stopping policy on frozen reasoning LLMs cuts CoT length 20–60% with minimal accuracy loss by trading answer correctness against token cost via a tunable λ.","lead":"OS-Pruner treats chain-of-thought pruning as an optimal stopping problem: after each reasoning step, a small policy decides whether more tokens are worth the expected accuracy gain. It cuts generation length 20–60% on math benchmarks with little accuracy loss and beats fixed-threshold early-exit baselines.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Forced boxed-answer accuracy labels (and matching inference) are the linchpin of both training rewards and the reported 20–60 % length cuts; free-generation ablations already show compensation and weaker frontiers.","rationale":"The reader correctly isolates the forced-answer premise as the weakest link that enters at data construction (§5.1) and is only partially stress-tested in Appendix A. All other elements—the optimal-stopping objective (Eqs. 2–4), the simple but correct Theorem 1, the lightweight head + last-two-layer fine-tune, and the multi-model math results—are internally consistent and support the reported numbers under the paper’s own protocol. Because the free-generation ablation already exists and already softens the frontiers, the concern is real but does not overturn the conditional acceptance; it simply confirms why the verdict should remain CONDITIONAL rather than ACCEPT. No stronger internal inconsistency or missing proof step was found.","tokens_in":23004,"tokens_out":619,"duration_ms":24257,"concrete_test":"Take the single best-AES OS-Pruner checkpoint for each of the three base models (the λ values reported in Table 1). Re-run the full evaluation suite (GSM8K, MATH-500, AIME24/25) under the free-thinking protocol of Appendix A, measuring total tokens (reasoning + answer) and Pass@1. Recompute AES with the paper’s α/β/γ. If average length reduction falls below 20 % or the accuracy drop relative to the free baseline exceeds the forced-case drop by more than 2 pp, the headline efficiency claim does not transfer to unconstrained generation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on A(y≤i|x) being a faithful, non-gaming signal of prefix sufficiency. §5.1 constructs every training reward by forcibly terminating after each paragraph, injecting a boxed-answer prefix, and grading with Math-Verify; the same forcing is used at inference for the main tables. Appendix A (and Tables 5–7) shows that when models are allowed to continue thinking in the answer section, DeepSeek-R1-Distill-Qwen-7B and DRPO-7B systematically compensate, total (CoT+Ans) length reductions shrink, accuracy–length frontiers become less controllable, and baseline accuracies rise because the model finishes the solution after the stop decision. Consequently the 20–60 % figures and the AES dominance over classification probes (Table 1, Fig. 2) are measured under an evaluation protocol that is not the natural free-generation regime. If the forced signal systematically mis-ranks prefixes relative to free utility, both the learned policy and the claimed Pareto improvement become artifacts of that protocol rather than genuine optimal-stopping gains.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper formulates chain-of-thought pruning as an optimal stopping problem. After each reasoning step the policy decides whether to stop and emit a final answer or continue, maximizing the expected reward r = A(y≤i|x) − λ L(y≤i). A lightweight linear head on the last hidden state (with only the final two transformer layers lightly fine-tuned) is trained offline from precomputed prefix rewards obtained by forcibly terminating each paragraph, boxing the answer, and grading with Math-Verify. Theorem 1 shows that any fixed-threshold correctness classifier can be arbitrarily suboptimal relative to the optimal stopping policy. Empirically, on DeepSeek-R1-Distill-Qwen-7B, GPT-OSS-20B and DRPO-7B across GSM8K, MATH-500 and AIME, OS-Pruner reports 20–60 % length reductions at low accuracy cost and lies on or near the accuracy–efficiency Pareto frontier relative to matched-architecture classification, answer-convergence and FlashThink baselines (Fig. 2, Table 1).","tokens_in":23294,"tokens_out":1199,"duration_ms":16224,"significance":"If the claims hold under realistic generation, the work supplies a clean, theoretically grounded alternative to heuristic early-exit and expensive model-side compression. The explicit λ-controlled utility, the offline reward construction that avoids on-policy rollouts, and the plug-in architecture are practical strengths. Theorem 1 is a short, correct finite-horizon construction that cleanly separates optimal stopping from fixed-threshold classification. The DRPO-7B experiment further shows that the method can still add value on top of a strong model-side compressor. These elements make the paper a useful contribution to efficient reasoning provided the evaluation protocol is shown to be faithful.","major_comments":[{"comment":"§5.1 and the main experimental protocol force a boxed final answer after every prefix both for training rewards and for the reported tables/figures. Appendix A (and Tables 5–7) demonstrates that when models are allowed free thinking in the answer section, DeepSeek-R1-Distill-Qwen-7B and DRPO-7B systematically compensate: total (CoT+Ans) length reductions shrink, frontiers become less controllable, and baseline accuracies rise. Consequently the headline 20–60 % reductions and AES dominance (Table 1, Fig. 2) are measured under a non-default generation regime. Either primary results under free generation must be elevated, or a stronger argument must be given that forced boxing is the intended deployment setting and that the free-thinking compensation does not reverse the ranking versus baselines.","section":null},{"comment":"AES (§6, Eq. 8) raises the accuracy-penalty coefficient from the literature value γ=3 to γ=7. Because best-AES operating points are used to select the reported policies in Table 1, the ranking of methods (especially on AIME and on DRPO-7B) can change under the original coefficients. A short sensitivity table with the literature (α,β,γ) would confirm that the claimed Pareto improvement is not an artifact of the altered metric.","section":null},{"comment":"Progressive λ annealing (§5.3, Fig. 1) is essential to escape the trivial “never-stop” local minimum for small λ, yet the schedule (initial value, annealing points, number of steps) is described only qualitatively. Without a precise schedule or an ablation that starts from a cold small-λ initialization, it is hard to judge how much of the reported frontier is due to the optimal-stopping objective versus the curriculum.","section":null}],"minor_comments":[{"comment":"Notation inconsistency: Eq. (3) writes R_θ(y|x) while Eq. (4) writes R_θ(x,y); unify.","section":null},{"comment":"Fig. 2 captions and axis labels would benefit from explicit units (tokens) and a note that length is measured only up to the stop decision under forced answering.","section":null},{"comment":"The claim “lightweight during both training and inference” should be qualified by the multi-day offline data-generation cost reported in §6 (4 / 2.5 / 1 day on 4 A100s).","section":null},{"comment":"Related-work coverage of training-free entropy/redundancy methods is adequate, but a one-sentence comparison of wall-clock overhead versus HALT-CoT / REFRAIN would help readers place the plug-in cost.","section":null},{"comment":"Typos: “Thisoverthinkingbehavior” (p. 1), missing spaces after periods in a few places, and “App. C” referenced before the appendix letter is introduced.","section":null}],"recommendation":"major_revision","confidential_remarks":"The forced-answer protocol is the single load-bearing empirical assumption; once the authors either move free-generation results into the main body or convincingly justify the protocol, the paper is close to a solid contribution. The theoretical construction and the plug-in design are clean. Scope is appropriate for a methods-oriented AI venue."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful core is the framing: treat CoT pruning as optimal stopping with reward A − λL, train a tiny policy head on precomputed prefix rewards from a frozen base model, and get an explicit accuracy–length knob. That is cleaner than the usual “train a correctness classifier and threshold it” pattern, and Theorem 1 is a short, correct construction showing fixed-threshold classification can lose arbitrarily large value. The implementation is practical—last two layers + linear head, step-boundary decisions, progressive λ annealing—and the experiments are more thorough than most early-exit papers: three models (including DRPO-7B), three difficulty tiers, matched architecture for the classification baselines, full λ/γ sweeps, and AES-selected operating points.\n\nWhat it does well is show that the optimal-stopping policy sits on or near the Pareto front in the forced-answer regime and still adds value on top of a strong model-side compressor. The math is elementary but honest; the citation pattern covers the right early-stop, budget, and length-compression lines without obvious padding.\n\nThe soft spot that matters is exactly the one the stress-test flags. Training rewards and main tables force a boxed final answer after every prefix and grade with Math-Verify. Appendix A and Tables 5–7 show that when DeepSeek and DRPO are allowed to keep thinking in the answer section they compensate, total length reductions shrink, and frontiers get messier. GPT-OSS is better behaved. So the 20–60% numbers and the clean dominance over probes are protocol-dependent; they are not pure free-generation gains. AES with a custom γ=7 and no uncertainty bands are secondary issues. Scope is math-only.\n\nThis is for people building efficient-reasoning systems who want a plug-in stop policy rather than another full retrain. It deserves a serious referee. I would engage with it, cite the formulation and the theorem, and treat the free-generation ablation as the real stress test rather than a footnote.","headline":"Clean optimal-stopping framing for CoT early exit with a real theorem and solid multi-model Pareto curves, but the headline 20–60% cuts live under forced boxed-answer evaluation that free-generation ablations already weaken.","tokens_in":23926,"tokens_out":519,"would_cite":true,"duration_ms":5080,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"OS-Pruner frames chain-of-thought pruning as optimal stopping and cuts generation length by 20–60% with little accuracy loss.","keywords":["chain-of-thought","optimal stopping","early stopping","reasoning efficiency","length reduction","plug-in policy","large language models"],"falsifier":"On the same held-out math sets, run free (unforced) final-answer decoding: if OS-Pruner’s length reductions vanish or accuracy falls well below full traces and fixed-threshold baselines, the optimal-stopping claim fails under realistic decoding.","tokens_in":23901,"feed_emoji":"✂️","tokens_out":806,"duration_ms":19510,"temperature":0.7,"pith_summary":"Reasoning models often keep writing after the answer is already determined, burning tokens without raising accuracy. This paper treats the decision to stop as an optimal stopping problem: after each reasoning step, continue only if the expected accuracy gain is worth the extra length cost. A small plug-in policy is trained on precomputed rewards of the form accuracy-minus-λ-times-tokens, leaving the base model frozen. Across math benchmarks and several reasoning models, the method sits on or near the accuracy–efficiency frontier and beats fixed-threshold early-exit classifiers, while λ gives users direct control of the trade-off.","feed_headline":"Stop reasoning when tokens stop buying accuracy","feed_subtitle":"A lightweight plug-in decides after each step whether more thinking is worth the cost.","key_machinery":"The stopping utility r = A(prefix) − λ L(prefix) and the Bellman comparison of immediate accuracy value versus continuation value; a linear head on the last hidden state (with light fine-tuning of the last two layers) learns the stop probability at paragraph boundaries.","core_discovery":"Chain-of-thought pruning is best cast as optimal stopping with reward equal to final-answer accuracy of a prefix minus a length penalty λ. A lightweight policy head trained on that objective, without retraining the base reasoner, yields stopping rules that dominate classification-style early exit on the accuracy–length frontier and deliver roughly 20–60% shorter generations at low accuracy cost.","pith_inferences":["The same prefix-level reward pipeline could transfer to coding or tool-use traces whenever an intermediate correctness oracle exists.","Progressive annealing of λ is a practical fix for “always continue” local minima and may apply to other sparse sequential decisions.","On instruction-weak models, adding training-free entropy or redundancy signals as policy features could stabilize frontiers when forced-answer labels are noisy.","Serving systems that already prefill whole paragraphs can host the two-layer policy head with negligible extra latency."],"forward_implications":["Sweeping λ traces a controllable accuracy–efficiency frontier without changing the frozen base model.","Even models already shortened by model-side compression still overthink and can be further pruned by the same plug-in.","Fixed-threshold correctness classifiers can lose arbitrarily large value relative to optimal stopping (Theorem 1), so early-exit training should compare continuation value, not only a confidence cutoff.","Easier and medium problems admit large token savings while pass@1 stays nearly intact; hard olympiad problems stay more conservative.","Inference cost and latency of reasoning models can drop substantially when the policy is invoked only at paragraph boundaries."],"fun_headline_variants":["Prune CoT via optimal stopping on accuracy vs length cost","Light head stops reasoners when extra tokens stop paying off","Dynamic early exit: utility of answer accuracy minus λ length","Optimal stopping prunes 20-60% of reasoning with little accuracy loss","Plug-in decides after each step if more thinking is still worth it"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"Forcing a boxed final answer after every intermediate reasoning prefix must give a faithful accuracy signal that the model does not systematically undo by continuing to think in the answer section.","fun_headline_variants_meta":{"raw":{"variants":["Prune CoT via optimal stopping on accuracy vs length cost","Light head stops reasoners when extra tokens stop paying off","Dynamic early exit: utility of answer accuracy minus λ length","Optimal stopping prunes 20-60% of reasoning with little accuracy loss","Plug-in decides after each step if more thinking is still worth it"]},"model":"grok-4.5","effort":"low","cost_usd":0.00279,"raw_usage":{"total_tokens":1037,"prompt_tokens":762,"num_sources_used":0,"completion_tokens":72,"cost_in_usd_ticks":27900000,"prompt_tokens_details":{"text_tokens":762,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":203,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":762,"tokens_out":72,"duration_ms":2700,"temperature":1.0,"reasoning_tokens":203,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-14T07:07:54.680921+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On the same held-out math sets, run free (unforced) final-answer decoding: if OS-Pruner’s length reductions vanish or accuracy falls well below full traces and fixed-threshold baselines, the optimal-stopping claim fails under realistic decoding.","supporting_citations":[],"review_version":1}