Pith. sign in

REVIEW 2 major objections 5 minor 26 references

Learning when to stop reasoning only pays when answers oscillate and evidence is spread; on multiple-choice tasks a simple confidence threshold wins, and hard small sets admit no safe early exit.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-07-12 10:29 UTC pith:O2K7OBCS

load-bearing objection Solid systems paper that finally compares early-exit rules at matched lost-correct risk and shows when multi-feature learning actually pays. the 2 major comments →

arxiv 2606.30852 v2 pith:O2K7OBCS submitted 2026-06-29 cs.AI cs.CLcs.LG

When Does Learning to Stop Help? A Cost-Aware Study of Early Exits in Reasoning Models

Zhe Dong (University of Maine at Presque Isle) , Fang Qin (Stanford University) , Manish Shah (Independent Researcher) This is my paper
classification cs.AI cs.CLcs.LG
keywords early exitreasoning modelstest-time computelost-correct riskLearnStoptrajectory decompositionKV-cache forkingmatched-risk calibration
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Reasoning models waste test-time compute on easy questions and still need long chains on hard ones. Many early-exit rules exist—confidence, entropy, answer stability, and learned stoppers—but they are usually compared under mismatched risk and without charging for the probes that read intermediate answers. This paper runs a controlled 18-setting study that forces every policy, including a simple logistic stopper called LearnStop over eight prefix-observable features, to the same lost-correct risk budget before asking which one saves more tokens. The result is a three-regime map: multi-feature learning wins on free-form math where answers flip and signals complement each other; a calibrated scalar threshold wins on multiple-choice work where most items are already solved at the first checkpoint; and small hard benchmarks admit no certifiable aggressive policy at all. A five-way trajectory decomposition predicts which regime a workload will fall into, and cost accounting shows that the same policy that saves tens of percent under KV-cache forking can cost more than it saves under black-box repeated prefilling. The practical upshot is a decision procedure: measure the trajectory profile, pick the simplest matching rule, calibrate it to an explicit risk target, and verify that the serving stack reuses prefixes.

Core claim

Under matched lost-correct risk at α=0.15, LearnStop certifies 3.2 to 21.2 percentage points more total-token saving than the best calibrated scalar exit on the primary free-form math settings, loses on multiple-choice MMLU-Pro, and neither policy can be certified as aggressive on AIME-90 or GPQA. The regime is predicted by a five-way trajectory decomposition of early-solved, beneficial-thinking, harmful-overthinking, unsolved, and oscillating questions, and the certified savings reverse under black-box prefilling.

What carries the argument

LearnStop: a hidden-state-free logistic classifier over eight prefix-observable features (budget fraction, checkpoint index, answer log-probability, entropy, previous-match, run length, prefix vote share, backtracking density) that stops when the predicted probability that the forced prefix answer is correct exceeds a risk-calibrated threshold τ.

Load-bearing premise

That charging every probe at a fixed 48-token cap (actual lengths were never logged) and using output-level proxies for other published exits under one shared checkpoint schedule is faithful enough that the three-regime ranking would survive measured probe costs and full end-to-end reimplementations.

What would settle it

Re-run the matched-risk comparison on the same four Qwen3 free-form math settings with measured rather than capped probe lengths and with true end-to-end reimplementations of the competing systems; if the LearnStop advantage over BestScalar disappears or reverses on any primary math setting at α=0.15, the central claim fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper studies when multi-feature learned early-exit policies beat calibrated scalar exits for reasoning models under a fixed lost-correct risk budget and under explicit probe-cost regimes. Using LearnStop-8 (logistic regression over eight prefix-observable features) as the learned instrument, it compares against confidence, entropy, confidence-leap, and run-stability exits (plus output-level proxies for DEER/EAT/PUMA/TERMINATOR) across 18 task–model settings on Qwen3 and DeepSeek-R1 distillations. Under matched risk at α=0.15 with finite-grid Hoeffding UCB calibration (Eq. 6) and BestScalar selected on calibration data alone, it reports three regimes: LearnStop certifies +3.2 to +21.2 pp extra total-token savings on primary Qwen3 free-form math (Table 3); calibrated scalars win on MMLU-Pro; and AIME-90/GPQA admit no certifiable aggressive policy. A five-way trajectory decomposition (Table 2) is offered as a predictive mechanism, and KV-fork vs prefix-cache vs black-box cost accounting shows that the same policy can reverse from savings to large overhead.

Significance. If the matched-risk map and trajectory predictor hold, the paper replaces a method-ranking race with a deployable decision procedure: measure trajectory structure, pick the simplest matching signal, calibrate to an explicit lost-correct risk, and cost under the actual serving stack. Strengths include a clean risk-matched protocol (identical splits, finite-grid correction, grouped OOF scores, paired bootstrap CIs), α-sweeps, feature/classifier ablations, transfer and temperature checks, and explicit three-regime cost accounting with code/data release. The overhead result—that KV-fork savings can become large black-box costs—is practically important and under-emphasized in prior early-exit work. The contribution is empirical and systems-facing rather than a new theoretical guarantee, but the protocol and map are useful for the field.

major comments (2)
  1. Limitations §5 and Eq. (2): actual probe lengths were not logged, so all total-token savings use a fixed 48-token cap per probe. The paper correctly notes this overcharges short math answers and is therefore conservative for the math wins, but the MMLU-Pro scalar advantage (−2.7 to −3.9 pp in Table 3) could shrink or reverse under measured lengths. A short measured-length audit (or a sensitivity table with A ∈ {8,16,32,48}) on the primary six Qwen3 settings would make the regime boundary load-bearing rather than accounting-dependent.
  2. Appendix G.3 and Related Work: comparisons to DEER, EAT, PUMA, and TERMINATOR are output-level proxies under a shared checkpoint schedule, not end-to-end reimplementations. The paper scopes this honestly, and the three-regime contrast is driven mainly by trajectory structure (Table 2) and finite-sample certification (Eq. 6) rather than any single proxy. Still, the claim that the study answers “which policy saves more” for the broader family would be stronger if at least one original system (e.g., DEER or TERMINATOR) were run end-to-end on GSM8K/MATH-500 under the same α=0.15 protocol, or if the proxy gap were bounded more tightly.
minor comments (5)
  1. Figure 2 caption and right-column labels: “inc.” and the asterisk convention are clear in the text but dense in the figure; a one-line legend for “inc. = no certifiable aggressive policy” would help.
  2. §3.2 / Table 4: backtracking-marker density uses a fixed English marker list; a brief note on whether this list was tuned on calibration data or held fixed a priori would clarify leakage risk.
  3. Table 1 vs Table 3: the paper already explains that frontier rankings and risk-controlled rankings can disagree (GSM8K-8B); elevating that sentence earlier in §4.2 would prevent readers from treating Table 1 as the deployment answer.
  4. Appendix K cost table: black-box numbers for AIME are large negatives; stating the exact prefilling formula (full prefix re-send each probe) once in the main §4.5 would make the 121% extra claim easier to audit without the appendix.
  5. Minor notation: Li(τ) in Eq. (3) is an indicator; later text sometimes reads as if it were already an expectation—consistent E[Li] wording would avoid confusion.

Circularity Check

0 steps flagged

No significant circularity: empirical bake-off on held-out external benchmarks with independent risk targets and gold-defined trajectory types.

full rationale

This paper is a controlled empirical comparison of early-exit policies, not a first-principles derivation. LearnStop is a logistic classifier trained on prefix-observable features with gold checkpoint labels; BestScalar is selected on the same calibration split from a fixed pool of scalar exits; both are then evaluated on a disjoint test split under a finite-grid lost-correct UCB (Eq. 6) at a pre-specified α. Reported total-token savings and regime rankings are measured outcomes on GSM8K, MATH-500, MMLU-Pro, AIME-90, and GPQA, not quantities forced by construction from fitted constants. The five-way trajectory decomposition (early-solved / beneficial / harmful / unsolved / oscillating) is defined from gold checkpoint correctness sequences, independent of LearnStop’s predicted probabilities, and is used only as a post-hoc explanatory correlate of which policy wins—not as an input that algebraically produces the savings. Cost regimes (KV-fork, prefix-cache, black-box) are alternative accounting conventions applied to the same stopping decisions; they do not redefine correctness or risk. There is no self-definitional loop, no fitted parameter renamed as a prediction of a closely related target, no load-bearing uniqueness theorem imported from overlapping authors, and no ansatz smuggled via self-citation. Related-work citations are to external methods (DEER, EAT, PUMA, TERMINATOR, conformal risk control) used as baselines or protocol motivation. The study is self-contained against external benchmarks with held-out evaluation; circularity score is therefore zero.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 2 invented entities

The central claim is empirical, not axiomatic. Load-bearing free choices are the risk target, probe cap, budget grids, and the logistic feature set used as the learned instrument. Domain assumptions are standard supervised calibration with gold answers and the ability to force short answer probes from a reasoning prefix. Invented constructs are the LearnStop-8 instrument and the five trajectory types used as a predictive mechanism; both are operational definitions with direct measurement procedures, not unobservable entities.

free parameters (5)
  • lost-correct risk target α = 0.15 (primary)
    Primary operating point α=0.15 (also swept {0.05,0.10,0.20}); the certified savings and regime rankings are reported at this hand-chosen risk budget.
  • probe answer cap A = 48 tokens
    Every probe is charged A=48 tokens under capped accounting because raw probe lengths were not logged; this number directly enters total-token savings.
  • budget checkpoint grid = main 10-point grid; 6-linear default recommended
    Main grid [0,128,...,1536] (AIME longer to 6144) and schedule variants (4/6/8/14 checkpoints) determine when features are observed and how much probe overhead is paid.
  • logistic regularization C and feature set = C=1.0; 8 deployment features
    LearnStop-8 uses logistic regression (C=1.0, lbfgs) over eight hand-chosen prefix features; ablations show the combination matters, so the feature list is a free design choice of the instrument.
  • calibration split fraction and δ = 40/60; δ=0.05
    40/60 cal/test split (seed 123) and δ=0.05 enter the finite-grid UCB that selects feasible thresholds; small n_cal on AIME/GPQA forces the no-policy regime.
axioms (5)
  • standard math Lost-correct risk L_i(τ)=1{F_i=1,S_i(τ)=0} upper-bounds accuracy drop relative to full thinking (Eqs. 3–5).
    Used to justify calibrating on lost-correct risk rather than accuracy directly; standard inequality, invoked in §3.4.
  • standard math Finite-grid Hoeffding UCB U(c)=R̂_cal(c)+√(log(K/δ)/(2n)) certifies risk for the selected policy–threshold pair at level 1−δ.
    Concentration tool for simultaneous selection over K candidates; §3.4 Eq. (6).
  • domain assumption Gold answers are available for labeling probe correctness on training/calibration questions, and forced short answers after a stop-thinking marker are valid early-exit candidates.
    Required to train LearnStop and to define trajectory types; standard for supervised early-exit work but not free at pure test time without a labeled cal set.
  • domain assumption KV-cache forking (or prefix reuse) is a realistic serving regime in which probe cost equals only decoded answer tokens.
    Primary savings numbers assume this; black-box regime is shown separately to reverse savings (§4.5).
  • ad hoc to paper Output-level proxies under a common checkpoint protocol are informative enough to rank DEER/EAT/PUMA/TERMINATOR-style signals against LearnStop.
    Explicitly not end-to-end reproductions; used in Appendix G.3 and Limitations.
invented entities (2)
  • LearnStop-8 no independent evidence
    purpose: Hidden-state-free logistic stopper over eight prefix-observable features, used as the learned-policy instrument for all matched-risk comparisons.
    Deliberately simple classifier so conclusions are attributed to multi-feature information content rather than architecture; no independent existence outside this study.
  • Five-way trajectory decomposition (early-solved, beneficial thinking, harmful overthinking, unsolved, oscillating) independent evidence
    purpose: Operational taxonomy of checkpoint-correctness sequences that is claimed to predict which stopping regime a workload falls into.
    Defined from gold labels at checkpoints; measurable on the same probes used for calibration, but the five labels are a paper-specific partition.

reviewed 2026-07-12 · how reviews work

0 comments
Cite this review

Pith. "Pith review of When Does Learning to Stop Help? A Cost-Aware Study of Early Exits in Reasoning Models." pith.science (2026). https://pith.science/paper/O2K7OBCS

@misc{pith2026260630852,
  author       = {Pith},
  title        = {Pith review of: When Does Learning to Stop Help? A Cost-Aware Study of Early Exits in Reasoning Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/O2K7OBCS}},
  note         = {Machine review of arXiv:2606.30852}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Reasoning models spend test-time compute unevenly across instances, and a growing family of early-exit rules -- confidence thresholds, entropy monitors, answer-stability checks, and learned stoppers -- promises to reclaim the waste. These rules, however, are evaluated under heterogeneous protocols that leave the deployment question unanswered: at a fixed tolerance for losing correct answers, which policy saves more compute, and does the saving survive probe overhead? We answer this question with a controlled study across 18 task-model settings spanning GSM8K, MATH-500, MMLU-Pro, AIME-90, and GPQA on Qwen3 and DeepSeek-R1-distilled models, using LearnStop, a hidden-state-free logistic stopper over prefix-observable features, as the learned policy instrument. Under matched lost-correct risk at $\alpha$ = 0.15, with the scalar competitor selected on calibration data from confidence, entropy, confidence-leap, and run-stability exits, the answer forms three regimes. Learned stopping wins on all four primary Qwen3 free-form math settings (+3.2 to +21.2 pp additional total-token saving); calibrated scalar exits win on multiple-choice MMLU-Pro; and small hard benchmarks (AIME-90, GPQA) admit no certifiable aggressive policy at all. A trajectory decomposition predicts the regime: learning pays where answers oscillate and correctness evidence is spread across complementary signals, while a single confidence threshold suffices where most instances are already solved at the first checkpoint. Cost accounting sharpens the picture further -- the same policy that saves 32% of tokens under KV-cache forking costs 121% extra under black-box repeated prefilling. Together, these results replace the single-method race with a decision procedure for choosing a stopping rule from the trajectory structure and serving regime of the target workload.

Figures

Figures reproduced from arXiv: 2606.30852 by Fang Qin (Stanford University), Manish Shah (Independent Researcher), Zhe Dong (University of Maine at Presque Isle).

Figure 1
Figure 1. Figure 1: Accuracy–cost frontier on GSM8K with Qwen3-32B. The learned stopper improves over [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Why the result is task-dependent. Left: paired difference between learned stopping and [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 2
Figure 2. Figure 2: The 18-setting map. Left: post-hoc frontier difference (pp) between LearnStop-8 and [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Deployment sensitivity. Left: checkpoint schedule sweep on GSM8K-32B. Six linear [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 3
Figure 3. Figure 3: Matched-risk saving difference (LearnStop-8 [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Risk-control summary for Qwen3-32B. Left: savings at [PITH_FULL_IMAGE:figures/full_fig_p013_4.png] view at source ↗
Figure 4
Figure 4. Figure 4: Deployment sensitivity on GSM8K-32B. Six linear checkpoints retain 96% of the ten [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Transfer protocols across selected source-target pairs. [PITH_FULL_IMAGE:figures/full_fig_p014_5.png] view at source ↗
Figure 5
Figure 5. Figure 5: Transfer protocols across selected source-target pairs. [PITH_FULL_IMAGE:figures/full_fig_p021_5.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

26 extracted references · 7 linked inside Pith

  1. [1]

    AI-MO/aimo-validation-aime: AIME validation problems

    AI-MO. AI-MO/aimo-validation-aime: AIME validation problems. Hugging Face dataset, 2024. URLhttps://huggingface. co/datasets/AI-MO/aimo-validation-aime

    2024

  2. [2]

    Angelopoulos, Stephen Bates, Adam Fisch, Lihua Lei, and Tal Schuster

    Anastasios N. Angelopoulos, Stephen Bates, Adam Fisch, Lihua Lei, and Tal Schuster. Conformal risk control. In International Conference on Learning Representations, 2024. URLhttps://openreview.net/forum?id=33XGfHLtZg

    2024

  3. [3]

    Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021. URLhttps://arxiv.org/abs/2110.14168

    Pith/arXiv arXiv 2021

  4. [4]

    DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning.Nature, 645:633–638,

    DeepSeek-AI. DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning.Nature, 645:633–638,

  5. [5]

    URLhttps://doi.org/10.1038/s41586-025-09422-z

    doi: 10.1038/s41586-025-09422-z. URLhttps://doi.org/10.1038/s41586-025-09422-z. arXiv:2501.12948

    Pith/arXiv arXiv doi:10.1038/s41586-025-09422-z

  6. [6]

    Rosing, Ion Stoica, and Hao Zhang

    Yichao Fu, Junda Chen, Siqi Zhu, Zheyu Fu, Zhongdongming Dai, Yonghao Zhuang, Yian Ma, Aurick Qiao, Ta- jana S. Rosing, Ion Stoica, and Hao Zhang. Efficiently scaling LLM reasoning programs with Certaindex. InAd- vances in Neural Information Processing Systems, 2025. URLhttps://papers.nips.cc/paper_files/paper/2025/hash/ d037fd021c9aace128b8ce25001cdb6c-A...

    2025

  7. [7]

    Token-budget-aware LLM reasoning

    Tingxu Han, Zhenting Wang, Chunrong Fang, Shiyu Zhao, Shiqing Ma, and Zhenyu Chen. Token-budget-aware LLM reasoning. InFindings of the Association for Computational Linguistics: ACL 2025, pages 24842–24855, 2025. doi: 10.18653/v1/2025.findings-acl.1274. URLhttps://aclanthology.org/2025.findings-acl.1274/

    doi:10.18653/v1/2025.findings-acl.1274 2025

  8. [8]

    Measuring mathematical problem solving with the MATH dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. InProceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks (NeurIPS Datasets and Benchmarks), volume 1, 2021. URLhttps://datasets- benchmarks-pro...

    2021

  9. [9]

    Let’s verify step by step

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. InInternational Conference on Learning Representations, 2024. URLhttps://openreview.net/forum?id=OMD3OAojCe

    2024

  10. [10]

    MAA invitational competitions: American invitational mathematics examination,

    Mathematical Association of America. MAA invitational competitions: American invitational mathematics examination,

  11. [11]

    URLhttps://maa.org/maa-invitational-competitions/

  12. [12]

    Stop when reasoning converges: Semantic-preserving early exit for reasoning models, 2026

    Dehai Min, Giovanni Vaccarino, Huiyi Chen, Yongliang Wu, Gal Yona, and Lu Cheng. Stop when reasoning converges: Semantic-preserving early exit for reasoning models, 2026. URLhttps://arxiv.org/abs/2605.17672

    Pith/arXiv arXiv 2026

  13. [13]

    s1: Simple test-time scaling

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 20275–20321. Association for Computational Linguistics,

    2025

  14. [14]

    URLhttps://aclanthology.org/2025.emnlp-main.1025/

    doi: 10.18653/v1/2025.emnlp-main.1025. URLhttps://aclanthology.org/2025.emnlp-main.1025/

    doi:10.18653/v1/2025.emnlp-main.1025 2025

  15. [15]

    TERMINA- TOR: Learning optimal exit points for early stopping in chain-of-thought reasoning, 2026

    Alliot Nagle, Jakhongir Saydaliev, Dhia Garbaya, Michael Gastpar, Ashok Vardhan Makkuva, and Hyeji Kim. TERMINA- TOR: Learning optimal exit points for early stopping in chain-of-thought reasoning, 2026. URLhttps://arxiv.org/abs/ 2603.12529

    Pith/arXiv arXiv 2026

  16. [16]

    Conformal language modeling

    Victor Quach, Adam Fisch, Tal Schuster, Adam Yala, Jae Ho Sohn, Tommi Jaakkola, and Regina Barzilay. Conformal language modeling. InInternational Conference on Learning Representations, 2024. URLhttps://openreview.net/forum? id=pzUhfQ74c5

    2024

  17. [17]

    Qwen3 technical report, 2025

    Qwen Team. Qwen3 technical report, 2025. URLhttps://arxiv.org/abs/2505.09388

    Pith/arXiv arXiv 2025

  18. [18]

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate-level google-proof q&a benchmark. InFirst Conference on Language Modeling (COLM), 2024. URLhttps://openreview.net/forum?id=Ti67584b98

    2024

  19. [19]

    Confidence leaps in LLM reasoning: Early stopping and cross-model transfer

    Pavel Tikhonov, Ivan Oseledets, and Elena Tutubalina. Confidence leaps in LLM reasoning: Early stopping and cross-model transfer. InProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers), pages 602–616. Association for Computational Linguistics, 2026. doi: 10.18653/v1/2026.eacl-...

    doi:10.18653/v1/2026.eacl- 2026

  20. [20]

    Entropy after</think> for reasoning model early exiting,

    Xi Wang, James McInerney, Lequn Wang, and Nathan Kallus. Entropy after</think> for reasoning model early exiting,

  21. [21]

    URLhttps://arxiv.org/abs/2509.26522

    Pith/arXiv arXiv

  22. [22]

    Conformal thinking: Risk control for reasoning on a compute budget

    Xi Wang, Anushri Suresh, Alvin Zhang, Rishi More, William Jurayj, Benjamin Van Durme, Mehrdad Farajtabar, Daniel Khashabi, and Eric Nalisnick. Conformal thinking: Risk control for reasoning on a compute budget. InProceedings of the 43rd International Conference on Machine Learning (ICML), volume 306 ofProceedings of Machine Learning Research,

  23. [23]

    URLhttps://arxiv.org/abs/2602.03814

    Pith/arXiv arXiv

  24. [24]

    MMLU- Pro: A more robust and challenging multi-task language understanding benchmark

    Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. MMLU- Pro: A more robust and challenging multi-task language understanding benchmark. InAdvances in Neural Information Processing Systems Datasets...

    2024

  25. [25]

    Thought calibration: Efficient and confident test-time scaling

    Menghua Wu, Cai Zhou, Stephen Bates, and Tommi Jaakkola. Thought calibration: Efficient and confident test-time scaling. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 2025. URL https://aclanthology.org/2025.emnlp-main.722/

    2025

  26. [26]

    Dynamic early exit in reasoning models

    Chenxu Yang, Qingyi Si, Yongjie Duan, Zheliang Zhu, Chenyu Zhu, Qiaowei Li, Minghui Chen, Zheng Lin, and Weiping Wang. Dynamic early exit in reasoning models. InInternational Conference on Learning Representations, 2026. URL https://openreview.net/forum?id=NpU7ZXafRi. 23

    2026

This paper was first reviewed by grok-4.5 on July 12, 2026.