REVIEW 2 minor 39 references
Shorter Reasoning, Earlier Answers? An Evaluation of Reasoning Interfaces
T0 review · 0 major / 2 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read In these tests, a shorter reasoning trace mostly means the model stopped sooner, not that its early reasoning is better.
desk verdict Careful matched-horizon evaluation that cleanly separates stopping earlier from reasoning better; the gpt-oss 'mostly earlier completion' headline needs a symmetric-readout robustness check before the exact number is trusted. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The matched-horizon probe: paired runs of the same item are cut at a common token budget $B$ (for Qwen) or at the lower-effort run's realized terminal length $L_c$ (for gpt-oss); a token-exact replay reconstructs the stopped prefix without retokenizing, and a candidate-logit readout closes the reasoning block and scores every valid one-token spelling of each answer option from full-vocabulary logits. Completion is tracked separately, giving the terminal-aware policy accuracy $Y^{z,T}_{ir}(B)$, the both-active common-context contrast, and a four-stratum decomposition. This machinery is what lets the paper attribute an early-accuracy difference to finishing sooner versus carrying a better unfinished prefix.
What would settle it
Take a sample of unfinished high-effort gpt-oss prefixes at the lower-effort stopping point, record the candidate-logit forced answer, then let each high-effort run finish naturally and compare the two answers; if agreement on these unfinished prefixes drops far below the 98.3-100% agreement the paper reports for completed prefixes, the matched-horizon gap is an artifact of the probe rather than a property of the reasoning.
Extended reading notes
Core claim
In the tested settings, a reasoning control that produces a shorter trace and a control that produces a better answer at a fixed horizon are empirically distinct. For Qwen3-14B the numeric/concision prompt reduces geometric-mean reasoning length by 12-17% while same-horizon accuracy deltas sit within about two percentage points of zero; the only consistent early gain comes from a separate concise/early-answer instruction (+3.8 points at 512 tokens, +2.7 points among pairs where both runs are unfinished). For gpt-oss-20b and gpt-oss-120b, completed low- and medium-effort reasoning blocks beat matched-horizon high-effort prefixes by 14.5 to 26.3 points, but at the fixed 512-token checkpoint the dominant contribution is the lower-effort policy having already stopped and committed, while jointly unfinished prefixes differ by small and mixed amounts. The paper claims that accuracy-versus-trace-length curves conflate stopping policy with prefix quality; the right question is whether an answer is available at a horizon, whether it is correct, and whether it came from stopping or from the reasoning itself.
Load-bearing premise
The load-bearing premise is that the candidate-logit forced-answer readout on an unfinished reasoning prefix reveals the answer that would actually be available if the run were stopped there; if this proxy misreads unfinished prefixes, the paper's separation between completion timing and prefix quality is mis-measured.
Editorial extensions
If this is right
- Accuracy plotted against trace length should be split into correct completion by a deadline and correctness of the stopped-prefix answer, because the two move independently.
- For gpt-oss under a tight 512-token budget, lower effort is often the better deployment choice, but the advantage shrinks or reverses when both policies remain unfinished, and high effort can recover higher accuracy if allowed to finish.
- A prompt that announces a token budget can shorten reasoning without improving the early answer, and matching the announced number to the stop does not by itself help.
- Proper scores qualify accuracy: lower-effort terminal answers often concentrate probability on the chosen option, so Brier score and conditional log loss diverge, and evaluations that report only accuracy or only log loss will mislead.
Reading between the lines
- If the matched-horizon decomposition is adopted by benchmark suites, reported speedups from reasoning controls will need to be re-derived as stopping-policy effects plus prefix-quality effects, which would change how reasoning-effort settings are compared.
- The same probe could be applied to other controls, such as trained token budgets, control tokens, or draft chains, to test whether their early gains are mostly early stopping; the paper's case-study design is a template rather than a proof for other models.
- A direct testable extension would measure whether the candidate-logit forced answer on an unfinished prefix matches the answer the model gives after a small number of extra continuation tokens; the paper's generated-answer continuation partially checks this, and a larger unfinished-only cohort would tighten or weaken the central decomposition.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a matched-horizon evaluation protocol for reasoning interfaces that separates whether an intervention shortens a trace by making the model stop sooner or by changing the answer available from an unfinished prefix. It applies the protocol to Qwen3-14B numeric/concision prompting and to gpt-oss-20b/-120b effort settings on GPQA Diamond and a stratified MMLU-Pro subset, with additional calibration on Qwen3-4B/-8B and Omni-MATH-2. The central empirical claims are that the numeric/concision prompt shortens reasoning by 12–17% without a consistent matched-horizon accuracy gain; a concise/early-answer instruction gives a +3.8 point gain at B=512 including +2.7 points for jointly unfinished runs; and lower gpt-oss effort is 14.5–26.3 points more accurate at its own stopping point, with most of the fixed B=512 advantage attributable to earlier completion. The paper includes extensive validation of the forced-answer readout, proper-score analyses, and a scripted positive control.
Significance. If the results hold, the paper makes a valuable methodological contribution by showing that accuracy-versus-trace-length evaluations conflate stopping policy with prefix quality, and it provides a replicable template for separating these factors. The manuscript is unusually careful: it uses paired item–replicate designs, token-exact replay, multiple readout contexts, bootstrap and crossed-intercept inference, validation against natural answers and generated continuations, and a scripted positive control. The claims are presented with appropriate scope restrictions, and the limitations are honestly stated. The paper also ships reproducible code and data, which strengthens the empirical contribution.
minor comments (2)
- [Section 5, Table A16] The fixed-checkpoint decomposition supporting 'Earlier completion explains most of the gpt-oss advantage at the fixed B=512 checkpoint' mixes natural terminal answers for the lower-complete arm with candidate-logit forced-answer readouts for the high-active arm. The forced-answer readout is validated against natural answers for completed blocks (Table A4) and against generated continuations for the matched-horizon all-replayable comparison (Table A14), but not explicitly for this stratum. Because the lower-complete/high-active stratum is the largest positive component in all eight comparisons, I recommend adding a sensitivity analysis that expresses the B=512 decomposition under a symmetric readout (e.g., common-context replay of completed lower-effort blocks) or uses greedy continuations for the active high arm. The both-active stratum and the matched-horizon continuation checks provide indirect support, so this is a robustness improvement rather than a blocking defect.
- [Abstract and Section 5] The sentence 'Most of the 512-token advantage comes from lower effort finishing earlier' could be qualified to note that it holds under the candidate-logit readout and depends on the forced-answer readout's fidelity for unfinished prefixes. The Limitations section acknowledges this, but the abstract states the claim unconditionally; a brief qualifier would align the headline with the measurement caveat.
Circularity Check
No significant circularity: the matched-horizon estimands are measured quantities, and the key decomposition is arithmetic rather than a prediction forced by construction.
full rationale
The central comparisons define observable estimands in Section 2 — τQ,pol(B), τa_Q,BA(B), and τc_G,R — that pair independently generated traces at an imposed or itemwise horizon. No parameter is fitted to the outcome being explained, and no quoted equation reduces a reported effect to the definition of the intervention. The forced-answer readout is an instrument, not a conclusion: it is validated against natural terminal answers (Table A4, 98.3–100% agreement), a scripted positive control (Table A29), generated-answer continuations (Table A14), and length-matched swapped/shuffled/random prefix controls (Figures 2d, A3). The readout is explicitly described as a standardized probe that 'measures which answer is available, not when the model decided,' so the paper does not rename the probe as the target finding. The strongest-sounding claim, that 'earlier completion explains most of the gpt-oss advantage at the fixed B=512 checkpoint' (Section 5), is a termination-stratum decomposition of an observed accuracy gap: the lower-complete/high-active contribution is computed from within-stratum accuracy differences and shares, and it is corroborated by the separate both-active stratum, whose contributions are small and mixed. That decomposition is arithmetic partitioning of measured outcomes, not a self-fulfilling construction. Self-citations appear in refs. [10], [28], and [34], but they are used for background, for prior probing methodology, and for the secondary Omni-MATH-2 extension; the primary matched-horizon claims rest on external GPQA Diamond and MMLU-Pro data with token-exact replay. No uniqueness theorem is imported from the authors' prior work, and no ansatz is smuggled in via citation. The acknowledged limitation that 'forced-answer readout affects unfinished-prefix accuracy' is a measurement-validity caveat, disclosed and partially checked, not a circular derivation. Overall, the reasoning chain is self-contained: measurements are independent of the conclusions they support.
Assumptions & free parameters
assumptions (4)
- ad hoc to paper Candidate-logit forced-answer readout faithfully measures the answer available from an unfinished prefix.
- domain assumption Token-exact replay reconstructs stopped inputs without material information loss.
- ad hoc to paper A common answer context isolates prefix-carried differences from answer-time prompt effects.
- domain assumption The sampled items and three global replicates support within-family inference.
Cite this review
Pith. "Pith review of Shorter Reasoning, Earlier Answers? An Evaluation of Reasoning Interfaces." pith.science (2026). https://pith.science/paper/FRIBUOCE
@misc{pith2026260803401,
author = {Pith},
title = {Pith review of: Shorter Reasoning, Earlier Answers? An Evaluation of Reasoning Interfaces},
year = {2026},
howpublished = {\url{https://pith.science/paper/FRIBUOCE}},
note = {Machine review of arXiv:2608.03401}
}
read the original abstract
Large language models often reason at length before answering, increasing cost and latency. Prompts and trained settings can shorten this reasoning, but a shorter trace may only show that the model stopped sooner. Here, we evaluate paired runs of the same question at matched reasoning horizons across 198 GPQA Diamond and 500 MMLU-Pro questions. We test a numeric/concision prompt that announces a token limit for Qwen3-14B and the trained effort settings of gpt-oss-20b and -120b. The Qwen prompt shortens reasoning traces by 12-17%, while accuracy changes at matched token limits are small and mixed. A concise/early-answer instruction raises MMLU-Pro accuracy by 3.8 percentage points at 512 tokens, including +2.7 points when both runs are unfinished. Its gain at 2,048 tokens is uncertain. For gpt-oss, candidate-logit answers from completed low- and medium-effort reasoning are 14.5-26.3 points more accurate than matched-horizon high-effort answers. Most of the 512-token advantage comes from lower effort finishing earlier, while differences among unfinished runs are smaller and mixed. Wrong early answers often concentrate probability on the chosen option, so earlier stopping does not uniformly improve probability quality. In these tests, a tight deadline can favor lower effort or a concise instruction, whereas allowing high effort to finish can recover higher final accuracy. Evaluations should report correct completion before a deadline, the answer obtained when a run is stopped, differences among unfinished runs, and probability assigned to the correct answer separately.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems, 35:24824–24837, 2022. doi: 10.52202/068431-1800. URL https://doi.org/10.52202/ 068431-1800
-
[2]
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners.Advances in neural information processing systems, 35:22199–22213, 2022. doi: 10.52202/068431-1613. URLhttps://doi.org/10.52202/068431-1613
-
[3]
OpenAI o1 system card.arXiv preprint arXiv:2412.16720, 2024
Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. OpenAI o1 system card.arXiv preprint arXiv:2412.16720, 2024. URL https://arxiv.org/abs/2412.16720
arXiv 2024
-
[4]
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning.Nature, 645(8081):633–638, 2025. ISSN 1476-4687. doi: 10.1038/s41586-025-09422-z. URL https://doi.org/10. 1038/s41586-025-09422-z
-
[6]
Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM test-time compute optimally can be more effective than scaling model parameters.arXiv preprint arXiv:2408.03314, 2024. URL https://arxiv.org/ abs/2408.03314
arXiv 2024
-
[7]
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori B Hashimoto. s1: Simple test-time scaling. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 20286–20332. Association for Computational Linguistics, 2025. d...
-
[8]
Pranjal Aggarwal, Marjan Ghazvininejad, Seungone Kim, Ilia Kulikov, Jack Lanchantin, Xian Li, Tianjian Li, Bo Liu, Graham Neubig, Anaelia Ovalle, et al. Reasoning over mathematical objects: on-policy reward modeling and test time aggregation.arXiv preprint arXiv:2603.18886, 2026. URL https://arxiv.org/abs/2603. 18886
arXiv 2026
-
[9]
Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, et al. Do not think that much for 2+3=? On the overthinking of o1-like LLMs.arXiv preprint arXiv:2412.21187, 2024. URLhttps://arxiv.org/abs/2412.21187
arXiv 2024
Show all 39 references
-
[10]
The relationship between reasoning and performance in large language models—o3 (mini) Thinks Harder, Not Longer.arXiv preprint arXiv:2502.15631, 2025
Marthe Ballon, Andres Algaba, and Vincent Ginis. The relationship between reasoning and performance in large language models—o3 (mini) Thinks Harder, Not Longer.arXiv preprint arXiv:2502.15631, 2025. URL https://arxiv.org/abs/2502.15631
2025 arXiv
-
[11]
Between underthinking and overthinking: An empirical study of reasoning length and correctness in LLMs.arXiv preprint arXiv:2505.00127, 2025
Jinyan Su, Jennifer Healey, Preslav Nakov, and Claire Cardie. Between underthinking and overthinking: An empirical study of reasoning length and correctness in LLMs.arXiv preprint arXiv:2505.00127, 2025. URL https://arxiv.org/abs/2505.00127
2025 arXiv
-
[12]
Stop when enough: Adaptive early-stopping for chain-of-thought reasoning.arXiv preprint arXiv:2510.10103, 2025
Renliang Sun, Wei Cheng, Dawei Li, Haifeng Chen, and Wei Wang. Stop when enough: Adaptive early-stopping for chain-of-thought reasoning.arXiv preprint arXiv:2510.10103, 2025. URL https://arxiv.org/abs/2510. 10103
2025
-
[13]
An analysis of time-dependent planning
Thomas Dean and Mark Boddy. An analysis of time-dependent planning. InProceedings of the Seventh National Conference on Artificial Intelligence, pages 49–54, 1988
1988
-
[14]
Using anytime algorithms in intelligent systems.AI Magazine, 17(3):73–83, 1996
Shlomo Zilberstein. Using anytime algorithms in intelligent systems.AI Magazine, 17(3):73–83, 1996
1996
-
[15]
Russell and Eric H
Stuart J. Russell and Eric H. Wefald.Do the Right Thing: Studies in Limited Rationality. MIT Press, Cambridge, MA, 1991. doi: 10.7551/mitpress/2474.001.0001. URL https://doi.org/10.7551/mitpress/2474.001. 0001
1991 doi
-
[16]
L1: Controlling how long a reasoning model thinks with reinforcement learning.arXiv preprint arXiv:2503.04697, 2025
Pranjal Aggarwal and Sean Welleck. L1: Controlling how long a reasoning model thinks with reinforcement learning.arXiv preprint arXiv:2503.04697, 2025. URLhttps://arxiv.org/abs/2503.04697
2025 arXiv
-
[17]
Token-budget-aware LLM reasoning
Tingxu Han, Chunrong Fang, Shiyu Zhao, Shiqing Ma, Zhenyu Chen, and Zhenting Wang. Token-budget-aware LLM reasoning. InFindings of the Association for Computational Linguistics: ACL 2025, pages 24842– 24855. Association for Computational Linguistics, 2025. doi: 10.18653/v1/202...
2025 doi
-
[18]
Training language models to reason efficiently.arXiv preprint arXiv:2502.04463, 2025
Daman Arora and Andrea Zanette. Training language models to reason efficiently.arXiv preprint arXiv:2502.04463, 2025. URLhttps://arxiv.org/abs/2502.04463
2025
-
[20]
Steering LLM thinking with budget guidance.arXiv preprint arXiv:2506.13752, 2025
Junyan Li, Wenshuo Zhao, Yang Zhang, and Chuang Gan. Steering LLM thinking with budget guidance.arXiv preprint arXiv:2506.13752, 2025. URLhttps://arxiv.org/abs/2506.13752
2025 arXiv
-
[21]
An empirical study of LLM reasoning ability under strict output length constraint
Yi Sun, Han Wang, Jiaqiang Li, Jiacheng Liu, Xiangyu Li, Hao Wen, Yizhen Yuan, Huiwen Zheng, Yan Liang, Yuanchun Li, and Yunxin Liu. An empirical study of LLM reasoning ability under strict output length constraint. InProceedings of the 2025 Conference on Empirical Methods in ...
2025 doi
-
[22]
BudgetThinker: Empowering budget-aware LLM reasoning with control tokens.arXiv preprint arXiv:2508.17196, 2025
Hao Wen, Xinrui Wu, Yi Sun, Feifei Zhang, Liye Chen, Jie Wang, Yunxin Liu, Yunhao Liu, Ya-Qin Zhang, and Yuanchun Li. BudgetThinker: Empowering budget-aware LLM reasoning with control tokens.arXiv preprint arXiv:2508.17196, 2025. URLhttps://arxiv.org/abs/2508.17196
2025 arXiv
-
[23]
The benefits of a concise chain of thought on problem-solving in large language models.arXiv preprint arXiv:2401.05618, 2024
Matthew Renze and Erhan Guven. The benefits of a concise chain of thought on problem-solving in large language models.arXiv preprint arXiv:2401.05618, 2024. URLhttps://arxiv.org/abs/2401.05618
2024 arXiv
-
[24]
Concise thoughts: Impact of output length on LLM reasoning and cost.arXiv preprint arXiv:2407.19825, 2024
Sania Nayab, Giulio Rossolini, Marco Simoni, Andrea Saracino, Giorgio Buttazzo, Nicolamaria Manes, and Fabrizio Giacomelli. Concise thoughts: Impact of output length on LLM reasoning and cost.arXiv preprint arXiv:2407.19825, 2024. URLhttps://arxiv.org/abs/2407.19825
2024 arXiv
-
[25]
Chain of draft: Thinking faster by writing less.arXiv preprint arXiv:2502.18600, 2025
Silei Xu, Wenhao Xie, Lingxiao Zhao, and Pengcheng He. Chain of draft: Thinking faster by writing less.arXiv preprint arXiv:2502.18600, 2025. URLhttps://arxiv.org/abs/2502.18600
2025 arXiv
-
[26]
Stop overthinking: A survey on efficient reasoning for large language models.arXiv preprint arXiv:2503.16419, 2025
Yang Sui, Yu-Neng Chuang, Guanchu Wang, et al. Stop overthinking: A survey on efficient reasoning for large language models.arXiv preprint arXiv:2503.16419, 2025. URLhttps://arxiv.org/abs/2503.16419
2025 arXiv
-
[27]
How inference compute shapes frontier LLM evaluation.arXiv preprint arXiv:2606.17930, 2026
Jessica McFadyen, Ole Jorgensen, Harry Coppock, Kevin Wei, and Cozmin Ududec. How inference compute shapes frontier LLM evaluation.arXiv preprint arXiv:2606.17930, 2026. URL https://arxiv.org/abs/2606. 17930
2026 arXiv
-
[28]
Probing the trajectories of reasoning traces in large language models.arXiv preprint arXiv:2601.23163, 2026
Marthe Ballon, Brecht Verbeken, Vincent Ginis, and Andres Algaba. Probing the trajectories of reasoning traces in large language models.arXiv preprint arXiv:2601.23163, 2026. URL https://arxiv.org/abs/2601.23163
2026
-
[29]
Large language models decide early and explain later.arXiv preprint arXiv:2604.22266, 2026
Ayan Datta, Zhixue Zhao, Bhuvanesh Verma, Radhika Mamidi, Mounika Marreddy, and Alexander Mehler. Large language models decide early and explain later.arXiv preprint arXiv:2604.22266, 2026. URL https: //arxiv.org/abs/2604.22266
2026 arXiv
-
[30]
Chan, Trevor Darrell, Narges Norouzi, and Joseph E
Tsung-Han Wu, Mihran Miroyan, David M. Chan, Trevor Darrell, Narges Norouzi, and Joseph E. Gonzalez. Are large reasoning models interruptible?arXiv preprint arXiv:2510.11713, 2025. URL https://arxiv.org/abs/ 2510.11713
2025 arXiv
-
[31]
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate-level Google-proof Q&A benchmark. InFirst Conference on Language Modeling, 2024. URLhttps://openreview.net/forum?id=Ti67584b98
2024
-
[32]
MMLU-Pro: A more robust and challenging multi-task language un- derstanding benchmark.Advances in Neural Information Processing Systems, 37:95266–95290, 2024
Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. MMLU-Pro: A more robust and challenging multi-task language un- derstanding benchmark.Advances in Neural Information Processing Systems, 37:...
2024 doi
-
[33]
Omni-MATH: A universal olympiad level mathematic benchmark for large language models
Bofei Gao, Feifan Song, Zhe Yang, Zefan Cai, Yibo Miao, Qingxiu Dong, Lei Li, Chenghao Ma, Liang Chen, Runxin Xu, et al. Omni-MATH: A universal olympiad level mathematic benchmark for large language models. arXiv preprint arXiv:2410.07985, 2024. URLhttps://arxiv.org/abs/2410.07985
-
[34]
Benchmarks saturate when the model gets smarter than the judge.arXiv preprint arXiv:2601.19532, 2026
Marthe Ballon, Andres Algaba, Brecht Verbeken, and Vincent Ginis. Benchmarks saturate when the model gets smarter than the judge.arXiv preprint arXiv:2601.19532, 2026. URL https://arxiv.org/abs/2601.19532
2026
-
[35]
Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025. URL https: //arxiv.org/abs/2505.09388
2025 arXiv
-
[36]
Qwen3 thinking budget
Qwen Team. Qwen3 thinking budget. https://github.com/QwenLM/Qwen3/blob/main/docs/source/ getting_started/thinking_budget.md, 2025. Accessed 20 June 2026. 12 Shorter Reasoning, Earlier Answers? An Evaluation of Reasoning Interfaces
2025
-
[37]
Monitoring reasoning models for misbehavior and the risks of promoting obfuscation.arXiv preprint arXiv:2503.11926, 2025
Bowen Baker, Joost Huizinga, Leo Gao, Zehao Dou, Melody Y Guan, Aleksander Madry, Wojciech Zaremba, Jakub Pachocki, and David Farhi. Monitoring reasoning models for misbehavior and the risks of promoting obfuscation.arXiv preprint arXiv:2503.11926, 2025. URLhttps://arxiv.org/a...
2025 arXiv
-
[38]
Chain of thought monitorability: A new and fragile opportunity for AI safety.arXiv preprint arXiv:2507.11473, 2025
Tomek Korbak, Mikita Balesni, Elizabeth Barnes, Yoshua Bengio, Joe Benton, Joseph Bloom, Mark Chen, Alan Cooney, Allan Dafoe, Anca Dragan, et al. Chain of thought monitorability: A new and fragile opportunity for AI safety.arXiv preprint arXiv:2507.11473, 2025. URLhttps://arxi...
2025 arXiv
-
[39]
Finished
Quinn McNemar. Note on the sampling error of the difference between correlated proportions or percentages.Psy- chometrika, 12(2):153–157, 1947. doi: 10.1007/BF02295996. URL https://doi.org/10.1007/BF02295996. 13 Shorter Reasoning, Earlier Answers? An Evaluation of Reasoning In...
1947 doi
-
[2022]
URLhttps://arxiv.org/abs/2203.11171
-
[2025]
URLhttps://arxiv.org/abs/2508.10925
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.