Pith. sign in

REVIEW 4 major objections 7 minor 49 references

Large language models can be trained to refuse unsolvable problems, cutting futile reasoning from 79% to 1% on hard tasks while preserving accuracy.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 11:32 UTC pith:6J5DUYAK

load-bearing objection A solid empirical diagnosis of futile reasoning and a promising RL fix, but the 'preserved performance' claim is weakened by over-refusal on OOD and an internal inconsistency about vanilla refusal rates. the 4 major comments →

arxiv 2607.29211 v1 pith:6J5DUYAK submitted 2026-07-31 cs.CL

Knowing When to Quit: Diagnosing and Training LLMs to Abort Futile Reasoning

Xinyan Guan , Jiali Zeng , Chunlei Xin , Yaojie Lu , Hongyu Lin , Xianpei Han , Le Sun , Fandong Meng This is my paper
classification cs.CL
keywords futile reasoningcapability alignmentrefusal trainingreinforcement learningreward shapinghindsight augmentationLLM reasoningover-confidence
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Large language models, when faced with problems beyond their abilities, often produce long, plausible-sounding but incorrect reasoning instead of admitting they cannot solve them. The paper calls this 'futile reasoning' and shows it is pervasive across model sizes and difficulty levels. It introduces CaRL, a reinforcement-learning method that teaches models when to abort and refuse, using a reward hierarchy that favors refusal over hallucination and a hindsight-augmentation step that turns every failed attempt into a refusal training example. On Countdown and Sudoku tasks, CaRL cuts the futile rate from 65.5% to 7.0% (8B) and 78.6% to 1.0% (14B) while maintaining or improving reliability and shortening outputs. If true, this offers a concrete way to make LLMs safer for high-stakes use.

Core claim

The paper's central claim is that 'futile reasoning'—the generation of superficially valid but incorrect derivations on tasks a model cannot solve—is a distinct, dominant failure mode that can be diagnosed and suppressed by aligning model behavior with its actual capability boundary. The authors establish three phenomena: models uniformly overreach (near-zero refusal even when error rates exceed 65%), the prevalent failure style is 'specious reasoning' that escalates with difficulty, and models are miscalibrated by a roughly 6:1 bias toward over-confidence rather than over-conservatism. They then show that their CaRL method—combining a reward function that ranks refusal above hallucination w

What carries the argument

Futile reasoning is the paper's central object: reasoning traces that look coherent but are mathematically wrong, produced when the model exceeds its capability boundary. The key mechanism is CaRL, which works through two complementary parts. Capability-Calibrated Reward Shaping replaces the usual binary reward with a hierarchy (+1 for correct, 0 for a valid refusal, -1 for incorrect), making refusal strictly better than hallucination. Hindsight Refusal Augmentation addresses the scarcity of refusal examples by taking every incorrect on-policy trajectory and retrofitting it into a refusal response—keeping the reasoning prefix, adding 'Sorry, I don't know', and assigning the refusal reward—so

Load-bearing premise

The load-bearing premise is stated in Section 4.2: 'every failed reasoning attempt implicitly reveals a situation where refusal would have been the appropriate action.' In other words, the method assumes that whenever the current policy produces a wrong answer, the problem is beyond the model's capability and deserves a refusal. If some failures are merely inference-time misses on problems the model can actually solve, this training signal teaches the model to over-refuse.

What would settle it

A direct test: take a CaRL-trained model and a set of problems where the model's failures are known to be solvable (e.g., problems it answers correctly under a different seed or with a few extra reasoning steps, or problems verified solvable by a brute-force solver). Measure how often the trained model now refuses these solvable-but-previously-missed problems. If the refusal rate on that set is high (near the failure rate), the method is teaching over-refusal rather than true capability alignment—the Sudoku accuracy drop on 14B (50.75→44.87) is already suggestive of this.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • CaRL-trained models should refuse more on unsolvable queries and produce shorter outputs, cutting compute by roughly one-third on hard reasoning tasks.
  • Reliability, defined as correct answers plus half-credit for refusals, rises by 0.13–0.16 points on 8B and 14B models even when accuracy stays flat, meaning the same model is safer to deploy in high-stakes settings.
  • The method transfers to out-of-distribution tasks (Sudoku) and to general benchmarks (AIME, GPQA) with less than 2% accuracy change, implying capability-boundary awareness is not just task memorization.
  • Standard reward-based RL without hindsight augmentation fails to reduce futile reasoning (futile rate stays above 95% on 8B), implying that dense refusal supervision, not reward shaping alone, is the load-bearing ingredient.
  • Larger models benefit more from CaRL (14B futile rate drops to 1.0% vs 7.0% for 8B), suggesting scale amplifies the ability to internalize refusal boundaries.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The core training assumption—that every failed attempt is a refusal-worthy failure—may teach over-refusal on solvable problems the model simply missed; the paper's own OOD Sudoku accuracy drop (50.75 to 44.87 on 14B) hints at this risk, and a dedicated evaluation on 'falsely refused but actually solvable' items would quantify it.
  • CaRL's reward function treats all refusals as equal (0), so a model could learn to refuse early and cheaply rather than after genuine exploration; ablating on refusal timing or conditioning reward on reasoning effort could distinguish 'knowing when to quit' from 'lazy refusal'.
  • The findings suggest a general principle for aligning LLMs: capability boundaries can be learned from the model's own errors by relabeling them as refusals, which might be applicable beyond arithmetic puzzles to knowledge-boundary refusal if the hindsight signal can be made reliable.
  • If scaled, the compute savings from aborting futile reasoning could change the economics of test-time scaling, making reasoning models more practical for budget-constrained deployments.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper defines and characterizes 'futile reasoning' — long, plausible-looking but incorrect chains produced by LLMs on tasks beyond their capability — using a difficulty-graded Countdown testbed. It reports three phenomena: universal capability overreach (models rarely refuse), specious reasoning as the dominant failure mode, and a capability-behavior misalignment biased toward overconfidence. To address this, the authors propose CaRL, a GRPO-based training method combining reward shaping that ranks refusal above incorrect answers with Hindsight Refusal Augmentation (HRA), which converts every failed rollout into a refusal training example. Experiments on Qwen3-8B/14B report large reductions in futile rate (65.5→7.0% and 78.6→1.0%) while retaining or improving in-distribution accuracy, and a smaller but positive reliability gain on OOD Sudoku and general benchmarks. The central claim is that CaRL achieves capability-aligned behavior without sacrificing utility.

Significance. If the results hold, the paper makes a useful contribution: it provides a clean controlled testbed for studying over-confidence in reasoning models, identifies a concrete failure taxonomy, and proposes a training intervention that appears to substantially reduce hallucinated derivations while keeping task accuracy roughly intact. The authors are appropriately careful to separate reasoning capability from knowledge, and they include OOD and general-task evaluations, which is more than most intervention papers do. The strongest assets are the controlled difficulty design, the explicit refusal-vs-futile metric, and the comparison against Standard RL and RFT baselines, including a cautionary note that RFT's low futile rate on OOD is merely an artifact of near-total refusal. However, the significance is conditional: the central 'without sacrificing utility' claim depends on whether HRA's relabeling of every failed trajectory as refusal-worthy is actually capability-aligned, and the current evidence does not establish that the refused instances are the unsolvable ones. The paper's own OOD accuracy drops and the absence of uncertainty estimates make this a substantive correctness risk rat

major comments (4)
  1. [§4.2, Eq. (1), Algorithm 1] HRA relabels every r=-1 trajectory as a refusal demonstration (r=0), based on the premise that 'every failed reasoning attempt implicitly reveals a situation where refusal would have been the appropriate action' (§4.2). Since the Countdown training set is generated with golden answers (Appendix A.1), a substantial fraction of failures are likely inference-time misses on queries the model can solve, not evidence of beyond-capability tasks. Training on these relabeled failures teaches the policy to refuse solvable queries. This is exactly the risk visible in Table 1: the 14B OOD Sudoku accuracy drops from 50.75 to 44.87 while refusal rises to 38.87%, and the abstract's 'without sacrificing utility' therefore rests on an untested assumption. The authors should decompose refused/evaluated instances by whether they are truly unsolvable, and/or ablate HRA using only failures on known-unsolvabl
  2. [§3.4, §4.2] The capability boundary is operationalized circularly. In §3.4 a query is 'solvable' if the model's own sampled accuracy exceeds 0.5; HRA then uses the same model's failures as ground-truth refusal labels. Consequently, the reported 'calibration' improvements may be self-confirming: the model is trained to treat its own failures as refusal-worthy, then evaluated on whether it refuses its own failures. An external criterion (e.g., provable solvability via the golden answer, or an independent solver's success) is needed to show that refusals concentrate on genuinely unsolvable queries rather than on hard-but-solvable ones. Without this, the headline futile-rate reduction does not establish capability-aligned behavior.
  3. [§3.2, §4.2, Table 1] The text states that 'vanilla models never refuse (0% across all difficulty levels)' and that baseline models exhibit '0% refusal rate' (§3.2, §4.2). Table 1 reports vanilla refusal rates of 13.92% (Qwen3-8B ID), 7.88% (Qwen3-14B ID), 10.62% (8B OOD), and 18.62% (14B OOD). These are not small rounding effects. The contradiction weakens the motivation for HRA (which is justified by the alleged absence of on-policy refusals) and must be reconciled. If 'refusal' is being counted differently in the two places, that definitional difference should be explicit.
  4. [Table 1, §5.4, Appendix C.1] All headline comparisons are reported as point estimates without error bars, multiple seeds, or significance tests. The paper states that 8 responses per query are averaged, but that does not quantify variation across runs or across query samples. The OOD accuracy drop of 5.9 points for Qwen3-14B, the RFT collapse to 0% accuracy, and the claimed 'preserving performance' are load-bearing; without variance estimates these could be noise. The authors should rerun at least the main comparisons with 3-5 seeds and report means and standard deviations, or provide bootstrap confidence intervals.
minor comments (7)
  1. [§1] CaRL is spelled 'CARL' once in the introduction; use 'CaRL' consistently.
  2. [Figure 11 caption] Typo: 'Supicous' should be 'Specious'.
  3. [§3.1, §3.2, §5.1] Difficulty levels are inconsistent: §3.2 says N=3 to N=8, Figure 2 shows N=3-8, but §5.1 says only N=4,6,8 are generated and Table 2 uses only those. Clarify which difficulty levels were actually evaluated and why N=3 is excluded from training/evaluation.
  4. [§4.1, Eq. (1)] The detection of a 'valid refusal' is not formalized. Please provide the exact refusal-snippet matching procedure (e.g., regex patterns, threshold on confidence) used in both reward computation and evaluation.
  5. [§5.2, Table 1] The notation 'RL unk=0' and 'RL unk=0.5' is hard to read; use subscripts or a table column label such as 'RL_unk=0'.
  6. [§5.5, Table 2] Table 2 reports futile rate and length only for Qwen3-8B. If 14B results differ materially, include them or state why they are omitted.
  7. [§5.3] The reliability score weights correct=1.0 and refusal=0.5 arbitrarily. Since the paper's central evaluation metric is not parameter-free, a brief sensitivity analysis (e.g., weight 0.25/0.75) would strengthen the conclusions.

Circularity Check

0 steps flagged

No significant circularity: CaRL's headline reductions are empirical training/evaluation results, not fitted parameters or self-citation chains renamed as predictions.

full rationale

The paper contains no derivation in which a predicted quantity is equal to an input by construction. The central measures are defined independently of the training batches: Futile Rate = Ni/(Ni+Nr) and Reliability = (Nc+0.5Nr)/N are computed on held-out Countdown and OOD Sudoku sets sampled 8 times per query. CaRL's reward shaping (Eq. 1) does encode a preference for refusal over incorrect answers, so a drop in futile rate is expected from the optimization objective, but the reported effect is an empirical outcome on held-out data including an OOD stress test, not a fitted parameter renamed as a prediction. The HRA premise—'every failed reasoning attempt implicitly reveals a situation where refusal would have been the appropriate action'—is a strong assumption, and the paper's own 14B OOD Sudoku accuracy drop (50.75→44.87) plus absence of error bars are legitimate validity concerns, as is the internal inconsistency between the '0% refusal' motivation and Table 1's 13.92%/7.88% vanilla refusal rates. These undermine generalization claims but are not circular reductions. The Reliability metric's 0.5 credit for refusal aligns with the reward hierarchy, so reliability gains should be read as partially reflecting the chosen objective; this is an evaluation-design caveat rather than a circularity because accuracy is also reported and the headline futile-rate reduction is not derived from the metric. The only self-citation (Xin et al. 2025 in §5.6) supports a peripheral layer-localization interpretation and is not load-bearing. No uniqueness theorem, imported ansatz, or definitional equivalence was found.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 2 invented entities

The central contribution rests on hand-set reward values, an arbitrary solvability threshold, and the strong assumption that all training failures should be turned into refusals. The named constructs 'futile reasoning' and 'capability boundary' are descriptive labels without standalone external evidence. No new physical entities are introduced.

free parameters (3)
  • Reward values in Eq. (1) = +1 / 0 / -1
    Hand-chosen hierarchy rewarding correct answers over refusals over incorrect answers; no sensitivity analysis is provided.
  • Solvability threshold for capability quadrants = 0.5
    In §3.4 a task is classified as solvable if the model's accuracy exceeds 0.5 over sampled responses; this arbitrary cutoff defines the capability boundary and the misalignment statistics.
  • Reliability metric weights = 1.0 correct / 0.5 refusal / 0 incorrect
    Hand-assigned utility weights; the 0.5 credit for refusal directly shapes the reported reliability improvements.
axioms (5)
  • domain assumption Countdown and Sudoku instances from reasoning-gym are verifiable and difficulty-graded as intended
    The entire diagnosis and training supervision rely on exact verifiability of arithmetic expressions and Sudoku solutions (§5.1).
  • domain assumption Model accuracy over 8 sampled responses estimates true capability
    The capability quadrants in §3.4 and the solvable/unsolvable labeling use an accuracy threshold on sampled responses, not ground-truth task solvability.
  • domain assumption A valid refusal can be reliably detected from explicit refusal snippets
    The reward function in Eq. (1) requires automatic identification of refusals; the paper does not report the precision/recall of this detector.
  • ad hoc to paper Every failed reasoning trajectory during training is a refusal-worthy outcome
    Hindsight Refusal Augmentation relabels every r=-1 trajectory as a refusal demonstration (§4.2), including failures on solvable tasks that the policy simply failed to solve.
  • standard math GRPO with group-relative advantages provides a valid policy optimization objective
    The training relies on GRPO as a standard RL algorithm without re-derivation (§4.3).
invented entities (2)
  • Futile reasoning no independent evidence
    purpose: Names and taxonomizes the failure mode into specious reasoning, endless generation, and degenerate repetition.
    The category is operationalized only through the paper's own metrics (error rate, refusal rate, futile rate); there is no external benchmark or independent measurement of 'futility'.
  • Capability boundary no independent evidence
    purpose: Latent construct that CaRL is claimed to align behavior with.
    Operationalized by the model's own sampled accuracy threshold of 0.5 (§3.4), not measured independently; its existence and location are inferred from behavior.

pith-pipeline@v1.3.0-daily-deepseek · 14810 in / 16319 out tokens · 166381 ms · 2026-08-03T11:32:01.931240+00:00 · methodology

0 comments
read the original abstract

Large language models generate computationally expensive yet semantically void reasoning on beyond-capability tasks, creating risks where plausible-sounding but incorrect derivations mislead users. We characterize this \textit{futile reasoning} phenomenon through systematic analysis, revealing universal capability overreach and systematic miscalibration between capability and behavior. The dominant failure mode is specious reasoning, which outputs look superficially valid but contain subtle errors, escalating with task difficulty. To address this, we introduce \textbf{CaRL} (\textbf{Ca}pability-\textbf{a}ligned \textbf{R}einforcement \textbf{L}earning), which aligns model behavior with capability boundaries through reward shaping that incentivizes refusal over futile reasoning and hindsight refusal augmentation that converts failures into refusal supervision. Experiments demonstrate a substantial reduction in futile reasoning while preserving performance across task difficulties, effectively achieving capability-aligned behavior without sacrificing utility. \footnote{https://github.com/icip-cas/Knowing-When-to-Quit}

Figures

Figures reproduced from arXiv: 2607.29211 by Chunlei Xin, Fandong Meng, Hongyu Lin, Jiali Zeng, Le Sun, Xianpei Han, Xinyan Guan, Yaojie Lu.

Figure 1
Figure 1. Figure 1: Illustration of futile reasoning generated by DeepSeek-R1 (Guo et al., 2025). When faced with problems beyond its capability, the model generates plausible-looking but fundamentally incorrect reasoning traces instead of refusing to answer. semble valid solutions. Due to the lengthy and convoluted nature of these generated traces, distin￾guishing valid reasoning from such hallucinations is notoriously diffi… view at source ↗
Figure 2
Figure 2. Figure 2: Universal Capability Overreach. We first investigate whether models possess an in￾trinsic mechanism to acknowledge ignorance when facing unsolvable problems. We conduct experi￾ments on Qwen3-8B (Yang et al., 2025), Qwen3- 32B (Yang et al., 2025), gpt-oss-120b (OpenAI, 2025), Qwen3-235B-A22B (Yang et al., 2025), and DeepSeek-V3.2 (Liu et al., 2025) across difficulty levels (N = 3 to N = 8). We experiment on… view at source ↗
Figure 3
Figure 3. Figure 3: Distribution of Futile Reasoning Patterns. [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: The Alignment Trade-off. Naive prompting leads to a collapse in Refusal Recall (Green) on hard tasks while simultaneously increasing Capability Loss (Pink) on solvable tasks. 5000 7500 10000 12500 15000 17500 20000 22500 Average Response Length (Tokens) 0.00000 0.00005 0.00010 0.00015 0.00020 0.00025 Density quadrant D. Ideal Refusal A. Ideal Answer B. Over Conservative C. Hallucination [PITH_FULL_IMAGE:f… view at source ↗
Figure 6
Figure 6. Figure 6: Reasoning Depth Distribution. Refusal behaviors show decisive termination (Peaks), whereas Over-Confidence exhibits a long-tail distribution, con￾firming the high computational cost of futile reasoning. Computational Cost of Misalignment. Beyond correctness, misalignment imposes computational costs [PITH_FULL_IMAGE:figures/full_fig_p004_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Overview of CaRL. Our framework combines (1) Capability-Calibrated Reward Shaping that establishes a preference hierarchy favoring refusal over hallucination, and (2) Hindsight Refusal Augmentation that converts failed trajectories into refusal trajectories, enabling capability-aligned behavior. framework that aligns model behavior with capa￾bility boundaries through explicit reinforcement learning. As sho… view at source ↗
Figure 8
Figure 8. Figure 8: Case study on the countdown task. late to harder tasks. In contrast, CaRL maintains robust generalization (1.96% → 8.12%), demon￾strating that reinforcement learning enables capa￾bility boundary internalization across difficulties. Efficiency. RFT produces the longest responses (9133 tokens at Level 8), exhibiting verbose fail￾ure from prolonged futile reasoning. On the con￾trary, CaRL achieves nearly 33% … view at source ↗
Figure 9
Figure 9. Figure 9: Initial Reasoning Phase. The model system￾atically explores combinations (e.g., 97 + 66 = 163, 3 × 51 = 153), attempting to construct the target value 275. Early attempts show valid mathematical reasoning but fail to reach the exact target [PITH_FULL_IMAGE:figures/full_fig_p011_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Final Output After Degenerate Repeti￾tion. After 50+ failed attempts, the model outputs (97 + 66 + 51 + 38 + 37) − (3 + 3 + 3) = 280 while incorrectly asserting it equals 275. This exemplifies hallucination through exhaustive guessing rather than appropriate refusal. B.2 Endless Generation We illustrate this failure mode using a Countdown task with numbers [58, 89, 15, 47, 37, 84, 86, 70] and target 168. … view at source ↗
Figure 11
Figure 11. Figure 11: Supicous Reasoning Example [PITH_FULL_IMAGE:figures/full_fig_p013_11.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

49 extracted references · 22 linked inside Pith

  1. [1]

    Art of Problem Solving . n.d. AIME problems and solutions. https://artofproblemsolving.com/wiki/index.php/AIME_Problems_and_Solutions. Accessed: 2026-01-06

    2026

  2. [10]

    Richard A Davis, Keh-Shin Lii, and Dimitris N Politis. 2011. Remarks on some nonparametric estimates of a density function. In Selected Works of Murray Rosenblatt, pages 95--100. Springer

    2011

  3. [12]

    Jie Huang and Kevin Chen-Chuan Chang. 2023. Towards reasoning in large language models: A survey. In Findings of the association for computational linguistics: ACL 2023, pages 1049--1065

    2023

  4. [13]

    Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and 1 others. 2025 a . A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems, 43(2):1--55

    2025

  5. [17]

    Gautam Siddharth Kashyap, Mark Dras, and Usman Naseem. 2025. Too helpful, too harmless, too honest or just right? In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 29711--29722

    2025

  6. [23]

    OpenAI. 2025. https://arxiv.org/abs/2508.10925 gpt-oss-120b & gpt-oss-20b model card . Preprint, arXiv:2508.10925

    Pith/arXiv arXiv 2025

  7. [24]

    Emanuel Parzen. 1962. On estimation of a probability density function and mode. The annals of mathematical statistics, 33(3):1065--1076

    1962

  8. [25]

    Ethan Perez, Sam Ringer, Kamile Lukosiute, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, and 1 others. 2023. Discovering language model behaviors with model-written evaluations. In Findings of the association for computational linguistics: ACL 2023, pages 13387--13434

    2023

  9. [26]

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2024. Gpqa: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling

    2024

  10. [35]

    Aho and Jeffrey D

    Alfred V. Aho and Jeffrey D. Ullman , title =. 1972

    1972

  11. [36]

    Publications Manual , year = "1983", publisher =

    1983

  12. [37]

    Chandra and Dexter C

    Ashok K. Chandra and Dexter C. Kozen and Larry J. Stockmeyer , year = "1981", title =. doi:10.1145/322234.322243

    arXiv 1981

  13. [38]

    Scalable training of

    Andrew, Galen and Gao, Jianfeng , booktitle=. Scalable training of

  14. [39]

    Dan Gusfield , title =. 1997

    1997

  15. [40]

    Tetreault , title =

    Mohammad Sadegh Rasooli and Joel R. Tetreault , title =. Computing Research Repository , volume =. 2015 , url =

    2015

  16. [41]

    A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =

    Ando, Rie Kubota and Zhang, Tong , Issn =. A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =. Journal of Machine Learning Research , Month = dec, Numpages =

  17. [42]

    arXiv preprint arXiv:2412.21187 , year=

    Do not think that much for 2+ 3=? on the overthinking of o1-like llms , author=. arXiv preprint arXiv:2412.21187 , year=

    Pith/arXiv arXiv

  18. [43]

    arXiv preprint arXiv:2502.03373 , year=

    Demystifying long chain-of-thought reasoning in llms , author=. arXiv preprint arXiv:2502.03373 , year=

    Pith/arXiv arXiv

  19. [44]

    arXiv preprint arXiv:2401.13275 , year=

    Can AI assistants know what they don't know? , author=. arXiv preprint arXiv:2401.13275 , year=

    Pith/arXiv arXiv

  20. [45]

    arXiv preprint arXiv:2207.05221 , year=

    Language models (mostly) know what they know , author=. arXiv preprint arXiv:2207.05221 , year=

    Pith/arXiv arXiv

  21. [46]

    arXiv preprint arXiv:2512.08093 , year=

    Training LLMs for Honesty via Confessions , author=. arXiv preprint arXiv:2512.08093 , year=

    arXiv

  22. [47]

    arXiv preprint arXiv:2501.09686 , year=

    Towards large reasoning models: A survey of reinforced reasoning with large language models , author=. arXiv preprint arXiv:2501.09686 , year=

    Pith/arXiv arXiv

  23. [48]

    arXiv preprint arXiv:2402.06196 , year=

    Large language models: A survey , author=. arXiv preprint arXiv:2402.06196 , year=

    Pith/arXiv arXiv

  24. [49]

    arXiv preprint arXiv:2303.18223 , volume=

    A survey of large language models , author=. arXiv preprint arXiv:2303.18223 , volume=

    Pith/arXiv arXiv

  25. [50]

    arXiv preprint arXiv:2308.05374 , year=

    Trustworthy llms: a survey and guideline for evaluating large language models' alignment , author=. arXiv preprint arXiv:2308.05374 , year=

    Pith/arXiv arXiv

  26. [51]

    ACM Transactions on Information Systems , volume=

    A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions , author=. ACM Transactions on Information Systems , volume=. 2025 , publisher=

    2025

  27. [52]

    arXiv preprint arXiv:2405.09589 , year=

    A comprehensive survey of hallucination in large language, image, video and audio foundation models , author=. arXiv preprint arXiv:2405.09589 , year=

    Pith/arXiv arXiv

  28. [53]

    arXiv preprint arXiv:2205.14334 , year=

    Teaching models to express their uncertainty in words , author=. arXiv preprint arXiv:2205.14334 , year=

    Pith/arXiv arXiv

  29. [54]

    Selected Works of Murray Rosenblatt , pages=

    Remarks on some nonparametric estimates of a density function , author=. Selected Works of Murray Rosenblatt , pages=. 2011 , publisher=

    2011

  30. [55]

    The annals of mathematical statistics , volume=

    On estimation of a probability density function and mode , author=. The annals of mathematical statistics , volume=. 1962 , publisher=

    1962

  31. [56]

    arXiv preprint arXiv:2412.19437 , year=

    Deepseek-v3 technical report , author=. arXiv preprint arXiv:2412.19437 , year=

    Pith/arXiv arXiv

  32. [57]

    arXiv preprint arXiv:2501.12948 , year=

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning , author=. arXiv preprint arXiv:2501.12948 , year=

    Pith/arXiv arXiv

  33. [58]

    arXiv preprint arXiv:2509.06948 , year=

    Beyond two-stage training: Cooperative sft and rl for llm reasoning , author=. arXiv preprint arXiv:2509.06948 , year=

    Pith/arXiv arXiv

  34. [59]

    arXiv preprint arXiv:2504.11468 , year=

    Sft or rl? an early investigation into training r1-like reasoning large vision-language models , author=. arXiv preprint arXiv:2504.11468 , year=

    Pith/arXiv arXiv

  35. [60]

    arXiv preprint arXiv:2508.11408 , year=

    On-policy rl meets off-policy experts: Harmonizing supervised fine-tuning and reinforcement learning via dynamic weighting , author=. arXiv preprint arXiv:2508.11408 , year=

    arXiv

  36. [61]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    Too Helpful, Too Harmless, Too Honest or Just Right? , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  37. [62]

    arXiv preprint arXiv:2502.06059 , year=

    Position: We need an adaptive interpretation of helpful, honest, and harmless principles , author=. arXiv preprint arXiv:2502.06059 , year=

    arXiv

  38. [63]

    arXiv preprint arXiv:2204.05862 , year=

    Training a helpful and harmless assistant with reinforcement learning from human feedback , author=. arXiv preprint arXiv:2204.05862 , year=

    Pith/arXiv arXiv

  39. [64]

    arXiv preprint arXiv:2308.04371 , year=

    Cumulative reasoning with large language models , author=. arXiv preprint arXiv:2308.04371 , year=

    Pith/arXiv arXiv

  40. [65]

    arXiv preprint arXiv:2505.09388 , year=

    Qwen3 technical report , author=. arXiv preprint arXiv:2505.09388 , year=

    Pith/arXiv arXiv

  41. [66]

    Findings of the association for computational linguistics: ACL 2023 , pages=

    Towards reasoning in large language models: A survey , author=. Findings of the association for computational linguistics: ACL 2023 , pages=

    2023

  42. [67]

    arXiv preprint arXiv:2501.17161 , year=

    Sft memorizes, rl generalizes: A comparative study of foundation model post-training , author=. arXiv preprint arXiv:2501.17161 , year=

    Pith/arXiv arXiv

  43. [68]

    arXiv preprint arXiv:2211.03540 , year=

    Measuring progress on scalable oversight for large language models , author=. arXiv preprint arXiv:2211.03540 , year=

    Pith/arXiv arXiv

  44. [69]

    Findings of the association for computational linguistics: ACL 2023 , pages=

    Discovering language model behaviors with model-written evaluations , author=. Findings of the association for computational linguistics: ACL 2023 , pages=

    2023

  45. [70]

    2025 , eprint=

    gpt-oss-120b & gpt-oss-20b Model Card , author=. 2025 , eprint=

    2025

  46. [71]

    2: Pushing the frontier of open large language models , author=

    Deepseek-v3. 2: Pushing the frontier of open large language models , author=. arXiv preprint arXiv:2512.02556 , year=

    Pith/arXiv arXiv

  47. [72]

    Sparse Latents Steer Retrieval-Augmented Generation

    Xin, Chunlei and Zhou, Shuheng and Zhu, Huijia and Wang, Weiqiang and Chen, Xuanang and Guan, Xinyan and Lu, Yaojie and Lin, Hongyu and Han, Xianpei and Sun, Le. Sparse Latents Steer Retrieval-Augmented Generation. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025. doi:10.18653/v1/2025.ac...

    doi:10.18653/v1/2025.acl-long.228 2025

  48. [73]

    arXiv preprint arXiv:2312.04333 , year=

    Is bigger and deeper always better? probing llama across scales and layers , author=. arXiv preprint arXiv:2312.04333 , year=

    Pith/arXiv arXiv

  49. [74]

    First Conference on Language Modeling , year=

    Gpqa: A graduate-level google-proof q&a benchmark , author=. First Conference on Language Modeling , year=