Pith. sign in

REVIEW 3 major objections 6 minor 7 cited by

Long-CoT reasoning is no trustworthiness upgrade, survey finds.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-05 10:34 UTC pith:P5SZ5446

load-bearing objection Useful map of reasoning trustworthiness literature, but the abstract's 'comparable or even greater' claim is undercut by the body's own benchmark-dependent evidence; revise, don't reject. the 3 major comments →

arxiv 2509.03871 v1 pith:P5SZ5446 submitted 2025-09-04 cs.CL cs.AIcs.CR

A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models

Yanbo Wang , Yongcan Yu , Jian Liang , Ran He This is my paper
classification cs.CL cs.AIcs.CR
keywords chain-of-thought reasoninglarge reasoning modelstrustworthinessjailbreak safetyhallucinationfaithfulnessrobustnessprivacy leakage
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This survey asks what long chain-of-thought reasoning does to trustworthiness across five axes: truthfulness, safety, robustness, fairness, and privacy. It argues that reasoning techniques genuinely help in some areas, such as hallucination detection and harmful-content screening, while current reasoning models simultaneously show comparable or larger vulnerabilities than ordinary chat models on safety, robustness, and privacy. The recurring object is the thinking trace itself: it can be attacked, it can be unfaithful, it leaks private attributes, and it can make harmful answers more detailed and persuasive. The survey's organizing conclusion is that reasoning capability is not an automatic trustworthiness upgrade; it shifts where failures occur and expands the attack surface.

Core claim

On the paper's own terms, the central claim is that while chain-of-thought prompting and reasoning models can improve truthfulness through hallucination mitigation and support safety defenses such as guardrail models, the same reasoning capability produces new failure modes: reasoning models hallucinate in simple non-reasoning tasks, generate more harmful content after jailbreak, are less robust to small input perturbations, preserve or amplify bias, and leak more private information through their thinking traces. The synthesis, stated in the abstract, is that current reasoning models 'often suffer from comparable or even greater vulnerabilities' in safety, robustness, and privacy, with the

What carries the argument

The key machinery is the survey's taxonomy of trustworthy reasoning: five dimensions (truthfulness, safety, robustness, fairness, privacy) crossed with two implementation paradigms (chain-of-thought prompting and end-to-end large reasoning models). Within that grid, the load-bearing object is the chain of thought itself—the generated intermediate reasoning steps that serve simultaneously as a faithfulness measurement target, a jailbreak surface, a backdoor trigger channel, a source of overthinking and underthinking, and a privacy side channel.

Load-bearing premise

The survey's cross-model conclusions assume that safety and robustness results measured on different benchmarks, languages, and attack templates can be combined into a single verdict; the paper itself reports cases where model rankings flip depending on the dataset.

What would settle it

Run a standardized safety battery on the same reasoning models across many languages and attack templates and check whether open-source reasoning models remain consistently less safe than their base counterparts; if the rank order reverses across datasets, the pooled conclusion about comparable or greater vulnerabilities weakens.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Safety auditing of deployed reasoning models should monitor the thinking trace, not just the final answer, because several cited studies find the reasoning content is less safe than the output.
  • Jailbreak defenses validated on chat models cannot be assumed to transfer to reasoning models; new attacks in this taxonomy specifically exploit reasoning traces, ciphers, or stepwise decomposition.
  • Faithfulness evaluation needs standardized protocols; current intervention metrics can confound model strength with apparent unfaithfulness.
  • Aligning reasoning models requires chain-of-thought-specific data and training, since safety reasoning must be taught rather than assumed to follow from general reasoning ability.
  • Privacy and interpretability conflict: visible thinking traces improve transparency but also enable attribute inference and unlearning-recovery attacks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Not claimed by the paper, but the dataset-dependence of safety rankings suggests that single-number attack-success-rate scores can mislead; reporting per-category and per-language results would be a cheap test of which vulnerabilities are stable.
  • Not claimed by the paper, but the finding that shortened reasoning improves harmlessness points to a concrete deployable intervention: force short reasoning at inference time and measure safety across languages.
  • Not claimed by the paper, but if the safety tax scales with reasoning-length incentives, separating rewards for correctness from rewards for safety during reinforcement learning may decouple the trade-off.
  • Not claimed by the paper, but if the multilingual vulnerabilities are a form of mismatched generalization, then training and evaluating safety in a few dominant languages systematically underestimates real-world deployment risk.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This survey reviews the trustworthiness of chain-of-thought prompting and large reasoning models (LRMs) across five dimensions: truthfulness, safety, robustness, fairness, and privacy. It organizes recent work into a taxonomy, summarizes methods and findings in each area, and identifies open problems. The central claim, stated in the abstract and Introduction, is that reasoning techniques can improve some aspects of trustworthiness (e.g., hallucination mitigation, harmful-content detection, robustness) while cutting-edge reasoning models themselves often exhibit comparable or greater vulnerabilities in safety, robustness, and privacy than non-reasoning models. The paper positions itself as the first comprehensive survey of trustworthy reasoning and provides a public repository of related papers.

Significance. If the synthesis is reliable, the survey would be a useful resource for the AI-safety community. Its main strengths are the breadth of covered topics (five trustworthiness dimensions), the explicit separation of early CoT techniques from end-to-end reasoning models, and the identification of several concrete open problems, such as the need for standardized faithfulness metrics and more fine-grained benchmarks. The paper also highlights a genuinely important tension: improved reasoning capability does not automatically translate into improved trustworthiness. The structured taxonomy and the GitHub resource add practical value. However, the central comparative claim—that reasoning models 'often suffer from comparable or even greater vulnerabilities'—is only partially supported by the evidence the paper itself presents, because the underlying evaluations are heterogeneous and sometimes contradictory. The survey also contains several citation and attribution errors that undermine reliability, especially for a resource whose purpose is to guide readers to the correct literature.

major comments (3)
  1. [Abstract and §4.1] The abstract claims that reasoning models 'often suffer from comparable or even greater vulnerabilities' in safety, robustness, and privacy. This comparative claim is not stable under the evidence reported in §4.1. The paper's own third bullet explicitly states that 'pairwise safety ranks between models depend on datasets': AirBench finds DeepSeek-R1 safer than DeepSeek-V3, while CNSafe finds the opposite with an average ASR margin of 21.7%, and WildGuard Jailbreak vs CNSafe_RT again reverse the picture. Similar benchmark-dependence appears in robustness (§5.2: reasoning models beat non-reasoning models on CodeCrash but lose on Math-RoB and other perturbation benchmarks). Since these evaluations use different attack templates, languages, and model versions, pooling them into a single 'comparable or even greater' verdict is unsupported. The abstract and conclusion should either be qualifi
  2. [Introduction and §2.2] Several references are mischaracterized or misassigned. In the Introduction, reference [12] is described as 'a related survey [12] provided valuable discussions on safety-related aspects,' but [12] is 'Safety Reasoning with Guidelines' (ICML 2025), an original research paper, not a survey. In §2.2, the sentence 'zero-shot-CoT [16]' attributes zero-shot CoT to Wei et al. [16]; the correct reference for zero-shot CoT is Kojima et al. [17]. In addition, 'few-shot-CoT [19]' in the same paragraph is wrong: [19] is the Chain-of-Scrutiny backdoor-detection paper, not the few-shot CoT paper. For a survey whose purpose is to organize and signpost the literature, these citation errors are material and should be corrected throughout.
  3. [§3.2.2 and §8] The faithfulness section reports directly contradictory conclusions (e.g., 'larger models are generally more faithful' [85,78] vs 'models with higher accuracy tend to exhibit lower faithfulness' [79,86]; 'length penalties may result in unfaithful responses' [81] vs 'unfaithful CoTs are usually longer' [82]). The survey notes these contradictions and calls for standardized metrics, which is appropriate. However, the synthesis does not identify which differences are likely due to evaluation methodology (e.g., intervention type, task difficulty, model family) versus genuine model properties. Since the paper explicitly lists 'standard measurements of faithfulness' as a future direction, it should at least organize the existing evidence around the methodological axes that are already discussed in §3.2.1. Without that, the 'factors that influence faithfulness' subsection is more a list of conf
minor comments (6)
  1. [§2.2] The citation typo 'zero-shot-CoT [16]' should read [17]; the nearby 'few-shot-CoT [19]' should read [16] (or the intended reference). Please check all citation numbers against the bibliography.
  2. [§3.2.1] Typo: 'Lakage-Adjusted Simulatability' should be 'Leakage-Adjusted Simulatability'; 'Paulet al.' should be 'Paul et al.'
  3. [§4.3.3] Typo: 'convolution neural network' should be 'convolutional neural network.'
  4. [Figure 2] The taxonomy figure is dense and the font is small. Consider grouping by sub-theme or providing an accompanying table to improve readability.
  5. [References] Reference formats are inconsistent: some entries include only arXiv identifiers and no publication venue, while others include venue names. For a survey, adding DOIs or stable URLs would improve usability.
  6. [§6] The fairness section is comparatively short and does not discuss the interaction of fairness with reasoning-model training (e.g., RLVR) beyond citing a few benchmarks. A brief discussion of open problems in this specific intersection would strengthen the survey.

Circularity Check

0 steps flagged

No material circularity: survey synthesis rests on external literature; self-citations are minor and not load-bearing.

full rationale

This is a survey paper rather than a derivation; its central claim—that reasoning techniques can improve some aspects of trustworthiness while reasoning models themselves show comparable or greater vulnerabilities—is a synthesis of many independent empirical studies, not the output of a fitted model or a self-referential equation. No step was found where a parameter is fit to a subset of data and then renamed a prediction, and no uniqueness theorem from the authors' own prior work is invoked to force a conclusion. The authors do cite several of their own papers ([260], [304], [305], [340], [344], [345]), but in all cases these are background or supporting citations within broader enumerations of related work; the central conclusions are supported by numerous third-party benchmarks and studies. The paper even explicitly acknowledges benchmark-dependent disagreements in Section 4.1 ('Pairwise safety ranks between models depend on datasets'), which is a limitation on comparability rather than a circularity. Under the hard rules requiring a specific quoted reduction for a circularity finding, no such reduction can be exhibited. The score of 2 reflects the presence of minor self-citations that are not load-bearing, not any actual circular reasoning.

Axiom & Free-Parameter Ledger

0 free parameters · 3 axioms · 0 invented entities

The survey is a literature synthesis, not a derivation, so it has no free parameters or invented entities. Its claims rest on the adequacy of its taxonomy, the reliability of the cited evaluations, and the comparability of heterogeneous benchmarks.

axioms (3)
  • domain assumption The five categories (truthfulness, safety, robustness, fairness, privacy) constitute an adequate decomposition of trustworthiness.
    Section 1 and Figure 2 organize the entire survey around these five dimensions; the paper does not justify exhaustiveness or test alternative taxonomies.
  • domain assumption The summarized findings in the cited papers are reliable and representative of the literature up to June 2025.
    The survey does not re-run evaluations; its synthesis about reasoning models having comparable or greater vulnerabilities depends on trusting the cited benchmarks.
  • domain assumption Conflicting evaluation results across benchmarks can be reconciled into a cross-model conclusion.
    Section 4.1 states that pairwise safety ranks depend on datasets (Airbench vs CNSafe), yet the paper still concludes that reasoning models are vulnerable; the reconciliation rule is not stated.

pith-pipeline@v1.4.0-alltime-deepseek-medium · 48819 in / 10647 out tokens · 95426 ms · 2026-08-05T10:34:41.229731+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models." pith.science (2026). https://pith.science/paper/P5SZ5446

@misc{pith2026250903871,
  author       = {Pith},
  title        = {Pith review of: A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P5SZ5446}},
  note         = {Machine review of arXiv:2509.03871}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

The development of Long-CoT reasoning has advanced LLM performance across various tasks, including language understanding, complex problem solving, and code generation. This paradigm enables models to generate intermediate reasoning steps, thereby improving both accuracy and interpretability. However, despite these advancements, a comprehensive understanding of how CoT-based reasoning affects the trustworthiness of language models remains underdeveloped. In this paper, we survey recent work on reasoning models and CoT techniques, focusing on five core dimensions of trustworthy reasoning: truthfulness, safety, robustness, fairness, and privacy. For each aspect, we provide a clear and structured overview of recent studies in chronological order, along with detailed analyses of their methodologies, findings, and limitations. Future research directions are also appended at the end for reference and discussion. Overall, while reasoning techniques hold promise for enhancing model trustworthiness through hallucination mitigation, harmful content detection, and robustness improvement, cutting-edge reasoning models themselves often suffer from comparable or even greater vulnerabilities in safety, robustness, and privacy. By synthesizing these insights, we hope this work serves as a valuable and timely resource for the AI safety community to stay informed on the latest progress in reasoning trustworthiness. A full list of related papers can be found at \href{https://github.com/ybwang119/Awesome-reasoning-safety}{https://github.com/ybwang119/Awesome-reasoning-safety}.

Figures

Figures reproduced from arXiv: 2509.03871 by Jian Liang, Ran He, Yanbo Wang, Yongcan Yu.

Figure 1
Figure 1. Figure 1: Illustration of typical CoT prompting. Few-shot-CoT uses several examples with the reasoning process to elicit CoT, and [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Taxonomy of trustworthiness in reasoning with large language models. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: An example of H-CoT jailbreak prompt, which is from “DukeCEICenter/Malicious_Educator_hcot_o1” dataset [117]. [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Examples of prompts for CoT data synthesis. Minor modifications are executed for better readability. [PITH_FULL_IMAGE:figures/full_fig_p017_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Where Do CoT Training Gains Land in LLM based Agents?

    cs.AI 2026-06 unverdicted novelty 6.0

    CoT training in LLM agents improves prompt-action quality more than the advantage of generated reasoning, and selectively masking action supervision improves out-of-domain generalization.

  2. Auditing Reasoning-Trace Memorization Claims after Unlearning with Head-Conditioned Canaries

    cs.LG 2026-05 conditional novelty 6.0

    Swapping the reasoning trace prefill on unlearned weights can replicate or reverse the parser-split bypass gap, showing that the gap alone does not identify or rule out weight-level memorization.

  3. Pause or Fabricate? Training Language Models for Grounded Reasoning

    cs.CL 2026-04 conditional novelty 6.0

    GRIL uses stage-specific RL rewards to train LLMs to detect missing premises, pause proactively, and resume grounded reasoning after clarification, yielding up to 45% better premise detection and 30% higher task succe...

  4. From $P(y|x)$ to $P(y)$: Investigating Reinforcement Learning in Pre-train Space

    cs.LG 2026-04 unverdicted novelty 6.0

    PreRL applies reward-driven updates to P(y) in pre-train space, uses Negative Sample Reinforcement to prune bad reasoning paths and boost reflection, and combines with standard RL in Dual Space RL to outperform baseli...

  5. Stop Tracking Me! Proactive Defense Against Attribute Inference Attack in LLMs

    cs.CR 2026-02 conditional novelty 6.0

    TRACE-RPS drops LLM attribute inference accuracy from around 50% to below 5% via fine-grained anonymization plus a two-stage rejection optimization.

  6. Be Faithful When Response: Returning Fluent and Grounded Answers for Vision-Language Models Reinforcement Learning

    cs.AI 2026-06 unverdicted novelty 5.0

    Faithful Warm-Start pre-training on causally consistent vision-language samples improves accuracy, stabilizes RL, and reduces unsupported reasoning in VLMs.

  7. Strengthening Human-Centric Chain-of-Thought Reasoning Integrity in LLMs via a Structured Prompt Framework

    cs.CR 2026-04 unverdicted novelty 5.0

    A 16-factor structured prompt framework strengthens CoT reasoning in LLMs for security analysis, yielding up to 40% reasoning gains in smaller models and stable accuracy improvements validated by human raters with Coh...

Reference graph

Works this paper leans on

299 extracted references · 6 canonical work pages · cited by 7 Pith papers · 5 internal anchors

  1. [1]

    Safechain: Safety of language models with long chain-of-thought reasoning capabilities

    Fengqing Jiang, Zhangchen Xu, Yuetai Li, Luyao Niu, Zhen Xiang, Bo Li, Bill Yuchen Lin, and Radha Poovendran. Safechain: Safety of language models with long chain-of-thought reasoning capabilities. arXiv preprint arXiv:2502.12025, 2025

    Pith/arXiv arXiv 2025

  2. [2]

    Does Chain-of-Thought Reasoning Really Reduce Harmfulness from Jailbreaking? arXiv preprint arXiv:2505.17650, 2025

    Chengda Lu, Xiaoyu Fan, Yu Huang, Rongwu Xu, Jijie Li, and Wei Xu. Does Chain-of-Thought Reasoning Really Reduce Harmfulness from Jailbreaking? arXiv preprint arXiv:2505.17650, 2025

    Pith/arXiv arXiv 2025

  3. [3]

    Towards understanding the safety boundaries of deepseek models: Evaluation and findings

    Zonghao Ying, Guangyi Zheng, Yongxin Huang, Deyue Zhang, Wenxin Zhang, Quanchen Zou, Aishan Liu, Xianglong Liu, and Dacheng Tao. Towards understanding the safety boundaries of deepseek models: Evaluation and findings. arXiv preprint arXiv:2503.15092, 2025

    Pith/arXiv arXiv 2025

  4. [4]

    A comprehensive survey in llm (-agent) full stack safety: Data, training and deployment

    Kun Wang, Guibin Zhang, Zhenhong Zhou, Jiahao Wu, Miao Yu, Shiqian Zhao, Chenlong Yin, Jinhu Fu, Yibo Yan, Hanjun Luo, et al. A comprehensive survey in llm (-agent) full stack safety: Data, training and deployment. arXiv preprint arXiv:2504.15585, 2025

    Pith/arXiv arXiv 2025

  5. [5]

    Attacks, defenses and evaluations for llm conversation safety: A survey

    Zhichen Dong, Zhanhui Zhou, Chao Yang, Jing Shao, and Yu Qiao. Attacks, defenses and evaluations for llm conversation safety: A survey. In Proc. NAACL, 2024

    2024

  6. [6]

    Large language model safety: A holistic survey

    Dan Shi, Tianhao Shen, Yufei Huang, Zhigen Li, Yongqi Leng, Renren Jin, Chuang Liu, Xinwei Wu, Zishan Guo, Linhao Yu, et al. Large language model safety: A holistic survey. arXiv preprint arXiv:2412.17686, 2024

    Pith/arXiv arXiv 2024

  7. [7]

    Towards reasoning era: A survey of long chain-of-thought for reasoning large language models

    Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wangxiang Che. Towards reasoning era: A survey of long chain-of-thought for reasoning large language models. arXiv preprint arXiv:2503.09567, 2025

    Pith/arXiv arXiv 2025

  8. [8]

    Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Models

    Fengli Xu, Qianyue Hao, Zefang Zong, Jingwei Wang, Yunke Zhang, Jingyi Wang, Xiaochong Lan, Jiahui Gong, Tianjian Ouyang, Fanjin Meng, et al. Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Models. arXiv preprint arXiv:2501.09686, 2025

    Pith/arXiv arXiv 2025

  9. [9]

    A survey of efficient reasoning for large reasoning models: Language, multimodality, and beyond

    Xiaoye Qu, Yafu Li, Zhaochen Su, Weigao Sun, Jianhao Yan, Dongrui Liu, Ganqu Cui, Daizong Liu, Shuxian Liang, Junxian He, et al. A survey of efficient reasoning for large reasoning models: Language, multimodality, and beyond. arXiv preprint arXiv:2503.21614, 2025

    arXiv 2025

  10. [10]

    Stop overthinking: A survey on efficient reasoning for large language models

    Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Hanjie Chen, et al. Stop overthinking: A survey on efficient reasoning for large language models. arXiv preprint arXiv:2503.16419, 2025

    Pith/arXiv arXiv 2025

  11. [11]

    Efficient reasoning models: A survey

    Sicheng Feng, Gongfan Fang, Xinyin Ma, and Xinchao Wang. Efficient reasoning models: A survey. arXiv preprint arXiv:2504.10903, 2025

    arXiv 2025

  12. [12]

    Safety Reasoning with Guidelines

    Haoyu Wang, Zeyu Qin, Li Shen, Xueqian Wang, Dacheng Tao, and Minhao Cheng. Safety Reasoning with Guidelines. In Proc. ICML, 2025

    2025

  13. [13]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    Pith/arXiv arXiv 2023

  14. [14]

    The Claude 3 Model Family: Opus, Sonnet, Haiku, 2024

    Anthropic. The Claude 3 Model Family: Opus, Sonnet, Haiku, 2024

    2024

  15. [15]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    Pith/arXiv arXiv 2025

  16. [16]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. In Proc. NeurIPS, 2022

    2022

  17. [17]

    Large language models are zero-shot reasoners

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. In Proc. NeurIPS, 2022

    2022

  18. [18]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. In Proc. NeurIPS, 2020

    2020

  19. [19]

    Chain-of-scrutiny: Detecting backdoor attacks for large language models

    Xi Li, Yusen Zhang, Renze Lou, Chen Wu, and Jiaqi Wang. Chain-of-scrutiny: Detecting backdoor attacks for large language models. arXiv preprint arXiv:2406.05948, 2024

    arXiv 2024

  20. [20]

    Openai o1 system card

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card. arXiv preprint arXiv:2412.16720, 2024

    Pith/arXiv arXiv 2024

  21. [21]

    Openr: An open source framework for advanced reasoning with large language models

    Jun Wang, Meng Fang, Ziyu Wan, Muning Wen, Jiachen Zhu, Anjie Liu, Ziqin Gong, Yan Song, Lei Chen, Lionel M Ni, et al. Openr: An open source framework for advanced reasoning with large language models. arXiv preprint arXiv:2410.09671, 2024

    Pith/arXiv arXiv 2024

  22. [22]

    O1 Replication Journey: A Strategic Progress Report–Part 1

    Yiwei Qin, Xuefeng Li, Haoyang Zou, Yixiu Liu, Shijie Xia, Zhen Huang, Yixin Ye, Weizhe Yuan, Hector Liu, Yuanzhi Li, et al. O1 Replication Journey: A Strategic Progress Report–Part 1. arXiv preprint arXiv:2410.18982, 2024

    Pith/arXiv arXiv 2024

  23. [23]

    O1 Replication Journey–Part 2: Surpassing O1-preview through Simple Distillation, Big Progress or Bitter Lesson? arXiv preprint arXiv:2411.16489, 2024

    Zhen Huang, Haoyang Zou, Xuefeng Li, Yixiu Liu, Yuxiang Zheng, Ethan Chern, Shijie Xia, Yiwei Qin, Weizhe Yuan, and Pengfei Liu. O1 Replication Journey–Part 2: Surpassing O1-preview through Simple Distillation, Big Progress or Bitter Lesson? arXiv preprint arXiv:2411.16489, 2024. 24 References A Preprint

    Pith/arXiv arXiv 2024

  24. [24]

    O1 Replication Journey–Part 3: Inference-time Scaling for Medical Reasoning

    Zhongzhen Huang, Gui Geng, Shengyi Hua, Zhen Huang, Haoyang Zou, Shaoting Zhang, Pengfei Liu, and Xiaofan Zhang. O1 Replication Journey–Part 3: Inference-time Scaling for Medical Reasoning. arXiv preprint arXiv:2501.06458, 2025

    Pith/arXiv arXiv 2025

  25. [25]

    Llama-berry: Pairwise optimization for o1-like olympiad-level mathematical reasoning

    Di Zhang, Jianbo Wu, Jingdi Lei, Tong Che, Jiatong Li, Tong Xie, Xiaoshui Huang, Shufei Zhang, Marco Pavone, Yuqiang Li, et al. Llama-berry: Pairwise optimization for o1-like olympiad-level mathematical reasoning. arXiv preprint arXiv:2410.02884, 2024

    Pith/arXiv arXiv 2024

  26. [26]

    A survey of monte carlo tree search methods

    Cameron B Browne, Edward Powley, Daniel Whitehouse, Simon M Lucas, Peter I Cowling, Philipp Rohlfshagen, Stephen Tavener, Diego Perez, Spyridon Samothrakis, and Simon Colton. A survey of monte carlo tree search methods. IEEE Transactions on Computational Intelligence and AI in games, pages 1–43, 2012

    2012

  27. [27]

    Training Verifiers to Solve Math Word Problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training Verifiers to Solve Math Word Problems. arXiv preprint arXiv:2110.14168, 2021

    Pith/arXiv arXiv 2021

  28. [28]

    Measuring mathematical problem solving with the math dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. In Proc. NeurIPS D&B Track, 2021

    2021

  29. [29]

    MARIO: MAth Reasoning with code Interpreter Output–A Reproducible Pipeline

    Minpeng Liao, Wei Luo, Chengxi Li, Jing Wu, and Kai Fan. MARIO: MAth Reasoning with code Interpreter Output–A Reproducible Pipeline. In Findings of Proc. ACL, page 905–924, 2024

    2024

  30. [30]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In Proc. NeurIPS, 2023

    2023

  31. [31]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    Pith/arXiv arXiv 2017

  32. [32]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Pith/arXiv arXiv 2024

  33. [33]

    Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach

    Jonas Geiping, Sean McLeish, Neel Jain, John Kirchenbauer, Siddharth Singh, Brian R Bartoldson, Bhavya Kailkhura, Abhinav Bhatele, and Tom Goldstein. Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach. arXiv preprint arXiv:2502.05171, 2025

    Pith/arXiv arXiv 2025

  34. [34]

    Training large language models to reason in a continuous latent space

    Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in a continuous latent space. In ICLR Workshop on LLM Reason and Plan, 2024

    2024

  35. [35]

    Deepseek-v3 technical report

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024

    Pith/arXiv arXiv 2024

  36. [36]

    Qwen2.5 technical report

    A Yang Qwen, Baosong Yang, B Zhang, B Hui, B Zheng, B Yu, Chengpeng Li, D Liu, F Huang, H Wei, et al. Qwen2.5 technical report. arXiv preprint, 2024

    2024

  37. [37]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv e-prints, pages arXiv–2407, 2024

    2024

  38. [38]

    Solving math word problems with process-and outcome-based feedback

    Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. Solving math word problems with process-and outcome-based feedback. arXiv preprint arXiv:2211.14275, 2022

    Pith/arXiv arXiv 2022

  39. [39]

    Star: Self-taught reasoner bootstrapping reasoning with reasoning

    Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D Goodman. Star: Self-taught reasoner bootstrapping reasoning with reasoning. In Proc. NeurIPS, volume 1126, 2024

    2024

  40. [40]

    Let’s verify step by step

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. In Proc. ICLR, 2023

    2023

  41. [41]

    T \" ulu 3: Pushing frontiers in open language model post-training

    Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, et al. T \" ulu 3: Pushing frontiers in open language model post-training. arXiv preprint arXiv:2411.15124, 2024

    Pith/arXiv arXiv 2024

  42. [42]

    Reinforcement Learning with Verifiable Rewards: GRPO’s Effective Loss, Dynamics, and Success Amplifi- cation

    Youssef Mroueh. Reinforcement Learning with Verifiable Rewards: GRPO’s Effective Loss, Dynamics, and Success Amplifi- cation. arXiv preprint arXiv:2503.06639, 2025

    arXiv 2025

  43. [43]

    Perception, reason, think, and plan: A survey on large multimodal reasoning models

    Yunxin Li, Zhenyu Liu, Zitao Li, Xuanyu Zhang, Zhenran Xu, Xinyu Chen, Haoyuan Shi, Shenyuan Jiang, Xintong Wang, Jifang Wang, et al. Perception, reason, think, and plan: A survey on large multimodal reasoning models. arXiv preprint arXiv:2505.04921, 2025

    Pith/arXiv arXiv 2025

  44. [44]

    Multimodal chain-of- thought reasoning: A comprehensive survey

    Yaoting Wang, Shengqiong Wu, Yuecheng Zhang, Shuicheng Yan, Ziwei Liu, Jiebo Luo, and Hao Fei. Multimodal chain-of- thought reasoning: A comprehensive survey. arXiv preprint arXiv:2503.12605, 2025

    Pith/arXiv arXiv 2025

  45. [45]

    Multimodal chain-of-thought reasoning in language models

    Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. Multimodal chain-of-thought reasoning in language models. Transactions on Machine Learning Research, 2024

    2024

  46. [46]

    Video-of-thought: Step-by-step video reasoning from perception to cognition

    Hao Fei, Shengqiong Wu, Wei Ji, Hanwang Zhang, Meishan Zhang, Mong-Li Lee, and Wynne Hsu. Video-of-thought: Step-by-step video reasoning from perception to cognition. In Proc. ICML, 2024

    2024

  47. [47]

    Thinking before looking: Improving multimodal llm reasoning via mitigating visual hallucination

    Haojie Zheng, Tianyang Xu, Hanchi Sun, Shu Pu, Ruoxi Chen, and Lichao Sun. Thinking before looking: Improving multimodal llm reasoning via mitigating visual hallucination. arXiv preprint arXiv:2411.12591, 2024. 25 References A Preprint

    Pith/arXiv arXiv 2024

  48. [48]

    Llava-o1: Let vision language models reason step-by-step

    Guowei Xu, Peng Jin, Li Hao, Yibing Song, Lichao Sun, and Li Yuan. Llava-o1: Let vision language models reason step-by-step. arXiv preprint arXiv:2411.10440, 2024

    Pith/arXiv arXiv 2024

  49. [49]

    Llamav-o1: Rethinking step-by-step visual reasoning in llms

    Omkar Thawakar, Dinura Dissanayake, Ketan More, Ritesh Thawkar, Ahmed Heakl, Noor Ahsan, Yuhao Li, Mohammed Zumri, Jean Lahoud, Rao Muhammad Anwer, et al. Llamav-o1: Rethinking step-by-step visual reasoning in llms. arXiv preprint arXiv:2501.06186, 2025

    Pith/arXiv arXiv 2025

  50. [50]

    RedStar: Does Scaling Long-CoT Data Unlock Better Slow-Reasoning Systems? arXiv preprint arXiv:2501.11284, 2025

    Haotian Xu, Xing Wu, Weinong Wang, Zhongzhi Li, Da Zheng, Boyuan Chen, Yi Hu, Shijia Kang, Jiaming Ji, Yingying Zhang, et al. RedStar: Does Scaling Long-CoT Data Unlock Better Slow-Reasoning Systems? arXiv preprint arXiv:2501.11284, 2025

    Pith/arXiv arXiv 2025

  51. [51]

    Mulberry: Empowering mllm with o1-like reasoning and reflection via collective monte carlo tree search

    Huanjin Yao, Jiaxing Huang, Wenhao Wu, Jingyi Zhang, Yibo Wang, Shunyu Liu, Yingjie Wang, Yuxin Song, Haocheng Feng, Li Shen, et al. Mulberry: Empowering mllm with o1-like reasoning and reflection via collective monte carlo tree search. arXiv preprint arXiv:2412.18319, 2024

    Pith/arXiv arXiv 2024

  52. [52]

    Improve vision language model chain-of-thought reasoning

    Ruohong Zhang, Bowen Zhang, Yanghao Li, Haotian Zhang, Zhiqing Sun, Zhe Gan, Yinfei Yang, Ruoming Pang, and Yiming Yang. Improve vision language model chain-of-thought reasoning. arXiv preprint arXiv:2410.16198, 2024

    Pith/arXiv arXiv 2024

  53. [53]

    Insight-v: Exploring long-chain visual reasoning with multimodal large language models

    Yuhao Dong, Zuyan Liu, Hai-Long Sun, Jingkang Yang, Winston Hu, Yongming Rao, and Ziwei Liu. Insight-v: Exploring long-chain visual reasoning with multimodal large language models. In Proc. CVPR, pages 9062–9072, 2025

    2025

  54. [54]

    Mammoth-vl: Eliciting multimodal reasoning with instruction tuning at scale

    Jarvis Guo, Tuney Zheng, Yuelin Bai, Bo Li, Yubo Wang, King Zhu, Yizhi Li, Graham Neubig, Wenhu Chen, and Xiang Yue. Mammoth-vl: Eliciting multimodal reasoning with instruction tuning at scale. arXiv preprint arXiv:2412.05237, 2024

    Pith/arXiv arXiv 2024

  55. [55]

    Mm-verify: Enhancing multimodal reasoning with chain-of-thought verification

    Linzhuang Sun, Hao Liang, Jingxuan Wei, Bihui Yu, Tianpeng Li, Fan Yang, Zenan Zhou, and Wentao Zhang. Mm-verify: Enhancing multimodal reasoning with chain-of-thought verification. arXiv preprint arXiv:2502.13383, 2025

    Pith/arXiv arXiv 2025

  56. [56]

    Think More, Hallucinate Less: Mitigating Hallucinations via Dual Process of Fast and Slow Thinking

    Xiaoxue Cheng, Junyi Li, Wayne Xin Zhao, and Ji-Rong Wen. Think More, Hallucinate Less: Mitigating Hallucinations via Dual Process of Fast and Slow Thinking. arXiv preprint arXiv:2501.01306, 2025

    Pith/arXiv arXiv 2025

  57. [57]

    HalluMeasure: Fine-grained hallucination measurement using chain-of-thought reasoning

    Shayan Ali Akbar, Md Mosharaf Hossain, Tess Wood, Si-Chi Chin, Erica M Salinas, Victor Alvarez, and Erwin Cornejo. HalluMeasure: Fine-grained hallucination measurement using chain-of-thought reasoning. In Proc. EMNLP, pages 15020– 15037, 2024

    2024

  58. [58]

    CLATTER: Comprehensive Entailment Reasoning for Hallucination Detection

    Ron Eliav, Arie Cattan, Eran Hirsch, Shahaf Bassan, Elias Stengel-Eskin, Mohit Bansal, and Ido Dagan. CLATTER: Comprehensive Entailment Reasoning for Hallucination Detection. arXiv preprint arXiv:2506.05243, 2025

    Pith/arXiv arXiv 2025

  59. [59]

    Order Matters in Hallucination: Reasoning Order as Benchmark and Reflexive Prompting for Large-Language- Models

    Zikai Xie. Order Matters in Hallucination: Reasoning Order as Benchmark and Reflexive Prompting for Large-Language- Models. arXiv preprint arXiv:2408.05093, 2024

    Pith/arXiv arXiv 2024

  60. [60]

    Grounded chain-of- thought for multimodal large language models

    Qiong Wu, Xiangcong Yang, Yiyi Zhou, Chenxin Fang, Baiyang Song, Xiaoshuai Sun, and Rongrong Ji. Grounded chain-of- thought for multimodal large language models. arXiv preprint arXiv:2503.12799, 2025

    Pith/arXiv arXiv 2025

  61. [61]

    CoMT: Chain-of- Medical-Thought Reduces Hallucination in Medical Report Generation

    Yue Jiang, Jiawei Chen, Dingkang Yang, Mingcheng Li, Shunli Wang, Tong Wu, Ke Li, and Lihua Zhang. CoMT: Chain-of- Medical-Thought Reduces Hallucination in Medical Report Generation. In Proc. ICASSP, 2025

    2025

  62. [62]

    MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM

    Bowen Dong, Minheng Ni, Zitong Huang, Guanglei Yang, Wangmeng Zuo, and Lei Zhang. MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM. arXiv preprint arXiv:2505.24238, 2025

    Pith/arXiv arXiv 2025

  63. [63]

    The Hallucination Tax of Reinforcement Finetuning.arXiv preprint arXiv:2505.13988, 2025

    Linxin Song, Taiwei Shi, and Jieyu Zhao. The Hallucination Tax of Reinforcement Finetuning.arXiv preprint arXiv:2505.13988, 2025

    Pith/arXiv arXiv 2025

  64. [64]

    More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models

    Chengzhi Liu, Zhongxing Xu, Qingyue Wei, Juncheng Wu, James Zou, Xin Eric Wang, Yuyin Zhou, and Sheng Liu. More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models. arXiv preprint arXiv:2505.21523, 2025

    Pith/arXiv arXiv 2025

  65. [65]

    Are Reasoning Models More Prone to Hallucination? arXiv preprint arXiv:2505.23646, 2025

    Zijun Yao, Yantao Liu, Yanxu Chen, Jianhui Chen, Junfeng Fang, Lei Hou, Juanzi Li, and Tat-Seng Chua. Are Reasoning Models More Prone to Hallucination? arXiv preprint arXiv:2505.23646, 2025

    Pith/arXiv arXiv 2025

  66. [66]

    AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions

    Polina Kirichenko, Mark Ibrahim, Kamalika Chaudhuri, and Samuel J Bell. AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions. arXiv preprint arXiv:2506.09038, 2025

    Pith/arXiv arXiv 2025

  67. [67]

    Auditing Meta-Cognitive Hallucinations in Reasoning Large Language Models

    Haolang Lu, Yilian Liu, Jingxin Xu, Guoshun Nan, Yuanlong Yu, Zhican Chen, and Kun Wang. Auditing Meta-Cognitive Hallucinations in Reasoning Large Language Models. arXiv preprint arXiv:2505.13143, 2025

    arXiv 2025

  68. [68]

    The Hallucination Dilemma: Factuality-Aware Reinforcement Learning for Large Reasoning Models

    Junyi Li and Hwee Tou Ng. The Hallucination Dilemma: Factuality-Aware Reinforcement Learning for Large Reasoning Models. arXiv preprint arXiv:2505.24630, 2025

    arXiv 2025

  69. [69]

    Analyzing Logical Fallacies in Large Language Models: A Study on Hallucination in Mathematical Reasoning

    Dang Hoang Anh, Vu Tran, and Le Minh Nguyen. Analyzing Logical Fallacies in Large Language Models: A Study on Hallucination in Mathematical Reasoning. In JSAI International Symposium on Artificial Intelligence, pages 179–195. Springer, 2025

    2025

  70. [70]

    Detection and Mitigation of Hallucination in Large Reasoning Models: A Mechanistic Perspective

    Zhongxiang Sun, Qipeng Wang, Haoyu Wang, Xiao Zhang, and Jun Xu. Detection and Mitigation of Hallucination in Large Reasoning Models: A Mechanistic Perspective. arXiv preprint arXiv:2505.12886, 2025

    Pith/arXiv arXiv 2025

  71. [71]

    Mathematical Proof as a Litmus Test: Revealing Failure Modes of Advanced Large Reasoning Models

    Dadi Guo, Jiayu Liu, Zhiyuan Fan, Zhitao He, Haoran Li, Yumeng Wang, et al. Mathematical Proof as a Litmus Test: Revealing Failure Modes of Advanced Large Reasoning Models. arXiv preprint arXiv:2506.17114, 2025

    arXiv 2025

  72. [72]

    Fine-grained Hallucination Detection and Mitigation in Language Model Mathematical Reasoning

    Ruosen Li, Ziming Luo, and Xinya Du. Fine-grained Hallucination Detection and Mitigation in Language Model Mathematical Reasoning. arXiv preprint arXiv:2410.06304, 2024. 26 References A Preprint

    arXiv 2024

  73. [73]

    Reasoning Models Know When They’re Right: Probing Hidden States for Self-Verification

    Anqi Zhang, Yulin Chen, Jane Pan, Chen Zhao, Aurojit Panda, Jinyang Li, and He He. Reasoning Models Know When They’re Right: Probing Hidden States for Self-Verification. arXiv preprint arXiv:2504.05419, 2025

    Pith/arXiv arXiv 2025

  74. [74]

    Joint Evaluation of Answer and Reasoning Consistency for Hallucination Detection in Large Reasoning Models

    Changyue Wang, Weihang Su, Qingyao Ai, and Yiqun Liu. Joint Evaluation of Answer and Reasoning Consistency for Hallucination Detection in Large Reasoning Models. arXiv preprint arXiv:2506.04832, 2025

    arXiv 2025

  75. [75]

    Measuring faithfulness in chain-of-thought reasoning

    Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Hernandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, et al. Measuring faithfulness in chain-of-thought reasoning. arXiv preprint arXiv:2307.13702, 2023

    Pith/arXiv arXiv 2023

  76. [76]

    Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting

    Miles Turpin, Julian Michael, Ethan Perez, and Samuel Bowman. Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting. In Proc. NeurIPS, 2023

    2023

  77. [77]

    Measuring faithfulness of chains of thought by unlearning reasoning steps

    Martin Tutek, Fateme Hashemi Chaleshtori, Ana Marasović, and Yonatan Belinkov. Measuring faithfulness of chains of thought by unlearning reasoning steps. arXiv preprint arXiv:2502.14829, 2025

    arXiv 2025

  78. [78]

    Measuring the Faithfulness of Thinking Drafts in Large Reasoning Models

    Zidi Xiong, Chen Shan, Zhenting Qi, and Himabindu Lakkaraju. Measuring the Faithfulness of Thinking Drafts in Large Reasoning Models. arXiv preprint arXiv:2505.13774, 2025

    Pith/arXiv arXiv 2025

  79. [79]

    Chain-of-Thought Unfaithfulness as Disguised Accuracy

    Oliver Bentham, Nathan Stringham, and Ana Marasovic. Chain-of-Thought Unfaithfulness as Disguised Accuracy. Transac- tions on Machine Learning Research, 2024

    2024

  80. [80]

    Chain-of- thought reasoning in the wild is not always faithful

    Iván Arcuschin, Jett Janiak, Robert Krzyzanowski, Senthooran Rajamanoharan, Neel Nanda, and Arthur Conmy. Chain-of- thought reasoning in the wild is not always faithful. arXiv preprint arXiv:2503.08679, 2025

    Pith/arXiv arXiv 2025

Showing first 80 references.