Towards Understanding the Cognitive Habits of Large Reasoning Models

Chao Zhang; Chuanrui Hu; Han Qiu; Jianshuo Dong; Yujia Fu

REVIEW 4 major objections 5 minor 62 references

Towards Understanding the Cognitive Habits of Large Reasoning Models

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Reasoning models show stable, human-like cognitive habits in their chains of thought.

desk verdict CogTest is a genuinely new benchmark idea and the model comparisons are informative, but the unvalidated LLM judge and the assumed CoT faithfulness make the cognitive-habit claims provisional until the measurement is checked against humans. read the letter →

arxiv 2506.21571 v3 pith:OT6LZKGK submitted 2025-06-13 cs.CL cs.AIcs.CR

Jianshuo Dong , Yujia Fu , Chuanrui Hu , Chao Zhang , Han Qiu This is my paper

classification cs.CLcs.AIcs.CR

keywords cognitivehabitslargereasoningmodelschainofthoughtmetacognitionLLMsafetyCoTevaluationMindbenchmark

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Large reasoning models (LRMs) generate an explicit chain of thought before answering; this paper asks whether the recurring meta-cognitive statements in those chains are actually stable cognitive habits of the kind that contribute to successful human problem-solving. To test this, the authors build CogTest, a benchmark of 16 cognitive habits, each probed with 25 tasks designed to elicit them without mentioning them. They find that LRMs display these habits persistently across tasks and deploy different habits for different kinds of problems, while ordinary LLMs asked to "think first" do not. They also find that habit use tracks model family and training method, and that some habits—especially Taking Responsible Risks—appear disproportionately in the chains of thought that lead to harmful answers. If right, the paper gives a new way to characterize, compare, and monitor reasoning models from the inside.

What carries the argument

The load-bearing instrument is CogTest, a benchmark pairing each of 16 cognitive habits with 25 task prompts that induce the habit without naming it, plus an evidence-first habit extraction method: a judge model (GPT-4.1-mini) must quote a verbatim meta-cognitive sentence from the CoT before it may mark a habit as present. Habit profiles—the frequency with which each of the 16 habits fires across the benchmark—are then clustered across models with agglomerative clustering. The design blocks sycophantic mimicry by never telling the model which habit is being tested, and blocks judge hallucination by requiring quoted evidence; the same pipeline is then transferred to 200 harmful-query CoTs to compare habit presence in harmful versus harmless responses.

What would settle it

Take a model that scores high on Taking Responsible Risks in harmful CoTs, re-run the same safety queries with a system prompt that forbids the model from discussing risks or consequences in its reasoning, and check whether the harmful-response rate drops; if it does not, the habit is a correlate, not a cause.

Watch

Extended reading notes

Core claim

The central discovery is that the "Wait, did I miss anything?" style of CoT text is not incidental noise: it is evidence that LRMs possess the 16 cognitive habits codified in the Habits of Mind framework, and that different models possess them to different degrees. The paper claims that LRMs, unlike non-reasoning LLMs, do not merely imitate these habits superficially but adaptively select them according to the task (metacognition and precision-seeking on math, empathy on interpersonal requests), and that the resulting habit profiles are stable enough to cluster models into families. It further claims that these profiles have practical teeth: across five LRMs tested on harmful queries, habits such as Taking Responsible Risks, Listening with Understanding and Empathy, and Creating, Imagining, and Innovating appear far more often in CoTs that end in harmful answers than in harmless ones.

Load-bearing premise

The whole measurement assumes that the meta-cognitive sentences written in a model's chain of thought truthfully reveal the cognitive habits that produced its answer, and that the judge model reads those sentences without bias; if CoTs are performative or the judge is biased, the habit profiles and safety correlations would be artifacts of wording rather than properties of the model.

Editorial extensions

If this is right

If CoT habits are stable and task-adaptive, then a model's internal reasoning can be characterized by a compact 16-dimensional habit profile rather than by per-task performance.
Because non-reasoning LLMs fail to sustain extended CoTs and show weak metacognition habits, the results make a concrete case that reasoning-oriented reinforcement learning is what instills these problem-solving habits.
Family-level habit similarity (e.g., Qwen-3 and DeepSeek-R1) implies that training methodology or training data shapes cognitive style, so habit profiles can serve as evidence in audits of distillation or data contamination.
The safety correlation suggests habit monitoring of CoTs could flag harmful outputs before the final answer is produced, extending the monitoring agenda the paper adopts from Baker et al.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Going beyond the paper, a direct test of its causal reading would be to ask a model to suppress meta-cognitive statements while answering the same safety queries; if harm rates do not change, the habits are predictive markers rather than drivers.
The benchmark's reliance on verbalized meta-thinking implies silent habits are invisible to it; one could extend it by comparing habit profiles of the same model under prompts that encourage versus discourage explicit self-talk, to see whether the profiles are intrinsic or induced.
If habit profiles are treated as a fingerprint, they could be used to screen third-party fine-tunes for hidden alignment changes without needing access to weights.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know two things. First, the CogTest benchmark is a real contribution: applying Costa and Kallick's Habits of Mind to LRM CoTs is a fresh framing, the task design shows care (spontaneity constraints, no habit cues, human-verified tasks), and the paper ships code. Second, the empirical claims rest on an unvalidated measurement. The entire pipeline depends on GPT-4.1-mini classifying 16 habits from CoT text, but there is no human agreement study, no precision/recall check, and no ablation showing the judge isn't just latching onto surface phrases like “Wait” or “Let me think.” The paper cites both Chen et al. (2025) and Baker et al. (2025) on CoT unfaithfulness, then proceeds as if CoTs are transparent. Section 7 acknowledges missing implicit habits but never addresses the possibility that explicit statements are unreliable. That is a load-bearing gap, not a minor one.

What the paper does well: the 16-model comparison is comprehensive, the clustering results (Qwen-3 and DeepSeek-R1 resemblance, Claude-3.7-sonnet distinct) look visually robust, and the safety extension is an interesting seed even if post hoc. The evidence-first extraction prompt is a reasonable mitigation against hallucination, and the authors are appropriately cautious about inferring training-data contamination. The benchmarking itself is reusable and scalable.

Where it is soft: (1) the judge validation gap described above; (2) the safety analysis selects models and habits after seeing the data, reports raw percentage differences without error bars or significance tests, and uses small counts (e.g., DeepSeek-R1 with 30 harmful cases; one habit at 80.8% vs 3.3% is striking but needs a test); (3) the occurrence threshold in the safety analysis is a free parameter with no robustness check; (4) the clustering is presented without stability analysis. None of these are fatal in themselves, but together they mean the central “adaptive deployment” conclusion is stronger than the measurement currently supports.

Who should read this: anyone building behavioral benchmarks for reasoning models, and people monitoring LRM safety. It deserves a serious referee. My recommendation: send it to review, but require human annotation of a sample of judge classifications, a CoT-length/Style control, and a more careful safety analysis before accepting the strong claims.

Referee Report

4 major / 5 minor

Summary. The paper adapts the Habits of Mind framework (16 human cognitive habits) to large reasoning models (LRMs), introducing CogTest, a benchmark of 25 tasks per habit with a mix of MATH-500/AIME problems and LLM-generated real-world scenarios. The authors elicit chains of thought from 13 LRMs and 3 non-reasoning LLMs, then use GPT-4.1-mini with an evidence-first prompt to label whether each habit appears in the CoT. They report habit profiles, clustering results, inter-family similarities, and an extension to 200 HarmBench queries in which certain habits (e.g., Taking Responsible Risks) are claimed to be associated with harmful responses. The central claim is that LRMs, unlike conventional LLMs, exhibit human-like cognitive habits and deploy them adaptively across tasks.

Significance. If the measurement pipeline is valid, the paper provides a new behavioral lens on LRMs that connects an established human psychology framework to CoT analysis, with potential applications in monitoring and safety. The benchmark is substantial (16 habits x 25 tasks x 16 models), the evidence-first extraction is a sensible attempt to reduce judge hallucination, the code is public, and the safety extension addresses a timely concern. However, the significance is conditional: all quantitative conclusions rest on an unvalidated LLM judge and on the untested assumption that CoT text faithfully reflects cognitive processes. The current evidence does not yet support the strong phrasing in the abstract and Section 4.

major comments (4)

[Section 3.2 (Prompt 2) and Section 3.3 (Habit Extraction)] The sole annotator of habit presence is GPT-4.1-mini, but the paper reports no human agreement study, no precision/recall against a gold standard, and no analysis of potential judge biases such as CoT length, self-referential phrasing, or model family. Since every quantitative result in Figures 3-4 and Table 1 is derived from these binary labels, the claimed 'reliable habit identification' is not yet supported; I ask for a human annotation study on a stratified sample, with inter-annotator agreement statistics and an error analysis of the judge.
[Section 3.2 (CoT Observation) and Section 7] The measurement treats CoT text as evidence of internal cognitive habits, but the manuscript itself cites Chen et al. (2025) and Baker et al. (2025), which show that reasoning models can produce CoTs that do not reflect their actual reasoning and can conceal or fabricate statements. The limitation paragraph in Section 7 only notes that implicit habits may be missed; it does not address the possibility that explicit meta-thinking statements are unfaithful. The authors should discuss this threat and provide a sanity check, for example by comparing habit labels extracted from CoTs with labels from final responses or from targeted interventions that suppress or encourage specific habits.
[Section 4 (Table 1)] The safety analysis is post hoc and unadjusted: the five models and the top-3 'evidently differentiating' habits are selected after inspecting the percentages, and no significance tests, confidence intervals, or multiple-testing corrections are reported. With denominators as small as 30 harmful responses for DeepSeek-R1, raw percentage differences such as 80.8% versus 3.3% do not by themselves establish a 'strong association.' I ask for a prespecified selection rule, the full 16-habit results for all evaluated models, and statistical tests (e.g., chi-square or permutation tests with correction) before drawing the paper's safety conclusions.
[Abstract and Section 3.4] The central claim that LRMs 'adaptively deploy' cognitive habits according to different tasks is not operationalized. Each habit is evaluated with its own curated set of 25 tasks, so the reported profiles measure per-habit frequency within task sets designed for that habit; the paper does not compare the same habit across different task families or show that deployment varies with task demands. The authors should define 'adaptive deployment' precisely and present evidence that directly tests it, rather than inferring it from habit profiles.

minor comments (5)

[Abstract] There is a missing period between 'adaptively deploy them according to different tasks' and 'Finer-grained analyses'.
[Section 4 (Table 1)] The model name is inconsistent: the experimental setup lists 'Doubao-1.5-thinking-pro' while Table 1 uses 'Doubao-1.5-thinking'; please unify the naming.
[Figure 3 and Figure 4 captions] The caption for Figure 3 says 'Measurement results of the 16 LRMs' but Figure 4 includes three non-reasoning LLMs; please clarify which models appear in each figure.
[References] Liu et al. (2023a) and Liu et al. (2023b) refer to the same paper ('Visual instruction tuning', NeurIPS); one duplicate citation should be removed.
[Prompt 3 (Section 3.3)] There is a typo in Prompt 3: 'Theuser asks' should read 'The user asks'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the CogTest habit measurements are empirical observations, not derivations forced by task design or self-citation.

full rationale

The paper's central claims are empirical measurements of whether certain meta-thinking statements appear in LRM CoTs, aggregated into habit profiles and correlated with safety outcomes. There is no equation, fitted parameter, or statistical model whose output is predetermined by its inputs. The task-generation prompts ask for tasks that 'naturally elicit' a habit, and the extraction prompt asks the judge to find explicit supporting evidence, but the presence labels are not mathematically entailed: the paper reports substantial variation across models and habits (e.g., DeepSeek-R1 scores only 8/25 on Responding with Wonderment and Awe), which would not occur if the measurement were circular. The reliance on GPT-4.1-mini as a judge and on CoT faithfulness is a validity and robustness concern, not a circularity concern. The paper even cites Chen et al. (2025) and Baker et al. (2025), which cast doubt on CoT faithfulness, and its Section 7 limitation discusses missing implicit habits; these caveats reduce confidence in the measurements but do not make the derivation equivalent to its inputs. The self-citations that appear (Xu et al. 2024a/b; Zhang et al. 2025a) are in related work, not load-bearing for the benchmark's construction or conclusions. No uniqueness theorem, ansatz, or fitted value is imported from the authors' prior work. The Habits of Mind framework is an external, established human taxonomy, and the benchmark is self-contained against it. Therefore, no specific circular step can be exhibited, and the appropriate score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claims rest on the validity of the Habits of Mind mapping, the reliability of LLM-based habit extraction, and the faithfulness of CoT text. None of these are independently established in the paper. The safety analysis also depends on the choice of a 10% occurrence threshold and the HarmBench classifier.

free parameters (2)

Habit occurrence threshold for safety analysis = 10%
Habits with occurrence rate below 10% in both harmful and harmless CoTs are excluded from the differentiating set, which changes the reported top habits (Section 4).
Number of clusters in agglomerative clustering = 4
The habit profiles are clustered into 4 groups without a stability analysis or justification (Section 3.4).

assumptions (4)

domain assumption The 16 Habits of Mind defined for human problem-solving can be meaningfully attributed to LRMs based on CoT text.
The paper maps surface CoT statements to a human cognitive framework without validating that humans exhibit these habits on the same tasks or that the mapping is causally valid.
domain assumption The evidence-first LLM annotator (GPT-4.1-mini) reliably identifies habits in CoTs.
No human agreement study or gold-standard validation is provided; the judge is prompted with examples that may bias detection.
domain assumption CoTs generated by LRMs are faithful traces of the model's reasoning and habits.
The paper cites Chen et al. (2025) showing reasoning models do not always reveal their true reasoning, but does not account for this in interpreting results.
domain assumption The HarmBench official classifier correctly labels responses as harmful or harmless.
Used to partition safety CoTs; no error analysis is provided.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Towards Understanding the Cognitive Habits of Large Reasoning Models." pith.science (2026). https://pith.science/paper/OT6LZKGK

@misc{pith2026250621571,
  author       = {Pith},
  title        = {Pith review of: Towards Understanding the Cognitive Habits of Large Reasoning Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OT6LZKGK}},
  note         = {Machine review of arXiv:2506.21571}
}

read the original abstract

Large Reasoning Models (LRMs), which autonomously produce a reasoning Chain of Thought (CoT) before producing final responses, offer a promising approach to interpreting and monitoring model behaviors. Inspired by the observation that certain CoT patterns -- e.g., ``Wait, did I miss anything?'' -- consistently emerge across tasks, we explore whether LRMs exhibit human-like cognitive habits. Building on Habits of Mind, a well-established framework of cognitive habits associated with successful human problem-solving, we introduce CogTest, a principled benchmark designed to evaluate LRMs' cognitive habits. CogTest includes 16 cognitive habits, each instantiated with 25 diverse tasks, and employs an evidence-first extraction method to ensure reliable habit identification. With CogTest, we conduct a comprehensive evaluation of 16 widely used LLMs (13 LRMs and 3 non-reasoning ones). Our findings reveal that LRMs, unlike conventional LLMs, not only exhibit human-like habits but also adaptively deploy them according to different tasks. Finer-grained analyses further uncover patterns of similarity and difference in LRMs' cognitive habit profiles, particularly certain inter-family similarity (e.g., Qwen-3 models and DeepSeek-R1). Extending the study to safety-related tasks, we observe that certain habits, such as Taking Responsible Risks, are strongly associated with the generation of harmful responses. These findings suggest that studying persistent behavioral patterns in LRMs' CoTs is a valuable step toward deeper understanding of LLM misbehavior. The code is available at: https://github.com/jianshuod/CogTest.

Figures

Figures reproduced from arXiv: 2506.21571 by the authors.

**Figure 1.** Pipeline of measuring cognitive habits of LRMs. Operationally, we instantiate each habit with 25 tasks that can effectively differentiate the LRMs’ inherent possession of specific cognitive habits (Habit Specificity and Comprehensiveness). These tasks are carefully designed to reflect the nature of each habit while avoiding any explicit mention or implicit cues about the targeted habit (Spontaneity). Furthermore, t… view at source ↗

**Figure 2.** The Habits of Mind framework and the corresponding examples of meta-thinking statements. Habits are evaluated via math problems (blue) and LLM-generated tasks (purple). distinct cognitive habits. Furthermore, we extend our evaluation to safety-related contexts, revealing that certain cognitive habits are strongly associated with generating harmful responses. 2 Background Large Reasoning Models (LRMs). Wei et al. (… view at source ↗

**Figure 3.** Measurement results of the 16 LRMs’ cognitive habits on CogTest. All other LRMs, with habit names omitted for brevity, follow the same habit display ordering as DeepSeek-R1. We define cognitive habit profiles as the frequency with which cognitive habits are activated on CogTest by a given LRM. Another intriguing observation reveals notable similarities and differences across models. To systematically analyze these… view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Measurement results of the 16 LRMs’ cognitive habits on CogTest. All other LRMs, with habit names omitted for brevity, follow the same habit display ordering as DeepSeek-R1. about Thinking. Meanwhile, non-reasoning models do exhibit certain cognitive habits; for examp…

**Figure 5.** Figure 5: Qualitative example of similarity between DeepSeek-R1 and Qwen-3-235B-A22B in CoT trajectories and certain steps. This task belongs to the Listening with Understanding and Empathy habit. CoTs reveals strikingly similar reasoning steps and highly analogous CoT trajector…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

62 extracted references · 29 canonical work pages

[1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Anthropic. 2025. Claude 3.7 Sonnet and Claude Code . https://www.anthropic.com/news/claude-3-7-sonnet

work page 2025
[4]

Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, and 1 others. 2025. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923

arXiv 2025
[5]

Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, and 1 others. 2022. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073

arXiv 2022
[6]

Guan, Aleksander Madry, Wojciech Zaremba, Jakub Pachocki, and David Farhi

Bowen Baker, Joost Huizinga, Leo Gao, Zehao Dou, Melody Y. Guan, Aleksander Madry, Wojciech Zaremba, Jakub Pachocki, and David Farhi. 2025. Monitoring reasoning models for misbehavior and the risks of promoting obfuscation

work page 2025
[7]

Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders. 2023. Language models can explain neurons in language models. https://openaipublic.blob.core.windows.net/neuron-explainer/paper/index.html

2023
[8]

Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, and 1 others. 2024. Do not think that much for 2+ 3=? on the overthinking of o1-like llms. arXiv preprint arXiv:2412.21187

arXiv 2024

Show all 62 references

[9]

Yanda Chen, Joe Benton, Ansh Radhakrishnan, Jonathan Uesato Carson Denison, John Schulman, Arushi Somani, Peter Hase, Misha Wagner Fabien Roger Vlad Mikulik, Sam Bowman, Jan Leike Jared Kaplan, and 1 others. 2025. Reasoning models don’t always say what they think. Anthropic Research

2025
[10]

Art Costa and Bena Kallick. 2005. Habits of mind. Hawker Brownlow Melbourne

2005
[11]

Google DeepMind. 2025. Gemini 2.5: Our most intelligent ai model. https://blog.google/technology/google-deepmind/gemini-model-thinking-updates-march-2025

2025
[12]

Hao Fei, Yuan Zhou, Juncheng Li, Xiangtai Li, Qingshan Xu, Bobo Li, Shengqiong Wu, Yaoting Wang, Junbao Zhou, Jiahao Meng, Qingyu Shi, Zhiyuan Zhou, Liangtao Shi, Minghe Gao, Daoan Zhang, Zhiqi Ge, Weiming Wu, Siliang Tang, Kaihang Pan, and 13 others. 2025. On path to multimod...

2025
[13]

Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D Goodman. 2025. Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars. arXiv preprint arXiv:2503.01307

2025 arXiv
[14]

Jiahui Geng, Fengyu Cai, Yuxia Wang, Heinz Koeppl, Preslav Nakov, and Iryna Gurevych. 2024. A survey of confidence estimation and calibration in large language models. In NAACL

2024
[15]

Xinyu Guan, Li Lyna Zhang, Yifei Liu, Ning Shang, Youran Sun, Yi Zhu, Fan Yang, and Mao Yang. 2025. rstar-math: Small llms can master math reasoning with self-evolved deep thinking. arXiv preprint arXiv:2501.04519

2025 arXiv
[16]

Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

2025 arXiv
[17]

Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. In NeurIPS Datasets and Benchmarks Track

2021
[18]

Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276

2024 arXiv
[19]

Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, and 1 others. 2024. Openai o1 system card. arXiv preprint arXiv:2412.16720

2024 arXiv
[20]

Erik Jones and Jacob Steinhardt. 2022. Capturing failures of large language models via human cognitive biases. NeurIPS

2022
[21]

Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. In NeurIPS

2022
[22]

Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In SOSP

2023
[23]

Xiaojian Li, Haoyuan Shi, Rongwu Xu, and Wei Xu. 2025 a . Ai awareness. arXiv preprint arXiv:2504.20084

2025 arXiv
[24]

Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Pei-Jie Wang, Xiuyi Chen, and 1 others. 2025 b . From system 1 to system 2: A survey of reasoning large language models. arXiv preprint arXiv:2502.17419

2025 arXiv
[25]

Ruixi Lin and Hwee Tou Ng. 2023. Mind the biases: Quantifying cognitive biases in language model prompting. In ACL

2023
[26]

Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437

2024 arXiv
[27]

Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023 a . Visual instruction tuning. In NeurIPS

2023
[28]

Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023 b . Visual instruction tuning. In NeurIPS

2023
[29]

Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, and 1 others. 2024. Harmbench: a standardized evaluation framework for automated red teaming and robust refusal. In ICML

2024
[30]

Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Cand \`e s, and Tatsunori Hashimoto. 2025. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393

2025 arXiv
[31]

Fionn Murtagh and Pierre Legendre. 2014. Ward’s hierarchical agglomerative clustering method: which algorithms implement ward’s criterion? Journal of classification

2014
[32]

OpenAI. 2025 a . Introducing OpenAI o3 and o4-mini . https://openai.com/index/introducing-o3-and-o4-mini/

2025
[33]

OpenAI. 2025 b . https://model-spec.openai.com/2025-04-11.html OpenAI Model Spec

2025
[34]

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. Training language models to follow instructions with human feedback. NeurIPS

2022
[35]

Keyu Pan and Yawen Zeng. 2023. Do llms possess a personality? making the mbti test an amazing evaluation for large language models. arXiv preprint arXiv:2307.16180

2023 arXiv
[36]

Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. OpenAI blog

2018
[37]

ByteDance Seed, Yufeng Yuan, Yu Yue, Mingxuan Wang, Xiaochen Zuo, Jiaze Chen, Lin Yan, Wenyuan Xu, Chi Zhang, Xin Liu, and 1 others. 2025. Seed-thinking-v1. 5: Advancing superb reasoning models with reinforcement learning. arXiv preprint arXiv:2504.13914

2025
[38]

Ammar Shaikh, Raj Abhijit Dandekar, Sreedath Panat, and Rajat Dandekar. 2024. Cbeval: A framework for evaluating and interpreting cognitive biases in llms. arXiv preprint arXiv:2412.03605

2024 arXiv
[39]

Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, YK Li, Yu Wu, and Daya Guo. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300

2024 arXiv
[40]

Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R. Bowman, Esin DURMUS, Zac Hatfield-Dodds, Scott R Johnston, Shauna M Kravec, Timothy Maxwell, Sam McCandlish, Kamal Ndousse, Oliver Rausch, Nicholas Schiefer, Da Yan, Miranda Zhang, and Ethan Pere...

2024
[41]

Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, Dipanjan Das, and Jason Wei. 2023. Language models are multilingual chain-of-thought reasoners. In ICLR

2023
[42]

Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2024. Scaling llm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314

2024 arXiv
[43]

Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Hanjie Chen, and 1 others. 2025. Stop overthinking: A survey on efficient reasoning for large language models. arXiv preprint arXiv:2503.16419

2025 arXiv
[44]

Zhongxiang Sun, Qipeng Wang, Haoyu Wang, Xiao Zhang, and Jun Xu. 2025. Detection and mitigation of hallucination in large reasoning models: A mechanistic perspective. arXiv preprint arXiv:2505.12886

2025 arXiv
[45]

Qwen Team. 2025. https://qwenlm.github.io/blog/qwq-32b/ Qwq-32b: Embracing the power of reinforcement learning

2025
[46]

Xuezhi Wang and Denny Zhou. 2024. Chain-of-thought reasoning without prompting. arXiv preprint arXiv:2402.10200

2024 arXiv
[47]

Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. Self-instruct: Aligning language models with self-generated instructions. In ACL

2023
[48]

Yue Wang, Qiuzhi Liu, Jiahao Xu, Tian Liang, Xingyu Chen, Zhiwei He, Linfeng Song, Dian Yu, Juntao Li, Zhuosheng Zhang, and 1 others. 2025. Thoughts are all over the place: On the underthinking of o1-like llms. arXiv preprint arXiv:2501.18585

2025 arXiv
[49]

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. In NeurIPS

2022
[50]

Fengli Xu, Qianyue Hao, Zefang Zong, Jingwei Wang, Yunke Zhang, Jingyi Wang, Xiaochong Lan, Jiahui Gong, Tianjian Ouyang, Fanjin Meng, and 1 others. 2025. Towards large reasoning models: A survey of reinforced reasoning with large language models. arXiv preprint arXiv:2501.09686

2025 arXiv
[51]

Rongwu Xu, Yishuo Cai, Zhenhong Zhou, Renjie Gu, Haiqin Weng, Liu Yan, Tianwei Zhang, Wei Xu, and Han Qiu. 2024 a . Course-correction: Safety alignment using synthetic preferences. In EMNLP

2024
[52]

Rongwu Xu, Brian Lin, Shujian Yang, Tianqi Zhang, Weiyan Shi, Tianwei Zhang, Zhixuan Fang, Wei Xu, and Han Qiu. 2024 b . The earth is flat because...: Investigating llms’ belief towards misinformation via persuasive conversation. In ACL

2024
[53]

An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388

2025 arXiv
[54]

An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, and 1 others. 2024. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115

2024 arXiv
[55]

Longhui Yu, Weisen Jiang, Han Shi, Jincheng YU, Zhengying Liu, Yu Zhang, James Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. 2024. Metamath: Bootstrap your own mathematical questions for large language models. In ICLR

2024
[56]

Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Shiji Song, and Gao Huang. 2025. Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model? arXiv preprint arXiv:2504.13837

2025 arXiv
[57]

Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. 2022. Star: Bootstrapping reasoning with reasoning. In NeurIPS

2022
[58]

Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, and Weiyan Shi. 2024. How johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge ai safety by humanizing llms. In ACL

2024
[59]

Qingjie Zhang, Han Qiu, Di Wang, Haoting Qian, Yiming Li, Tianwei Zhang, and Minlie Huang. 2025 a . Understanding the dark side of llms' intrinsic self-correction. In ACL

2025
[60]

Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, and 1 others. 2023. Siren's song in the ai ocean: a survey on hallucination in large language models. arXiv preprint arXiv:2309.01219

2023 arXiv
[61]

Zhexin Zhang, Xian Qi Loye, Victor Shea-Jay Huang, Junxiao Yang, Qi Zhu, Shiyao Cui, Fei Mi, Lifeng Shang, Yingkang Wang, Hongning Wang, and 1 others. 2025 b . How should we enhance the safety of large reasoning models: An empirical study. arXiv preprint arXiv:2505.15404

2025 arXiv
[62]

Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, and 1 others. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. In NeurIPS

2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Anthropic. 2025. Claude 3.7 Sonnet and Claude Code . https://www.anthropic.com/news/claude-3-7-sonnet

work page 2025

[4] [4]

Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, and 1 others. 2025. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923

arXiv 2025

[5] [5]

Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, and 1 others. 2022. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073

arXiv 2022

[6] [6]

Guan, Aleksander Madry, Wojciech Zaremba, Jakub Pachocki, and David Farhi

Bowen Baker, Joost Huizinga, Leo Gao, Zehao Dou, Melody Y. Guan, Aleksander Madry, Wojciech Zaremba, Jakub Pachocki, and David Farhi. 2025. Monitoring reasoning models for misbehavior and the risks of promoting obfuscation

work page 2025

[7] [7]

Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders. 2023. Language models can explain neurons in language models. https://openaipublic.blob.core.windows.net/neuron-explainer/paper/index.html

2023

[8] [8]

Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, and 1 others. 2024. Do not think that much for 2+ 3=? on the overthinking of o1-like llms. arXiv preprint arXiv:2412.21187

arXiv 2024