Pith. sign in

REVIEW 50 references

Unlocking Recursive Thinking of LLMs: Alignment via Refinement

T0 review · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that aligning a language model on refinement-aware rewards—rewarding each successive criticism and improvement step rather than only the final answer—unlocks recursive thinking with far less data than conventional…

desk verdict Solid empirical paper with a plausible two-stage refinement pipeline; the reward-model dependence is the main soft spot but not fatal. read the letter →

arxiv 2506.06009 v1 pith:QD6TL24K submitted 2025-06-06 cs.CL cs.AI

Haoke Zhang , Xiaobo Liang , Cunxiang Wang , Juntao Li , Min Zhang This is my paper
classification cs.CLcs.AI
keywords recursivethinkingalignmentrefinement-awarerewardpreferenceoptimizationchain-of-thoughttest-timescalingself-correctionDPO
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces AvR (Alignment via Refinement), a two-stage method to make large language models recursively refine their own responses instead of producing a single final answer. Stage 1 trains the model to maximize a refinement-aware reward: every criticism-and-improvement step must beat the previous response, judged by a Bradley-Terry reward model, with DPO on paired refinements. Stage 2 uses the trained model to greedily search long chains of self-criticism and improvement, then fine-tunes on those trajectories so the model performs recursive thinking autonomously at inference time. The central empirical claim is that this approach outperforms standard preference-optimization and RL baselines trained on 60k samples, using only about 10–14k synthetic examples—with a 20.2-point win-rate gain on AlpacaEval 2 over the LLaMA-3-8B-Instruct seed. If true, it suggests that teaching the process of refinement, not just the preference between final answers, is what unlocks test-time scaling in open-ended chat.

What carries the argument

The central object is the refinement-aware reward, a scalar signal computed by a Bradley-Terry reward model over every transition in a refinement tree. The method defines a multi-step MDP whose actions are whole responses or refinements, and accepts a trajectory only if each refinement scores higher than the previous step (and higher than the initial response). In Stage 1 a refinement tree of criticism and improvement nodes is built by parallel sampling, then RSFT and DPO train the policy to prefer the best-improving branch; in Stage 2 a greedy search over criticisms and improvements synthesizes long chains that become the training data for autonomous recursive thinking. The rejection-sampling condition—discard any trajectory where reward does not increase monotonically—is what guarantees the model only learns from cumulatively beneficial refinements.

What would settle it

Re-run the Stage 2 synthesis with a different reward model of comparable quality at every scoring point; if the AlpacaEval 2 win-rate gain over the seed falls by more than a few points or reverses, the effect is bound to the specific judge, not to the refinement-aware reward.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that a refinement-aware reward—one that scores each round of criticism and improvement against the previous round—can convert a standard instruction-tuned model into a self-correcting reasoner. Concretely, AvR Stage II, trained on 10k long-form CoT trajectories synthesized by its own Stage I model, reaches a 51.0% win rate and 51.4% length-controlled win rate on AlpacaEval 2 against GPT-4-Turbo, and 34.5% on Arena-Hard v0.1, starting from a seed model at 25.0%. The paper also reports that 3k synthetic samples already produce a >20-point win-rate improvement, and that the trained 8B model can refine the outputs of stronger models such as GPT-4o and GPT-4o-mini. The authors position this as an efficient alternative to compute-heavy RL pipelines that distill recursive behavior.

Load-bearing premise

The entire pipeline trusts one reward model's rankings: if Skywork-Reward-Gemma-2-27B-v0.2 mis-ranks a refinement, the synthetic data and the learned recursive behavior inherit that error, so the gains may be specific to this judge and not transfer.

Editorial extensions

If this is right

  • If the 3k-sample result holds, practitioners can unlock recursive reasoning on an 8B model with orders of magnitude less preference data than current RL pipelines.
  • Because Stage 2 data is generated entirely by the model itself after Stage 1, the pipeline removes the need for expert-curated CoT distillation.
  • The length-controlled DPO stage shows that verbosity from reward-model bias can be corrected with a few thousand self-generated preference pairs, yielding an 8.9-point gain in LC win rate.
  • The AvR Stage I model can improve responses of GPT-4o and GPT-4o-mini on AlpacaEval 2, suggesting refinement ability transfers across model strengths.
  • The method's gains concentrate in the first few refinement rounds (best iteration typically 1–3), implying a short recursive loop is sufficient.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the mechanism is right, the same refinement-aware reward could be applied to other test-time-scaling regimes, e.g., tool-calling or multi-step code execution, where each step's improvement can be scored by execution feedback.
  • The paper's reliance on a single reward model implies a natural stress-test: swapping Skywork-Reward for a different judge may change which refinement branches survive, predicting that data quality, not just scale, is the bottleneck.
  • A testable extension is to make the reward model itself part of the loop, letting the trained model generate critiques that the reward model scores, effectively co-training the verifier.
  • Because the stage-1 model is trained to maximize improvement over the initial response, it may be especially suited for settings where an external system (retrieval, code interpreter) supplies the initial draft.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the headline win rates are measured by external GPT-4-Turbo judges independent of the Skywork reward model used to synthesize training data; the few overlapping-author citations are non-load-bearing.

full rationale

The paper's derivation chain is: (1) Stage I builds a refinement tree whose criticism/improvement nodes are scored by the Bradley-Terry reward model Skywork-Reward-Gemma-2-27B-v0.2, and preference pairs are formed by selecting the highest-scoring refinements (Sections 4.1, 5.1); (2) Stage II uses that same reward model to greedily select the best improvement at each step and to decide when to stop, yielding recursive CoT trajectories (Section 4.2); (3) RSFT + DPO on these trajectories produces the reported model. The headline empirical claims—51.0%/51.4% win rate/LC win rate on AlpacaEval 2 and 34.5% on Arena-Hard—are computed by the GPT-4-Turbo auto-evaluator, an external measurement channel wholly separate from Skywork. No benchmark outcome is used to fit any parameter of the method; the algorithm's internal selections and stopping decisions are determined by Skywork, and the evaluations are then measured by a different judge on benchmark queries that are not among the 60k UltraFeedback training prompts. Hence the predictions are not identified with the fitting signal by construction. The overlapping-author citations (Wang et al. 2023 Pandalm and Liang et al. 2024 Fennec in the Limitations paragraph; Wang et al. 2025 on underthinking in the Introduction) are background or future-work references only; none is load-bearing, as the surrounding claims are independently supported by non-self citations (e.g., Kamoi et al. 2024; Jiang et al. 2024). The Limitations section honestly concedes that reward-model preference affects synthetic data quality; that is a real robustness/correctness risk (single reward model, no independent-reward ablation), not an instance of circular derivation, because the reported evaluations are external to Skywork. Verdict: no significant circularity; score 2 reflects only the presence of minor, non-load-bearing self-citations.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central empirical claim rests on the reliability of the Skywork reward model for all data synthesis and on the transferability of the resulting behavior to GPT-4-judged benchmarks. The ledger lists the main hand-set hyperparameters and domain assumptions. No parameter is fitted to the evaluation benchmarks.

free parameters (4)
  • Number of criticism/improvement branches per iteration = 2
    Section 4.2 sets step = 2 for the greedy search; this controls the width of the refinement tree and is chosen by hand.
  • DPO length-control data scale = 4k
    Section 4.2 and Table 1 use 4k preference pairs for the length-control DPO stage; the size is an empirical choice.
  • Sampling temperature and top_p = 0.7 / 0.8
    Section 5.1 sets temperature 0.7 and top_p 0.8 for all data generation and testing; these are hand-set.
  • Stage II SFT data scale = 10k
    Section 5.6 selects 10k long CoT trajectories as the standard setting; 20k gives marginal gains, so 10k is used.
assumptions (4)
  • domain assumption Skywork-Reward-Gemma-2-27B-v0.2 provides a valid scalar preference score for ranking any two responses to the same prompt.
    Used throughout Section 4 to score refinements, construct DPO pairs, select greedy-search branches, and stop iteration. If this reward model is biased, all synthetic data is affected.
  • domain assumption The GPT-4-based judges on AlpacaEval 2 and Arena-Hard v0.1 approximate human preferences for open-domain chat.
    Section 5.1 defines win rate via these auto-evaluators; the paper's headline gains are measured with them.
  • domain assumption Greedy search guided by the reward model produces high-quality long chain-of-thought trajectories for SFT.
    Section 4.2 synthesizes Stage II training data this way; the paper calls the trajectories 'high-quality' but provides no external validation beyond the final benchmark scores.
  • ad hoc to paper Mixing 60k original Llama-3-8B-Instruct responses into the RSFT corpus preserves generative ability without interfering with refinement learning.
    Footnote 8 in Section 5.2 introduces this mix to mitigate degeneration from fixed-format training; it is a setup-specific fix with no independent support.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Unlocking Recursive Thinking of LLMs: Alignment via Refinement." pith.science (2026). https://pith.science/paper/QD6TL24K

@misc{pith2026250606009,
  author       = {Pith},
  title        = {Pith review of: Unlocking Recursive Thinking of LLMs: Alignment via Refinement},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QD6TL24K}},
  note         = {Machine review of arXiv:2506.06009}
}
read the original abstract

The OpenAI o1-series models have demonstrated that leveraging long-form Chain of Thought (CoT) can substantially enhance performance. However, the recursive thinking capabilities of Large Language Models (LLMs) remain limited, particularly in the absence of expert-curated data for distillation. In this paper, we propose \textbf{AvR}: \textbf{Alignment via Refinement}, a novel method aimed at unlocking the potential of LLMs for recursive reasoning through long-form CoT. AvR introduces a refinement process that integrates criticism and improvement actions, guided by differentiable learning techniques to optimize \textbf{refinement-aware rewards}. As a result, the synthesized multi-round data can be organized as a long refinement thought, further enabling test-time scaling. Experimental results show that AvR significantly outperforms conventional preference optimization methods. Notably, with only 3k synthetic samples, our method boosts the performance of the LLaMA-3-8B-Instruct model by over 20\% in win rate on AlpacaEval 2.0. Our code is available at Github (https://github.com/Banner-Z/AvR.git).

Figures

Figures reproduced from arXiv: 2506.06009 by the authors.

Figure 1
Figure 1. Reward assignment comparison between tra [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration of two distinct reasoning paradigms. The left side depicts a [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Illustration of our framework: on the left is the first stage, enabling recursive thinking in multi-step [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The results demonstrate our model’s ability to [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 6
Figure 6. Figure 6: Experimental results on the scaling of DPO [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

50 extracted references · 11 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    AI@Meta. 2024. https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md Llama 3 model card

    2024

  4. [4]

    Chang, and Prithviraj Ammanabrolu

    Zachary Ankner, Mansheej Paul, Brandon Cui, Jonathan D. Chang, and Prithviraj Ammanabrolu. 2024. https://arxiv.org/abs/2408.11791 Critique-out-loud reward models . Preprint, arXiv:2408.11791

    arXiv 2024

  5. [5]

    Carl Bereiter and Marlene Scardamalia. 2013. The psychology of written composition. Routledge

    work page 2013

  6. [6]

    Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, et al. 2024. Do not think that much for 2+ 3=? on the overthinking of o1-like llms. arXiv preprint arXiv:2412.21187

    arXiv 2024

  7. [7]

    Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Wei Zhu, Yuan Ni, Guotong Xie, Zhiyuan Liu, and Maosong Sun. 2023. https://arxiv.org/abs/2310.01377 Ultrafeedback: Boosting language models with high-quality feedback . Preprint, arXiv:2310.01377

    arXiv 2023

  8. [8]

    Tri Dao. 2023. Flashattention-2: Faster attention with better parallelism and work partitioning. arXiv preprint arXiv:2307.08691

    arXiv 2023

Show all 50 references
  1. [9]

    DeepSeekTeam. 2024. Deepseek-r1-lite-preview is now live: unleashing supercharged reasoning power

    2024

  2. [10]

    Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. 2024. Kto: Model alignment as prospect theoretic optimization. arXiv preprint arXiv:2402.01306

    2024 arXiv

  3. [11]

    Xinyu Guan, Li Lyna Zhang, Yifei Liu, Ning Shang, Youran Sun, Yi Zhu, Fan Yang, and Mao Yang. 2025. rstar-math: Small llms can master math reasoning with self-evolved deep thinking. arXiv preprint arXiv:2501.04519

    2025 arXiv

  4. [12]

    Jiwoo Hong, Noah Lee, and James Thorne. 2024. Reference-free monolithic preference optimization with odds ratio. arXiv e-prints, pages arXiv--2403

    2024

  5. [13]

    Dongwei Jiang, Jingyu Zhang, Orion Weller, Nathaniel Weir, Benjamin Van Durme, and Daniel Khashabi. 2024. Self-[in] correct: Llms struggle with refining self-generated responses. arXiv preprint arXiv:2404.04298

    2024 arXiv

  6. [14]

    Ryo Kamoi, Yusen Zhang, Nan Zhang, Jiawei Han, and Rui Zhang. 2024. When can llms actually correct their own mistakes? a critical survey of self-correction of llms. Transactions of the Association for Computational Linguistics, 12:1417--1440

    2024

  7. [15]

    Geunwoo Kim, Pierre Baldi, and Stephen McAleer. 2023. Language models can solve computer tasks. Advances in Neural Information Processing Systems, 36:39648--39677

    2023

  8. [16]

    Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D Co-Reyes, Avi Singh, Kate Baumli, Shariq Iqbal, Colton Bishop, Rebecca Roelofs, et al. 2024. Training language models to self-correct via reinforcement learning. arXiv preprint arXiv:2409.12917

    2024 arXiv

  9. [17]

    Tianle Li, Wei-Lin Chiang, Evan Frick, Lisa Dunlap, Tianhao Wu, Banghua Zhu, Joseph E Gonzalez, and Ion Stoica. 2024. From crowdsourced data to high-quality benchmarks: Arena-hard and benchbuilder pipeline. arXiv preprint arXiv:2406.11939

    2024 arXiv

  10. [18]

    Hashimoto

    Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Alpacaeval: An automatic evaluator of instruction-following models. https://github.com/tatsu-lab/alpaca_eval

    2023

  11. [19]

    Xiaobo Liang, Haoke Zhang, Juntao Li, Jun Xu, Min Zhang, et al. 2024. Fennec: Fine-grained language model evaluation and correction extended through branching and bridging. arXiv preprint arXiv:2405.12163

    2024 arXiv

  12. [20]

    Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let's verify step by step. arXiv preprint arXiv:2305.20050

    2023 arXiv

  13. [21]

    Chris Yuhao Liu, Liang Zeng, Jiacai Liu, Rui Yan, Jujie He, Chaojie Wang, Shuicheng Yan, Yang Liu, and Yahui Zhou. 2024. Skywork-reward: Bag of tricks for reward modeling in llms. arXiv preprint arXiv:2410.18451

    2024 arXiv

  14. [22]

    Yu Meng, Mengzhou Xia, and Danqi Chen. 2025. Simpo: Simple preference optimization with a reference-free reward. Advances in Neural Information Processing Systems, 37:124198--124235

    2025

  15. [23]

    Yingqian Min, Zhipeng Chen, Jinhao Jiang, Jie Chen, Jia Deng, Yiwen Hu, Yiru Tang, Jiapeng Wang, Xiaoxue Cheng, Huatong Song, et al. 2024. Imitate, explore, and self-improve: A reproduction report on slow-thinking reasoning systems. arXiv preprint arXiv:2412.09413

    2024 arXiv

  16. [24]

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Cand \`e s, and Tatsunori Hashimoto. 2025. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393

    2025 arXiv

  17. [25]

    OpenAI . 2024. Learning to reason with llms. https://openai.com/index/learning-to-reason-with-llms/

    2024

  18. [26]

    Ryan Park, Rafael Rafailov, Stefano Ermon, and Chelsea Finn. 2024. Disentangling length from quality in direct preference optimization. arXiv preprint arXiv:2403.19159

    2024 arXiv

  19. [27]

    Yiwei Qin, Xuefeng Li, Haoyang Zou, Yixiu Liu, Shijie Xia, Zhen Huang, Yixin Ye, Weizhe Yuan, Hector Liu, Yuanzhi Li, et al. 2024. O1 replication journey: A strategic progress report--part 1. arXiv preprint arXiv:2410.18982

    2024 arXiv

  20. [28]

    Yuxiao Qu, Tianjun Zhang, Naman Garg, and Aviral Kumar. 2024. Recursive introspection: Teaching language model agents how to self-improve. arXiv preprint arXiv:2407.18219

    2024 arXiv

  21. [29]

    QwenTeam. 2024. Qwq: Reflect deeply on the boundaries of the unknown, november 2024. URL https://qwenlm. github. io/blog/qwq-32b-preview

    2024

  22. [30]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728--53741

    2023

  23. [31]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2024. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36

    2024

  24. [32]

    Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. 2020. Zero: Memory optimizations toward training trillion parameter models. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1--16. IEEE

    2020

  25. [33]

    Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. 2020. Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, p...

    2020

  26. [34]

    Donald A Sch \"o n. 1979. The reflective practitioner. New York

    1979

  27. [35]

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347

    2017 arXiv

  28. [36]

    Herbert A Simon and Allen Newell. 1971. Human problem solving: The state of the theory in 1970. American psychologist, 26(2):145

    1971

  29. [37]

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2024. Scaling llm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314

    2024 arXiv

  30. [38]

    Richard S Sutton. 1988. Learning to predict by the methods of temporal differences. Machine learning, 3:9--44

    1988

  31. [39]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. 2025. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599

    2025 arXiv

  32. [40]

    Qwen Team. 2024. https://qwenlm.github.io/blog/qwen2.5/ Qwen2.5: A party of foundation models

    2024

  33. [41]

    Jiaan Wang, Fandong Meng, Yunlong Liang, and Jie Zhou. 2024 a . Drt-o1: Optimized deep reasoning translation via long chain-of-thought. arXiv preprint arXiv:2412.17498

    2024 arXiv

  34. [42]

    Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. 2024 b . Math-shepherd: Verify and reinforce llms step-by-step without human annotations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguis...

    2024

  35. [43]

    Yidong Wang, Zhuohao Yu, Zhengran Zeng, Linyi Yang, Cunxiang Wang, Hao Chen, Chaoya Jiang, Rui Xie, Jindong Wang, Xing Xie, et al. 2023. Pandalm: An automatic evaluation benchmark for llm instruction tuning optimization. arXiv preprint arXiv:2306.05087

    2023 arXiv

  36. [44]

    Yue Wang, Qiuzhi Liu, Jiahao Xu, Tian Liang, Xingyu Chen, Zhiwei He, Linfeng Song, Dian Yu, Juntao Li, Zhuosheng Zhang, et al. 2025. Thoughts are all over the place: On the underthinking of o1-like llms. arXiv preprint arXiv:2501.18585

    2025 arXiv

  37. [45]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824--24837

    2022

  38. [46]

    Tianhao Wu, Weizhe Yuan, Olga Golovneva, Jing Xu, Yuandong Tian, Jiantao Jiao, Jason Weston, and Sainbayar Sukhbaatar. 2024. Meta-rewarding language models: Self-improving alignment with llm-as-a-meta-judge. arXiv preprint arXiv:2407.19594

    2024 arXiv

  39. [47]

    Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho, Sainbayar Sukhbaatar, Jing Xu, and Jason Weston. 2024. Self-rewarding language models. arXiv preprint arXiv:2401.10020

    2024 arXiv

  40. [48]

    Qingjie Zhang, Han Qiu, Di Wang, Haoting Qian, Yiming Li, Tianwei Zhang, and Minlie Huang. 2024 a . Understanding the dark side of llms' intrinsic self-correction. arXiv preprint arXiv:2412.14959

    2024 arXiv

  41. [49]

    Yuxiang Zhang, Shangxi Wu, Yuqi Yang, Jiangming Shu, Jinlin Xiao, Chao Kong, and Jitao Sang. 2024 b . o1-coder: an o1 replication for coding. arXiv preprint arXiv:2412.00154

    2024 arXiv

  42. [50]

    Yu Zhao, Huifeng Yin, Bo Zeng, Hao Wang, Tianqi Shi, Chenyang Lyu, Longyue Wang, Weihua Luo, and Kaifu Zhang. 2024. Marco-o1: Towards open reasoning models for open-ended solutions. arXiv preprint arXiv:2411.14405

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.