Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

Logit Arithmetic Elicits Long Reasoning Capabilities Without Training

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A frozen 32B language model can be steered into long chain-of-thought reasoning by adding the logit difference between a small reasoning model and its base.

desk verdict A cheap, practical way to elicit long CoT from a frozen large model using a small guider; worth a referee, but the direction control is missing and alpha tuning on test sets is a real weakness. read the letter →

arxiv 2507.12759 v1 pith:2EAINMEF submitted 2025-07-17 cs.CL cs.AI

Yunxiang Zhang , Muhammad Khalifa , Lechen Zhang , Xin Liu , Ayoung Lee , Xinliang Frederick Zhang , Farima Fatahi Bayat , Lu Wang This is my paper
classification cs.CLcs.AI
keywords logitarithmeticchain-of-thoughtreasoningdecoding-timesteeringfrozenlanguagemodelsmathematicalpreferenceoptimizationsmall-to-largemodeltransferdistillation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether a large language model can be made to reason at length—backtracking, checking, and correcting itself—without any training of that model. It proposes ThinkLogit, a decoding-time method that adds, at every token step, the difference between the logits of a small reasoning model and the logits of its small non-reasoning base to the logits of a frozen large target. On four mathematical benchmarks this raises the target's pass@1 (the fraction of single-attempt answers that are correct) by 26% relative, and by 29% when the small guider is first aligned to the target with preference optimization (ThinkLogit-DPO). The same recipe transfers reasoning skills learned by reinforcement learning in a 1.5B model, improving the 32B target by 13%. The point is that long chain-of-thought behavior can be treated as a steerable direction in score space rather than something that must be trained into each large model.

What carries the argument

The mechanism is logit arithmetic on the pre-softmax score vectors: at each step after a 100-token warm-up, the target's logits are replaced by $\tilde{\ell}_{t+1} = \ell^{(L)}_{t+1} + \alpha(\ell^{(S^\star)}_{t+1} - \ell^{(S)}_{t+1})$, with $\alpha = 1$. The delta $\ell^{(S^\star)} - \ell^{(S)}$ is the "become a long reasoner" direction computed from a small model pair, and the paper's premise is that this direction is portable to a larger model. ThinkLogit-DPO is a second mechanism: it first trains the small guider with DPO on two kinds of preference pairs—cases where the large model's short answer is right and the small model's long one is wrong, and the reverse—so the guider learns to preserve the target's strengths while fixing its errors, and then applies the same logit arithmetic with the improved guider.

What would settle it

Apply the identical ThinkLogit setup to a 32B target from a different pretrained family whose tokenizer does not overlap with the 1.5B guider's, after aligning the vocabularies by projection; if the 26% relative pass@1 gain disappears or turns negative, the claim that the small-model delta transfers to arbitrary large models is false. A cheaper control: add a sign-reversed delta ($-\alpha$ instead of $+\alpha$) or a random delta of the same magnitude; if either also improves accuracy, the specific long-vs-short direction is not what carries the gain.

Watch

Extended reading notes

Core claim

The central claim is that long chain-of-thought reasoning is not a capacity that has to be installed in a large model by fine-tuning; it can be elicited by steering the frozen model's next-token scores. Concretely, the paper shows that the vector difference between a small long-CoT model's logits and its short-CoT base model's logits—the shift that makes the small model generate long, self-correcting reasoning—can be added to a much larger model's logits at decoding time, transferring the behavior with no weight updates. The paper further claims that this transfer is not an artifact of longer outputs: a budget-forcing control that simply makes the large model write more tokens lowers accuracy, while the logit-guided model improves it. On AIME2024, AIME2025, AMC23, and a hard subset of MATH500, ThinkLogit raises the frozen Qwen2.5-32B's average pass@1 from 32.7 to 41.2, and ThinkLogit-DPO to 42.2; with an RL-trained 1.5B guider the gain is from 32.7 to 36.8.

Load-bearing premise

The load-bearing premise is that the direction in which a small model's word-scores change when it is turned into a long reasoner is the same direction that makes a much larger model reason at length, so adding that difference to the large model's scores nudges it correctly.

Editorial extensions

If this is right

  • Long reasoning can be added to a frozen large model at inference time for roughly the cost of one extra forward pass of a much smaller model, with no gradient updates to the large model.
  • Reasoning skills developed cheaply in small models—by SFT on distilled traces or by RL on a handful of questions—can be deployed in a larger model without re-training it.
  • Preference-aligning the guider to the target recovers a large share of the gains of full fine-tuning: ThinkLogit-DPO reaches pass@8 of 65.6 versus 67.5 for a 32B model fine-tuned on 1K distilled examples.
  • The method broadens the reasoning boundary, since pass@8 improves more than pass@1, indicating that more of the sampled solution space becomes correct.
  • Since a length-only control fails, output length alone is not the active ingredient; the content direction of the logit delta matters.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Not in the paper: if the same logit-delta direction transfers across model families and tokenizers, the recipe could be applied to targets whose weights are frozen or inaccessible, but only when the target exposes logits; API models that hide logits would need a different interface.
  • Not in the paper: the shared Qwen ancestry of the target and guiders may make the score-space alignment easier than in cross-family pairs; testing with a target from a different pretrained family would bound the transfer claim.
  • Not in the paper: an adaptive guidance strength $\alpha$ that fades out once the trace enters a stable reasoning pattern could reduce the overlong, repetitive outputs the paper notes, and might make the gains larger than the fixed $\alpha=1$ used here.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes THINKLOGIT, a decoding-time method that adds a scaled difference between the logits of a small long-CoT reasoning model (S*) and its base model (S) to the logits of a frozen large target model L, with the goal of eliciting long chain-of-thought reasoning without training L. A variant, THINKLOGIT-DPO, first fine-tunes the guider with Direct Preference Optimization on preference pairs sampled from both the target and guider models. Experiments on AIME2024, AIME2025, AMC23, and a MATH-hard subset with Qwen2.5-32B as target report average pass@1 improvements of 26% and 29% over the base model for the two variants, plus a 13% improvement when the guider is an RFT-trained 1.5B model. The paper includes ablations on guidance strength, warm-up, budget forcing, and DPO data construction.

Significance. If the reported results hold, the paper demonstrates a cheap and practical way to recover a substantial share of long-reasoning gains from full fine-tuning by transferring a contrastive logit direction from a small reasoning model to a larger frozen model. The strengths are the simplicity of the method, the public code release, the consistent direction of the gains across all four datasets for the SFT guider, and the budget-forcing control, which credibly rules out the trivial explanation that the gains come from output length alone. The DPO ablations comparing pair types and SFT alternatives are informative. However, as detailed below, the central transfer claim needs a direction control, and the headline numbers currently lack statistical support because alpha is selected on the test sets and no confidence intervals are reported. With those additions the contribution would be solid.

major comments (3)
  1. [Section 3 (definition of ˜ℓ_{t+1})] The paper's central transfer claim is not tested against any direction control. The method is equivalent to adding alpha times the contrastive vector Δ = ℓ(S*) − ℓ(S) to the target logits, and the interpretation that this 'encodes the probability shift that turns a short-CoT model into a long-CoT one' presupposes that this direction transfers across architectures. The ablations in Tables 2 and 3 vary alpha, warm-up, and DPO data, but none removes or inverts the subtraction, replaces Δ with a same-norm random vector, or compares against simply adding ℓ(S*) without subtracting ℓ(S). The non-monotonic alpha sweep in Table 2 (on AMC23, alpha=0.5 gives 36.6 and alpha=1.5 gives 41.9, both below the 57.2 baseline) makes it especially important to show that the effect is due to the semantic direction rather than to a narrow perturbation magnitude. Adding these controls is necessary to support the abstract's claim that logit arithmetic elicits long reasoning.
  2. [Section 4.1 / Table 2] The guidance strength alpha=1 is selected on the evaluation sets themselves. The ablation in Table 2 is reported on AIME2025 and AMC23, which also appear in the main results in Table 1, and no validation split or nested selection procedure is described. The headline improvement of 26%/29% therefore partly reflects test-set selection of alpha. Please report how alpha and T_warmup were chosen, or perform selection on a hold-out set and report the corresponding test results.
  3. [Table 1] No error bars, confidence intervals, or significance tests are reported, despite the fact that AIME2024 and AIME2025 contain only 30 problems each and AMC23 contains 40 problems. A single pass@1 difference of a few problems (e.g., AIME2024: 14.6 vs 22.5) can be within sampling noise, especially with 8 samples per problem. The paper should report bootstrap or exact binomial confidence intervals, and ideally multiple decoding seeds, before claiming that the 26%/29% relative improvement is robust.
minor comments (4)
  1. [Table 1] The 'Average' column for pass@8 omits MATH-hard, so the pass@8 averages are over three datasets rather than four; the text should state this explicitly to avoid implying a four-dataset average.
  2. [Section 4.1 / Appendix A.2] Section 4.1 says 10K preference pairs are randomly selected from a total of 50K, while Appendix A.2 reports 11,974 + 43,209 = 55,183 pairs; these numbers should be reconciled.
  3. [Equation (1)] The relationship between lambda and the description 'directly concatenating datasets as DPO training data' could be clarified; if lambda = |D1|/(|D1|+|D2|), the weighted sum equals the average over the union, but this equivalence is not stated.
  4. [Section 2] The related-work discussion says THINKLOGIT 'sidesteps these constraints entirely' compared with representation-engineering methods, but THINKLOGIT still requires white-box logit access to the base and guider small models; the sentence should be softened.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the logit delta is derived from external small-model outputs and evaluated on held-out benchmarks, with no fitted parameter renamed as a prediction.

full rationale

The paper's derivation chain is self-contained: the guiding delta l(S_star)-l(S) is computed from two small external models, not fitted to the target's outputs or to the evaluation benchmarks, and it is added to the frozen target logits as a token-level intervention. The DPO variant trains the guider on preference pairs labeled by final-answer correctness on the MATH training split (level 4-5), which is disjoint from the evaluation sets, including MATH-hard from MATH500 (a test-set subset); thus the reported pass@1 gains are measured rather than forced by the training objective. The guidance strength alpha=1 is a hyperparameter; the ablation in Table 2 is a robustness check and at most a test-set selection effect, not a fit that makes the main results equivalent to its inputs. No load-bearing self-citation or imported uniqueness theorem appears: the method is explicitly positioned as following external logit-arithmetic work (Liu et al., 2024) and is compared against external fine-tuned baselines (s1.1-32B, R1-Distill-Qwen-32B). The absence of a direction-control ablation (e.g., replacing the delta with a random or negative vector) is an experimental limitation concerning attribution, not circularity. Therefore no circular step is present.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim does not introduce new physical entities. It rests on the transferability of logit deltas across model scales, a shared tokenizer, and the use of final-answer correctness as a training signal.

free parameters (2)
  • alpha (guidance strength) = 1.0
    Chosen by ablation on AIME2025 and AMC23 (Table 2); using 0.5 or 1.5 lowers pass@1. Tuning on the reported test sets is a selection risk.
  • Twarmup (warm-up length) = 100 tokens
    Chosen by hand to stabilize decoding; the paper notes applying guidance at every step causes repetitive outputs, and warm-up improves accuracy. No sweep is reported.
assumptions (3)
  • domain assumption Logit-space transferability: the delta ℓ(S⋆) − ℓ(S) from small models can be added to the large model's logits to induce the same behavioral shift.
    Section 3, definition of ℓ̃; the entire method rests on this. The DPO variant is an attempt to reduce the mismatch.
  • domain assumption Tokenizers/vocabularies are aligned across the models so logits are directly comparable.
    Section 4.1: target is Qwen2.5-32B, guider based on Qwen2.5-Math-1.5B; the paper does not discuss tokenizer compatibility, which is required for the addition.
  • domain assumption Final-answer correctness is a sufficient label for reasoning quality when building DPO preference pairs.
    Appendix A.2: labels are computed by exact match with the gold answer, and these labels are used for both Type-1 and Type-2 pairs.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Logit Arithmetic Elicits Long Reasoning Capabilities Without Training." pith.science (2026). https://pith.science/paper/2EAINMEF

@misc{pith2026250712759,
  author       = {Pith},
  title        = {Pith review of: Logit Arithmetic Elicits Long Reasoning Capabilities Without Training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2EAINMEF}},
  note         = {Machine review of arXiv:2507.12759}
}
read the original abstract

Large reasoning models (LRMs) can do complex reasoning via long chain-of-thought (CoT) involving cognitive strategies such as backtracking and self-correction. Recent studies suggest that some models inherently possess these long reasoning abilities, which may be unlocked via extra training. Our work first investigates whether we can elicit such behavior without any training. To this end, we propose a decoding-time approach, ThinkLogit, which utilizes logits arithmetic (Liu et al., 2024) to tune a target large LM for long reasoning using a substantially smaller model as guider. We then show that we can further boost performance by training the guider model with preference optimization over correct/incorrect reasoning pairs sampled from both the target and guider model -- a setup we refer to as ThinkLogit-DPO. Our experiments demonstrate that ThinkLogit and ThinkLogit-DPO achieve a relative improvement in pass@1 by 26% and 29%, respectively, over four mathematical datasets using the Qwen2.5-32B when guided by R1-Distill-Qwen-1.5B -- a model 21x smaller. Lastly, we show that ThinkLogit can transfer long reasoning skills acquired through reinforcement learning, improving pass@1 by 13% relative compared to the Qwen2.5-32B base model. Our work presents a computationally-efficient method to elicit long reasoning in large models with minimal or no additional training.

Figures

Figures reproduced from arXiv: 2507.12759 by the authors.

Figure 1
Figure 1. Overview of our THINKLOGIT and THINKLOGIT-DPO approaches to elicit long chain-of-thought reasoning from a large pre-trained model. complete reasoning traces and scoring them after they are generated, which both raises costs and keeps them in the short-CoT regime. Auxiliary-model approaches modify the output of a frozen target model on the fly. Contrastive decoding subtracts logits from an “amateur” model or layer to… view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Video Reasoning without Training

    cs.CV 2025-10 conditional novelty 6.0 of 10

    An entropy-guided, inference-time value-cache controller improves video reasoning accuracy and cuts output tokens versus RL-trained baselines.

Reference graph

Works this paper leans on

62 extracted references · 17 canonical work pages · cited by 1 Pith paper

  1. [1]

    Bradley C. A. Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V. Le, Christopher R \' e , and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling. CoRR, abs/2407.21787, 2024. doi:10.48550/ARXIV.2407.21787. URL https://doi.org/10.48550/arXiv.2407.21787

    arXiv doi:10.48550/arxiv.2407.21787 2024

  2. [2]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pond \' e de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bava...

    arXiv 2021

  3. [3]

    Do NOT think that much for 2+3=? on the overthinking of o1-like llms

    Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. Do NOT think that much for 2+3=? on the overthinking of o1-like llms. CoRR, abs/2412.21187, 2024. doi:10.48550/ARXIV.2412.21187. URL https://doi.org/10.48550/arXiv.2412.21187

    arXiv doi:10.48550/arxiv.2412.21187 2024

  4. [4]

    Glass, and Pengcheng He

    Yung - Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R. Glass, and Pengcheng He. Dola: Decoding by contrasting layers improves factuality in large language models. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net, 2024. URL https://openreview.net/forum?id=Th6NyL07na

    work page 2024

  5. [5]

    Training verifiers to solve math word problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. CoRR, abs/2110.14168, 2021. URL https://arxiv.org/abs/2110.14168

    arXiv

  6. [6]

    Reinforcement learning for reasoning in small llms: What works and what doesn't

    Quy-Anh Dang and Chris Ngo. Reinforcement learning for reasoning in small llms: What works and what doesn't. arXiv preprint arXiv:2503.16219, 2025

    arXiv 2025

  7. [7]

    DeepSeek - AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai...

    arXiv doi:10.48550/arxiv.2501.12948 2025

  8. [8]

    On giant's shoulders: Effortless weak to strong by dynamic logits fusion

    Chenghao Fan, Zhenyi Lu, Wei Wei, Jie Tian, Xiaoye Qu, Dangyang Chen, and Yu Cheng. On giant's shoulders: Effortless weak to strong by dynamic logits fusion. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang (eds.), Advances in Neural Information Processing Systems 38: Annual Conference on N...

    work page 2024

Show all 62 references
  1. [9]

    On giant's shoulders: Effortless weak to strong by dynamic logits fusion

    Chenghao Fan, Zhenyi Lu, Wei Wei, Jie Tian, Xiaoye Qu, Dangyang Chen, and Yu Cheng. On giant's shoulders: Effortless weak to strong by dynamic logits fusion. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024 b

    2024

  2. [10]

    Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D. Goodman. Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars. CoRR, abs/2503.01307, 2025. doi:10.48550/ARXIV.2503.01307. URL https://doi.org/10.48550/arX...

    2025 doi arXiv

  3. [11]

    Measuring mathematical problem solving with the MATH dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. In Joaquin Vanschoren and Sai - Kit Yeung (eds.), Proceedings of the Neural Information Processing...

    2021

  4. [12]

    Courville, Alessandro Sordoni, and Rishabh Agarwal

    Arian Hosseini, Xingdi Yuan, Nikolay Malkin, Aaron C. Courville, Alessandro Sordoni, and Rishabh Agarwal. V-star: Training verifiers for self-taught reasoners. CoRR, abs/2402.06457, 2024. doi:10.48550/ARXIV.2402.06457. URL https://doi.org/10.48550/arXiv.2402.06457

    2024 doi arXiv

  5. [13]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022 ....

    2022

  6. [14]

    Huang, Wenxuan Zhou, Fei Wang, Fred Morstatter, Sheng Zhang, Hoifung Poon, and Muhao Chen

    James Y. Huang, Wenxuan Zhou, Fei Wang, Fred Morstatter, Sheng Zhang, Hoifung Poon, and Muhao Chen. Offset unlearning for large language models. Trans. Mach. Learn. Res., 2025, 2025. URL https://openreview.net/forum?id=A4RLpHPXCu

    2025

  7. [15]

    GRACE: discriminator-guided chain-of-thought reasoning

    Muhammad Khalifa, Lajanugen Logeswaran, Moontae Lee, Honglak Lee, and Lu Wang. GRACE: discriminator-guided chain-of-thought reasoning. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, Decembe...

    2023 doi

  8. [16]

    Process reward models that think

    Muhammad Khalifa, Rishabh Agarwal, Lajanugen Logeswaran, Jaekyeom Kim, Hao Peng, Moontae Lee, Honglak Lee, and Lu Wang. Process reward models that think. CoRR, abs/2504.16828, 2025. doi:10.48550/ARXIV.2504.16828. URL https://doi.org/10.48550/arXiv.2504.16828

    2025 doi

  9. [17]

    Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D

    Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V. Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luc...

    2024 doi arXiv

  10. [18]

    Fast inference from transformers via speculative decoding

    Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative decoding. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), International Conference on Machine Learning, ICML 2023...

    2023

  11. [19]

    Patil, Matei Zaharia, Joseph E

    Dacheng Li, Shiyi Cao, Tyler Griggs, Shu Liu, Xiangxi Mo, Eric Tang, Sumanth Hegde, Kourosh Hakhamaneshi, Shishir G. Patil, Matei Zaharia, Joseph E. Gonzalez, and Ion Stoica. Llms can easily learn to reason from demonstrations structure, not content, is what matters! CoRR, abs...

    2025 doi arXiv

  12. [20]

    Contrastive decoding: Open-ended text generation as optimization

    Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, and Mike Lewis. Contrastive decoding: Open-ended text generation as optimization. In Anna Rogers, Jordan L. Boyd - Graber, and Naoaki Okazaki (eds.), Proceedings of the...

    2023 doi

  13. [21]

    Reward-guided speculative decoding for efficient LLM reasoning

    Baohao Liao, Yuhui Xu, Hanze Dong, Junnan Li, Christof Monz, Silvio Savarese, Doyen Sahoo, and Caiming Xiong. Reward-guided speculative decoding for efficient LLM reasoning. CoRR, abs/2501.19324, 2025. doi:10.48550/ARXIV.2501.19324. URL https://doi.org/10.48550/arXiv.2501.19324

    2025 doi arXiv

  14. [22]

    Let's verify step by step

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-1...

    2024

  15. [23]

    Smith, and Yejin Choi

    Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A. Smith, and Yejin Choi. Dexperts: Decoding-time controlled text generation with experts and anti-experts. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59...

    2021

  16. [24]

    Alisa Liu, Xiaochuang Han, Yizhong Wang, Yulia Tsvetkov, Yejin Choi, and Noah A. Smith. Tuning language models by proxy. volume abs/2401.08565, 2024. doi:10.48550/ARXIV.2401.08565. URL https://doi.org/10.48550/arXiv.2401.08565

    2024 doi arXiv

  17. [25]

    Understanding r1-zero-like training: A critical perspective

    Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding r1-zero-like training: A critical perspective. arXiv preprint arXiv:2503.20783, 2025

    2025 arXiv

  18. [26]

    Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica

    Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y. Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica. Deepscaler: Surpassing o1-preview with a 1.5b model by scaling rl. Notion Blog https://pretty-radio-b75.notion.site/DeepSca...

    2025

  19. [27]

    Eric Mitchell, Rafael Rafailov, Archit Sharma, Chelsea Finn, and Christopher D. Manning. An emulator for fine-tuning large language models using small language models. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 20...

    2024

  20. [28]

    Cand \` e s, and Tatsunori Hashimoto

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei - Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel J. Cand \` e s, and Tatsunori Hashimoto. s1: Simple test-time scaling. CoRR, abs/2501.19393, 2025. doi:10.48550/ARXIV.2501.19393. URL https://...

    2025 doi arXiv

  21. [29]

    Learning to reason with LLMs , September 2024

    OpenAI . Learning to reason with LLMs , September 2024. URL https://openai.com/index/learning-to-reason-with-llms/

    2024

  22. [30]

    OpenAI o3 and o4-mini System Card

    OpenAI. OpenAI o3 and o4-mini System Card . https://cdn.openai.com/pdf/2221c875-02dc-4789-800b-e7758f3722c1/o3-and-o4-mini-system-card.pdf, April 2025. Accessed: 2025-07-05

    2025

  23. [31]

    Comblm: Adapting black-box language models through small fine-tuned models

    Aitor Ormazabal, Mikel Artetxe, and Eneko Agirre. Comblm: Adapting black-box language models through small fine-tuned models. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023...

    2023 doi

  24. [32]

    Qwen3: Think Deeper, Act Faster | Qwen , April 2025

    Qwen Team . Qwen3: Think Deeper, Act Faster | Qwen , April 2025. URL https://qwenlm.github.io/blog/qwen3/

    2025

  25. [33]

    Manning, Stefano Ermon, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (e...

    2023

  26. [34]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. CoRR, abs/2402.03300, 2024. doi:10.48550/ARXIV.2402.03300. URL https://doi.org/10.4...

    2024 doi arXiv

  27. [35]

    Trusting your evidence: Hallucinate less with context-aware decoding

    Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Wen - tau Yih. Trusting your evidence: Hallucinate less with context-aware decoding. In Kevin Duh, Helena G \' o mez - Adorno, and Steven Bethard (eds.), Proceedings of the 2024 Conference of the Nor...

    2024

  28. [36]

    Scaling LLM test-time compute optimally can be more effective than scaling model parameters

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM test-time compute optimally can be more effective than scaling model parameters. CoRR, abs/2408.03314, 2024. doi:10.48550/ARXIV.2408.03314. URL https://doi.org/10.48550/arXiv.2408.03314

    2024 doi arXiv

  29. [37]

    Unlocking general long chain-of-thought reasoning capabilities of large language models via representation engineering

    Xinyu Tang, Xiaolei Wang, Zhihao Lv, Yingqian Min, Wayne Xin Zhao, Binbin Hu, Ziqi Liu, and Zhiqiang Zhang. Unlocking general long chain-of-thought reasoning capabilities of large language models via representation engineering. arXiv preprint arXiv:2503.11314, 2025

    2025 arXiv

  30. [38]

    Qwq-32b: Embracing the power of reinforcement learning, March 2025

    Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025. URL https://qwenlm.github.io/blog/qwq-32b/

    2025

  31. [39]

    Gram: A generative foundation reward model for reward generalization

    Chenglong Wang, Yang Gan, Yifu Huo, Yongyu Mu, Qiaozhi He, Murun Yang, Bei Li, Tong Xiao, Chunliang Zhang, Tongran Liu, et al. Gram: A generative foundation reward model for reward generalization. arXiv preprint arXiv:2506.14175, 2025 a

    2025

  32. [40]

    Math-shepherd: Verify and reinforce llms step-by-step without human annotations

    Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. Math-shepherd: Verify and reinforce llms step-by-step without human annotations. In Lun - Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeti...

    2024 doi

  33. [41]

    Chain-of-thought reasoning without prompting

    Xuezhi Wang and Denny Zhou. Chain-of-thought reasoning without prompting. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang (eds.), Advances in Neural Information Processing Systems 38: Annual Conference on Neura...

    2024

  34. [42]

    Le, Ed H

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali,...

    2023

  35. [43]

    Reinforcement learning for reasoning in large language models with one training example

    Yiping Wang, Qing Yang, Zhiyuan Zeng, Liliang Ren, Lucas Liu, Baolin Peng, Hao Cheng, Xuehai He, Kuan Wang, Jianfeng Gao, et al. Reinforcement learning for reasoning in large language models with one training example. arXiv preprint arXiv:2504.20571, 2025 b

    2025 arXiv

  36. [44]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Ad...

    2022

  37. [45]

    Self-evaluation guided beam search for reasoning

    Yuxi Xie, Kenji Kawaguchi, Yiran Zhao, James Xu Zhao, Min - Yen Kan, Junxian He, and Michael Qizhe Xie. Self-evaluation guided beam search for reasoning. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Infor...

    2023

  38. [46]

    Redstar: Does scaling long-cot data unlock better slow-reasoning systems? CoRR, abs/2501.11284, 2025

    Haotian Xu, Xing Wu, Weinong Wang, Zhongzhi Li, Da Zheng, Boyuan Chen, Yi Hu, Shijia Kang, Jiaming Ji, Yingying Zhang, Zhijiang Guo, Yaodong Yang, Muhan Zhang, and Debing Zhang. Redstar: Does scaling long-cot data unlock better slow-reasoning systems? CoRR, abs/2501.11284, 202...

    2025 doi arXiv

  39. [47]

    Qwen2.5 technical report

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    2024 doi arXiv

  40. [48]

    Qwen2.5-math technical report: Toward mathematical expert model via self-improvement

    An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang Ren, and Zhenru Zhang. Qwen2.5-math technical report: Toward mathematical expert model via se...

    2024 doi arXiv

  41. [49]

    Speculative thinking: Enhancing small-model reasoning with large model guidance at inference time

    Wang Yang, Xiang Yue, Vipin Chaudhary, and Xiaotian Han. Speculative thinking: Enhancing small-model reasoning with large model guidance at inference time. CoRR, abs/2504.12329, 2025. doi:10.48550/ARXIV.2504.12329. URL https://doi.org/10.48550/arXiv.2504.12329

    2025 doi arXiv

  42. [50]

    LIMO: less is more for reasoning

    Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. LIMO: less is more for reasoning. CoRR, abs/2502.03387, 2025. doi:10.48550/ARXIV.2502.03387. URL https://doi.org/10.48550/arXiv.2502.03387

    2025 doi arXiv

  43. [51]

    DAPO: an open-source LLM reinforcement learning system at scale

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wa...

    2025 doi arXiv

  44. [52]

    Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model? arXiv preprint arXiv:2504.13837, 2025 a

    Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Shiji Song, and Gao Huang. Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model? arXiv preprint arXiv:2504.13837, 2025 a

    2025 arXiv

  45. [53]

    Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model? CoRR, abs/2504.13837, 2025 b

    Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, and Gao Huang. Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model? CoRR, abs/2504.13837, 2025 b . doi:10.48550/ARXIV.2504.13837. URL https://doi.org/10.4...

    2025 doi arXiv

  46. [54]

    Generative verifiers: Reward modeling as next-token prediction

    Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. Generative verifiers: Reward modeling as next-token prediction. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025 . Open...

    2025

  47. [55]

    Small language models need strong verifiers to self-correct reasoning

    Yunxiang Zhang, Muhammad Khalifa, Lajanugen Logeswaran, Jaekyeom Kim, Moontae Lee, Honglak Lee, and Lu Wang. Small language models need strong verifiers to self-correct reasoning. In Lun - Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Comput...

    2024 doi

  48. [56]

    Weak-to-strong jailbreaking on large language models

    Xuandong Zhao, Xianjun Yang, Tianyu Pang, Chao Du, Lei Li, Yu - Xiang Wang, and William Yang Wang. Weak-to-strong jailbreaking on large language models. CoRR, abs/2401.17256, 2024. doi:10.48550/ARXIV.2401.17256. URL https://doi.org/10.48550/arXiv.2401.17256

    2024 doi arXiv

  49. [57]

    Activation control for efficiently eliciting long chain-of-thought ability of language models

    Zekai Zhao, Qi Liu, Kun Zhou, Zihan Liu, Yifei Shao, Zhiting Hu, and Biwei Huang. Activation control for efficiently eliciting long chain-of-thought ability of language models. arXiv preprint arXiv:2505.17697, 2025

    2025 arXiv

  50. [58]

    Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J

    Andy Zou, Long Phan, Sarah Li Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann - Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J....

    doi arXiv

  51. [59]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  52. [60]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  53. [61]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  54. [62]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.