Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Cautious Next Token Prediction

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read CNTP is a training-free decoding strategy that branches at uncertain tokens and keeps the lowest-perplexity continuation, claiming consistent gains over greedy and nucleus decoding.

desk verdict A plausible training-free decoding heuristic that overclaims its evidence; the local-PPL selection is well ablated, but the theorem is assumption-bound and the tuning protocol inflates the headline numbers. read the letter →

arxiv 2507.03038 v2 pith:NWWGYTMN submitted 2025-07-03 cs.CL cs.AIcs.LG

Yizhou Wang , Lingzhi Zhang , Yue Bai , Mang Tik Chiu , Zhengmian Hu , Mingyuan Zhang , Qihua Dong , Yu Yin
show 2 more authors
Sohrab Amirghodsi Yun Fu
This is my paper
classification cs.CLcs.AIcs.LG
keywords cautiousnexttokenpredictiondecodingstrategyentropy-guidedsamplingperplexityselectioninference-timeadaptationchain-of-thoughtself-consistencymultimodallanguagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that next-token decoding can be made more accurate without retraining by letting the model explore more exactly where it is unsure. At each decoding step, it computes the entropy of the next-token distribution: low entropy means decode normally, while high entropy triggers several independent short continuations, each ending at the next punctuation mark, and the model keeps the continuation with the lowest perplexity. The number of trials grows with entropy up to a fixed budget, so extra compute is spent only on uncertain steps. Reported experiments on math, commonsense reasoning, truthfulness, and multimodal question answering show gains over greedy decoding and nucleus sampling, and CNTP combined with self-consistency usually beats self-consistency alone. A theorem states that, under two assumptions, the probability of producing the exact correct sequence is never worse than single-sample decoding and strictly better whenever CNTP branches.

What carries the argument

The load-bearing object is the local trial path: a short continuation sampled from a high-entropy step and truncated at the next punctuation token. CNTP scores each trial path by its mean negative log-likelihood, or perplexity, and appends the lowest-scoring path to the running sequence. The trial count is an entropy-controlled gate, $N = \max\!\left(1,\; \min\!\left(N_{\max},\; \tfrac{H-H_{\min}}{H_{\max}-H_{\min}}N_{\max}\right)\right)$, so the method is a runtime policy that switches between greedy single-token decoding and local multi-sample search depending on model confidence. This mechanism is what lets CNTP spend computation on uncertain branches while preserving the speed and coherence of ordinary decoding elsewhere.

What would settle it

On a benchmark with known correct answers, record each high-entropy step where the correct continuation appears among the sampled trials and check whether the lowest-perplexity rule selects it. If the correct continuation is chosen no more often than random selection among the candidates, Assumption 1 fails and the claimed gain should disappear.

Watch

Extended reading notes

Core claim

The paper's central claim is that a language model can police its own uncertainty during decoding: sample ahead only when the predictive distribution flattens, and trust the branch the model itself scores as most probable. The proposed algorithm treats a punctuation-bounded continuation as a local decision unit. At each step it computes the entropy $H(s)$ of the token distribution, sets the trial count $N$ by a clamped linear ramp between thresholds $H_{\min}$ and $H_{\max}$, draws $N$ independent continuations ending at punctuation, scores each by $\mathrm{PPL}(s_i)=\exp\!\left(-\tfrac1{|s_i|}\sum_t \log p_\theta(w_t\mid s_{<t})\right)$, and appends the minimum-perplexity continuation. Theorem 1 asserts that when the correct continuation is among the sampled candidates and has strictly the lowest perplexity (Assumption 1) and high entropy means the correct token is unlikely in one draw (Assumption 2), the full-sequence correctness probability satisfies $P_{\mathrm{CNTP}}(\mathrm{correct}) \ge P_{\mathrm{Single}}(\mathrm{correct})$, strict whenever branching occurs, at expected cost $L[1+p(N_{\max}-1)]$ forward passes.

Load-bearing premise

The proof assumes that whenever the correct next chunk of text is among the sampled options, it always scores best on the model's own likelihood measure; the paper offers no direct measurement of how often that ranking holds.

Editorial extensions

If this is right

  • On Llama-3.1-8B-Instruct, CNTP raises GSM8K accuracy from 79.8 to 81.6, MATH from 41.5 to 47.1, and StrategyQA from 72.9 to 73.2 over greedy decoding.
  • On TruthfulQA with Llama-2-7B-Chat, CNTP reaches 84.8 percent truth accuracy and 74.0 percent truth-and-info accuracy, beating both greedy and stochastic decoding.
  • On multimodal benchmarks, CNTP improves accuracy over greedy decoding by up to 5.5 points on MMVet and by 0.4 points on MathVista across Llama-3.2-11B-Vision-Instruct and LLaVA-CoT.
  • CNTP plus 40-path self-consistency exceeds vanilla self-consistency on all three reported reasoning benchmarks with Llama-3.1-8B-Instruct.
  • Expected compute stays at $L[1+p(N_{\max}-1)]$ forward passes, below uniform multi-sample decoding when high-entropy steps are rare.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because CNTP uses only logits and a punctuation set, it is a drop-in wrapper around any autoregressive model; a natural stress test is whether the gains persist on models with better calibrated or deliberately miscalibrated probabilities.
  • The theorem relies on correct branches having lowest perplexity, which is a directly measurable local property; turning that measurement into a design criterion would let practitioners choose where branching helps.
  • The sensitivity analysis shows accuracy can saturate or drop as $N_{\max}$ grows, unlike self-consistency's monotone scaling, suggesting an exploration-exploitation trade-off that could be tuned or annealed per task.
  • A direct head-to-head against the concurrent entropy-based branching decoder noted in the paper is left for future work and would clarify which design choices carry the gain.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces Cautious Next Token Prediction (CNTP), a training-free decoding strategy that monitors next-token entropy and, when entropy is high, samples multiple punctuation-bounded continuation paths and appends the path with the lowest perplexity. The number of trials is scaled with entropy, so extra computation is concentrated on uncertain steps. The authors evaluate CNTP against greedy, stochastic/nucleus, beam search, and self-consistency on GSM8K, MATH, StrategyQA, TruthfulQA, MMVet, and MathVista using several Llama, DeepSeek-R1-distilled, and vision-language models, and report gains in most settings. They also present Theorem 1, which claims that CNTP is provably more likely than single-sample decoding to generate the correct full sequence under Assumptions 1 and 2.

Significance. If the empirical results hold, CNTP would be a simple, practical inference-time method that focuses sampling effort on uncertain steps and can be combined with self-consistency. The paper ships code and includes useful ablations of confidence measures, trial-scaling strategies, and perplexity-computation ranges. However, the formal claim is currently not supported: Assumption 1 essentially assumes the desired selection behavior, and the proof's probabilistic accounting is inconsistent with the multi-token path sampling. The evaluation is weakened by test-set hyperparameter tuning, a non-reproducible 200-question MATH subset, and at least one benchmark cell (Table 3, StrategyQA on DeepSeek-R1-Distill-Qwen-1.5B) where CNTP is below greedy. These issues are fixable but block acceptance in the current form.

major comments (4)
  1. [§3.4 and Appendix B] Assumption 1 states that whenever the ground-truth token or short path is among the sampled candidates, it has the strictly lowest perplexity among incorrect candidates. This is exactly the property that CNTP needs to work, and it is not derived, measured, or argued for. The proof of Theorem 1 then uses this assumption to conclude that once the correct candidate appears, CNTP selects it; without Assumption 1, the inequality PCNTP(correct) ≥ PSingle(correct) can fail. In addition, Eq. (11) treats the probability that a trial contains the correct token as pθ(ct | s<t), but trials are multi-token punctuation-bounded paths: the probability that a trial produces the entire correct path is the product of several token probabilities, and the event At in the proof (token t is correct given previous tokens correct) does not account for later tokens inside the appended path being wrong. Thus Theorem 1, as stated, does not follow from the proof. Please either provide empirical evidence that Assumption 1 holds for local branches (e.g., a measurement of how often the correct continuation is the lowest-PPL candidate in high-entropy steps) or restate the theorem as a conditional statement, and correct the proof's probabilistic model for multi-token paths. Table 11 shows that whole-sequence perplexity selection is worse than greedy, so the low-perplexity-leads-to-correctness premise is non-obvious.
  2. [§4.1 and Appendix C] The experimental protocol tunes hyperparameters on the test sets themselves: temperature is chosen from [0.6, 0.8, 1.0, 1.2] to select the best performance, and beam size is tuned from [2, 3, 5, 10, 20, 40]. Because the same test sets are used for both hyperparameter selection and final result reporting, the reported margins are selection artifacts, and the generality of the method is not established. This is particularly important because the abstract claims a consistent and clear margin, which is a comparative claim. Please re-run the evaluation with hyperparameters chosen on a validation set (or fixed a priori), or at least clearly separate validation and test performance and report how much of the gain survives under fixed hyperparameters.
  3. [Table 3] On StrategyQA with DeepSeek-R1-Distill-Qwen-1.5B, CNTP reports 53.0 ± 1.3, which is below the greedy decoding baseline of 53.6. This single cell directly contradicts the abstract's statement that CNTP outperforms standard decoding strategies consistently by a clear margin. The paper provides no statistical test for this difference, and the overlapping standard errors suggest the result may not be significant. Please reconcile the abstract's claim with this result, e.g., by narrowing the consistency claim to the settings that support it or by reporting a proper significance analysis across all benchmarks.
  4. [§4.1] The MATH evaluation uses only 200 randomly selected test questions, and the paper does not specify the random seed or the exact subset. This makes the reported MATH improvements (e.g., +5.6 over greedy in Table 2) non-reproducible and potentially sensitive to the particular sample drawn. Please provide the seed and subset, or use a fixed public subset, and preferably report the full MATH test set to confirm the magnitude of the gains.
minor comments (5)
  1. [§3.2 and §3.4] The notation is inconsistent: Algorithm 1 uses "s<t" but does not define it, and Definition 1 says "at some step Nmax" when Nmax is also used for the maximum number of trials; this makes the formal part hard to follow. Please clean up the notation.
  2. [Table 1] Table 1 marks CNTP as computationally efficient with a checkmark, which is only true relative to beam search and self-consistency; on high-entropy steps CNTP is several times more expensive than greedy or stochastic decoding. The table should be annotated to clarify this comparison.
  3. [Figure 1] Figure 1 is dense: the trial labels (Trial 1–6) are not clearly mapped to the sampled paths, and the "PPL min" annotations are ambiguous. Please redraw the figure to make the trial selection process clearer.
  4. [Tables 2 and 3] The self-consistency rows do not report standard deviations across the 40 paths or across repeated runs, so it is unclear whether differences such as Ours+SC vs. SC (e.g., 85.2 vs. 84.8 on GSM8K in Table 2) are significant.
  5. [Section 3.4 title] The section title "CNTP is Prone to Lead to the Correct Answer Provably" is informal and unclear; consider renaming to "Theoretical Analysis" or "Provable Improvement under Assumptions".

Circularity Check

1 steps flagged · score 6.0 of 10

Theorem 1's formal correctness claim reduces to Assumption 1, which restates the algorithm's lowest-perplexity selection rule; the empirical benchmark comparisons remain independent.

  1. self definitional [Section 3.4 (Assumption 1, Theorem 1) and Appendix B (proof)]
    "Assumption 1. Whenever the ground-truth token (or short path) c is among the sampled candidates {si}, it attains strictly the lowest perplexity among all incorrect candidates. ... Once ct is among the candidates, Assumption 1 ensures it will be selected due to the lowest perplexity."

    Algorithm 1 selects the candidate with the lowest PPL (line 15: s ← s + arg min_{s_i} PPL(s_i)). The proof then sets PCNTP(A_t) = 1 − (1 − pθ(c_t | s_<t))^{N_t}; this equality is valid only if Assumption 1 guarantees that a sampled correct continuation is always the PPL-minimizing candidate. If any incorrect candidate had lower PPL, CNTP would append the wrong branch and the inequality PCNTP ≥ PSingle could fail. The theorem is a valid conditional statement, but its correctness gain is not an independent derivation: it is the algorithm's lowest-PPL selection rule repackaged as Assumption 1, plus the coverage effect of extra samples. Hence the formal prediction reduces by construction to the unverified premise that the ground-truth continuation has the lowest perplexity.

full rationale

The paper's empirical contribution is largely self-contained: Tables 2, 3, 8, and 9 compare CNTP against greedy, stochastic, beam, and self-consistency decoding on standard benchmarks, and these evaluations do not depend on Theorem 1. The theorem's assumptions are explicitly labeled, and the proof is internally valid as a conditional argument. However, the load-bearing formal claim that CNTP is 'provably' more likely to produce the correct answer is obtained by assuming exactly the property the method needs—that the correct sampled continuation is the lowest-perplexity one. In the proof, the success probability at a high-entropy step is equated with the sampling coverage probability, suppressing the selection mechanism behind Assumption 1. Thus the formal result exhibits partial circularity: the predicted correctness advantage is, by construction, the assumption that the target answer has the lowest perplexity. Separately, Appendix C's test-set hyperparameter tuning (selecting temperature and beam sizes that give the best results) weakens the 'consistently by a clear margin' empirical claim, and Table 3 shows a case where CNTP is below greedy, but that is a methodological concern rather than a derivation-chain circularity. No load-bearing self-citation chain was found. Overall score 6: one central formal prediction reduces by construction, while the benchmark evidence provides independent empirical grounding.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central method depends on three tuned thresholds, per-dataset temperatures, a punctuation set, and a scaling formula. The theoretical guarantee is conditional on two assumptions that essentially state the desired behavior, so the proof adds little independent support. No new physical or conceptual entities are introduced.

free parameters (6)
  • Nmax (maximum trial count) = 10
    Set globally for all experiments; sensitivity analysis in Figure 4 shows performance varies with it, so the choice affects results.
  • Hmin (low entropy threshold) = 0.01
    Below this entropy N=1, above it the model starts multi-trial sampling; chosen by hand, not derived.
  • Hmax (high entropy threshold) = 1.5
    Above this entropy N=Nmax; chosen by hand, not derived, and likely model-dependent.
  • Temperature for CNTP = GSM8K 1.2, MATH 0.6, StrategyQA 0.8, TruthfulQA 0.8, MMVet 0.8, MathVista 0.8
    Tuned per dataset on the evaluation set and the best value reported, as stated in Appendix C.
  • top-p = 0.9 (Llama-based), 0.95 (DeepSeek-R1-distill-Qwen)
    Set to model defaults and also swept in Figure 5; the value influences the candidate pool.
  • Punctuation stop set = {.,?!:;)]}\n}
    Chosen as the trial stopping condition; this set bounds trial length and affects which paths can be selected.
assumptions (4)
  • domain assumption Assumption 1: the ground-truth short path has strictly the lowest perplexity among sampled candidates whenever it is included
    Used in Theorem 1 and as the selection rule in Section 3.2; the paper provides no independent validation of this ranking.
  • domain assumption Assumption 2: high entropy implies the correct token's probability is small
    Used in the proof of Theorem 1 to argue that multiple trials are needed when entropy is high.
  • domain assumption Low perplexity is a reliable proxy for output correctness or reliability
    The method selects the lowest-perplexity continuation and treats it as the best path; this is the core heuristic.
  • domain assumption The language model's next-token distribution is well-calibrated enough for entropy to indicate uncertainty
    The adaptive branching in Section 3.2 depends on entropy thresholds; the ablation in Figure 3 compares confidence measures but does not calibrate the thresholds.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Cautious Next Token Prediction." pith.science (2026). https://pith.science/paper/NWWGYTMN

@misc{pith2026250703038,
  author       = {Pith},
  title        = {Pith review of: Cautious Next Token Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NWWGYTMN}},
  note         = {Machine review of arXiv:2507.03038}
}
read the original abstract

Next token prediction paradigm has been prevailing for autoregressive models in the era of LLMs. The current default sampling choice for popular LLMs is temperature scaling together with nucleus sampling to balance diversity and coherence. Nevertheless, such approach leads to inferior performance in various NLP tasks when the model is not certain about testing questions. To this end, we propose a brand new training-free decoding strategy, dubbed as Cautious Next Token Prediction (CNTP). In the decoding process, if the model has comparatively high prediction entropy at a certain step, we sample multiple trials starting from the step independently and stop when encountering any punctuation. Then we select the trial with the lowest perplexity score viewed as the most probable and reliable trial path given the model's capacity. The trial number is negatively correlated with the prediction confidence, i.e., the less confident the model is, the more trials it should sample. This is consistent with human beings' behaviour: when feeling uncertain or unconfident, one tends to think more creatively, exploring multiple thinking paths, to cautiously select the path one feels most confident about. Extensive experiments on both LLMs and MLLMs show that our proposed CNTP approach outperforms existing standard decoding strategies consistently by a clear margin. Moreover, the integration of CNTP with self consistency can further improve over vanilla self consistency. We believe our proposed CNTP has the potential to become one of the default choices for LLM decoding. Code is available at https://github.com/wyzjack/CNTP.

Figures

Figures reproduced from arXiv: 2507.03038 by the authors.

Figure 1
Figure 1. Overview of Cautious Next Token Prediction (CNTP) method. Given a question in the testing phase, the AR model predict next tokens in an iterative way. Different from traditional one-token-by-one-token generation style, we leverage the prediction entropy of the token probability vectors and sample multiple trials to look ahead if the entropy is high enough. The different trials are sampled independently and stop when… view at source ↗
Figure 2
Figure 2. One QA example of Truthful-QA. When using random sampling with temperature scaling and nucleus [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 4
Figure 4. Performance curves of CNTP when scaling up the max trial numbers on GSM8K and StrategyQA datasets using Llama-3.1-8B-Instruct [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Accuracy of CNTP using Llama-3.1-8B￾Instruct on MATH w.r.t. temperature and top p values. tion strategy. We further try Best-of-N sampling using the whole perplexity of the generated an￾swers (including the complete CoT path). The best path is selected based on the low…
Figure 3
Figure 3. Figure 3: Comparison between [PITH_FULL_IMAGE:figures/full_fig_p008_3.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Entropy-Guided Loop: Achieving Reasoning through Uncertainty-Aware Generation

    cs.AI 2025-08 conditional novelty 5.0 of 10

    A lightweight entropy-triggered refinement loop improves a small LLM's answer quality to roughly 95% of a reasoning model's, at about one-third the cost.

Reference graph

Works this paper leans on

49 extracted references · 13 canonical work pages · cited by 1 Pith paper

  1. [1]

    Pranjal Aggarwal, Aman Madaan, Yiming Yang, and Mausam . 2023. Let ' s sample step by step: Adaptive-consistency for efficient reasoning and coding with LLM s. In EMNLP

    work page 2023

  2. [2]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877--1901

    2020

  3. [3]

    Xinyun Chen, Renat Aksitov, Uri Alon, Jie Ren, Kefan Xiao, Pengcheng Yin, Sushant Prakash, Charles Sutton, Xuezhi Wang, and Denny Zhou. 2023. Universal self-consistency for large language model generation. arXiv preprint arXiv:2311.17311

    arXiv 2023

  4. [4]

    Zhawnen Chen, Tianchun Wang, Yizhou Wang, Michal Kosinski, Xiang Zhang, Yun Fu, and Sheng Li. 2024. Through the theory of mind's eye: Reading minds with multimodal video large language models. arXiv preprint arXiv:2406.13763

    arXiv 2024

  5. [5]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

    arXiv 2021

  6. [6]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    arXiv 2024

  7. [7]

    David Farr, Iain Cruickshank, Nico Manzonelli, Nicholas Clark, Kate Starbird, and Jevin West. 2024. Llm confidence evaluation measures in zero-shot css classification. arXiv preprint arXiv:2410.13047

    arXiv 2024

  8. [8]

    Hao Fei, Shengqiong Wu, Wei Ji, Hanwang Zhang, Meishan Zhang, Mong-Li Lee, and Wynne Hsu. 2024. Video-of-thought: Step-by-step video reasoning from perception to cognition. In Forty-first International Conference on Machine Learning

    work page 2024

Show all 49 references
  1. [9]

    Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. 2021. Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies. Transactions of the Association for Computational Linguistics, 9:346--361

    2021

  2. [10]

    Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. 2023. Critic: Large language models can self-correct with tool-interactive critiquing. arXiv preprint arXiv:2305.11738

    2023 arXiv

  3. [11]

    Alex Graves. 2012. Sequence transduction with recurrent neural networks. arXiv preprint arXiv:1211.3711

    2012 arXiv

  4. [12]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    2025 arXiv

  5. [13]

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874

    2021 arXiv

  6. [14]

    Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2019. The curious case of neural text degeneration. arXiv preprint arXiv:1904.09751

    2019 arXiv

  7. [15]

    Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. 2023. Large language models cannot self-correct reasoning yet. arXiv preprint arXiv:2310.01798

    2023 arXiv

  8. [16]

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. NeurIPS, 35:22199--22213

    2022

  9. [17]

    Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D Co-Reyes, Avi Singh, Kate Baumli, Shariq Iqbal, Colton Bishop, Rebecca Roelofs, et al. 2024. Training language models to self-correct via reinforcement learning. arXiv preprint arXiv:2409.12917

    2024 arXiv

  10. [18]

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th Symposium on Operating Systems Princip...

    2023

  11. [19]

    Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, et al. 2024. T " ulu 3: Pushing frontiers in open language model post-training. arXiv preprint arXiv:2411.15124

    2024 arXiv

  12. [20]

    Yaniv Leviathan, Matan Kalman, and Yossi Matias. 2023. Fast inference from transformers via speculative decoding. In International Conference on Machine Learning, pages 19274--19286. PMLR

    2023

  13. [21]

    Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, Limin Wang, and Yu Qiao. 2024. Mvbench: A comprehensive multi-modal video understanding benchmark. In CVPR

    2024

  14. [22]

    Stephanie Lin, Jacob Hilton, and Owain Evans. 2021. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.07958

    2021 arXiv

  15. [23]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024. Improved baselines with visual instruction tuning. In CVPR

    2024

  16. [24]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. In NeurIPS

    2023

  17. [25]

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. 2023. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255

    2023 arXiv

  18. [26]

    Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Khan. 2024. Video- C hat GPT : Towards detailed video understanding via large vision and language models. In ACL

    2024

  19. [27]

    Minh Nguyen, Andrew Baker, Clement Neo, Allen Roush, Andreas Kirsch, and Ravid Shwartz-Ziv. 2024. Turning up the heat: Min-p sampling for creative and coherent llm outputs. arXiv preprint arXiv:2407.01082

    2024

  20. [28]

    OpenAI. 2023. Gpt-4 technical report. CoRR, abs/2303.08774

    2023 arXiv

  21. [29]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. pages 8748--8763. PMLR

    2021

  22. [30]

    Xuan Shen, Yizhou Wang, Xiangxi Shi, Yanzhi Wang, Pu Zhao, and Jiuxiang Gu. 2025. Efficient reasoning with hidden thinking. arXiv preprint arXiv:2501.19201

    2025 arXiv

  23. [31]

    Enxin Song, Wenhao Chai, Guanhong Wang, Yucheng Zhang, Haoyang Zhou, Feiyang Wu, Haozhe Chi, Xun Guo, Tian Ye, Yanting Zhang, Yan Lu, Jenq-Neng Hwang, and Gaoang Wang. 2024. Moviechat: From dense token to sparse memory for long video understanding. In CVPR

    2024

  24. [32]

    Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. 2024. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.06525

    2024 arXiv

  25. [33]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. 2025. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599

    2025 arXiv

  26. [34]

    Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. 2024. Visual autoregressive modeling: Scalable image generation via next-scale prediction. arXiv preprint arXiv:2404.02905

    2024 arXiv

  27. [35]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023 a . Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    2023 arXiv

  28. [36]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023 b . Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  29. [37]

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171

    2022 arXiv

  30. [38]

    Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023 a . Self-consistency improves chain of thought reasoning in language models. In ICLR

    2023

  31. [39]

    Yizhou Wang, Ruiyi Zhang, Haoliang Wang, Uttaran Bhattacharya, Yun Fu, and Gang Wu. 2023 b . Vaquita: Enhancing alignment in llm-assisted video understanding. arXiv preprint arXiv:2312.02310

    2023 arXiv

  32. [40]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824--24837

    2022

  33. [41]

    xjdr alt. 2024. Entropix: Entropy based sampling and parallel cot decoding. https://github.com/xjdr-alt/entropix. Accessed: 2025-07-21

    2024

  34. [42]

    Guowei Xu, Peng Jin, Li Hao, Yibing Song, Lichao Sun, and Li Yuan. 2024. Llava-o1: Let vision language models reason step-by-step. arXiv preprint arXiv:2411.10440

    2024 arXiv

  35. [43]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. 2024. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115

    2024 arXiv

  36. [44]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2024. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36

    2024

  37. [45]

    Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. 2023. Mm-vet: Evaluating large multimodal models for integrated capabilities. arXiv preprint arXiv:2308.02490

    2023 arXiv

  38. [46]

    Mingyuan Zhang, Yue Bai, Huan Wang, Yizhou Wang, Qihua Dong, and Yun Fu. 2025. Boosting large language models with mask fine-tuning. arXiv preprint arXiv:2503.22764

    2025

  39. [47]

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2024. Mini GPT -4: Enhancing vision-language understanding with advanced large language models. In ICLR

    2024

  40. [48]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  41. [49]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.