Pith. sign in

REVIEW 3 major objections 5 minor 31 references

Unleashing the True Potential of LLMs: A Feedback-Triggered Self-Correction with Long-Term Multipath Decoding

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that LLM self-correction succeeds when regeneration is triggered by external feedback rather than by the model's own critique, and that multipath decoding with minimum-perplexity selection carries the improvement.

desk verdict A practical feedback-triggered regeneration idea with a mostly known multipath decoding twist, but the headline comparison is rigged in FTR's favor and the PPL-based answer selection is unvalidated. read the letter →

arxiv 2509.07676 v1 pith:A4322OW7 submitted 2025-09-09 cs.AI

Jipeng Li , Zeyu Gao , Yubin Qi , Hande Dong , Weijian Chen , Qiang Lin This is my paper · ORCID
classification cs.AI
keywords self-correctionfeedback-triggeredregenerationlong-termmultipathdecodingperplexity-basedselectionmathematicalreasoningcodegenerationstrategieslargelanguagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that self-correction of large language models fails because the model itself is a poor judge of its own answers, and because next-token decoding cannot look far enough ahead to repair a bad reasoning path. To fix this, it proposes Feedback-Triggered Regeneration (FTR), which regenerates a response only when negative user feedback arrives and, in that second pass, decodes with Long-Term Multipath (LTM) sampling that keeps multiple candidate paths and picks the final answer by lowest perplexity. On math word problems (GSM8K, MultiArith) and code generation (HumanEval), using six open-source models from 1B to 13B parameters, FTR reports consistent 10–20% accuracy gains over the initial output and over two prompt-based self-correction baselines. The practical stake is that deployed assistants could improve their answers without the instability that self-evaluation prompts introduce.

What carries the argument

Long-Term Multipath (LTM) decoding is the load-bearing mechanism: it maintains a dynamic set of $k_i$ candidate sequences at each step, chosen as the smallest set whose cumulative probability satisfies $\sum_{j=0}^{k_i} P(s_i^j) \geq p^* \sum_{j=0}^{k_{i-1}V-1} P(s_i^j)$, subject to a hard cap $k^*$, and the final answer is selected by minimum perplexity $PPL_i = P(s_i)^{-1/(i+1)}$ over the surviving paths. FTR is the other half: user feedback is a binary signal that either accepts the initial output or triggers regeneration of the original prompt with LTM, so the model never receives a corrective prompt that could bias it.

What would settle it

Run LTM over a held-out set of GSM8K or HumanEval problems, record every generated candidate with its perplexity, and compare the accuracy of the minimum-perplexity answer against a majority vote over final answers and against a random candidate; if a wrong answer with lower perplexity than the correct answer is found, or if non-PPL selection does as well or better, the paper's claimed source of gain is contradicted.

Watch

Extended reading notes

Core claim

The central discovery is that two-stage self-correction works better when the second stage is triggered externally rather than by the model's own judgment. Prompt-based correction with Critic prompts and If-or-Else prompts frequently turns correct initial answers into wrong ones; FTR avoids this by using user feedback as a binary 'regenerate' trigger and re-running the original input without any corrective prompt. In the regeneration stage, LTM decoding replaces greedy next-token search: at each step it keeps the smallest set of candidate sequences whose cumulative probability exceeds a threshold, caps the set at $k^*$, and after generating all candidates emits the answer with the minimum perplexity, defined as the inverse geometric mean of token probabilities. In the reported experiments, FTR improves accuracy substantially over initial outputs and over both prompt-based baselines, and LTM alone outperforms greedy, beam search, and adaptive decoding in single-turn generation.

Load-bearing premise

The method assumes that the candidate answer the model finds most fluent, as measured by the lowest perplexity, is also the one most likely to be correct.

Editorial extensions

If this is right

  • FTR consistently outperforms both the initial output and the two prompt-based correction baselines on GSM8K, MultiArith, and HumanEval across six open models from 1B to 13B parameters, with Protocol 1 gains of about 10–20%.
  • Using feedback only as a regeneration trigger beats embedding the feedback in a corrective prompt, indicating that the corrective prompt itself causes much of the degradation seen in prompt-based self-correction.
  • LTM decoding alone, without any self-correction wrapper, beats greedy decoding, fixed-width beam search, and adaptive top-k decoding on the same benchmarks.
  • Because regeneration is skipped for accepted answers, FTR's total compute is about $N \times t \times (1 + p \times n)$ instead of the baselines' $2 \times N \times t$, making it cheaper than mandatory double-pass correction when the regeneration rate stays below roughly 54%.
  • A noisy automated proxy for human feedback (GPT-4o) is enough to reproduce most of the gains, suggesting the method does not depend on perfect or oracle error signals.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next experiment is to replace minimum-PPL selection inside LTM with a lightweight task verifier (running unit tests for code, or exact answer checking for arithmetic); the paper does not report this ablation, and the gains could be larger or smaller depending on how well perplexity tracks correctness.
  • The efficiency advantage over double-pass baselines depends on negative feedback being relatively rare; in applications where users reject most outputs, the multipath regeneration cost would approach or exceed the 2× cost of the prompt-based baselines.
  • The adaptive-width pruning in LTM can be viewed as a randomized beam search whose width reacts to the flatness of the token distribution, so the reported gains over fixed-width beam search may come substantially from that adaptivity rather than from 'long-term' evaluation per se.
  • FTR's trigger design suggests a direct deployment test: log real thumbs-up/thumbs-down signals over a few weeks and measure whether accuracy on accepted versus regenerated answers shifts in the direction the paper's protocols predict.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Feedback-Triggered Regeneration (FTR): when user feedback marks an LLM output as incorrect, the model regenerates from the original prompt using Long-Term Multipath (LTM) decoding, a tree-search decoder that prunes candidate sequences by cumulative probability and selects the final answer by minimum perplexity. Experiments on GSM8K, MultiArith, and HumanEval with six open-weight LLMs compare FTR with Critic Prompt and IoE Prompt self-correction baselines under a ground-truth-triggered protocol and a GPT-4o proxy protocol, and separately compare LTM with greedy, beam, nucleus, and adaptive decoding. The paper reports consistent gains of 10%–20% over the baselines and argues that prompt-based self-correction often degrades initial accuracy.

Significance. If the results hold, the feedback-triggered design makes a useful conceptual point: it replaces the model's unreliable self-assessment with an external trigger, preserving correct initial answers and focusing computation on incorrect ones. The LTM decoder is also an interesting alternative to fixed-width beam search, with an explicit mechanism for dynamically widening the beam at uncertain steps. The paper has concrete strengths: it evaluates six model sizes, includes a human-feedback proxy with reported false-positive/negative rates, and provides an inference-time analysis and a case study. However, the headline comparison in Protocol 1 is structurally unfair, the LTM hyperparameters appear to be selected on the test sets, and the min-PPL selection rule is unvalidated as a correctness criterion. These issues are load-bearing for the central claims and must be addressed before the results can be accepted.

major comments (3)
  1. [§5.1, Table 1 (Protocol 1)] The Protocol 1 comparison is structurally unfair and the reported gains over the initial output are guaranteed by the protocol. FTR is given the ground-truth label to decide when to regenerate, whereas the Critic and IoE baselines must self-assess every output. Since FTR by construction never regenerates a correct initial answer, its accuracy is initial-correct plus corrected-incorrect, which cannot fall below the Initial Input score; the baselines can and do degrade correct answers. For example, on Llama2-7B GSM8K, Initial Input is 0.206, Critic drops to 0.171, IoE drops to 0.136, and FTR rises to 0.360. The gain of FTR over Initial Input is therefore not an empirical discovery about the method's quality, and the 10%–20% claim in Section 5.1 does not by itself validate FTR. The paper should compare FTR against baselines that receive the same external trigger (as Protocol 2 attempts), and should report FTR with standard nucleus sampling under the same trigger to isolate the contribution of LTM.
  2. [§5.3, Table 6] The LTM hyperparameters p* and k* appear to be tuned per dataset and per model on the test sets. Table 6 reports different (p*, k*) pairs for every model-dataset combination, and the same chosen values are used to produce the results in Tables 1 and 2. The paper does not describe a validation split, a tuning procedure, or a sensitivity analysis, and no variance or confidence intervals are reported. This is particularly concerning because several LTM gains are small (e.g., 0.852 vs 0.851 for Qwen-3B on GSM8K in Table 2); test-set selection of two free parameters could account for such differences. The authors should tune on held-out data, report sensitivity to p* and k*, and provide multiple-seed results with confidence intervals.
  3. [§3.2, Eq. (2)–(5), Table 2] The central decoding contribution of LTM rests on an unvalidated assumption that lower perplexity selects the correct answer. The final answer is selected by minimum PPL evaluation (Section 3.2), and the pruning thresholds in Eqs. (4) and (5) operate on sequence probabilities; on mathematical reasoning and code generation, token likelihood is not established as a correctness signal. A verbose, fluent but incorrect chain of thought can easily receive a lower PPL than a terse correct answer. Table 2 compares LTM only against likelihood-based decoding baselines and never against a correctness-aware selection rule such as self-consistency majority voting over sampled paths. The authors should provide evidence that min-PPL selection recovers correct answers on these tasks (e.g., a correlation analysis between PPL and correctness, or an ablation comparing min-PPL selection with majority voting); without this, the LTM contribution to FTR's gains is unsupported.
minor comments (5)
  1. [Eq. (1)] The text says the probability is the product of the likelihoods of the first i tokens, but the formula includes y_0 through y_i, so it should refer to the first i+1 tokens.
  2. [Figure 4] The figure caption contains the undefined acronym "LTPD" in the illustration; this appears to be a typo for LTM and should be fixed.
  3. [§A.2, Table 3] The efficiency discussion should be reconciled with Table 3: FTR's measured inference time exceeds the 2× cost of the Critic/IoE baselines in most cells (e.g., Llama2-7B on HumanEval is 3.87× vs 2×), while the text claims "net efficiency gains in low error rate scenarios"; the condition p*n < 1 is not enough given the reported numbers and should be restated with the actual measured overheads.
  4. [Abstract and §1] The phrase "state-of-the-art prompt-based self-correction methods" overstates the comparison, since only two prompting baselines (Critic Prompt and IoE Prompt) are evaluated; the authors should either add more baselines or soften the wording.
  5. [References] Reference [30] contains a typo in the author name "Amir Globersons" and should be corrected to "Amir Globerson".

Circularity Check

1 steps flagged · score 6.0 of 10

Protocol 1's FTR gains over Initial are oracle-gated by construction; the LTM mechanism and Protocol 2 comparisons provide independent evidence.

  1. self definitional [Section 3.1 (Stage 2) and Section 5.1 (Protocol 1)]
    "FTR simulates supervised feedback by leveraging ground-truth labels to identify initially incorrect answers and trigger regeneration. ... If user feedback indicates that the LLM’s output yinit is problematic, the original prompt x and an advanced decoding strategy, LTM, are employed to regenerate the output. Otherwise, no further action is taken. ... Under Protocol 1, FTR achieves substantial performance gains (10%–20%) consistently across all evaluated scenarios."

    Under Protocol 1, the correctness label is both the regeneration trigger and the accuracy metric. Because FTR’s Stage 2 rule keeps correct outputs unchanged ('Otherwise, no further action is taken'), final accuracy equals initial accuracy plus the correction success rate on initially wrong answers; a non-negative gain over Initial is therefore guaranteed by the protocol rather than discovered. The reported 10%–20% magnitude is empirical, but the existence and structure of the improvement are definitional. The prompt-based baselines in Protocol 1 are not gated and can turn correct answers wrong, so FTR’s margin over them conflates the oracle gate with any intrinsic advantage of the method.

full rationale

The only identified circular step is the Protocol 1 evaluation, where FTR's use of ground-truth labels to trigger regeneration makes its improvement over Initial partly self-definitional. I do not find a self-citation chain: the cited self-correction and decoding baselines are external, and no uniqueness theorem is imported from the authors' own prior work. The LTM min-PPL selection (Eq. 2) is a monotone transformation of sequence likelihood, and whether PPL tracks correctness on math or code is an empirical validity concern, not a circularity, because the paper does not claim to derive PPL from correctness. Similarly, per-dataset hyperparameters in Table 6 raise an in-sample reporting concern, but the paper frames them as computational-budget standardization, not as fitting to test labels; without explicit tuning evidence, I do not count that as a circular step. Protocol 2 applies the same GPT-4o trigger to all methods, so the FTR-versus-baseline comparison there is not gated unfairly by construction. Overall, the LTM decoding comparison (Table 2) and Protocol 2 results are independent evidence; only the Protocol 1 headline gain over Initial is partially forced by the protocol, yielding a score of 6.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central claim rests on two unvalidated assumptions: PPL as a selection criterion, and the reliability of the feedback trigger. The only tuned parameters are the LTM thresholds p* and k*; no new physical or model entities are introduced.

free parameters (2)
  • p* (LTM pruning threshold) = 0.8 to 0.95 per model and dataset (Table 6)
    Controls the cumulative probability threshold for retaining candidate sequences; tuned separately for each model and dataset with no validation split reported.
  • k* (LTM candidate cap) = 7 or 8 per dataset (Table 6)
    Maximum number of retained sequences per decoding step; tuned per model and dataset.
assumptions (5)
  • ad hoc to paper Perplexity (PPL) is a valid proxy for answer quality
    Section 2.1 and 3.2: LTM selects the final output by minimum PPL. The paper provides no evidence that lower PPL correlates with correctness on reasoning tasks; long correct solutions may have lower probability.
  • domain assumption User feedback is accurate enough to serve as a reliable regeneration trigger
    Underlies FTR. Protocol 2 uses GPT-4o as a proxy with false-negative rates up to 34.9% and false-positive rates up to 35.7% (Table 4), so real feedback would be noisier; still FTR shows gains in the proxy setup.
  • domain assumption The language model's initial response to a well-formed query is generally optimal
    Section 3.1 states 'properly trained LLMs generally produce optimal initial responses to well-formed queries,' which motivates preserving initially correct outputs; this is asserted, not justified.
  • standard math Autoregressive token probability factorization (Eq. 1)
    Standard chain rule of probability for autoregressive language models; used to compute sequence probabilities P(s).
  • ad hoc to paper Cumulative probability thresholding retains the most promising paths
    Algorithmic heuristic in Eqs. (4)-(5); no theoretical guarantee that these paths contain the correct answer; the authors motivate it with a single case study.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Unleashing the True Potential of LLMs: A Feedback-Triggered Self-Correction with Long-Term Multipath Decoding." pith.science (2026). https://pith.science/paper/A4322OW7

@misc{pith2026250907676,
  author       = {Pith},
  title        = {Pith review of: Unleashing the True Potential of LLMs: A Feedback-Triggered Self-Correction with Long-Term Multipath Decoding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/A4322OW7}},
  note         = {Machine review of arXiv:2509.07676}
}
read the original abstract

Large Language Models (LLMs) have achieved remarkable performance across diverse tasks, yet their susceptibility to generating incorrect content during inference remains a critical unsolved challenge. While self-correction methods offer potential solutions, their effectiveness is hindered by two inherent limitations: (1) the absence of reliable guidance signals for error localization, and (2) the restricted reasoning depth imposed by conventional next-token decoding paradigms. To address these issues, we propose Feedback-Triggered Regeneration (FTR), a novel framework that synergizes user feedback with enhanced decoding dynamics. Specifically, FTR activates response regeneration only upon receiving negative user feedback, thereby circumventing error propagation from faulty self-assessment while preserving originally correct outputs. Furthermore, we introduce Long-Term Multipath (LTM) decoding, which enables systematic exploration of multiple reasoning trajectories through delayed sequence evaluation, effectively overcoming the myopic decision-making characteristic of standard next-token prediction. Extensive experiments on mathematical reasoning and code generation benchmarks demonstrate that our framework achieves consistent and significant improvements over state-of-the-art prompt-based self-correction methods.

Figures

Figures reproduced from arXiv: 2509.07676 by the authors.

Figure 1
Figure 1. The percentage distribution of answer changes induced by self-correction using If or Else(IoE) Prompts [ [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. (a) Framework of the prompt-based self-correction approach. (b) Framework of our feedback-triggered [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Comparison of different self-correction methods. (a) Self-assessment and update; (b) Revision with user [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Illustration of LTM decoding strategies ( [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Performance comparison of different user feedback utilization approaches. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Comparison of decoding processes between Beam Search (fixed beam width=3) and LTM on a MultiArith [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 15 canonical work pages

  1. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023. 10 Running Title for Header

    arXiv 2023

  3. [3]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    arXiv 2025

  4. [4]

    Llm lies: Hallucinations are not bugs, but features as adversarial examples.arXiv preprint arXiv:2310.01469, 2023

    Jia-Yu Yao, Kun-Peng Ning, Zhen-Hui Liu, Mu-Nan Ning, Yu-Yang Liu, and Li Yuan. Llm lies: Hallucinations are not bugs, but features as adversarial examples.arXiv preprint arXiv:2310.01469, 2023

    arXiv 2023

  5. [5]

    Exploring and evaluating hallucinations in llm-powered code generation.arXiv preprint arXiv:2404.00971, 2024

    Fang Liu, Yang Liu, Lin Shi, Houkun Huang, Ruifeng Wang, Zhen Yang, Li Zhang, Zhongqi Li, and Yuchi Ma. Exploring and evaluating hallucinations in llm-powered code generation.arXiv preprint arXiv:2404.00971, 2024

    arXiv 2024

  6. [6]

    Towards mitigating LLM hallucina- tion via self reflection

    Ziwei Ji, Tiezheng Yu, Yan Xu, Nayeon Lee, Etsuko Ishii, and Pascale Fung. Towards mitigating LLM hallucina- tion via self reflection. In Houda Bouamor, Juan Pino, and Kalika Bali, editors,Findings of the Association for Computational Linguistics: EMNLP, 2023

    work page 2023

  7. [7]

    Self-refine: Iterative refinement with self-feedback.Advances in Neural Information Processing Systems, 2024

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback.Advances in Neural Information Processing Systems, 2024

    work page 2024

  8. [8]

    Language models can solve computer tasks.Advances in Neural Information Processing Systems, 2024

    Geunwoo Kim, Pierre Baldi, and Stephen McAleer. Language models can solve computer tasks.Advances in Neural Information Processing Systems, 2024

    work page 2024

Show all 31 references
  1. [9]

    Confidence matters: Revisiting intrinsic self-correction capabilities of large language models.arXiv preprint arXiv:2402.12563, 2024

    Loka Li, Zhenhao Chen, Guangyi Chen, Yixuan Zhang, Yusheng Su, Eric Xing, and Kun Zhang. Confidence matters: Revisiting intrinsic self-correction capabilities of large language models.arXiv preprint arXiv:2402.12563, 2024

    2024 arXiv

  2. [10]

    Iterative translation refinement with large language models

    Pinzhen Chen, Zhicheng Guo, Barry Haddow, and Kenneth Heafield. Iterative translation refinement with large language models. InProceedings of the 25th Annual Conference of the European Association for Machine Translation (Volume 1), EAMT 2024, 2024

    2024

  3. [11]

    Large language models cannot self-correct reasoning yet

    Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. Large language models cannot self-correct reasoning yet. InThe Twelfth International Conference on Learning Representations, ICLR 2024, 2024

    2024

  4. [12]

    Perplexity—a measure of the difficulty of speech recognition tasks.The Journal of the Acoustical Society of America, 62(S1):S63–S63, 1977

    Fred Jelinek, Robert L Mercer, Lalit R Bahl, and James K Baker. Perplexity—a measure of the difficulty of speech recognition tasks.The Journal of the Acoustical Society of America, 62(S1):S63–S63, 1977

    1977

  5. [13]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Nee- lakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners.Advances in Neural Information Processing Systems 33: Annual Conference on N...

    2020

  6. [14]

    Take care of your prompt bias! investigating and mitigating prompt bias in factual knowledge extraction

    Ziyang Xu, Keqin Peng, Liang Ding, Dacheng Tao, and Xiliang Lu. Take care of your prompt bias! investigating and mitigating prompt bias in factual knowledge extraction. In Nicoletta Calzolari, Min-Yen Kan, Véronique Hoste, Alessandro Lenci, Sakriani Sakti, and Nianwen Xue, edi...

    2024

  7. [15]

    Self-reflection outcome is sensitive to prompt construction.arXiv preprint arXiv:2406.10400, 2024

    Fengyuan Liu, Nouar AlDahoul, Gregory Eady, Yasir Zaki, Bedoor AlShebli, and Talal Rahwan. Self-reflection outcome is sensitive to prompt construction.arXiv preprint arXiv:2406.10400, 2024

    2024 arXiv

  8. [16]

    Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    arXiv

  9. [17]

    Solving general arithmetic word problems

    Subhro Roy and Dan Roth. Solving general arithmetic word problems. In Lluís Màrquez, Chris Callison-Burch, Jian Su, Daniele Pighin, and Yuval Marton, editors,Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, 2015

    2015

  10. [18]

    Evaluating large language models trained on code

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021

    2021 arXiv

  11. [19]

    Instructeval: Towards holistic evaluation of instruction-tuned large language models.arXiv preprint arXiv:2306.04757, 2023

    Yew Ken Chia, Pengfei Hong, Lidong Bing, and Soujanya Poria. Instructeval: Towards holistic evaluation of instruction-tuned large language models.arXiv preprint arXiv:2306.04757, 2023

    2023 arXiv

  12. [20]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  13. [21]

    Qwen2.5 technical report.arXiv preprint arXiv:2412.15115, 2024

    An Yang, Baosong Yang, and Beichen Zhang et al. Qwen2.5 technical report.arXiv preprint arXiv:2412.15115, 2024. 11 Running Title for Header

    2024 arXiv

  14. [22]

    The curious case of neural text degeneration

    Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. In 8th International Conference on Learning Representations, ICLR 2020, 2020

    2020

  15. [23]

    Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  16. [24]

    Measuring mathematical problem solving with the MATH dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. InProceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, Neu...

    2021

  17. [25]

    Human or llm? A comparative study on accessible code generation capability.CoRR, abs/2503.15885, 2025

    Hyunjae Suh, Mahan Tafreshipour, Sam Malek, and Iftekhar Ahmed. Human or llm? A comparative study on accessible code generation capability.CoRR, abs/2503.15885, 2025

    2025 arXiv

  18. [26]

    Angela Fan, Mike Lewis, and Yann N. Dauphin. Hierarchical neural story generation. In Iryna Gurevych and Yusuke Miyao, editors,Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, 2018

    2018

  19. [27]

    Learning to write with cooperative discriminators

    Ari Holtzman, Jan Buys, Maxwell Forbes, Antoine Bosselut, David Golub, and Yejin Choi. Learning to write with cooperative discriminators. In Iryna Gurevych and Yusuke Miyao, editors,Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, 2018

    2018

  20. [28]

    Varshney

    Sourya Basu, Govardana Sachitanandam Ramachandran, Nitish Shirish Keskar, and Lav R. Varshney. Mirostat: a neural text decoding algorithm that directly controls perplexity. In9th International Conference on Learning Representations, ICLR 2021, 2021

    2021

  21. [29]

    Improving open-ended text generation via adaptive decoding

    Wenhong Zhu, Hongkun Hao, Zhiwei He, Yiming Ai, and Rui Wang. Improving open-ended text generation via adaptive decoding. InForty-first International Conference on Machine Learning, ICML 2024, 2024

    2024

  22. [30]

    Chain-of-thought reasoning without prompting

    Xuezhi Wang and Denny Zhou. Chain-of-thought reasoning without prompting. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang, editors,Advances in Neural Information Processing Systems 38: Annual Conference on Neur...

    2024

  23. [31]

    Deductive beam search: Decoding deducible rationale for chain-of-thought reasoning.arXiv preprint arXiv:2401.17686, 2024

    Tinghui Zhu, Kai Zhang, Jian Xie, and Yu Su. Deductive beam search: Decoding deducible rationale for chain-of-thought reasoning.arXiv preprint arXiv:2401.17686, 2024. A Appendices A.1 Case Study In this section, we present a case study comparing LTM with Beam Search to demonst...

    2024 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.