Pith. sign in

REVIEW 5 major objections 7 minor 42 references

HARP: Hesitation-Aware Reframing in Transformer Inference Pass

T0 review · 5 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read By letting a language model 'reconsider' when it is uncertain, a training-free tweak raises accuracy by up to 5.16% at modest extra cost.

desk verdict HARP is a training-free entropy-gated embedding-dropout inference trick with modest real accuracy gains, but its efficiency claim is tied to a no-KV-cache setup and its hyperparameters are tuned on the test tasks. read the letter →

arxiv 2412.07282 v2 pith:23XIM2IY submitted 2024-12-10 cs.CL cs.AIcs.LG

Romain Storaï , Seung-won Hwang This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords adaptivecomputationShannonentropyembeddingdropoutuncertaintyestimationTransformerinferencetraining-freedecodinglargelanguagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that off-the-shelf Transformer language models can be made more accurate without any retraining by letting them spend a little extra computation exactly when they are uncertain. The method, HARP, measures the Shannon entropy of the next-token distribution and, for high-entropy steps, runs a second forward pass on embeddings that have been partially dropped out, then averages the two predictions. Across three model families and five benchmarks the authors report consistent gains, the largest being +5.16% on LAMBADA, with an average inference slowdown of about 1.25x compared to greedy decoding—roughly two and a half times faster than beam search. If correct, this makes token-level uncertainty a cheap, model-agnostic switch for adaptive computation.

What carries the argument

The central mechanism is an entropy-gated second forward pass. Shannon entropy of the softmaxed next-token distribution is the hesitation signal; dropout applied to the embedding matrix at rate δ creates the 'reframed' input; and a convex combination with β=0.5 fuses the two logit distributions. The gate is what makes the added computation selective, and the dropout is what makes the second pass a different perspective rather than a duplicate.

What would settle it

Replace the Shannon-entropy trigger with a random trigger that fires at the same rate and run HARP on the same benchmarks; if the accuracy gain persists, the entropy gate is not the cause of the improvement.

Watch

Extended reading notes

Core claim

HARP is a plug-in modification to the Transformer forward pass that operates per generated token. The model first computes its normal logits and converts them to a probability distribution; if the Shannon entropy of that distribution lies above a threshold θ, the token is deemed uncertain and the model 'hesitates'. The embeddings of the input are then passed through dropout with rate δ, a complete second forward pass is executed, and the original and reframed logits are merged with a convex combination β. The authors set θ=1.0, δ=0.20, and β=0.5, and report accuracy improvements of up to +5.16% on five datasets spanning reasoning, multiple-choice, next-word prediction, and summarization, using LLaMA-3.1 8B, Mistral 7B, and Phi-3.5 Mini. They interpret the second pass as a reframing of the input—an inference-time analogue of the human framing effect—and argue that the entropy gate is what directs extra compute to genuinely hard decisions.

Load-bearing premise

The method's gains rest on the premise that zeroing out 20% of embedding values and averaging the two resulting predictions is a genuinely informative reframing for high-entropy tokens, rather than a benefit that any small stochastic perturbation could supply at the same gate frequency.

Editorial extensions

If this is right

  • Any off-the-shelf Transformer can be upgraded by swapping in the modified forward pass; no gradient updates, no fine-tuning, and no architectural changes are needed.
  • Greedy decoding with HARP outperforms beam search on most of the tested tasks while running about 2.5x faster, so it offers a better accuracy-latency trade-off.
  • HARP composes with nucleus sampling and Chain-of-Thought prompting, producing additional gains on top of those techniques.
  • The added cost is selective: on average the slowdown is about 1.25x, but the worst-case cost for a sequence of entirely uncertain tokens is nearly a full second forward pass per token.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A control experiment that triggers the second pass on randomly chosen tokens at the same rate as the entropy gate would isolate whether the uncertainty signal itself, rather than the extra averaged forward pass, is responsible for the gains.
  • The fixed hyperparameters (θ=1.0, δ=0.20) were tuned on a subset of the same benchmarks, so a natural test is whether they transfer to new tasks; an adaptive threshold per example would be a plausible improvement.
  • The authors note that reframing could be applied to deeper representations or to the key-value cache, which would avoid the cache invalidation and VRAM spike they identify as the main practical limitation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. HARP proposes a training-free, model-agnostic modification to Transformer inference: when the Shannon entropy of the next-token distribution exceeds a threshold theta, the model performs an extra forward pass with dropout-perturbed embeddings and linearly averages the original and reframed logits (Eq. 4). The method is evaluated on five tasks (CsQA, GSM8K, LAMBADA, MMLU Pro, CNN/DM) with three instruction-tuned models under greedy and nucleus sampling, reporting accuracy gains up to +5.16% and an average inference-time overhead of about 1.25x over vanilla decoding, which is claimed to be faster than beam search. The paper also includes analyses of the uncertainty gate, multiple reframing steps, a NEFTune-versus-dropout comparison, a threshold sensitivity study, and an extended evaluation with LM-Evaluation-Harness.

Significance. If the empirical claims hold, HARP would be a simple and useful adaptive-computation method for off-the-shelf LLMs, requiring no retraining and no architectural changes. The paper has genuine strengths: the method is conceptually simple, the code is promised to be public, and Appendix D provides a welcome attempt at a more standard evaluation with multiple seeds. However, the reported gains are not stable across the main and extended evaluations, the hyperparameters were selected on the same benchmarks that are then used to report results, and the efficiency claim is tied to a no-KV-cache setting that is not representative of standard inference. In its current form, the paper supports a proof of concept rather than the abstract's general claim of 'minimal additional cost' and performance improvements up to 5.16%.

major comments (5)
  1. [Section 3.4 and Section 4.3, with Appendices A and C] The hyperparameters theta=1.0, delta=0.20, and beta=0.5, as well as the choice of dropout over NEFTune noise, were selected using the same five benchmark datasets whose results are then reported as the main findings. This is a form of test-set tuning, so the reported improvements are optimistic and cannot be interpreted as unbiased estimates of generalization. The authors should either use a held-out validation set for hyperparameter selection or present results under a fixed configuration with sensitivity analyses on unseen tasks.
  2. [Algorithm 1, Section 4.3, and Section 8] All timing comparisons in Table 6 and Figure 2 are run "without caching (KVCache)". In standard inference with a KV cache, the reframing pass (Eq. 3) re-embeds the whole sequence and recomputes the full prefix, so each high-entropy token incurs O(L) extra work rather than a constant-factor overhead, and VRAM usage can spike. The Limitations section explicitly concedes that "embedding dropout may temporarily invalidate the KVCache" and that VRAM usage could double. The abstract's "minimal additional cost" and Section 5's "twice faster than beam search" claim are therefore not established for the cached setting; the authors should either re-measure with an actual KV-cache implementation or clearly qualify the claims to the no-cache setting.
  3. [Table 5 versus Table 1] For LLaMA-3.1 Instruct 8B with greedy decoding, Table 1 reports HARP's MMLU Pro score as 48.21 and CNN/DM ROUGE-1 as 34.03, while Appendix A's Table 5 reports 71.43 and 48.21 for the same HARP(DROPOUT) setting. These differences of roughly 14 points are too large to be explained by formatting and appear to be an internal inconsistency. The authors must correct the table or explain the discrepancy before the main results can be trusted.
  4. [Table 9 versus Table 1] The extended LM-Evaluation-Harness results in Appendix D differ dramatically from the main results for the same model: LAMBADA vanilla accuracy is 55.52 in Table 9 versus 30.86 in Table 1, MMLU Pro is 33.30 versus 46.42, and CNN/DM ROUGE-1 is 35.77 versus 32.44. The extended gains are also much smaller (e.g., MMLU Pro +0.26, CNN/DM +0.36). The main experiments use unquantified subsets with a single seed, and the paper does not reconcile these differences. Until the evaluation protocol and scoring are clarified and the two sets of numbers are reconciled, the headline improvement of +5.16% cannot be considered robust.
  5. [Section 5 and Appendix D] Because dropout is applied at inference time, HARP introduces stochasticity even under greedy decoding, yet the main results in Table 1 are reported without error bars. Appendix D reports standard deviations for some extended runs and states that "certain seeds can still negatively impact results," which confirms that the method is noisy. The authors should provide variance estimates, significance tests, and the exact subset sizes and seeds used for all main claims.
minor comments (7)
  1. [Section 2.3] The word "Parallely" should be "In parallel" or "Parallelly".
  2. [Equation (2)] The notation "SHANNON(P) = - |V| X ..." is typeset in a way that obscures the formula; the summation range and the function name should be formatted more clearly.
  3. [Table 2] The formatting of the Mistral CoT row is garbled: "Ours CoT75.23(+4.59) 46.00(-2.00) 32.95(+0.58)" needs proper column separation and model labeling, and the -2.00 drop on GSM8K should be discussed in the text.
  4. [Figure 2 and Figure 3] Both figures contain garbled unicode/rendering artifacts that make them difficult to read; the PDF rendering should be fixed.
  5. [Section 4.3] The paper says a "subset" of each dataset is evaluated but never states the subset sizes; this information is essential for assessing the reliability of the main results.
  6. [Appendix C] Table 8 is labeled "Relative accuracy," but the values appear to be absolute accuracy gains in percentage points; the label should be clarified.
  7. [Section 4.3] The phrase "with a top-k of 5" after beam search parameters is missing a space before "with," and the beam-search length penalty alpha should be stated more explicitly as a hyperparameter that is fixed across datasets.

Circularity Check

2 steps flagged · score 4.0 of 10

Headline LAMBADA gain is tuned on LAMBADA and the perturbation choice is made on the same five test sets, but the uncertainty-gating claim retains independent empirical content.

  1. fitted input called prediction [Section 3.4; Section 4.3; Appendix C (Tables 7, 8); Table 1]
    "In our experiments, we set these parameters to values identified as optimal during preliminary testing. ... The hyperparameters used in our experiments are fixed at a dropout rate of δ = 0.20 and an uncertainty threshold of θ = 1.0. ... We evaluate two datasets with different θ values and report relative accuracy and inference time in Table 7 and Table 8 respectively."

    Appendix C's θ sweep is run on CsQA and LAMBADA, the same two datasets whose Table 1 headline gains are then attributed to HARP. Table 8 shows LAMBADA +5.16 at θ=1.0 and CsQA +1.52 at θ=1.0, and Table 1 reports exactly Ours (Greedy) LAMBADA +5.16 and CsQA +1.52 with θ fixed to 1.0. Thus the flagship numbers are the values at which the threshold was selected, not out-of-sample predictions. The global θ=1.0 is not tuned per task for the other benchmarks, so only part of the result table is affected, but the most prominent improvement is fitted on the benchmark where it is reported.

  2. other [Section 2.3; Appendix A (Table 5); Table 1]
    "We choose to use dropout among them, as it consistently yields the best results. In Appendix A, we demonstrate that dropout performs better than NEFTune's random uniform noise in this context. ... Comparison of NEFTune noise (with hyperparameter α in (5, 10)) and dropout (δ = 20%) in the HARP method using LLaMA-3.1 Instruct 8B with greedy decoding."

    The core reframing mechanism is selected by comparing dropout against NEFTune on exactly the five evaluation datasets (CsQA, GSM8K, LAMBADA, MMLU Pro, CNN/DM) reported in Table 5, and the same five datasets then appear in Table 1 as HARP's benchmark improvements. The choice of dropout is therefore fitted to the test data whose gains are later reported, so those reported gains are not a fully independent evaluation of the perturbation choice. The uncertainty gate and the multi-model generalization remain separately testable, but the selection-on-test weakens the independence of the headline comparison.

full rationale

The paper contains no equation-level circularity: Eq. (4) defines how logits are combined, and no load-bearing claim is justified by a self-citation chain. The central intuition — that uncertainty-gated extra computation helps — is tested by an ablation against unconditional extra steps (Table 3) and by an external-harness evaluation (Appendix D), which give independent empirical content. The circularity burden is limited to benchmark selection: the threshold θ is swept on CsQA and LAMBADA, and the exact values later reported as HARP's headline gains on those tasks (+5.16 LAMBADA, +1.52 CsQA) are the values at which θ was chosen; similarly, the dropout-versus-NEFTune choice is made on the same five datasets whose results are then reported. This makes the flagship numbers partially fitted rather than out-of-sample predictions, but it does not force the multi-model, multi-task pattern or the uncertainty-gating conclusion.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on several unproved modeling choices: entropy as a reliable difficulty signal, dropout as a useful reframing, linear interpolation of logits, and transferability of the tuned hyperparameters to new tasks and models. The paper contributes no theory for these choices, so they are axioms for the method's effectiveness.

free parameters (4)
  • uncertainty threshold theta = 1.0
    Reported as identified as optimal during preliminary testing; no held-out validation, and Appendix C shows accuracy varies with theta on CsQA and LAMBADA.
  • dropout rate delta = 0.20
    Set to 0.20 for all experiments; no sensitivity analysis reported in the paper.
  • combination factor beta = 0.5
    Empirically found to balance original and reframed logits; no sensitivity analysis.
  • beam search length penalty alpha = 0.6
    Used for the beam search baseline; tuned on a subset of three datasets (CNN/DM, GSM8K, LAMBADA), affecting the speed and accuracy comparison.
assumptions (5)
  • domain assumption Shannon entropy of the next-token distribution is a sufficient signal for detecting tokens that benefit from extra computation.
    Section 3.2 adopts entropy as the uncertainty estimator without comparing to other token-level uncertainty measures or justifying the threshold's universality.
  • ad hoc to paper Embedding dropout produces a useful reframing of the input that improves predictions for uncertain tokens.
    Section 3.3 and Eq. 3 borrow the framing-effect intuition from psychology; no mechanism or analysis shows why zeroing 20% of embedding entries creates a beneficial alternative perspective.
  • ad hoc to paper Averaging original and reframed logits linearly (Eq. 4) is a valid way to combine the two perspectives.
    Section 3.4, Eq. 4 with beta=0.5 chosen empirically; no derivation that this combination improves calibration or accuracy.
  • domain assumption Hyperparameters theta=1.0 and delta=0.20, tuned on the evaluated benchmarks, transfer to new tasks and models.
    Appendix C explores theta only on two tasks; transfer to unseen tasks and 70B-scale models is untested.
  • domain assumption Results on INT8-quantized models reflect the behavior of full-precision models.
    Main experiments use INT8 quantization (Section 4.1); the extended FP16 evaluation covers only LLaMA-3.1 8B, so the interaction with quantization is not fully characterized.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HARP: Hesitation-Aware Reframing in Transformer Inference Pass." pith.science (2026). https://pith.science/paper/23XIM2IY

@misc{pith2026241207282,
  author       = {Pith},
  title        = {Pith review of: HARP: Hesitation-Aware Reframing in Transformer Inference Pass},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/23XIM2IY}},
  note         = {Machine review of arXiv:2412.07282}
}
read the original abstract

This paper aims to improve the performance of large language models by addressing the variable computational demands in inference steps, where some tokens require more computational resources than others. We present HARP, a simple modification to "off-the-shelf" Transformer forward pass. Drawing from hesitation and the framing effect in decision-making, HARP selectively applies additional computation when the model encounters uncertainty during token generation. Our method mimics human cognitive processes by pausing at difficult decision points and reframing inputs for a different perspective. Unlike other approaches, HARP is model-agnostic, training-free, and easy to implement. We evaluate our method across various downstream tasks and model sizes, demonstrating performance improvements up to +5.16%. Notably, HARP achieves these gains while maintaining inference times twice faster than beam search. Simple and yet with significant gains, HARP provides insights into the potential of adaptive computation for enhancing the performance of Transformer-based language models.

Figures

Figures reproduced from arXiv: 2412.07282 by the authors.

Figure 1
Figure 1. The left side represents the Transformer’s [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. LLaMA 3.1 Instruct (8B) average relative [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Answer to the given prompt generated using [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Multiple-choice Question prompt (CommonsenseQA and MMLU Pro). [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: GSM8K prompt. LAMBADA Prompt You have to predict the next word of the sentence given the context. Your answer should be a single word. Context: {context (minus one sentence)} Sentence to continue: {last sentence (minus one word)} [PITH_FULL_IMAGE:figures/full_fig_p015…
Figure 6
Figure 6. Figure 6: LAMBADA prompt. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: CNN/DailyMail prompt. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 7 canonical work pages

  1. [1]

    Rajendra Acharya, Vladimir Makarenkov, and Saeid Nahavandi

    Moloud Abdar, Farhad Pourpanah, Sadiq Hussain, Dana Rezazadegan, Li Liu, Mohammad Ghavamzadeh, Paul Fieguth, Xiaochun Cao, Abbas Khosravi, U. Rajendra Acharya, Vladimir Makarenkov, and Saeid Nahavandi. 2020. https://doi.org/10.1016/j.inffus.2021.05.008 A Review of Uncertainty Quantification in Deep Learning : Techniques , Applications and Challenges

    doi:10.1016/j.inffus.2021.05.008 2020

  2. [2]

    Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, and Harkirat Behl et al. 2024. https://arxiv.org/abs/2404.14219 Phi-3 technical report: A highly capable language model locally on your phone . Preprint, arXiv:2404.14219

    arXiv 2024

  3. [3]

    Chinmaya Andukuri, Jan-Philipp Fr \"a nken, Tobias Gerstenberg, and Noah Goodman. 2024. https://openreview.net/forum?id=CrzAj0kZjR ST ar- GATE : Teaching language models to ask clarifying questions . In First Conference on Language Modeling

    2024

  4. [4]

    Arteaga, Thomas B

    Gabriel Y. Arteaga, Thomas B. Schön, and Nicolas Pielawski. 2024. https://arxiv.org/abs/2409.02976 Hallucination detection in llms: Fast and memory-efficient finetuned models . Preprint, arXiv:2409.02976

    arXiv 2024

  5. [5]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, and Amanda et al. Askell. 2020. https://proceedings.neurips.cc/paper_files/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf Language models are few-shot learners . In Advances in Neural Information Processing...

    work page 2020

  6. [6]

    Kakade, Karen Livescu, and Karthik Sridharan

    Kamalika Chaudhuri, Sham M. Kakade, Karen Livescu, and Karthik Sridharan. 2009. https://doi.org/10.1145/1553374.1553391 Multi-view clustering via canonical correlation analysis . In Proceedings of the 26th Annual International Conference on Machine Learning, ICML '09, page 129–136, New York, NY, USA. Association for Computing Machinery

    arXiv 2009

  7. [7]

    Charlie Chen, Sebastian Borgeaud, Geoffrey Irving, Jean-Baptiste Lespiau, Laurent Sifre, and John Jumper. 2023. https://arxiv.org/abs/2302.01318 Accelerating large language model decoding with speculative sampling . Preprint, arXiv:2302.01318

    arXiv 2023

  8. [8]

    Christiano, Jan Leike, Tom B

    Paul F. Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS'17, page 4302–4310, Red Hook, NY, USA. Curran Associates Inc

    2017

Show all 42 references
  1. [9]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. https://arxiv.org/abs/2110.14168 Training verifiers to solve math word problems . ...

    2021 arXiv

  2. [10]

    Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Łukasz Kaiser. 2019. https://arxiv.org/abs/1807.03819 Universal transformers . Preprint, arXiv:1807.03819

    2019 arXiv

  3. [11]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, and Angela Fan et al. 2024. https://arxiv.org/abs/2407.21783 The llama 3 herd of models . Preprint, arXiv:2407.21783

    2024 arXiv

  4. [12]

    Mostafa Elhoushi, Akshat Shrivastava, Diana Liskovich, Basil Hosmer, Bram Wasti, Liangzhen Lai, Anas Mahmoud, Bilge Acun, Saurabh Agarwal, Ahmed Roman, Ahmed Aly, Beidi Chen, and Carole-Jean Wu. 2024. https://doi.org/10.18653/v1/2024.acl-long.681 L ayer S kip: Enabling early e...

    2024 doi

  5. [13]

    Muennighoff

    Leo Gao, Jonathan Tow, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Kyle McDonell, and Niklas et al. Muennighoff. 2021. https://doi.org/10.5281/zenodo.5371628 A framework for few-shot language model evaluation . Zenodo

    2021 doi

  6. [14]

    Sachin Goyal, Ziwei Ji, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv Kumar, and Vaishnavh Nagarajan. 2024. https://openreview.net/forum?id=ph04CRkPdC Think before you speak: Training language models with pause tokens . In The Twelfth International Conference on Learning Rep...

    2024

  7. [15]

    Alex Graves. 2017. https://arxiv.org/abs/1603.08983 Adaptive computation time for recurrent neural networks . Preprint, arXiv:1603.08983

    2017 arXiv

  8. [16]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. https://openreview.net/forum?id=d7KBjmI3GmQ Measuring massive multitask language understanding . In International Conference on Learning Representations

    2021

  9. [17]

    Arian Hosseini, Xingdi Yuan, Nikolay Malkin, Aaron Courville, Alessandro Sordoni, and Rishabh Agarwal. 2024. https://openreview.net/forum?id=stmqBSW2dV V- ST ar: Training verifiers for self-taught reasoners . In First Conference on Language Modeling

    2024

  10. [18]

    Bartoldson, Bhavya Kailkhura, Avi Schwarzschild, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein

    Neel Jain, Ping yeh Chiang, Yuxin Wen, John Kirchenbauer, Hong-Min Chu, Gowthami Somepalli, Brian R. Bartoldson, Bhavya Kailkhura, Avi Schwarzschild, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein. 2024. https://openreview.net/forum?id=0bMmZ3fkCk NEFT une: No...

    2024

  11. [19]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, and Lucile Saulnier et al. 2023. https://arxiv.org/abs/2310.06825 Mistral 7b . Preprint, arXiv:2310.06825

    2023 arXiv

  12. [20]

    Daniel Kahneman. 2012. Thinking, fast and slow. Penguin, London

    2012

  13. [21]

    Jungo Kasai, Keisuke Sakaguchi, Ronan Le Bras, Dragomir Radev, Yejin Choi, and Noah A. Smith. 2024. https://aclanthology.org/2024.lrec-main.7 A call for clarity in beam search: How it works and when it stops . In LREC/COLING, pages 77--90

    2024

  14. [22]

    Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. 2023. https://openreview.net/forum?id=VD-AYtP0dve Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation . In The Eleventh International Conference on Learning Representations

    2023

  15. [23]

    Yaniv Leviathan, Matan Kalman, and Yossi Matias. 2023. Fast inference from transformers via speculative decoding. In Proceedings of the 40th International Conference on Machine Learning, ICML'23. JMLR.org

    2023

  16. [24]

    Yaniv Leviathan, Matan Kalman, and Yossi Matias. 2024. https://arxiv.org/abs/2410.02703 Selective attention improves transformer . Preprint, arXiv:2410.02703

    2024 arXiv

  17. [25]

    Ziqin Luo, Haixia Han, Haokun Zhao, Guochao Jiang, Chengyu Du, Tingyun Li, Jiaqing Liang, Deqing Yang, and Yanghua Xiao. 2024. https://arxiv.org/abs/2405.16552 Sed: Self-evaluation decoding enhances large language models for better generation . Preprint, arXiv:2405.16552

    2024 arXiv

  18. [26]

    Potsawee Manakul, Adian Liusie, and Mark Gales. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.557 S elf C heck GPT : Zero-resource black-box hallucination detection for generative large language models . In Proceedings of the 2023 Conference on Empirical Methods in Natural...

    2023 doi

  19. [27]

    Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, Çağlar Gulçehre, and Bing Xiang. 2016. https://doi.org/10.18653/v1/K16-1028 Abstractive text summarization using sequence-to-sequence RNN s and beyond . In Proceedings of the 20th SIGNLL Conference on Computational Natural Langu...

    2016 doi

  20. [28]

    Denis Paperno, Germ \'a n Kruszewski, Angeliki Lazaridou, Ngoc Quan Pham, Raffaella Bernardi, Sandro Pezzelle, Marco Baroni, Gemma Boleda, and Raquel Fern \'a ndez. 2016. https://doi.org/10.18653/v1/P16-1144 The LAMBADA dataset: Word prediction requiring a broad discourse cont...

    2016 doi

  21. [29]

    Claude Elwood Shannon. 1948. http://plan9.bell-labs.com/cm/ms/what/shannonday/shannon1948.pdf A mathematical theory of communication . The Bell System Technical Journal, 27:379--423

    1948

  22. [30]

    Botvinick, and Jonathan D

    Amitai Shenhav, Matthew M. Botvinick, and Jonathan D. Cohen. 2013. https://doi.org/10.1016/j.neuron.2013.07.007 The expected value of control: An integrative theory of anterior cingulate cortex function . Neuron, 79(2):217--240. Funding Information: This work is supported by t...

    2013 doi

  23. [31]

    Chufan Shi, Haoran Yang, Deng Cai, Zhisong Zhang, Yifan Wang, Yujiu Yang, and Wai Lam. 2024. https://doi.org/10.48550/arXiv.2402.06925 A thorough examination of decoding methods in the era of llms . CoRR, abs/2402.06925

    2024 doi arXiv

  24. [32]

    John Sweller. 1988. https://doi.org/10.1207/s15516709cog1202\_4 Cognitive load during problem solving: Effects on learning . Cognitive Science, 12(2):257--285

    1988 doi

  25. [33]

    Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. https://doi.org/10.18653/v1/N19-1421 C ommonsense QA : A question answering challenge targeting commonsense knowledge . In Proceedings of the 2019 Conference of the North A merican Chapter of the Associa...

    2019 doi

  26. [34]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf Attention is all you need . In Advances in Ne...

    2017

  27. [35]

    Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. 2024. https://arxiv.org/abs/2406.01574 Mmlu-pro: A more robust and...

    2024 arXiv

  28. [36]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2024. Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the 36th International Conference on Neural Information Processi...

    2024

  29. [37]

    Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, and Klaus Macherey et al

    Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, and Klaus Macherey et al. 2016. https://arxiv.org/abs/1609.08144 Google's neural machine translation system: Bridging the gap between human and machine tr...

    2016 arXiv

  30. [38]

    Chang Xu, Dacheng Tao, and Chao Xu. 2013. https://arxiv.org/abs/1304.5634 A survey on multi-view learning . Preprint, arXiv:1304.5634

    2013 arXiv

  31. [39]

    Eric Zelikman, Georges Raif Harik, Yijia Shao, Varuna Jayasiri, Nick Haber, and Noah Goodman. 2024. https://openreview.net/forum?id=oRXPiSOGH9 Quiet- ST ar: Language models can teach themselves to think before speaking . In First Conference on Language Modeling

    2024

  32. [40]

    Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. 2022. https://openreview.net/forum?id=_3ELRdg2sgI ST ar: Bootstrapping reasoning with reasoning . In Advances in Neural Information Processing Systems

    2022

  33. [41]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  34. [42]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.