Pith. sign in

REVIEW 3 major objections 5 minor 88 references

Most lossy verification methods in speculative decoding fall into two mechanisms, and the truncation-based variant induces a distribution that can significantly degrade quality relative to its matched baseline.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 12:00 UTC pith:64O77SQY

load-bearing objection A genuinely useful reframing of lossy verification with a solid empirical core; the formalism overreaches in its mapping to existing implementations, but the main practical conclusion holds. the 3 major comments →

arxiv 2607.26627 v1 pith:64O77SQY submitted 2026-07-29 cs.CL

Revisiting Lossy Verification in Speculative Decoding: Mechanisms, Trade-offs, and Failure Modes

Tianyu Wang , Yuxuan Zhou , Wenbin Wang , Heng Li , Zikai Xiao , Junyuan Shang This is my paper
classification cs.CL
keywords lossy verificationspeculative decodingtruncation-based verificationcollaborative verificationovershoot ceilingmin-p samplingeta-samplingdistributional distortion
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that the apparent diversity of lossy verification methods in speculative decoding hides two underlying mechanisms. Truncation-based verification—accepting any draft token that falls inside a truncation-sampling allowed set—produces the renormalized draft distribution over that set, not the truncated target distribution, so it can underperform the very truncation-sampling baseline it superficially resembles; under EAGLE-3 tree verification the per-token divergence does not vanish even with a perfect draft. Collaborative verification, by contrast, blends draft and target, and the paper argues its quality-preserving ingredient is a specific ceiling: wherever the draft probability exceeds the target probability divided by a lenience factor, generation probability is capped. A reader should care because these claims would change how lossy methods are evaluated and designed: compare against distribution-matched truncation baselines, and when relaxing verification, suppress draft overshoot rather than interpolating uniformly.

Core claim

The paper's central claim is a mechanistic unification: SpecCascade and the 'typical acceptance' variant of Medusa are truncation-based verification, accepting a draft token exactly when it lies in the allowed set of min-p or η-sampling, respectively; CoS and lenience-based relaxation are collaborative verification, replacing the target distribution with a mixture or capped mixture of draft and target. The formal consequence of truncation-based verification is an induced distribution q/ZΘ(q) on the allowed set, and the paper derives the per-token KL divergence from the matched truncation-sampling target in Lemma 2. Proposition 1 shows that under standard speculative decoding this divergence

What carries the argument

The load-bearing object is Definition 1, truncation-based verification: replace the lossless acceptance rule with h(x)=1[x∈AΘ], where AΘ is the allowed set of a truncation-sampling strategy, and reject with no resampling. This gives the induced distribution q(x)/ZΘ(q) on AΘ, which is the renormalized draft distribution and therefore need not match the truncated target pΘ(x)=p(x)/ZΘ(p). Lemma 2 computes the per-token KL between these two distributions for standard SD and for EAGLE-3; Proposition 1 shows that as q→p the standard-SD KL vanishes while the EAGLE-3 KL stays strictly positive. The second key mechanism is the overshoot ceiling in collaborative verification: in lenience-based relaxat

Load-bearing premise

The empirical comparisons presuppose that SpecCascade and Medusa's typical acceptance really compute the min-p and η-sampling allowed sets exactly as formalized in Equations (5), (6), and (28); if their acceptance rules differ in implementation, the induced distributions in Equation (13) and the matched-baseline gaps in Tables 2 and 3 describe a different algorithm.

What would settle it

Measure the per-token KL divergence between the distribution emitted by typical acceptance under EAGLE-3 and the η-sampling truncated target when the draft equals the target (q=p). Proposition 1 predicts this KL is strictly positive (≈E[log ZΘ(p)/p(X)]); if it approaches zero, the predicted non-vanishing gap is wrong. Alternatively, instrument SpecCascade's accept/reject decisions against the formal min-p allowed set and check whether accepted tokens coincide with AΘ; any mismatch means the benchmark comparisons are not about the formalized method.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Truncation-based verification should be compared against its matched truncation-sampling baseline; otherwise apparent gains can be misattributed to the truncation strategy itself.
  • Under EAGLE-3-style tree verification, improving the draft cannot close the quality gap: Proposition 1 predicts a positive per-token KL even for q=p.
  • For collaborative verification, selective overshoot suppression is enough to preserve task quality; uniform draft-target interpolation is not necessary.
  • A combined rule—gate acceptance by a min-p allowed set and cap overshoot outside it—achieved +3.7% block efficiency at unchanged Pass@1 on MBPP+ in the paper's experiments, pointing to a concrete design direction.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the unification holds, the design space for lossy verification collapses to a small set of induced-distribution choices, so new methods can be evaluated by asking which distribution they emit rather than by the name of their acceptance rule.
  • Proposition 1 suggests that any truncation-based lossy verification under tree-based drafting has a quality ceiling set by the allowed set, independent of draft quality; a testable prediction is that better drafts will not improve benchmark scores beyond the matched truncation baseline.
  • The overshoot-ceiling principle might transfer to training: a draft model regularized to avoid probabilities above p/ℓ, or a verification rule that clips only those tokens, could capture most of the speed gain without task loss—though this is not tested in the paper.
  • The paper's classification depends on the exact equivalence between published methods and the formal allowed sets; a source-level check of alternative implementations would either confirm or narrow the scope of the two categories.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a taxonomy of lossy verification methods in speculative decoding: truncation-based verification (Medusa's typical acceptance, SpecCascade) and collaborative verification (CoS, lenience-based relaxation). It derives the distributions induced by these rules, gives KL-divergence characterizations for standard SD and EAGLE-3 tree verification, and conducts benchmark experiments on MATH, MBPP+, INCLUDE, and BFCL. The main empirical claims are that truncation-based verification underperforms its matched truncation-sampling baseline, that this gap widens sharply under EAGLE-3 (Proposition 1), and that for collaborative verification the overshoot ceiling—capping draft probabilities at p/ℓ in the q ≥ p/ℓ region—is the effective quality-preserving mechanism (Section 4, Table 1). The paper closes with an ablation, qualitative failure-mode examples, and an appendix containing extended results and proofs.

Significance. If the results hold, the paper provides an important correction to the evaluation practice of lossy verification: methods such as SpecCascade and Medusa's typical acceptance should be compared against the same truncation sampling applied directly to the target model, not against default decoding. The overshoot-ceiling principle is a concise, actionable design insight, and the ablation isolating this mechanism (Table 1) is clean. The paper also ships code, uses public benchmarks, and gives parameter-free analytical expressions for the induced distributions and KL gaps, which are notable strengths. The main risk is that the formal identification of the two named methods with the paper's truncation sets is asserted rather than verified, and one central lemma's proof is referenced to an appendix section that does not contain it. These issues are fixable but currently load-bearing.

major comments (3)
  1. [§3.2, Definition 1 / Eq. (13); §H.1, Eq. (28)] The claim that SpecCascade uses the min-p allowed set (Eq. 5) and Medusa's typical acceptance uses the η-sampling allowed set (Eq. 6) is asserted but not verified against the original implementations. The internal tension is visible in Eq. (28): τ is defined using the truncated target pΘ, while Eq. (6) defines η-sampling via H(p). If the actual methods compute their allowed sets from p or use a different threshold schedule, the 'matched baseline' comparisons in Tables 2–3 and the KL analysis in Lemma 2/Proposition 1 do not describe those methods. Please provide source-level evidence or an empirical check (e.g., compare the accepted sets on sampled distributions) that the formalized rules coincide with the implementations.
  2. [§5.1, Lemma 1 (Eq. 14); §H] The proof of Lemma 1 is deferred to 'Section H', but Section H contains no proof of Lemma 1; it proves Lemma 2, Lemma 3, Corollary 1, and Proposition 2. Lemma 1 is used to interpret the simulation in Fig. 4 and the benchmark efficiency trends in Fig. 5. A missing proof for this lemma is a gap in the theoretical support. Either supply the proof in the appendix or correct the reference.
  3. [§5.2, Lemma 2 / Proposition 1; §H.3] The derivation of KLEAGLE and the non-vanishing limit in Proposition 1 depend on the specific deterministic tree-verification model in §H.1, where the accepted token is the first candidate in AΘ and draft probabilities play no role. The proof in §H.3 does not make explicit the distribution over the accepted token x with respect to which the expectation in KLEAGLE is taken, nor does it show that as q→p the induced distribution of x does not concentrate on a token with pΘ(x)→1. The current lower bound log(ZΘ(p)/p(x)) > 0 for every x in AΘ does not by itself rule out the expectation converging to zero if the first candidate became the max-probability token. Please clarify the stochastic model and either prove the limit or state the additional assumptions needed.
minor comments (5)
  1. [§C, Table 4 vs §5, Table 2] The row label 'Cascade (Chen et al., 2024)' in Table 4 does not match 'SpecCascade (Narasimhan et al., 2024)' used in the main text. Align the naming and citations across tables.
  2. [§H.1, Eq. (26)] The notation h(x) = min{pΘ(x)/q, 1}|_{q=1} is confusing; explain that under greedy drafting the candidate's proposal probability is treated as 1 for the candidate token, or rewrite the acceptance rule without the q placeholder.
  3. [§5.1, Eq. (6) and Eq. (28)] Equation (6) defines η-sampling with a general δ, but Eq. (28) silently sets δ = √ε. State this substitution explicitly in the main text.
  4. [Figure 1 caption] The terms 'True baseline' and 'Wrong baseline' are used without defining which method is 'SOTA' (SpecCascade). Clarify the caption so the gap directions are unambiguous.
  5. [Limitations] The limitations paragraph states that the theoretical results assume the specific formalized rules; it would be useful to add an explicit admission that the equivalence of those rules to the original implementations has not been checked.

Circularity Check

1 steps flagged

EAGLE-3 'typical acceptance' allowed set is defined via H(pΘ) (Eq. 28), making the matched-baseline comparison and KL_EAGLE prediction self-referential; core SD analysis is independent.

specific steps
  1. self definitional [Appendix H.1, Eq. (28) with Eq. (4) and Section 3.2; Table 2 caption]
    "Typical acceptance. h(x) = min{pΘ(x)/τ,1} = 1[x∈Aη], τ = min(ε, √ε e^{−H(pΘ)}). ... each verification method is compared only with its matched truncation sampling baseline using the same allowed set."

    Eq. (28) defines the typical-acceptance threshold via the entropy of the truncated target, H(pΘ), whereas the η-sampling allowed set in Eq. (6) uses H(p). But pΘ(x)=p(x)/ZΘ(p) is defined only after an allowed set AΘ is chosen (Eq. 4). Thus the method's allowed set Aη depends on pΘ, which depends on Aη — a fixed point the paper never solves. The paper asserts that Eq. (28) is 'the indicator of Definition 1 with Θ=η (Equation (6), δ=√ε)', an equality that holds only when pΘ=p. Consequently the 'matched truncation sampling baseline using the same allowed set' is not actually using the same allowed set, and the KL_EAGLE expression / non-vanishing gap in Prop. 1 is a consequence of this self-referential definition rather than an independent prediction about the implemented Medusa/EAGLE-3 method

full rationale

The paper's core formal results are mostly self-contained. Lemma 1 is a bookkeeping identity; Lemma 2 and Prop. 1 are algebraic consequences of Definition 1 and Lemma 3, not fitted to the empirical gaps; Table 1 is a genuine ablation; and self-citations (Zhou et al. 2026, 2025) are background, not load-bearing. No parameter is fitted and then relabeled as a prediction. The one concrete circularity is in the EAGLE-3 treatment of 'typical acceptance': Eq. (28) defines τ via H(pΘ) whereas the η-sampling allowed set in Eq. (6) uses H(p), and pΘ is itself defined only after an allowed set is chosen (Eq. 4). The paper asserts these are the same allowed set, but the equality holds only in the special case pΘ=p. Thus the KL_EAGLE gap in Prop. 1 and the amplified degradation in Table 3 are partly consequences of the self-referential definition rather than an independent empirical discovery about Medusa. Because the standard-SD comparison and the collaborative-verification ablation retain independent content, the overall circularity is partial.

Axiom & Free-Parameter Ledger

0 free parameters · 4 axioms · 0 invented entities

The central claims rest on two structural assumptions: (i) the reported methods are faithfully represented by the min-p/η allowed sets, and (ii) EAGLE-3 verification is per-position first-in-set acceptance. Neither is verified against source code of the original methods in the paper. No free parameters are fitted to make the claims work; the derived distributions are parameter-free functions of the defined rules.

axioms (4)
  • standard math Standard probability and information-theoretic identities (TV distance, KL divergence, convex mixtures) as applied to next-token distributions.
    Used throughout Sections 2-3 and Appendix A for the derivations of induced distributions and Lemma 2.
  • domain assumption SpecCascade's allowed set is exactly the min-p allowed set (5) and Medusa's typical acceptance is exactly the η-sampling allowed set (6).
    This equivalence (Section 3.2, Definition 1; Eq 27-28) is what makes the categorization and the matched-baseline comparison valid. If the real implementations use different thresholds, the induced distribution (13) and the pitfall claims do not apply to them.
  • domain assumption EAGLE-3 tree verification can be modeled as per-position inspection of a fixed candidate set, where truncation-based rules accept the first in-set candidate and otherwise terminate with a bonus token from the truncated target pΘ.
    Used in Lemma 3, Corollary 1, and Proposition 1 to derive the point-mass emission and the non-vanishing KL gap. Real EAGLE-3 may use draft probabilities or stochastic acceptance, which would change the distributional conclusions.
  • domain assumption Draft tokens in the EAGLE-3 experiments are greedily generated so that the candidate ordering and probabilities used in verification match the described procedure.
    Required for Lemma 3(i) that truncation-sampling+SD emits pΘ exactly and for the per-position acceptance analysis in H.2.

pith-pipeline@v1.3.0-daily-deepseek · 21644 in / 19273 out tokens · 181605 ms · 2026-08-01T12:00:09.888389+00:00 · methodology

0 comments
read the original abstract

Speculative Decoding (SD) accelerates large language model inference by allowing a lightweight draft model to propose tokens that are subsequently verified in parallel by a larger target model. Recent approaches introduce lossy verification schemes to further improve efficiency by relaxing strict distributional matching. Yet such relaxation silently rewrites the decoding distribution, and the resulting acceleration can come at the cost of unstable, sometimes severely degraded generation quality. In this work, we present a principled analysis of the distributions induced by lossy verification methods. We show that many seemingly distinct approaches differ only superficially and can be classified into two categories: truncation-based verification and collaborative verification. We further construct a diagnostic evaluation framework across curated benchmarks. For truncation-based methods, we identify a fundamental pitfall: performance can degrade significantly compared to the true truncation sampling baseline due to distributional distortion. For collaborative verification, we uncover a key principles: controlling the overshoot of draft probabilities relative to target probabilities is essential to prevent low-quality outputs. Our code is available at https://github.com/ZhouYuxuanYX/Fast-HSD.

Figures

Figures reproduced from arXiv: 2607.26627 by Heng Li, Junyuan Shang, Tianyu Wang, Wenbin Wang, Yuxuan Zhou, Zikai Xiao.

Figure 1
Figure 1. Figure 1: Accuracy gap between the lossless baseline [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Comparison of distributions induced by collaborative and truncation-based methods. Tokens are sorted by [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Efficiency and task performance tradeoff for collaborative verification across four benchmarks. Each [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Net change in ∆BE under different draft– target alignment ratios for Min-p and η-sampling. Each triplet reports the ratios of matching, partially overlap￾ping, and unrelated candidate tokens, respectively. baselines (η-sampling and Min-p sampling, re￾spectively) directly applied on the target model with lossless verification (Leviathan et al., 2023). This setup isolates whether their claimed compa￾rable pe… view at source ↗
Figure 5
Figure 5. Figure 5: Block efficiency of truncation-based methods across hyperparameter settings on four benchmarks. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Efficiency and task performance tradeoff for truncation methods across hyperparameter settings on [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Efficiency and task performance tradeoff for truncation methods across hyperparameter settings on [PITH_FULL_IMAGE:figures/full_fig_p016_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Two views of lenience-based relaxation. Fig. [PITH_FULL_IMAGE:figures/full_fig_p016_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

88 extracted references · 32 linked inside Pith

  1. [1]

    Aho and Jeffrey D

    Alfred V. Aho and Jeffrey D. Ullman , title =. 1972

    1972

  2. [2]

    Publications Manual , year = "1983", publisher =

    1983

  3. [3]

    Chandra and Dexter C

    Ashok K. Chandra and Dexter C. Kozen and Larry J. Stockmeyer , year = "1981", title =. doi:10.1145/322234.322243

    arXiv 1981

  4. [4]

    Scalable training of

    Andrew, Galen and Gao, Jianfeng , booktitle=. Scalable training of

  5. [5]

    Dan Gusfield , title =. 1997

    1997

  6. [6]

    Tetreault , title =

    Mohammad Sadegh Rasooli and Joel R. Tetreault , title =. Computing Research Repository , volume =. 2015 , url =

    2015

  7. [7]

    A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =

    Ando, Rie Kubota and Zhang, Tong , Issn =. A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =. Journal of Machine Learning Research , Month = dec, Numpages =

  8. [8]

    arXiv preprint arXiv:2311.13581 , year=

    Pass: Parallel speculative sampling , author=. arXiv preprint arXiv:2311.13581 , year=

    Pith/arXiv arXiv

  9. [9]

    Online Speculative Decoding , author=

  10. [10]

    2023 , url =

    Significant-Gravitas , title =. 2023 , url =

    2023

  11. [11]

    Ensemble learning for heterogeneous large language models with deep parallel collaboration , author=

  12. [12]

    arXiv preprint arXiv:2410.03777 , year=

    Determine-Then-Ensemble: Necessity of Top-k Union for Large Language Model Ensembling , author=. arXiv preprint arXiv:2410.03777 , year=

    Pith/arXiv arXiv

  13. [13]

    Breaking the ceiling of the llm community by treating token generation as a classification for ensembling , author=

  14. [14]

    Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , year=

    Contrastive Decoding: Open-ended Text Generation as Optimization , author=. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , year=

  15. [15]

    arXiv preprint arXiv:2309.09117 , year=

    Contrastive decoding improves reasoning in large language models , author=. arXiv preprint arXiv:2309.09117 , year=

    Pith/arXiv arXiv

  16. [16]

    arXiv preprint arXiv:2401.07851 , year=

    Unlocking efficiency in large language model inference: A comprehensive survey of speculative decoding , author=. arXiv preprint arXiv:2401.07851 , year=

    Pith/arXiv arXiv

  17. [17]

    Decoding-time Realignment of Language Models , author=

  18. [18]

    Decoding-time language model alignment with multiple objectives , author=

  19. [19]

    React: Synergizing reasoning and acting in language models , author=

  20. [20]

    Toolformer: Language models can teach themselves to use tools , author=

  21. [21]

    Fast inference from transformers via speculative decoding , author=

  22. [22]

    Spectr: Fast speculative decoding via optimal transport , author=

  23. [23]

    Transformers: State-of-the-art natural language processing , author=

  24. [24]

    arXiv preprint arXiv:2311.08252 , year=

    Rest: Retrieval-based speculative decoding , author=. arXiv preprint arXiv:2311.08252 , year=

    Pith/arXiv arXiv

  25. [25]

    arXiv preprint arXiv:2302.01318 , year=

    Accelerating large language model decoding with speculative sampling , author=. arXiv preprint arXiv:2302.01318 , year=

    Pith/arXiv arXiv

  26. [26]

    Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generation , author=

  27. [27]

    Specinfer: Accelerating large language model serving with tree-based speculative inference and verification , author=

  28. [28]

    EAGLE: speculative sampling requires rethinking feature uncertainty , author=

  29. [29]

    Sequoia: Scalable and robust speculative decoding , author=

  30. [30]

    arXiv preprint arXiv:2401.06706 , year=

    Multi-candidate speculative decoding , author=. arXiv preprint arXiv:2401.06706 , year=

    Pith/arXiv arXiv

  31. [31]

    Optimized Multi-Token Joint Decoding With Auxiliary Model for LLM Inference , author=

  32. [32]

    Speculative decoding with big little decoder , author=

  33. [33]

    Efficient Memory Management for Large Language Model Serving with PagedAttention , author=

  34. [34]

    DistillSpec: Improving Speculative Decoding via Knowledge Distillation , author=

  35. [35]

    arXiv preprint arXiv:2402.15758 , year=

    Chimera: A Lossless Decoding Method for Accelerating Large Language Models Inference by Fusing all Tokens , author=. arXiv preprint arXiv:2402.15758 , year=

    Pith/arXiv arXiv

  36. [36]

    Accelerated speculative sampling based on tree Monte Carlo , author=

  37. [37]

    arXiv preprint arXiv:2502.18779 , year=

    Towards optimal multi-draft speculative decoding , author=. arXiv preprint arXiv:2502.18779 , year=

    Pith/arXiv arXiv

  38. [38]

    arXiv preprint arXiv:2404.11912 , year=

    Triforce: Lossless acceleration of long sequence generation with hierarchical speculative decoding , author=. arXiv preprint arXiv:2404.11912 , year=

    Pith/arXiv arXiv

  39. [39]

    arXiv preprint arXiv:2408.11049 , year=

    Magicdec: Breaking the latency-throughput tradeoff for long context generation with speculative decoding , author=. arXiv preprint arXiv:2408.11049 , year=

    Pith/arXiv arXiv

  40. [40]

    arXiv preprint arXiv:2304.04487 , year=

    Inference with reference: Lossless acceleration of large language models , author=. arXiv preprint arXiv:2304.04487 , year=

    Pith/arXiv arXiv

  41. [41]

    arXiv preprint arXiv:2405.19261 , year=

    Faster Cascades via Speculative Decoding , author=. arXiv preprint arXiv:2405.19261 , year=

    Pith/arXiv arXiv

  42. [42]

    GLIDE with a CAPE: a low-hassle method to accelerate speculative decoding , author=

  43. [43]

    EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees , author=

  44. [44]

    Draft& Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding , author=

  45. [45]

    arXiv preprint arXiv:2403.10444 , year=

    Block Verification Accelerates Speculative Decoding , author=. arXiv preprint arXiv:2403.10444 , year=

    Pith/arXiv arXiv

  46. [46]

    Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads , author=

  47. [47]

    LANTERN: Accelerating Visual Autoregressive Models with Relaxed Speculative Decoding , author=

  48. [48]

    arXiv preprint arXiv:2303.08774 , year=

    Gpt-4 technical report , author=. arXiv preprint arXiv:2303.08774 , year=

    Pith/arXiv arXiv

  49. [49]

    arXiv preprint arXiv:2307.09288 , year=

    Llama 2: Open foundation and fine-tuned chat models , author=. arXiv preprint arXiv:2307.09288 , year=

    Pith/arXiv arXiv

  50. [50]

    arXiv preprint arXiv:2309.16609 , year=

    Qwen technical report , author=. arXiv preprint arXiv:2309.16609 , year=

    Pith/arXiv arXiv

  51. [51]

    2026 , eprint=

    OpenAI o1 System Card , author=. 2026 , eprint=

    2026

  52. [52]

    arXiv preprint arXiv:2501.12948 , year=

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning , author=. arXiv preprint arXiv:2501.12948 , year=

    Pith/arXiv arXiv

  53. [53]

    arXiv preprint arXiv:1803.03635 , year=

    The lottery ticket hypothesis: Finding sparse, trainable neural networks , author=. arXiv preprint arXiv:1803.03635 , year=

    Pith/arXiv arXiv

  54. [54]

    arXiv preprint arXiv:2306.11695 , year=

    A simple and effective pruning approach for large language models , author=. arXiv preprint arXiv:2306.11695 , year=

    Pith/arXiv arXiv

  55. [55]

    Q-bert: Hessian based ultra low precision quantization of bert , author=

  56. [56]

    Smoothquant: Accurate and efficient post-training quantization for large language models , author=

  57. [57]

    arXiv preprint arXiv:2110.14168 , year=

    Training verifiers to solve math word problems , author=. arXiv preprint arXiv:2110.14168 , year=

    Pith/arXiv arXiv

  58. [58]

    arXiv preprint arXiv:2107.03374 , year=

    Evaluating large language models trained on code , author=. arXiv preprint arXiv:2107.03374 , year=

    Pith/arXiv arXiv

  59. [59]

    Get to the point: Summarization with pointer-generator networks , author=

  60. [60]

    arXiv preprint arXiv:1805.04833 , year=

    Hierarchical neural story generation , author=. arXiv preprint arXiv:1805.04833 , year=

    Pith/arXiv arXiv

  61. [61]

    arXiv preprint arXiv:1904.09751 , year=

    The curious case of neural text degeneration , author=. arXiv preprint arXiv:1904.09751 , year=

    Pith/arXiv arXiv 1904

  62. [62]

    Cascade speculative drafting for even faster llm inference , author=

  63. [63]

    Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignment , author=

  64. [64]

    2024 , month =

    Vivien Tran-Thien , title =. 2024 , month =

    2024

  65. [65]

    arXiv preprint arXiv:2407.01082 , year=

    Turning up the heat: Min-p sampling for creative and coherent llm outputs , author=. arXiv preprint arXiv:2407.01082 , year=

    arXiv

  66. [66]

    arXiv preprint arXiv:2210.15191 , year=

    Truncation sampling as language model desmoothing , author=. arXiv preprint arXiv:2210.15191 , year=

    Pith/arXiv arXiv

  67. [67]

    Mmlu-pro: A more robust and challenging multi-task language understanding benchmark , author=

  68. [68]

    The flores-101 evaluation benchmark for low-resource and multilingual machine translation , author=

  69. [69]

    Truncation Sampling as Language Model Desmoothing , author=

  70. [70]

    arXiv preprint arXiv:2502.01662 , year=

    Fast large language model collaborative decoding via speculation , author=. arXiv preprint arXiv:2502.01662 , year=

    Pith/arXiv arXiv

  71. [71]

    arXiv preprint arXiv:2412.15115 , year =

    Qwen2.5 Technical Report , author =. arXiv preprint arXiv:2412.15115 , year =

    Pith/arXiv arXiv

  72. [72]

    arXiv preprint arXiv:2411.19799 , year=

    Include: Evaluating multilingual language understanding with regional knowledge , author=. arXiv preprint arXiv:2411.19799 , year=

    Pith/arXiv arXiv

  73. [73]

    arXiv preprint arXiv:2601.05724 , year=

    Overcoming Joint Intractability with Lossless Hierarchical Speculative Decoding , author=. arXiv preprint arXiv:2601.05724 , year=

    arXiv

  74. [74]

    arXiv preprint arXiv:2103.03874 , year=

    Measuring mathematical problem solving with the math dataset , author=. arXiv preprint arXiv:2103.03874 , year=

    Pith/arXiv arXiv

  75. [75]

    Balancing diversity and risk in llm sampling: How to select your method and parameter for open-ended text generation , author=

  76. [76]

    Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? , author=

  77. [77]

    Balancing Diversity and Risk in LLM Sampling: How to Select Your Method and Parameter for Open-Ended Text Generation , author =

  78. [78]

    Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation , author=

  79. [79]

    arXiv preprint arXiv:2210.17323 , year=

    Gptq: Accurate post-training quantization for generative pre-trained transformers , author=. arXiv preprint arXiv:2210.17323 , year=

    Pith/arXiv arXiv

  80. [80]

    arXiv preprint arXiv:2505.23281 , year=

    Matharena: Evaluating llms on uncontaminated math competitions , author=. arXiv preprint arXiv:2505.23281 , year=

    Pith/arXiv arXiv

Showing first 80 references.