Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

HiSpec uses early-exit layers to verify draft tokens midway through the model, reporting 1.28×-2.01× faster decoding over baseline speculative decoding, but the accuracy claim is not empirically tested.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 12:55 UTC pith:CTTX4GMK

load-bearing objection Genuinely new intermediate-verification mechanism, but the accuracy-preservation claim is unverified and the headline speedup numbers are mislabeled. the 4 major comments →

arxiv 2510.01336 v2 pith:CTTX4GMK submitted 2025-10-01 cs.CL cs.AIcs.LG

HiSpec: Hierarchical Speculative Decoding for LLMs

Avinash Kumar , Sujay Sanghavi , Poulami Das This is my paper
classification cs.CL cs.AIcs.LG
keywords intermediateverificationtextithispecmodeldecodingdraftmodels
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Speculative decoding is a trick to make large language models respond faster: a small, fast 'draft' model guesses several likely next words, and the big, slow model checks them all at once. The check is usually the slow part — sometimes four times slower than the guessing. Most earlier work tried to make the guessing faster, not the checking. HiSpec instead tackles the checking step.

HiSpec uses 'early-exit' models, which are trained so that they can stop partway through their layers and still give a usable prediction. HiSpec treats one early layer as the draft guesser, a slightly deeper layer as a middle-checker, and the full model as the final checker. The middle-checker rejects bad guesses quickly, so only the survivors reach the final check. It also reuses intermediate computations so the three checks do not duplicate work, and it periodically sends the accepted tokens to the full model to make sure the answer still matches what that model would have written alone.

On benchmarks like chat, math, and code, HiSpec reports moderate speedups over earlier speculative decoding methods — on average about 1.3×, and up to 2× on some tasks. But the paper does not report any direct measurement of answer quality, so the claim that accuracy is preserved is asserted rather than shown. The settings that give the best speedups were also chosen using the same datasets used for reporting, so the headline numbers may be optimistic.

Core claim

The abstract states: 'HiSpec improves throughput by 1.28× on average and by up to 2.01× compared to the baseline single-layer speculation without compromising accuracy.' If true, this means the verification bottleneck in speculative decoding can be substantially alleviated without a quality penalty, making it an important practical advance for LLM inference.

Load-bearing premise

The claim that output quality is preserved rests on the unstated assumption that the intermediate verifier's TOPPREDICTIONS check, combined with the periodic full-model verification, makes the final output distribution identical to the target model's (or at least equally accurate). The paper never defines TOPPREDICTIONS (top-1 vs top-k) nor proves the distribution-preservation property, and it presents no accuracy metrics. If TOPPREDICTIONS admits tokens beyond the target's argmax, the periodic verification only checks the final prefix and can commit tokens the target would not have chosen, silently changing the output distribution.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. HiSpec proposes hierarchical speculative decoding that uses early-exit (EE) layers within a single model for both draft generation and intermediate verification, with periodic full-model verification to preserve output quality. The draft generates tokens at a shallow exit layer; an intermediate exit layer tentatively accepts/rejects tokens; after a window of tentative acceptances, the full model verifies and commits the final prefix. The paper claims throughput improvements of 1.28× on average and up to 2.01× over single-layer speculative decoding baselines, without accuracy loss, across five benchmarks and six model sizes (7B–70B), while reusing KV caches and hidden states to avoid redundant computation.

Significance. If the correctness and throughput claims hold, HiSpec addresses a real bottleneck—verification latency in speculative decoding—without the training overhead of auxiliary verifier models. The idea of reusing EE layers as a free intermediate verifier is elegant and practical, and the evaluation covers a broad range of models and tasks, consistently showing gains over LayerSkip, AdaDecode, Lookahead, and SWIFT. However, the central claim of accuracy preservation is not supported by any accuracy metric or a precise definition of the acceptance predicate, and the headline speedup is inconsistently reported. These issues are fixable within the scope of a revision, but they are load-bearing for the paper's main contribution.

major comments (4)
  1. [Algorithm 1 (line 21) and Algorithm 2 (lines 17, 35)] The function TOPPREDICTIONS, on which the entire correctness argument rests, is never defined. If it means 'top-1 prediction' under greedy decoding, then the periodic full-model verification in Algorithm 2 can be shown to preserve the target model's greedy output distribution. If it means top-k with k>1, the full-model check can commit tokens that the target would not have generated, and the claimed consistency with vanilla autoregressive decoding is false. The paper reports no accuracy metric (task accuracy, exact match, or output-equality rate) to support the abstract's 'without compromising accuracy' claim. Please define TOPPREDICTIONS, state the decoding mode (greedy vs. sampling), provide a proof of output distribution preservation (or an empirical output-equality test), and report accuracy on the evaluated benchmarks.
  2. [Abstract, Section 5, Conclusion] The headline speedup is inconsistent. The abstract and conclusion claim '1.28× on average and up to 2.01× compared to the baseline single-layer speculation,' while Section 5 states '1.7× on average and up to 2.08× compared to vanilla auto-regressive decoding.' Table 1 reports HiSpec relative to vanilla, and the implied HiSpec-vs-LayerSkip ratios are different from 2.01× (e.g., ShareGPT Llama3-8B: 2.01/1.14≈1.76×). Clarify the exact baseline, report both vanilla-relative and LayerSkip-relative numbers, and ensure all stated averages are reproducible from the table. Also report variance/error bars across runs, since the measured differences between methods are often small.
  3. [Section 6 and Appendix C] The design parameters (draft exit layer L_d, intermediate verifier layer L_i, draft proposal length N_d, and tentative acceptance window N_i) are all tuned on ShareGPT with Llama3-8B, and the same configuration is then used to produce the headline 2.01× number on the same benchmark/model in Table 1. This is a form of test-set overfitting. The paper should either report performance for a fixed, pre-specified default configuration across all benchmarks, or apply a cross-validation/held-out scheme for hyperparameter selection, so that the reported gains are not simply the result of tuning on the evaluation set.
  4. [Algorithm 2, lines 33–35] The full-model verification step is underspecified to the point of being unclear. The pseudocode computes q ← L_f.PREDICTNEXT(current context, KVi), where KVi is the KV cache from the intermediate verifier. Since L_f contains more layers than L_i, the KV from L_i alone cannot directly produce a prediction at L_f; the model must continue through layers L_i+1...L_f using the hidden states from L_i. The paper's Section 3.1 discusses KV/hidden-state reuse at a high level, but the algorithm as written is not a valid forward pass. Please specify the exact mechanism by which the intermediate hidden states and KV caches are extended to the full model, or correct the pseudocode.
minor comments (6)
  1. [Section 1 and Section 7] The repeated string 'HiHiHiHiHierachical Spec...' in the proposal paragraph and conclusion appears to be a rendering artifact and should be fixed.
  2. [Figure 3 caption] The caption contains corrupted text ('FǦEDABČ') that is unreadable; please provide a clean version.
  3. [Section 6] The text says the one-fourth intermediate verifier 'maximizes both throughput and token acceptance rates,' but Figure 5 shows only throughput, not acceptance rates. Either add acceptance-rate data or rephrase.
  4. [General] The paper should state explicitly that all experiments use greedy decoding (or, if sampling is used, describe how acceptance is defined for non-greedy sampling). This is relevant to the TOPPREDICTIONS definition.
  5. [Table 1] The caption says 'These methods do not rely on an auxiliary draft model,' which is true for the baselines listed, but HiSpec also does not use an auxiliary model. This is fine, but the wording could be clarified to avoid implying HiSpec uses one.
  6. [Appendix C, Figure 8] The heatmap would benefit from labeled axes and a colorbar legend to make the swept parameter ranges and the chosen default (L_d=3, L_i=8) easier to read.

Circularity Check

1 steps flagged

Mild same-benchmark tuning inflates the 2.01x headline, but HiSpec's core result is independent and not circular.

specific steps
  1. fitted input called prediction [Section 6 (Ablation), Appendix C/Figure 8, Table 1 (ShareGPT/Llama3-8B row)]
    "We analyze the impact of these parameters on HiSpec's throughput using the ShareGPT dataset and Llama3-8B model. ... By default, HiSpec uses Ni = 4. ... Our evaluations show that among all configurations, the setting with draft generation at Layer 3 and intermediate verification at Layer 8 achieves the highest throughput. ... HiSpec(Ours) 2.01×"

    The configuration behind the headline 'up to 2.01x' speedup was selected by sweeping Nd, Ni, and the draft/intermediate layer positions on ShareGPT/Llama3-8B to maximize throughput on that exact benchmark/model (Section 6 and Appendix C). Table 1 then reports 2.01x for that same selected configuration and same benchmark. Hence that specific number is an in-sample selected maximum of the objective used to choose the design, not an independent out-of-sample evaluation. The other rows of Table 1 were not used in the sweeps and provide independent support, so the circularity is partial rather than load-bearing for the whole paper.

full rationale

HiSpec's algorithm (Algorithms 1-2) is not self-referential: draft generation, intermediate acceptance at Li, buffering until Ni, and final verification at Lf are distinct operations, and the throughput result is a measured systems outcome rather than a quantity defined in terms of itself. There are no load-bearing self-citations; the paper builds on external EE checkpoints (LayerSkip, AdaDecode) and benchmarks. The accuracy claim is under-supported because TOPPREDICTIONS is never defined and no accuracy or output-equality metric is reported, but that is a missing specification/measurement, not a derivational circularity. The score is raised only by the same-benchmark hyperparameter selection that directly produces the 2.01x ShareGPT datapoint; because most reported configurations (other models and benchmarks) were not tuned and still show consistent speedups, the central claim retains independent content.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The paper's contribution rests on pre-existing EE models, an unverified accuracy-preservation assumption, a generalization heuristic for layer placement, and a Linux-style HF implementation assumption. The main free parameters (L_d, L_i, N_d, N_i) are fitted via sweeps on the evaluation benchmarks.

free parameters (4)
  • Draft exit layer L_d = Layer 3 (Llama3-8B), Layer 7 (CodeLlama-34B), Layer 10 (Llama2-70B)
    Chosen via throughput sweeps in Appendix C and Section 3.3 to maximize throughput; acts as a design parameter fitted to the evaluation data.
  • Intermediate verifier layer L_i = Layer 8 (Llama3-8B), Layer 12 (CodeLlama-34B), Layer 20 (Llama2-70B)
    Same as above; position selected by sweeps on ShareGPT and CNN/DM to maximize throughput.
  • Draft proposal length N_d = 2 tokens
    Ablation in Figure 7(a) shows N_d=2 yields highest throughput; selected on ShareGPT with Llama3-8B.
  • Tentative acceptance window N_i = 4 tokens
    Ablation in Figure 7(b) shows N_i=4 yields highest throughput; selected on the same dataset.
axioms (4)
  • domain assumption Early-exit models with interpretable hidden states at selected layers are available and used as provided (LayerSkip checkpoints).
    Paper relies on prior trained EE models; no training is done in HiSpec.
  • ad hoc to paper The TOPPREDICTIONS check at the intermediate verifier plus periodic full-model verification preserves the target model's output distribution.
    Section 3.3 asserts target verification maintains accuracy, but the acceptance rule is not defined (top-1 vs top-k) and no proof or accuracy measurement is given.
  • ad hoc to paper The heuristic that the first quarter of model layers produces ~69% of correct tokens generalizes across models and tasks for positioning the verifier.
    Section 3.2 uses this observation to choose default layer ratios; it is measured on the same task families used for evaluation.
  • domain assumption The HuggingFace implementation's measured throughput is representative of real serving conditions.
    No batch scheduling, GPU utilization, or production framework analysis is provided; vLLM measurements in Appendix B are separate from the HiSpec implementation.

pith-pipeline@v1.3.0-alltime-deepseek · 12986 in / 17701 out tokens · 125328 ms · 2026-08-04T12:55:17.801701+00:00 · methodology

0 comments
read the original abstract

Speculative decoding accelerates LLM inference by using a smaller draft model to speculate tokens that a larger target model verifies. Verification is often the bottleneck (e.g. verification is $4\times$ slower than token generation when a 3B model speculates for a 70B target model), but most prior works focus only on accelerating drafting. $\textit{``Intermediate"}$ verification reduces verification time by discarding inaccurate draft tokens early, but existing methods incur substantial training overheads in incorporating the intermediate verifier, increase the memory footprint to orchestrate the intermediate verification step, and compromise accuracy by relying on approximate heuristics. We propose $\underline{\textit{Hi}}\textit{erarchical }\underline{\textit{Spec}}\textit{ulative Decoding (HiSpec)}$, a framework for high-throughput speculative decoding that exploits $\textit{early-exit (EE) models}$ for low-overhead intermediate verification. EE models allow tokens to exit early by skipping layer traversal and are explicitly trained so that hidden states at selected layers can be interpreted, making them uniquely suited for intermediate verification without drastically increasing compute and memory overheads. To improve resource-efficiency even further, we design a methodology that enables HiSpec to re-use key-value caches and hidden states between the draft, intermediate verifier, and target models. To maintain accuracy, HiSpec periodically validates the draft tokens accepted by the intermediate verifier against the target model. Our evaluations using various representative benchmarks and models show that HiSpec improves throughput by 1.28$\times$ on average and by up to 2.01$\times$ compared to the baseline single-layer speculation without compromising accuracy.

Figures

Figures reproduced from arXiv: 2510.01336 by Avinash Kumar, Poulami Das, Sujay Sanghavi.

Figure 1
Figure 1. Figure 1: Throughput of various representative benchmarks for the Llama3-8B model relative to [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Latency of the draft token generation and token verification phases for different draft and [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: (a) Standard speculative decoding. (b) Our proposal, [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Percentage of tokens produced by one-fourth the model that are accepted by the final layer. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Throughput of HiSpec for different draft and intermediate layer combinations relative to [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Comparison of token acceptance rates and throughput ( [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Throughput (higher is better) with increasing (a) number of draft tokens per step (Nd) and (b) number of tokens tentatively accepted (Ni). Lower values of Nd and Ni yield higher throughput, which is expected, because they limit the formation of longer chains of unverified inaccurate tokens. ▷ Impact of number of draft tokens per step (Nd): A large number of draft tokens per step gener￾ally increases the nu… view at source ↗
Figure 8
Figure 8. Figure 8: Throughput of all valid combinations of drafts and intermediate verifiers for the 32 layer [PITH_FULL_IMAGE:figures/full_fig_p014_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Cumulative distribution of input and output token lengths across (a) ShareGPT, (b) [PITH_FULL_IMAGE:figures/full_fig_p015_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. FASER: Fine-Grained Phase Management for Speculative Decoding in Dynamic LLM Serving

    cs.DC 2026-04 unverdicted novelty 7.0

    FASER delivers up to 53% higher throughput and 1.92x lower latency in dynamic LLM serving by adjusting speculative lengths per request, early pruning of rejects, and overlapping draft/verification phases via frontiers.

Reference graph

Works this paper leans on

20 extracted references · 15 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Sharegpt, 2023.https://sharegpt.com

    Hugging face, 2016.https://huggingface.co. Sharegpt, 2023.https://sharegpt.com. Tianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng, Jason D. Lee, Deming Chen, and Tri Dao. Medusa: Simple llm inference acceleration framework with multiple decoding heads.arXiv preprint arXiv: 2401.10774,

    Pith/arXiv arXiv 2016

  2. [4]

    Layerskip: Enabling early exit inference and self-speculative decoding.arXiv preprint arXiv:2404.16710,

    Mostafa Elhoushi, Akshat Shrivastava, Diana Liskovich, Basil Hosmer, Bram Wasti, Liangzhen Lai, Anas Mahmoud, Bilge Acun, Saurabh Agarwal, Ahmed Roman, et al. Layerskip: Enabling early exit inference and self-speculative decoding.arXiv preprint arXiv:2404.16710,

    Pith/arXiv arXiv

  3. [5]

    Break the sequential dependency of llm infer- ence using lookahead decoding.arXiv preprint arXiv:2402.02057,

    Yichao Fu, Peter Bailis, Ion Stoica, and Hao Zhang. Break the sequential dependency of llm infer- ence using lookahead decoding.arXiv preprint arXiv:2402.02057,

    Pith/arXiv arXiv

  4. [6]

    Draft- based approximate inference for llms.arXiv preprint arXiv:2506.08373,

    Kevin Galim, Ethan Ewer, Wonjun Kang, Minjae Lee, Hyung Il Koo, and Kangwook Lee. Draft- based approximate inference for llms.arXiv preprint arXiv:2506.08373,

    arXiv

  5. [8]

    Eagle: Speculative sampling requires rethinking feature uncertainty.arXiv preprint arXiv:2401.15077,

    10 HiSpec: Hierarchical Speculative Decoding for LLMs Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. Eagle: Speculative sampling requires rethinking feature uncertainty.arXiv preprint arXiv:2401.15077,

    Pith/arXiv arXiv

  6. [9]

    Specinfer: Accelerating genera- tive large language model serving with tree-based speculative inference and verification.arXiv preprint arXiv:2305.09781,

    Xupeng Miao, Gabriele Oliaro, Zhihao Zhang, Xinhao Cheng, Zeyu Wang, Zhengxin Zhang, Rae Ying Yee Wong, Alan Zhu, Lijie Yang, Xiaoxiang Shi, et al. Specinfer: Accelerating genera- tive large language model serving with tree-based speculative inference and verification.arXiv preprint arXiv:2305.09781,

    Pith/arXiv arXiv

  7. [12]

    Efficient speculative decoding for llama at scale: Challenges and solutions.arXiv preprint arXiv:2508.08192,

    Bangsheng Tang, Carl Chengyan Fu, Fei Kou, Grigory Sizov, Haoci Zhang, Jason Park, Jiawen Liu, Jie You, Qirui Yang, Sachin Mehta, et al. Efficient speculative decoding for llama at scale: Challenges and solutions.arXiv preprint arXiv:2508.08192,

    Pith/arXiv arXiv

  8. [14]

    Adadecode: Accelerating llm decoding with adaptive layer parallelism.arXiv preprint arXiv:2506.03700,

    Zhepei Wei, Wei-Lin Chen, Xinyu Zhu, and Yu Meng. Adadecode: Accelerating llm decoding with adaptive layer parallelism.arXiv preprint arXiv:2506.03700,

    Pith/arXiv arXiv

  9. [15]

    Transformers: State-of-the-art natural language processing

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, et al. Transformers: State-of-the-art natural language processing. InProceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, pp. 38–45,

    2020

  10. [17]

    Decoding speculative decoding

    Minghao Yan, Saurabh Agarwal, and Shivaram Venkataraman. Decoding speculative decoding. arXiv preprint arXiv:2402.01528,

    Pith/arXiv arXiv

  11. [18]

    Algorithm 2Speculative Decoding with HiSpec 1:Input:Draft layerL d, 2:Full layerL f , 3:Intermediate layerL i |L d < Li < Lf 4:Prompt= [T 0, T1,

    11 HiSpec: Hierarchical Speculative Decoding for LLMs A HISPECALGORITHM Algorithm A describes an elaborate version of the HiSpec design. Algorithm 2Speculative Decoding with HiSpec 1:Input:Draft layerL d, 2:Full layerL f , 3:Intermediate layerL i |L d < Li < Lf 4:Prompt= [T 0, T1, . . . , Tk], 5:Draft proposal lengthN d, // Tokens proposed byL d per step ...

    2023

  12. [20]

    intermediate verification

    For example, summarization tasks such as CNN/DM, Xsum contain longer input sequences compared to other benchmarks. These variations ensure that our evaluations encompasses a diverse range of input-output configurations. (a) (b) (c) (d) (e) Figure 9: Cumulative distribution of input and output token lengths across (a) ShareGPT, (b) CNN/DM, (c) GSM8K, (d) X...

    2024

  13. [31]

    Figure 8 shows the relative throughput of each combination compared to vanilla decoding for each draft and intermediate verifier combination

    We analyze the throughput of all these draft and intermediate verifier combinations. Figure 8 shows the relative throughput of each combination compared to vanilla decoding for each draft and intermediate verifier combination. Our evaluations show that among all configurations, the setting with draft generation at Layer 3 and intermediate verification at ...

    2023

  14. [2016]

    doi: 10.18653/v1/K16-1028

    Association for Compu- tational Linguistics. doi: 10.18653/v1/K16-1028. URLhttps://aclanthology.org/ K16-1028. Shashi Narayan, Shay B. Cohen, and Mirella Lapata. Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization.ArXiv, abs/1808.08745,

    Pith/arXiv arXiv

  15. [2018]

    Code llama: Open foundation models for code.arXiv preprint arXiv:2308.12950,

    Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, et al. Code llama: Open foundation models for code.arXiv preprint arXiv:2308.12950,

    Pith/arXiv arXiv

  16. [2020]

    Swift: On-the-fly self-speculative decoding for llm inference acceleration.arXiv preprint arXiv:2410.06916,

    Heming Xia, Yongqi Li, Jun Zhang, Cunxiao Du, and Wenjie Li. Swift: On-the-fly self-speculative decoding for llm inference acceleration.arXiv preprint arXiv:2410.06916,

    Pith/arXiv arXiv

  17. [2021]

    Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168,

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168,

    Pith/arXiv arXiv

  18. [2023]

    Self-distilled sparse drafters.arXiv preprint arXiv:2504.08838,

    Mike Lasby, Nish Sinnadurai, Valavan Manohararajah, Sean Lie, Yani Ioannou, and Vithursan Thangarasa. Self-distilled sparse drafters.arXiv preprint arXiv:2504.08838,

    Pith/arXiv arXiv

  19. [2024]

    Accelerating large language model decoding with speculative sampling.arXiv preprint arXiv:2302.01318,

    Charlie Chen, Sebastian Borgeaud, Geoffrey Irving, Jean-Baptiste Lespiau, Laurent Sifre, and John Jumper. Accelerating large language model decoding with speculative sampling.arXiv preprint arXiv:2302.01318,

    Pith/arXiv arXiv

  20. [2025]

    Llama 2: Open founda- tion and fine-tuned chat models.arXiv preprint arXiv:2307.09288,

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models.arXiv preprint arXiv:2307.09288,

    Pith/arXiv arXiv