Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

DYNAMAX: Dynamic computing for Transformers and Mamba based architectures

T0 review · 4 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Mamba's own blocks can cut LLM compute by deciding when to exit

desk verdict First early-exit framework for Mamba with a clever Mamba-as-classifier design, but the headline savings curve rests on an unspecified rule that drops degenerate low-threshold runs. read the letter →

arxiv 2504.20922 v1 pith:V647IXI4 submitted 2025-04-29 cs.CL cs.AIcs.LG

Miguel Nogales , Matteo Gambella , Manuel Roveri This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords earlyexitMambastate-spacemodelsdynamicinferencedecoder-onlytransformersknowledgedistillationcomputationalefficiencylargelanguage
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DYNAMAX is a framework for adding early exits to decoder-only large language models, and the paper claims it is the first to do so for Mamba-based architectures. The core assertion is that Mamba blocks can serve not only as the backbone being exited but also as the confidence classifiers that decide when to exit, for both Mamba and Transformer hosts. If the claim holds, Mamba-based LLMs can stop computing per token as soon as confidence is sufficient, and the stateful structure of Mamba makes it a cheaper and more flexible exit predictor than feed-forward alternatives. The paper reports that early exits beat layer pruning on knowledge-focused QA tasks at similar compute savings, and that the Mamba classifier gives the widest range of useful confidence thresholds.

What carries the argument

The central object is the token-forwarding scheme of Algorithm 1 together with the three early-exit classifier heads: a CALM-style one-layer feed-forward net, a transformer-like FFN, and a Mamba block whose output projection is reduced to two logits. The load-bearing comparison is the operational-cost decomposition: a Transformer block costs about $24Td_{\text{model}}^2 + 4T^2d_{\text{model}}$ operations, while a Mamba block costs about $6d_{\text{model}}^2 + 2n_{\text{groups}}d_{\text{state}}d_{\text{model}}$, which the paper uses to argue that a Mamba classifier adds only constant per-token cost while carrying a recurrent state that helps it track the generated text. For Transformers, the efficiency trick is copying the KV cache entries from the exit layer directly to all subsequent layers instead of recomputing partial forwards for the exited token.

What would settle it

Run DYNAMAX on a Mamba model with identical pretraining data and objective as a matched Transformer, or on the same Mamba weights before and after code specialization, and check whether the Mamba classifier's advantage over feed-forward exits persists; if the advantage vanishes, the claimed architectural benefit is confounded by pretraining.

Watch

Extended reading notes

Core claim

The paper claims that Mamba-based large language models can be made dynamically computable: early-exit classifiers attached to the latter half of a Mamba backbone can stop per-token inference when confidence crosses a threshold, and, more strongly, that a Mamba block itself—with its output projection reduced to two logits—is a better early-exit classifier than a one-layer CALM-style head or a feed-forward block for both Mamba and Transformer hosts. The framework trains these exits by knowledge distillation against the full model's output, using a top-k relaxation of the oracle target to keep training stable, and reports that on TriviaQA, CoQA, and TruthfulQA the Mamba-classifier configuration preserves accuracy better than layer pruning at comparable compute savings, with the Mamba backbone showing a larger gap between recomputing and not recomputing states than the Transformer does.

Load-bearing premise

The empirical case rests on comparing Mistral 7B with Codestral Mamba 7B as if they differ only in architecture, when the Mamba model was pretrained specifically for code tasks; differences in early-exit behavior could come from that pretraining gap.

Editorial extensions

If this is right

  • In Mamba-based LLMs, inference can be terminated per token when an exit confidence threshold is met, cutting compute on easy tokens and in low-latency settings.
  • A Mamba block with a two-logit output projection can act as a reusable early-exit classifier on Transformer backbones, not just on Mamba backbones.
  • For Transformer early exits, copying the KV cache from the exit layer to later layers avoids partial recomputation, lowering the overhead of exiting.
  • In knowledge-heavy tasks such as TriviaQA, early exits degrade accuracy more gracefully than layer pruning at the same compute reduction.
  • Skipping state recomputation in Mamba changes the cost-performance trade-off more than it does in Transformers, so the choice is architecture-specific.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If Mamba's recurrent state is what gives its exit classifier a wider threshold range, then hybrid or linear-attention models with explicit recurrent states might inherit the same benefit; this is an inference, not a paper claim.
  • The KV-cache copying trick could compose with speculative decoding or draft-model pipelines, where early-exited tokens are later re-verified with full computation.
  • A natural testable extension is per-layer adaptive thresholds driven by context or latency budgets, which the paper lists as future work but does not evaluate.
  • The top-k relaxation of the distillation target is a free parameter; varying k could expose a trade-off between training stability and exit accuracy that the paper does not quantify.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces DYNAMAX, a framework that adds early exit (EE) mechanisms to decoder-only Transformers and Mamba-based LLMs, and also repurposes Mamba blocks as lightweight EE classifiers for both Mamba and Transformer backbones. The classifiers are trained by knowledge distillation from the full model, with a top-k relaxation of the oracle target to improve training stability. During inference, tokens may exit early when a classifier's confidence exceeds a threshold; for Transformers, the paper proposes copying KV caches from the exit layer to subsequent blocks, while for Mamba it considers either partial recomputation or skipping state updates. Experiments compare Mistral 7B (Transformer) with Codestral Mamba 7B on TriviaQA, CoQA, and TruthfulQA, measuring accuracy versus computational savings, and compare EE against layer pruning. The authors claim this is the first framework to integrate EEs into Mamba architectures and to use Mamba as an EE classifier, and report that Mamba-based classifiers provide a favorable savings-accuracy trade-off.

Significance. If the central claims hold, the paper makes a useful contribution by extending early-exit techniques to state-space models, which are increasingly important for efficient long-context inference, and by demonstrating that Mamba blocks can serve as effective confidence predictors. The release of source code and the use of public models and datasets support reproducibility and facilitate follow-up work. However, the empirical support for the headline claim is currently weakened by an unquantified post-hoc exclusion rule for degenerate low-threshold configurations, a confounded comparison between two models that differ in pretraining data and specialization, and an incompletely specified KV-cache copying mechanism. These issues are load-bearing because the paper's quantitative conclusions rest on the threshold-sweep curves and on the architectural comparison.

major comments (4)
  1. [Section V] The exclusion rule for degenerate outputs is not specified quantitatively: the paper states 'If this event occurs a certain number of times, that configuration for the evaluation is not considered valid and then not shown in the graphs,' but it does not give the number, the total count of excluded configurations, or the number of degenerate sequences per configuration. Since the x-axis of every EE plot is the confidence-threshold sweep, removing low-threshold degenerate points preferentially trims the high-savings end of the curve, which directly inflates the reported savings-accuracy trade-off. Moreover, the rule is applied only to EE configurations and not to layer-pruning baselines, biasing the comparison. The authors should report the exclusion criterion, the number of excluded configurations per task and model, and a sensitivity analysis in which degenerate outputs are counted as failures; the figures should be re-plotted accordingly.
  2. [Section IV-A and V] The architectural comparison is confounded by pretraining: Mistral 7B is a general-purpose Transformer, whereas Codestral Mamba 7B is 'specifically optimized for code tasks' (Section IV-A). Any observed difference in early-exit behavior between the two backbones could be due to the different pretraining corpora and domains rather than to the architectural properties under study. The authors should either use Mamba and Transformer models trained on the same data (e.g., the same family with comparable checkpoints), or explicitly argue why the pretraining difference is irrelevant to early-exit behavior, or provide a matched ablation that isolates architecture from pretraining.
  3. [Section IV-D] The proposed KV-cache copying trick is incompletely specified and unvalidated: the passage describing the update of subsequent Transformer blocks ends mid-sentence ('will have its KV cache updated with the keys and value from the'), and no experiment isolates the effect of copying versus recomputation on generation quality. The claim that copying cached representations from the exit layer preserves enough information for correct generation is an axiom that needs empirical support. Please complete the algorithm description, state exactly which keys and values are copied, and provide an ablation comparing (a) copying, (b) partial forward recomputation, and (c) no state update, with respect to accuracy and consistency.
  4. [Figures 1–5] All experimental results are presented as single deterministic-looking curves without error bars, repeated runs, or statistical significance tests. The training of EE classifiers involves stochastic optimization, and threshold sweeps produce curves whose differences between EE configurations and layer pruning may be within run-to-run variance. Adding multiple seeds (or bootstrap confidence intervals over the evaluation samples) is necessary to support the qualitative claims such as 'Mamba shows as a better model for EE prediction' and 'EE configurations provide better performance.'
minor comments (6)
  1. [Algorithm 1] In Algorithm 1, the condition 'exit output ≥ θ' is ambiguous because the classifier outputs two values passed through a softmax; clarify that the exit decision uses the probability of the 'exit' class, and define the notation for 'partial forward(z)'.
  2. [Section IV-D] The sentence describing the KV-cache copying mechanism is syntactically incomplete (it ends with 'the keys and value from the'); please finish the sentence and provide a precise description of what is copied and how the cache tensors are sized.
  3. [Section V] The phrase 'a larger spawn over the x axis' appears to be a typo; 'span' is likely intended. Also, the phrase 'a certain number of times' should be replaced with a concrete numeric criterion.
  4. [Section V-B] The paper states that a configuration of four early exits is placed in the second half of the model's backbone, but it does not specify the exact layer indices for either Mistral or Codestral Mamba. Providing these indices would improve reproducibility.
  5. [Section IV-B] The repetition penalty is mentioned in the Mamba inference paragraph but its value and how it is applied are not described; please give the formula or reference.
  6. [Related Work] The related-work discussion would benefit from a direct quantitative comparison with existing EE frameworks for decoder-only LLMs (e.g., CALM and EE-LLM) rather than only citing them; at minimum, the chosen baselines (layer pruning and full model) should be supplemented with a CALM-style baseline on the same architecture.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DYNAMAX is evaluated against external task metrics with swept thresholds, not fitted to produce the headline result.

full rationale

The claimed derivation chain is empirical rather than definitional. The early-exit classifiers are trained by knowledge distillation from the full model (the 'oracle'), but the reported accuracy, exact match, F1, BLEU, and ROUGE scores are measured against external ground-truth datasets (TriviaQA, CoQA, TruthfulQA), not against the training objective or the oracle itself. The confidence threshold is swept, not fitted to a subset of the evaluation data, and no parameter is optimized to make the savings-accuracy curves appear. The self-citation [31] appears in a general list of training methods and is not load-bearing for any central claim. The only notable issue is the post-hoc exclusion of degenerate low-threshold configurations described in Section V ('Sometimes, when the threshold is set too low... that configuration for the evaluation is not considered valid and then not shown in the graphs'); this is an unquantified selection rule that could bias the plotted trade-off, but it does not reduce the reported predictions to their inputs by construction. Because no equation is equated with an input and no fitted parameter is renamed as a prediction, the paper does not exhibit circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The ledger shows a handful of free design choices (threshold, exit placement, k, repetition penalty) that are either swept or unreported. The load-bearing assumptions are the approximate FLOP model for Mamba and the two state-handling approximations (KV-copy, skipped Mamba updates), none of which are validated directly.

free parameters (4)
  • confidence threshold theta = swept over range, not fixed
    The threshold is swept to trace the savings-performance frontier, and a single shared value is used across all exits. The choice of threshold determines which points are reported and which invalid ones are excluded.
  • number and placement of exits = 4 exits in the second half of the backbone
    Chosen to balance overhead and accuracy, but no ablation is shown for other placements.
  • top-k relaxation parameter k = not reported
    Tolerates mismatches between early-exit and oracle token sets during training; the value is not given, so its influence on the results is unknown.
  • repetition penalty coefficient = not reported
    Used to avoid degenerate repeated-token output, but the coefficient is not specified, making the post-hoc filtering rule partially dependent on an unreported choice.
assumptions (5)
  • domain assumption Mamba block computational cost is dominated by its projection matrices, with the 1D convolution and SSM being at least an order of magnitude smaller.
    Equation 2 is used to compute the savings factor; if the small components are not negligible in practice, the reported efficiency gains are overestimated.
  • ad hoc to paper Copying the KV cache from the exit layer to subsequent Transformer blocks preserves enough information for correct generation.
    Section IV-D proposes copying cached values directly instead of recomputation, but the sentence is cut off and no experiment validates the approximation.
  • ad hoc to paper Skipped state updates in Mamba are eventually corrected by tokens that later pass through the full model.
    The paper relies on this assumption when skipping recomputation in the Mamba backbone; no analysis supports that the recurrent state stays coherent.
  • standard math The full model's output can serve as a reliable oracle for distillation.
    Knowledge distillation from the full model is standard practice, so this is not a weakness, though it ties exit quality to the base model's behavior.
  • domain assumption The selected QA datasets and metrics capture the intended generation-preserving efficiency tradeoff.
    Results rely on exact match, F1, BLEU and Rouge from a 1000-sample TriviaQA subset; the subsetting rule is described only as 'longer answers'.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DYNAMAX: Dynamic computing for Transformers and Mamba based architectures." pith.science (2026). https://pith.science/paper/V647IXI4

@misc{pith2026250420922,
  author       = {Pith},
  title        = {Pith review of: DYNAMAX: Dynamic computing for Transformers and Mamba based architectures},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/V647IXI4}},
  note         = {Machine review of arXiv:2504.20922}
}
read the original abstract

Early exits (EEs) offer a promising approach to reducing computational costs and latency by dynamically terminating inference once a satisfactory prediction confidence on a data sample is achieved. Although many works integrate EEs into encoder-only Transformers, their application to decoder-only architectures and, more importantly, Mamba models, a novel family of state-space architectures in the LLM realm, remains insufficiently explored. This work introduces DYNAMAX, the first framework to exploit the unique properties of Mamba architectures for early exit mechanisms. We not only integrate EEs into Mamba but also repurpose Mamba as an efficient EE classifier for both Mamba-based and transformer-based LLMs, showcasing its versatility. Our experiments employ the Mistral 7B transformer compared to the Codestral 7B Mamba model, using data sets such as TruthfulQA, CoQA, and TriviaQA to evaluate computational savings, accuracy, and consistency. The results highlight the adaptability of Mamba as a powerful EE classifier and its efficiency in balancing computational cost and performance quality across NLP tasks. By leveraging Mamba's inherent design for dynamic processing, we open pathways for scalable and efficient inference in embedded applications and resource-constrained environments. This study underscores the transformative potential of Mamba in redefining dynamic computing paradigms for LLMs.

Figures

Figures reproduced from arXiv: 2504.20922 by the authors.

Figure 1
Figure 1. Comparison of EE and layer pruning performances in the TriviaQA [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 4
Figure 4. Performance of EE and layer pruning in the Truthful QA generation [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 3
Figure 3. Performance of EE and layer pruning in the CoQA set for Mamba. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SEAL: Searching Expandable Architectures for Incremental Learning

    cs.LG 2025-05 conditional novelty 5.0 of 10

    SEAL jointly searches a neural network architecture and an expansion policy, expanding the network only when a capacity threshold is exceeded, and reports competitive accuracy with lower average forgetting on CIFAR-10...

Reference graph

Works this paper leans on

44 extracted references · 38 canonical work pages · cited by 1 Pith paper

  1. [1]

    Scaling Autoregressive Multi- Modal Models: Pretraining and Instruction Tuning,

    L. Yu, B. Shi, R. Pasunuru, B. Muller, O. Golovneva, T. Wang, A. Babu, B. Tang, B. Karrer, S. Sheynin, C. Ross, A. Polyak, R. Howes, V . Sharma, P. Xu, H. Tamoyan, O. Ashual, U. Singer, S.-W. Li, S. Zhang, R. James, G. Ghosh, Y . Taigman, M. Fazel-Zarandi, A. Celikyilmaz, L. Zettlemoyer, and A. Aghajanyan, “Scaling Autoregressive Multi- Modal Models: Pret...

    work page 2023

  2. [2]

    Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study,

    B. Wang, W. Ping, P. Xu, L. McAfee, Z. Liu, M. Shoeybi, Y . Dong, O. Kuchaiev, B. Li, C. Xiao, A. Anandkumar, and B. Catanzaro, “Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study,” ACL Anthology, pp. 7763–7786, Dec. 2023

    work page 2023

  3. [3]

    Attention is All you Need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is All you Need,” Advances in Neural Information Processing Systems , vol. 30, 2017

    work page 2017

  4. [4]

    LLaMA: Open and Efficient Foundation Language Models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi `ere, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample, “LLaMA: Open and Efficient Foundation Language Models,” arXiv, Feb. 2023

    work page 2023

  5. [5]

    Mistral 7B,

    A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. d. l. Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, L. R. Lavaud, M.-A. Lachaux, P. Stock, T. L. Scao, T. Lavril, T. Wang, T. Lacroix, and W. E. Sayed, “Mistral 7B,” arXiv, Oct. 2023

    work page 2023

  6. [6]

    Language models are unsupervised multitask learners,

    A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, “Language models are unsupervised multitask learners,” 2019

    2019

  7. [7]

    Claude 3 model card,

    Anthropic, “Claude 3 model card,” tech. rep., Anthropic Inc., 2023. 2024-03-04

    work page 2023

  8. [8]

    Scaling LLM Test-Time Com- pute Optimally can be More Effective than Scaling Model Parameters,

    C. Snell, J. Lee, K. Xu, and A. Kumar, “Scaling LLM Test-Time Com- pute Optimally can be More Effective than Scaling Model Parameters,” arXiv, Aug. 2024

    work page 2024

Show all 44 references
  1. [9]

    Learning to reason with LLMs,

    OpenAI, “Learning to reason with LLMs,” Sept. 2024

    2024

  2. [10]

    DeepSeek-R1: Incentiviz- ing Reasoning Capability in LLMs via Reinforcement Learning,

    D.-A. I., D. Guo, D. Yang, and e. A. Zhang, “DeepSeek-R1: Incentiviz- ing Reasoning Capability in LLMs via Reinforcement Learning,” arXiv, Jan. 2025

    2025

  3. [11]

    Mamba: Linear-Time Sequence Modeling with Selective State Spaces,

    A. Gu and T. Dao, “Mamba: Linear-Time Sequence Modeling with Selective State Spaces,” arXiv, Dec. 2023

    2023

  4. [12]

    Early exit optimizations for additive machine learned ranking systems,

    B. B. Cambazoglu, H. Zaragoza, O. Chapelle, J. Chen, C. Liao, Z. Zheng, and J. Degenhardt, “Early exit optimizations for additive machine learned ranking systems,” in Proceedings of the third ACM international conference on Web search and data mining , pp. 411–420, 2010

    2010

  5. [13]

    Branchynet: Fast inference via early exiting from deep neural networks,

    S. Teerapittayanon, B. McDanel, and H.-T. Kung, “Branchynet: Fast inference via early exiting from deep neural networks,” in 2016 23rd international conference on pattern recognition (ICPR) , pp. 2464–2469, IEEE, 2016

    2016

  6. [14]

    SkipNet: Learning Dynamic Routing in Convolutional Networks,

    X. Wang, F. Yu, Z.-Y . Dou, T. Darrell, and J. E. Gonzalez, “SkipNet: Learning Dynamic Routing in Convolutional Networks,” arXiv, Nov. 2017

    2017

  7. [15]

    Dynabert: Dynamic bert with adaptive width and depth,

    L. Hou, Z. Huang, L. Shang, X. Jiang, X. Chen, and Q. Liu, “Dynabert: Dynamic bert with adaptive width and depth,” Advances in Neural Information Processing Systems , vol. 33, pp. 9782–9793, 2020

    2020

  8. [16]

    The Unreasonable Ineffectiveness of the Deeper Layers,

    A. Gromov, K. Tirumala, H. Shapourian, P. Glorioso, and D. A. Roberts, “The Unreasonable Ineffectiveness of the Deeper Layers,” arXiv, Mar. 2024

    2024

  9. [17]

    BERT: Pre- training of Deep Bidirectional Transformers for Language Understand- ing,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre- training of Deep Bidirectional Transformers for Language Understand- ing,” arXiv, Oct. 2018

    2018

  10. [18]

    AdaViT: Adaptive Tokens for Efficient Vision Transformer,

    H. Yin, A. Vahdat, J. Alvarez, A. Mallya, J. Kautz, and P. Molchanov, “AdaViT: Adaptive Tokens for Efficient Vision Transformer,” arXiv, Dec. 2021

    2021

  11. [19]

    Lgvit: Dy- namic early exiting for accelerating vision transformer,

    G. Xu, J. Hao, L. Shen, H. Hu, Y . Luo, H. Lin, and J. Shen, “Lgvit: Dy- namic early exiting for accelerating vision transformer,” in Proceedings of the 31st ACM International Conference on Multimedia , pp. 9103– 9114, 2023

    2023

  12. [20]

    Nodeformer: A scalable graph structure learning transformer for node classification,

    Q. Wu, W. Zhao, Z. Li, D. P. Wipf, and J. Yan, “Nodeformer: A scalable graph structure learning transformer for node classification,” Advances in Neural Information Processing Systems , vol. 35, pp. 27387–27401, 2022

    2022

  13. [21]

    Cas- cadeBERT: Accelerating Inference of Pre-trained Language Models via Calibrated Complete Models Cascade,

    L. Li, Y . Lin, D. Chen, S. Ren, P. Li, J. Zhou, and X. Sun, “Cas- cadeBERT: Accelerating Inference of Pre-trained Language Models via Calibrated Complete Models Cascade,” arXiv, Dec. 2020

    2020

  14. [22]

    FastBERT: a Self-distilling BERT with Adaptive Inference Time,

    W. Liu, P. Zhou, Z. Zhao, Z. Wang, H. Deng, and Q. Ju, “FastBERT: a Self-distilling BERT with Adaptive Inference Time,” arXiv, Apr. 2020

    2020

  15. [23]

    Depth-Adaptive Trans- former,

    M. Elbayad, J. Gu, E. Grave, and M. Auli, “Depth-Adaptive Trans- former,” arXiv, Oct. 2019

    2019

  16. [24]

    Confident adaptive language modeling,

    T. Schuster, A. Fisch, J. Gupta, M. Dehghani, D. Bahri, V . Tran, Y . Tay, and D. Metzler, “Confident adaptive language modeling,” Advances in Neural Information Processing Systems, vol. 35, pp. 17456–17472, 2022

    2022

  17. [25]

    Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decodingt5,

    S. Bae, J. Ko, H. Song, and S.-Y . Yun, “Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decodingt5,” arXiv, Oct. 2023

    2023

  18. [26]

    EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelism,

    Y . Chen, X. Pan, Y . Li, B. Ding, and J. Zhou, “EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelism,” arXiv, Dec. 2023

    2023

  19. [27]

    Exploring the limits of transfer learning with a unified text-to-text transformer,

    C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y . Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” Journal of machine learning research, vol. 21, no. 140, pp. 1–67, 2020

    2020

  20. [28]

    Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey,

    Z. Han, C. Gao, J. Liu, J. Zhang, and S. Q. Zhang, “Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey,” arXiv, Mar. 2024

    2024

  21. [29]

    LoRA: Low-Rank Adaptation of Large Language Models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-Rank Adaptation of Large Language Models,” arXiv, June 2021

    2021

  22. [30]

    Transformers are SSMs: Generalized models and efficient algorithms through structured state space duality,

    T. Dao and A. Gu, “Transformers are SSMs: Generalized models and efficient algorithms through structured state space duality,” in Interna- tional Conference on Machine Learning (ICML) , 2024

    2024

  23. [31]

    Edanas: Adaptive neural architecture search for early exit neural networks,

    M. Gambella and M. Roveri, “Edanas: Adaptive neural architecture search for early exit neural networks,” in 2023 International Joint Conference on Neural Networks (IJCNN) , pp. 1–8, 2023

    2023

  24. [32]

    Greedy layer- wise training of deep networks,

    Y . Bengio, P. Lamblin, D. Popovici, and H. Larochelle, “Greedy layer- wise training of deep networks,” in Advances in Neural Information Processing Systems (B. Sch ¨olkopf, J. Platt, and T. Hoffman, eds.), vol. 19, MIT Press, 2006

    2006

  25. [33]

    Distilling the Knowledge in a Neural Network,

    G. Hinton, O. Vinyals, and J. Dean, “Distilling the Knowledge in a Neural Network,” arXiv, Mar. 2015

    2015

  26. [34]

    A survey on deep neural network pruning: Taxonomy, comparison, analysis, and recommendations,

    H. Cheng, M. Zhang, and J. Q. Shi, “A survey on deep neural network pruning: Taxonomy, comparison, analysis, and recommendations,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2024

    2024

  27. [35]

    The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits,

    S. Ma, H. Wang, L. Ma, L. Wang, W. Wang, S. Huang, L. Dong, R. Wang, J. Xue, and F. Wei, “The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits,” arXiv, Feb. 2024

    2024

  28. [36]

    Linformer: Self- Attention with Linear Complexity,

    S. Wang, B. Z. Li, M. Khabsa, H. Fang, and H. Ma, “Linformer: Self- Attention with Linear Complexity,” arXiv, June 2020

    2020

  29. [37]

    Longformer: The Long- Document Transformer,

    I. Beltagy, M. E. Peters, and A. Cohan, “Longformer: The Long- Document Transformer,” arXiv, Apr. 2020

    2020

  30. [38]

    Codestral mamba,

    Mistral AI Team, “Codestral mamba,” 2024. 2024-07-16

    2024

  31. [39]

    HuggingFace’s Transformers: State-of-the-art Natural Language Processing,

    T. Wolf, L. Debut, V . Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y . Jernite, J. Plu, C. Xu, T. L. Scao, S. Gugger, M. Drame, Q. Lhoest, and A. M. Rush, “HuggingFace’s Transformers: State-...

    2019

  32. [40]

    Fineweb-edu,

    A. Lozhkov, L. Ben Allal, L. von Werra, and T. Wolf, “Fineweb-edu,” May 2024

    2024

  33. [41]

    TruthfulQA: Measuring How Models Mimic Human Falsehoods,

    S. Lin, J. Hilton, and O. Evans, “TruthfulQA: Measuring How Models Mimic Human Falsehoods,” arXiv, Sept. 2021

    2021

  34. [42]

    Coqa: A conversational question answering challenge,

    S. Reddy, D. Chen, and C. D. Manning, “Coqa: A conversational question answering challenge,” Transactions of the Association for Computational Linguistics, vol. 7, pp. 249–266, 2019

    2019

  35. [43]

    TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehen- sion,

    M. Joshi, E. Choi, D. S. Weld, and L. Zettlemoyer, “TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehen- sion,” arXiv, May 2017

    2017

  36. [44]

    A framework for few-shot language model evaluation,

    L. Gao, J. Tow, B. Abbasi, S. Biderman, S. Black, A. DiPofi, C. Foster, L. Golding, J. Hsu, A. Le Noac’h, H. Li, K. McDonell, N. Muen- nighoff, C. Ociepa, J. Phang, L. Reynolds, H. Schoelkopf, A. Skowron, L. Sutawika, E. Tang, A. Thite, B. Wang, K. Wang, and A. Zou, “A framewo...

    2024

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.