Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

GREEN-CODE: Learning to Optimize Energy Efficiency in LLM-based Code Generation

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An RL agent decides when a code-generation LLM can stop early, cutting inference energy by 23–50% without meaningful accuracy loss.

desk verdict GREEN-CODE offers a plausible integration of LITE-style fine-tuning with an RL-based exit policy for code generation, but the headline 23–50% energy claim is overstated and not yet validated for the standard KV-cached deployment setting. read the letter →

arxiv 2501.11006 v2 pith:RVOU4YRW submitted 2025-01-19 cs.DC cs.AIcs.PFcs.SE

Shashikant Ilager , Lukas Florian Briem , Ivona Brandic This is my paper · ORCID
classification cs.DCcs.AIcs.PFcs.SE
keywords energy-efficientinferenceearlyexitreinforcementlearningcodegenerationlargelanguagemodelsLLMgreenAIdynamic
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

GREEN-CODE claims that a reinforcement-learning agent can decide, token by token, when a code-generation transformer has processed enough layers to produce a good prediction, and that acting on those decisions cuts inference energy by 23–50% on average while keeping code quality close to running all layers. The paper shows this on two open-source LLMs, Llama 3.2 3B and OPT 2.7B, and two code-completion datasets, JavaCorpus and PY150. The authors argue that, unlike static early-exit heuristics or methods that add extra language-model heads, their approach is dynamic, adds little overhead, and treats energy as a first-class objective. If the claim holds, energy-hungry code-assistance tools could become substantially cheaper to operate without retraining or degrading the developer experience.

What carries the argument

The central object is a small policy network that maps a transformer layer's hidden state to one of two actions, continue or exit, trained with proximal policy optimization. Three components carry the argument. First, fine-tuning with a weighted aggregated loss, adapted from the LITE approach, makes intermediate layers able to produce token predictions through the original single language-model head, removing the need for extra heads at each exit. Second, the reward function defines the optimal exit layer as the first layer whose token prediction equals the final layer's prediction, and gives a reward of 1 for exiting or continuing at that point, with penalties proportional to layer distance when the agent exits too early or too late. Third, a softmax threshold at inference turns the policy's stochastic output into a deterministic exit decision, so the same trained agent covers the whole range from aggressive savings to near-full accuracy. The KV-cache incompatibility introduced by nonmonotone exits is mitigated by propagating cached states from earlier layers.

What would settle it

Run GREEN-CODE with the default reward coefficients on a third code dataset, such as JavaScript or C#, and compare the agent's chosen exit layers against the layers where the final layer's prediction first becomes stable; if the agent's exits do not concentrate near those optimal layers, or if agreement with the final layer does not track human-judged code correctness, the reported energy savings will not generalize beyond the two evaluated datasets.

Watch

Extended reading notes

Core claim

The paper claims that a reinforcement-learning agent can decide token by token when a code-generation transformer has processed enough layers, and that this dynamic early exit cuts inference energy by 23–50% on average while keeping code quality close to the full-model baseline. The agent observes the hidden state at the current layer and chooses 'continue' or 'exit'; its reward, defined in Equations 2 and 3, compares the current layer's token prediction with the final layer's prediction and measures how far the current layer is from the shallowest layer that agrees with the final layer. After fine-tuning the model with an aggregated loss so intermediate layers can decode with the original LM head, exit decisions are made at alternating early layers and every fourth layer deeper in the network. At the most conservative threshold T=0.92 on JavaCorpus, Llama 3.2 reaches RougeL 0.41 versus 0.425 for the full model with 23% energy savings, and OPT reaches 0.39 versus 0.392 with at least 28% savings; on PY150, Llama reaches RougeL 0.46 versus 0.44 with roughly 29% savings. The conclusion is that the energy-accuracy trade-off can be controlled at runtime by a single threshold.

Load-bearing premise

The results rest on the assumption that a small policy reading only the current layer's hidden state can reliably detect when that layer's prediction already agrees with the model's final layer, and that this agreement is a faithful proxy for code quality.

Editorial extensions

If this is right

  • At T=0.92, GREEN-CODE saves roughly 23% of inference energy on JavaCorpus and about 29% on PY150 while keeping RougeL within a few points of the full model.
  • One trained policy, selected by threshold, spans the full trade-off curve from about half the energy at T=0.6 to near-full-model accuracy at T=0.92.
  • Because only one language-model head is used and the policy is a two-action network, per-token overhead stays below one fifth of total runtime, which the paper argues is acceptable for interactive code completion.
  • The reported results transfer across Llama 3.2 and OPT and across Java and Python datasets, indicating the mechanism is not tied to a single architecture or language.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to reward the agent for agreement with a stronger teacher or with the ground-truth token rather than the final layer's own prediction, which could let early exits beat the full model on some tokens instead of merely matching it.
  • The reward coefficients were set per dataset (beta=gamma=1 for JavaCorpus, 0.5 for PY150); testing GREEN-CODE on a third language or domain without retuning would reveal whether the learned exit policy generalizes or simply memorizes layer-accuracy profiles.
  • Since the agent sees only the current hidden state, exit decisions are made without memory of previous tokens; adding a small recurrent or summary state could exploit correlations among exit decisions within a line of code.
  • The sensitivity analysis shows larger accuracy losses at higher context lengths, so an adaptive threshold that rises with context length could recover accuracy while preserving most savings.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes GREEN-CODE, a framework for energy-efficient LLM-based code generation via dynamic early exit. It first fine-tunes Llama 3.2 3B and OPT 2.7B using an aggregated intermediate-layer loss (adapted from LITE) so that the models can decode from intermediate layers with a single LM head. It then trains a PPO agent that observes the current layer's hidden state and decides whether to exit or continue, with a reward function balancing accuracy, latency, and energy. The framework is evaluated on JavaCorpus and PY150 for line completion, reporting 23–50% energy savings with "comparable accuracy" compared with full-layer inference. The paper also reports a KV-cache experiment, a sensitivity analysis, and a VS Code extension demonstration.

Significance. If the central claim holds, GREEN-CODE is a useful contribution to green AI for code generation, an area where most early-exit work targets NLP accuracy and latency rather than directly measured energy. The single-LM-head fine-tuning approach is a practical way to avoid the overhead of multiple exit heads, and framing exit decisions as RL is a natural fit for dynamic accuracy-efficiency trade-offs. The paper is also commendable for shipping an open-source implementation, measuring energy with a hardware monitor, and explicitly analyzing the overhead of the RL agent. However, the current evidence does not fully support the headline "23–50% without significantly affecting accuracy" claim: the energy numbers come from a non-KV-cache setting, while the KV-cache experiment reports no energy or latency, and the abstract conflates conservative and aggressive thresholds with very different accuracy trade-offs. These gaps are fixable and do not undermine the overall approach, but they need to be addressed before the central claim can be accepted as stated.

major comments (4)
  1. [Section VI-G and Figs. 8–11] The headline energy-savings claim (23–50%) is computed from experiments run without KV caching, as Section VI-G explicitly states. Since KV caching is the standard deployment mode for code completion, the reported savings are not yet shown to transfer to the intended setting. The KV-cache experiment (Fig. 13) reports only accuracy and mean layers, not energy or latency, so it cannot be used to verify the energy claim. Please add energy and latency measurements for the KV-cache configuration, or clearly bound the claim to non-KV-cache inference.
  2. [Abstract and Section VI-E1] The abstract's statement that the method "reduces the energy consumption between 23-50% on average ... without significantly affecting accuracy" conflates two regimes. At the conservative threshold T=0.92, savings are 23–29% with near-parity RougeL (e.g., 0.41 vs 0.425 for Llama on JavaCorpus), while the upper end of the range (about 50% savings) corresponds to T=0.6, where RougeL drops from 0.42 to 0.29 (Fig. 8a). Please revise the abstract to present the accuracy–efficiency trade-off explicitly and to attribute the 23–50% range to the full threshold sweep.
  3. [Section IV-D, Eq. (2)] The reward function defines the "ground truth" y as the final-layer token prediction of the same model, not the actual token label. This makes the RL objective self-referential: the agent is trained to imitate the full model's predictions, and if the full model is wrong, early exit at a wrong prediction is rewarded. The paper should justify why matching the final layer is an appropriate proxy for code quality and should report whether the accuracy comparisons in Section VI-E use the actual ground-truth tokens.
  4. [Section VI-D] The reward coefficients β and γ are set per dataset (β=γ=1 for JavaCorpus and 0.5 for PY150) based on observed differences in lower-layer accuracy. This means the method requires dataset-specific reward tuning; the paper should state this explicitly as a limitation and discuss how the framework would be adapted to a new dataset without such manual adjustment.
minor comments (6)
  1. [Section III-D] The text "around around 15k samples" contains a duplicated word; please fix.
  2. [Table I] The entry "12.934" should read "12,934" to avoid confusion with a decimal number.
  3. [Equation (2)] The constant ϵ in the final condition of the reward is never defined; please specify its value and how it was chosen.
  4. [Section VI-H] The overhead numbers in Table IV are reported relative to the full model; it would be clearer to also report net energy savings after subtracting this overhead.
  5. [References] References [21] and [36] cite the same work with different titles; please unify them.
  6. [Section VI-G] The phrase "native incompatibility of KV caching with early exits" is vague; please explain the exact mechanism that makes caches unavailable after a shallow exit.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: GREEN-CODE's RL reward uses the model's own final-layer prediction as a training surrogate, but the reported accuracy and energy claims are measured externally against held-out ground truth and hardware energy readings.

full rationale

The claimed derivation chain is self-contained and externally grounded. The early-exit capability is obtained by fine-tuning with the LITE aggregated loss from external prior work [21], and the RL agent is trained on a reward (Eq. 2) whose "ground truth" y is the final layer's own token prediction and whose optimal exit l_opt is the first layer matching that prediction. This is a surrogate objective for preserving full-model behavior, not a circular definition: the paper never claims to derive external code accuracy from this reward. On the contrary, Section VI-E evaluates the deployed agent with RougeL, BLEU, CodeBLEU, syntax, and dataflow scores against held-out test tokens, and energy is measured with ZeusMonitor, so the 23-50% energy-reduction claim is an empirical measurement rather than a consequence of the reward design. The per-dataset reward coefficients beta and gamma in Section VI-D are transparently reported and reflect observed layer-wise accuracy; this is ordinary hyperparameter adjustment, not a fitted quantity renamed as a prediction. The KV-cache limitation acknowledged in Section VI-G, namely that the main experiments ran without caching and the KV-cache figure reports accuracy and layer counts but no energy, is a missing measurement for the deployment setting rather than a circular step. There are no load-bearing self-citations: the only author self-citation, [20], supports background on energy-aware GPU scheduling and is not used to justify GREEN-CODE's mechanism. The fine-tuning weights and exit-point schedule are stated design choices, not imported via an unsupported uniqueness claim. Therefore no step in the derivation reduces by construction to its own inputs.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The GREEN-CODE result rests on a stack of design choices: the LITE fine-tuning method, a hand-designed reward with per-dataset coefficients, an exit-layer schedule, and a threshold knob. None of these is derived from first principles; together they define the operating point at which the energy and accuracy numbers are measured.

free parameters (4)
  • Fine-tuning layer weight budgets (first half, second half, final) with decay factor = Alpha=0.7, 0.2, 0.1; r=0.9
    Hand-chosen in Section III-D to bias the model toward early-exit performance via geometric sequence weights.
  • RL reward coefficients beta and gamma = beta=gamma=1 for JavaCorpus; beta=gamma=0.5 for PY150
    Set per dataset in Section VI-D based on observed token-level accuracy of lower layers; alpha is only constrained to be <= beta and not explicitly given.
  • Inference softmax threshold T = 0.6, 0.8, 0.9, 0.91, 0.92
    Runtime knob swept in evaluation; headline energy savings are reported at the conservative end (T=0.92).
  • Exit layer configuration = Earliest exit at layer 4; alternating layers in first half, every fourth in second half (9 exits for Llama, 10 for OPT)
    Hand-designed in Section III-D to define candidate exit points.
assumptions (4)
  • domain assumption Fine-tuning with weighted aggregated loss (LITE) makes intermediate layers decodable with the original single LM head.
    Adopted from [21]/[36] without independent verification; the whole early-exit mechanism depends on it.
  • ad hoc to paper The model's own final-layer token prediction is a suitable target (ground truth) for learning exit decisions.
    Eq. 2 defines y as the last-layer prediction and uses it to compute all rewards; the paper does not show this proxy correlates with external code quality.
  • domain assumption A small MLP policy (2 layers, 32/64 hidden units) operating on the hidden state can learn a good exit policy.
    The state in Section IV-B is the current layer's hidden state; the paper asserts it encapsulates the current contextual understanding without an analysis of representational sufficiency.
  • domain assumption Reward stabilization implies converged RL policy.
    Section IV-F assumes convergence when cumulative reward stabilizes; no formal convergence guarantee is given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GREEN-CODE: Learning to Optimize Energy Efficiency in LLM-based Code Generation." pith.science (2026). https://pith.science/paper/RVOU4YRW

@misc{pith2026250111006,
  author       = {Pith},
  title        = {Pith review of: GREEN-CODE: Learning to Optimize Energy Efficiency in LLM-based Code Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RVOU4YRW}},
  note         = {Machine review of arXiv:2501.11006}
}
read the original abstract

Large Language Models (LLMs) are becoming integral to daily life, showcasing their vast potential across various Natural Language Processing (NLP) tasks. Beyond NLP, LLMs are increasingly used in software development tasks, such as code completion, modification, bug fixing, and code translation. Software engineers widely use tools like GitHub Copilot and Amazon Q, streamlining workflows and automating tasks with high accuracy. While the resource and energy intensity of LLM training is often highlighted, inference can be even more resource-intensive over time, as it's a continuous process with a high number of invocations. Therefore, developing resource-efficient alternatives for LLM inference is crucial for sustainability. This work proposes GREEN-CODE, a framework for energy-aware code generation in LLMs. GREEN-CODE performs dynamic early exit during LLM inference. We train a Reinforcement Learning (RL) agent that learns to balance the trade-offs between accuracy, latency, and energy consumption. Our approach is evaluated on two open-source LLMs, Llama 3.2 3B and OPT 2.7B, using the JavaCorpus and PY150 datasets. Results show that our method reduces the energy consumption between 23-50 % on average for code generation tasks without significantly affecting accuracy.

Figures

Figures reproduced from arXiv: 2501.11006 by the authors.

Figure 1
Figure 1. Performance, energy and latency of OPT-2.7B and [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. A high-level view of the system model. predicting exit points. Finally, we evaluate RL agent’s perfor￾mance capturing both model-related performance metrics and resource efficiency metrics, like energy consumption. Once the RL agent’s training converges, during the online phase, the RL agent and the fine-tuned LLMs are deployed on a service end point for runtime usage. In the following subsection, we discuss each of… view at source ↗
Figure 4
Figure 4. shows the loss curves of both models for two datasets with our fine-tuning method. As observed, after a few thousand training steps, the loss converges and further training has only insignificant effects. (a) Loss on JavaCorpus. (b) Loss on PY150 [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (8 more)
Figure 5
Figure 5. Figure 5: Illustration of the RL environ￾ment. One of the main components of for￾mulating any problem into the RL domain is to create an environ￾ment where an agent can interact and learn the complexities and trade-offs between different configurable and dependent pa￾rameters an…
Figure 6
Figure 6. Figure 6: Mean step reward per episode of PPO training with [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Occurrences of optimal exits during training of RL [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Results of Llama on JavaCorpus, with different GC [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Results of OPT on JavaCorpus, with different GC agent [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: Results of Llama on PY150, with different GC agent [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]
Figure 12
Figure 12. Figure 12: Results of Llama on JavaCorpus, with different GC [PITH_FULL_IMAGE:figures/full_fig_p009_12.png]
Figure 13
Figure 13. Figure 13: Results of Llama on JavaCorpus with KV caching. [PITH_FULL_IMAGE:figures/full_fig_p010_13.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Evaluating the Energy-Efficiency of the Code Generated by LLMs

    cs.SE 2025-05 conditional novelty 5.0 of 10

    LLM-generated Python solutions typically consume more energy than canonical human-written solutions, with DeepSeek-v3 and GPT-4o the most efficient LLMs and worst-case gaps near 450 times on certain problems.

Reference graph

Works this paper leans on

39 extracted references · 28 canonical work pages · cited by 1 Pith paper

  1. [1]

    Github copilot,

    “Github copilot,” accessed: October 4, 2024. [Online]. Available: https://github.com/features/copilot

    work page 2024

  2. [2]

    Amazon q developer, ai for software development

    Amazon, “Amazon q developer, ai for software development.” [Online]. Available: https://aws.amazon.com/de/q/developer/

    work page

  3. [3]

    Evaluating the code quality of ai-assisted code generation tools: An empirical study on github copilot, amazon codewhisperer, and chatgpt,

    B. Yetis ¸tiren, I. ¨Ozsoy, M. Ayerdem, and E. T ¨uz¨un, “Evaluating the code quality of ai-assisted code generation tools: An empirical study on github copilot, amazon codewhisperer, and chatgpt,” 2023. [Online]. Available: https://arxiv.org/abs/2304.10778

    arXiv 2023

  4. [4]

    The growing energy footprint of artificial intelligence,

    A. de Vries, “The growing energy footprint of artificial intelligence,” Joule, vol. 7, no. 10, pp. 2191–2194, 2023

    2023

  5. [5]

    Qlora: efficient finetuning of quantized llms,

    T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer, “Qlora: efficient finetuning of quantized llms,” in Proceedings of the 37th International Conference on Neural Information Processing Systems , 2024

    work page 2024

  6. [6]

    Quip: 2-bit quanti- zation of large language models with guarantees,

    J. Chee, Y . Cai, V . Kuleshov, and C. M. De Sa, “Quip: 2-bit quanti- zation of large language models with guarantees,” Advances in Neural Information Processing Systems , vol. 36, 2024

    work page 2024

  7. [7]

    Fp8 quantization: the power of the exponent,

    A. Kuzmin, M. Van Baalen, Y . Ren, M. Nagel, J. Peters, and T. Blankevoort, “Fp8 quantization: the power of the exponent,” in Proceedings of the 36th International Conference on Neural Information Processing Systems, 2022

    work page 2022

  8. [8]

    Post-training quantization with multiple points: Mixed precision without mixed precision,

    X. Liu, M. Ye, D. Zhou, and Q. Liu, “Post-training quantization with multiple points: Mixed precision without mixed precision,” in Proceedings of the AAAI Conference on Artificial Intelligence , 2021

    work page 2021

Show all 39 references
  1. [9]

    TinyBERT: Distilling BERT for natural language understanding,

    X. Jiao, Y . Yin, L. Shang, X. Jiang, X. Chen, L. Li, F. Wang, and Q. Liu, “TinyBERT: Distilling BERT for natural language understanding,” in Findings of the Association for Computational Linguistics: EMNLP

  2. [10]

    Mixed distillation helps smaller language models reason better,

    L. Chenglin, Q. Chen, L. Li, C. Wang, F. Tao, Y . Li, Z. Chen, and Y . Zhang, “Mixed distillation helps smaller language models reason better,” in Findings of the Association for Computational Linguistics: EMNLP, 2024

    2024

  3. [11]

    Baby llama: knowledge distillation from an ensemble of teachers trained on a small dataset with no performance penalty,

    J.-L. Tastet and I. Timiryasov, “Baby llama: knowledge distillation from an ensemble of teachers trained on a small dataset with no performance penalty,” arXiv preprint arXiv:2308.02019 , Aug 2023

    2023 arXiv

  4. [12]

    Llm-pruner: On the structural pruning of large language models,

    X. Ma, G. Fang, and X. Wang, “Llm-pruner: On the structural pruning of large language models,” Advances in neural information processing systems, vol. 36, pp. 21 702–21 720, 2023

    2023

  5. [13]

    LaCo: Large language model pruning via layer collapse,

    Y . Yang, Z. Cao, and H. Zhao, “LaCo: Large language model pruning via layer collapse,” in Findings of the Association for Computational Linguistics: EMNLP 2024. Association for Computational Linguistics, 2024

    2024

  6. [14]

    Structured optimal brain pruning for large language models,

    J. Wei, Q. Lu, N. Jiang, S. Li, J. Xiang, J. Chen, and Y . Liu, “Structured optimal brain pruning for large language models,” in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2024

    2024

  7. [15]

    DeeBERT: Dynamic early exiting for accelerating BERT inference,

    J. Xin, R. Tang, J. Lee, Y . Yu, and J. Lin, “DeeBERT: Dynamic early exiting for accelerating BERT inference,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , 2020

    2020

  8. [16]

    Berxit: Early exiting for bert with better fine-tuning and extension to regression,

    J. Xin, R. Tang, Y . Yu, and J. Lin, “Berxit: Early exiting for bert with better fine-tuning and extension to regression,” in Proceedings of the 16th conference of the European chapter of the association for computational linguistics, 2021, pp. 91–104

    2021

  9. [17]

    Confident adaptive language modeling,

    T. Schuster, A. Fisch, J. Gupta, M. Dehghani, D. Bahri, V . Tran, Y . Tay, and D. Metzler, “Confident adaptive language modeling,” in Advances in Neural Information Processing Systems , S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds., vol. 35. Curran As...

    2022

  10. [18]

    When neural code completion models size up the situation: Attaining cheaper and faster completion through dynamic model inference,

    Z. Sun, X. Du, F. Song, S. Wang, and L. Li, “When neural code completion models size up the situation: Attaining cheaper and faster completion through dynamic model inference,” in Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, ser. ICSE ’24, 2024

    2024

  11. [19]

    Consistentee: A consistent and hardness-guided early exiting method for accelerating language models inference,

    Z. Zeng, Y . Hong, H. Dai, H. Zhuang, and C. Chen, “Consistentee: A consistent and hardness-guided early exiting method for accelerating language models inference,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 17, pp. 19 506–19 514, Mar. 2024

    2024

  12. [20]

    A data-driven frequency scaling approach for deadline-aware energy efficient schedul- ing on graphics processing units (gpus),

    S. Ilager, R. Muralidhar, K. Rammohanrao, and R. Buyya, “A data-driven frequency scaling approach for deadline-aware energy efficient schedul- ing on graphics processing units (gpus),” in 2020 20th IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing (CCGR...

    2020

  13. [21]

    Accelerating llama inference by enabling intermediate layer decoding via instruction tuning with lite,

    N. Varshney, A. Chatterjee, M. Parmar, and C. Baral, “Accelerating llama inference by enabling intermediate layer decoding via instruction tuning with lite,” 2023

    2023

  14. [22]

    Codexglue: A machine learning benchmark dataset for code understanding and generation,

    S. Lu, D. Guo, S. Ren, J. Huang, A. Svyatkovskiy, A. Blanco, C. Clement, D. Drain, D. Jiang, D. Tang, G. Li, L. Zhou, L. Shou, L. Zhou, M. Tufano, M. Gong, M. Zhou, N. Duan, N. Sundaresan, S. K. Deng, S. Fu, and S. Liu, “Codexglue: A machine learning benchmark dataset for code...

    2021

  15. [23]

    Mining source code repositories at mas- sive scale using language modeling,

    M. Allamanis and C. Sutton, “Mining source code repositories at mas- sive scale using language modeling,” in 2013 10th Working Conference on Mining Software Repositories (MSR) , 2013, pp. 207–216

    2013

  16. [24]

    Probabilistic model for code with decision trees,

    V . Raychev, P. Bielik, and M. Vechev, “Probabilistic model for code with decision trees,” in Proceedings of the 2016 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications, ser. OOPSLA, 2016

    2016

  17. [25]

    Opt: Open pre-trained transformer language models,

    S. Zhang, S. Roller, N. Goyal, M. Artetxe, M. Chen, S. Chen, C. Dewan, M. Diab, X. Li, X. V . Lin, T. Mihaylov, M. Ott, S. Shleifer, K. Shuster, D. Simig, P. S. Koura, A. Sridhar, T. Wang, and L. Zettlemoyer, “Opt: Open pre-trained transformer language models,” 2022

    2022

  18. [26]

    Llama: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi `ere, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample, “Llama: Open and efficient foundation language models,” 2023

    2023

  19. [27]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems , 2017

    2017

  20. [28]

    R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 2nd ed. The MIT Press, 2018

    2018

  21. [29]

    Gymnasium: A standard interface for reinforcement learning environments,

    M. Towers, A. Kwiatkowski, J. Terry, J. U. Balis, G. D. Cola, T. Deleu, M. Goul ˜ao, A. Kallinteris, M. Krimmel, A. KG, R. Perez-Vicente, A. Pierr´e, S. Schulhoff, J. J. Tai, H. Tan, and O. G. Younis, “Gymnasium: A standard interface for reinforcement learning environments,” 2024

    2024

  22. [30]

    Openai gym,

    G. Brockman, V . Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba, “Openai gym,” 2016

    2016

  23. [31]

    Stable-baselines3: reliable reinforcement learning implementa- tions,

    A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus, and N. Dor- mann, “Stable-baselines3: reliable reinforcement learning implementa- tions,” J. Mach. Learn. Res. , vol. 22, no. 1, Jan. 2021

    2021

  24. [32]

    Proximal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” 2017

    2017

  25. [33]

    Out of the bleu: How should we assess quality of the code generation models?

    M. Evtikhiev, E. Bogomolov, Y . Sokolov, and T. Bryksin, “Out of the bleu: How should we assess quality of the code generation models?” Journal of Systems and Software , vol. 203, p. 111741, Sep. 2023

    2023

  26. [34]

    Codebleu: a method for automatic evaluation of code synthesis,

    S. Ren, D. Guo, S. Lu, L. Zhou, S. Liu, D. Tang, N. Sundaresan, M. Zhou, A. Blanco, and S. Ma, “Codebleu: a method for automatic evaluation of code synthesis,” 2020

    2020

  27. [35]

    LayerSkip: Enabling early exit inference and self-speculative decoding,

    M. Elhoushi, A. Shrivastava, D. Liskovich, B. Hosmer, B. Wasti, L. Lai, A. Mahmoud, B. Acun, S. Agarwal, A. Roman, A. Aly, B. Chen, and C.-J. Wu, “LayerSkip: Enabling early exit inference and self-speculative decoding,” in Proceedings of the 62nd Annual Meeting of the Associat...

    2024

  28. [36]

    Investigating ac- celeration of LLaMA inference by enabling intermediate layer decoding via instruction tuning with ‘LITE’,

    N. Varshney, A. Chatterjee, M. Parmar, and C. Baral, “Investigating ac- celeration of LLaMA inference by enabling intermediate layer decoding via instruction tuning with ‘LITE’,” in Findings of the Association for Computational Linguistics: NAACL , 2024

    2024

  29. [37]

    Jump to conclusions: Short-cutting transformers with linear transformations,

    A. Yom Din, T. Karidi, L. Choshen, and M. Geva, “Jump to conclusions: Short-cutting transformers with linear transformations,” in Proceedings of the 2024 Joint International Conference on Computational Lin- guistics, Language Resources and Evaluation (LREC-COLING 2024) . ELRA ...

    2024

  30. [38]

    A simple hash-based early exiting approach for language understanding and generation,

    T. Sun, X. Liu, W. Zhu, Z. Geng, L. Wu, Y . He, Y . Ni, G. Xie, X. Huang, and X. Qiu, “A simple hash-based early exiting approach for language understanding and generation,” in Findings of the Association for Computational Linguistics: ACL 2022 . Association for Computational ...

    2022

  31. [2020]

    Association for Computational Linguistics, 2020

    2020

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.