Pith. sign in

REVIEW 4 major objections 6 minor 34 references

Multi-Level Modeling of Large Language Model Inference Latency and Energy via Hybrid Analytical--Machine-Learning Predictors

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read HYMELL predicts LLM inference latency and energy from architecture alone, with under 5% error on LLaMA 3 8B in both prefill and decode.

desk verdict A useful three-level hybrid cost model whose headline accuracy numbers currently can't be separated from interpolation because the train/test split is never pinned down. read the letter →

arxiv 2608.06723 v1 pith:3RJNFW3L submitted 2026-08-07 cs.LG cs.AI

Saeid Shokoufa , Mohammad Erfan Sadeghi , Mehdi Kamal , Massoud Pedram This is my paper · ORCID
classification cs.LGcs.AI
keywords LargelanguagemodelsInferencelatencyestimationEnergymodelingGPUpowerHybridanalytical-machinelearningMixture-of-expertsPrefillanddecodeDesignspaceexploration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

HYMELL is a three-level predictor that estimates the latency and energy of large-language-model inference directly from architectural parameters, so no GPU measurement is needed at prediction time. The paper claims that combining analytical regressions for primitive GPU operations (GEMM, Softmax, RMSNorm) with lightweight MLPs at the attention and feed-forward block levels, and then an end-to-end MLP, yields roughly 10% mean absolute percentage error across dense and mixture-of-experts architectures, and under 5% error for LLaMA 3 8B in both prefill and decode. This matters because it turns architecture exploration into a fast software computation: designers can sweep hidden widths, head counts, layer counts, and MoE routing settings to find energy-efficient or latency-efficient configurations without running each candidate on hardware. The framework also reports transfer to a second GPU after re-profiling, and to a linear-attention variant after adding a block estimator.

What carries the argument

The carrying mechanism is the three-level HYMELL hierarchy. Level 1 is a set of regime-aware analytical regressions: RMSNorm and Softmax each split into launch-bound versus memory-bound regimes by a workload threshold ($2.5\times10^6$ for RMSNorm, $N=1024$ for Softmax), while GEMM splits into five regimes keyed by total work $F$ and arithmetic intensity $\mathrm{AI}$, with a parallelism factor $\mathrm{PF}=MN/K$ and chosen interaction terms in a polynomial model. Level 2 consists of small multilayer perceptrons that take the level-1 predictions plus architecture features (heads, head dimension, sequence length, $d_{\text{model}}$, $d_{\text{intermediate}}$, MoE expert counts) and predict attention-block and FFN-block time and energy. Level 3 is a final MLP over block predictions plus global features (layers, sequence length, batch size, inference mode) that yields end-to-end estimates. The division of labor is what carries the argument: the analytical layer tracks the dominant, hardware-regular kernels, and each MLP layer absorbs only the residual variance left by the level below it.

What would settle it

Run a held-out set of GEMM shapes whose arithmetic intensities lie just on either side of the reported thresholds (AI_low=81.92, AI_mid=166.05, AI_high=288.91 FLOPs/byte, F_launch=8.39e6 FLOPs) and compare the polynomial estimator's predictions against measured cuBLAS latency and measured energy; if the error on those boundary shapes systematically exceeds the reported 9.4% balanced-regime MAPE, the regime partition is not capturing kernel-switching discontinuities, and end-to-end predictions will inherit the distortion on workloads concentrated in those regimes.

Watch

Extended reading notes

Core claim

The central claim is that inference cost for modern LLMs can be predicted accurately from architecture-level numbers by splitting the problem into three levels, each matched to the granularity where hardware behavior is regular. At level 1, analytical regressions with regime thresholds estimate runtime and energy of RMSNorm, Softmax, and GEMM; the GEMM model uses five regimes defined by total FLOPs and arithmetic intensity, with features such as a parallelism factor $MN/K$ and interaction terms to capture kernel selection. At level 2, small MLPs consume those operator-level predictions plus architecture parameters to predict attention and FFN block costs, absorbing residual overheads like reshaping, masking, activations, and MoE routing. At level 3, a final MLP combines block predictions with global parameters (layers, sequence length, batch size, prefill versus decode) to produce end-to-end estimates. The paper reports operator MAPEs below 7%, block-level MAPEs below 6%, end-to-end MAPEs of roughly 10–13% across dense and MoE configurations, under 5% for LLaMA 3 8B, and similar or better accuracy in cross-device and linear-attention ablations.

Load-bearing premise

The load-bearing premise is that a small set of fixed regime thresholds and a fixed polynomial form can approximate how the closed-source cuBLAS library actually performs on every GEMM shape; the paper's own balanced-regime errors, up to 9.4% MAPE, mark where cuBLAS switches kernels and tiling strategies, so if those boundaries are mis-specified the operator-level inputs to the MLPs are distorted and the end-to-end accuracy is unlikely to generalize beyond the profiled shape distribution.

Editorial extensions

If this is right

  • A designer can sweep model width, depth, head counts, and MoE configurations and receive latency and energy estimates for each point, making architecture-level search for efficient LLMs a software-only process.
  • Porting to a new GPU reduces to re-profiling the operator regressions and fine-tuning the small MLPs; the paper demonstrates this with an RTX A6000 ablation averaging about 8% MAPE.
  • New attention mechanisms can be folded into the framework by profiling a block-level estimator, as shown by the linear-attention ablation with sub-4% MAPE.
  • Because batch size is an explicit input feature, the framework claims to capture the performance and energy effects of serving optimizations such as PagedAttention and FlashAttention without changing the estimators.
  • The framework can be extended to heterogeneous continuous batching by evaluating the effective aggregated kernel workload rather than summing independent requests.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the reported accuracy survives broader shape distributions, HYMELL could serve as a cheap surrogate for GPU profiling inside neural architecture search, letting optimizers test thousands of LLM configurations per minute rather than per measurement run.
  • The fixed regime thresholds are the brittle point: since cuBLAS switches kernels and tiling strategies continuously, a natural hardening is to replace the hard thresholds with a learned or interpolated mixture over regimes, or to add features that track kernel-selection boundaries.
  • The claim that the framework is hardware-free at prediction time still presumes a one-time profiling pass per device; on accelerators without closed-source library optimizations the regime discontinuities may be smoother, which could make the analytical layer more portable rather than less.
  • A direct per-configuration head-to-head against AMALI on identical models and sequence lengths, rather than comparing against AMALI's reported aggregate MAPE, would give a crisper picture of where each approach wins.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents HYMELL, a three-level hybrid framework for predicting latency and energy of LLM inference on GPUs. Level 1 consists of 'analytical' estimators for RMSNorm, softmax, and GEMM, which are actually polynomial regressions with features motivated by workload and memory traffic. Level 2 uses lightweight MLPs to predict attention and FFN block costs from the Level-1 outputs plus architectural parameters. Level 3 combines block-level predictions with global model parameters in an MLP to estimate end-to-end prefill and decode latency/energy. Evaluation on an NVIDIA H100 reports operator-level MAPE of 1.7%-6.4%, block-level MAPE of 0.4%-6.3%, and end-to-end MAPE of about 10-13%, with under 5% error for LLaMA 3 8B at batch size 1. A cross-device experiment on an RTX A6000 and an ablation on linear attention are also included. The central claim is that HYMELL enables fast, hardware-free design-space exploration by predicting execution costs directly from architectural parameters.

Significance. If the reported accuracy generalizes beyond the profiled configurations, HYMELL would provide a useful surrogate for LLM inference cost estimation, supporting design-space exploration without repeated hardware measurements. The hierarchical decomposition into operator, block, and end-to-end levels is a sensible way to combine low-level kernel behavior with high-level architectural parameters, and the inclusion of dense and MoE FFNs, MHA and GQA, and both prefill and decode phases is a strength. The cross-device experiment (Table 8) and linear-attention ablation (Table 9) extend the evidence base beyond a single architecture/hardware combination. However, the validation methodology is not yet rigorous enough to establish the generalization claim: the construction of the held-out end-to-end set is not described, the operator-level estimators lack a train/test split, and no measure of variability is given for any of the reported metrics. The paper does not mention code or data release, which would also help reproducibility.

major comments (4)
  1. [Section 5.1, Eqs. (1)-(6), Tables 1-3] The operator-level 'analytical' estimators are polynomial regressions whose coefficients are fitted to measured data, but the manuscript never states whether the MAPE and R^2 values in Tables 1-3 are computed on the training data, a validation split, or held-out test data. Since these estimates are consumed by the upper-level MLPs, any in-sample optimism at Level 1 propagates upward and undermines the end-to-end generalization claim. The paper must specify the train/test split or cross-validation procedure for each operator estimator and report out-of-sample metrics, ideally with variance across splits.
  2. [Section 5.1-5.2, Tables 6-7, Figure 4] The end-to-end evaluation is described as being on 'held-out LLM architectures' (Table 6) and specific models (Table 7, Figure 4), but the paper does not explain how the held-out set was constructed, whether LLaMA 3 8B, Mistral 7B, and Qwen MoE were excluded from the ~1,200-configuration training set, or whether the tested models lie outside the architectural distribution covered by training. Without this information, the under-5% per-model results may simply reflect interpolation over the profiled families rather than the architecture-level extrapolation promised in the abstract and conclusions. The authors should describe the held-out split in detail and, ideally, evaluate on architecture families not present in the training set.
  3. [Section 3.3, Table 3, Section 5.2] The GEMM regime partition depends on five thresholds (F_thr, AI_low, AI_mid, AI_high) whose values are reported in Section 5.2, but no procedure for selecting these thresholds is given. If the thresholds were tuned on the same data used to evaluate the estimator, the regime-specific MAPE values (including the 9.399% for the balanced compute-dominant regime) may be optimistically biased, and the model may not generalize near regime boundaries where cuBLAS kernel switches occur. The authors should state how the thresholds were chosen (e.g., via a validation set or prior knowledge) and assess sensitivity to these values.
  4. [Section 5.2, Tables 1-9, Figure 4] None of the reported MAPE or R^2 values include error bars, confidence intervals, or repeated-training-seed variation. Given that all MLPs use stochastic optimizers and the datasets are finite, some variance across seeds and data splits is expected, and this variance is necessary to judge whether differences such as dense vs. MoE end-to-end MAPE are meaningful. The authors should add repeated-seed or repeated-split statistics, or at least report the range across a small number of runs.
minor comments (6)
  1. [Section 5.2, Table 7] The AMALI values in Table 7 are aggregate MAPE figures repeated across sequence lengths, not per-sequence measurements. The text acknowledges this, but the table layout may still mislead readers. Please add an explicit note in the table caption that AMALI does not report per-sequence errors or energy predictions.
  2. [Section 5.2, Figure 4 and Conclusions] The conclusion states that HYMELL achieves 'less than 5%' error for Llama3, Mistral, and Qwen models, but Table 7 reports numerical error only for LLaMA 3 8B; the claims for Mistral and Qwen rest on the visual representation in Figure 4, which does not show exact per-sequence values or error bars. Provide a table of per-model, per-sequence errors, or clarify that the figure is qualitative.
  3. [Section 3.3, Eqs. (5)-(6)] The feature set for the GEMM estimator is said to have been 'obtained via ablation', but the ablation procedure and the marginal contribution of each term are not reported. Including an ablation table or a brief description of the selection criterion would improve interpretability and support the claim that all retained features are significant.
  4. [Section 3.1, Eqs. (1)-(2)] The interpretation of log2(d_model) and N*log2(d_model) as 'tree-based reduction' depth is plausible for reductions but may not hold for all implementations of RMSNorm on modern GPUs. A brief justification or reference would help, since the regression coefficients absorb whatever the kernel actually does.
  5. [Section 5.1] The measurement methodology says each configuration was repeated until cumulative execution time exceeded 10 seconds, then later says large prefill workloads used only five repetitions. It would be helpful to state explicitly how many repetitions were used for the operator, block, and end-to-end datasets, and whether the 10-second rule applied uniformly.
  6. [General] Neither the abstract nor the experiments section mentions code or data availability. For a paper whose value depends on reproducible accuracy numbers, providing the profiling data and training scripts (or at least a clear statement of availability) would substantially strengthen the contribution.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: HYMELL's predictions are empirical ML fits evaluated on held-out data, not derivations equivalent to their inputs.

full rationale

The claimed derivation chain is an empirical pipeline: operator-level regressors (Eqs. 1–6) are fitted to measured primitive latencies and energies; block-level MLPs (Sections 3.4–3.5) are trained to predict measured attention and FFN costs from those operator estimates plus architectural features; the end-to-end MLP (Section 3.6) is trained on measured full-model costs from block predictions plus global features. At no point is a target quantity defined as a function of the inputs in a way that forces equality; each MLP has free parameters optimized against independent measurements, and Table 6 reports held-out evaluation. The word 'analytical' is used loosely—the operator models are polynomial regressions with fitted coefficients, and their reported per-regime accuracy (Tables 1–3) is not stated to be held-out—but this is a validation-reporting concern, not circularity. Self-citations ([16], [17], [19]) are not load-bearing for the central claim. The main unresolved risk is the undisclosed construction of the held-out set and whether LLaMA 3 8B, Mistral, and Qwen were excluded from training; if they were not, the under-5% figures would be interpolation rather than generalization, but the paper provides no evidence either way, so this cannot be scored as demonstrated circularity.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The framework's central claim rests on dozens of fitted regression coefficients, six manual thresholds, and MLP weights learned from profiled data. The polynomial forms are physically motivated but the coefficients are empirical, so the model is a calibrated surrogate rather than a first-principles derivation.

free parameters (5)
  • RMSNorm regression coefficients alpha_i^E, alpha_i^T = not disclosed
    Fit to profiled RMSNorm measurements; define the 'analytical' RMSNorm estimator (Eqs. 1-2).
  • Softmax regression coefficients beta_i^E, beta_i^T = not disclosed
    Fit to profiled softmax measurements; define the Softmax estimator (Eqs. 3-4).
  • GEMM regression coefficients gamma_i^E, gamma_i^T = not disclosed
    Fit to profiled GEMM measurements; define the GEMM estimator (Eqs. 5-6).
  • Regime thresholds th_RMS, th_Softmax, F_thr, AI_low, AI_mid, AI_high = 2.5e6, 1024, 8.39e6 FLOPs, 81.92, 166.05, 288.91 FLOPs/byte
    Chosen or fitted thresholds that determine which regression applies; no tuning procedure is described (Section 5.2).
  • MLP hyperparameters (layers, hidden units, learning rates) = 4 layers; attn 384, FFN 64, final 32; learning rates as listed
    Selected by the authors, not systematically searched; affect the fitted functions (Section 5.1).
assumptions (4)
  • domain assumption cuBLAS GEMM execution is well approximated by the five-regime polynomial model despite kernel discontinuity.
    Used throughout Section 3.3; the paper acknowledges balanced-regime errors from cuBLAS kernel switches.
  • domain assumption NVML energy readings and CUDA event timings during profiled regions accurately reflect the operation costs.
    Measurement methodology in Section 5.1 assumes these readings are ground truth.
  • domain assumption The roughly 1,200 model configurations are representative and are split into disjoint train and test sets.
    Section 5.1 describes the dataset but does not describe the split, randomization, or held-out protocol.
  • domain assumption RMSNorm and softmax costs are dominated by the polynomial terms N*d_model and N^2 respectively.
    Equations (1)-(4) assume this scaling; alternative memory access patterns might require different forms.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multi-Level Modeling of Large Language Model Inference Latency and Energy via Hybrid Analytical--Machine-Learning Predictors." pith.science (2026). https://pith.science/paper/3RJNFW3L

@misc{pith2026260806723,
  author       = {Pith},
  title        = {Pith review of: Multi-Level Modeling of Large Language Model Inference Latency and Energy via Hybrid Analytical--Machine-Learning Predictors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3RJNFW3L}},
  note         = {Machine review of arXiv:2608.06723}
}
read the original abstract

The rapid scaling of Large Language Models (LLMs) has significantly increased computational cost, energy consumption, and inference latency, making accurate estimation essential for sustainable artificial intelligence deployment and hardware-aware design. In this work, we introduce Hybrid Modeling for Energy and Latency of LLMs (HYMELL), a hybrid three-level framework for estimating LLM inference latency and energy by combining analytical modeling with machine learning (ML). HYMELL models LLM execution through a three-level hierarchy: analytical estimation of primitive operations, ML prediction of higher-level components, and an end-to-end model that captures system-level overheads across both prefill and decode phases. The framework supports diverse architectures, including dense and mixture-of-experts (MoE) feed-forward networks (FFNs), as well as multi-head attention (MHA) and grouped-query attention (GQA) mechanisms. Evaluated on an NVIDIA H100 graphics processing unit (GPU), HYMELL achieves high predictive accuracy; notably, for LLaMA 3 8B, it attains less than 5% error for both prefill and decode phases. By predicting execution costs directly from architectural parameters, it enables fast, hardware-free design space exploration and energy-efficient optimization.

Figures

Figures reproduced from arXiv: 2608.06723 by the authors.

Figure 1
Figure 1. General architecture of an LLM (𝐿 and 𝑁𝐻 represent the number of layers and attention heads, respectively). research has focused on reducing NN and LLM energy through tech￾niques such as pruning, quantization, efficient attention, sparsity, and memory-efficient serving [6–14]. NNs, and especially LLMs, are now used across a wide range of applications, including natural￾language processing, engineering, healthcare, f… view at source ↗
Figure 2
Figure 2. Overview of the HYMELL hierarchical prediction framework. Level 1 analytically models primitive GPU operators. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Effect of the number of attention heads on execution [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Normalized prediction accuracy (batch=1). Top: time [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 20 canonical work pages

  1. [1]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. InAdvances in Neural Information Processing Systems, volume 30, 2017

    2017

  2. [2]

    Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity.Journal of Machine Learning Research, 23(120):1–39, 2022

    William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity.Journal of Machine Learning Research, 23(120):1–39, 2022

    2022

  3. [3]

    Smith, and Oren Etzioni

    Roy Schwartz, Jesse Dodge, Noah A. Smith, and Oren Etzioni. Green ai, 2019

    work page 2019

  4. [4]

    Power hungry processing: Watts driving the cost of AI deployment? InProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, pages 85–99, 2024

    Alexandra Sasha Luccioni, Yacine Jernite, and Emma Strubell. Power hungry processing: Watts driving the cost of AI deployment? InProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, pages 85–99, 2024

    work page 2024

  5. [5]

    Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David So, Maud Texier, and Jeff Dean

    David Patterson, Joseph Gonzalez, Quoc V. Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David So, Maud Texier, and Jeff Dean. Carbon emissions and large neural network training.arXiv preprint arXiv:2104.10350, 2021

    arXiv 2021

  6. [6]

    Song Han, Huizi Mao, and William J. Dally. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. InInternational Conference on Learning Representations, 2016

    2016

  7. [7]

    Integration of oscillator-based feature extraction for energy- efficient convolutional neural networks.Journal of Applied Physics, 139(23), 2026

    Seyedeh Atiyeh Abbasi Jalal, Ragib Ahsan, Zezhi Wu, Mirbehrad Mousavi, and Rehan Kapadia. Integration of oscillator-based feature extraction for energy- efficient convolutional neural networks.Journal of Applied Physics, 139(23), 2026

    work page 2026

  8. [8]

    Data driven control of defect formation in solution de- posited sb2se3 thin films.Advanced Materials Interfaces, 12(24):e00798, 2025

    Marissa J Strumolo, Zhaohong Sun, Iliya Larimojarad, Rehan Kapadia, and Richard L Brutchey. Data driven control of defect formation in solution de- posited sb2se3 thin films.Advanced Materials Interfaces, 12(24):e00798, 2025

    work page 2025

Show all 34 references
  1. [9]

    GPTQ: Ac- curate post-training quantization for generative pre-trained transformers

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. GPTQ: Ac- curate post-training quantization for generative pre-trained transformers. In International Conference on Learning Representations (ICLR), 2023

    2023

  2. [10]

    LLM.int8(): 8-bit matrix multiplication for transformers at scale

    Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. LLM.int8(): 8-bit matrix multiplication for transformers at scale. InAdvances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  3. [11]

    Fu, Stefano Ermon, Atri Rudra, and Christopher Ré

    Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashatten- tion: Fast and memory-efficient exact attention with io-awareness. InAdvances in Neural Information Processing Systems, volume 35, 2022

    2022

  4. [12]

    Flashattention-2: Faster attention with better parallelism and work partitioning, 2023

    Tri Dao. Flashattention-2: Faster attention with better parallelism and work partitioning, 2023

    2023

  5. [13]

    SparseGPT: Massive language models can be accurately pruned in one-shot

    Elias Frantar and Dan Alistarh. SparseGPT: Massive language models can be accurately pruned in one-shot. InInternational Conference on Machine Learning (ICML), 2023

    2023

  6. [14]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with PagedAttention. InACM SIGOPS Symposium on Operating Systems Principles (SOSP), 2023. 8

    2023

  7. [15]

    Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374, 2021

    Mark Chen, Jerry Tworek, Heewoo Jun, et al. Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374, 2021

    2021 arXiv

  8. [16]

    Unified LLM model for power, performance, and area prediction from hardware code

    Armin Abdollahi, Mehdi Kamal, and Massoud Pedram. Unified LLM model for power, performance, and area prediction from hardware code. InNinth Conference on Machine Learning and Systems, 2026

    2026

  9. [17]

    Hdlforge: A two-stage multi-agent framework for efficient verilog code generation with adaptive model escalation, 2026

    Armin Abdollahi, Saeid Shokoufa, Negin Ashrafi, Mehdi Kamal, and Massoud Pedram. Hdlforge: A two-stage multi-agent framework for efficient verilog code generation with adaptive model escalation, 2026

    2026

  10. [18]

    Semi-supervised gan with hybrid regulariza- tion and evolutionary hyperparameter tuning for accurate melanoma detection

    Alireza Golkarieh, Parsa Razmara, Ahmadreza Lagzian, Amirhosein Dolatabadi, and Seyed Jalaleddin Mousavirad. Semi-supervised gan with hybrid regulariza- tion and evolutionary hyperparameter tuning for accurate melanoma detection. Scientific Reports, 15(1):31977, 2025

    2025

  11. [19]

    Coft: Counterfactual- conformal decoding for fair chain-of-thought reasoning in large language models

    Arya Fayyazi, Mehdi Kamal, and Massoud Pedram. Coft: Counterfactual- conformal decoding for fair chain-of-thought reasoning in large language models. arXiv preprint arXiv:2605.30641, 2026

    2026 arXiv

  12. [20]

    Preserving privacy and utility in llm-based product recommendations

    Tina Khezresmaeilzadeh, Jiang Zhang, Dimitrios Andreadis, and Konstantinos Psounis. Preserving privacy and utility in llm-based product recommendations. arXiv preprint arXiv:2505.00951, 2025

    2025 arXiv

  13. [21]

    Large language models encode clinical knowledge.Nature, 620:172–180, 2023

    Karan Singhal, Shekoofeh Azizi, Tao Tu, et al. Large language models encode clinical knowledge.Nature, 620:172–180, 2023

    2023

  14. [22]

    BloombergGPT: A large language model for finance.arXiv preprint arXiv:2303.17564, 2023

    Shijie Wu, Ozan Irsoy, Steven Lu, et al. BloombergGPT: A large language model for finance.arXiv preprint arXiv:2303.17564, 2023

    2023 arXiv

  15. [23]

    Ho, et al

    Neel Guha, Julian Nyarko, Daniel E. Ho, et al. LegalBench: A collaboratively built benchmark for measuring legal reasoning in large language models. InAdvances in Neural Information Processing Systems, volume 36, 2023

    2023

  16. [24]

    ChatGPT for good? on opportunities and challenges of large language models for education.Learning and Individual Differences, 103:102274, 2023

    Enkelejda Kasneci, Kathrin Seßler, Stefan Küchemann, et al. ChatGPT for good? on opportunities and challenges of large language models for education.Learning and Individual Differences, 103:102274, 2023

    2023

  17. [25]

    Amali: An analytical model for accurately modeling llm inference on modern gpus

    Shiheng Cao, Junmin Wu, Junshi Chen, Hong An, and Zhibin Yu. Amali: An analytical model for accurately modeling llm inference on modern gpus. In Proceedings of the 52nd Annual International Symposium on Computer Architecture, pages 1495–1508, 2025

    2025

  18. [26]

    Forecasting llm inference performance via hardware-agnostic analytical modeling, 2025

    Rajeev Patwari, Ashish Sirasao, and Devleena Das. Forecasting llm inference performance via hardware-agnostic analytical modeling, 2025

    2025

  19. [27]

    Espos- ito, Francesco Antici, Daniele Cesarini, Zeynep Kiziltan, and Andrea Bartolini

    Hiari Pizzini Cavagna, Andrea Proia, Giacomo Madella, Giovanni B. Espos- ito, Francesco Antici, Daniele Cesarini, Zeynep Kiziltan, and Andrea Bartolini. Sweetspot: An analytical model for predicting energy efficiency of llm inference, 2026

    2026

  20. [28]

    Calatrava-Nicolas, Vishal Banwari, Paul Lukowicz, and Jakob Karolus

    Lars Krupp, Daniel Geißler, Francisco M. Calatrava-Nicolas, Vishal Banwari, Paul Lukowicz, and Jakob Karolus. This is taking too long – investigating time as a proxy for energy consumption of llms, 2026

    2026

  21. [29]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention, 2023

    2023

  22. [30]

    Fu, Stefano Ermon, Atri Rudra, and Christopher Ré

    Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashat- tention: Fast and memory-efficient exact attention with io-awareness, 2022

    2022

  23. [31]

    Flashattention-3: Fast and accurate attention with asynchrony and low-precision, 2024

    Jay Shah, Ganesh Bikshandi, Ying Zhang, Vijay Thakkar, Pradeep Ramani, and Tri Dao. Flashattention-3: Fast and accurate attention with asynchrony and low-precision, 2024

    2024

  24. [32]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  25. [33]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, De- vendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thoma...

    2023

  26. [34]

    Qwen technical report

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Si...

    2023 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.