Pith. sign in

REVIEW 4 major objections 5 minor 51 references

PreNeT: Leveraging Computational Features to Predict Deep Neural Network Training Time

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Adding computational features to layer-wise predictors lowers training-time prediction error by up to 72%.

desk verdict Modest feature-engineering extension with useful new layer benchmarks; the paper overclaims SOTA status and the full-model additivity assumption needs validation before the headline numbers can be trusted. read the letter →

arxiv 2412.15519 v2 pith:5UUMO44Z submitted 2024-12-20 cs.LG

Alireza Pourali , Arian Boukani , Hamzeh Khazaei This is my paper · ORCID
classification cs.LG
keywords trainingtimepredictiondeepneuralnetworkslargelanguagemodelscomputationalcomplexitymemoryfootprintGPUbenchmarkinglayer-wiseresourceallocation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PreNeT is a framework for predicting how long a single training epoch will take before the training run starts. Its central claim is that per-layer prediction accuracy improves sharply when each layer is described not only by its shape and the GPU it runs on, but also by three computational features: the number of arithmetic operations it performs, the memory its weights and tensors occupy, and a hardware-normalized computational time equal to those operations divided by the GPU's theoretical peak FLOP rate. In experiments across attention, convolutional, embedding, recurrent, dense, and normalization layers, the added features lowered RMSE by up to 72% compared with a layer-wise baseline, with the largest gains on attention and recurrent layers, and the trained models predicted per-epoch times on two GPUs they had never seen. If the claim holds, practitioners can compare model configurations and GPU choices by cost and runtime before spending compute.

What carries the argument

The load-bearing machinery is the per-layer feature vector $F_{\text{layer}}$ that combines layer-specific shapes with three computational feature families: CO, the arithmetic operation count (e.g., $CO_{\text{Dense}} = S \cdot R_{\text{in}} \cdot R_{\text{out}}$ and $CO_{\text{Attention}} = P \cdot X^2 \cdot R$); CM, the memory required for weights, input, and output tensors; and CT, the computational time $t(O,M) = CO(L)/Z_{\text{FLOPs}}(M)$, where $Z_{\text{FLOPs}}(M)$ is the GPU's theoretical peak FLOP rate. A regressor per layer type maps these vectors to median measured runtimes, and the predicted epoch time is the sum over all layers and all batches of the per-layer predictions. These features let the regressor separate how much work a layer demands from how fast a given GPU can supply that work, which is why the framework claims to generalize to GPUs it has not been benchmarked on.

What would settle it

Run a real VGG-16 or BERT training step while profiling each layer's forward and backward time with standard profiler hooks, then compare the sum of the individually measured layer times with the whole-model step time across batch sizes. If the gap grows systematically with batch size or with the number of memory-heavy layers, the per-layer summation that PreNeT's epoch prediction relies on is wrong.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that training-time prediction is materially improved by making the feature set carry the layer's computational load explicitly, rather than leaving the regressor to infer it from shapes alone. PreNeT benchmarks each layer type in isolation on seven GPUs, records median runtimes over 20,000 random configurations, trains per-layer regressors on layer-specific and computational features, and then predicts a full epoch as the sum of the per-layer, per-batch predictions. Across all layer types the computational features beat the baseline, with the best per-layer models reaching RMSE values such as 0.312 ms for attention layers and 7.909 ms for convolutional layers, and with a 72.60% RMSE reduction for attention layers under gradient-boosted trees. The framework also holds out two GPU models during training and reports that the same feature set predicts their per-epoch times with 19.4% to 72.5% lower error than the baseline, and it validates the end-to-end sum on VGG-16 and BERT. The paper takes this as evidence that the computational features, not just the layer shapes or hardware identifiers, carry the signal that determines training duration.

Load-bearing premise

The result depends on assuming that the time for one training epoch is exactly the sum of independently predicted per-layer, per-batch runtimes, and that runtimes measured on isolated layers with random configurations behave the same inside a real model; memory contention, kernel launch overhead, optimizer updates, and data loading could all break that additivity.

Editorial extensions

If this is right

  • Transformer and LLM configurations can be costed before training, because attention and embedding layers, absent from the earlier baseline, are benchmarked and predicted explicitly.
  • A new GPU can be scored with no benchmark runs, since the regressors take only hardware specifications and layer features as input.
  • Layer-type-specific model selection matters: dense, attention, and recurrent layers favor MLP regressors, while convolutional and fully connected layers favor random forests, so a production predictor should keep a per-layer model ensemble.
  • The feature set can be trimmed by layer type: computational time mainly helps compute-bound layers such as CNN and dense layers, while memory features drive accuracy on memory-bound layers such as attention, embedding, and RNN layers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the additivity assumption holds under real training conditions, the same feature design should extend to distributed training by summing per-device segment times, but it would need an explicit communication and synchronization overhead term; the paper does not test that case.
  • A natural testable extension is a memory-bandwidth-normalized time feature alongside the peak-FLOP time; the paper's results suggest it would improve the already memory-sensitive attention and embedding layers further.
  • The epoch-level predictor becomes a total-cost predictor only when chained to a convergence-step estimator; the paper names this as future work, but the epoch feature set here is the component that would make such a chain practical.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. PreNeT is a framework for predicting the per-epoch training time of deep neural networks by training machine-learning models on isolated layer measurements across seven Nvidia GPUs. The model uses hand-crafted computational features—layer-specific parameters, computational operations (CO), computational memory (CM), and computational time (CT)—to predict the runtime of each layer, then sums these predictions over layers and batches (Eq. 13) to estimate an epoch's total time. The paper reports that PreNeT achieves up to 72% lower RMSE than a re-implemented 2018 baseline (Justus et al.) for attention, convolutional, embedding, recurrent, and fully connected layers, and that it generalizes to two unseen Nvidia GPUs. Full-model experiments on VGG-16 and BERT report epoch-level prediction errors, with the best case around 4.5% and worst around 27.65%. The central claim is that the proposed feature set materially improves layer-wise training-time prediction and that summing these predictions yields accurate full-model epoch times.

Significance. If the central claim were rigorously supported, PreNeT would be a practical tool for hardware selection and configuration planning, and its extension to transformer-based layers would address a gap in older performance models. The release of code and data on GitHub is a positive step for reproducibility. However, the current evidence is insufficient in several load-bearing respects: the additivity assumption in Eq. (13) is unvalidated, the reported improvements lack statistical error bars, and the comparison is only against a re-implemented 2018 baseline rather than contemporary frameworks. The idea is plausible and the per-layer results are suggestive, but the full-model claims need substantial additional validation.

major comments (4)
  1. [Section 3.4 / Section 5.3] Eq. (13) assumes that the per-epoch training time equals the sum of independently predicted per-layer, per-batch runtimes. These per-layer rimes were measured in isolation (Section 3.1) on randomly generated configurations, without optimizer updates, data loading, or memory contention. The paper never validates this additivity against in-situ per-layer timings inside VGG-16 or BERT; Section 5.3 reports only aggregate epoch-level errors. The unexplained large gap between VGG-16 errors on V100 (4.50%) and P100 (27.65%) suggests a systematic bias that may arise from the additivity assumption breaking down. The authors should profile at least one full model to compare predicted versus actual layer-wise times, or otherwise bound the effects of kernel launch, data loading, optimizer, and memory contention.
  2. [Section 3.1 vs. Section 4.2] Section 3.1 states that the median runtime was taken over multiple runs, while Section 4.2 says the mean of five repetitions was used to calculate the runtime. These are different statistics, and the paper must specify which one was used for the reported results. In addition, no variance or confidence intervals are given for any RMSE in Tables 4 and 5, so the improvement percentages (e.g., 72.60% in the Attention/GDBT row) cannot be assessed for statistical significance. The authors should provide error bars, confidence intervals, or significance tests for the key comparisons.
  3. [Abstract / Section 5.1] The abstract claims up to 72% improvement over 'contemporary state-of-the-art frameworks,' but the only quantitative comparison is to a re-implemented version of Justus et al. (2018) as described in Section 5.1. Section 6 discusses DNNPerf and Habitat, yet no comparison to these or any other recent framework is made. For Attention, Embedding, and Normalization layers, whose baseline did not exist in the original Justus et al. work, the 'baseline' is the authors' own layer-feature-only model. The claim should either be qualified as 'improvement over a re-implemented 2018 baseline' or be backed by experiments against genuinely contemporary predictors.
  4. [Section 5.2 / Table 1] The unseen-hardware experiments cover only two Nvidia GPUs (L4 and RTX A4000), which does not support the abstract's claim of predicting training time on 'novel accelerator architectures' such as TPU, IPU, or Gaudi. Additionally, GPU Model is listed as a one-hot feature in Table 1; for an unseen GPU, the one-hot category is new, and the paper does not explain how this is handled at test time. The evidence therefore supports at most 'unseen Nvidia GPU models' using the continuous hardware features, and the authors should either adjust the claim or add experiments on genuinely novel accelerator types.
minor comments (5)
  1. [Section 3.4] Equation (13) is garbled in the manuscript (OCR artifacts), making the summation notation illegible; it should be rewritten in standard notation. Also, the variables O and L are used for numbers of layers and batches, while L is used earlier for batch size; notation should be unified.
  2. [Throughout] The text contains numerous OCR artifacts (e.g., '!gure', 'con!guration', 'e"ciency') that obscure meaning. A thorough proofreading and typesetting pass is needed before publication.
  3. [Section 4] The GitHub repository link is described as 'anonymous' but the URL is https://github.com/pacslab/PreNeT, which reveals the lab name. If double-blind review is intended, the link and repository should be anonymized.
  4. [Section 3.2.1] The formulas for computational operations for RNN, LSTM, GRU, and Attention are stated without derivation or citation; a brief justification or a note that these are established approximations (with reference) would help readers assess their validity.
  5. [Section 6] DNNPerf and Habitat are discussed in Related Work but are not included in the experimental comparison. Even a short discussion of why they are not directly comparable would strengthen the evaluation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: PreNeT's computational features and ML predictions are independent of the measured target times.

full rationale

PreNeT's derivation chain is supervised regression from measured per-layer runtimes to runtime, using features (layer dimensions, CO, CM, CT) that are computed from architecture and hardware specifications, not from the target runtime. The CO formulas (Eqs. 2-8), CT (Eq. 9), and CM (Eqs. 10-12) are deterministic functions of layer parameters and GPU peak FLOPs; none requires the measured training time, so adding them is genuine feature engineering rather than fitting the target. The ML models are trained on an 80/10/10 split of the 20,000 measured configurations per GPU and layer type, and the headline improvements are RMSE differences on held-out test data relative to a feature-ablation baseline, not reconstruction of training data. The unseen-GPU experiments (Section 5.2) hold out L4 and A4000 entirely, which is a legitimate generalization test. The full-model aggregation (Eq. 13) is an additive modeling assumption that may be empirically wrong (e.g., the VGG-16/P100/batch-64 error of 27.65%), but an unvalidated additivity assumption is a correctness and robustness concern, not circularity: the per-layer predictions are not derived from the epoch-level target. There are no load-bearing self-citations, no imported uniqueness theorems, and no parameter fitted to the target and then renamed as a prediction. Accordingly, no circular step is present.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on standard benchmark construction and supervised learning, not on new physical or mathematical postulates. No new entities are introduced. The main uncharged assumptions are the transferability of isolated-layer runtimes and the additivity of per-layer times, both of which the full-model evaluation only partially tests. The ML hyperparameters and sampling ranges are the principal free choices.

free parameters (3)
  • MLP architecture and training hyperparameters = 6 hidden layers [32,64,128,128,128,128], dropout 0.2, batch size 128, learning rate 0.001, 300 epochs
    Chosen by hand without a tuning protocol; these choices directly affect the RMSE of the fully connected predictor and the headline improvements.
  • Random configuration sampling ranges = Ranges in Table 3, e.g., batch size 1-64, sequence length 64-512, embedding dim 64-1024
    The benchmark and test data are generated from these ranges, so the predictive performance is conditional on them.
  • Train/validation/test split = 80/10/10
    Chosen without justification; the split determines the RMSE estimates and the unseen-GPU evaluation uses a subset of 7 GPUs.
assumptions (3)
  • domain assumption Runtimes measured on isolated layers transfer to full-model training.
    Section 3.4 and 5.3 sum per-layer predictions to estimate VGG-16 and BERT epoch time, assuming isolated layer behavior is representative.
  • domain assumption Training time per epoch is additive over layers and batches.
    Equation (13) defines epoch time as a double sum of independent per-layer per-batch predictions, ignoring interactions such as memory contention and launch overhead.
  • domain assumption GPU theoretical peak FLOPs is a meaningful hardware feature for runtime prediction.
    Section 3.2.2 uses ZFLOPs as the denominator of the computational time feature; the paper acknowledges that actual achieved FLOPs vary with memory access patterns and kernel optimizations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PreNeT: Leveraging Computational Features to Predict Deep Neural Network Training Time." pith.science (2026). https://pith.science/paper/5UUMO44Z

@misc{pith2026241215519,
  author       = {Pith},
  title        = {Pith review of: PreNeT: Leveraging Computational Features to Predict Deep Neural Network Training Time},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5UUMO44Z}},
  note         = {Machine review of arXiv:2412.15519}
}
read the original abstract

Training deep learning models, particularly Transformer-based architectures such as Large Language Models (LLMs), demands substantial computational resources and extended training periods. While optimal configuration and infrastructure selection can significantly reduce associated costs, this optimization requires preliminary analysis tools. This paper introduces PreNeT, a novel predictive framework designed to address this optimization challenge. PreNeT facilitates training optimization by integrating comprehensive computational metrics, including layer-specific parameters, arithmetic operations and memory utilization. A key feature of PreNeT is its capacity to accurately predict training duration on previously unexamined hardware infrastructures, including novel accelerator architectures. This framework employs a sophisticated approach to capture and analyze the distinct characteristics of various neural network layers, thereby enhancing existing prediction methodologies. Through proactive implementation of PreNeT, researchers and practitioners can determine optimal configurations, parameter settings, and hardware specifications to maximize cost-efficiency and minimize training duration. Experimental results demonstrate that PreNeT achieves up to 72% improvement in prediction accuracy compared to contemporary state-of-the-art frameworks.

Figures

Figures reproduced from arXiv: 2412.15519 by the authors.

Figure 1
Figure 1. PreNeT Framework Overview RQ2: How does integrating computational features such as arithmetic operations, memory usage, and layer-speci!c inputs enhance training time prediction for deep learning models? PreNeT is equipped with a wide range of computational data, including memory usage (the memory required when a model is running), arithmetic operations (catching the computational complexity of each layer), and laye… view at source ↗
Figure 2
Figure 2. PreNeT vs. Baseline MLP on Unseen Hardware [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. PreNeT vs. Baseline XGBoost on Unseen Hardware [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Prediction Errors for PreNeT and Baseline vs. Actual [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Prediction Errors for PreNeT and Baseline vs. Actual [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Prediction Errors for PreNeT and Baseline vs. Actual [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

51 extracted references · 39 canonical work pages

  1. [1]

    Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching Word Vectors with Subword Information. InTransactions of the Associa- tion for Computational Linguistics, Vol. 5. 135–146. https://aclanthology.org/Q17- 1010/

    work page 2017

  2. [2]

    Tom B Brown. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020)

    arXiv 2020

  3. [3]

    Cerebras Systems. 2024. Cerebras Wafer Scale Engine (WSE). https://www. cerebras.net/wafer-scale-engine/. Accessed: 2024-10-19

    work page 2024

  4. [4]

    Kyunghyun Cho, Bart van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio

  5. [5]

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Se- bastian Gehrmann, et al . 2023. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research 24, 240 (2023), 1–113

    work page 2023

  6. [6]

    Neural Computation. 2016. Long short-term memory. Neural Comput 9 (2016), 1735–1780

    work page 2016

  7. [7]

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Im- ageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition . Ieee, 248–255

    work page 2009

  8. [8]

    Jacob Devlin. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

    arXiv 2018

Show all 51 references
  1. [9]

    Xuanyi Dong, Mingxing Tan, Adams Wei Yu, Daiyi Peng, Bogdan Gabrys, and Quoc V Le. 2020. AutoHAS: E"cient hyperparameter and architecture search. arXiv preprint arXiv:2006.03656 (2020)

    2020 arXiv

  2. [10]

    Yanjie Gao, Xianyu Gu, Hongyu Zhang, Haoxiang Lin, and Mao Yang. 2023. Runtime performance prediction for deep learning models with graph neural network. In 2023 IEEE/ACM 45th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) . IEEE...

    2023

  3. [11]

    X Yu Geo#rey, Yubo Gao, Pavel Golikov, and Gennady Pekhimenko. 2021. Habi- tat: A {Runtime-Based} computational performance predictor for deep neural network training. In 2021 USENIX Annual Technical Conference (USENIX ATC 21) . 503–521

    2021

  4. [12]

    Graphcore. 2024. Graphcore Intelligence Processing Units (IPUs). https://www. graphcore.ai/. Accessed: 2024-10-19

    2024

  5. [13]

    Alex Graves. 2012. Sequence Transduction with Recurrent Neural Networks. In International Conference on Machine Learning (ICML) Workshop on Representation Learning. https://arxiv.org/abs/1211.3711

    2012 arXiv

  6. [14]

    Habana Labs. 2024. Habana Labs: AI Processors for Deep Learning. https: //habana.ai/. Accessed: 2024-10-19

    2024

  7. [15]

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition (CVPR) . 770–778. https://doi.org/10.1109/ CVPR.2016.90

    2016

  8. [16]

    Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam

    Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. 2017. MobileNets: E"cient Convolutional Neural Networks for Mobile Vision Applications. In arXiv preprint arXiv:1704.04861. https://arxiv.org/abs/1704.04861

    2017 arXiv

  9. [17]

    Norman P Jouppi, Cli# Young, Nishant Patil, David Patterson, Gaurav Agrawal, Raminder Bajwa, Sarah Bates, Suresh Bhatia, Nan Boden, Al Borchers, et al . 2017. In-datacenter performance analysis of a tensor processing unit. In Proceedings of the 44th annual international sympos...

    2017

  10. [18]

    Daniel Justus, John Brennan, Stephen Bonner, and Andrew Stephen McGough

  11. [19]

    Alex Krizhevsky, Ilya Sutskever, and Geo#rey E. Hinton. 2012. ImageNet Clas- si!cation with Deep Convolutional Neural Networks. In Advances in Neural Information Processing Systems (NeurIPS) . 1097–1105. https://papers.nips.cc/ paper/2012/!le/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf

    2012

  12. [20]

    Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Ha#ner. 1998. Gradient- based learning applied to document recognition. Proc. IEEE 86, 11 (1998), 2278– 2324

    1998

  13. [21]

    Xiaolong Ma, Feng Yan, Lei Yang, Ian Foster, Michael E Papka, Zhengchun Liu, and Rajkumar Kettimuthu. 2024. MalleTrain: Deep Neural Networks Train- ing on Un!llable Supercomputer Nodes. In Proceedings of the 15th ACM/SPEC International Conference on Performance Engineering . 190–200

    2024

  14. [22]

    Tomas Mikolov, Kai Chen, Greg Corrado, and Je#rey Dean. 2013. E"cient Estima- tion of Word Representations in Vector Space. In Proceedings of the International Conference on Learning Representations (ICLR) . https://arxiv.org/abs/1301.3781

    2013 arXiv

  15. [23]

    NVIDIA Corporation. 2016. NVIDIA Tesla P100 GPU Architecture. NVIDIA Product Page. https://www.nvidia.com/en-us/data-center/tesla-p100/

    2016

  16. [24]

    NVIDIA Corporation. 2016. NVIDIA Tesla P4 GPU Architecture. NVIDIA Whitepaper. https://www.nvidia.com/en-us/data-center/tesla-p4/

    2016

  17. [25]

    NVIDIA Corporation. 2017. NVIDIA Tesla V100 GPU Architecture. NVIDIA Product Page. https://www.nvidia.com/en-us/data-center/v100/

    2017

  18. [26]

    NVIDIA Corporation. 2018. NVIDIA T4 Tensor Core GPU. NVIDIA Product Page. https://www.nvidia.com/en-us/data-center/tesla-t4/

    2018

  19. [27]

    NVIDIA Corporation. 2021. NVIDIA RTX A4000 Graphics Card. NVIDIA Product Page. https://www.nvidia.com/en-us/design-visualization/rtx-a4000/

    2021

  20. [28]

    NVIDIA Corporation. 2022. NVIDIA GeForce RTX 4090 Graphics Card. NVIDIA Product Page. https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/ rtx-4090/

    2022

  21. [29]

    NVIDIA Corporation. 2023. NVIDIA L4 Tensor Core GPU. NVIDIA Product Page. https://www.nvidia.com/en-us/data-center/l4/

    2023

  22. [30]

    Long Ouyang, Je#rey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al . 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems 35...

    2022

  23. [31]

    Jay H Park, Gyeongchan Yun, M Yi Chang, Nguyen T Nguyen, Seungmin Lee, Jaesik Choi, Sam H Noh, and Young-ri Choi. 2020. {HetPipe}: Enabling large {DNN} training on (whimpy) heterogeneous {GPU } clusters through integra- tion of pipelined model parallelism and data parallelism....

    2020

  24. [32]

    Ziqian Pei, Chensheng Li, Xiaowei Qin, Xiaohui Chen, and Guo Wei. 2019. Itera- tion time prediction for cnn in multi-gpu platform: modeling and analysis. IEEE Access 7 (2019), 64788–64797

    2019

  25. [33]

    Yanghua Peng, Yixin Bao, Yangrui Chen, Chuan Wu, Chen Meng, and Wei Lin

  26. [34]

    Benjamin JJ P!ster, Dominik Scheinert, Morgan K Geldenhuys, and Odej Kao

  27. [35]

    Hang Qi, Evan R Sparks, and Ameet Talwalkar. 2017. Paleo: A performance model for deep neural networks. InInternational Conference on Learning Representations

    2017

  28. [36]

    Colin Ra#el, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the Limits of Transfer Learning with a Uni!ed Text-to-Text Transformer.Journal of Machine Learning Research 21, 140 (2020), 1–67. http:/...

    2020

  29. [37]

    Sebastian Schelter, Felix Biessmann, Tim Januschowski, David Salinas, Stephan Seufert, and Gyuri Szarvas. 2018. On Challenges in Machine Learning Model Management. Data Engineering (2018), 5

    2018

  30. [38]

    Mike Schuster and Kuldip K. Paliwal. 1997. Bidirectional Recurrent Neural Networks. In IEEE Transactions on Signal Processing , Vol. 45. 2673–2681. https: //doi.org/10.1109/78.650093

    1997 doi

  31. [39]

    Karen Simonyan and Andrew Zisserman. 2015. Very Deep Convolutional Net- works for Large-Scale Image Recognition. In 3rd International Conference on Learning Representations (ICLR). https://arxiv.org/abs/1409.1556

    2015 arXiv

  32. [40]

    Ravi Kumar Singh, Likhith Bandamudi, Shruti Kunde, Mayank Mishra, and Rekha Singhal. 2024. Leftovers for LLaMA. In Proceedings of the 15th ACM/SPEC International Conference on Performance Engineering . 201–210

    2024

  33. [41]

    I Sutskever. 2014. Sequence to Sequence Learning with Neural Networks. arXiv preprint arXiv:1409.3215 (2014)

    2014 arXiv

  34. [42]

    Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going Deeper with Convolutions. In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition (CVPR) . ...

    2015

  35. [43]

    Mingxing Tan and Quoc V. Le. 2019. E"cientNet: Rethinking Model Scaling for Convolutional Neural Networks. In Proceedings of the 36th International Conference on Machine Learning (ICML) . 6105–6114. http://proceedings.mlr. press/v97/tan19a.html PreNeT: Leveraging Computational...

    2019

  36. [44]

    A Vaswani. 2017. Attention is all you need. Advances in Neural Information Processing Systems (2017)

    2017

  37. [45]

    Chuan-Chi Wang, Ying-Chiao Liao, Ming-Chang Kao, Wen-Yew Liang, and Shih- Hao Hung. 2021. Toward accurate platform-aware performance modeling for deep neural networks. ACM SIGAPP Applied Computing Review 21, 1 (2021), 50–61

    2021

  38. [46]

    Gyeongsik Yang, Changyong Shin, Jeunghwan Lee, Yeonho Yoo, and Chuck Yoo. 2022. Prediction of the resource consumption of distributed deep learning systems. Proceedings of the ACM on Measurement and Analysis of Computing Systems 6, 2 (2022), 1–25

    2022

  39. [47]

    Luca Zancato, Alessandro Achille, Avinash Ravichandran, Rahul Bhotika, and Stefano Soatto. 2020. Predicting training time without training. Advances in Neural Information Processing Systems 33 (2020), 6136–6146

    2020

  40. [2014]

    In Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation , Dekai Wu, Marine Carpuat, Xavier Carreras, and Eva Maria Vecchi (Eds.)

    On the Properties of Neural Machine Translation: Encoder–Decoder Approaches. In Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation , Dekai Wu, Marine Carpuat, Xavier Carreras, and Eva Maria Vecchi (Eds.). Association for Comput...

    doi

  41. [2018]

    In 2018 IEEE international conference on big data (Big Data)

    Predicting the computational cost of deep learning models. In 2018 IEEE international conference on big data (Big Data) . IEEE, 3873–3882

    2018

  42. [2021]

    IEEE Transactions on Parallel and Distributed Systems 32, 8 (2021), 1947–1960

    DL2: A deep learning-driven scheduler for deep learning clusters. IEEE Transactions on Parallel and Distributed Systems 32, 8 (2021), 1947–1960

    2021

  43. [2024]

    In Proceedings of the 15th ACM/SPEC International Conference on Performance Engineering

    Daedalus: Self-Adaptive Horizontal Autoscaling for Resource E"ciency of Distributed Stream Processing Systems. In Proceedings of the 15th ACM/SPEC International Conference on Performance Engineering . 130–141

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.