Pith. sign in

REVIEW 2 major objections 6 minor 1 cited by

Frugal Machine Learning for Energy-efficient, and Resource-aware Artificial Intelligence

T0 review · 2 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A survey argues frugal machine learning is a coherent field organized by input, learning-process, and model frugality.

desk verdict Useful but rough survey of Frugal ML: broad technique coverage, but the two taxonomies never line up and the editing undercuts the clarity claim. read the letter →

arxiv 2506.01869 v1 pith:XEZARSLB submitted 2025-06-02 cs.LG cs.AI

John Violos , Konstantina-Christina Diamanti , Ioannis Kompatsiaris , Symeon Papadopoulos This is my paper
classification cs.LGcs.AI
keywords FrugalMachineLearningmodelcompressionknowledgedistillationpruningquantizationfeatureselectiondatasamplingedgecomputing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Frugal Machine Learning (FML) is the practice of designing models that keep acceptable performance while cutting the data, computation, time, energy, memory, and money needed for training and inference. The paper's central claim is that FML forms a coherent research area distinct from TinyML, organized by three complementary strategies: input frugality, learning-process frugality, and model frugality. It supports this claim with a taxonomy of six method families, including model compression, frugal algorithms, feature selection, data sampling, hyperparameter optimization, and hardware optimization, and illustrates them across IoT, wearables, autonomous systems, healthcare, ambient intelligence, and cybersecurity. The chapter also identifies open challenges, chiefly missing standardized benchmarks and the risk that compression harms interpretability and fairness. If its framing is right, these techniques determine whether AI can run affordably on edge devices and with a lower environmental footprint.

What carries the argument

The organizing mechanism is the three-way split among input frugality, learning-process frugality, and model frugality, which gives the survey its structure and its definition of the field. Within that split, the chapter catalogs six method families: model compression through pruning, quantization, knowledge distillation, low-rank factorization, and dynamic neural networks; frugal and optimized algorithms such as adaptive model utilization and zero-time-waste early exits; feature selection; data sampling; hyperparameter optimization; and hardware optimization. The taxonomy is the load-bearing instrument, converting scattered efficiency tricks into a map that lets a practitioner see which stage of the machine-learning pipeline a given technique acts on.

What would settle it

A systematic literature search with a predefined protocol that surfaces a substantial, established frugal technique absent from this taxonomy, or that shows a large share of published frugal methods falling outside the input/learning-process/model split, would show the survey's central organization is incomplete. A simpler test is whether two independent experts using the same protocol would produce materially different taxonomies of the same literature.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that the many disconnected tricks for making machine learning cheaper, such as pruning, quantization, knowledge distillation, low-rank factorization, dynamic networks, feature and data selection, hyperparameter search, and hardware accelerators, are unified by a single cost-driven mindset and fall into one of three frugalities: reducing what goes into the model, reducing what it costs to train, or reducing what it costs to store and run. The paper surveys each family, gives representative methods and recent advances, and maps them onto application domains with tight resource budgets. It closes by arguing that wider adoption depends on standardized benchmarks that measure energy, memory, latency, and carbon footprint together, and on better guidance for sequencing frugal techniques through the model lifecycle.

Load-bearing premise

The survey's claim to be comprehensive rests on the assumption that its selection of referenced works fairly represents the whole frugal-ML field, but the chapter does not state a systematic search or inclusion protocol that would guarantee this.

Editorial extensions

If this is right

  • A practitioner can pick frugality tools by pipeline stage: shrink the input first, then cheapen training, then compress the model, instead of applying a single one-size-fits-all trick.
  • Combining techniques compounds the savings, since pruning with quantization, quantization-aware factorization, and quantized distillation all appear in the survey as compatible pairings.
  • Dynamic architectures and incremental updates let models absorb new tasks without full retraining, making continual on-device learning feasible.
  • Hardware-aware design, from neural processing units to once-for-all networks, means frugality is co-designed with the deployment device rather than added afterward.
  • Wider adoption waits on standardized benchmarks that measure memory, latency, energy, and carbon footprint together, which the chapter identifies as a key blocker.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is a benchmark suite that measures input cost, training cost, and inference cost as separate axes, which would show whether the three-way taxonomy corresponds to genuinely independent engineering trade-offs.
  • The importance of sequencing implies that an end-to-end toolchain which searches over technique order, such as whether to prune before or after quantizing, could itself become a frugal-ML method, since the chapter notes that order changes final performance.
  • Because FML is broader than TinyML, many FML ideas such as entropy-based data selection and early exits could be imported into ultra-low-power microcontroller settings, where current practice concentrates mainly on model compression.
  • The sustainability claim implies lifecycle accounting: verifying that reduced model size actually lowers carbon footprint requires standardized energy and carbon metering across both training and deployment, which the chapter flags as missing.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The manuscript is a book-chapter-style survey of Frugal Machine Learning (FML), defined as the design of ML models under constraints on computational resources, energy, data, and cost. Section 2 proposes a taxonomy with six method families: model compression, frugal/optimized algorithms, feature selection, data sampling, hyperparameter optimization, and hardware optimization. Section 3 gives short application vignettes in IoT/edge AI, wearables, robotics, healthcare, ambient intelligence, bandwidth-constrained systems, consumer electronics, and cybersecurity. Section 4 introduces a higher-level categorization of frugality into input, learning-process, and model frugality, and Section 5 lists open challenges such as noisy environments, interpretability/fairness trade-offs, lack of standardized benchmarks, and missing deployment guidance. The stated main contribution is a comprehensive and clearly structured survey of FML.

Significance. If made internally coherent, the survey would be a useful entry point to the frugal-ML literature: it covers a wide range of techniques, includes a practical objectives table (Table 1), and draws attention to underserved issues such as lifecycle cost assessment and the absence of standardized resource-efficiency benchmarks. The paper makes no new empirical or theoretical contributions and contains no machine-checked artifacts; its value rests entirely on the quality of its organization and coverage. That value is currently limited by the unreconciled double taxonomy and the absent taxonomy figure.

major comments (2)
  1. [Section 2 and Section 4] The paper presents two different taxonomies that are never reconciled. Section 2 organizes FML methods into six families (model compression, frugal/optimized algorithms, feature selection, data sampling, hyperparameter optimization, hardware optimization), while the abstract and Section 4 state that FML is 'broadly categorized into input frugality, learning process frugality, and model frugality.' No mapping is provided between these two organizational schemes, so the reader cannot determine, for example, whether feature selection and data sampling fall under input frugality, whether hyperparameter optimization falls under learning-process frugality, or whether hardware optimization falls under model frugality. Since the paper's main contribution is a 'comprehensive survey... structured to ensure clarity and accessibility' (Section 1), this unreconciled taxonomy directly weakens the central claim. Additionally, Figure 1, which presumably would illustrate the taxonomy, is referenced but absent from the manuscript text, making the intended structure impossible to verify.
  2. [Section 1] The introduction states that the main contribution is 'a comprehensive survey of FML' but does not describe the literature selection process. There is no mention of a systematic search protocol, inclusion/exclusion criteria, databases queried, or screening method. Without such information, the comprehensiveness claim cannot be audited, and the reader cannot distinguish an exhaustive survey from a selective overview. This is a load-bearing gap for the survey's central claim and should be addressed by adding a methodology paragraph or by softening the claim to 'an overview' with a clearly stated scope.
minor comments (6)
  1. [Section 2.1] The citation placeholder '[18,?]' in the first paragraph is unresolved; it should be completed with the actual reference or removed.
  2. [Section 2.3] The text reads 'In SMVs, feature selection has been formulated...' and should read 'In SVMs' (Support Vector Machines).
  3. [Section 2.1] The sentence 'The two main categories of pruning is structured and unstructured.' has a subject-verb agreement error and should be 'The two main categories of pruning are structured and unstructured.'
  4. [Section 3.8] The final sentence of Section 3.8, 'ensuring strong security while data requirements and computational load,' is incomplete; it should end with a verb phrase such as 'while reducing data requirements and computational load.'
  5. [Abstract] The abstract contains a typo: 'Approach in this field aim to achieve satisfactory performance' should be 'Approaches in this field aim to achieve satisfactory performance.'
  6. [References] Several reference entries are incomplete, lacking publication venues or full titles; examples include [47] ('Slimmable Neural Networks') and [85] ('Once-for-all'), which are missing conference/book information.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the paper is a descriptive survey whose organizational claims are not derived from, or reduced to, its own inputs.

full rationale

This paper is a survey chapter, not a derivation. It contains no fitted parameters, no equations, and no quantity that is predicted from another quantity. The central claim is that the chapter provides 'a comprehensive survey of FML, structured to ensure clarity and accessibility' (Section 1), supported by a taxonomy in Section 2 and a high-level three-way categorization in Section 4. That categorization is asserted descriptively, and while the relationship between the six method families of Section 2 and the three categories of Section 4 is left unstated, that is a structural or completeness weakness, not a circular reduction. The paper cites several works by its own authors (e.g., refs 1, 2, 5, 14, 15, 18, 30, 60, 73), but these are independently published papers used to support specific descriptive statements about particular techniques; none is invoked as an authority that forces the survey's taxonomy or that forbids alternative organizations. The absence of a documented literature-search protocol is a legitimate concern about comprehensiveness and auditability, but it is not a case of a prediction being equivalent to its input by construction. No self-definitional step, fitted-input-as-prediction step, or load-bearing self-citation chain is present. The survey is therefore not circular; editorial issues such as the unresolved placeholder '[18,?]' and the typo 'In SMVs' are quality concerns, not circularity.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

This is a review paper; it introduces no free parameters or invented entities. Its central claim of comprehensiveness rests on the representative selection of cited works and on the validity of those works.

assumptions (2)
  • domain assumption The cited literature is representative of the field of frugal machine learning.
    The survey's comprehensiveness claim rests on the authors' selection of references; no systematic search protocol is provided (Section 1).
  • domain assumption FML methods can achieve acceptable performance while reducing resource consumption.
    This premise underlies the whole survey and is supported only by citations, not by new experiments (throughout).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Frugal Machine Learning for Energy-efficient, and Resource-aware Artificial Intelligence." pith.science (2026). https://pith.science/paper/XEZARSLB

@misc{pith2026250601869,
  author       = {Pith},
  title        = {Pith review of: Frugal Machine Learning for Energy-efficient, and Resource-aware Artificial Intelligence},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XEZARSLB}},
  note         = {Machine review of arXiv:2506.01869}
}
read the original abstract

Frugal Machine Learning (FML) refers to the practice of designing Machine Learning (ML) models that are efficient, cost-effective, and mindful of resource constraints. This field aims to achieve acceptable performance while minimizing the use of computational resources, time, energy, and data for both training and inference. FML strategies can be broadly categorized into input frugality, learning process frugality, and model frugality, each focusing on reducing resource consumption at different stages of the ML pipeline. This chapter explores recent advancements, applications, and open challenges in FML, emphasizing its importance for smart environments that incorporate edge computing and IoT devices, which often face strict limitations in bandwidth, energy, or latency. Technological enablers such as model compression, energy-efficient hardware, and data-efficient learning techniques are discussed, along with adaptive methods including parameter regularization, knowledge distillation, and dynamic architecture design that enable incremental model updates without full retraining. Furthermore, it provides a comprehensive taxonomy of frugal methods, discusses case studies across diverse domains, and identifies future research directions to drive innovation in this evolving field.

Figures

Figures reproduced from arXiv: 2506.01869 by the authors.

Figure 1
Figure 1. Illustration of the FML Taxonomy [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Position: Stop Preaching and Start Practising Data Frugality for Responsible Development of AI

    cs.LG 2026-02 conditional novelty 5.0 of 10

    Data frugality is practical: pruning 25% of ImageNet-like datasets can cut training energy by roughly 29–33% with negligible accuracy loss, while dataset-level carbon costs are substantial.

Reference graph

Works this paper leans on

88 extracted references · 45 canonical work pages · cited by 1 Pith paper

  1. [14]

    Kinnas, J

    M. Kinnas, J. Violos, N. I. Karapiperis, I. Kompatsiaris, Selecting Images With Entropy for Frugal Knowledge Distillation, IEEE Access 13 (2025) 28189–28203, conference Name: IEEE Access. doi:10.1109/ACCESS.2025.3540384. URL https://ieeexplore.ieee.org/abstract/document/10878975

    work page arXiv 2025

  2. [1]

    Bimpas, J

    A. Bimpas, J. Violos, A. Leivadeas, I. Varlamis, Leveraging pervasive com- puting for ambient intelligence: A survey on recent advancements, ap- plications and open challenges, Computer Networks 239 (2024) 110156. doi:10.1016/j.comnet.2023.110156. URL https://www.sciencedirect.com/science/article/pii/ S1389128623006011

    arXiv 2024

  3. [2]

    Kinnas, J

    M. Kinnas, J. Violos, I. Kompatsiaris, S. Papadopoulos, Reducing inference energy consumption using dual complementary CNNs, Future Generation Computer Systems 165 (2025) 107606. doi:10.1016/j.future.2024.107606. URL https://www.sciencedirect.com/science/article/pii/ S0167739X24005703

    arXiv 2025

  4. [3]

    Llisterri Gim´ enez, M

    N. Llisterri Gim´ enez, M. Monfort Grau, R. Pueyo Centelles, F. Freitag, On-device training of machine learning models on microcontrollers with federated learning 11 (4) 573. doi:10.3390/electronics11040573. URL https://www.mdpi.com/2079-9292/11/4/573

    work page doi:10.3390/electronics11040573

  5. [4]

    D. Li, X. Chen, M. Becchi, Z. Zong, Evaluating the energy efficiency of deep convolutional neural networks on CPUs and GPUs, in: 2016 IEEE International Conferences on Big Data and Cloud Computing (BDCloud), Social Comput- ing and Networking (SocialCom), Sustainable Computing and Communications (SustainCom) (BDCloud-SocialCom-SustainCom), pp. 477–484. doi...

    work page arXiv 2016

  6. [5]

    Violos, S

    J. Violos, S. Papadopoulos, I. Kompatsiaris, Towards Optimal Trade-Offs in Knowledge Distillation for CNNs and Vision Transformers at the Edge, in: 2024 32nd European Signal Processing Conference (EUSIPCO), 2024, pp. 1896–1900, iSSN: 2076-1465. doi:10.23919/EUSIPCO63174.2024.10715301. URL https://ieeexplore.ieee.org/abstract/document/10715301

    arXiv 2024

  7. [6]

    T. Tan, G. Cao, Deep Learning on Mobile Devices With Neural Processing Units, Computer 56 (8) (2023) 48–57, conference Name: Computer. doi:10.1109/MC. 2022.3215780. URL https://ieeexplore.ieee.org/abstract/document/10207072 18 Artificial Intelligence, Data and Robotics: Frugal Machine Learning

    arXiv 2023

  8. [7]

    Capra, B

    M. Capra, B. Bussolino, A. Marchisio, G. Masera, M. Martina, M. Shafique, Hard- ware and software optimizations for accelerating deep neural networks: Survey of current trends, challenges, and the road ahead 8 225134–225180, conference Name: IEEE Access. doi:10.1109/ACCESS.2020.3039858. URL https://ieeexplore.ieee.org/document/9269334/\?arnumber=9269334

    arXiv

Show all 88 references
  1. [8]

    Y. Yin, J. Jang-Jaccard, W. Xu, A. Singh, J. Zhu, F. Sabrina, J. Kwak, IGRF- RFE: a hybrid feature selection method for MLP-based network intrusion detection on UNSW-NB15 dataset, Journal of Big Data 10 (1) (2023) 15. doi:10.1186/ s40537-023-00694-8 . URL https://doi.org/10.11...

    2023 doi

  2. [9]

    Cheng, M

    H. Cheng, M. Zhang, J. Q. Shi, A Survey on Deep Neural Network Pruning: Taxonomy, Comparison, Analysis, and Recommendations, IEEE Transactions on Pattern Analysis and Machine Intelligence 46 (12) (2024) 10558–10578, confer- ence Name: IEEE Transactions on Pattern Analysis and ...

    2024

  3. [10]

    Cherniuk, S

    D. Cherniuk, S. Abukhovich, A.-H. Phan, I. Oseledets, A. Cichocki, J. Gusak, Quantization Aware Factorization for Deep Neural Network Compression, Journal of Artificial Intelligence Research 81 (2024) 973–988. doi:10.1613/jair.1.16167. URL https://www.jair.org/index.php/jair/a...

    2024 doi

  4. [11]

    J. Gou, B. Yu, S. J. Maybank, D. Tao, Knowledge distillation: A survey 129 (6) 1789–1819. arXiv:2006.05525[cs], doi:10.1007/s11263-021-01453-z . URL http://arxiv.org/abs/2006.05525

    2006 arXiv

  5. [12]

    Pandey, P

    P. Pandey, P. Basu, K. Chakraborty, S. Roy, GreenTPU: Improving Timing Error Resilience of a Near-Threshold Tensor Processing Unit, in: Proceedings of the 56th Annual Design Automation Conference 2019, DAC ’19, Association for Computing Machinery, New York, NY, USA, 2019, pp. ...

    2019

  6. [13]

    Cancela, V

    B. Cancela, V. Bol´ on-Canedo, A. Alonso-Betanzos, E2E-FS: An End-to-End Fea- ture Selection Method for Neural Networks, IEEE Transactions on Pattern Anal- ysis and Machine Intelligence 45 (7) (2023) 8311–8323, conference Name: IEEE Transactions on Pattern Analysis and Machine...

    2023

  7. [15]

    Karathanasis, J

    A. Karathanasis, J. Violos, I. Kompatsiaris, S. Papadopoulos, A Brief Re- view for Compression and Transfer Learning Techniques in DeepFake Detection, arXiv:2504.21066 [cs] (Apr. 2025). doi:10.48550/arXiv.2504.21066. URL http://arxiv.org/abs/2504.21066

    2025 doi arXiv

  8. [16]

    Salmani Pour Avval, N

    S. Salmani Pour Avval, N. D. Eskue, R. M. Groves, V. Yaghoubi, Systematic review on neural architecture search, Artificial Intelligence Review 58 (3) (2025)

    2025

  9. [17]

    Curry, F

    E. Curry, F. Heintz, M. Irgens, A. W. M. Smeulders, S. Stramigioli, Partnership on AI, data, and robotics, Commun. ACM 65 (4) (2022) 54–55.doi:10.1145/3513000. URL https://dl.acm.org/doi/10.1145/3513000 Artificial Intelligence, Data and Robotics: Frugal Machine Learning 19

    2022 doi

  10. [18]

    Karathanasis, J

    A. Karathanasis, J. Violos, I. Kompatsiaris, A Comparative Analysis of Compres- sion and Transfer Learning Techniques in DeepFake Detection Models, Mathemat- ics 13 (5) (2025) 887, number: 5 Publisher: Multidisciplinary Digital Publishing Institute. doi:10.3390/math13050887. U...

    2025 doi

  11. [19]

    Liebenwein, C

    L. Liebenwein, C. Baykal, B. Carter, D. Gifford, D. Rus, Lost in Pruning: The Effects of Pruning Neural Networks beyond Test Accuracy, arXiv:2103.03014 [cs] (Mar. 2021). doi:10.48550/arXiv.2103.03014. URL http://arxiv.org/abs/2103.03014

    2021 doi arXiv

  12. [20]

    Y. He, L. Xiao, Structured Pruning for Deep Convolutional Neural Networks: A Survey, IEEE Transactions on Pattern Analysis and Machine Intelligence 46 (5) (2024) 2900–2919, conference Name: IEEE Transactions on Pattern Analysis and Machine Intelligence. doi:10.1109/TPAMI.2023....

    2024

  13. [21]

    J. Zou, T. Rui, Y. Zhou, C. Yang, S. Zhang, Convolutional neural network simpli- fication via feature map pruning 70 950–958. doi:10.1016/j.compeleceng.2018. 01.036. URL https://linkinghub.elsevier.com/retrieve/pii/\S0045790617326393

    2018 doi

  14. [22]

    H. Li, A. Kadav, I. Durdanovic, H. Samet, H. P. Graf, Pruning Filters for Efficient ConvNets, arXiv:1608.08710 [cs] (Mar. 2017). doi:10.48550/arXiv.1608.08710. URL http://arxiv.org/abs/1608.08710

    2017 doi arXiv

  15. [23]

    C. Chen, K. Li, X. Zou, Y. Li, DyGNN: Algorithm and Architecture Support of Dynamic Pruning for Graph Neural Networks, in: 2021 58th ACM/IEEE Design Automation Conference (DAC), 2021, pp. 1201–1206, iSSN: 0738-100X. doi:10. 1109/DAC18074.2021.9586298. URL https://ieeexplore.ie...

    2021

  16. [24]

    G. Li, P. Yang, C. Qian, R. Hong, K. Tang, Stage-Wise Magnitude-Based Prun- ing for Recurrent Neural Networks, IEEE Transactions on Neural Networks and Learning Systems 35 (2) (2024) 1666–1680, conference Name: IEEE Transactions on Neural Networks and Learning Systems. doi:10....

    2024

  17. [25]

    Molchanov, S

    P. Molchanov, S. Tyree, T. Karras, T. Aila, J. Kautz, Pruning Convolutional Neu- ral Networks for Resource Efficient Inference, arXiv:1611.06440 [cs] (Jun. 2017). doi:10.48550/arXiv.1611.06440. URL http://arxiv.org/abs/1611.06440

    2017 doi arXiv

  18. [26]

    K. Song, W. Yao, X. Zhu, Filter Pruning via Similarity Clustering for Deep Convolutional Neural Networks, in: Neural Information Processing: 29th Inter- national Conference, ICONIP 2022, Virtual Event, November 22–26, 2022, Pro- ceedings, Part I, Springer-Verlag, Berlin, Heide...

    2022 doi

  19. [27]

    Y. Guo, A. Yao, Y. Chen, Dynamic Network Surgery for Efficient DNNs, in: Advances in Neural Information Processing Systems, Vol. 29, Curran Associates, Inc., 2016. URL https://proceedings.neurips.cc/paper/2016/hash/ 2823f4797102ce1a1aec05359cc16dd9-Abstract.html

    2016

  20. [28]

    Frankle, M

    J. Frankle, M. Carbin, The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks, arXiv:1803.03635 [cs] (Mar. 2019). doi:10.48550/arXiv.1803. 03635. URL http://arxiv.org/abs/1803.03635 20 Artificial Intelligence, Data and Robotics: Frugal Machine Learning

    2019 doi arXiv

  21. [29]

    C. Pham, T. Hoang, T.-T. Do, Collaborative Multi-Teacher Knowledge Distillation for Learning Low Bit-Width Deep Neural Networks, 2023, pp. 6435–6443. URL https://openaccess.thecvf.com/content/WACV2023/html/Pham_ Collaborative_Multi-Teacher_Knowledge_Distillation_for_Learning_L...

    2023

  22. [30]

    Tsanakas, A

    S. Tsanakas, A. Hameed, J. Violos, A. Leivadeas, A light-weight edge-enabled knowledge distillation technique for next location prediction of multitude transportation means, Future Generation Computer Systems 154 (2024) 45–58. doi:10.1016/j.future.2023.12.025. URL https://www....

    2024 doi

  23. [31]

    C. Yang, X. Yu, Z. An, Y. Xu, Categories of Response-Based, Feature-Based, and Relation-Based Knowledge Distillation, in: W. Pedrycz, S.-M. Chen (Eds.), Advancements in Knowledge Distillation: Towards New Horizons of Intelligent Sys- tems, Springer International Publishing, Ch...

    2023 doi

  24. [32]

    T. Wu, C. Tao, J. Wang, R. Yang, Z. Zhao, N. Wong, Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models, in: O. Rambow, L. Wanner, M. Apidianaki, H. Al-Khalifa, B. D. Eugenio, S. Schockaert (Eds.), Proceedings of the 31st International Con...

    2025

  25. [33]

    G. Cai, J. Li, X. Liu, Z. Chen, H. Zhang, Learning and Compressing: Low-Rank Ma- trix Factorization for Deep Neural Network Compression, Applied Sciences 13 (4) (2023) 2704, number: 4 Publisher: Multidisciplinary Digital Publishing Institute. doi:10.3390/app13042704. URL https...

    2023 doi

  26. [34]

    H. Yang, M. Tang, W. Wen, F. Yan, D. Hu, A. Li, H. Li, Y. Chen, Learning Low- Rank Deep Neural Networks via Singular Vector Orthogonality Regularization and Singular Value Sparsification, 2020, pp. 678–679. URL https://openaccess.thecvf.com/content_CVPRW_2020/html/w40/ Yang_Le...

    2020

  27. [35]

    Park, S.-M

    S. Park, S.-M. Moon, CURing Large Models: Compression via CUR Decomposi- tion, arXiv:2501.04211 [cs] (Jan. 2025). doi:10.48550/arXiv.2501.04211. URL http://arxiv.org/abs/2501.04211

    2025 doi arXiv

  28. [36]

    Nalepa, M

    J. Nalepa, M. Antoniak, M. Myller, P. Ribalta Lorenzo, M. Marcinkiewicz, To- wards resource-frugal deep convolutional neural networks for hyperspectral image segmentation 73 102994. doi:10.1016/j.micpro.2020.102994. URL https://linkinghub.elsevier.com/retrieve/\pii/S0141933119302844

    2020

  29. [37]

    Menghani, Efficient deep learning: A survey on making deep learning models smaller, faster, and better 55 (12) 1–37

    G. Menghani, Efficient deep learning: A survey on making deep learning models smaller, faster, and better 55 (12) 1–37. doi:10.1145/3578938. URL https://dl.acm.org/doi/10.1145/3578938

    doi

  30. [38]

    Tabani, A

    H. Tabani, A. Balasubramaniam, S. Marzban, E. Arani, B. Zonooz, Improving the efficiency of transformers for resource-constrained devices. arXiv:2106.16006[cs]. URL http://arxiv.org/abs/2106.16006

    arXiv

  31. [39]

    Polino, R

    A. Polino, R. Pascanu, D. Alistarh, Model compression via distillation and quan- tization, arXiv:1802.05668 [cs] (Feb. 2018). doi:10.48550/arXiv.1802.05668. URL http://arxiv.org/abs/1802.05668 Artificial Intelligence, Data and Robotics: Frugal Machine Learning 21

    2018 doi arXiv

  32. [40]

    H. Liu, Y. Zhou, B. Liu, J. Zhao, R. Yao, Z. Shao, Incremental learning with neural networks for computer vision: a survey 56 (5) 4557–4589. doi:10.1007/ s10462-022-10294-2 . URL https://doi.org/10.1007/s10462-022-10294-2

    doi

  33. [41]

    X. Wang, F. Yu, Z.-Y. Dou, T. Darrell, J. E. Gonzalez, SkipNet: Learning Dynamic Routing in Convolutional Networks, 2018, pp. 409–424. URL https://openaccess.thecvf.com/content_ECCV_2018/html/Xin_Wang_ SkipNet_Learning_Dynamic_ECCV_2018_paper.html

    2018

  34. [42]

    L. Liu, J. Deng, Dynamic Deep Neural Networks: Optimizing Accuracy-Efficiency Trade-Offs by Selective Execution, Proceedings of the AAAI Conference on Artifi- cial Intelligence 32 (1), number: 1 (Apr. 2018). doi:10.1609/aaai.v32i1.11630. URL https://ojs.aaai.org/index.php/AAAI...

    2018 doi

  35. [43]

    Verelst, T

    T. Verelst, T. Tuytelaars, Dynamic Convolutions: Exploiting Spatial Sparsity for Faster Inference, 2020, pp. 2320–2329. URL https://openaccess.thecvf.com/content_CVPR_2020/html/Verelst_ Dynamic_Convolutions_Exploiting_Spatial_Sparsity_for_Faster_ Inference_CVPR_2020_paper.html

    2020

  36. [44]

    A. J. Varghese, A. Bora, M. Xu, G. E. Karniadakis, TransformerG2G: Adaptive time-stepping for learning temporal graph embeddings using transformers, Neural Networks 172 (2024) 106086. doi:10.1016/j.neunet.2023.12.040. URL https://www.sciencedirect.com/science/article/pii/ S089...

    2024 doi

  37. [45]

    Y. Zhou, T. Lei, H. Liu, N. Du, Y. Huang, V. Zhao, A. M. Dai, Z. Chen, Q. V. Le, J. Laudon, Mixture-of-Experts with Expert Choice Routing, Advances in Neural Information Processing Systems 35 (2022) 7103–7114. URL https://proceedings.neurips.cc/paper_files/paper/2022/hash/ 2f0...

    2022

  38. [46]

    Elkerdawy, M

    S. Elkerdawy, M. Elhoushi, H. Zhang, N. Ray, Fire Together Wire Together: A Dynamic Pruning Approach With Self-Supervised Mask Prediction, in: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 12454–12463. URL https://openacc...

    2022

  39. [47]

    J. Yu, L. Yang, N. Xu, J. Yang, T. Huang, Slimmable Neural Networks, arXiv:1812.08928 [cs] (Dec. 2018). doi:10.48550/arXiv.1812.08928. URL http://arxiv.org/abs/1812.08928

    2018 doi arXiv

  40. [48]

    C. Li, G. Wang, B. Wang, X. Liang, Z. Li, X. Chang, Dynamic Slimmable Network, 2021, pp. 8607–8617. URL https://openaccess.thecvf.com/content/CVPR2021/html/Li_Dynamic_ Slimmable_Network_CVPR_2021_paper.html

    2021

  41. [49]

    J. Qiu, C. Chen, S. Liu, H.-Y. Zhang, B. Zeng, SlimConv: Reducing Channel Redundancy in Convolutional Neural Networks by Features Recombining, IEEE Transactions on Image Processing 30 (2021) 6434–6445, conference Name: IEEE Transactions on Image Processing. doi:10.1109/TIP.202...

    2021

  42. [50]

    G. Xu, X. Wang, X. Wu, X. Leng, Y. Xu, Development of Skip Connection in Deep Neural Networks for Computer Vision and Medical Image Analysis: A Survey, arXiv:2405.01725 [eess] (May 2024). doi:10.48550/arXiv.2405.01725. URL http://arxiv.org/abs/2405.01725 22 Artificial Intellig...

    2024 doi arXiv

  43. [51]

    K. H. Lee, N. Verma, A low-power processor with configurable embedded machine- learning accelerators for high-order and adaptive analysis of medical-sensor signals 48 (7) 1625–1637, conference Name: IEEE Journal of Solid-State Circuits. doi: 10.1109/JSSC.2013.2253226. URL http...

  44. [52]

    Apicharttrisorn, X

    K. Apicharttrisorn, X. Ran, J. Chen, S. V. Krishnamurthy, A. K. Roy-Chowdhury, Frugal following: power thrifty object detection and tracking for mobile augmented reality, in: Proceedings of the 17th Conference on Embedded Networked Sensor Systems, ACM, pp. 96–109. doi:10.1145/...

  45. [53]

    E. Park, D. Kim, S. Kim, Y.-D. Kim, G. Kim, S. Yoon, S. Yoo, Big/little deep neural network for ultra low power inference, in: 2015 International Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS), pp. 124–132. doi:10.1109/CODESISSS.2015.7331375. URL ht...

    2015

  46. [54]

    Wojcik, M

    B. Wojcik, M. Przewiezlikowski, F. Szatkowski, M. Wolczyk, K. Balazy, B. Krzep- kowski, I. Podolak, J. Tabor, M. Smieja, T. Trzcinski, Zero time waste in pre-trained early exit neural networks, Neural Networks 168 (2023) 580–601. doi:10.1016/j.neunet.2023.10.003. URL https://w...

    2023 doi

  47. [55]

    Rahmath P, https://orcid.org/0000-0003-1128-5964, View Profile, V

    H. Rahmath P, https://orcid.org/0000-0003-1128-5964, View Profile, V. Sri- vastava, https://orcid.org/0000-0002-2064-5805, View Profile, K. Chaura- sia, https://orcid.org/0000-0003-3471-782X, View Profile, R. G. Pacheco, https://orcid.org/0000-0002-6763-7255, View Profile, R. ...

    2024

  48. [56]

    Cholet, E

    S. Cholet, E. Biabiany, A framework for frugal supervised learning with incremen- tal neural networks 13 (9) 5489, number: 9 Publisher: Multidisciplinary Digital Publishing Institute. doi:10.3390/app13095489. URL https://www.mdpi.com/2076-3417/13/9/5489

    doi

  49. [57]

    Kawewong, S

    A. Kawewong, S. Tangruamsub, P. Kankuekul, O. Hasegawa, Fast online incremen- tal transfer learning for unseen object classification using self-organizing incremen- tal neural networks, in: The 2011 International Joint Conference on Neural Net- works, 2011, pp. 749–756, iSSN: ...

    2011

  50. [58]

    Grossberg, Nonlinear neural networks: Principles, mechanisms, and architec- tures 1 (1) 17–61

    S. Grossberg, Nonlinear neural networks: Principles, mechanisms, and architec- tures 1 (1) 17–61. doi:10.1016/0893-6080(88)90021-4 . URL https://www.sciencedirect.com/science/article/pii/ \0893608088900214

    doi

  51. [59]

    Adadi, A survey on data-efficient algorithms in big data era, Journal of Big Data 8 (1) (2021) 24

    A. Adadi, A survey on data-efficient algorithms in big data era, Journal of Big Data 8 (1) (2021) 24. doi:10.1186/s40537-021-00419-9 . URL https://doi.org/10.1186/\s40537-021-00419-9

    2021 doi

  52. [60]

    Sarantos, J

    P. Sarantos, J. Violos, A. Leivadeas, Enabling semi-supervised learning in intrusion detection systems, Journal of Parallel and Distributed Computing 196 (2025) 105010. doi:10.1016/j.jpdc.2024.105010. URL https://www.sciencedirect.com/science/article/pii/ S0743731524001746 Art...

    2025

  53. [61]

    H. Tu, T. Menzies, FRUGAL: Unlocking Semi-Supervised Learning for Soft- ware Analytics, in: 2021 36th IEEE/ACM International Conference on Auto- mated Software Engineering (ASE), 2021, pp. 394–406, iSSN: 2643-1572. doi: 10.1109/ASE51524.2021.9678617. URL https://ieeexplore.iee...

    2021

  54. [62]

    Senthamarai Kannan, N

    S. Senthamarai Kannan, N. Ramaraj, A novel hybrid feature selection via Symmet- rical Uncertainty ranking based local memetic search algorithm, Knowledge-Based Systems 23 (6) (2010) 580–585. doi:10.1016/j.knosys.2010.03.016. URL https://www.sciencedirect.com/science/article/pi...

    2010 doi

  55. [63]

    R. J. Urbanowicz, M. Meeker, W. La Cava, R. S. Olson, J. H. Moore, Relief-based feature selection: Introduction and review, Journal of Biomedical Informatics 85 (2018) 189–203. doi:10.1016/j.jbi.2018.07.014. URL https://www.sciencedirect.com/science/article/pii/ S1532046418301400

    2018 doi

  56. [64]

    P. Saha, S. Patikar, S. Neogy, A Correlation - Sequential Forward Selection Based Feature Selection Method for Healthcare Data Analysis, in: 2020 IEEE Interna- tional Conference on Computing, Power and Communication Technologies (GU- CON), 2020, pp. 69–72. doi:10.1109/GUCON488...

    2020

  57. [65]

    H. B. Nguyen, B. Xue, I. Liu, M. Zhang, Filter based backward elimination in wrapper based PSO for feature selection in classification, in: 2014 IEEE Congress on Evolutionary Computation (CEC), 2014, pp. 3111–3118, iSSN: 1941-0026. doi: 10.1109/CEC.2014.6900657. URL https://ie...

    2014

  58. [66]

    Jim´ enez-Cordero, J

    A. Jim´ enez-Cordero, J. M. Morales, S. Pineda, A novel embedded min-max approach for feature selection in nonlinear Support Vector Machine classi- fication, European Journal of Operational Research 293 (1) (2021) 24–35. doi:10.1016/j.ejor.2020.12.009. URL https://www.scienced...

    2021 doi

  59. [67]

    Seijo-Pardo, I

    B. Seijo-Pardo, I. Porto-D ´ ıaz, V. Bol´ on-Canedo, A. Alonso-Betanzos, Ensemble feature selection: Homogeneous and heterogeneous approaches, Knowledge-Based Systems 118 (2017) 124–139. doi:10.1016/j.knosys.2016.11.017. URL https://www.sciencedirect.com/science/article/pii/ S...

    2017 doi

  60. [68]

    Mohammed, J

    R. Mohammed, J. Rawashdeh, M. Abdullah, Machine learning with oversam- pling and undersampling techniques: Overview study and experimental results, in: 2020 11th International Conference on Information and Communication Systems (ICICS), pp. 243–248, ISSN: 2573-3346. doi:10.110...

    2020

  61. [69]

    Werner de Vargas, J

    V. Werner de Vargas, J. A. Schneider Aranda, R. Dos Santos Costa, P. R. Da Silva Pereira, J. L. Vict´ oria Barbosa, Imbalanced data preprocessing tech- niques for machine learning: a systematic mapping study 65 (1) 31–57. doi: 10.1007/s10115-022-01772-8 . URL https://link.spri...

    doi

  62. [70]

    S. Chen, S. Dorn, M. Lell, M. Kachelrieß, A. Maier, Manifold Learning-based Data Sampling for Model Training, in: A. Maier, T. M. Deserno, H. Han- dels, K. H. Maier-Hein, C. Palm, T. Tolxdorff (Eds.), Bildverarbeitung f¨ ur die 24 Artificial Intelligence, Data and Robotics: Fr...

    2018

  63. [71]

    Wang, J.-Y

    T. Wang, J.-Y. Zhu, A. Torralba, A. A. Efros, Dataset Distillation, arXiv:1811.10959 [cs] (Feb. 2020). doi:10.48550/arXiv.1811.10959. URL http://arxiv.org/abs/1811.10959

    2020 doi arXiv

  64. [72]

    R. Yu, S. Liu, X. Wang, Dataset distillation: A comprehensive review. arXiv: 2301.07014[cs]. URL http://arxiv.org/abs/2301.07014

    arXiv

  65. [73]

    URL https://doi.org/10.1007/s10462-024-11058-w

    doi:10.1007/s10462-024-11058-w . URL https://doi.org/10.1007/s10462-024-11058-w

    doi

  66. [74]

    Tsanakas, A

    S. Tsanakas, A. Hameed, J. Violos, A. Leivadeas, An Innovative Neuro-Genetic Algorithm and Geometric Loss Function for Mobility Prediction, in: Proceedings of the 19th ACM International Symposium on Mobility Management and Wireless Access, MobiWac ’21, Association for Computin...

    2021

  67. [75]

    H. Cho, Y. Kim, E. Lee, D. Choi, Y. Lee, W. Rhee, Basic enhancement strategies when using bayesian optimization for hyperparameter tuning of deep neural net- works 8 52588–52608. doi:10.1109/ACCESS.2020.2981072. URL https://ieeexplore.ieee.org/document/9037259/

    2020

  68. [76]

    A. Diaw, M. McKerns, I. Sagert, L. G. Stanton, M. S. Murillo, Efficient learn- ing of accurate surrogates for simulations of complex systems, Nature Machine Intelligence 6 (5) (2024) 568–577, publisher: Nature Publishing Group. doi: 10.1038/s42256-024-00839-1 . URL https://www...

    2024 doi

  69. [77]

    T. C. Y. Chan, R. Mahmood, I. Y. Zhu, Inverse Optimization: Theory and Ap- plications, Operations ResearchPublisher: INFORMS (Dec. 2023). doi:10.1287/ opre.2022.0382. URL https://pubsonline.informs.org/doi/full/10.1287/opre.2022.0382

    2023

  70. [78]

    A. M. Shetty, M. F. Aljunid, D. H. Manjaiah, A. M. S. Shaik Afzal, Hyperparameter Optimization of Machine Learning Models Using Grid Search for Amazon Review Sentiment Analysis, in: S. J. Nanda, R. P. Yadav, A. H. Gandomi, M. Saraswat (Eds.), Data Science and Applications, Spr...

    2024

  71. [79]

    Bischl, M

    B. Bischl, M. Binder, M. Lang, T. Pielok, J. Richter, S. Coors, J. Thomas, T. Ullmann, M. Becker, A.-L. Boulesteix, D. Deng, M. Lindauer, Hyperparam- eter optimization: Foundations, algorithms, best practices, and open challenges, WIREs Data Mining and Knowledge Discovery 13 (...

    2023 doi

  72. [80]

    Bergstra, Y

    J. Bergstra, Y. Bengio, Random Search for Hyper-Parameter Optimization, Jour- nal of Machine Learning Research 13 (10) (2012) 281–305. URL http://jmlr.org/papers/v13/bergstra12a.html

    2012

  73. [81]

    F. Yan, A. Koch, O. Sinnen, A survey on FPGA-based accelerator for ML models, arXiv:2412.15666 [cs] (Dec. 2024). doi:10.48550/arXiv.2412.15666. URL http://arxiv.org/abs/2412.15666 Artificial Intelligence, Data and Robotics: Frugal Machine Learning 25

    2024 doi arXiv

  74. [82]

    Parnell, C

    T. Parnell, C. Duenner, K. Atasu, M. Sifalakis, H. Pozidis, Large-scale stochas- tic learning using GPUs, in: 2017 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), 2017, pp. 419–428. doi:10.1109/ IPDPSW.2017.140. URL https://ieeexplore.ieee....

    2017

  75. [83]

    C. D. Schuman, T. E. Potok, R. M. Patton, J. D. Birdwell, M. E. Dean, G. S. Rose, J. S. Plank, A Survey of Neuromorphic Computing and Neural Networks in Hardware, arXiv:1705.06963 [cs] (May 2017). doi:10.48550/arXiv.1705.06963. URL http://arxiv.org/abs/1705.06963

    2017 doi arXiv

  76. [84]

    Balaji, A

    A. Balaji, A. Das, Y. Wu, K. Huynh, F. G. Dell’Anna, G. Indiveri, J. L. Krichmar, N. D. Dutt, S. Schaafsma, F. Catthoor, Mapping Spiking Neural Networks to Neu- romorphic Hardware, IEEE Transactions on Very Large Scale Integration (VLSI) Systems 28 (1) (2020) 76–86, conference...

    2020

  77. [85]

    H. Cai, C. Gan, T. Wang, Z. Zhang, S. Han, Once-for-all: Train one network and specialize it for efficient deployment. arXiv:1908.09791[cs,stat]. URL http://arxiv.org/abs/1908.09791

    1908 arXiv

  78. [86]

    D. T. Nguyen, N. H. Hung, H. Kim, H.-J. Lee, An approximate memory archi- tecture for energy saving in deep learning applications 67 (5) 1588–1601, con- ference Name: IEEE Transactions on Circuits and Systems I: Regular Papers. doi:10.1109/TCSI.2019.2962516. URL https://ieeexp...

  79. [88]

    Kuhrt, N

    M. Kuhrt, N. Glombiewski, M. K¨ orber, A. Morgen, D. Brandenstein, B. Seeger, Efficient Event Processing on Modern Hardware, in: K.-U. Sattler, A. Kemper, T. Neumann, J. Teubner (Eds.), Scalable Data Management for Future Hard- ware, Springer Nature Switzerland, Cham, 2025, pp...

    2025 doi

  80. [474]

    doi:$10.1007/978-981-99-7814-4\_36$

    doi

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.