REVIEW 4 major objections 5 minor 22 references
Self-Supervised Learning at the Edge: The Cost of Labeling
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Including label acquisition in the energy budget, semi-supervised contrastive learning matches supervised accuracy on roughly a quarter of the energy.
desk verdict Useful framing with honest measurements, but the headline energy ratios don't survive a recomputation from the paper's own tables. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is a two-term energy ledger, $E_{\text{total}} = E_{\text{train}} + E_{\text{labeling}}$, with labeling cost computed from Eq. (3), $E_{\text{labeling}} = P_{\text{energy}} \times K T_{\text{label}} / 3600$ kWh. The argument hinges on inserting the per-sample annotation time $T_{\text{label}}=10$ s at $P_{\text{energy}}=30$ W and comparing four supervision regimes (supervised cross-entropy, SimCLR, SupCon, CCSSL) under the same training procedures. This ledger converts accuracy differences into energy differences, and it is what makes the quantitative claim about CCSSL's roughly 4x advantage legible.
What would settle it
A controlled annotation-time study for CIFAR-10 or EuroSAT reporting mean seconds per label at measured desktop power would settle the core claim: if the true mean is below roughly 2-3 seconds per image, the label bill no longer dominates training energy and the paper's reported 2x and 4x energy ratios collapse.
Extended reading notes
Core claim
On its own terms, the paper establishes that a fair energy comparison of learning methods at the edge must add label-acquisition energy to training energy, and that once this is done the ranking changes. Using a ResNet-18 backbone on CIFAR-10 (with EuroSAT as a second dataset), the authors measure GPU, CPU, and RAM energy during 1,000-epoch training, then add a per-sample labeling cost of 10 seconds at 30 W. In this accounting, fully supervised approaches (a cross-entropy baseline and supervised contrastive learning, SupCon) reach the highest accuracies but carry a labeling bill that can outweigh their training energy, while self-supervised SimCLR trains cheaply but visibly loses accuracy. The central result is that the semi-supervised contrastive method CCSSL, trained with a labeled subset and unlabeled data, reaches accuracy within a point or less of SupCon while consuming roughly a quarter of SupCon's total energy; the paper reports the advantage persists and sharpens in low-data regimes.
Load-bearing premise
The whole quantitative comparison rests on assuming every labeled image costs 10 seconds of human attention at 30 W of desktop power, regardless of dataset, labeler, or interface.
Editorial extensions
If this is right
- Energy benchmarking of learning systems should include labeling as a first-class cost; otherwise supervised methods appear cheaper than they are in edge deployments.
- Semi-supervised contrastive learning becomes the default energy-accuracy tradeoff point for resource-constrained devices, since it matches supervised accuracy at a fraction of the total energy.
- The advantage of partial labeling grows as data becomes scarcer: in the 20% and 50% data regimes, the accuracy gap between CCSSL and SupCon stays small while the energy gap widens.
- In dynamic edge environments where models must be retrained on freshly observed data, the per-sample labeling bill recurs, making label-efficient SSL increasingly attractive relative to one-time supervised training.
Reading between the lines
- Editorial inference: the 10 s/30 W labeling estimate is for natural images; if annotation time varies by domain, the crossover point between supervised and semi-supervised energy moves accordingly.
- Editorial inference: the paper's ledger excludes the energy of transmitting data to a human annotator; for remote or federated edge settings, communication energy would further favor locally trained SSL.
- Editorial inference: a testable extension is to run the same comparison with active learning, where labels are requested only for the most informative samples; this could lower the label bill of CCSSL below what the paper reports.
- Editorial inference: if annotation becomes automated or near-free, supervised methods regain competitiveness, so the paper's conclusion is conditioned on the scarcity and cost of human labeling that motivates SSL in the first place.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents an empirical comparison of supervised, self-supervised, and semi-supervised contrastive learning methods (cross-entropy baseline, SimCLR, SupCon, CCSSL) on CIFAR-10 and EuroSAT, measuring training energy with CodeCarbon and adding an estimated labeling energy cost from Eq. (3). The authors claim that label acquisition can consume over twice the training energy for supervised methods and that semi-supervised CCSSL achieves near-supervised accuracy with up to 4x less total energy, motivating energy-aware benchmarking for edge learning.
Significance. If the numbers were correct, the paper would address a genuinely underexplored component of ML energy footprints (labeling) and provide a useful framework for comparing supervision regimes. The experimental setup is transparent (ResNet-18, fixed epochs, CodeCarbon tracking), and the inclusion of both accuracy and energy is valuable. However, the headline quantitative claims are not supported by the paper's own reported data, which weakens the contribution as written. The qualitative insight—that labeling energy can dominate training energy under certain assumptions and that semi-supervised methods may reduce total energy—is plausible and worth publishing, but only after the numerical claims are corrected and made reproducible from the reported tables and equations.
major comments (4)
- [Abstract & Section 2] The statement that for SupCon 'label acquisition may consume twice the amount of energy consumed for training' is not supported by the paper's own numbers. Using Eq. (3) with T_label=10 s and P_energy=30 W, labeling 25,000 CIFAR-10 samples (the 50% subset) costs 2.08 kWh, while Table 1 reports SupCon training energy of 1.25 kWh at 50% data and 2.51 kWh at 100% data. The labeling/training ratio is 1.66 in both cases, not 'over twice'. The ratio exceeds two only for the cross-entropy baseline at 50% data (2.08/0.63 = 3.3), which is not the claim made in Section 2.
- [Section 2 & Section 4.3] The claim that CCSSL requires 'almost 4× less energy' than SupCon to reach similar accuracies is contradicted by the paper's own Section 4.3, which states that CCSSL(50)(50) consumes 2.4× the energy of SimCLR while SupCon consumes 4× SimCLR's energy. These ratios imply CCSSL/SupCon ≈ 0.6, i.e., about 40% less energy, not 4x less (which would be 25% of SupCon's energy). Recomputing from Table 1 and Eq. (3) gives CCSSL(50)(50) total energy ≈ 1.08 + 1.04 = 2.12 kWh and SupCon total energy ≈ 1.25 + 2.08 = 3.33 kWh, again a ratio of 0.64. The 'up to 4X' claim in the Abstract is therefore unsupported by any reported comparison.
- [Section 4.3 & Fig. 2] The energy ratios quoted in Section 4.3 (2.4× and 4× relative to SimCLR) are not traceable to any reported table or figure values. From Table 1 and the labeling model, CCSSL(50)(50)/SimCLR ≈ 2.12/1.26 ≈ 1.68, and SupCon/SimCLR ≈ 3.33/1.26 ≈ 2.64, not 2.4 and 4. Either Fig. 2 contains additional energy components not described in the text, or the numbers in the text are misreported. The authors must provide the actual measured totals (e.g., in a table) and reconcile the text with those totals, because the central energy-efficiency comparison is based on these ratios.
- [Section 3.1, Eq. (3)] The labeling energy model depends critically on the assumed T_label = 10 s per sample and P_energy = 30 W, both taken from a single citation for natural images and applied uniformly to both datasets. The paper's qualitative conclusion that labeling dominates supervised training energy is load-bearing on these parameters. Since the authors aim to guide edge deployments, they should provide a sensitivity analysis over T_label (e.g., 1–30 s) and P_energy (e.g., 5–100 W), and discuss how the trade-off shifts when labeling is faster, automated, or crowdsourced. Without such an analysis, the robustness of the central claim is unclear.
minor comments (5)
- [Title] The title has a spacing issue: 'SELF-SUPERVISED LEARNING A T THE EDGE' should read 'AT THE EDGE'.
- [Table 1] The header 'EuroSA T' contains an unnecessary space; it should be 'EuroSAT'.
- [Equation (3)] The notation 'Penergy(Watts) × KT label' is ambiguous; please write it as 'K · T_label' to clarify the multiplication.
- [Figure 2 caption] The caption says 'labeling cost of ∼ 300 Joules per sample' (consistent with 10 s × 30 W), but the text in Section 4.3 refers to 'the cost of labeling a single sample' without stating the per-sample energy; consider stating it explicitly in the figure caption or main text.
- [Section 4.3] The notation 'CCSSL(20)(50)' is confusing: it means CCSSL using 50% of the training set with 20% of that subset labeled, but this should be stated more clearly, e.g., 'CCSSL with 20% of the selected subset labeled'.
Circularity Check
No significant circularity: the paper's energy and accuracy results are empirical measurements plus an externally cited labeling-cost model, not derived from their own conclusions.
full rationale
The paper's central claims are empirical: training energies are measured with CodeCarbon, and accuracies are measured under a stated protocol. The labeling energy model in Eq. (3) is an externally cited assumption (T_label = 10 s, P_energy = 30 W, from DualLabel [13]), not a parameter fitted to the paper's own results. There is no fitted input renamed as a prediction, no self-citation chain carrying a load-bearing premise, and no uniqueness theorem imported from the authors' prior work. The 'almost 4x less' and 'over twice' energy statements may be internally inconsistent with Table 1 and Fig. 2, but internal inconsistency is a correctness or reporting issue, not circularity. The derivation chain—defining total energy as training energy plus labeling energy, then evaluating it empirically—does not reduce to its own inputs. No circular step could be identified under the required evidence standard.
Assumptions & free parameters
free parameters (3)
- T_label (time to label one sample) =
10 s (from citation [13])
- P_energy (labeling power draw) =
30 W
- Labeled fraction for CCSSL =
50% (Table 1) or 10-20% (Fig 2)
assumptions (4)
- domain assumption CodeCarbon accurately measures CPU/GPU/RAM energy on the test machine.
- domain assumption The 10 s per-image annotation time from ref [13] transfers to CIFAR-10 and EuroSAT.
- domain assumption A 30 W desktop is representative of the labeler's hardware.
- standard math InfoNCE objective Eq (1) correctly represents the SSL methods compared.
Cite this review
Pith. "Pith review of Self-Supervised Learning at the Edge: The Cost of Labeling." pith.science (2026). https://pith.science/paper/ULLKUBGM
@misc{pith2026250707033,
author = {Pith},
title = {Pith review of: Self-Supervised Learning at the Edge: The Cost of Labeling},
year = {2026},
howpublished = {\url{https://pith.science/paper/ULLKUBGM}},
note = {Machine review of arXiv:2507.07033}
}
read the original abstract
Contrastive learning (CL) has recently emerged as an alternative to traditional supervised machine learning solutions by enabling rich representations from unstructured and unlabeled data. However, CL and, more broadly, self-supervised learning (SSL) methods often demand a large amount of data and computational resources, posing challenges for deployment on resource-constrained edge devices. In this work, we explore the feasibility and efficiency of SSL techniques for edge-based learning, focusing on trade-offs between model performance and energy efficiency. In particular, we analyze how different SSL techniques adapt to limited computational, data, and energy budgets, evaluating their effectiveness in learning robust representations under resource-constrained settings. Moreover, we also consider the energy costs involved in labeling data and assess how semi-supervised learning may assist in reducing the overall energy consumed to train CL models. Through extensive experiments, we demonstrate that tailored SSL strategies can achieve competitive performance while reducing resource consumption by up to 4X, underscoring their potential for energy-efficient learning at the edge.
Reference graph
Works this paper leans on
-
[1]
INTRODUCTION With the increasing number of edge devices, in domains such as healthcare, autonomous systems, and environmental mon- itoring, there is a growing demand for intelligent algorithms that can operate efficiently and locally [1]. These devices gen- erate large amounts of unlabeled data, making traditional su- pervised learning approaches at the e...
-
[2]
BACKGROUND AND MOTIV A TION Supervised learning has long been the dominant paradigm in AI/ML solutions, offering high predictive performance and relatively straightforward optimization pipelines. Nonethe- less, in real-world deployments, models trained on static, fully-labeled datasets often fail to generalize due to misalign- ment between the training di...
-
[3]
ENERGY CONSUMPTION In what follows, we provide a detailed breakdown of the energy consumption associated with learning under different supervision regimes. Our analysis considers both: (i) the cost of labeling (both manual and with the help of an external server), often neglected in traditional evaluations, and (ii) the computational energy associated wit...
-
[4]
NUMERICAL RESULTS In this section, we evaluate the total energy consumption and accuracy trade-offs of vanilla cross-entropy and CL methods. We consider four approaches: a standard supervised model trained with cross-entropy loss (baseline); SimCLR as a rep- resentative self-supervised CL method [2]; SupCon as a su- pervised CL approach [9]; and a semi-su...
-
[5]
CONCLUSION In this work, we presented a comprehensive analysis of the energy-performance trade-offs in contrastive learning meth- ods with a focus on edge scenarios. By measuring both training energy and the often-overlooked cost of labeling, we uncover critical trade-offs between performance and resource consumption. Our empirical analysis reveals that w...
-
[6]
Convergence of edge computing and deep learning: A comprehensive survey,
Xiaofei Wang, Yiwen Han, Victor CM Leung, Dusit Niyato, Xueqiang Yan, and Xu Chen, “Convergence of edge computing and deep learning: A comprehensive survey,” IEEE communications surveys & tutorials, vol. 22, no. 2, pp. 869–904, 2020
work page 2020
-
[7]
A simple framework for contrastive learning of visual representations,
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton, “A simple framework for contrastive learning of visual representations,” inInternational con- ference on machine learning. PmLR, 2020
work page 2020
-
[8]
Momentum contrast for unsupervised visual representation learning,
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick, “Momentum contrast for unsupervised visual representation learning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 9729–9738
work page 2020
Show all 22 references
-
[9]
Exploring simple siamese representation learning,
Xinlei Chen and Kaiming He, “Exploring simple siamese representation learning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 15750–15758
2021
-
[10]
Bootstrap your own latent-a new approach to self-supervised learning,
Jean-Bastien Grill, Florian Strub, Florent Altch ´e, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mo- hammad Gheshlaghi Azar, et al., “Bootstrap your own latent-a new approach to self-supervised learning,” Ad- vances in...
2020
-
[11]
Class-aware contrastive semi-supervised learn- ing,
Fan Yang, Kai Wu, Shuyi Zhang, Guannan Jiang, Yong Liu, Feng Zheng, Wei Zhang, Chengjie Wang, and Long Zeng, “Class-aware contrastive semi-supervised learn- ing,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 14421–14430
2022
-
[12]
Tinyml meets IoT: A comprehensive survey,
Lachit Dutta and Swapna Bharali, “Tinyml meets IoT: A comprehensive survey,” Internet of Things , vol. 16, pp. 100461, 2021
2021
-
[13]
Rep- resentation learning with contrastive predictive coding,
Aaron van den Oord, Yazhe Li, and Oriol Vinyals, “Rep- resentation learning with contrastive predictive coding,” arXiv preprint arXiv:1807.03748, 2018
2018 arXiv
-
[14]
Supervised Contrastive Learn- ing,
Prannay et. al. Khosla, “Supervised Contrastive Learn- ing,” Advances in neural information processing sys- tems, vol. 33, pp. 18661–18673, 2020
2020
-
[15]
Weakly supervised contrastive learning,
Mingkai Zheng, Fei Wang, Shan You, Chen Qian, Changshui Zhang, Xiaogang Wang, and Chang Xu, “Weakly supervised contrastive learning,” in Pro- ceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 10042–10051
2021
-
[16]
Dissecting supervised contrastive learn- ing,
Florian Graf, Christoph Hofer, Marc Niethammer, and Roland Kwitt, “Dissecting supervised contrastive learn- ing,” in International Conference on Machine Learning. PMLR, 2021, pp. 3821–3830
2021
-
[17]
Un- biased supervised contrastive learning,
Carlo Alberto Barbano, Benoit Dufumier, Enzo Tartaglione, Marco Grangetto, and Pietro Gori, “Un- biased supervised contrastive learning,” arXiv preprint arXiv:2211.05568, 2022
2022 arXiv
-
[18]
DualLabel: secondary labels for chal- lenging image annotation,
Chia-Ming Chang, Yi He, Xi Yang, Haoran Xie, and Takeo Igarashi, “DualLabel: secondary labels for chal- lenging image annotation,” in Graphics Interface, 2022
2022
-
[19]
mlco2/codecarbon: v2.4.1,
Benoit Courty, Victor Schmidt, and et. all., “mlco2/codecarbon: v2.4.1,” May 2024
2024
-
[20]
How to estimate carbon footprint when training deep learning models? A guide and review,
Luc ´ıa Bouza, Aur ´elie Bugeau, and Lo ¨ıc Lannelongue, “How to estimate carbon footprint when training deep learning models? A guide and review,” Environmental Research Communications, vol. 5, no. 11, pp. 115014, 2023
2023
-
[21]
Evalu- ating the RAM energy consumption at the stage of soft- ware development,
DA Maevsky, EJ Maevskaya, and ED Stetsuyk, “Evalu- ating the RAM energy consumption at the stage of soft- ware development,” Green IT Engineering: Concepts, Models, Complex Systems Architectures, 2017
2017
-
[22]
Unsupervised feature learning via non-parametric instance discrimination,
Zhirong Wu, Yuanjun Xiong, Stella X Yu, and Dahua Lin, “Unsupervised feature learning via non-parametric instance discrimination,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 3733–3742
2018
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.