{"id":"08efa7af-409a-42a4-a4bb-ffeb98c4c226","arxiv_id":"2501.16397","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A layer-wise Gaussian Process model estimates DNN training energy from measured probe networks, reducing MAPE from about 40% to about 10% versus FLOPs-based estimation.","lead":"THOR estimates the energy cost of training a neural network on a phone or edge device by measuring small probe networks, learning each layer type's energy profile with a Gaussian Process, and summing the layer predictions. It reports roughly 10% average error across five devices, compared with about 40% for a FLOPs-based baseline, and uses the estimates to guide pruning to a 50% energy budget.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The layer-wise additivity/subtractivity premise behind Eqs. (1) and (2) is validated only on plain stacked conv layers (Fig. 2); without a direct test on fused/residual/recurrent architectures, hidden-layer GP labels are potentially biased and the ~10% MAPE may not transfer.","rationale":"Agreement: the reader's weakest_assumption identifies the same concern I consider most load-bearing. The method is only as sound as the additivity/subtractivity decomposition, since without it the per-layer GP labels are not true layer-energy measurements and Eq. (4) is not a physically meaningful sum. The paper offers only Fig. 2, which uses plain stacked Conv2d layers on a single CNN; this does not generalize to the BN-ReLU-fused blocks, residual connections, LSTM cells, and Transformer attention used in the evaluation. The internal tension with Sec. 2.3 (which describes kernel fusion and in-place optimizations making execution a 'black box') reinforces the need for a direct test. The end-to-end MAPE is necessary but not sufficient because aggregation can hide compensating errors. The proposed test isolates subtractivity by using measured totals rather than GP estimates, so it can distinguish a genuinely additive system from one where THOR happens to fit the data. I keep the verdict at CONDITIONAL: the paper is a plausible engineering contribution, but the central premise is not yet established. Providing the additivity/subtractivity check (and, ideally, releasing code and data) would satisfy the condition and make the claim substantially stronger.","tokens_in":17211,"tokens_out":8449,"duration_ms":76303,"concrete_test":"Using measured (not GP-estimated) total energies on one device (e.g., Xavier), take a fixed input layer I, output layer O, and a hidden conv+BN+ReLU block H. For at least 10 random channel configurations (C1, C2), measure E(I+H+O), E(I+O), E(I+H), and E(I). Compute two independent estimates of E_H: (i) E(I+H+O) - E(I+O) and (ii) E(I+H) - E(I). Check whether |(i) - (ii)| / max(i, ii) remains within measurement noise (e.g., under 5%) across all configurations. If the difference is significant, the subtractivity assumption behind Eqs. (1) and (2) fails and all hidden-layer GP labels are biased; the reported MAPE would then not establish the validity of Eq. (4).","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim depends on layer-wise energy additivity and subtractivity (Sec. 3.2), which convert measured totals of 1-, 2-, and 3-layer probe networks into training labels for the hidden-layer GPs through Eqs. (1) and (2). This premise is weakly supported: Fig. 2 shows a roughly linear increase in observed energy when identical Conv2d layers are added to a CNN, but it does not exercise batch-normalization fusion, residual/skip connections, attention, or recurrent cells. The paper's own Sec. 2.3 argues that framework optimizations such as Conv-BN-ReLU fusion make execution \"more like a black box\", which directly undercuts the assumption that each layer's energy is independently additive. Because hidden-layer labels are obtained by subtracting GP predictions of the other layers, any cross-layer coupling (kernel fusion, memory reuse, power-state transitions) introduces a systematic bias into those labels. The end-to-end MAPE of ~10% does not refute this: over- and under-estimates from different layers can cancel in the sum, so Eq. (4) may be numerically accurate for the tested models even with badly biased per-layer predictions. The pruning case study relies on per-layer gradients from the same GPs, so its \"50% energy reduction\" claim is also sensitive to this bias. Appendix A4 lists parallel execution as a limitation, but sequential models with fused kernels are not exempt; a direct validation of subtractivity is needed before the per-layer decomposition can be trusted.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes THOR, a per-layer Gaussian Process (GP) based energy estimator for on-device DNN training. The method assumes layer-wise energy additivity and subtractivity, profiles one-, two-, and three-layer probe networks to obtain per-layer energy labels, fits GP models to those labels, and then predicts total training energy as the sum of per-layer GP predictions (Eq. 4). The evaluation compares THOR against a FLOPs-based linear baseline on five devices (OPPO, iPhone, Xavier, TX2, Server) and several model families (LeNet-5, a 5-layer CNN, HAR, LSTM, Transformer, ResNet), reporting an average MAPE around 10% versus roughly 40% for the baseline, and a pruning case study on Xavier that claims a 50% energy reduction.","tokens_in":17493,"tokens_out":4311,"duration_ms":40573,"significance":"If the layer-wise additivity premise holds, THOR is a practically attractive approach: it is device-agnostic, uses active-learning-style GP profiling with uncertainty-based termination, and its per-layer decomposition could enable energy-aware scheduling and pruning. The breadth of the empirical study across heterogeneous devices and model types is a real strength, as is the use of a nonparametric model that does not require hardware-specific analytical simulators. However, the central additivity/subtractivity assumption is validated only qualitatively, the per-layer labels are derived by subtracting GP estimates rather than by direct measurement, and the main MAPE figures lack error bars. These issues currently limit confidence in the central claim and in the pruning case study.","major_comments":[{"comment":"The layer-wise energy additivity/subtractivity premise is load-bearing but is validated only visually on a CNN with identical stacked Conv2d layers. The paper's own Sec. 2.3 states that frameworks fuse operations such as Conv-BN-ReLU into a single kernel, making execution 'more like a black box'; under such fusion, per-layer energy is not independently measurable, so subtracting estimated energies of other layers from measured totals can produce biased hidden-layer labels. Please provide a direct validation of subtractivity on architectures that include BatchNorm fusion, residual connections, attention blocks, and recurrent cells, or otherwise demonstrate that the per-layer decomposition in Eqs. (1), (2), and (4) does not inherit a systematic bias from cross-layer coupling.","section":"Sec. 3.2, Fig. 2, Eqs. (1)-(2)"},{"comment":"The hidden- and input-layer labels are formed by subtracting GP-based estimates (\\hat E_output and \\hat E_input) from measured totals. Any systematic error in those GPs enters the training labels for the remaining GPs and propagates into the final sum in Eq. (4). The reported end-to-end MAPE can be accurate even when per-layer estimates are badly biased, because over- and under-estimates from different layers may cancel in the sum. The paper should report per-layer estimation errors (e.g., quantified MAPE/RMSE for the surfaces in Figs. 11-12), provide an error-propagation analysis, or compare the subtraction-derived labels against a direct per-layer measurement on at least one device.","section":"Eqs. (1)-(2), Sec. 4.2"},{"comment":"The main quantitative claim (MAPE around 10% versus roughly 40% for the FLOPs baseline) is presented as bar charts without error bars or confidence intervals, even though Appendix A5.1 states that each experiment was repeated three times and that the mean and standard error are reported. Without these values, the reader cannot judge whether the observed improvements are statistically significant, particularly for models with higher error such as HAR or LSTM on smartphones. Please add error bars to Figs. 8-10 and include a table with per-device, per-model means and standard errors.","section":"Sec. 4.1, Fig. 8, Appendix A5.1"},{"comment":"The pruning case study claims that THOR reduces energy consumption by 50% while preserving accuracy, but it is unclear whether the reported 49.2% figure is a measured energy value or a THOR prediction. The claim also depends on per-layer GP gradients whose accuracy is not independently established. Please report the actually measured energy of the pruned model, specify the measurement procedure, and include multiple devices or models with variance across runs to support the generality of the claim.","section":"Sec. 4.3, Fig. 13"}],"minor_comments":[{"comment":"There is a typo in the sentence 'Based on the presented layer-wise energy additivity pf DNN'; it should read 'of DNN'. Similarly, Sec. 3.3 contains 'we use the the upper and lower bounds', which should be 'the upper and lower bounds'.","section":"Sec. 3.2"},{"comment":"The caption says 'Energy consumption from NeuralPower estimation and from observation for a CNN', but the surrounding text describes an adapted training-phase profiling method. Please clarify exactly what is plotted, how NeuralPower was adapted to training, and how the linear trend supports additivity rather than merely showing a correlation.","section":"Fig. 2 caption"},{"comment":"The subcaptions are garbled and repeated: the six panels are labeled '(a) H=W=42, N=10 (b) ... (a) Xavier, H=W=42 ...' in an overlapping way. Please relabel the panels uniquely by device and spatial size so the reader can map each surface to the correct setting.","section":"Fig. 11"},{"comment":"The bibliography appears to contain a duplicated and largely unrelated block of references (e.g., Abelson et al. 1985, Baumgartner et al. 2001, Brachman and Schmolze 1985, Gottlob 1992, Levesque 1984, Nebel 2000) that are not cited in the text. Please remove this block and ensure every reference is cited and every citation appears in the reference list.","section":"References"},{"comment":"The phrase 'reduced the Mean Absolute Percentage Error (MAPE) by up to 30%' is ambiguous because MAPE is already a percentage. Since the reported improvement is from about 40% to about 10%, please state the change as an absolute percentage-point reduction or as a relative reduction to avoid confusion.","section":"Abstract and Sec. 1"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid engineering paper with a real, if modest, contribution. The core idea—profile one-, two-, and three-layer probe networks, subtract the fitted GP estimates to get per-layer training-energy models, then sum them for new architectures—is sensible and, to my knowledge, not done before. The experiments span five devices and several architecture families (CNNs, LSTM, Transformer, ResNet), and the reported MAPE of around 10% versus 40% for a FLOPs linear baseline is a meaningful improvement. That part deserves credit.\n\nThe soft spot is exactly where the reader's concern lands: the layer-wise additivity/subtractivity assumption. Fig. 2 validates it only on a plain stacked Conv2d CNN with no BatchNorm fusion, no residual connections, no attention. The paper's own Sec. 2.3 argues that framework optimizations like Conv-BN-ReLU fusion make execution 'more like a black box', which cuts against the assumption. And because hidden-layer labels are obtained by subtracting GP predictions of other layers (Eqs. 1 and 2), any bias in those GPs corrupts the labels. The end-to-end MAPE could still look good if per-layer errors cancel in the sum, so it doesn't prove the decomposition is sound. The pruning case study inherits this: it uses per-layer gradients from the same GPs.\n\nOther weaknesses are minor but real. The baseline is only FLOPs + linear regression; there is no comparison with a learned surrogate or a simulation-based estimator. The headline MAPE figure lacks error bars, even though the appendix claims three repeats. No code or data was released. And the reference list contains a block of uncited, unrelated classic AI references, which looks like a cleanup artifact.\n\nNone of this sinks the paper. The empirical scope and the end-to-end results make a decent case that the method works for the tested sequential models. What's missing is a direct test of subtractivity on architectures with fused or residual operators, and ideally an examination of per-layer error rather than only the sum. If the authors add that, plus the artifacts, I'd be comfortable with the claims.\n\nRecommendation: send it to peer review. The flaws are addressable; a serious referee should ask for the additivity validation and the error bars. The work deserves that time.","headline":"A sensible engineering method for on-device training energy estimation, with a load-bearing additivity assumption that is only weakly tested; deserves a referee but needs a direct subtractivity check.","tokens_in":18101,"tokens_out":2791,"would_cite":true,"duration_ms":24138,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a DNN's on-device training energy can be estimated as the sum of per-layer Gaussian-process predictions, cutting mean absolute percentage error from about 40% to around 10% across five devices.","keywords":["energy estimation","on-device training","Gaussian process","layer-wise additivity","DNN energy profiling","energy-aware pruning","mobile devices","MAPE"],"falsifier":"Build a two-convolutional-layer model and measure its training energy directly; then measure a one-layer model with the same first convolution and a one-layer model with the same second convolution, add them, and compare with the two-layer measurement over a range of channel counts on a device with operator fusion enabled. If the mismatch systematically exceeds the measurement noise, additivity fails and the subtractive profiling labels in Eqs. (1) and (2) are biased.","tokens_in":16960,"feed_emoji":"⚡","tokens_out":5292,"duration_ms":45848,"temperature":0.7,"pith_summary":"To make on-device training energy predictable before it runs, the paper proposes treating a DNN as a sum of independent layers and fitting a Gaussian process to each layer type. The load-bearing claim is that total training energy equals the sum of per-layer energies, and that a layer's energy can be read off by subtracting the other layers' contributions from measured totals of small probe networks. On five heterogeneous devices, the summed estimates reach roughly 10% mean absolute percentage error, compared with roughly 40% for a FLOPs-based baseline. The authors then use the estimator to guide channel pruning and report cutting energy by 50% while keeping accuracy. A sympathetic reader would care because accurate, cheap energy estimates are the missing piece for job scheduling and energy budgets on battery-powered devices.","feed_headline":"Per-layer sum predicts training energy within ~10% on five devices","feed_subtitle":"Gaussian-process fits on isolated layers beat FLOPs-based energy estimates and guide pruning to a 50% energy budget.","key_machinery":"The machinery is the layer-wise energy additivity and subtractivity assumption, stated in Section 3.2 and encoded in Eqs. (1) and (2), together with the Gaussian processes that fit each layer's energy as a smooth function of channel counts. A GP is a non-parametric probabilistic regressor whose predictions come with uncertainty, which the paper exploits to guide the choice of the next profiling point by maximum variance. The Matérn kernel with $\\nu = 2.5$ supplies the GP's covariance, chosen to tolerate runtime optimization artifacts. The final estimator, Eq. (4), simply sums the per-layer GP predictions; profiling is a one-time cost per device and framework, and the paper reports most profiling and fitting runs finish within 20 minutes.","core_discovery":"The paper's central discovery candidate is that training energy obeys layer-wise additivity well enough to be instrumented: a model's total energy is the sum of its input, hidden, and output layer energies, and each layer's cost can be isolated by subtracting the other layers' costs from the measured total of a small variant network. Using this, THOR profiles one-, two-, and three-layer probe networks, fits a Gaussian process per layer type with the Matérn kernel, and predicts an unseen network's energy as $\\hat{E}_{\\mathrm{model}} = \\hat{E}_{\\mathrm{input}}(C_1) + \\sum_{i=2}^{n-1} \\hat{E}_{\\mathrm{hidden}}(C_{i-1}, C_i) + \\hat{E}_{\\mathrm{output}}(C_{n-1})$. The paper's evaluation reports MAPE around 10% across LeNet-5, a 5-layer CNN, HAR, LSTM, Transformer, and ResNet variants on five devices, versus roughly 40% for FLOPs-based estimation. The paper acknowledges that the decomposition assumes sequential layer execution, that parallel-branch architectures such as GoogleNet and SqueezeNet fall outside the current treatment, and that estimates degrade when the framework version changes.","pith_inferences":["If additivity holds beyond the tested models, the same subtractive recipe could produce per-layer estimates for other resources, such as latency, memory bandwidth, or thermal load, extending the profiling idea beyond energy.","The GP's predictive variance is a natural input for scheduling under probabilistic energy budgets, a use the paper hints at but does not develop.","A direct stress test would be to profile GoogleNet or SqueezeNet-style multi-branch blocks; until that is done, the additivity claim is effectively established only for sequential architectures.","The framework-version sensitivity reported (MAPE rising from 8% to 11% after an upgrade) suggests that a lightweight re-calibration rule could be derived from the GP's posterior variance, an extension the paper leaves implicit."],"forward_implications":["Energy-aware job schedulers can treat the per-layer GP sum as a cheap, differentiable budget oracle for on-device training.","Energy-constrained pruning can use THOR as the objective: the case study hits a 50% energy budget while the FLOPs-based baseline misses it.","One-time profiling per device and framework yields reusable layer models, so unseen architectures built from profiled layer types can be estimated without new measurements.","Because the estimator tracks nonlinear layer behavior (plateaus and ridges) that FLOPs misses, it can detect slow spots or energy cliffs missed by compute-based proxies.","The method's scope is sequential execution; the paper explicitly leaves parallel-branch models and framework-version drift as limitations."],"supporting_citations":[{"why":"The NeuralPower profiling baseline that THOR adapts to training and shows overestimates per-layer costs (Fig. 2).","marker":"[Cai et al., 2017]"},{"why":"Provides the Gaussian-process formulation and the Matérn kernel used to fit layer energy as a function of channel counts.","marker":"[Williams and Rasmussen, 2006]"},{"why":"Random channel pruning strategy used in the energy-aware pruning case study.","marker":"[Li et al., 2022]"},{"why":"Energy-aware pruning and FLOPs-based energy estimation, representative of the proxy-based baseline THOR compares against.","marker":"[Yang et al., 2017]"}],"fun_headline_variants":["Energy additivity enables training cost estimates within 10%","Gaussian process layers cut training energy error to ~10%","Layer-sum method slashes training energy estimation error","Training energy predicted per-layer, error under 10%","GP-based layer profiling guides pruning to 50% energy cut"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a layer's training energy is independent of its neighbors, so the cost of any layer can be recovered by subtracting the other layers' costs from the measured total of a small probe network, and the whole model's energy is exactly the sum of these independent layer costs.","fun_headline_variants_meta":{"raw":{"variants":["Energy additivity enables training cost estimates within 10%","Gaussian process layers cut training energy error to ~10%","Layer-sum method slashes training energy estimation error","Training energy predicted per-layer, error under 10%","GP-based layer profiling guides pruning to 50% energy cut"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000182,"raw_usage":{"total_tokens":1350,"prompt_tokens":1027,"completion_tokens":323,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":643,"completion_tokens_details":{"reasoning_tokens":252}},"tokens_in":643,"tokens_out":323,"duration_ms":3670,"temperature":1.0,"reasoning_tokens":252,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T13:58:14.669744+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build a two-convolutional-layer model and measure its training energy directly; then measure a one-layer model with the same first convolution and a one-layer model with the same second convolution, add them, and compare with the two-layer measurement over a range of channel counts on a device with operator fusion enabled. If the mismatch systematically exceeds the measurement noise, additivity fails and the subtractive profiling labels in Eqs. (1) and (2) are biased.","supporting_citations":[{"cited_title":"Neuralpower: Predict and Deploy Energy-efficient Convolutional Neural Networks","cited_arxiv_id":null,"evidence_quote":"The NeuralPower profiling baseline that THOR adapts to training and shows overestimates per-layer costs (Fig. 2)."},{"cited_title":"Gaussian Processes for Ma- chine Learning, volume","cited_arxiv_id":null,"evidence_quote":"Provides the Gaussian-process formulation and the Matérn kernel used to fit layer energy as a function of channel counts."},{"cited_title":"Designing Energy-Efficient Convolutional Neu- ral Networks using Energy-Aware Pruning","cited_arxiv_id":null,"evidence_quote":"Energy-aware pruning and FLOPs-based energy estimation, representative of the proxy-based baseline THOR compares against."}],"review_version":1}