{"id":"83ea4aa3-e3b3-4cb2-9e17-7bee5de5bc93","arxiv_id":"2501.14995","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"An automated platform that cuts edge-device AI energy by searching an expanded neural architecture space with energy predictors and a Pareto front.","lead":"GreenAuto is an automated platform that searches for energy-efficient deep learning models for edge devices by combining an expanded neural architecture space, pre-trained energy predictors, and automated power measurements. A generalist might read it to see a concrete pipeline for lowering the electricity and carbon cost of on-device AI model design.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 1 requires gradients over a discrete architecture space without defining any relaxation; until this is specified, the claimed efficiency of gradient-guided sampling is unsubstantiated.","rationale":"The reader's weakest assumption identifies exactly the same load-bearing gap: Algorithm 1 assumes meaningful gradients over discrete architectural choices without defining how they are computed. This is not a disagreement with current consensus; it is an internal specification gap. The measured energy values and the reported gains over MobileNet-V2 are concrete and independently checkable in principle, and the use of actual on-device measurements to update the Pareto front is a reasonable design. However, the search algorithm is the platform's advertised contribution, and the paper provides no way to verify that gradient-guided sampling outperforms random sampling or any other heuristic on the same budget. Secondary concerns (no error bars, a carbon-footprint comparison across different GPU generations, weak Kendall's tau for the energy predictor, and a 5000 Hz sampling rate that may be coarse for very short inferences) reinforce the conditional verdict but do not replace the central gap. Since the reader already conditions acceptance on filling these gaps, no verdict change is needed; the proposed ablation would directly test whether the gradient mechanism actually carries the efficiency claim.","tokens_in":9619,"tokens_out":6214,"duration_ms":62425,"concrete_test":"Run a controlled comparison of Algorithm 1's gradient-similarity selection against random sampling with the same 170-model measurement budget, using the hypervolume of the final energy-accuracy Pareto front as the metric. If the fronts are statistically indistinguishable, the gradient signal is not doing load-bearing work; if gradient sampling is better, report the exact relaxation or finite-difference scheme that defines g_i(x) over the Table 1 space, resolving the specification gap.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central efficiency claim rests on Algorithm 1, which selects new models by the similarity between each candidate's gradient g_i(x) and the optimal gradient direction g*(x) (Eq. 4, lines 6-13). The search variables, however, are discrete output-channel counts, kernel sizes, and strides from Table 1, and the paper never supplies a differentiable relaxation, an embedding, or a finite-difference rule that would make g_i(x) well-defined for an unevaluated architecture. If g_i(x) cannot be computed, the line 9 selection is formally unspecified, and the reported Pareto-front improvements over NASNet-A cannot be attributed to gradient-guided search rather than to the energy predictor plus repeated random sampling. This is load-bearing because the abstract and Section 2.2 advertise gradient descent as the mechanism that makes exploration efficient, so without a definition the search-efficiency claim reduces to an untested heuristic.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"GreenAuto is an end-to-end automated platform for neural architecture search targeting energy-efficient image classifiers on edge devices. The paper expands the NAS-Bench-201 cell space by varying output channels, kernel sizes, and strides (Table 1), yielding 959,417 candidate models. A kernel-level energy predictor from the authors' prior work and the NASWOT zero-cost accuracy proxy provide initial estimates; a Pareto-front search with two proposed 'gradient descent' algorithms (Algorithms 1 and 2) iteratively selects candidates for physical training and on-device energy measurement using a Monsoon power monitor. On CIFAR-10, the platform reports a best model at 0.88 accuracy and 0.2 mJ inference energy and another at 0.93 accuracy and 4.27 mJ, both compared favorably to MobileNet-V2 (0.88, 7.53 mJ), and a carbon-footprint comparison against NASNet-A showing 0.0478% of the per-model search emissions. The paper claims the pipeline operates without human intervention.","tokens_in":9767,"tokens_out":5514,"duration_ms":46318,"significance":"If the results hold, GreenAuto would be a useful demonstration that an automated, hardware-aware NAS pipeline can find image classifiers with substantially lower inference energy than a common mobile baseline, and the reported physical measurements on a real device are a strength. The platform's modular design and the use of a pre-trained energy predictor with a reported Kendall's tau of 0.526 on 1,179 physically measured models provide a credible basis for further study. However, the core gradient-guided sampling mechanism is not mathematically specified for the discrete architecture space, and the external baseline comparison is not controlled; these issues currently prevent the efficiency and sustainability claims from being fully supported.","major_comments":[{"comment":"The search selects models by comparing each candidate's gradient g_i(x) with the optimal gradient direction g*(x), but the search space (Table 1) is discrete: output channels, kernel sizes, and strides take categorical values. The paper never defines a differentiable parameterization, continuous relaxation, embedding, or finite-difference rule that would make g_i(x) meaningful for an unevaluated architecture. Consequently, the inner-product similarity in line 9 is formally unspecified, and the claimed efficiency gain from gradient-guided sampling is not established. Please provide an explicit definition (e.g., a straight-through estimator over a continuous relaxation, or finite differences over the discrete configuration grid) or revise the claims to present the sampler as a heuristic.","section":"Section 2.2, Eq. (4) and Algorithm 1"},{"comment":"The comparison with NASNet-A uses different hardware (450 K40 GPUs over seven days versus a single RTX 4090 over roughly one week), different search algorithms, and different training pipelines, so the reported 0.0478% carbon-emission ratio conflates methodology with hardware generation and is not a controlled result. A fair comparison would require re-running NASNet-A (or an equivalent evolutionary baseline) on the same GPU and measurement setup, or clearly labeling the numbers as an illustrative order-of-magnitude estimate rather than a benchmark result.","section":"Section 4, Table 2 and the carbon comparison"},{"comment":"The evaluation reports a single run without error bars, confidence intervals, or repeated-seed results. Since the initial k models and per-iteration samples are drawn from distributions (Algorithm 1, lines 3 and 9), the observed Pareto front and best models could be substantially affected by stochasticity. Please report variance over multiple independent runs, or at minimum a sensitivity analysis of the reported accuracy and energy values.","section":"Section 4, Fig. 3 and Table 2"},{"comment":"The energy predictor from the authors' prior work is used both as the search objective and as the guide for candidate sampling, making the search self-referential. While the final candidates are physically measured and the Kendall's tau of 0.526 on 1,179 models is an external check, the paper should clarify whether these 1,179 models are independent of the search process and discuss how predictor bias (e.g., systematic underestimation of energy for certain configurations) could affect which regions of the search space are explored. This is important because the stopping criterion and the reported best models depend on the predictor's ranking during search.","section":"Sections 2.1 and 2.2"}],"minor_comments":[{"comment":"The text states that the best model in Fig. 3d achieves 0.88 accuracy and 0.2 mJ, but the caption for (d) lists 0.88 accuracy and 1.00 mJ; additionally, the progression from iteration 5 to iteration 7 appears to increase energy, which would contradict the definition of a Pareto front. Please correct the caption or the text.","section":"Fig. 3 and Section 4 text"},{"comment":"The heading contains a typo: 'Pareto fornt-Based Model Search' should read 'Pareto front-Based Model Search'.","section":"Section 2.2 heading"},{"comment":"The text says 'In the first iteration, neither the Pareto front nor the best model changes,' but Fig. 3b is labeled 'Pareto Frontier for iteration 1' and 'Best Model of iteration 1'; please clarify whether these labels refer to the previous front or a new front.","section":"Section 4, Fig. 3b"},{"comment":"The carbon-footprint notation '0.013/model1' contains a stray '1' and the metric is not defined; please clarify whether the reported value includes training energy, measurement energy, and any idle overhead.","section":"Section 4, Table 2"},{"comment":"The notation is inconsistent: the algorithm uses 'k' for the initial sample count and 'm' for the per-iteration sample count, while Section 4 states the opposite (initial model number m = 100, per-iteration k = 10). Please standardize the notation.","section":"Algorithm 1 and Section 2.2"},{"comment":"The stopping criterion (accuracy > 0.9 and energy < 7 mJ) is introduced only in Section 4, but Algorithm 1 refers to 'predefined constraints'; the criterion should be stated in Section 2.2 where the algorithm is defined.","section":"Section 2.2"},{"comment":"The 'average prediction accuracy of 86.2%' for the energy predictor is cited from prior work but not defined; please state the metric (e.g., percentage of predictions within a tolerance band) so that the Kendall's tau value can be interpreted alongside it.","section":"Section 2.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is a systems-oriented workshop submission and the core idea is promising, but the undefined gradient over a discrete space is a formal gap that currently leaves the main search-efficiency claim unsupported. The carbon comparison with NASNet-A should be reframed as an illustrative estimate rather than a rigorous benchmark. The paper would also benefit from releasing the code and data, and from reporting repeated-run statistics. These issues are fixable within the scope of a revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"GreenAuto is a workshop-scale demonstration that the search loop actually works: it combines an expanded NAS-Bench-201 space (959,417 models), a kernel-level energy predictor from the authors' prior work, a zero-cost accuracy proxy, and automated on-device power measurement into one pipeline. The integration is real and the measured results are concrete. On a single Android phone, the best found model reaches 0.88 accuracy at 0.2 mJ inference energy, versus MobileNet-V2 at 0.88/7.53 mJ on the same device. That is a meaningful gain. The Kendall's Tau of 0.526 against 1,179 physically measured models is a genuine independent check on the predictor and should be credited.\n\nThe soft spot is the one the stress-test note flags. Algorithm 1 samples new architectures by comparing each candidate's gradient g_i(x) to the optimal gradient direction g*(x), but the search variables are discrete—output channel counts, kernel sizes, strides from Table 1—and the paper never defines a differentiable relaxation, an embedding, or a finite-difference rule. So 'gradient descent guided' sampling is formally unspecified. The empirical results do not depend on that formalism; the same pipeline could be run with random sampling and probably produce similar Pareto fronts, but the paper's efficiency claim rests on the gradient story and that claim is not backed up. Two smaller gaps: the evaluation has no error bars on energy measurements, and the carbon comparison against NASNet-A uses different GPU generations (K40 vs 4090), which muddies the '0.0478% of emissions' claim. There is also no baseline of random sampling on the same search space, which would be the cheapest way to test whether the gradient mechanism adds anything.\n\nWho gets value from this: researchers in hardware-aware NAS and energy-efficient edge inference who want a concrete automated measurement loop and a set of energy-accuracy trade-offs on one device. Read it critically, trust the measurements more than the search-efficiency claims. The gaps are presentation-level rather than fatal; a serious referee would ask for the gradient definition and a random baseline, and those are reasonable requests.\n\nRecommendation: accept for peer review. The integration is novel enough, the measurements are honest, and the underspecified gradient step is fixable in revision without changing the core demonstration.","headline":"A concrete automated energy-aware NAS demo with real measured gains on one phone; the gradient-guided search step is underspecified, so the measured results are worth more than the stated mechanism.","tokens_in":10323,"tokens_out":2874,"would_cite":true,"duration_ms":25175,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims GreenAuto, an automated pipeline, can find image classifiers for edge phones that match or beat MobileNet-V2's accuracy while using a fraction of the energy, without human intervention.","keywords":["sustainable AI","energy-efficient deep learning","neural architecture search","Pareto front","edge devices","on-device energy measurement","carbon footprint","gradient descent sampling"],"falsifier":"Run the same experiment with the same 170-model training budget on the same edge device using randomly sampled architectures; if random sampling reaches comparable accuracy-energy Pareto points or finds a model with similar accuracy and energy, the claimed gradient-guided sampling efficiency is not supported.","tokens_in":9384,"feed_emoji":"🌱","tokens_out":3869,"duration_ms":34272,"temperature":0.7,"pith_summary":"GreenAuto is an end-to-end automated platform for designing image-classification models on edge devices, optimizing for both accuracy and measured inference energy. The paper claims the pipeline explores a search space of over 959,000 model variants and finds models that outperform a standard mobile baseline while using a fraction of its energy, with no human in the loop. If this holds, energy-efficient on-device AI can be treated as an automated search problem rather than a manual tuning exercise, and the approach could scale to billions of edge devices.","feed_headline":"Automated AI search cuts edge inference energy 37-fold","feed_subtitle":"GreenAuto finds a CIFAR model at 0.88 accuracy and 0.2 mJ, beating MobileNet-V2 while running without human oversight.","key_machinery":"The load-bearing mechanism is a three-stage loop: an expanded NAS search space varying output channels, kernel sizes, and strides (959,417 models); a Pareto-front multi-objective search that uses kernel-level energy-predictor estimates and NASWOT accuracy scores, guided by a gradient descent direction g*(x) (a weighted combination of per-objective gradients) to sample the next model candidates; and an automated on-device energy measurement using an external power monitor synchronized by inference start and stop triggers. The iteratively updated Pareto front and a final gradient-magnitude-based selection procedure pick the best model.","core_discovery":"The paper's central claim is that an automated pipeline combining an expanded neural architecture search space, pre-trained kernel-level energy predictors, a zero-cost accuracy proxy, Pareto-front search with gradient-guided sampling, and automated on-device power measurement can efficiently identify sustainable DNN models. On CIFAR-10 with a smartphone edge device, GreenAuto reports best models of 0.93 accuracy at 4.27 mJ and 0.88 accuracy at 0.2 mJ, both outperforming MobileNet-V2's 0.88 accuracy and 7.53 mJ on the same device. The search trained only 170 models on a single GPU over roughly one week, with a per-model search carbon footprint of 0.013 kgCO2 versus 0.231 kgCO2 for the NASNet-A baseline. The authors conclude that this pipeline demonstrates efficient identification of sustainable models without human intervention.","pith_inferences":["Editorial extension: if the kernel-level energy predictors transfer to new devices without retraining, the same global-view search could be reused across hardware; the paper does not establish this transferability.","Editorial extension: the paper never defines a differentiable relaxation of the discrete search space, so the gradient-guided sampling may be equivalent to a heuristic; a random-search or evolutionary baseline with the same measurement budget could test whether the gradients are doing real work.","Editorial extension: the carbon comparison is per-model, not total search cost; the reported ratio depends on the number and type of GPUs used, which differ between the two searches.","Editorial extension: the same Pareto-front pipeline could be extended to transformer-based or LLM-style models on edge devices, as the authors note in their future-work section, though energy predictors for those architectures would be needed."],"forward_implications":["Sustainable model search can run without human intervention, automating the measurement, training, and refinement loop on a target edge device.","The search cost is drastically lower than conventional NAS: 170 trained models versus 20,000, with per-model search carbon emissions of 0.013 kgCO2 compared with 0.231 kgCO2 for NASNet-A.","Models found by the pipeline can beat a common mobile baseline: 0.93 accuracy at 4.27 mJ and 0.88 accuracy at 0.2 mJ versus MobileNet-V2's 0.88 accuracy at 7.53 mJ.","Because the platform is modular and reconfigurable, the same pipeline can be redirected to other objectives, hardware, or search-space definitions."],"supporting_citations":[{"why":"Supplies the foundational NAS-Bench-201 cell-based search space that GreenAuto expands.","marker":"[9]"},{"why":"Supplies the pre-trained kernel-level energy predictors used to estimate energy for all model candidates.","marker":"[10]"},{"why":"Supplies the zero-cost NASWOT accuracy proxy used for initial accuracy estimates during search.","marker":"[11]"},{"why":"Supplies the CIFAR-10 dataset on which models are trained and validated.","marker":"[12]"},{"why":"Supplies the TensorFlow benchmark tool used for inference latency profiling on the edge device.","marker":"[13]"},{"why":"Supplies the Monsoon power monitor that collects high-frequency power data for automated energy measurement.","marker":"[14]"},{"why":"Supplies the NASNet-A search baseline used for carbon footprint comparison.","marker":"[15]"}],"fun_headline_variants":["Automated NAS finds edge AI 37x more energy-efficient","GreenAuto: self-driving search cuts edge inference energy 37x","Zero-intervention AI design: 37x energy cut on edge devices","Automated Pareto search delivers 37x greener edge models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The search loop assumes that 'gradient descent' over a discrete set of architectural choices—output channel counts, kernel sizes, and strides—produces meaningful guidance, even though the paper does not define a differentiable parameterization of the search space.","fun_headline_variants_meta":{"raw":{"variants":["Automated NAS finds edge AI 37x more energy-efficient","GreenAuto: self-driving search cuts edge inference energy 37x","Zero-intervention AI design: 37x energy cut on edge devices","Automated Pareto search delivers 37x greener edge models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000727,"raw_usage":{"total_tokens":3194,"prompt_tokens":816,"completion_tokens":2378,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":432,"completion_tokens_details":{"reasoning_tokens":2305}},"tokens_in":432,"tokens_out":2378,"duration_ms":31634,"temperature":1.0,"reasoning_tokens":2305,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T14:43:27.244267+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same experiment with the same 170-model training budget on the same edge device using randomly sampled architectures; if random sampling reaches comparable accuracy-energy Pareto points or finds a model with similar accuracy and energy, the claimed gradient-guided sampling efficiency is not supported.","supporting_citations":[{"cited_title":"NAS-Bench-201: Extending the scope of reproducible neural architecture search","cited_arxiv_id":null,"evidence_quote":"Supplies the foundational NAS-Bench-201 cell-based search space that GreenAuto expands."},{"cited_title":"Unveiling energy efficiency in deep learning: Measurement, prediction, and scoring across edge devices","cited_arxiv_id":null,"evidence_quote":"Supplies the pre-trained kernel-level energy predictors used to estimate energy for all model candidates."},{"cited_title":"Neural architecture search without training","cited_arxiv_id":null,"evidence_quote":"Supplies the zero-cost NASWOT accuracy proxy used for initial accuracy estimates during search."},{"cited_title":"https://ai.google.dev/edge/litert/models/measurement","cited_arxiv_id":null,"evidence_quote":"Supplies the TensorFlow benchmark tool used for inference latency profiling on the edge device."},{"cited_title":"https://www.msoon.com/high-voltage-power-monitor","cited_arxiv_id":null,"evidence_quote":"Supplies the Monsoon power monitor that collects high-frequency power data for automated energy measurement."},{"cited_title":"Regularized evolution for image classifier architecture search","cited_arxiv_id":null,"evidence_quote":"Supplies the NASNet-A search baseline used for carbon footprint comparison."}],"review_version":1}