{"id":"075eb5cc-580d-4972-bceb-85b8ab8df5d3","arxiv_id":"2412.03950","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"BEFL is a client-selection framework for federated learning that jointly optimizes model accuracy, total device energy, and the variance of energy consumption across devices, evaluated on simulated CIFAR-10 and MNIST tasks.","lead":"This paper proposes BEFL, a method for deciding which battery-powered devices train a shared machine learning model each round, aiming to keep their energy use balanced. It combines a hand-designed scheduling rule with reinforcement learning, and reports lower overall energy use and more even battery drain in simulated mobile edge settings.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline energy-variance and total-energy reductions are computed in a synthetic relative-energy metric that also defines the RL reward; with a per-device exponent δ the reported gains may be an optimization artifact rather than real battery-balancing behavior.","rationale":"The reader's weakest-assumption analysis identifies the energy model as the load-bearing premise, and my stress test agrees: the paper's strongest quantitative claims are about energy, and those claims are entirely computed inside a simulator whose energy model is also the RL objective. I sharpen the concern to a specific mechanism: the per-device exponent δ makes Er_i a dimensionally inconsistent, non-physical quantity, and because the same formula generates both the reward and the reported metric, the reported variance reduction may reflect optimization of a synthetic objective rather than true battery-drain balancing. This is not a claim of misconduct; the paper explicitly says 'the energy consumption mentioned in this article specifically refers to the relative energy consumption under this formula,' so the internal logic is coherent. The problem is external validity. The provided GitHub repository and the reproducible simulator are real supporting evidence, and the method may well be useful, but without a physical or hardware validation step the energy claims cannot be accepted as evidence about real MEIoT devices. The reader's CONDITIONAL verdict remains exactly appropriate: the condition is to validate or replace the energy metric, disclose the δ distribution and reward thresholds, and report seed variability. I therefore recommend no change to the reader's verdict.","tokens_in":7257,"tokens_out":3464,"duration_ms":37986,"concrete_test":"Rerun the full evaluation with δ fixed to 1 for all devices, so that Er_i becomes the physical battery fraction (Etrain_i + Etrans_i)/E_total_i, and use this same physical fraction for both the RL reward and all reported energy/variance metrics; retrain BEFL from scratch under this metric and compare against the baselines with the same change. If BEFL's variance reduction vs FedAvg falls substantially below the reported 72.7%, or its total-energy reduction falls substantially below 28.2%, the headline claims are artifacts of the δ mechanism. As a secondary check, repeat all runs with three random seeds and disclose the exact δ distribution and reward thresholds T, E, V.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claims for energy rest on the relative energy definition Er_i = (Etrain_i + Etrans_i) / E_total_i^δ in Section II-A3, where δ is a per-device 'sensitivity coefficient' sampled from an undisclosed normal distribution on [0,1] (Section IV-A2). This Er_i is not merely an evaluation metric: the same quantity enters the RL reward through the energy and variance thresholds E_t and V_t in Section III-C. Consequently, the optimizer directly minimizes the exact metric used to report the 72.7% variance reduction and 28.2% total-energy reduction. The metric itself is dimensionally questionable: for δ≠1, Er_i has units of energy divided by (battery capacity)^δ, so it is neither physical energy nor a comparable fraction of battery depletion. Figure 3 even labels it in KJ/mAh, which only makes sense when δ=1. With independently drawn δ values, variance across clients can be reduced by selecting devices with favorable denominators, not by equalizing actual drain on device batteries. No hardware measurements, power traces, or independent physical energy model validate Er_i. Thus the load-bearing premise—that variance reduction in Er_i corresponds to meaningful energy balancing on mobile IoT devices—is unverified, and the self-referential loop between training reward and evaluation metric makes the reported gains vulnerable to artifacts of the unvalidated model.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes BEFL, a federated-learning client-selection and resource-allocation framework for mobile edge IoT that jointly optimizes global model accuracy, total energy consumption, and energy-consumption variance across devices. The framework combines SLSQP-based communication-resource allocation, a cluster-and-utility heuristic client-selection algorithm, offline imitation learning from that heuristic, and an online ranking-based reinforcement-learning stage. The evaluation is simulation-based on CIFAR-10 and MNIST with 100 virtual clients of heterogeneous hardware, compared against FedAvg, FedProx, AFL, Favor, and FlashRL. The paper reports that BEFL improves accuracy by 1.6%, reduces energy variance by 72.7%, and lowers total energy by 28.2% over existing methods.","tokens_in":7570,"tokens_out":4126,"duration_ms":40194,"significance":"If the energy-balancing claims hold, BEFL addresses a real and under-studied problem: preventing premature dropout of energy-sensitive devices in federated learning. The accuracy results are evaluated on standard external datasets and are not circular, and the combination of SLSQP allocation, heuristic warm-start, and RL is a reasonable design direction. The paper also provides a public code link, which is a useful reproducibility aid. However, the central energy claims are built on a synthetic relative-energy metric that is simultaneously the RL reward signal and the evaluation metric, and no independent physical energy validation is provided; the reported gains therefore are not yet established as real battery-energy improvements.","major_comments":[{"comment":"The relative energy metric Er_i = (Etrain_i + Etrans_i) / Etotal_i^delta is used both as the evaluation metric for energy consumption and as the objective that the RL reward directly minimizes through the energy and variance thresholds E_t and V_t. The reported 72.7% variance reduction and 28.2% total-energy reduction are therefore reductions of the same modeled quantity that generates the training reward. This is a self-referential evaluation loop: the optimizer may be exploiting properties of the synthetic model rather than balancing real battery drain. The authors should evaluate energy claims with an independent physical energy model, power traces, or hardware measurements, and should report absolute physical energy (J or mAh) alongside the normalized metric.","section":"Section II-A3 and Section III-C"},{"comment":"Several load-bearing parameters are undisclosed: the mean and standard deviation of the normal distribution for the sensitivity coefficient delta, the parameters of the CPU-frequency normal distributions, the penalty thresholds T, E, V, the reward exponents alpha, beta, gamma, and the efficiency factor alpha in the heuristic selection function. Without these values, the experiments cannot be reproduced, and the sensitivity of the reported gains to these choices is unknown. The authors should provide the full parameter settings and a sensitivity analysis for the key thresholds and exponents.","section":"Section IV-A2, Section III-B, Section III-C"},{"comment":"The three headline percentages in the Abstract are taken from different settings and baselines: the 72.7% variance reduction is BEFL versus FedAvg in the IID CIFAR-10 row (27.3% vs 100%), while the 28.2% energy reduction and the 1.6% accuracy improvement are from the Non-IID CIFAR-10 row (71.8% vs 100% energy, 43.44% vs 41.84% accuracy). The paper should state which baseline and setting each headline number refers to. Moreover, Table II reports single values without error bars, standard deviations, or number of seeds, so the statistical significance of the accuracy and energy differences is not established.","section":"Table II and Abstract"},{"comment":"For delta not equal to 1, Er_i has units of energy divided by (battery capacity)^delta, which is neither a physical energy nor a dimensionless fraction. Figure 3 labels the quantity in KJ/mAh, which is only dimensionally valid when delta = 1, yet Section IV-A2 states that delta is drawn from a normal distribution on [0,1] with per-device values. Comparing such heterogeneous quantities across clients makes the reported 'variance' a variance of objects with different units. The authors should either justify the metric, set delta = 1, or report physical energy values for the variance and total-energy claims.","section":"Section II-A3 and Figure 3"}],"minor_comments":[{"comment":"The formula for relative energy should be written as Er_i = (Etrain_i + Etrans_i) / (Etotal_i)^delta to remove the ambiguity in the denominator notation.","section":"Section II-A3"},{"comment":"Constraint (1), Ephi_h >= sum_{theta_i in phi_h}(Etrain_i + Etrans_i), does not appear to impose a meaningful restriction as written; the symbol Ephi_h and the direction of the inequality should be clarified.","section":"Section II-B"},{"comment":"The caption contains a typo: 'energy consumption ensitivity' should be 'energy consumption sensitivity.'","section":"Figure 1 caption"},{"comment":"Reference [7] is incomplete; it lists only an author name and a journal title with no article title, volume, or year, and should be completed.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper does not compare against FedRank [17], which is the closest prior work combining ranking-based client selection with imitation and reinforcement learning. It is in the reference list but absent from Table II, and an explicit comparison would strengthen the novelty assessment. I recommend major revision rather than rejection because the accuracy results are externally evaluated and the energy claims, while currently unvalidated, could in principle be supported by adding an independent energy model and full parameter disclosure."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this one. First, it tackles a real problem—uneven battery drain that drops devices out of federated learning—and the proposed fix, a cluster-based heuristic that pre-trains an RL client selector, is a sensible extension of the FedRank line. Second, the headline numbers (72.7% variance reduction, 28.2% total energy cut) are produced by the same synthetic energy model that generates the RL reward. That circularity is the thing to check.\n\nThe paper's actual contribution is honest: it defines relative energy consumption as Er_i = (Etrain+Etrans)/E_total^δ, where δ is a per-device 'sensitivity coefficient.' I don't know the distribution parameters—they're undisclosed. The evaluation uses this metric throughout. The same E_t and V_t thresholds appear in the reward, and Table II reports variance relative to FedAvg in this same Er space. So the optimizer is directly minimizing the metric being reported. That's not fraud, but it does mean the 72.7% is a property of the model, not of real batteries. The dimensions don't even work for δ≠1: energy divided by battery capacity to the δ is not an energy. Figure 3 labels it KJ/mAh, which only makes sense at δ=1.\n\nOn the positive side, the method itself is new: clustering devices by ideal energy consumption and reallocating communication resources is a clean heuristic, and using it to warm-start an RL agent is a practical way to reduce RL sample complexity. The MDP definition is explicit, and the code is promised though I didn't run it. Accuracy gains are modest but consistent across settings, and those come from external datasets, so they're not circular.\n\nSoft spots besides the metric: no seeds, no error bars, no parameter values for reward thresholds or the normal distributions. The headline 72.7% is cherry-picked across baselines and settings. The ablation only shows a loss curve, not quantitative energy numbers. These are fixable.\n\nBottom line: this paper is for the FL-systems crowd who care about client selection and battery life. The idea is worth reading, but I wouldn't cite its energy claims without an independent validation. If it lands on your desk, send it to review: the problem is real, the approach is coherent, and the flaws are addressable—but the energy model needs to be validated or replaced before the numbers can be trusted.","headline":"Useful incremental idea for energy-balanced FL client selection, but the headline energy savings are computed with a self-referential synthetic metric that needs hardware validation before I'd trust the 72.7%.","tokens_in":8051,"tokens_out":2635,"would_cite":false,"duration_ms":25163,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"BEFL cuts federated-learning energy variance by 72.7% while improving accuracy.","keywords":["federated learning","energy consumption","client selection","reinforcement learning","imitation learning","resource allocation","mobile edge IoT","energy fairness"],"falsifier":"Run BEFL on a physical testbed of the five device types listed in Table I, record actual battery drain and dropout rates over 100 rounds, and check whether energy variance drops by the simulated amount; if the reduction in modeled relative energy does not appear as reduced variance in actual milliampere-hours, the central claim is not supported.","tokens_in":7053,"feed_emoji":"🔋","tokens_out":3959,"duration_ms":32137,"temperature":0.7,"pith_summary":"The paper argues that energy imbalance among devices, not just total energy use, is what hurts federated learning on battery-powered mobile IoT devices. It proposes BEFL, a joint optimization framework that allocates communication resources with the SLSQP algorithm and selects clients through a heuristic that clusters devices by ideal energy consumption, then uses that heuristic to pre-train a reinforcement learning agent. In simulations on CIFAR-10 and MNIST, BEFL improves global model accuracy by about 1.6%, cuts variance in device energy consumption by up to 72.7%, and lowers total energy consumption by 28.2% relative to existing methods. The significance is that federated learning deployments on energy-sensitive devices could last longer and lose fewer participants if energy fairness is made an explicit objective.","feed_headline":"BEFL cuts federated-learning energy variance by 72.7%","feed_subtitle":"RL-based client selection also lifts accuracy 1.6% and trims total energy 28.2% in simulated mobile IoT settings.","key_machinery":"The load-bearing mechanism is the relative energy consumption model $E^r_i = (E^{train}_i + E^{trans}_i)/E^{total,\\delta}_i$, where $\\delta$ is a per-device sensitivity coefficient, together with a client-selection MDP whose reward multiplies accuracy gain by penalty factors for latency, total energy, and energy variance. The heuristic splits devices into high- and low-ideal-energy clusters, reallocates communication resources via SLSQP, and defines a utility $F(\\varphi_i, E_i) = \\alpha \\varphi_i / (E^{trans}_i + E^{train}_i)$ that discourages over-selection of any device. This heuristic generates state-action pairs used to pre-train the RL agent, which then refines selection online with a target network for stable Q-value updates.","core_discovery":"The central claim is that energy fairness can be optimized jointly with accuracy and total energy without sacrificing any of them, by treating client selection as a ranking problem learned from a heuristic warm start. In the paper's simulations, BEFL consistently outperforms FedAvg, FedProx, AFL, Favor, and FlashRL: best accuracy reaches 52.46% on non-IID CIFAR-10 and 43.44% on non-IID MNIST, while energy variance drops to as low as 24.6% of the FedAvg baseline. The mechanism is the relative energy consumption model $E^r_i = (E^{train}_i + E^{trans}_i)/E^{total,\\delta}_i$, which makes energy-sensitive devices costly to overuse, and a reward function that penalizes energy variance alongside latency and total energy.","pith_inferences":["If the relative energy model matches hardware, the framework could adaptively tune $\\delta$ per device based on battery state-of-health instead of drawing it from a fixed normal distribution.","The two-cluster heuristic suggests a testable extension: an adaptive number of clusters might better capture multi-modal device populations in larger fleets.","The results imply that total energy and energy variance are partly in tension; real deployments would need to set the penalty thresholds $(T, E, V)$ to match operator priorities.","The reported 72.7% variance reduction is relative to modeled energy; a hardware testbed is needed to confirm the effect transfers to physical battery drain."],"forward_implications":["Energy fairness can be built into FL client selection without a separate fairness constraint; the variance penalty in the reward suffices.","The heuristic warm start shortens RL convergence, so the method works in training budgets where pure RL would not.","The same reward structure could be applied to other resource-constrained FL settings, such as bandwidth-limited or latency-critical edge networks.","Including relative energy rather than absolute energy in the objective shifts selection away from devices with small battery capacity, which should reduce premature dropout."],"supporting_citations":[{"why":"Defines FedAvg, the basic FL baseline that BEFL compares against and builds upon.","marker":"[2]"},{"why":"Supplies the SLSQP algorithm used for communication resource allocation.","marker":"[15]"},{"why":"Introduces ranking-based client selection with imitation learning and RL, the foundation for BEFL's learning approach.","marker":"[17]"},{"why":"FedProx serves as a baseline that adjusts local iterations for stability.","marker":"[21]"},{"why":"AFL is a heuristic-based baseline that selects devices based on model and client data.","marker":"[22]"},{"why":"Favor is a learning-based baseline using accuracy to guide local weight selection.","marker":"[24]"},{"why":"FlashRL is a learning-based baseline using Double Deep Q-Learning for system and static heterogeneity.","marker":"[25]"}],"fun_headline_variants":["BEFL trims energy variance 72.7% in mobile IoT federated learning","BEFL lowers total energy 28.2% and lifts accuracy 1.6%","Energy-fair client selection: BEFL cuts variance 72.7%","BEFL: balancing energy use in federated learning for IoT"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The framework's claimed energy gains depend on the simulated relative energy model, with its per-device sensitivity coefficient drawn from a normal distribution, faithfully capturing how real mobile IoT devices drain their batteries.","fun_headline_variants_meta":{"raw":{"variants":["BEFL trims energy variance 72.7% in mobile IoT federated learning","BEFL lowers total energy 28.2% and lifts accuracy 1.6%","Energy-fair client selection: BEFL cuts variance 72.7%","BEFL: balancing energy use in federated learning for IoT"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000756,"raw_usage":{"total_tokens":3378,"prompt_tokens":983,"completion_tokens":2395,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":599,"completion_tokens_details":{"reasoning_tokens":2310}},"tokens_in":599,"tokens_out":2395,"duration_ms":15510,"temperature":1.0,"reasoning_tokens":2310,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T21:54:15.799073+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run BEFL on a physical testbed of the five device types listed in Table I, record actual battery drain and dropout rates over 100 rounds, and check whether energy variance drops by the simulated amount; if the reduction in modeled relative energy does not appear as reduced variance in actual milliampere-hours, the central claim is not supported.","supporting_citations":[{"cited_title":"Model-contrastive federated learning,","cited_arxiv_id":null,"evidence_quote":"FedProx serves as a baseline that adjusts local iterations for stability."},{"cited_title":"Optimizing federated learning on non-iid data with reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Favor is a learning-based baseline using accuracy to guide local weight selection."},{"cited_title":"Communication-efficient learning of deep networks from decentralized data,","cited_arxiv_id":null,"evidence_quote":"Defines FedAvg, the basic FL baseline that BEFL compares against and builds upon."},{"cited_title":"Sequential quadratic programming,","cited_arxiv_id":null,"evidence_quote":"Supplies the SLSQP algorithm used for communication resource allocation."},{"cited_title":"”Ranking-based Client Imitation Selection for Efficient Federated Learning.” Forty-first International Conference on Machine Learning","cited_arxiv_id":null,"evidence_quote":"Introduces ranking-based client selection with imitation learning and RL, the foundation for BEFL's learning approach."},{"cited_title":"FLASH-RL: Federated Learning Addressing System and Static Heterogeneity using Reinforcement Learning,","cited_arxiv_id":null,"evidence_quote":"FlashRL is a learning-based baseline using Double Deep Q-Learning for system and static heterogeneity."}],"review_version":1}