{"id":"d27053a5-86bf-4e3e-adb7-aa3b2a767e81","arxiv_id":"2411.10619","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A federated LSTM for household load forecasting that chooses each client's learning rate by validation loss is reported to beat federated averaging and LSTM on a single-household dataset.","lead":"Smart meters in this study train a shared LSTM forecast model without sharing raw power data, and each meter picks its own learning rate from three options using a small validation sample. The authors report that this personalized learning-rate choice beats plain federated learning and LSTM on one public household dataset.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Adaptive learning-rate selection is the whole claimed advantage, but the algorithm as written cannot identify a best rate: line 9's loss does not depend on α, and the prose's 10-round trial is absent from the pseudocode. No support for transfer.","rationale":"In good faith, the paper is a modest empirical proposal: a federated LSTM with per-client learning-rate selection, and the central claim is that this selection improves forecasting accuracy over plain FL and centralized LSTM. That claim is plausible and testable. The paper deserves credit for using a real dataset and for reporting comparisons against FedAvg and LSTM. However, the claimed mechanism is the only source of the alleged improvement, and it is both internally inconsistent and unsupported. The load-bearing assumption is not simply that a 20% temporary validation slice is representative; it is that the selection rule is well defined and that early validation loss correctly ranks candidate learning rates for the later training phase. The paper does not establish either. The reader's weakest assumption identified the transfer issue; the additional pseudocode/prose mismatch makes the concern stronger. Because a re-run with a simple ablation could settle the question, the empirical verdict should remain REJECT; I do not see a path to accepting the central claim on the current evidence.","tokens_in":8236,"tokens_out":6938,"duration_ms":77662,"concrete_test":"Implement the procedure literally as described in Sec. IV-B (10 trial local rounds with each of the three learning rates on Dtemp, pick the rate with lowest MAE, then 50 local rounds with that rate) on the same UCI dataset and five-client non-IID split. Compare this against (i) fixed learning rates 0.05, 0.001, and 0.0001, (ii) random per-client learning-rate choice, and (iii) an oracle that picks the rate by final Dtest loss, over at least five seeds. Report per-client mean±std MAE/RMSE. If the proposed selection is not significantly better than the best fixed/random choice and does not approach the oracle, the claimed advantage is unsupported. In addition, a literal run of Algorithm 1 as printed will show whether line 9 is meaningful.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's entire accuracy advantage is attributed to the per-client learning-rate selection in Sec. IV-B and Algorithm 1, lines 8-13. There are two unaddressed problems. First, the rule is not well defined: line 9 evaluates f'_i,k(α_i) = f_i,k((w^t_m,k, α_i), D^temp_m,k), but the loss functions defined in Eqs. (2)-(3) do not depend on a learning rate, so on the printed pseudocode all three candidate rates give the same loss and the 'best' rate cannot be selected. The prose in Sec. IV-B describes a different procedure--10 trial rounds with each rate on the temporary validation slice, then 50 training rounds with the winner--but Algorithm 1 puts the selection inside the per-epoch local loop. Second, even taking the prose at face value, the paper gives no theory, ablation, or sensitivity analysis showing that the rate with lowest loss after 10 rounds transfers to the 50-round training phase. If that transfer fails, the method is FedAvg with a data-dependent learning-rate schedule and the claimed advantage over fixed-LR FL disappears. The Section III Moreau-envelope/pFedMe equations do not describe the Algorithm 1 update, so they provide no independent support.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a personalized federated learning method for short-term electrical load forecasting in smart grids, called personalized meta-LSTM. The method lets each smart meter select one of three learning rates per global round by evaluating loss on a temporary validation subset, then trains a local LSTM with the chosen rate and aggregates local models through FedAvg. Experiments on the UCI Individual Household Electric Power Consumption dataset compare the method with standalone LSTM, centralized LSTM, and a federated LSTM baseline under IID and non-IID client partitions. The authors claim that the proposed approach outperforms state-of-the-art ML and FL methods in load forecasting accuracy.","tokens_in":8354,"tokens_out":4299,"duration_ms":40980,"significance":"If substantiated, per-client adaptive learning-rate selection would be a simple, server-light personalization mechanism for non-IID federated load forecasting and would preserve data privacy, which is a timely and important problem. The paper clearly motivates the privacy and heterogeneity challenges, uses a real-world dataset, and provides an intuitive architecture. However, the central contribution is not established: the algorithmic description is internally inconsistent, the experiments lack repeated runs and error bars, and the comparison omits the personalized federated learning baselines that the paper itself cites as closest work. Thus the claimed advantage over state-of-the-art methods is unsupported in the present version.","major_comments":[{"comment":"The loss functions in Eqs. (2) and (3) do not depend on the learning rate α_i, so the quantity f′_{i,k}(α_i) in line 9 is identical for every candidate rate and cannot identify a best rate. The prose in Sec. IV-B describes a different procedure—10 trial rounds with each rate on D^{temp}, then 50 training rounds with the winner—but Algorithm 1 places the selection inside the per-epoch local loop and has no trial rounds. The central claimed mechanism is therefore undefined as written; please reconcile the pseudocode with the prose and provide a well-defined selection rule.","section":"Algorithm 1, lines 8–13; Eqs. (2)–(3)"},{"comment":"The empirical comparison is based on a single run with no error bars, no multiple seeds, and no statistical tests. The differences between PFL and FL in Fig. 5 are small in absolute terms, so without repeated runs the conclusions that PFL is 'far more stable' and achieves 'better' accuracy are not supported. Please report mean and standard deviation over at least several independent runs and state the number of runs.","section":"Sec. IV-C, Figs. 3–5"},{"comment":"The comparison omits the personalized federated learning baselines cited in the paper, including pFedMe [18], Wang et al. [1], and Qu et al. [12]. The paper claims to outperform 'state-of-the-art ML and FL methods,' but without these PFL baselines the claim is unsupported. Please add the relevant PFL baselines or narrow the claim to the methods actually compared.","section":"Sec. IV-C, Fig. 5; Sec. III"},{"comment":"The experiments use one household from the UCI dataset and partition its data into synthetic clients; this does not reflect heterogeneous smart meters from different households, which is the stated motivation. The claim that the method handles 'diverse SMs' with different data sizes is only tested by splitting a single time series. A multi-household or multi-meter dataset, or a clear justification of why synthetic clients are sufficient, is needed to support the heterogeneity claims.","section":"Sec. IV-A and IV-B"},{"comment":"The Moreau-envelope/pFedMe formulation is not connected to Algorithm 1. The notation prox_{f_i/x_i}(w_k) in Eq. (13) is undefined, Eq. (14) is not a standard Moreau envelope, and the per-client learning-rate selection in Algorithm 1 does not correspond to any update in Eqs. (10)–(16). This section neither provides theoretical support nor clarifies the algorithm; either derive the algorithm from this formulation or remove the formalization.","section":"Sec. III, Eqs. (10)–(16)"}],"minor_comments":[{"comment":"The notation D^{temp}_{m,k}, Trainloader, and Testloader is not defined precisely; in particular, the relationship between D^{temp}_{m,k} and the 80%/20% split described in Sec. II-A should be stated in one place.","section":"Sec. II-B"},{"comment":"Line 9 has mismatched parentheses and uses f′_{i,k} before it is defined; also, the outer loop iteration over global rounds k and the inner loop over local epochs t make the variable k overloaded.","section":"Algorithm 1"},{"comment":"In Eq. (1), k is used both as the global-round index and as an index for the data sample, which is confusing; please use different symbols for these two roles.","section":"Eq. (1)"},{"comment":"The x-axis is labelled 'Epochs' but the text describes global rounds; the y-axis is labelled 'Loss' while the captions say MAE and RMSE. Please make the axes consistent and explicitly state which quantity is plotted.","section":"Sec. IV-C, Figs. 3–5"},{"comment":"Reference [11], a self-citation on modulation recognition, appears in the sentence about FL struggling with non-IID data but is not related to that claim; please replace it with an appropriate citation or remove it.","section":"References"}],"recommendation":"reject","confidential_remarks":"The manuscript appears to be an early draft with a central algorithmic inconsistency and insufficient experimental evidence for the stated claims. In my view, the issues require substantial rewriting and new experiments, so I recommend rejection. If the editor considers a resubmission path, the authors would need to fix Algorithm 1, add repeated runs with uncertainty quantification, include PFL baselines, and test on a genuinely multi-client dataset."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one thing worth knowing about this paper is that the only concrete idea—per-client learning-rate selection on a validation slice—is standard hyperparameter tuning, not meta-learning, and the paper's internal inconsistencies make the empirical claim untestable as written.\n\nWhat is actually new: nothing much. The algorithm reduces to FedAvg with an LSTM, where each client tries three learning rates on a temporary validation slice and trains with the best one. That is a reasonable engineering heuristic, but it is not personalized learning in any meaningful sense—all clients end up with the same averaged global model—and it is not meta-learning in the MAML or Per-FedAvg sense. The paper does earn some credit for a clear motivation: the privacy argument for federated load forecasting is real, and the dataset is public and reproducible. The comparison against vanilla LSTM and FL is fine as a start, but that is about all.\n\nThe soft spots are substantial, and the stress-test note is correct. Algorithm 1, line 9, evaluates the loss as a function of the learning rate, but the loss functions defined in Eqs. (2)–(3) do not depend on a learning rate. So on the printed pseudocode, all three candidate rates give identical loss and no best rate can be selected. The prose describes a 10-round trial before the 50-round training, but the pseudocode places the selection inside the per-epoch loop. That is a load-bearing inconsistency, not a typo. In addition, Section III's Moreau envelope and pFedMe equations do not describe the update actually run in Algorithm 1, so the theoretical framing provides no support. The experiments use a single household split into five synthetic clients, with no error bars, no repeated seeds, and no comparison against the PFL baselines cited in the paper (pFedMe, Per-FedAvg, [1], [12]). The abstract's claim of outperforming state-of-the-art ML and FL methods is therefore unsupported. On the citation pattern, the self-citation [11] is for a related PFL application and is not a problem by itself.\n\nWho is this for? A practitioner looking for a quick trick in federated load forecasting might find the learning-rate selection heuristic worth a test, but as a research paper it does not hold up. It is not a serious contribution in its current form, and the internal contradictions are severe enough that I would not send it to peer review. If the authors fixed the algorithm, ran proper baselines with error bars, and toned down the claims, it could become a minor conference paper. Right now, I would desk reject it.","headline":"This is a weakly validated hyperparameter-tuning trick dressed up as meta-learning, and the printed algorithm cannot even select a learning rate; not ready for peer review.","tokens_in":9033,"tokens_out":1589,"would_cite":false,"duration_ms":17081,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A personalized federated LSTM lets each smart meter choose its own learning rate and forecasts next-hour electricity load more accurately than fixed-rate federated or centralized LSTM baselines on non-IID household data.","keywords":["federated learning","personalized federated learning","load forecasting","smart meter","LSTM","meta-learning","non-IID data","smart grid"],"falsifier":"Take the exact non-IID five-client setup and rerun Algorithm 1 with the selection rule inverted—choose the learning rate with the highest loss on the temporary validation slice—while keeping every other detail fixed; if the inverted rule matches or beats the proposed rule on test MAE and RMSE, the validation-based meta-learning step is not the cause of the reported improvement. A second check is to compare the rate chosen at round 10 with the rate that would have produced the lowest loss at round 50; any substantial disagreement undermines the transfer assumption.","tokens_in":7861,"feed_emoji":"⚡","tokens_out":6073,"duration_ms":52811,"temperature":0.7,"pith_summary":"Electrical load forecasting keeps smart grids stable, but household usage patterns differ strongly from meter to meter and sending all readings to a utility raises privacy concerns. This paper tries to show that a personalized federated learning scheme can get the best of both: each smart meter trains a local LSTM on its own readings, participates in global federated averaging, and privately tests a short list of learning rates on a small validation slice to pick the rate that best fits its own data before the longer local training run. The reported simulations on a real household electricity dataset indicate that this per-meter learning-rate selection lowers both mean absolute error and root mean squared error on non-IID data compared with a centralized LSTM and with standard federated averaging using any single fixed learning rate. A sympathetic reader would take the contribution to be an inexpensive personalization step: no extra server computation, no raw data leaving the meter, and accuracy gains concentrated exactly where data heterogeneity hurts plain federated learning.","feed_headline":"Each smart meter tunes its own learning rate to forecast load","feed_subtitle":"A per-client validation step improves next-hour load forecasts on non-IID data without sharing raw readings.","key_machinery":"The load-bearing object is the temporary-validation learning-rate selector inside each smart meter: a small set of candidate learning rates $\\alpha_1,\\dots,\\alpha_j$, a 20 percent holdout slice $D^{(k)}_{\\text{temp},m}$ of the client's own training data, and a rule that after ten local rounds picks the rate $\\beta^{(k)}_m$ giving the lowest loss, then uses $\\beta^{(k)}_m$ for the remaining fifty local rounds. The server side is ordinary federated averaging, $w_{k+1} = \\frac{1}{M}\\sum_{m\\in M} w_{m,k}$, so the personalization is entirely client-local and adds no aggregation cost. The underlying sequence model is an LSTM with one 50-unit layer, dropout 0.2, and a dense output neuron, fed 24 hourly load values to forecast the next hour. The paper also connects its objective to a Moreau-envelope formulation of personalization via pFedMe, but the algorithm itself does not compute proximal updates; the mechanism that carries the reported gains is the per-client rate choice.","core_discovery":"The paper's central claim is that per-client learning-rate selection, framed as meta-learning, is enough to make federated load forecasting competitive under non-IID smart-meter data. In Algorithm 1, after receiving the global model, each sampled meter runs ten local rounds with each candidate learning rate on a temporary 20 percent slice of its training data, records the rate with the lowest loss, and then trains for fifty local rounds on its full training data using that chosen rate; the server aggregates the resulting local models by plain federated averaging. The simulations use five clients with deliberately different batch sizes and data fractions and a 24-hour input LSTM predicting next-hour load, and compare against fixed-rate PFL variants and against LSTM and federated-learning baselines. The paper interprets the results as showing that no single learning rate is best for all clients, that meta-learning effectively combines the strengths of the candidate rates, and that the resulting forecasts track rapid consumption changes better than the baselines.","pith_inferences":["The reported comparison is against fixed learning rates and standard baselines; an untested implication is that the same temporary-validation rule could be applied to batch size or number of local epochs, not just learning rate.","One way to stress the claim is to benchmark the selector against a cheap random or round-robin learning-rate schedule; if random selection matches the validation-selected rates, the paper's meta-learning label would overstate what the mechanism contributes.","Because the selector uses only a client's own validation slice, the idea should transfer to other non-IID time-series tasks such as decentralized forecasting of traffic or renewable generation, though the paper does not test those.","The method's per-client choice could in principle be combined with other personalization layers such as local fine-tuning or regularization, yielding further gains beyond those the paper reports."],"forward_implications":["If the reported gains are real, utilities can improve next-hour load forecasts without pulling household consumption histories to a central server, preserving the privacy that regulations have protected.","Meters with different hardware, batch sizes, and data volumes can participate in the same global round, because personalization lives in the local rate choice rather than in server-side tuning.","The method adds no server-side computation and no extra communication beyond the normal model upload and download, so the personalization step scales with the number of meters.","On non-IID splits the meta-learning curve stays below fixed-rate curves in the reported MAE and RMSE plots, meaning the chosen rate avoids both the fast-starting-but-unstable high rate and the too-slow low rate."],"supporting_citations":[{"why":"Supplies the Individual Household Electric Power Consumption dataset, the real-world data for all simulations.","marker":"[19]"},{"why":"The LSTM load-forecasting baseline whose reported accuracy the proposed method is claimed to beat.","marker":"[3]"},{"why":"The distributed federated learning baseline, FedAvg-style with recurrent networks, used as the main FL comparator.","marker":"[6]"},{"why":"The meta-learning formulation of personalized federated learning that motivates the per-client optimization view.","marker":"[15]"},{"why":"Defines the Moreau-envelope personalization objective the paper uses as the closest related formulation.","marker":"[18]"},{"why":"Provides the general meta-learning framing that the paper invokes for learning-rate selection.","marker":"[8]"}],"fun_headline_variants":["Per-client learning rates boost federated load forecasts on non-IID data","Meta-learning tunes each smart meter's rate for personalized load forecasts","Federated learning adapts learning rates per meter to forecast household load","Personalized federated learning picks best local rate for smart meter load prediction","Client-specific learning rates improve federated load forecasting in smart grids"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole method depends on the assumption that the learning rate with the lowest loss after ten quick validation rounds is also the learning rate that will train best over the following fifty local rounds on the full training data.","fun_headline_variants_meta":{"raw":{"variants":["Per-client learning rates boost federated load forecasts on non-IID data","Meta-learning tunes each smart meter's rate for personalized load forecasts","Federated learning adapts learning rates per meter to forecast household load","Personalized federated learning picks best local rate for smart meter load prediction","Client-specific learning rates improve federated load forecasting in smart grids"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000634,"raw_usage":{"total_tokens":2918,"prompt_tokens":930,"completion_tokens":1988,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":546,"completion_tokens_details":{"reasoning_tokens":1896}},"tokens_in":546,"tokens_out":1988,"duration_ms":14214,"temperature":1.0,"reasoning_tokens":1896,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T19:30:35.410891+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the exact non-IID five-client setup and rerun Algorithm 1 with the selection rule inverted—choose the learning rate with the highest loss on the temporary validation slice—while keeping every other detail fixed; if the inverted rule matches or beats the proposed rule on test MAE and RMSE, the validation-based meta-learning step is not the cause of the reported improvement. A second check is to compare the rate chosen at round 10 with the rate that would have produced the lowest loss at round 50; any substantial disagreement undermines the transfer assumption.","supporting_citations":[{"cited_title":"Individual household electric power con- sumption data set,","cited_arxiv_id":null,"evidence_quote":"Supplies the Individual Household Electric Power Consumption dataset, the real-world data for all simulations."},{"cited_title":"Optimal deep learning lstm model for electric load forecasting using feature selection and genetic algorithm: Comparison with machine learning approaches,","cited_arxiv_id":null,"evidence_quote":"The LSTM load-forecasting baseline whose reported accuracy the proposed method is claimed to beat."},{"cited_title":"Distributed load forecasting using smart meter data: Federated learning with recurrent neural net- works,","cited_arxiv_id":null,"evidence_quote":"The distributed federated learning baseline, FedAvg-style with recurrent networks, used as the main FL comparator."},{"cited_title":"Personalized federated learning with moreau envelopes,","cited_arxiv_id":null,"evidence_quote":"Defines the Moreau-envelope personalization objective the paper uses as the closest related formulation."}],"review_version":1}