{"id":"fb05fc5e-539e-4b63-8446-1df493b294f8","arxiv_id":"2411.19798","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"RMFL replaces the standard exponentially weighted average of local gradients in federated momentum with reverse-decayed weights, and reports accuracy gains over MFL on three benchmarks under non-IID splits.","lead":"A federated learning paper proposes reversed momentum: weighting early local gradients more than recent ones, on the assumption that recent gradients are more biased by client data heterogeneity. Experiments on MNIST, CIFAR10, and CIFAR100 report consistent accuracy gains over the momentum-based MFL baseline, though the comparison is limited to a single baseline.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (3) does not implement the claimed reverse-exponential weighting: the latest gradient receives β/(1−β)=9× the weight of the previous gradient, so the method as defined is inconsistent with its motivation.","rationale":"The reader's conditional verdict focuses on the unsupported assumption that local-gradient bias increases monotonically with local steps. I agree that this premise is under-supported and that the qualitative evidence for it is weak. However, there is a more immediate problem: Section III.B defines RMFL by Eq. (3), and that equation does not produce the reverse-exponential weighting described in the text. For β=0.9, the coefficient on the current gradient is nine times the coefficient on the immediately preceding gradient, so the most recent, allegedly most biased gradient still dominates at every local step. This is an internal inconsistency in the definition of the method, not a disagreement with existing literature or a missing convergence proof. Because Eq. (3) is the only formal specification of RMFL, the empirical tables and figures cannot be attributed to the claimed mechanism. The paper also contains supporting inconsistencies that aggravate this: Table I says results are averaged over 5 runs while Section IV.A says 10 runs, and no code is released, so the empirical claim cannot be independently checked. I am therefore not rejecting the underlying idea; a correctly defined reverse-weighted momentum might well help. But the current manuscript does not define what it evaluates, so the central claim is unverdictable rather than merely conditional. A short symbolic check settles the definitional issue; if the authors provide a corrected update rule, the headline experiments should be rerun and the learning-rate search reported. This is why I would move the verdict from CONDITIONAL to UNVERDICTED rather than to REJECT.","tokens_in":9061,"tokens_out":14442,"duration_ms":122339,"concrete_test":"Evaluate Eq. (3) symbolically for the experimental setting β=0.9 and t=1..30: compute the gradient weight sequence w_i = (1−β)β^i for i<t and w_t = β^t, and test whether it is monotonically decreasing. It is not, because w_t/w_{t−1} = β/(1−β) = 9. If the authors state that Eq. (3) contains a typo and the intended rule gives every gradient weight (1−β)β^i, rerun Table I with that corrected rule and report the learning rates selected for each method; the reported accuracy and F1 gaps must be re-established under the corrected definition.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"The central claim rests on the definition of RMFL in Eq. (3), but that equation is not a reverse-exponential average and does not match the prose in Section III. For the experimental value β=0.9, the coefficient of the current local gradient ∇f(x_t) is β^t, while the coefficient of the immediately preceding gradient ∇f(x_{t−1}) is (1−β)β^{t−1}; the ratio is β/(1−β)=9. Thus at every local step the most recent gradient receives nine times the weight of the second-most-recent gradient, not a smaller weight. The full weight sequence (excluding v0) is (1−β), (1−β)β, ..., (1−β)β^{t−1}, β^t; for t=10, ∇f(x_9) has weight 0.0387 and ∇f(x_10) has weight 0.3487, the largest single weight. Consequently, Eq. (3) does not de-emphasize the allegedly biased late-step gradients, and the experiments evaluate a mis-specified method. The ambiguity is compounded by Table I stating 5 runs while Section IV.A states 10 runs, and no code is released, so the empirical superiority claim cannot be independently checked. The monotonic-bias premise may or may not hold, but it cannot rescue a formula that fails to realize the proposed weighting.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes Reversed Momentum Federated Learning (RMFL), a modification of momentum in federated learning in which the standard exponential moving-average weighting of gradients is replaced by a weighting intended to give larger weight to early local gradients and smaller weight to later, putatively more biased gradients. The method is defined in Eq. (3). The authors report experiments on MNIST, CIFAR10, and CIFAR100 under Dirichlet heterogeneity levels alpha = 1, 0.1, 0.01 and local epochs 2, 5, and 10, claiming that RMFL consistently outperforms MFL in accuracy and macro F1-score.","tokens_in":9341,"tokens_out":7330,"duration_ms":60625,"significance":"The motivating observation that late local gradients are more biased in heterogeneous FL is worth studying, and the experimental matrix is broad. If the proposed weighting were correctly specified and robustly validated, the method would be a simple, practical contribution. However, the central definition in Eq. (3) does not realize the claimed reverse weighting: for beta = 0.9 the latest gradient receives nine times the weight of the immediately preceding one, so the experiments evaluate a method that is inconsistent with the paper's own description. In addition, the local update rule is under-specified and the experimental protocol has reproducibility and tuning issues. The contribution is therefore not established as written.","major_comments":[{"comment":"The proposed momentum estimate is not reverse-exponentially weighted. Expanding Eq. (3), the weights on ∇f(x_0), ..., ∇f(x_t) are (1−β), (1−β)β, ..., (1−β)β^{t−1}, β^{t}. For β = 0.9 and t = 10, ∇f(x_10) has weight 0.3487 while ∇f(x_9) has weight 0.0387, and the ratio of the most recent gradient to the immediately preceding one is β/(1−β) = 9. Thus the latest gradient receives the largest weight, directly contradicting the Section III claim that the weights are exponentially decayed with time going forward. Since Eq. (3) is the definition of RMFL, the experiments do not evaluate the proposed method as described.","section":"§III-B, Eq. (3)"},{"comment":"The local update rule is under-specified. Eq. (3) defines a momentum estimate, but the manuscript never states how this estimate is used to update the model parameters, for example whether x_{t+1} = x_t − η \\hat v_t or whether \\hat v_t is combined with an additional recursion. Without this equation, the implementation of RMFL is ambiguous and the reported results are not reproducible from the text alone, especially since no code is released.","section":"§III-B and §IV"},{"comment":"The experimental protocol is reported inconsistently and the comparison is not fully controlled. Section IV.A states that 10 repeated runs are conducted, while the Table I caption reports results averaged over 5 runs. The learning rate is searched per method in [0.3, 0.1, 0.03, 0.01, 0.003, 0.001] for the fastest convergence, but neither the selected learning rates nor the convergence criterion are reported; a per-method search without a fixed budget can favor one method. These issues make it impossible to verify that the reported gains are due to the momentum weighting rather than to tuning or implementation details.","section":"§IV-A and Table I"},{"comment":"The monotonic-bias premise is not established by the presented evidence. The text claims that gradient bias increases with local steps, but the supporting figures are mislabeled: Fig. 4 is captioned as cumulative momentum while the text describes average gradient cosine similarity, and Fig. 5 is captioned as active clients while the text describes average gradient projection. No numerical summary or statistical test is provided. If the bias does not increase monotonically, the reverse weighting has no principled basis; this premise must be quantified before the method can be assessed.","section":"§III-A"}],"minor_comments":[{"comment":"Typos include 'beed proved' in the abstract, 'address more basis' in Section III, 'As the MFL parer suggested' in Section IV.A, and 'the most challenge CIFAR100' in Section IV.A; these should be corrected.","section":"Throughout"},{"comment":"The text says Table I shows results on MNIST, CIFAR10, and CIFAR100, but the table caption says the results are on MNIST and CIFAR10 datasets; the caption should be aligned with the reported content.","section":"§IV.B and Table I"},{"comment":"The figure captions do not match the quantities described in the text; Fig. 4 is captioned as cumulative momentum while the text reports average gradient cosine similarity, and Fig. 5's caption says active clients while the text reports average gradient projection.","section":"Figures 4 and 5"},{"comment":"The cited FedDyn paper is described as a federated distillation approach on recommender systems, but the dynamic-regularization FedDyn commonly referenced in federated learning is a different paper (Acar et al., ICLR 2021); the reference should be checked.","section":"§II.B, reference [4]"}],"recommendation":"major_revision","confidential_remarks":"The central issue is the mismatch between Eq. (3) and the method's stated mechanism; fixing Eq. (3) would require rerunning all experiments, so I recommend major revision rather than rejection. The authors should also report the exact local update rule and the selected learning rates for each setting."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short take: the paper doesn't do what its headline says. Eq. (3), which is the entire proposal, is not a reversed exponential weighting. For β=0.9, the most recent local gradient gets 9× the weight of the one before it and about 9× the weight of the oldest, so RMFL as defined still emphasizes exactly the late, biased gradients the authors claim to suppress. The stress-test note is right; I checked the arithmetic.\n\nWhat's genuinely worthwhile: the underlying hypothesis — that in heterogeneous FL the later local steps carry more bias and standard momentum overweights them — is plausible and clearly stated. The experiments cover three datasets and three Dirichlet α levels, plus sensitivity to local epochs, and the reported gains for RMFL over MFL are consistent. If a corrected reversed-weighting scheme outperforms MFL that would be a simple, useful contribution.\n\nBut the soft spots are serious. Beyond the Eq. (3) mismatch, the paper never states how v̂_t enters the parameter update (the usual x ← x − η v̂_t is only implied). The comparison is against MFL only, not any of the other momentum/adaptive methods cited. The learning rate is tuned per method for 'fastest convergence', which can conceal optimizer-specific sensitivity. Table I says 5 runs while Section IV.A says 10. Figures 4 and 5 have captions that don't match what the text says they show. No code or data are provided. Any one of these would be minor; together they make the empirical claim unverifiable.\n\nThe math flaw is load-bearing. If the authors actually implemented Eq. (3), then the method is just a variant of MFL with a peculiar weighting, and the narrative is wrong. If they implemented the prose description, then Eq. (3) is a typo and we need a corrected paper with the true formula and re-run experiments. Either way, the current version cannot be used as is.\n\nBottom line: this is a paper with an interesting idea but a critical internal contradiction. I wouldn't cite it in its current form. A serious editor could send it to peer review in the hope that the authors fix the equation and provide code, but the review would be a heavy revision request. I'd rather treat it as a cautionary example in reading group than as a building block.","headline":"The method's defining equation contradicts its own motivation, and the empirical claims are unverifiable without code or a clear update rule.","tokens_in":9812,"tokens_out":4277,"would_cite":false,"duration_ms":35895,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that reversing the weight order in momentum accumulation—favoring early local gradients over recent ones—consistently improves federated learning accuracy and F1-score under heterogeneous data.","keywords":["federated learning","momentum","data heterogeneity","non-IID data","reversed momentum","momentum initialization","local epochs sensitivity"],"falsifier":"Measure the average cosine similarity (or projection) between client gradients and the mean gradient at each local step in a heterogeneous federated task. If the similarity does not decrease monotonically with the local step — for instance if it dips and then recovers — the premise of RMFL fails. A second check: run RMFL on a task where later gradients are known to be more aligned with the global objective (e.g., synthetic noise added only to early steps); if RMFL still wins, its benefit does not come from the claimed mechanism.","tokens_in":8869,"feed_emoji":"🔄","tokens_out":5289,"duration_ms":41039,"temperature":0.7,"pith_summary":"The paper argues that the standard way of accumulating momentum in federated learning is suboptimal when client data are heterogeneous: the usual exponential moving average gives the most weight to the most recent local gradients, but those are also the most biased by client divergence. It proposes Reversed Momentum Federated Learning (RMFL), which assigns exponentially decaying weights as training proceeds so early local gradients dominate the momentum sent to the server. Across MNIST, CIFAR10, and CIFAR100 under three heterogeneity levels, RMFL reports higher accuracy and F1-score than the momentum baseline in every setting tested, with the largest gains under the most skewed data. The result suggests that a simple reweighting of the momentum initialization can offset a major source of bias in federated learning.","feed_headline":"Reverse momentum weights boost federated learning accuracy","feed_subtitle":"Weighting early local gradients more lifts accuracy and F1 on MNIST, CIFAR10, CIFAR100 under heterogeneity.","key_machinery":"The central object is the reverse-exponential weighted average of Eq. (3), $\\hat v_t = (1-\\beta)v_0 + (1-\\beta)\\sum_{i=0}^{t-1}\\beta^i \\nabla f(x_i) + \\beta^t \\nabla f(x_t)$. It assigns the largest weight to the first local gradient and exponentially smaller weights to later gradients, the opposite of the standard momentum update in Eq. (1), so that the later, more biased local gradients contribute least to the momentum used to initialize the next local training round.","core_discovery":"The central claim is that the bias introduced by heterogeneous client data accumulates during local training, so a momentum buffer that emphasises recent gradients carries more bias than one that emphasises early gradients. The paper therefore replaces the standard exponential moving average of local gradients with the reverse-exponential weighted average defined in Eq. (3), and shows empirically that this estimated momentum outperforms the standard momentum baseline across three datasets, three heterogeneity levels, and local epochs of 2, 5, and 10. The authors interpret the gains as evidence that later local gradients are less useful for the global objective, supporting the reverse weighting.","pith_inferences":["A direct testable extension is to measure the actual gradient-bias-versus-local-step curve on a real heterogeneous task; the claimed mechanism predicts a monotonically increasing bias curve, and the benefit of RMFL should correlate with the slope of that curve.","The case-study figures in Section III appear to conflate cumulative momentum with gradient cosine similarity, so the direct empirical support for the monotonic-bias premise is thinner than the main accuracy comparison; readers should verify that premise independently.","If early local gradients were not more aligned with the global objective, RMFL could underperform standard momentum, so the method's success hinges on the direction of the bias, not merely its existence.","The same reweighting idea could generalize to adaptive optimizers such as Adam-style second-moment estimates in federated learning, where the bias pattern is likely similar."],"forward_implications":["RMFL improves accuracy and F1-score over the MFL baseline on all three benchmark datasets and all heterogeneity levels tested, with the largest gains at the most skewed data (CIFAR10 with $\\alpha=0.01$: accuracy 0.472 vs 0.247).","The advantage of RMFL grows with the number of local epochs, so it makes longer local training less harmful than standard momentum in heterogeneous settings.","RMFL shows smaller standard deviation than MFL in most settings, indicating more stable convergence across random seeds.","The reweighting idea could be applied to other momentum-based federated optimization methods that use exponential moving averages, as the paper itself notes."],"supporting_citations":[{"why":"Defines the federated averaging setup (FedAvg) that forms the training loop RMFL builds on.","marker":"[1]"},{"why":"The momentum federated learning (MFL) baseline that RMFL is compared against; supplies the momentum aggregation scheme that RMFL modifies.","marker":"[2]"},{"why":"The MNIST dataset used for evaluation.","marker":"[9]"},{"why":"The CIFAR10 and CIFAR100 datasets used for evaluation.","marker":"[10]"},{"why":"The LeNet-5 network used for the CIFAR10 experiments.","marker":"[11]"},{"why":"The ResNet20 network used for the CIFAR100 experiments.","marker":"[12]"}],"fun_headline_variants":["Reverse momentum weighting curbs federated data bias","Heterogeneous federated learning gains from reversed momentum","Flip momentum weighting to cut federated heterogeneity bias","Late local gradients less useful? Reverse momentum says yes","Reversing momentum order lifts federated accuracy on skewed data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the bias of local gradients grows monotonically with the number of local training steps, so that early gradients are always less biased than recent ones; if that curve is not monotonic, the reverse weighting has no principled reason to help.","fun_headline_variants_meta":{"raw":{"variants":["Reverse momentum weighting curbs federated data bias","Heterogeneous federated learning gains from reversed momentum","Flip momentum weighting to cut federated heterogeneity bias","Late local gradients less useful? Reverse momentum says yes","Reversing momentum order lifts federated accuracy on skewed data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000654,"raw_usage":{"total_tokens":2946,"prompt_tokens":847,"completion_tokens":2099,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":463,"completion_tokens_details":{"reasoning_tokens":2023}},"tokens_in":463,"tokens_out":2099,"duration_ms":12262,"temperature":1.0,"reasoning_tokens":2023,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:48:21.380184+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the average cosine similarity (or projection) between client gradients and the mean gradient at each local step in a heterogeneous federated task. If the similarity does not decrease monotonically with the local step — for instance if it dips and then recovers — the premise of RMFL fails. A second check: run RMFL on a task where later gradients are known to be more aligned with the global objective (e.g., synthetic noise added only to early steps); if RMFL still wins, its benefit does not come from the claimed mechanism.","supporting_citations":[{"cited_title":"Communication-efficient learning of deep networks from decentralized data,","cited_arxiv_id":null,"evidence_quote":"Defines the federated averaging setup (FedAvg) that forms the training loop RMFL builds on."}],"review_version":1}