{"id":"05df88f0-eb36-4f22-a64d-1e0ca5d6ff9b","arxiv_id":"2608.07007","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A federated server that weights client updates by the inverse of each model's validation loss on a small proxy dataset outperforms data-size weighting, but its convergence proof assumes the weight concentration that the method is supposed to provide.","lead":"This paper proposes FedLBW, a federated learning aggregation rule that weights each client's update by the inverse of its validation loss on a small server-held proxy dataset instead of by dataset size. The authors report accuracy gains up to 7.6% over FedAvg in extreme non-IID settings and stable performance under client dropouts.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 4.1's proof is internally inconsistent: Lemma 4.2 drops the nonzero bias between uniform client sampling and the data-size-weighted global objective, so the convergence guarantee does not follow for the paper's own setup.","rationale":"The reader's conditional verdict is appropriate, and I found a concrete mathematical gap in Lemma 4.2 that matches the reader's weakest assumption. The empirical results are broad, with consistent but modest gains across datasets, architectures, non-IID levels, dropout rates, participation ratios, and proxy-shift ablations; the largest reported gain is plausible. The proof issue is internal rather than a disagreement with consensus: the stated theorem does not follow from the stated assumptions when client data sizes are unequal, which is precisely the setting of Table 1. This weakens the theoretical contribution but does not by itself overturn the empirical headline, which is supported by the experiments and by the proxy-shift robustness table. I would keep the conditional verdict: accept the empirical claim pending a corrected proof or explicit removal of the convergence claim, clarification of the proxy-ranking assumption, and release of code or data artifacts for independent verification. I do not see grounds to reject, because the main empirical finding does not depend on the theorem.","tokens_in":21428,"tokens_out":7459,"duration_ms":82889,"concrete_test":"Re-derive Lemma 4.2 in the uniform-beta case and verify whether the identity contains the extra term || (1/C) sum_c grad F_c(w) - sum_c (n_c/N) grad F_c(w) ||^2. Then evaluate this term on the actual partitions from Table 1 (for example CIFAR-10, alpha=0.1) at initialization and at a few training checkpoints using the released or recreated data split. If the term is nonzero at the order of zeta^2, Theorem 4.1's bound cannot hold as stated; the proof would need either equal client sizes, sampling proportional to n_c, or a new assumption bounding this size-driven bias.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The convergence proof is the load-bearing support for the paper's claimed theoretical contribution, and Lemma 4.2 does not follow from the stated assumptions when client data sizes are unequal. Equation (2) defines the global objective F with data-size weights n_c/N. Assumption 4 samples clients uniformly, and Eq. (4) aggregates with loss-based weights that ignore n_c. Lemma 4.2 bounds u_t = sum_{c in S_t} beta_c grad F_c(w_t) - grad F(w_t) by a sampling-variance term plus a term depending only on the spread of the beta's. In the special case beta uniform, u_t equals [sample-mean gradient - (1/C) sum_c grad F_c] + [(1/C) sum_c grad F_c - sum_c (n_c/N) grad F_c]. The first bracket has zero mean; the second bracket is a deterministic bias that is nonzero whenever client sizes differ. Table 1 shows n_c ranges from 14 to 1409 on CIFAR-10 at alpha=0.1, exactly the regime where the paper claims its largest gains. Lemma 4.2 has no term controlling this second bracket, so the stated bound is false and Theorem 4.1's convergence to a stationary point of F does not follow. Assumption 5(b) does not repair the gap because it only constrains the beta weights, not the mismatch between unweighted and data-size-weighted gradient averages. The empirical convergence plots may stand on their own, but the analytical claim as written is internally inconsistent with the problem setup.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces FedLBW, a federated aggregation rule that replaces dataset-size weights with weights inversely proportional to each client model's validation loss on a server-held proxy set. The authors motivate the method for wireless settings with non-IID data and dropouts, provide a convergence analysis in Section 4, and report experiments on FashionMNIST, CIFAR-10, and CIFAR-100 showing 1.5-7.7 percentage point accuracy gains over FedAvg and consistent gains over FedLAW, FedDkw, and other baselines, plus a dropout-robustness study and a proxy-shift sensitivity study.","tokens_in":21672,"tokens_out":7272,"duration_ms":75159,"significance":"The empirical study is broad and the core idea is simple and plausible: if the proxy ranking is informative, inverse-loss weighting can suppress poorly generalizing clients. The paper's strengths are the range of datasets, the three-seed repetitions, the comparison against six baselines, and the honest proxy-robustness experiments in Table 5. However, the theoretical contribution is not sound in its current form: the main convergence bound relies on a biased-sampling step and on unverified concentration assumptions, and the theorem's objective does not match the algorithm's actual round-dependent weighting. The empirical claims may survive a revision, but the paper's advertised theoretical guarantee should not be taken as established.","major_comments":[{"comment":"Lemma 4.2 does not follow from Assumptions 2-4 and Eq. (4). With uniform client sampling and loss-based weights beta_t, the conditional expectation of u_t = sum_{c in S_t} beta_c grad F_c(w_t) - grad F(w_t) contains a deterministic bias. In the special case beta_c = 1/|S_t|, E[u_t | w_t] = (1/C) sum_c grad F_c(w_t) - sum_c (n_c/N) grad F_c(w_t), which is not zero unless client sizes are equal or sampling is proportional to n_c. Table 1 shows n_c ranges from 14 to 1409 on CIFAR-10 at alpha=0.1, exactly the regime highlighted in the paper. The first term in the bound controls only the variance around the unweighted client mean; the second term is a pure weighting-bias term unrelated to the n_c mismatch. Therefore the stated bound is false, and Theorem 4.1's convergence to a stationary point of F in Eq. (2) is not established for the setup defined in Section 3.2. Repairing this requires either sampling clients with probability proportional to n_c or proving a new bound that explicitly controls the weighted-vs-unweighted gradient mismatch.","section":"Section 4.3, Lemma 4.2"},{"comment":"Assumption 5 is doing the entire work of the claimed theoretical advantage, but it is neither derived from the proxy-loss mechanism nor empirically verified. The theorem's method-specific bias term is G^2(rho-1), and Corollary 4.2 asserts FedLBW achieves smaller rho than FedAvg provided Lmin/Lmax >= delta. No result in the paper establishes this loss-ratio condition, and Section 6.4 shows the proxy ranking can be weakly aligned with client quality (Spearman rho=0.4219 for SVHN), so the condition cannot be taken for granted. The authors should either derive delta (or rho) from properties of the validation loss and proxy set, or report measured values of rho and B_max in the experiments; otherwise the comparison with FedAvg is circular.","section":"Section 4.2, Assumption 5; Section 4.4, Corollary 4.2"},{"comment":"The convergence analysis studies the static data-size-weighted objective F in Eq. (2), but Algorithm 1 implements the round-dependent objective in Eq. (5), whose coefficients beta_t depend on the current validation losses. The proof bounds descent on F while an update with beta_t != n_c/N is applied; even if Lemma 4.2 were repaired, the theorem would still be about a different optimization problem. The paper should state explicitly which objective FedLBW is claimed to converge to, and the analysis should handle the time-varying nature of the loss-based weights.","section":"Section 4.4, Theorem 4.1 vs Section 3.2, Eq. (5)"}],"minor_comments":[{"comment":"Section 3.3 refers to 'the aggregation step line 10', but in Algorithm 1 the aggregation happens on line 9 and the ClientUpdate function starts at line 11; please correct the cross-reference.","section":"Section 3.3, Algorithm 1"},{"comment":"The y-axis labels read 'Accuracy (%)' while the tick values are 0.0-0.9; please make the units consistent by either plotting percentages (e.g., 20-80) or changing the label to 'Accuracy'.","section":"Figures 2-4"},{"comment":"The symbol rho is used both for the Spearman rank correlation coefficient and for the weight concentration factor in Section 4.1; please use a different symbol for one of these quantities.","section":"Section 6.4, Table 5"},{"comment":"The 'Training Samples per Client' statistics are useful, but the header layout and the 'Data Distribution' entries make the table hard to parse; consider a clearer multi-row layout with a separate row for each dataset and alpha value.","section":"Table 1"},{"comment":"The reported gains over FedAvg are arithmetic differences of means; given the reported standard deviations (e.g., 58.63 +/- 3.02 vs 50.97 +/- 3.55 at CIFAR-10 alpha=0.1), a significance test or a statement about overlap would strengthen the comparison.","section":"Section 6.1, Table 2"},{"comment":"The complexity O(|V||S_t|) for the server-side loss computation should be stated as a per-round cost and should clarify that it assumes one forward pass per client model over the proxy set; otherwise it may be read as an end-to-end training cost.","section":"Section 3.5"}],"recommendation":"major_revision","confidential_remarks":"The advertised convergence guarantee is not valid as written, but the empirical comparison is substantial and the method is plausible. I would support a major revision that either removes or substantially rewrites the theoretical section, or repairs Lemma 4.2 and verifies Assumption 5 empirically before the theory is presented as a contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: FedLBW is a simple, plausible aggregation rule—inverse validation loss on a server proxy instead of data-size weighting—and the empirical case for it is broad and honest. The theory, though, has a load-bearing gap. Lemma 4.2 silently assumes client data sizes are equal (or that sampling is proportional to n_c) when the paper's own Eq. (2) uses data-size weights and Assumption 4 samples clients uniformly. With client sizes ranging from 14 to 1409 in Table 1, the bias term between the unweighted client average and the data-size-weighted global objective is nonzero, and the stated bound is false. This is not a minor typo; Theorem 4.1 does not follow as written. The fix is straightforward—analyze the unweighted global objective, sample clients with probability n_c/N, or carry the extra bias term—but it needs to be done.\n\nWhat is actually new: the specific inverse-loss weighting rule in Eq. (4). It is a natural special case of the proxy-based aggregation paradigm in FedLAW, and the paper acknowledges that lineage, but this direct formula is a cheap baseline worth having. The empirical work is the real contribution: three datasets, three architectures, multiple non-IID levels, dropout up to 0.5, varying participation ratios and client counts, plus a proxy-shift ablation. The gains are modest but consistent, and the largest CIFAR-10 gain (7.6 points at alpha=0.1) is plausible. Table 5 is a point in the authors' favor: even with an SVHN proxy that badly ranks clients (Spearman 0.42), accuracy drops only about two points, suggesting the method is robust to poor ranking rather than secretly dependent on it. That softens the proxy-alignment concern.\n\nSoft spots in order of severity: (1) the convergence proof gap above; (2) Assumption 5(b) essentially assumes the weight concentration the method is supposed to deliver—acceptable as an assumption for analysis, but Corollary 4.2 then credits the method for something assumed rather than shown; (3) no code released, so the numbers cannot be independently checked.\n\nThis paper is for people building practical FL systems in wireless settings, and for anyone benchmarking server-side proxy aggregation. It deserves a serious referee, conditional on the proof being repaired and code being released. I would not desk-reject it.","headline":"A simple, honestly evaluated proxy-based aggregation rule with a real gap in its convergence proof—worth refereeing once the sampling argument is fixed.","tokens_in":22264,"tokens_out":2737,"would_cite":true,"duration_ms":28696,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FedLBW claims that weighting client updates by inverse validation loss on a small server proxy set, instead of by dataset size, raises accuracy and convergence under non-IID data and client dropout.","keywords":["Federated learning","Non-IID data","Loss-based aggregation","Server proxy dataset","Client dropout","Wireless networks","Convergence analysis"],"falsifier":"Compute the Spearman correlation between client rankings induced by proxy validation loss and rankings induced by true held-out accuracy across many rounds; if that correlation is near zero or negative, FedLBW's weights systematically favor worse models, and the experiment should show FedLBW matching or falling below FedAvg. Table 5 already shows the correlation dropping to 0.42 for an SVHN proxy, so the sharp check is a stronger proxy mismatch where the ranking is reversed and FedLBW's accuracy falls below its data-size-weighted baseline.","tokens_in":21128,"feed_emoji":"📡","tokens_out":11480,"duration_ms":98439,"temperature":0.7,"pith_summary":"This paper argues that in federated learning the standard practice of weighting client updates by dataset size is the wrong signal under non-IID data and wireless dropouts. It proposes FedLBW, which weights each returned model by the inverse of its validation loss on a small, class-balanced proxy dataset held by the server, so low-loss models get more influence. The authors claim this substitution yields higher accuracy and faster convergence, with up to 7.6% higher accuracy on CIFAR-10 at the most skewed Dirichlet setting, and keeps accuracy stable when up to half of clients drop out. The reason to care is that the scheme replaces an unavailable global data distribution with a cheap, server-side quality signal during aggregation.","feed_headline":"Loss-based weighting lifts federated learning accuracy up to 7.6%","feed_subtitle":"The server uses a small proxy set to down-weight high-loss clients, making aggregation stable under non-IID data and client dropout.","key_machinery":"The load-bearing mechanism is the inverse-loss weighting rule at Eq. (4): replace the data-size coefficients $n_c/\\sum_j n_j$ in FedAvg with coefficients $\\beta_c = (1/L_c^{t+1})/\\sum_{j \\in S_t} (1/L_j^{t+1})$. Here $L_c^{t+1}$ is the average cross-entropy loss of client $c$'s updated model on the server's small class-balanced proxy set $V$. The inversion converts lower validation loss into higher aggregation weight while keeping the update a convex combination, and the proxy set lets the server evaluate every participating client on a common fixed reference distribution. The convergence analysis leans on Assumption 5, which bounds how far these loss-based weights deviate from uniform and how concentrated their squared values are, because that concentration factor $\\rho$ controls both the stochastic-noise term and the weighting-bias term in Theorem 4.1.","core_discovery":"The central claim is that the aggregation step in federated learning should weight client updates by the inverse of each client model's validation loss on a small server-side proxy dataset, rather than by the number of local samples. Formally, the global update is $w^{t+1} = \\sum_{c \\in S_t} \\beta_c w_c^{t+1}$ with $\\beta_c \\propto 1/L_c^{t+1}$ and $L_c^{t+1}$ the average cross-entropy loss of client $c$'s returned model on the proxy set. The paper reports that FedLBW outperforms FedAvg, FedAvgM, FedProx, FedNova, FedLAW, and FedDkw across FashionMNIST, CIFAR-10, and CIFAR-100, with the largest gains at extreme non-IID skew, and that it retains most of its accuracy under client dropout while FedAvg degrades sharply. It also contributes a nonconvex convergence bound that separates optimization error, stochastic noise, client heterogeneity, and loss-weight concentration, and argues that bounded loss ratios give FedLBW a smaller concentration factor, and thus less bias, than FedAvg.","pith_inferences":["Editorial inference: FedLBW effectively turns the server's proxy dataset into a learned similarity measure, so the method's ceiling is set by proxy representativeness; one natural extension is to update or diversify the proxy online when client data drift is detected.","Editorial inference: the inverse-loss rule is a deterministic performance-weighted ensemble, so combining it with client sampling that is proportional to $\\beta_c$ rather than to $n_c$ would directly align the partial-participation analysis with the algorithm actually run.","Editorial inference: a testable extension is temporal smoothing or clipping of the loss-based weights, which the paper mentions only as an enforcement mechanism for the theory but does not evaluate as a practical stabilizer."],"forward_implications":["In non-IID federated learning, switching aggregation weights from dataset size to inverse validation loss should improve final global accuracy, with the largest measured gain of 7.6% over FedAvg on CIFAR-10 at Dirichlet $\\alpha=0.1$.","Convergence should be faster in early rounds because the aggregation direction is dominated by low-loss clients; the reported convergence curves show smoother and steeper accuracy increases in the first rounds.","Under client dropout, FedLBW should stay near its no-dropout accuracy because surviving reliable clients keep high weight; at a dropout probability of 0.5 on CIFAR-10 it preserves about 90% of its baseline performance while FedAvg loses about 28%.","The convergence bound implies that the method's advantage is governed by the weight concentration factor $\\rho$: with tightly bounded validation-loss ratios, FedLBW's $\\rho$ can be smaller than FedAvg's, reducing the bias term $G^2(\\rho-1)$."],"supporting_citations":[{"why":"Defines FedAvg, the baseline algorithm whose data-size weighting FedLBW replaces.","marker":"[5]"},{"why":"FedLAW, the closest proxy-based weighted aggregation baseline; supplies the practice of using a small server proxy set for weighting.","marker":"[10]"},{"why":"Cited to support the premise that proxy datasets are already available in many federated learning systems.","marker":"[11]"},{"why":"FedProx baseline for heterogeneity; its proximal-term approach is compared against loss-based weighting.","marker":"[16]"},{"why":"SCAFFOLD; the convergence proof borrows standard assumptions and control-variate ideas from this line of work.","marker":"[23]"},{"why":"Provides the nonconvex FedAvg convergence framework that the FedLBW theorem extends with loss-based weights.","marker":"[33]"},{"why":"FedDkw, a dynamic KL-divergence weighting baseline that FedLBW claims to outperform.","marker":"[35]"}],"fun_headline_variants":["Loss-based weighting outperforms dataset-size weighting in FL","Inverse-loss weighting strengthens FL against non-IID and dropout","Steer federated aggregation by loss, not sample counts","FedLBW: loss-based weights keep FL accurate under dropouts","Weight clients by inverse loss to boost FL accuracy and resilience"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that validation losses computed on the server's small proxy dataset rank clients by their true generalization quality; if the proxy's ranking is wrong, FedLBW will up-weight the wrong clients and its advantage over FedAvg collapses.","fun_headline_variants_meta":{"raw":{"variants":["Loss-based weighting outperforms dataset-size weighting in FL","Inverse-loss weighting strengthens FL against non-IID and dropout","Steer federated aggregation by loss, not sample counts","FedLBW: loss-based weights keep FL accurate under dropouts","Weight clients by inverse loss to boost FL accuracy and resilience"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000777,"raw_usage":{"total_tokens":3524,"prompt_tokens":1124,"completion_tokens":2400,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":740,"completion_tokens_details":{"reasoning_tokens":2317}},"tokens_in":740,"tokens_out":2400,"duration_ms":18367,"temperature":1.0,"reasoning_tokens":2317,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T16:42:37.700821+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the Spearman correlation between client rankings induced by proxy validation loss and rankings induced by true held-out accuracy across many rounds; if that correlation is near zero or negative, FedLBW's weights systematically favor worse models, and the experiment should show FedLBW matching or falling below FedAvg. Table 5 already shows the correlation dropping to 0.42 for an SVHN proxy, so the sharp check is a stronger proxy mismatch where the ranking is reversed and FedLBW's accuracy falls below its data-size-weighted baseline.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"FedLAW, the closest proxy-based weighted aggregation baseline; supplies the practice of using a small server proxy set for weighting."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Cited to support the premise that proxy datasets are already available in many federated learning systems."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"FedProx baseline for heterogeneity; its proximal-term approach is compared against loss-based weighting."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"FedDkw, a dynamic KL-divergence weighting baseline that FedLBW claims to outperform."}],"review_version":1}