{"id":"0b780071-7fa5-4531-b68f-6b7a759e1d1a","arxiv_id":"2507.20871","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"FedABC combines prediction-similarity scoring with loss-based client values and an increasing participation threshold, reporting higher accuracy with fewer clients on CIFAR-10.","lead":"A Huawei-TUM team proposes FedABC, a method for choosing which devices train a shared AI model in each round of federated learning. It aims to reach similar accuracy while using fewer device participations, a useful property for 6G networks with bandwidth and battery limits.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The attention score's contribution is not isolated: the reported gains over Cho et al. may stem solely from the hand-tuned cumulative threshold schedule and score-based aggregation, not from the proposed similarity-weighted value in Eq. (13).","rationale":"The reader identified the reliability of the server-side loss proxy and the 5,000-sample server dataset as the weakest assumptions. These are genuine, but the most load-bearing gap is causal attribution: the paper's central claim is that a specific algorithmic component, the attention-based similarity weighting, is responsible for the reported gains, yet the experimental design never varies that component while holding everything else fixed. Since three components change simultaneously relative to the Cho et al. baseline, the empirical comparison cannot support the stated conclusion. A simple ablation directly settles this. This is more decisive than questioning the loss proxy, because even if the proxy is imperfect, the attention term could still add value; conversely, if the attention term adds nothing, the central claim fails regardless of proxy quality. The reader also noted the internal contradiction about the threshold direction and the erroneous citation [18], both of which reinforce the need for a cleaner, reproducible evaluation. We agree only partially with the reader's weakest_assumption because our concern is about isolating the mechanism, not about the proxy's absolute validity. The verdict remains CONDITIONAL: the paper should be accepted only after the proposed ablation (and ideally multi-seed error bars) confirms the attention term's contribution over the threshold schedule and aggregation weighting.","tokens_in":8690,"tokens_out":5783,"duration_ms":68776,"concrete_test":"Run an ablation of FedABC on CIFAR-10 (α=0.1) that replaces the attention score in Sec. IV-B with the raw loss, i.e., set c_{k,k}=1 and c_{k,j}=0 for j≠k in Eq. (13) so S(θ_t_k)=v_t_k, while keeping the cumulative-threshold selection, the τ_t schedule (start 0.2, +0.1 every two rounds), and the score-based aggregation weights from Sec. V-A exactly as in FedABC. Compare the resulting accuracy and participation ratio against full FedABC and Cho et al. over at least 5 random Dirichlet partitions and training seeds. If the ablation matches full FedABC within noise, the attention term is not load-bearing and the central claim should be revised to attribute gains to the threshold schedule/aggregation; if FedABC is significantly better, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"FedABC introduces three simultaneous changes relative to the loss-based baseline Cho et al. [10]: (i) the score S(θ_t_k) in Eq. (13) replaces raw loss v_t_k with a similarity-weighted average of losses; (ii) selection is made by ranking scores and selecting until the cumulative score exceeds τ_t, where τ_t starts at 0.2 and increases by 0.1 every two rounds (Sec. V-A), rather than any rule derived from Eq. (14); (iii) aggregation weights in Eq. (15) are normalized scores instead of a uniform or data-size weighting. The evaluation in Fig. 2 compares only the full FedABC to Cho and FedAvg; there is no ablation that isolates (i). The '3.5% higher accuracy with 2% fewer clients' headline could therefore be caused entirely by the hand-specified cumulative-threshold schedule (ii) or the score-based aggregation (iii), with the attention weighting contributing nothing. The optimization derivation in Sec. IV-A is also not what is evaluated: Eq. (14) selects each client independently when S > λ/η_t, but the experiments replace this with a cumulative threshold and a manually increased τ_t, so the theoretical justification does not cover the implemented algorithm. The central claim that attention-based selection drives the improvement is thus unsupported unless the attention term is ablated.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FedABC, a client-selection algorithm for federated learning that combines an attention-style score, defined as a similarity-weighted average of per-client losses (Eqs. 10-13), with a cumulative threshold rule and score-based aggregation. The authors formulate an optimization problem (Eq. 7) with temporal weighting, introduce a threshold schedule that grows over time to increase participation in later rounds, and evaluate on CIFAR-10 with 10 clients against FedAvg and the loss-based method of Cho et al. The claimed results are comparable accuracy to FedAvg with about 32% fewer clients and about 3.5% higher accuracy than Cho et al. with 2% fewer clients.","tokens_in":8970,"tokens_out":3423,"duration_ms":40773,"significance":"If the empirical claims hold, FedABC would be a useful contribution to resource-efficient federated learning in wireless/6G settings, where reducing client participation while maintaining accuracy is practically important. The paper includes a welcome cost analysis and a comparison of different threshold growth designs, and the attention-style similarity weighting is a reasonable idea. However, the current evidence is insufficient to establish the central claim: the evaluation consists of single-run comparisons without error bars, no ablation isolates the attention component from the threshold and aggregation changes, and the implemented selection rule differs from the derived optimization rule. These issues must be addressed before the claimed improvements can be considered reliable.","major_comments":[{"comment":"The headline result, a 3.5% accuracy improvement over Cho et al. with 2% fewer clients, rests on a single simulation run with no seeds or error bars. CIFAR-10 training over 20 rounds is stochastic, and the reported gain may fall within run-to-run variation. Please report mean and confidence intervals over at least three to five independent runs for all settings and baselines.","section":"V-A, Fig. 2"},{"comment":"The optimization derivation does not match the evaluated algorithm. Eq. (14) yields an independent per-client decision rule, selecting client k when S(theta_t_k) > lambda/eta_t, but the experiments in Sec. V-A instead rank clients by S and select until the cumulative score exceeds a hand-specified threshold tau_t. Thus the theoretical justification in Sec. IV-A does not cover the implemented selection rule. Either derive the cumulative-threshold rule from the objective or evaluate the rule in Eq. (14).","section":"IV-A, IV-C, V-A"},{"comment":"There is an internal contradiction in the threshold schedule. Sec. V-A states that the threshold tau 'starts at 0.2 and increases by 0.1 every two global rounds,' while Sec. VII states that FedABC uses 'an adaptive client selection threshold, progressively lowering over time.' If tau increases, the cumulative-score condition becomes harder to satisfy, which should increase participation in later rounds; this matches the 'later-is-better' narrative but contradicts the conclusion's wording. Please correct the inconsistency.","section":"V-A vs. VII"},{"comment":"The claimed advantage of attention-based selection is not isolated. Relative to the loss-based baseline Cho et al., FedABC changes three things simultaneously: the score S in Eq. (13) replaces raw loss with a similarity-weighted value, the selection rule becomes a cumulative threshold with a manually scheduled tau_t, and aggregation in Eq. (15) uses normalized scores instead of uniform weights. No ablation separates these effects. Without an ablation that, for example, uses the raw loss v_t^k in place of S while keeping the threshold and aggregation fixed, the paper cannot support the claim that the attention weighting itself drives the improvement.","section":"IV-B, V-B"},{"comment":"The load-bearing assumption that v_t^k = L(D_k, theta_s^{t-1}) reliably measures a client's marginal contribution to the global model is inherited from Cho et al. and is not validated in this setting. Likewise, the KL-divergence similarities in Eqs. (10)-(11) are computed on a 5,000-sample unlabeled server dataset, but the paper does not assess how well these similarities capture true client distribution structure. Please include a sensitivity analysis over server dataset size and a comparison of the similarity metric against a ground-truth distribution-based similarity on a synthetic or semi-synthetic setup.","section":"IV-B"}],"minor_comments":[{"comment":"The claim of 'comparable performance with 32% fewer clients than FedAvg' should specify the accuracy level at which parity is assessed; the figures do not identify a single operating point.","section":"Abstract, V-B"},{"comment":"The aggregation formula has a rendering issue: '1P k wk' should be '1 / sum_k w_k'. Please correct the notation.","section":"Eq. (2)"},{"comment":"There is a typo: 'threhold' should be 'threshold'.","section":"V-A"},{"comment":"Reference [18] appears unrelated to the topic, as it concerns a paper on reasoning processes of large language models rather than federated learning or client selection. Please verify and replace it with an appropriate reference or remove it.","section":"References"},{"comment":"The three threshold designs (linear, concave, convex) are described only qualitatively. Please specify the exact functional forms and parameter values, including how the average participation ratio is held approximately equal across designs.","section":"Fig. 3"},{"comment":"The related-work section could mention recent client-selection methods based on reinforcement learning, diversity, or fairness, and the experimental comparison is limited to FedAvg and Cho et al.; adding at least one more recent baseline would strengthen the evaluation.","section":"VI"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's central contribution is plausible but the current evidence is not yet convincing. The absence of ablations and seeds, together with the mismatch between the derived and implemented selection rules, are the main blockers. The paper may also need a scope check: the evaluation is a small-scale CIFAR-10 simulation, while the framing targets 6G networks; the authors should be explicit about what conclusions generalize beyond this setup."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: FedABC is a reasonable incremental idea—weight client loss values by prediction-similarity, then select with a rising cumulative threshold—but the paper's claims outrun its evidence. The attention-weighting itself is never isolated, so the reported gains cannot be attributed to it.\n\nWhat's genuinely useful: the combination of Cho et al.'s loss-based value with ClusterFL-style prediction-similarity is natural and not present in prior work. The 'later-is-better' threshold schedule is a practical heuristic, and the cost analysis is honest and useful. The paper is clearly written and the related work is mostly relevant.\n\nThe soft spots are real. The evaluation is a single CIFAR-10 setup with no seeds or error bars; the 3.5% accuracy gain over Cho et al. could easily be noise. There is an internal contradiction: Sec. V-A says the threshold starts at 0.2 and increases by 0.1 every two rounds, while Sec. VII says it progressively lowers. The reference [18] to a paper about LLM reasoning seems wrong. More importantly, the optimization derivation in Eq. (14) selects clients independently when S > λ/η_t, but the experiments rank scores and select until the cumulative score exceeds τ_t. So the theory does not cover the implemented rule. And because the paper changes three things at once—the score, the threshold rule, and the aggregation weights—without an ablation, the headline gain cannot be attributed to the attention score.\n\nWho this is for: researchers working on client-selection heuristics for FL in edge/6G settings might want to test the combination themselves. It deserves a serious referee because the idea is plausible and the flaws are fixable. But as it stands, the central claim is unsupported.\n\nMy recommendation: send it to review, but require multiple seeds, error bars, an ablation isolating the attention term, and a corrected threshold description. Fix the wrong reference too.","headline":"Plausible incremental client-selection heuristic whose headline gains are not yet supported by its evidence—needs ablations and error bars.","tokens_in":9516,"tokens_out":2347,"would_cite":false,"duration_ms":24955,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a similarity-weighted loss score, combined with a rising selection threshold, lets federated learning train to comparable accuracy while selecting roughly a third fewer clients, and beats loss-only selection by about…","keywords":["federated learning","client selection","attention mechanism","data heterogeneity","6G networks","participation efficiency","CIFAR-10","Dirichlet distribution"],"falsifier":"Run FedABC and a loss-only baseline (Cho et al.) on a heterogeneous CIFAR-10 split, then remove the similarity weights by setting $c_{k,j}$ to a uniform distribution; if accuracy does not fall or participation does not rise, the compatibility term is not doing the claimed work. Additionally, shrink the server's unlabeled dataset from 5,000 samples to a few hundred; if FedABC's advantage over the loss-only baseline disappears, the KL-based similarity estimates are not robust.","tokens_in":8488,"feed_emoji":"📡","tokens_out":5732,"duration_ms":57851,"temperature":0.7,"pith_summary":"Federated learning typically asks every client to train every round, which wastes communication and computation when many clients bring redundant or uninformative data. FedABC proposes a client selection rule that scores each client by an attention-like combination of two signals: how much the global model still loses on that client's data (its unique contribution) and how similar its model predictions are to other clients (its compatibility). Clients whose scores clear a threshold participate, and the threshold rises over the training schedule so that later rounds involve more clients, following a 'later-is-better' principle. The paper claims this scheme reaches accuracy comparable to the full-participation FedAvg baseline while using about 32% fewer clients, and beats a loss-only selection baseline by about 3.5% accuracy while using slightly fewer clients. The result matters because client participation is the dominant cost in federated learning, especially in resource-constrained 6G edge networks.","feed_headline":"Attention-based selection trims FL clients by a third","feed_subtitle":"A similarity-weighted loss score picks the most valuable clients, holding accuracy while cutting participation.","key_machinery":"The central object is the attention score $S(\\theta^t_k)$ of Eq. (13), which fuses two per-client signals into one selection criterion. Compatibility scores $c_{k,j}$ are built by applying a softmax to negative KL divergences between clients' prediction vectors over the server's small dataset, so a client is 'compatible' with another when their models make similar predictions. Values $v^t_j$ are the server model's loss on each client's local data, taken as a proxy for how much that client's data has yet to be learned. The score is then a compatibility-weighted sum of values, and the selection rule is simply to take all clients whose score exceeds the round-dependent threshold $\\lambda/\\eta_t$. This threshold schedule is the long-term component: it keeps participation selective early and broadens it later, implementing the 'later-is-better' idea from prior work.","core_discovery":"The discovery is that a single attention score, defined as the similarity-weighted sum of per-client loss values, identifies the clients whose participation most improves the global model. Specifically, for each client k, the score in Eq. (13) is $S(\\theta^t_k) = \\sum_j c_{k,j} v^t_j$, where $c_{k,j}$ is the normalized exponential of the negative KL divergence between client k's and client j's model predictions on a small server-held dataset (Eqs. 10-11), and $v^t_j$ is the server model's loss on client j's data (Eq. 12). The server selects clients with $S(\\theta^t_k)$ above the threshold $\\lambda/\\eta_t$, where $\\eta_t$ grows with the round index to encourage more participation later in training. The paper claims this selection rule, together with the rising threshold, accelerates convergence and improves final accuracy compared with full participation and with loss-only selection, with the gap widening as data heterogeneity increases.","pith_inferences":["The similarity term can be read as a diversity regularizer on loss-based selection: clients that are highly similar to an already-selected high-loss client get down-weighted, which may prevent the server from repeatedly picking the same redundant clients. The paper does not isolate this effect experimentally.","The hand-set threshold schedule (0.2, plus 0.1 every two rounds) could be replaced by a budget-constrained rule derived from a target participation cost, which would make the method adaptable to changing network conditions.","The compatibility scores are computed on a 5,000-sample unlabeled server dataset; a natural sensitivity test is to vary this dataset's size or distribution, since the whole method depends on those predictions being representative.","The same attention-score construction could be applied to other client selection goals, such as fair participation or minimizing wall-clock time, by changing the value $v^t_j$ accordingly."],"forward_implications":["On CIFAR-10, FedABC reaches accuracy comparable to FedAvg while selecting an average of about 65% of clients, a saving of roughly 32% in participation.","Relative to the loss-only selection baseline of Cho et al., FedABC gains about 3.5% accuracy with about 2% fewer selected clients, and the advantage grows with data heterogeneity (smaller Dirichlet $\\alpha$).","Among tested threshold schedules, a convex one that rises faster in later rounds performs best at fixed average participation, supporting the 'later-is-better' principle.","The extra cost of selection is small: clients run only forward passes to report loss values (a few bytes), and the server performs cheap evaluations and multiplications.","The method slots into a standard three-step federated round (configuration, selection, report) without requiring additional global communication."],"supporting_citations":[{"why":"Defines FedAvg, the full-participation baseline that FedABC must match with fewer clients.","marker":"[2]"},{"why":"Supplies the loss-based value function $v^t_k$ and the theoretical argument that high server loss indicates a client's data is under-represented.","marker":"[10]"},{"why":"Provides the long-term 'later-is-better' principle that motivates the rising threshold schedule.","marker":"[15]"},{"why":"Contributes the method of comparing client models by KL divergence over predictions, used to build compatibility scores.","marker":"[19]"},{"why":"Gives empirical evidence that loss-based valuation of clients selects informative participants.","marker":"[20]"},{"why":"Introduces the attention mechanism that inspires the compatibility-score-plus-value score structure.","marker":"[13]"}],"fun_headline_variants":["Attention scores pick top FL clients, cut participation 32%","Long-term client selection lifts FL accuracy, slashes clients","Later-is-better rule trims federated clients by a third","Attention-weighted loss selects clients that matter in FL","Smart attention selection boosts FL with fewer clients"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The approach assumes that the server model's loss on a client's private data reliably measures how much that client's participation would improve the global model, and that prediction similarities computed on a small server-held dataset capture the true relatedness of client data distributions.","fun_headline_variants_meta":{"raw":{"variants":["Attention scores pick top FL clients, cut participation 32%","Long-term client selection lifts FL accuracy, slashes clients","Later-is-better rule trims federated clients by a third","Attention-weighted loss selects clients that matter in FL","Smart attention selection boosts FL with fewer clients"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001162,"raw_usage":{"total_tokens":4847,"prompt_tokens":1020,"completion_tokens":3827,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":636,"completion_tokens_details":{"reasoning_tokens":3748}},"tokens_in":636,"tokens_out":3827,"duration_ms":31351,"temperature":1.0,"reasoning_tokens":3748,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T13:09:46.809781+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run FedABC and a loss-only baseline (Cho et al.) on a heterogeneous CIFAR-10 split, then remove the similarity weights by setting $c_{k,j}$ to a uniform distribution; if accuracy does not fall or participation does not rise, the compatibility term is not doing the claimed work. Additionally, shrink the server's unlabeled dataset from 5,000 samples to a few hundred; if FedABC's advantage over the loss-only baseline disappears, the KL-based similarity estimates are not robust.","supporting_citations":[{"cited_title":"Communication-efficient learning of deep networks from decentralized data,","cited_arxiv_id":null,"evidence_quote":"Defines FedAvg, the full-participation baseline that FedABC must match with fewer clients."},{"cited_title":"Towards understanding biased client selection in federated learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the loss-based value function $v^t_k$ and the theoretical argument that high server loss indicates a client's data is under-represented."},{"cited_title":"Client selection and bandwidth allocation in wireless federated learning networks: A long-term perspective,","cited_arxiv_id":null,"evidence_quote":"Provides the long-term 'later-is-better' principle that motivates the rising threshold schedule."},{"cited_title":"ClusterFL: a similarity-aware federated learning system for human activity recogni- tion,","cited_arxiv_id":null,"evidence_quote":"Contributes the method of comparing client models by KL divergence over predictions, used to build compatibility scores."},{"cited_title":"Attention is all you need,","cited_arxiv_id":null,"evidence_quote":"Introduces the attention mechanism that inspires the compatibility-score-plus-value score structure."}],"review_version":1}