{"id":"499b9eb6-02ef-4795-9ce9-2cdad6557614","arxiv_id":"2501.00170","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Selecting the local examples a model is least confident about, and training only part of the model, lets federated learning use far less client data and compute while keeping or improving accuracy.","lead":"This paper proposes FedFT-EDS, a federated learning method in which clients fine-tune only part of a pretrained model and train on the local samples the model is most uncertain about. The authors report that this reduces client workload and training time while matching or beating standard baselines on CIFAR-10, CIFAR-100, and a speech dataset.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Core entropy-selection gain is 0.4–3.8 points with no error bars or multiple seeds; without statistical evidence, the novelty over random selection is not established.","rationale":"I read the paper in good faith. The proposed combination of partial fine-tuning and per-sample entropy selection is clearly motivated, and the authors include the essential control, FedFT-RDS, which uses the same partial fine-tuning and random data selection. That control isolates the effect of the entropy criterion. The problem is that the isolated effect is small and statistically unverified. In Table II, FedFT-EDS beats FedFT-RDS by 0.4–2.7 points; in Table III (100 clients, 10% data) by 0.7–1.5; in Table III (50% data) by 0.8–1.5; in Table IV (GSC) by 0.6 at 10% and 3.8 at 50%. The largest gain is on GSC at 50%, but the 10% gain there is 0.64. Without multiple seeds or confidence intervals, these differences are within the noise expected for CIFAR-scale FL with Dirichlet-partitioned data. Furthermore, the temperature hyperparameter ρ=0.1 is tuned on CIFAR-100/α=0.1, the very benchmark used in the main tables, so the reported advantage may reflect overfitting to that configuration. A proper held-out validation or a sensitivity analysis across seeds is missing. I also note that the experimental section omits details for FedFT-ALL (Section IV-E) and includes MOON in Figure 6 without any description in the text, which hampers reproducibility but is secondary. These issues do not prove the method is wrong, but they make the central claim conditional. Therefore I maintain the reader's CONDITIONAL verdict, with the same primary concern.","tokens_in":16933,"tokens_out":6435,"duration_ms":61878,"concrete_test":"Run FedFT-EDS and FedFT-RDS for each configuration in Tables II–IV with at least 5 independent seeds (new Dirichlet partitions and model initializations), report mean±std of final test accuracy, and apply a paired Wilcoxon signed-rank test between the two methods. If the entropy-selection gain is not statistically significant (p<0.05) in a majority of settings, the central claim should be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's core contribution is the entropy-based data selection (FedFT-EDS vs FedFT-RDS). Across Tables II, III, IV the reported gains over random selection are 0.4–2.7, 0.7–1.5, and 0.6–3.8 percentage points respectively. These numbers come from single runs; no seeds, error bars, or statistical tests are reported. The temperature ρ=0.1 for the hardened softmax is selected from an ablation on exactly the CIFAR-100, α=0.1 setting used for evaluation, so the method is tuned on the benchmark it is judged on. Given typical run-to-run variance in non-IID FL, these small increments may not be significant. If the true effect is zero or negative, the central claim that entropy-based selection is beneficial collapses, leaving only the known benefit of partial fine-tuning. The comparison against FedAvg/FedProx is confounded by pretraining and partial fine-tuning, but the FedFT-RDS control is appropriate; the remaining load-bearing issue is statistical credibility of the entropy-selection gain.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FedFT-EDS, a federated learning method that combines fine-tuning of only the upper layers of a pretrained global model with per-round entropy-based local data selection. At each round, clients rank their local samples by Shannon entropy computed from a temperature-hardened softmax and train only on the highest-entropy subset; the server aggregates only the trainable upper part. Experiments on CIFAR-10, CIFAR-100, and Google Speech Commands report that FedFT-EDS outperforms FedAvg, FedProx, and random-selection controls while using 10–50% of local data, and that it roughly triples learning efficiency measured as test accuracy per unit of local training time. The paper also claims that entropy-selected 50% data can outperform training on all local data, supporting the thesis that not all client data is beneficial in FL.","tokens_in":17107,"tokens_out":7132,"duration_ms":72458,"significance":"If the reported results are reproducible, the paper makes a useful practical contribution: it demonstrates that a simple, cheap entropy-based selection rule combined with partial fine-tuning can reduce client workload without sacrificing global accuracy, and the construction of FedFT-RDS as a random-selection control is an appropriate way to isolate the effect of entropy selection. The CKA-based analysis of pretraining's effect on client model shift is a thoughtful addition, and the cross-domain speech experiment extends the evidence beyond image classification. The main weakness is statistical: the central entropy-selection gains over random selection are small (0.4–3.8 percentage points) and are reported from single runs without seeds, error bars, or significance tests, while the critical temperature hyperparameter is selected on the same benchmark used for evaluation. The paper does not provide code or reproducibility artifacts, which further limits verification.","major_comments":[{"comment":"The central claim that entropy-based selection outperforms random selection is supported only by single-run accuracy differences of 0.4–2.7 points in Table II and 0.6–3.8 points in Table III. No seeds, error bars, or statistical tests are reported, and these margins are small relative to typical run-to-run variance in non-IID federated learning. Please report multiple seeds with confidence intervals and a paired comparison (e.g., Wilcoxon signed-rank or bootstrap) across datasets and heterogeneity levels; without this, the core contribution over FedFT-RDS is not statistically established.","section":"IV-C, Tables II–III"},{"comment":"The hardened-softmax temperature rho=0.1 is selected from an ablation on CIFAR-100, which is also the benchmark used for evaluation, and the advantage over FedFT-RDS disappears or reverses for rho values greater than or equal to 1. Because the success of the method is contingent on this hyperparameter, the evaluation should either choose rho on a held-out validation split or demonstrate robustness across datasets and alpha values without per-benchmark tuning. As written, the main result may reflect tuning on the test benchmark rather than a general property of entropy selection.","section":"IV-H3, Fig. 10c"},{"comment":"The paper's headline insight that 'not all client data is beneficial for FL' rests on the comparison between FedFT-EDS (50%) and FedFT-ALL, where the observed advantage is 0.8–1.8 accuracy points in single runs without variance estimates. Please provide confidence intervals for this specific comparison, and ideally an analysis of why entropy selection removes harmful samples (e.g., label noise or distribution outliers), so that the claimed insight is supported beyond a single-run point estimate.","section":"IV-E, Table III"}],"minor_comments":[{"comment":"The abstract and conclusion emphasize using 50% of client data, while the main 10-client comparison in Table II and Figure 5 uses Pds=10%; please state explicitly which configuration is the headline result and clarify the role of each data-selection ratio.","section":"Abstract and Section IV-C"},{"comment":"The legends for the CIFAR-100 panels include 'MOON', but this baseline is not introduced in Section IV-A or anywhere else in the text; either describe MOON and cite it, or remove it from the figure.","section":"Fig. 6"},{"comment":"The text says the hardened softmax is 'parameterized by a temperature rho set to a value greater than 1', which is inconsistent with the following paragraph, where rho is set to a value smaller than 1, and with the experimental value rho=0.1; correct this contradiction.","section":"III-E, Eq. (6)"},{"comment":"The text contains typographical fragments such as 'FedAVG-' and 'FedA VG'; please proofread and fix these broken references to FedAvg.","section":"IV-C"},{"comment":"The pretraining phase is described only as pretraining on Small ImageNet 32x32; please specify the pretraining epochs, optimizer, learning rate, data augmentation, and architecture details, as these are needed for reproducibility.","section":"IV-A"},{"comment":"The number of communication rounds is not stated in the setup; Figures 5, 8, and 9 show 50 rounds, but this should be stated explicitly in the experimental protocol, along with the FedProx proximal term mu if it is used.","section":"IV-A"},{"comment":"For the cross-domain Google Speech Commands experiment, please specify the model architecture used for the speech task; it is not clear whether the same Wide ResNet is applied to 1-D audio inputs or whether a different architecture is used.","section":"IV-G"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely problem and the controlled comparison against FedFT-RDS is a good design choice. My main concern is statistical: the headline gains over random selection are small and come from single runs, and the critical temperature hyperparameter is selected on the evaluation benchmark. I recommend asking for repeated seeds, confidence intervals, and either validation-based selection of rho or a multi-benchmark robustness analysis. I would not reject on conceptual grounds; the partial-fine-tuning baseline is sensible and the CKA analysis is a useful addition."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper combines two known tricks—partial fine-tuning of a pretrained model and entropy-based sample selection—and the internal control against random selection (FedFT-RDS) is the right comparison. That control is the paper's real strength: it isolates the contribution of entropy selection from the larger confound of pretraining/partial fine-tuning. The sample-level entropy with hardened softmax is not exactly in the cited prior work, and the authors correctly distinguish it from FedAvg-BE's batch-level entropy. The CKA analysis of pretraining is a nice touch, and evaluating on speech in addition to CIFAR-10/100 adds breadth. The efficiency metric (test accuracy over local training time) is reasonable, and the finding that FedFT-EDS at 50% data beats FedFT-ALL at 100% is the kind of observation that makes the paper worth reading.\n\nThe soft spots are substantial but not fatal. First, there are no seeds or error bars anywhere. The entropy-selection gains over random selection are 0.4–3.8 points across tables; in non-IID FL, single-run differences of that size are exactly what run-to-run variance eats. Second, the temperature ρ=0.1 and the fine-tuning depth are chosen via ablation on CIFAR-100 with the same heterogeneity setting used for evaluation. That is tuning on the test benchmark. Third, the paper cites FedAvg-BE and FLRD but never compares against them, which is the minimal experiment needed to support the novelty claim. Fourth, the abstract says \"50% user data\" while Table II uses 10%, and the efficiency claim of \"up to 3 times\" conflates accuracy gains with data reduction.\n\nOne minor inconsistency: the ablation shows fine-tuning only the classifier is best, but the main experiments fine-tune from layer 3. The authors hedge this with a domain-similarity caveat, but it deserves a line in the main results.\n\nWho is this for? Anyone working on communication or computation efficient FL, and especially people building on pretrained models for edge training. It is an engineering contribution, not a new capability. It deserves a serious referee: the question is legitimate, the control is appropriate, and the gaps are fixable. But the central claim—entropy beats random—will not hold up until there are multi-seed runs, held-out hyperparameter selection, and direct comparison to the closest prior data-selection baselines. I would not cite it in its current form.","headline":"Entropy-based selection plus partial fine-tuning is a sensible combo, and the random-selection control is the right test, but the small single-run gains do not yet establish that entropy beats random.","tokens_in":17666,"tokens_out":1679,"would_cite":false,"duration_ms":17464,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that fine-tuning only a pretrained model's upper layers on the highest-entropy 10–50% of local data beats FedAvg and FedProx while using about one-third of the client training time.","keywords":["federated learning","entropy-based data selection","partial fine-tuning","hardened softmax","non-IID data","workload reduction","client efficiency","CIFAR-100"],"falsifier":"Run FedFT-EDS against FedFT-RDS on a benchmark with controlled label noise or outlier contamination, where high-entropy samples are known to include many mislabeled points; if the entropy-selection accuracy gain over random selection disappears or turns negative while the rest of the method is unchanged, the claim that entropy identifies the most beneficial data would be refuted.","tokens_in":16688,"feed_emoji":"⚡","tokens_out":5767,"duration_ms":53988,"temperature":0.7,"pith_summary":"The paper proposes FedFT-EDS, a federated learning method that combines two workload reductions: clients fine-tune only the upper layers of a pretrained model, and each round they train only on the local samples whose hardened-softmax prediction entropy is highest. On CIFAR-10, CIFAR-100, and a speech task under non-IID partitions, it reports higher global test accuracy than FedAvg and FedProx while using 10–50% of local data and roughly one-third of client training time. The paper's strongest claim is that selecting 50% of local data by entropy outperforms using all local data, concluding that not all client data is beneficial for federated learning at every round. This matters because it points to data selection, not just model compression, as a way to let resource-constrained devices participate.","feed_headline":"Entropy-picked local data beats full-data federated learning","feed_subtitle":"Clients use a tenth to half the data and a third of the training time, and still beat the standard baselines.","key_machinery":"The machinery has two coupled pieces. First, partial fine-tuning: the global model is pretrained on a source domain, the lower feature-extractor parameters $\\phi$ are frozen, and clients update only $\\theta$, the upper part of the model (in the experiments, layers from layer 3 upward, with the best results when only the classifier is trainable). Second, entropy-based data selection: at each round each client computes the softmax output $\\mathbf{p}$ for every local sample, applies a hardened softmax with temperature $\\rho = 0.1$ to sharpen the distribution, computes Shannon entropy $H = -\\sum_j p_j \\log p_j$, and keeps the samples with the highest entropy as the most informative. The temperature-hardening is the mechanism that makes selection work: it compresses confident predictions into very low entropy, so that small gains in confidence produce large drops in entropy and easy samples are filtered out. The server then aggregates only the updated upper parts using FedAvg-style weighting.","core_discovery":"On the paper's own terms, the central discovery is that federated learning can be made both more accurate and much cheaper by freezing a pretrained feature extractor and fine-tuning only a small trainable upper part on each client, while ranking local samples each round by Shannon entropy of a temperature-hardened softmax output and updating only on the highest-entropy subset. The paper reports that with 10% or 50% of client data, FedFT-EDS beats FedAvg and FedProx trained on all data, improves on random data selection by roughly 0.4 to 3.8 percentage points in its tables, and reaches higher test accuracy per unit of client training time. It also reports that selecting 50% of data by entropy outperforms training on 100% of local data, which the authors interpret as evidence that not all user data is equally beneficial in every round.","pith_inferences":["The entropy proxy is a heuristic tied to the current global model; a natural extension would be to validate it against gradient-based importance or per-sample loss reduction, which the paper does not do.","The benefit of partial fine-tuning likely depends on the pretraining source being close enough to the target that the frozen bottom features remain useful; on far-domain tasks, both the fine-tuning depth and the entropy ranking would need recalibration, a point the paper itself qualifies.","In deployment, the extra forward pass for entropy scoring costs time even though it reuses the training pass; a testable extension is to cache or refresh the scores every few rounds instead of every round.","The claim that not all data is beneficial concerns per-round usefulness, not permanent data value; a sample excluded in one round may become informative later, so an adaptive selection threshold or a small random exploration component could be explored."],"forward_implications":["Federated clients can cut local data to 10–50% and fine-tune only a fraction of the model without losing accuracy; FedFT-EDS reports top accuracy above FedAvg and FedProx in every tested setting.","Because only the upper-part parameters are communicated and updated, communication and client compute shrink together, easing the straggler problem and allowing a larger pool of clients to participate each round.","Entropy-based selection beats random selection at the same data budget, so the gain comes from which samples are chosen, not merely from training on fewer samples.","More data is not always better: with a pretrained model, FedFT-EDS at 50% selection beats FedFT-ALL at 100% data, suggesting that per-round data filtering can remove samples that would hurt or slow learning.","Learning efficiency, measured as test accuracy per unit of client training time, is roughly tripled on CIFAR-10 and improved even more on CIFAR-100.","The entropy ranking can be recomputed each round for free as part of the local forward pass, so the data-selection overhead is small relative to the training time saved."],"supporting_citations":[{"why":"Supplies the FedAvg aggregation rule and the standard baseline that FedFT-EDS extends and compares against.","marker":"[27]"},{"why":"Defines the FedProx proximal-term baseline that FedFT-EDS must beat in the main experiments.","marker":"[22]"},{"why":"Provides evidence on the impact of pretraining and initialization in federated learning, motivating the pretrained global model.","marker":"[31]"},{"why":"Introduces the temperature-scaled softmax for knowledge distillation, which the hardened softmax inverts for data selection.","marker":"[9]"},{"why":"The closest prior work using entropy for batched client data selection in FL, from which the paper distinguishes its sample-level selection.","marker":"[33]"},{"why":"Provides the CKA similarity metric used to show that pretraining reduces model shift between clients under data heterogeneity.","marker":"[15]"}],"fun_headline_variants":["Smarter data selection cuts FL workload by 90%","Fine-tune partial models, pick high-entropy data for FL","Federated learning with less data and time beats baselines","Entropy-guided data selection speeds up federated learning","Partial fine-tuning plus entropy selection boosts FL"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the current model's hardened-softmax prediction entropy marks the local samples whose training will most improve federated performance; if high-entropy samples are mostly mislabeled or outliers, or the temperature tuned on CIFAR-100 does not transfer to other tasks, the reported edge over random selection would vanish.","fun_headline_variants_meta":{"raw":{"variants":["Smarter data selection cuts FL workload by 90%","Fine-tune partial models, pick high-entropy data for FL","Federated learning with less data and time beats baselines","Entropy-guided data selection speeds up federated learning","Partial fine-tuning plus entropy selection boosts FL"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000239,"raw_usage":{"total_tokens":1534,"prompt_tokens":982,"completion_tokens":552,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":598,"completion_tokens_details":{"reasoning_tokens":472}},"tokens_in":598,"tokens_out":552,"duration_ms":5630,"temperature":1.0,"reasoning_tokens":472,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:57:27.821472+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run FedFT-EDS against FedFT-RDS on a benchmark with controlled label noise or outlier contamination, where high-entropy samples are known to include many mislabeled points; if the entropy-selection accuracy gain over random selection disappears or turns negative while the rest of the method is unchanged, the claim that entropy identifies the most beneficial data would be refuted.","supporting_citations":[{"cited_title":"Communication-efficient learning of deep networks from decentralized data,","cited_arxiv_id":null,"evidence_quote":"Supplies the FedAvg aggregation rule and the standard baseline that FedFT-EDS extends and compares against."},{"cited_title":"Entropy to mitigate non-iid data problem on federated learning for the edge intelligence environment,","cited_arxiv_id":null,"evidence_quote":"The closest prior work using entropy for batched client data selection in FL, from which the paper distinguishes its sample-level selection."}],"review_version":1}