{"id":"093fc3d1-9ef6-4381-971a-f2776f3d3ca5","arxiv_id":"2506.06337","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A DRL-based per-class data subset selection with post-federation fine-tuning is claimed to improve personalized federated learning, but the experiments and the theoretical bound do not substantiate the claim.","lead":"This paper proposes a reinforcement learning agent that decides how much of each class of a client's local data to use in federated learning, then fine-tunes that client on all its data afterward. The authors report accuracy gains, but they omit a control that would show whether the gains come from the data selection or just from the extra local training.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Post-FL fine-tuning on the complete local dataset is not controlled in the naive baseline, so the claimed superiority in Table I cannot be attributed to DRL data selection.","rationale":"The reader's weakest_assumption exactly identifies the missing control for post-FL fine-tuning. This is the load-bearing issue for the main empirical claim. The paper's own ablation study suggests the DRL-based partition actually hurts accuracy during the FL phase, which further undermines the attribution. The theoretical bound in Theorem 1 is a definitional identity about circle areas, not a performance bound, so it provides no independent support. A single controlled experiment with a fine-tuned naive baseline would settle the question; until then, the evidence is insufficient for the claimed superiority. The verdict should remain REJECT, as the core claim is not supported by the presented comparisons.","tokens_in":13849,"tokens_out":938,"duration_ms":10252,"concrete_test":"Rerun the FedAvg and FedMedian experiments with a 'fine-tuned naive' control: after the 100 FL rounds, apply exactly the same post-FL fine-tuning procedure (full local dataset, same epochs, same convergence criterion) to a naive client, and compare its validation accuracy against the optimized client. If the fine-tuned naive client matches or exceeds the optimized client's accuracy, the DRL selection is not the cause of the reported gains. Additionally, report the optimized client's validation accuracy immediately after round 100 (before fine-tuning) in Table I.","verdict_should_be":"REJECT","load_bearing_attack":"The central empirical claim is that the DRL-based data partition improves client performance. Table I compares each baseline aggregation algorithm with and without 'Our Method,' but the optimized client always receives an additional phase of local training on the full dataset Dk after the FL rounds (Algorithm 1 lines 13-18, Section V-C), while the naive clients do not. The table caption states the optimized client's numbers are taken 'after training on the complete local dataset, Dk, after the FL rounds.' Therefore the reported gains (e.g., FedAvg accuracy 43.01% vs. 26.64%) conflate two interventions: (1) DRL-driven per-class subsampling during FL and (2) extra fine-tuning on all local data. The ablation in Table II is consistent with this confound: during FL, the optimized client has lower accuracy than the naive baseline (e.g., FedAvg accuracy 26.91% vs. 34.98%), and the main-table advantage appears only after the additional training phase. Without a control that performs the same post-FL fine-tuning without DRL selection, or without reporting the optimized client's accuracy immediately after round 100, the headline claim that the DRL partition itself yields superior performance is not established. The paper itself acknowledges lower performance during FL rounds (Section V-C, Figure 4), yet the abstract and conclusion claim 'superior performance' without qualifying that this occurs only after an extra, non-baseline training stage.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a DRL-based per-class data subsampling scheme for one \"optimized\" client in federated learning. The agent observes the client's F1-score after aggregation and outputs per-class weights that define how much of the local dataset is used in the next local-training round; the reward is a function of loss reduction and data usage. After the FL rounds, the optimized client fine-tunes on its full local dataset. Experiments compare the optimized client against naive clients on CIFAR-10, CIFAR-100, and FashionMNIST under five aggregation algorithms, and the paper presents a theoretical \"performance upper bound\" for the optimized client during FL.","tokens_in":14187,"tokens_out":4657,"duration_ms":46005,"significance":"If the claimed gains were attributable to the DRL data partition, the method would be a useful client-side personalization technique for non-IID federated learning. Strengths include the breadth of aggregation baselines, the availability of code, and a clearly described client-side algorithm. However, the main empirical comparison is confounded by an uncontrolled post-FL fine-tuning phase, and the theoretical analysis is a tautology. As presented, the results do not establish that the DRL policy provides any benefit, and the evidence in the ablation study points in the opposite direction during the FL rounds.","major_comments":[{"comment":"The headline comparison is confounded. Table I reports the optimized client's accuracy \"after training on the complete local dataset, Dk, after the FL rounds,\" and Algorithm 1 lines 13-18 implement this extra fine-tuning phase. Naive clients do not receive an equivalent post-FL fine-tuning phase on their full local datasets. Therefore the reported gains (e.g., FedAvg accuracy 43.01% vs. 26.64%) conflate two interventions: the DRL-driven per-class data selection during FL and an additional local training stage on all local data. The ablation study in Table II makes the confound clear: during the FL rounds, the optimized client has lower accuracy than the baseline (FedAvg 26.91% vs. 34.98%). Without a control in which naive clients receive the same post-FL fine-tuning, or a report of the optimized client's accuracy immediately after round 100 without fine-tuning, the claim that the DRL partition itself yields superior performance is not established.","section":"Section V-C, Table I, Figure 4, Algorithm 1 lines 13-18"},{"comment":"Theorem 1 is a tautology and provides no predictive content. Equations (10)-(12) define P_k and P'_k as sums of circle areas πZ_c^2 and πz_c^2, and Ω is defined as πΣ(Z_c^2 − z_c^2). The proof then derives P_k − P'_k = Ω by algebraic rearrangement. This is true by definition, not by any property of learning, loss, or generalization. The bound therefore says nothing about accuracy, loss, or any actual performance measure, and it does not support the paper's claim of an upper bound on client performance. The authors should either connect the circle-area quantity to a real performance metric through a nontrivial argument or remove the theorem and present the calculation as what it is: a definitional identity.","section":"Section IV-B, Theorem 1"},{"comment":"The paper's own results contradict the unqualified claim of superior performance. Figure 4 and Table II show lower accuracy for the optimized client during the FL rounds under every aggregation algorithm, and the Discussion concedes that \"training on a smaller partition of the local data can cause the optimized client to marginally lag in performance relative to the naive clients during the FL rounds.\" The abstract and conclusion nevertheless state that the method yields \"superior performance\" without the crucial qualification that this occurs only after an additional post-FL fine-tuning phase that is not applied to the baselines. This is a reporting issue that goes beyond style: it obscures the fact that the method's only demonstrated advantage comes from a stage that is absent from the comparison.","section":"Section V-C, Figure 4, Table II, and the Abstract/Conclusion"}],"minor_comments":[{"comment":"Equation (5a) uses max(a'_c, |D_kc|)/|D_kc|, which cannot implement the stated goal of capping the selected count at the available class size; for any a'_c < |D_kc| the expression evaluates to 1, and for a'_c > |D_kc| it is also 1. The formula should use min(a'_c, |D_kc|)/|D_kc|.","section":"Section IV-A, Equation (5a)"},{"comment":"The condition in the reward function is written as \"if T < τ,\" but T denotes the total number of FL rounds, not the current round index t; the intended condition is presumably \"if t < τ.\"","section":"Section IV, Equation (9a)"},{"comment":"In the FashionMNIST block, the FedCDA recall entry is printed as \"57.91±4.59\" without a percent sign, unlike all other entries; this is likely a typographical error.","section":"Table I"},{"comment":"The parameter line \"C ∈ (0, 1) 7→ R\" is notationally nonsensical; it should be something like \"C ∈ (0, 1)\" for the client fraction. Also, \"meausred\" in Section II-B is a typo.","section":"Algorithm 1 and Section II-B"},{"comment":"The caption refers to a \"dark green line\" and \"blue line,\" but it would be clearer to use the legend labels directly (\"Optimized after FL,\" \"Optimized during FL\") given the multiple subplots.","section":"Figure 4 caption"}],"recommendation":"reject","confidential_remarks":"The paper is already accepted at IJCNN, and the editorial office may wish to consider whether the confounded comparison and the vacuous theorem were noted in the conference review. The code release is commendable, but the current manuscript does not provide a valid empirical or theoretical basis for its central claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The two things you need to know: the reader's take holds up, and the stress-test concern lands directly on the paper's own text. The central claim — that DRL-driven per-class data selection gives the optimized client superior performance — is not supported, because the optimized client's numbers in Table I are taken after an extra phase of fine-tuning on its full local dataset (Algorithm 1, lines 13–18), while the naive clients get no such phase. The table caption says exactly this. So the reported gains (FedAvg 43.01% vs 26.64%) bundle two interventions, and the ablation in Table II shows that during FL the optimized client is actually worse (FedAvg 26.91% vs 34.98%). The paper itself admits this in Section V-C and Figure 4; the abstract's \"superior performance\" claim just skips the qualifier that it only holds after the extra, non-baseline training stage.\n\nWhat is genuinely new: the specific combination — DDPG choosing per-class data weights during FL rounds, followed by full-data fine-tuning — is not in the cited literature. The algorithm is clearly specified, code is provided, and the experiments span five aggregation methods and three datasets. That is real work, and reporting the during-FL drop rather than hiding it earns some credit. A practitioner might plausibly use this as a personalization heuristic.\n\nWhere it falls apart: the confound is load-bearing, not cosmetic. There is no control client that fine-tunes on Dk without the DRL policy, and no measurement of the optimized client's accuracy right after round 100, before fine-tuning. Theorem 1 is a tautology: client performance is defined as π times the sum of squared per-class radii, so Pk − P'k equals Ω by construction. The proof bounds nothing about accuracy or loss. Minor issues: the reward has a pile of free parameters (u, v, λ, τ, η, ε, bl, bu), and 8 clients with 1 local epoch is a thin experimental base.\n\nWho this is for: anyone who wants a clear example of how a missing control can sink an otherwise reasonable ML idea. It is not a paper to cite for the claimed performance.\n\nRecommendation: send it to peer review — a serious referee would demand the fine-tuning control, and the paper is fixable. In current form the evidence does not support the headline claim. If the control shows the gain is mostly fine-tuning, what remains is a modest data-pruning heuristic with privacy/communication appeal, which is worth testing but not a strong result.","headline":"A reasonable RL-for-FL idea is undermined in its current form by a confounded main comparison and a tautological theorem; the claims need a fine-tuning control before they can be taken seriously.","tokens_in":14639,"tokens_out":7251,"would_cite":false,"duration_ms":62146,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A deep reinforcement learning agent that chooses per-class data proportions during federated rounds, followed by fine-tuning on the full local dataset, can make a client outperform naive federated clients on multiple benchmarks and…","keywords":["federated learning","deep reinforcement learning","data heterogeneity","non-IID data","client personalization","data selection","local updates","DDPG"],"falsifier":"Run the same federated setup with three clients: an optimized client, a naive client that trains on all local data throughout, and a control client that trains on all local data during FL and then performs the same post-FL full-data fine-tuning without any DRL agent. If the control client's final accuracy matches or beats the optimized client's, then the DRL data selection is not the source of the reported gain.","tokens_in":13677,"feed_emoji":"🤖","tokens_out":6480,"duration_ms":59228,"temperature":0.7,"pith_summary":"Federated learning trains a shared model across clients without centralizing raw data, but non-IID local datasets can make aggregation hurt individual clients. This paper argues that, for a given client, training locally on more data than necessary does not help and can even hurt, so the client should let a deep reinforcement learning agent pick, each round, how much of each class's local data to train on. After the federated rounds, the client fine-tunes on its complete local dataset, and the paper reports that this optimized client outperforms naive clients on CIFAR-10, CIFAR-100, and FashionMNIST under five aggregation algorithms, for example 43.01% versus 26.64% accuracy on CIFAR-10 with FedAvg. The paper also proves an upper bound on the optimized client's performance gap relative to full-data training during the federated rounds, framing the approach as a trade-off between during-FL and post-FL performance. If correct, the method gives a client a way to personalize the global model while feeding less data into the server's aggregation.","feed_headline":"Reinforcement learning picks the local data a federated client really needs","feed_subtitle":"A deep-RL-guided client beats standard clients across five aggregation algorithms after full-data fine-tuning.","key_machinery":"The load-bearing object is the DRL action vector $a_t = (z_1, \\dots, z_C)$, where $z_c$ is the proportion of class-$c$ samples selected from the client's local dataset for the next FL round. The state is the class-wise F1-score of the server-aggregated model on the client's local validation split; the reward is a piecewise function of relative training-loss reduction, the mean action, and a fitted loss-estimation curve; and the policy is trained with DDPG. Two exploration variants, the normalized action and the weighted-metric action, adapt the action vector to epsilon-greedy selection. The accompanying bound, $P_k - P'_k \\le \\Omega$, is a circle-area argument: performance for a class is treated as the area of a circle whose radius is the fraction of data used, so the gap between full-data training and partitioned training is bounded by the sum of squared-radius differences. This bound is what lets the client tolerate lower during-FL accuracy while expecting post-FL gains.","core_discovery":"The central claim is that a federated learning client can achieve better performance on its own data distribution by not using all its local data during the federated rounds. The proposed optimized client uses a DRL agent that observes the class-wise F1-score of the aggregated model on the client's local validation data as its state, outputs per-class data-selection weights as its action, and receives a reward based on the change in training loss and the amount of data used. The selected subset is used only for local FL training; after the final aggregation round, the client fine-tunes on its complete local dataset. Reported results compare the best mean performance of naive clients with the optimized client across FedAvg, FedAvgM, FedMedian, FedProx, and FedCDA, with the optimized client reaching 43.01% accuracy on CIFAR-10 versus 26.64% for naive FedAvg clients. The paper frames this as both a personalization mechanism and a way to avoid oversharing local information with the server.","pith_inferences":["A control experiment that gives a naive client the same post-FL full-data fine-tuning without the DRL agent would isolate whether the reported gains come from the learned data selection or from the extra training phase; the current comparison does not run that control.","The per-class selection policy is defined for only one optimized client; if several clients ran DRL agents simultaneously, their choices would interact through the server aggregation, so the bound $P_k - P'_k \\le \\Omega$ would need a coupled multi-agent version rather than a per-client one.","Because the state is a class-wise performance vector and the reward is pluggable, the same machinery could be repurposed for other objectives such as class-balanced fairness or energy budgets, but the paper does not test those objectives.","A natural next test is whether the learned selection policy transfers across clients or datasets: train the agent on one client's data distribution and deploy it on another client in the same federation to see whether the selected proportions generalize."],"forward_implications":["A client can improve its final personalized accuracy by letting a DRL agent decide per-class data proportions during federated rounds and then fine-tuning on its full local dataset after aggregation ends.","The reported improvement appears across all five tested aggregation algorithms, suggesting the data-selection mechanism does not depend on a particular aggregation rule.","During the federated rounds the optimized client may lag behind naive clients, but the upper-bound result says this lag is controlled by the gap between the selected action weights and the full per-class sample sizes.","Because the optimized client uses less local data in each round, it shares less information with the server during aggregation than a naive client, aligning personalization with privacy.","The ablation study indicates that replacing optimized actions with naive 80/20 actions changes the outcome, which the paper credits to the learned data-partitioning policy."],"supporting_citations":[{"why":"Supplies the FedAvg aggregation objective and loop that the optimized client modifies.","marker":"[30]"},{"why":"Supplies the DDPG actor-critic algorithm used to train the DRL policy.","marker":"[27]"},{"why":"Supplies the Dirichlet partitioner that creates the non-IID local datasets used in all experiments.","marker":"[48]"},{"why":"FedMedian is one of the five aggregation baselines the optimized client is compared against.","marker":"[45]"},{"why":"FedCDA is one of the five aggregation baselines the optimized client is compared against.","marker":"[38]"},{"why":"FedProx is one of the five aggregation baselines the optimized client is compared against.","marker":"[24]"},{"why":"FedAvgM is one of the five aggregation baselines the optimized client is compared against.","marker":"[14]"},{"why":"ResNet50 is the backbone architecture used for both server and client models.","marker":"[13]"},{"why":"Supplies the nonlinear regression technique used for loss estimation in the reward function.","marker":"[37]"}],"fun_headline_variants":["Reinforcement learning trims local data to boost federated client accuracy","Deep RL picks which local data a federated client needs most","Federated clients learn to skip irrelevant data using reinforcement learning","Optimized local data selection via reinforcement learning in federated setups"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the measured performance gain of the optimized client comes from the DRL-chosen data partitions, but the optimized client alone also receives extra fine-tuning on its complete local dataset after the FL rounds, and no naive control client gets that same post-FL fine-tuning without the DRL agent.","fun_headline_variants_meta":{"raw":{"variants":["Reinforcement learning trims local data to boost federated client accuracy","Deep RL picks which local data a federated client needs most","Federated clients learn to skip irrelevant data using reinforcement learning","Optimized local data selection via reinforcement learning in federated setups"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000654,"raw_usage":{"total_tokens":3036,"prompt_tokens":1024,"completion_tokens":2012,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":640,"completion_tokens_details":{"reasoning_tokens":1948}},"tokens_in":640,"tokens_out":2012,"duration_ms":14050,"temperature":1.0,"reasoning_tokens":1948,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:59:56.506531+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same federated setup with three clients: an optimized client, a naive client that trains on all local data throughout, and a control client that trains on all local data during FL and then performs the same post-FL full-data fine-tuning without any DRL agent. If the control client's final accuracy matches or beats the optimized client's, then the DRL data selection is not the source of the reported gain.","supporting_citations":[{"cited_title":"Continuous control with deep reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the DDPG actor-critic algorithm used to train the DRL policy."},{"cited_title":"Bayesian nonparametric federated learning of neural networks,","cited_arxiv_id":null,"evidence_quote":"Supplies the Dirichlet partitioner that creates the non-IID local datasets used in all experiments."},{"cited_title":"Byzantine-robust distributed learning: Towards optimal statistical rates,","cited_arxiv_id":null,"evidence_quote":"FedMedian is one of the five aggregation baselines the optimized client is compared against."},{"cited_title":"FedCDA: Federated learning with cross-rounds divergence-aware aggregation,","cited_arxiv_id":null,"evidence_quote":"FedCDA is one of the five aggregation baselines the optimized client is compared against."},{"cited_title":"Confidence region estimation techniques for nonlinear regression in groundwater flow: Three case studies,","cited_arxiv_id":null,"evidence_quote":"Supplies the nonlinear regression technique used for loss estimation in the reward function."}],"review_version":1}