{"id":"cbc90c68-39db-4ce7-8be8-65a15db86082","arxiv_id":"2505.03533","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A QMIX-based resource allocation strategy that adapts sub-band and power choices to small-scale fading improves federated learning accuracy in spectrum-sharing wireless networks.","lead":"This paper proposes a wireless resource allocation method for federated learning that adjusts spectrum and power choices within each short fading period, using multi-agent reinforcement learning. It targets faster, more accurate model training when many devices share a wireless channel and hold different local data.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reward in Eqs. (28)-(31) is a heuristic proxy for the true convergence objective: it substitutes the previous aggregate gradient for the current ideal gradient and drops the 1/|N_t|^2 factor, so the claimed QMIX advantage may stem from tuned sum-rate shaping rather than the convergence-driven…","rationale":"I agree with the reader's weakest assumption. The paper's own Eq. (29) is introduced as a heuristic assignment, and the ablation in Fig. 6 only removes observations; it does not validate that the learned policy minimizes the true convergence term. I do not rest the objection on the reader's claim of a false norm inequality in Appendix C; the 2/2/1 decomposition there can be repaired using the zero-mean structure of the stochastic-gradient noise, and the remaining arithmetic issues mostly affect constants C1/C3 rather than the reward-relevant second term. The reward proxy is the more central gap because it is the bridge between Theorem 1 and the RL objective. A direct offline computation of the true error under the trained policy, or an A/B retrain with the true current-round ideal gradient, would settle whether the claimed mechanism is real. Since the paper's conclusions would need conditional revision to add such validation, the reader's CONDITIONAL verdict stands unchanged.","tokens_in":19667,"tokens_out":21920,"duration_ms":212564,"concrete_test":"On the alpha=0.5, n_c=21 setting, after training the proposed QMIX policy, run it on at least 100 held-out channel and data realizations and record, per episode, (i) the cumulative reward of Eq. (31) and (ii) the realized true objective ||gbar_t - g_t||^2, which is available offline because the simulator can compute the exact current-round ideal average. Compute Spearman correlation between these two quantities, and compare the mean true error under the learned policy against the max-sum-rate baseline on the same realizations. If the correlation is weak or negative, or the learned policy's true error is not below the baseline's, the reward proxy is not faithful. As a second arm, retrain with the same hyperparameters but replace g_{t-1} by the true gbar_t in Eqs. (29)-(30); if final test accuracy moves outside the reported error bars, the previous-round approximation is load-bearing.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that QMIX policies trained on a reward derived from Theorem 1 outperform heuristics because they exploit small-scale fading and gradient deviation. For this to hold, the reward must be a faithful surrogate for the one-step convergence term E||gbar_t - g_t||^2, up to policy-ordering preservation, but that condition is not established. In Eq. (28), gbar_t is replaced by g_{t-1}, the aggregate from the previous FL round, although the true ideal gradient averages current-round stochastic gradients over all N clients. Eq. (29) then rewrites the squared term as a double sum of inner products divided by |N_t|^2, and the per-slot reward in Eqs. (30)-(31) drops that 1/|N_t|^2 normalization, adds lambda_1|N_t|, and appends a transmitted-rate term lambda_t * sum_n C_{n,ts}. Section IV-D explicitly calls the design heuristic. No monotonicity or bounded-error argument connects the optimum of the proposed per-slot reward to the minimizer of the true convergence term. A concrete failure mode is that the proxy rewards clients whose current gradient happens to be close to last round's aggregate, which can produce a self-consistent but biased schedule. The experiments never measure the realized true error E||gbar_t - g_t||^2 under the learned policy, so the reported accuracy gains may be due to the lambda_t sum-rate component and QMIX coordination rather than the convergence-derived gradient-deviation mechanism claimed in the ablation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper considers spectrum and power allocation for the uplink of wireless federated learning (FL) when small-scale fading varies within a single gradient-upload round, so the block-fading assumption in prior work is inadequate. The authors derive a one-step convergence bound for FedAvg under non-IID data (Theorem 1), whose second term depends on the set of clients that successfully upload, and use this term to design a Dec-POMDP: each client is an agent that chooses a sub-band and a power level in every coherence-time slot, with the problem solved by QMIX under centralized training and decentralized execution. Agent observations include large- and small-scale fading, transmission progress, and a gradient-deviation term defined as the distance between the client's current local gradient and the previous round's aggregate gradient. The reward combines a convergence-related component with a transmission-rate component. Experiments on CIFAR-10 with QuaDRiGa channels compare the proposed scheme with three heuristic baselines over nine (statistical-heterogeneity, channel-fluctuation) settings; the QMIX policy consistently attains the highest test accuracy and approaches a perfect-communication upper bound, and an ablation (Fig. 6) shows that removing either the small-scale-fading or the gradient-deviation observation degrades performance.","tokens_in":20002,"tokens_out":36912,"duration_ms":306872,"significance":"If the results hold, this is a solid systems contribution to wireless FL: the per-slot, small-scale-fading-aware allocation framework is a practical improvement over round-static schedulers, and the empirical gains over the three baselines are clear and consistent across heterogeneity levels. The paper's strongest assets are the well-motivated problem, the realistic QuaDRiGa evaluation, the breadth of tested scenarios, and the informative ablation of Fig. 6. The theoretical anchor (Theorem 1) is not proven as written, and the reward's link to the convergence bound is heuristic in three unquantified respects (the g-tilde_{t-1} proxy, the dropped 1/|N_t|^2 factor, and the non-telescoping per-slot decomposition). Because the paper's narrative attributes the gains to this convergence-informed design, the theory-reward chain needs to be repaired and validated before the attribution claim is credible. The errors appear fixable within the scope of a revision.","major_comments":[{"comment":"The proof of Theorem 1 is not correct as written, although the final statement appears salvageable. (i) In the bounding of E||grad F(w_t) - g-tilde_t||^2, the paper uses the deterministic inequality ||a+b+c||^2 <= 2||a||^2+2||b||^2+||c||^2 (implicit in the step bounding by 2(A1) + 2(A2) + (A3)); this inequality is false, e.g., a=(1,0), b=(0,1), c=(1,1) gives 8 <= 6. The intended estimate can be recovered in expectation from the unbiased-sampling assumption because g-bar_t - g-bar-tilde_t has conditional mean zero, but that argument is not supplied. (ii) The expansion of E||grad F(w_t) - g-bar_t||^2 displays the identity (1/E) sum_i grad F_n(w_t) - eta_l sum_i grad F_n(w_{n,t}^{(i-1)}) = (1 - eta_l E) sum_i grad F_n(w_t) - eta_l E (sum_i grad F_n(w_t) - sum_i grad F_n(w_{n,t}^{(i-1)})); expanding the right-hand side gives (1 - 2 eta_l E) sum_i grad F_n(w_t) + eta_l E sum_i grad F_n(w_{n,t}^{(i-1)}), which differs from the left-hand side, and the sign of the second term should be positive. The following lines (46)-(50) are consistent with the corrected identity, so the error is repairable, but the appendix must be rewritten. (iii) In Appendix A, the step between (38) and (39) asserts that eta_l <= 1/sqrt(8 E L) implies 1 + 1/(E-1) >= 1 + 1/(2E-1) + 4 E eta_l^2 L^2; this fails for E >= 3 (for E=3 and eta_l^2 L^2 = 1/24, the inequality reads 0.5 >= 0.7), and a sufficient condition would instead be eta_l^2 L^2 <= 1/(4(E-1)(2E-1)).","section":"Appendix C (Theorem 1); Appendix A (Lemma 1)"},{"comment":"The reward is a heuristic surrogate whose connection to the quantity Theorem 1 identifies (E||g-bar-tilde_t - g-tilde_t||^2) is unvalidated at three levels. First, g-bar-tilde_t is replaced by g-tilde_{t-1} (the previous round's aggregate over the then-successful subset) in the observation (26) and in (28)-(29), with no bound on ||g-bar-tilde_t - g-tilde_{t-1}||; the global model changes each round, so this approximation error is uncontrolled, particularly in early FL rounds. Second, the reward in (29) drops the 1/|N_t|^2 normalization present in (28) and adds lambda_1 |N_t|, so the reward landscape differs from the convergence term in a way that can change the optimal policy (e.g., favoring a small unnormalized deviation ||sum_{n in N_t} delta-tilde_n||^2 over a large |N_t|). Third, the per-slot reward (30) does not decompose (29): since N_ts is cumulative, applying (30) at every slot after the first success counts each client's terms repeatedly, and the factors 2 and 1 on the cross terms do not reproduce the pairwise expansion of (29). Because Section V.B attributes the QMIX gains to this convergence-derived reward and the ablation in Fig. 6 attributes them to the gradient-deviation observation, the paper should provide direct evidence that the proxy preserves policy ordering, e.g., by reporting the realized true term E||g-bar-tilde_t - g-tilde_t||^2 (computable in simulation) under the learned policy and baselines, or by ablating the reward components (with/without the proxy, with/without the 1/|N_t|^2 factor). Without such evidence, the reported gains could equally stem from the lambda_t sum-rate term and from QMIX coordination alone.","section":"Section IV-D, Eqs. (28)-(31)"},{"comment":"The reward weights in the central experiments are not reported: lambda_1 and lambda_2 in (29)-(30) are defined but their values are absent from Tables I and II, and lambda_c and lambda_t in (31), described as balancing and normalizing the convergence and throughput rewards, are given no values and no normalization procedure. The two reward components have very different scales (gradient-norm squared terms versus bits-per-slot rates), so without these values the reward actually optimized in the experiments is unknown to the reader and the results are not reproducible. Given that the paper's central claim is that the gradient-deviation mechanism drives the improvement, the authors should report the chosen weights and ideally include a sensitivity study over them.","section":"Section V-A, Tables I-II"}],"minor_comments":[{"comment":"The statement that the first and third terms in (25) 'are determined only by the initialization and FL hyperparameters' is inaccurate: the first term contains ||grad F(w_t)||^2, which depends on the whole training trajectory and the data, and C3 contains the data-dependent constants sigma_g^2 and sigma_l^2. A wording like 'do not depend on the current upload decisions' would preserve the design rationale.","section":"Section III-B"},{"comment":"The intermediate line writes '24/N eta_l^4 E^4 L^2 sum_n 4E ||F_n(w_t)||^2', which carries a spurious factor 4E; the final constant in (50) is consistent with the factor-free expression, so this appears to be a typographical slip that should be fixed.","section":"Appendix C, line preceding Eq. (50)"},{"comment":"The stochastic gradient in the local update is written as grad-tilde F_n(w_{n,t}^{(i)}) but should be evaluated at w_{n,t}^{(i-1)}; likewise the summation in (40) is unclear, since for i=1 the stated range p=1,...,i-1 is empty yet the bound should still be positive.","section":"Appendix A, Eqs. (36)-(40)"},{"comment":"The sentence 'each agent interacts with the recurring environment 20 times per FL round' is ambiguous; it should be clarified whether these are environment steps, RL episodes, or network gradient updates, and how they relate to the T_s=250-slot upload process and to the global model update in Algorithm 1.","section":"Section V-A"},{"comment":"The shaded bands are the 10th-90th percentile ranges of only 5 independent runs; the authors should state the percentile estimator used or show all runs, since percentile bands from 5 samples are noisy.","section":"Fig. 4"},{"comment":"Minor typos: 'gredients' appears twice in the text around Eq. (30) and should read 'gradients'; the caption of Fig. 6 says 'gradient derivation' where 'gradient deviation' is meant.","section":"Throughout"},{"comment":"The baseline set contains only heuristic schemes; comparing against a state-of-the-art FL-aware scheduler from the block-fading literature (e.g., [16] or [17]) would strengthen the claim that the gains come specifically from exploiting small-scale fading within the round rather than from better coordination in general.","section":"Section V-B"}],"recommendation":"major_revision","confidential_remarks":"The manuscript fits the journal's scope well, and the citation and positioning practices appear appropriate. My main concern for the editor is the gap between the paper's narrative, namely that the QMIX gains are driven by the convergence-bound-derived reward, and the direct evidence offered: the appendix proofs require correction, the reward proxy is unvalidated, and the reward weights are unreported. I would ask the authors, at revision, to supply a corrected proof of Theorem 1 (or explicitly restate the necessary learning-rate condition), to report the realized second term of the bound under the learned policy versus the baselines as a validation of the proxy, and to report the reward weights. These items are all within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Per-slot resource allocation under small-scale fading is a genuinely useful problem formulation, and the QMIX-based solution is a reasonable engineering choice. But the convergence theorem that motivates the reward is not proven as written: the Appendix C derivation contains a false algebraic step, so the theoretical anchor is broken.\n\nWhat is actually new: most prior work assumes block fading or static per-round decisions. Here each client picks a sub-band and power level in every coherence slot, with a reward decomposed per slot to avoid sparse reward. That is a meaningful extension. The ablation study is also informative: removing the gradient-deviation observation or the small-scale fading observation degrades performance, which suggests those features matter. The experimental coverage (three alpha levels, three nc levels) is reasonable, and the gains over the three heuristics are consistent across settings.\n\nWhere the paper is soft: Theorem 1's proof in Appendix C rewrites the gradient difference using a decomposition with (1-eta_l E) that is algebraically wrong for E > 1. I checked a simple E=2 example; the equality fails. So the one-step bound might still be true, but the presented derivation does not establish it. Second, the reward in Eqs. (28)-(31) is only loosely connected to that bound: it replaces the current ideal gradient with the previous round's aggregate, drops the 1/|N_t|^2 factor, and adds an ad hoc sum-rate term. The paper calls this heuristic, which is fair, but then it should not claim that the bound directly justifies the reward without some validation. Third, the evaluation does not include any MARL baseline such as FedQMIX or FedMarl; the comparison is only against heuristics, so \"significantly outperforms\" is a weaker claim than it appears. Finally, the reward weights lambda1, lambda2, lambdac, lambdat are not reported, which is a reproducibility problem.\n\nThe core engineering claim is plausible, and the work is honest about the heuristic reward design. The theorem flaw is real but fixable. I would send this to peer review: the problem formulation is worth refereeing, and the authors can probably repair the proof or downgrade it to a heuristic motivation. I would not cite it in its current form until the theorem is fixed and the reward weights are reported.\n\nRecommendation: accept for peer review with a request for major revision.","headline":"Per-slot small-scale-fading-aware allocation is a solid idea, but the convergence theorem that anchors the reward is unproven as written and the reward-to-bound link is heuristic.","tokens_in":20516,"tokens_out":9281,"would_cite":false,"duration_ms":73008,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A multi-agent reinforcement learning policy that reallocates spectrum and power on every fast-fading time slot brings wireless federated learning close to the perfect-communication accuracy ceiling.","keywords":["federated learning","small-scale fading","resource allocation","multi-agent reinforcement learning","QMIX","Dec-POMDP","convergence analysis","wireless communications"],"falsifier":"Train the same QMIX setup with the reward computed from the true ideally-uploaded gradient $\\bar{\\tilde g}_t$, available at the server during centralized training, instead of the previous-round proxy $\\tilde g_{t-1}$; if accuracy does not improve, or if removing the convergence-reward terms leaves performance unchanged, the reported gain is not caused by the convergence-derived reward.","tokens_in":19426,"feed_emoji":"📡","tokens_out":8573,"duration_ms":78017,"temperature":0.7,"pith_summary":"Wireless federated learning usually assumes the channel is fixed during a round of gradient uploads, but real uploads span many fast-fading coherence slots. This paper claims that letting each client re-decide its sub-band and transmit power at every coherence slot, coordinated by the multi-agent QMIX algorithm, improves final model accuracy when data are non-IID and channels fluctuate. The argument rests on a one-step convergence bound for FedAvg whose only round-controllable term is the gap between the ideally-uploaded aggregate gradient and the gradient the server actually receives; the paper converts that gap into a per-slot reward. In simulations the learned policy beats heuristic baselines and approaches the perfect-communication upper bound, and ablations show both the small-scale fading observation and a gradient-deviation observation are needed for the gain.","feed_headline":"Per-slot wireless scheduling brings FL near perfect accuracy","feed_subtitle":"The QMIX policy reallocates spectrum and power each coherence slot, beating heuristics on non-IID data.","key_machinery":"The central object is the convergence-reward link: Theorem 1 isolates $C_2\\mathbb{E}\\|\\bar{\\tilde g}_t-\\tilde g_t\\|^2$, the mean squared error between the aggregate stochastic gradient that would be uploaded if every client succeeded and the gradient the server actually aggregates, and this term depends on which clients the wireless schedule lets through. Because the true ideal gradient is unavailable, the paper approximates it with the previous round's aggregate gradient $\\tilde g_{t-1}$, which clients can recover from the difference of the last two broadcast global models, yielding per-client estimated deviations $\\tilde\\delta_{n,t}^2$. The reward then decomposes that expression over time slots as a client-count term plus gradient-deviation and pairwise inner-product terms, together with a sum-rate term. QMIX supplies centralized training via a monotonic mixing network and decentralized execution, so each client's DQN maps its local observation to sub-band and power actions per slot.","core_discovery":"The paper claims that resource allocation for wireless FedAvg should be decided below the round level: inside one gradient-upload round of $T_s$ coherence-time slots, each client can pick a sub-band and a power level per slot, and a QMIX-trained multi-agent policy can learn to make those choices well. The enabling analysis is Theorem 1, a one-step convergence bound of the form $\\mathbb{E}[F(w_{t+1})]-F(w_t)\\le C_1\\|\\nabla F(w_t)\\|^2+C_2\\mathbb{E}\\|\\bar{\\tilde g}_t-\\tilde g_t\\|^2+C_3$, where the middle term is the only component that the current round's upload decisions can shrink. The paper therefore builds a Dec-POMDP in which each client observes large-scale fading, per-slot small-scale fading across sub-bands, remaining transmission volume, an estimated gradient deviation $\\tilde\\delta_{n,t}^2=\\|\\tilde g_{t-1}-\\tilde g_{n,t}\\|^2$, and the round index, and receives a reward that heuristically splits the convergence term into per-slot pieces plus a throughput incentive. On CIFAR-10 with time-varying multipath channels generated from an urban microcell model, the resulting policy consistently beats max-sum-rate, max-individual-rate, and random baselines, and tracks the perfect-communication accuracy ceiling, most clearly under strong statistical heterogeneity ($\\alpha=0.5$).","pith_inferences":["A stress test the paper does not run is to compare the proxy reward against an oracle reward computed from the true ideally-uploaded gradient at the server; if the policy ordering changes, the reported gain may be driven by the throughput term rather than the convergence term.","The framework suggests an immediate extension to explicit client selection at the start of each round, named as future work in the conclusion, and the same reward structure could be adapted to FL variants such as SCAFFOLD whose drift terms enter the bound differently.","Because the proxy approximates the current ideal gradient with the previous round's aggregate, abrupt shifts in data distribution between rounds are a plausible failure mode worth testing.","The decentralized-execution phase requires every agent to know the large-scale fading of all clients, an overhead that grows with the number of clients and is not quantified in the paper."],"forward_implications":["Per-slot, per-client spectrum and power choices can shrink the round-level aggregation-error term in the convergence bound, something a static per-round allocation cannot do.","Under strong statistical heterogeneity, steering resources toward clients whose gradients align with the aggregate matters more than maximizing total throughput, which is why the QMIX policy outperforms sum-rate-maximizing baselines.","A policy trained at one heterogeneity level and channel condition remains effective when those settings change at test time, since the paper fixes $\\alpha=0.5$, $n_c=21$ during training and varies both during testing.","The ablation results imply that both the small-scale fading observation and the gradient-deviation observation carry information the agent needs; removing either one lowers accuracy."],"supporting_citations":[{"why":"Defines FedAvg and the global model update rule whose convergence the paper bounds.","marker":"[1]"},{"why":"Supplies the non-IID convergence framework and the bounded local and global variance assumptions adapted in Lemma 1 and Theorem 1.","marker":"[6]"},{"why":"Provides the spectrum-sharing MARL formulation and per-slot channel model that the resource allocation builds on.","marker":"[21]"},{"why":"Introduces QMIX, the monotonic value-factorization algorithm used for centralized training and decentralized execution.","marker":"[25]"},{"why":"Contributes the fingerprint technique for stabilizing experience replay in non-stationary multi-agent settings, used for the FL round index in the observation.","marker":"[27]"},{"why":"Generates the time-varying small-scale fading channel realizations with controllable cluster numbers used in the simulations.","marker":"[29]"},{"why":"Provides the Dirichlet-based data partitioning used to create the different statistical heterogeneity levels.","marker":"[30]"}],"fun_headline_variants":["Fading-aware per-slot allocation sharpens federated learning","Per-coherence-slot resource allocation boosts FL accuracy","Slot-level fading-aware scheduling closes FL accuracy gap","QMIX policy tracks perfect-communication FL accuracy","Wireless FL improved by slot-level fading-aware allocation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The scheme depends on the assumption that replacing the true ideally-uploaded gradient with the previous round's aggregate gradient, and then decomposing the convergence term into the heuristic per-slot reward of Eqs. (29)-(31), preserves which policies are best under the actual objective; if that proxy is poor, the learned allocation optimizes a reward only loosely connected to convergence.","fun_headline_variants_meta":{"raw":{"variants":["Fading-aware per-slot allocation sharpens federated learning","Per-coherence-slot resource allocation boosts FL accuracy","Slot-level fading-aware scheduling closes FL accuracy gap","QMIX policy tracks perfect-communication FL accuracy","Wireless FL improved by slot-level fading-aware allocation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000629,"raw_usage":{"total_tokens":2962,"prompt_tokens":1057,"completion_tokens":1905,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":673,"completion_tokens_details":{"reasoning_tokens":1829}},"tokens_in":673,"tokens_out":1905,"duration_ms":15916,"temperature":1.0,"reasoning_tokens":1829,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:49:11.744164+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same QMIX setup with the reward computed from the true ideally-uploaded gradient $\\bar{\\tilde g}_t$, available at the server during centralized training, instead of the previous-round proxy $\\tilde g_{t-1}$; if accuracy does not improve, or if removing the convergence-reward terms leaves performance unchanged, the reported gain is not caused by the convergence-derived reward.","supporting_citations":[{"cited_title":"Communication-efficient learning of deep networks from decentralized data,","cited_arxiv_id":null,"evidence_quote":"Defines FedAvg and the global model update rule whose convergence the paper bounds."},{"cited_title":"On the convergence of FedAvg on non-IID data,","cited_arxiv_id":null,"evidence_quote":"Supplies the non-IID convergence framework and the bounded local and global variance assumptions adapted in Lemma 1 and Theorem 1."},{"cited_title":"Spectrum sharing in vehicular networks based on multi-agent reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Provides the spectrum-sharing MARL formulation and per-slot channel model that the resource allocation builds on."},{"cited_title":"Monotonic value function factorisation for deep multi-agent reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Introduces QMIX, the monotonic value-factorization algorithm used for centralized training and decentralized execution."},{"cited_title":"Stabilising experience replay for deep multi-agent reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Contributes the fingerprint technique for stabilizing experience replay in non-stationary multi-agent settings, used for the FL round index in the observation."},{"cited_title":"QuaDRiGa: A 3-D multi-cell channel model with time evo- lution for enabling virtual field trials,","cited_arxiv_id":null,"evidence_quote":"Generates the time-varying small-scale fading channel realizations with controllable cluster numbers used in the simulations."}],"review_version":1}