{"id":"6d7efd63-7980-4b53-8a62-d9caf0d15cd5","arxiv_id":"2507.15256","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"An over-the-air federated distillation scheme with closed-form power control and SDR-based beamforming is derived from a convergence-rate bound, with a claimed proof that the beamforming relaxation is tight.","lead":"The authors design an over-the-air federated distillation system in which wireless devices share per-class model outputs rather than model parameters. They derive a convergence bound, use it to optimize transmit powers and receiver beamforming, and claim large communication savings with small accuracy loss.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Proposition 2's proof relies on inequality (53), |Σ z_r sqrt(Λ_r)|² ≥ Σ Λ_r |z_r|², which is false for complex z_r (e.g., z1=1, z2=-1, Λ1=Λ2=1/2 gives 0 ≥ 1). The constructed rank-one candidate may violate the constraints, so the claimed zero optimality gap is unproven.","rationale":"I read the paper in good faith. The contribution is an over-the-air federated distillation framework with a convergence-rate-driven transceiver design. The load-bearing theoretical claim is Proposition 2, which states that the relaxed SDP has a rank-one optimal solution, permitting exact recovery of the optimal beamformer. The proof in Appendix C is the only support for this claim. Its key step, inequality (53), is demonstrably false for complex z_r, and the constructed rank-one matrix can fail the constraints of (P6). The proof also assumes without justification that the same device attains equality in every per-class constraint. These are not matters of taste or disagreement with a heuristic; they are internal invalidities in the argument. The numerical section provides empirical evidence that the SDP solutions are approximately rank one in the tested configurations, but numerical rank-one behavior does not repair a false proof, especially since SDP solvers return approximate solutions. Therefore the paper's central optimality-gap guarantee is unsupported. The practical framework may still be valuable, and a corrected or weakened claim (e.g., heuristic rank-one recovery with numerical validation) could be acceptable, but the manuscript as written does not deliver the claimed theoretical result. The reader's verdict of REJECT with moderate confidence is appropriate, and my stress-test does not change it.","tokens_in":23290,"tokens_out":3037,"duration_ms":37602,"concrete_test":"Symbolically evaluate inequality (53) with z1=1, z2=-1, Λ1=Λ2=1/2: the left side is 0 and the right side is 1, so the inequality is false. To expose the consequence within Appendix C, set h1,t = v1 - v2 and W* with eigenvalues Λ1=Λ2=1/2, define v_hat = (v1+v2)/sqrt(2), and compute e_k from equation (50) using device 1's equality constraint. Then check whether e_k + Tr(W_hat H_i) ≥ 0 holds for all i,k. If any constraint is violated, the constructed rank-one solution is infeasible and the proof of Proposition 2 collapses.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central theoretical guarantee is Proposition 2: the relaxed SDP (P6) has a rank-one optimal solution, so the SDR is tight and the recovered beamformer is optimal for (P2). Appendix C attempts to prove this by taking an optimal W* of rank R>1 with eigendecomposition VΛV^H and constructing a rank-one candidate W_hat whose top eigenvector is v_hat = Σ_{r=1}^R sqrt(Λ_r) v_r. Feasibility of W_hat requires that for every class k and every device i, e_k^* + Tr(W_hat H_i) ≥ 0, i.e., |Σ_r z_{i,r} sqrt(Λ_r)|² ≥ Σ_r |z_{i,r}|² Λ_r. This is exactly inequality (53). But (53) is the reverse of Cauchy–Schwarz and is false for arbitrary complex z_{i,r}. A concrete failure is z1=1, z2=-1, Λ1=Λ2=1/2, for which the left side is 0 and the right side is 1. Thus the constructed rank-one matrix can be strictly worse than W* on some constraints, so the proof's claim that W_hat is feasible does not follow. A second unproved step is the 'without loss of generality' assumption in (49) that the same device index, device 1, attains equality in every per-class constraint; Lemma C.2 only guarantees at least one equality per class, and the index could vary with k. Both gaps invalidate the proof of Proposition 2. The numerical rank-one observation in Fig. 2 is suggestive but does not substitute for a proof, and the exact SDP solutions in CVX are computed to finite precision. Since the paper's title and abstract claim optimal transceiver design and no optimality gap, this is a load-bearing flaw. The reader's weakest_assumption correctly identifies the same root cause, and I agree that the verdict should stand as REJECT.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an over-the-air federated distillation framework in which wireless devices upload per-class averaged soft predictions, aggregated at a multi-antenna parameter server by over-the-air computation. It derives a convergence bound for the FD training process, formulates a transceiver design problem, obtains closed-form optimal transmit equalization factors and post-processing scalars, and proposes an SDR-based receiver beamforming method. The authors claim that the SDR relaxation has zero optimality gap (Proposition 2) and support this with numerical experiments.","tokens_in":23762,"tokens_out":15736,"duration_ms":168000,"significance":"If the results hold, the paper makes a useful contribution: it is the first transceiver design for over-the-air federated distillation, with a nontrivial convergence analysis and a low-complexity alternating algorithm. The closed-form power control and the SDR formulation are natural, and the numerical comparisons show meaningful communication-overhead reductions. However, the central theoretical guarantee—the zero optimality gap for the beamforming relaxation—rests on a proof that is not valid as written, so the paper's main optimality claim is not established by the current manuscript.","major_comments":[{"comment":"The inequality |∑_r z_r √Λ_r|² ≥ ∑_r Λ_r |z_r|² is stated to follow from Cauchy–Schwarz, but Cauchy–Schwarz gives the opposite bound. For z1=1, z2=−1 and Λ1=Λ2=1/2, the left side is 0 and the right side is 1, so the inequality fails. Since (53) is used to prove that the constructed matrix W_hat satisfies the constraints (54) and that Ω(W_hat) ≤ Ω(W*), the proof of Proposition 2 does not establish feasibility or optimality of the rank-one candidate.","section":"Appendix C, Eq. (53)"},{"comment":"The 'without loss of generality' assumption that device 1 attains equality in every per-class constraint is not justified. Lemma C.2 only guarantees at least one equality per constraint set, and the equality-achieving device index can vary with k; the subsequent construction using a single h1,t and z1,r for all k is therefore unsupported.","section":"Appendix C, Eq. (49)"},{"comment":"The numerical observation that the second eigenvalue of W*_t is near zero is only evidence for the tested random instances and is computed at finite precision. It does not replace the proof of Proposition 2, and the abstract's 'no optimality gap' claim therefore remains unproven.","section":"Section V-B and Fig. 2"}],"minor_comments":[{"comment":"The standard basis vectors used to complete the orthonormal basis should be indexed up to N, not K, since the ambient dimension is N; as written, the construction is undefined when K ≠ N.","section":"Appendix C, Eq. (52)"},{"comment":"The statement that the objective is coercive because it approaches infinity when e_k approaches infinity is imprecise; the objective decreases without bound as e_k → −∞, though feasibility restricts e_k from below. A correct compactness argument is available and should be stated.","section":"Lemma C.1"},{"comment":"The notation (w^H h)^H in the numerator is opaque; writing h^H w explicitly would make the formula easier to read.","section":"Proposition 1, Eq. (20)"},{"comment":"The remark that the optimal transceiver design is independent of L1 and L2 is correct for the power-control and beamforming steps, but the convergence bound in Theorem 1 still depends on these constants; the wording could be sharpened to avoid overstatement.","section":"Footnote 4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript contains a useful system model, a nontrivial convergence analysis, and a plausible numerical study, but the central rank-one tightness proof is invalid. I recommend asking the authors to either provide a correct proof of Proposition 2 or explicitly downgrade the optimality claim to a heuristic with empirical support. If neither is possible after revision, the paper should not be accepted. The earlier VTC publication is cited but is not a substitute for the missing proof."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague — This is a substantive paper that claims a stronger result than it proves. The new piece is the formulation of over-the-air federated distillation with per-class knowledge aggregation and a convergence-rate-driven transceiver design. That's a real extension of over-the-air FL, and the closed-form power control and post-processing scalars in Proposition 1 are clean and useful. The convergence analysis in Theorem 1 follows standard Lipschitz/smoothness assumptions and appears careful. The simulations are reasonably thorough and show communication savings versus FL baselines.\n\nThe soft spot is Proposition 2, and it's load-bearing. The proof of the zero optimality gap relies on inequality (53), |Σ z_r √Λ_r|² ≥ Σ Λ_r |z_r|², which is the reverse of Cauchy-Schwarz and false for complex z_r. For z1=1, z2=-1, Λ1=Λ2=1/2, the left side is 0 and the right side is 1. So the constructed rank-one matrix Ŵ is not guaranteed to satisfy the per-class constraints; it can be strictly worse than the relaxed optimum. The proof also assumes without justification that the same device index (device 1) attains equality in every per-class constraint, while Lemma C.2 only says at least one per class, possibly different devices. Both gaps invalidate the claimed proof. The numerical rank-one observation in Fig. 2 is suggestive but not a proof, and finite-precision SDP solves don't substitute.\n\nThis matters because the title and abstract promise an optimal transceiver design with no optimality gap. If the proof fails, the SDR approach is still a reasonable heuristic, and the power-control part stands on its own. But the central theoretical guarantee is unproven. The paper is not ready as is.\n\nThat said, this is a fixable flaw. The framework and the overall approach are worth engaging; the authors could either supply a correct proof (maybe under extra conditions) or soften the claim to low-rank recovery with numerical evidence. I'd send this to peer review with a clear request for major revision, and I'd expect the reviewers to focus on the SDR tightness.\n\nWho gets value: people working on communication-efficient federated distillation or over-the-air computation; the system model and Proposition 1 are transferable. I wouldn't cite it in the next year until the optimality claim is settled.","headline":"A useful over-the-air FD framework with a clean power-control result, but the zero-optimality-gap theorem in Proposition 2 rests on a false inequality.","tokens_in":24283,"tokens_out":2542,"would_cite":false,"duration_ms":27048,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Over-the-air federated distillation replaces model uploads with aggregated soft predictions, and its beamforming design is proved to have zero relaxation gap.","keywords":["federated distillation","over-the-air computation","transceiver design","beamforming","semidefinite relaxation","convergence rate","knowledge distillation","wireless edge learning"],"falsifier":"Run the relaxed SDP (P6) on a channel realization where the device with the smallest value of $|w^H h_{j,t}|/\\hat{q}_{j,t}^k$ differs across classes; if the optimal value of (P6) is strictly lower than the best achievable value under the rank-one constraint, then the zero-gap claim is false.","tokens_in":1558,"feed_emoji":"📱","tokens_out":2039,"duration_ms":90822,"temperature":0.7,"pith_summary":"This paper proposes a wireless federated distillation scheme in which devices upload only per-class averaged soft predictions, called their knowledge, and the parameter server aggregates these predictions over the air by exploiting channel superposition, avoiding transmission of large model parameters. It derives an upper bound on the expected gradient norm that serves as a convergence-rate proxy, then formulates a transceiver optimization problem over transmit powers, aggregation post-processing scalars, and receiver beamforming. The authors claim that, given the receiving strategy, the transmit powers and scalars have closed-form optimal solutions, and that the semidefinite relaxation of the beamforming problem is tight because its optimal solution is rank-one. If these claims hold, over-the-air federated distillation can approach error-free distillation accuracy with a fraction of the communication load of conventional federated learning on large models.","feed_headline":"Zero-gap beamforming for over-the-air distillation","feed_subtitle":"Devices upload soft predictions instead of model weights, and the receiver design is proved optimal under a rank-one relaxation.","key_machinery":"The engine of the argument is the convergence-rate upper bound in Theorem 1, in which the expected squared gradient norm is bounded by terms involving $\\Phi_{1,i,t}$, the signal misalignment caused by channel fading and power variation across devices, and $\\Phi_{2,i,t}$, the noise passed through the post-processing scalar. Minimizing that bound decouples into a closed-form power-and-estimator step and a beamforming problem that is reformulated, via slack variables $e_k$ and matrices $W_t=w_t w_t^H$, as the semidefinite program (P6). The rank-one proof then constructs a feasible rank-one point from the eigen-decomposition $W_t^* = V\\Lambda V^H$ by choosing $\\hat v = \\sum_{r=1}^R \\sqrt{\\Lambda_r} v_r$ and using inequality (53) to argue that this rank-one point has no larger objective.","core_discovery":"On the paper's own terms, the central discovery is that transceiver design for over-the-air federated distillation is not merely a good heuristic but is globally optimal. For a fixed receiver beamforming vector $w_t$, the optimal transmit equalization factors $P_{i,t}^{k*}$ and post-processing scalars $\\lambda_t^{k*}, a_{i,t}^{k*}$ are given in closed form in Proposition 1. For the remaining beamforming problem, the paper formulates a semidefinite relaxation and proves in Proposition 2 that the relaxed optimum $W_t^*$ is always rank-one, so the recovered beamforming vector $w_t^*$ solves the original non-convex problem with zero optimality gap. Together with the convergence-rate expression of Theorem 1, this yields a per-round optimization algorithm whose objective is the learning performance itself rather than a proxy such as mean squared error.","pith_inferences":["A reader should not treat the zero-gap proof as settled: the inequality in (53), $|\\sum_r z_r\\sqrt{\\Lambda_r}|^2 \\ge \\sum_r \\Lambda_r |z_r|^2$, is not valid for arbitrary complex $z_r$, so the constructed rank-one matrix may be strictly worse than the relaxed optimum.","A direct test is to generate channels where class 1's bottleneck device differs from class 2's bottleneck device; if the relaxed SDP beats every rank-one feasible point in such a setting, the optimality gap is real and a rank-restricted or regularized beamforming step is needed.","If the zero-gap claim survives, the same slack-variable SDR structure may apply to other multi-class over-the-air aggregation problems, such as multi-task or multi-modal federated distillation.","The convergence-rate objective being free of $L_1$ and $L_2$ suggests the design could be implemented without a learning-performance model, but it also leaves open how much accuracy is lost by optimizing a bound rather than the true convergence rate."],"forward_implications":["Each of the three design variables, transmit power, aggregation estimator, and receiver beamforming, has a per-round optimal form, and the beamforming step reduces to a convex SDP whose solution is rank-one.","The optimal design depends only on channel state, local class counts, and knowledge statistics, not on learning constants such as $L_1$ and $L_2$, so the same algorithm can be run without estimating those constants.","Communication per round scales with $K^2$, the number of classes squared, rather than with model size, which is the paper's route to applying federated distillation to very large models.","In the paper's simulations over FMNIST and CIFAR-10 with a 5-antenna server and 50 devices, the design approaches error-free FD accuracy and outperforms uniform beamforming and MSE-minimizing baselines in both IID and non-IID splits.","The per-round complexity $O(N^3+M)$ lets the server recompute the design at the start of each fading block, avoiding per-device channel state information at the transmitter."],"supporting_citations":[{"why":"Supplies the over-the-air computation mechanism: simultaneous analog transmission and superposition at the receiver for aggregation.","marker":"[7]"},{"why":"Introduces federated distillation, the paradigm of sharing model outputs rather than model parameters.","marker":"[16]"},{"why":"Provides online co-distillation, the local training-with-distillation regularizer used in the loss function.","marker":"[17]"},{"why":"Gives the Lipschitz assumptions on the local loss and model mapping used in Theorem 1's convergence bound.","marker":"[24]"},{"why":"Provides the step-decay convergence argument and weighting used to derive the expected gradient-norm bound.","marker":"[25]"},{"why":"Supplies the semidefinite relaxation technique and rank-recovery background for the beamforming problem.","marker":"[27]"},{"why":"Establishes the over-the-air federated learning beamforming benchmark, with MSE minimization, that the paper extends and compares against.","marker":"[11]"}],"fun_headline_variants":["Closed-form power and estimator for over-the-air distillation","Rank-one relaxation yields exact beamforming for air distillation","Zero-gap optimal transceiver for over-the-air federated distillation","Exact beamforming solution for over-the-air distillation","Optimal transceiver closed-form for over-the-air knowledge sharing"],"cache_read_input_tokens":26240,"weakest_assumption_plain":"The proof of Proposition 2 relies on an inequality that does not hold for arbitrary complex numbers, and it assumes that the same device is the bottleneck for every class, so the rank-one construction may not actually reach the relaxed optimum.","fun_headline_variants_meta":{"raw":{"variants":["Closed-form power and estimator for over-the-air distillation","Rank-one relaxation yields exact beamforming for air distillation","Zero-gap optimal transceiver for over-the-air federated distillation","Exact beamforming solution for over-the-air distillation","Optimal transceiver closed-form for over-the-air knowledge sharing"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000695,"raw_usage":{"total_tokens":3188,"prompt_tokens":1036,"completion_tokens":2152,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":652,"completion_tokens_details":{"reasoning_tokens":2071}},"tokens_in":652,"tokens_out":2152,"duration_ms":15233,"temperature":1.0,"reasoning_tokens":2071,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T15:37:34.606277+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the relaxed SDP (P6) on a channel realization where the device with the smallest value of $|w^H h_{j,t}|/\\hat{q}_{j,t}^k$ differs across classes; if the optimal value of (P6) is strictly lower than the best achievable value under the rank-one constraint, then the zero-gap claim is false.","supporting_citations":[{"cited_title":"Broadband Analog Aggregat ion for Low-Latency Federated Edge Learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the over-the-air computation mechanism: simultaneous analog transmission and superposition at the receiver for aggregation."},{"cited_title":"Communication-efficient on-device machine learning: fed erated distil- lation and augmentation under non-IID private data,","cited_arxiv_id":null,"evidence_quote":"Introduces federated distillation, the paradigm of sharing model outputs rather than model parameters."},{"cited_title":"The Best of Both Worlds: Accurate Global and Personalized Models through Federated Learning with Data-Free Hyper-Knowledge Distillation","cited_arxiv_id":"2301.08968","evidence_quote":"Gives the Lipschitz assumptions on the local loss and model mapping used in Theorem 1's convergence bound."},{"cited_title":"Sem idef- inite Relaxation of Quadratic Optimization Problems,","cited_arxiv_id":null,"evidence_quote":"Supplies the semidefinite relaxation technique and rank-recovery background for the beamforming problem."},{"cited_title":"Federated Learnin g via Over-the- Air Computation,","cited_arxiv_id":null,"evidence_quote":"Establishes the over-the-air federated learning beamforming benchmark, with MSE minimization, that the paper extends and compares against."}],"review_version":1}