{"id":"b7da61cd-f458-4e0a-a919-30cd4aa2b7cb","arxiv_id":"2506.16074","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"CAAC, a constrained actor-critic algorithm with attention-enhanced Q-networks and successive convex approximation policy updates, schedules WMMSE priorities and power to improve energy efficiency and QoS satisfaction in simulated MIMO downlinks.","lead":"Wireless base stations often use WMMSE precoding to serve many users, but choosing user priorities and transmit power in real time while satisfying quality-of-service constraints is hard. This paper proposes CAAC, a constrained reinforcement learning algorithm that learns these choices, and reports that it lowers power use and QoS violations in simulations.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The QoS constraint for delay-tolerant users is vacuous as written (U=-R, c=+5 Mbps), so the reported QoS violation ignores the rate constraint and the central claim is unsupported.","rationale":"The paper's central claim is that CAAC achieves lower transmit power and lower QoS violation than the baselines in a system with heterogeneous QoS requirements. That claim requires the QoS constraints to be meaningful for both user classes. As written, the delay-tolerant constraint is vacuous: with utility U = -R and threshold c = 5 Mbps, the constraint -R <= 5 is always satisfied, and the QoS-gap metric [ -R - 5 ]_+ is identically zero. Consequently, the reported QoS-violation numbers exclude any rate-constraint violations by construction, so the empirical comparison does not support the claim of superior QoS-aware scheduling for mixed delay-sensitive and delay-tolerant traffic. This is not merely a stylistic sign typo: it changes the optimization problem being solved. The reader's verdict correctly noted an internal sign error in Eq. (1) and the undefined feasible policy set Theta; the present concern is closely related but more specific and more directly damaging, because it invalidates the QoS metric used in the headline results. A corrected manuscript would need to fix the constraint sign (or equivalently use a negative threshold for rate users) and rerun all experiments, ideally with the delay-tolerant QoS violation reported separately. Until then, the current version does not establish its strongest claim, and I recommend rejecting it as submitted while leaving room for a corrected revision.","tokens_in":10637,"tokens_out":8008,"duration_ms":97973,"concrete_test":"Re-run the Section IV evaluation with the rate constraint implemented as R_k >= 5 Mbps: for delay-tolerant users compute the QoS gap as [5 - R_{k,t}]_+ (equivalently [-R_{k,t} - (-5)]_+) and keep c = 3 for delay-sensitive users; then report the delay-tolerant component of Fig. 4-b separately. If that component is nonzero, the published 2.98% figure and the comparison against PPO-Lag, CPO, and SCAOPO were computed under a vacuous constraint and must be redone. Also verify that Algorithm 1 minimizes average power as described in the text, rather than maximizing it as Eq. (1) currently states.","verdict_should_be":"REJECT","load_bearing_attack":"In Eq. (1), the QoS constraint is lim 1/T E [sum_t sum_n delta(n-rho_k) U(n,k,t)] <= c_k, with Section II-B defining U(2,k,t) = -R_{k,t} for delay-tolerant users and Section IV setting c_{k,b} = 5 Mbps. This yields -R <= +5, i.e., R >= -5 Mbps, which is always true. The same sign appears in the QoS-gap metric of Section IV: [sum_n delta(n-rho_k) U(n,k,t) - c_k]_+ = [-R - 5]_+ = 0 for every delay-tolerant user. Thus the rate-constrained half of the user population places no active constraint on the policy, and the reported '2.98% QoS violation' and all QoS comparisons in Figs. 3-4 do not actually test QoS-aware scheduling for rate-constrained users. The text also says Eq. (1) minimizes transmit power while displaying max, another sign inconsistency in the same formulation. The central claim therefore rests on a self-contradictory constraint formulation: either c_k should be -5 Mbps for delay-tolerant users, or the inequality and gap should be reversed, and the experiments must be rerun under the intended rate constraint before the performance comparison can be evaluated.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a constrained reinforcement learning algorithm, CAAC, for jointly scheduling user priority weights and transmit power for WMMSE precoding in a MIMO downlink. The actor uses a constrained stochastic successive convex approximation method, and the critic uses lightweight attention-enhanced Q-networks. The authors claim that CAAC achieves lower average transmit power and lower QoS violation than PPO-Lag, CPO, SCAOPO, greedy, and equal-priority baselines in simulations with 4 to 16 users. The paper also gives a brief convergence argument that relies on the authors' previous SCAOPO analysis.","tokens_in":10871,"tokens_out":3741,"duration_ms":43447,"significance":"If the performance claims hold, the paper would provide a practical model-free scheduler for QoS-aware WMMSE precoding, with a scalable critic architecture that addresses the growth of Q-network count with the number of users. The simulation setup is described in enough detail to be partially reproducible, and the comparison against multiple CRL baselines is useful. However, the theoretical novelty is limited because CSSCA is essentially a re-use of the authors' earlier SCAOPO method with an attention-augmented critic, and the convergence analysis is inherited from [13] rather than proved here. More importantly, the QoS constraint for delay-tolerant users is vacuous as written, which directly undermines the central claim of QoS-aware scheduling.","major_comments":[{"comment":"The objective is stated inconsistently. Eq. (1) reads as a maximization of average transmit power, while the surrounding text says the goal is to minimize energy consumption. In Eq. (2) the reformulation is a minimization with C'_0(s_t,a_t)=R(s_t,a_t)=p_t, so the sign of the objective flips without explanation. Since the objective surrogate (7) and the reported power values in Section IV all presume minimization, the formal problem statement is self-contradictory and should be corrected, e.g., by replacing max with min in Eq. (1).","section":"Section II-B, Eq. (1) and Section III-A, Eq. (2)"},{"comment":"The QoS constraint for delay-tolerant users is vacuous. With U(2,k,t)=-R_{k,t} and c_k=5 Mbps, the constraint lim (1/T) E[sum_t U(2,k,t)] <= c_k becomes -R <= 5, i.e., R >= -5 Mbps, which is always satisfied for positive rates. Correspondingly, the QoS gap metric in Section IV, [U(2,k,t)-c_k]_+ = [-R-5]_+, is identically zero for every delay-tolerant user. Thus the delay-tolerant half of the user population imposes no active constraint, and the reported '2.98% QoS violation' and the QoS comparisons in Figs. 3 and 4 do not test QoS-aware scheduling for rate-constrained users. The sign of c_k or of the inequality must be corrected, and the experiments must be rerun under a genuinely active rate constraint before the central claim can be evaluated.","section":"Section II-B and Section IV, QoS gap metric"},{"comment":"The feasible policy parameter set Theta is never defined. The surrogate problems (8) and (9) are solved over theta in Theta, and the convergence and feasibility statements in Section III-D depend on the properties of Theta, but the paper neither specifies Theta nor states assumptions on it. Please define Theta explicitly, including any boundedness or regularity conditions needed for the convergence argument.","section":"Section III-B, Eqs. (8) and (9)"},{"comment":"The convergence claim is not established within this paper. The argument invokes the authors' previous [13, Lemma 3] without proof, and then asserts that TD updates make the Q-networks accurate enough for B=200 and T_cri=10. Since CSSCA is essentially SCAOPO with a different critic, the theoretical contribution is heavily self-referential. Please state the precise assumptions under which the Q-network approximation error is small, and either prove or cite a result that covers the finite-B, finite-T_cri regime used in the simulations. Without this, the bias in the policy-gradient estimates (4) is not controlled.","section":"Section III-D"},{"comment":"All simulation results are reported as averages over 5 random seeds, but no error bars, standard deviations, or per-seed curves are shown. For the near-tie comparisons among CRL baselines at small user counts, the claimed advantage of CAAC cannot be assessed statistically. Please report the spread across seeds, and ideally release code so that the results can be reproduced independently.","section":"Section IV, Figs. 3 and 4"}],"minor_comments":[{"comment":"There is a typo: 'the obtrained precoder vector' should be 'the obtained precoder vector'.","section":"Section II-A"},{"comment":"The phrase 'unbias estimation' should be 'unbiased estimate'.","section":"Section III-D"},{"comment":"The symbol c_{k,b} is used in Section IV for the QoS threshold, but the problem formulation in Eq. (1) uses c_k. Please harmonize the notation.","section":"Section II-B"},{"comment":"The TD-update loop uses T_cri mini-batches but the notation 'tcri' is inconsistent; please use a clear subscript throughout, e.g., T_{cri} and t=1,...,T_{cri}.","section":"Section III-C.2"}],"recommendation":"major_revision","confidential_remarks":"The paper's main novelty relative to the authors' prior SCAOPO work [13] is the attention-enhanced shared critic and its application to WMMSE precoding. The convergence analysis, however, is almost entirely deferred to [13], and 'CSSCA' appears to be a rebranding of SCAOPO. Given the vacuous delay-tolerant QoS constraint, the numerical results currently do not support the advertised QoS-aware performance claim. I recommend major revision rather than rejection because the issues are fixable by correcting the constraint formulation, defining Theta, adding statistical evidence, and rerunning the experiments, but the revision needs to be substantial and the comparisons redone under an active rate constraint."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper has one genuinely new piece—the attention-enhanced shared-embedding Q-network—and a policy-optimization core that is effectively their own SCAOPO with a different critic. The simulation setup is described in enough detail to be reimplemented in principle, but the central claim is undermined by a formulation error that needs fixing before the results mean what they say.\n\nI checked the stress-test concern, and it lands. In Eq. (1), the QoS constraint is lim avg E[sum_n delta(n-rho_k) U(n,k,t)] <= c_k. For delay-tolerant users they define U = -R_{k,t}, and Section IV sets c_{k,b} = 5 Mbps. That yields -R <= 5, i.e., R >= -5 Mbps, which is automatically true for any non-negative rate. The QoS-gap metric [sum_n delta(n-rho_k) U - c_k]_+ is then [-R - 5]_+ = 0 for every delay-tolerant user. So half the user population never actively constrains the policy; the reported 2.98% violation is driven entirely by the delay-sensitive users. The text also says Eq. (1) maximizes power while the surrounding text and Eq. (2) minimize it. These are not cosmetic slips: with the intended rate constraint, the experiments need to be rerun.\n\nWhat is good: the shared embedding plus attention across Q-functions is a sensible way to cut parameter count and capture inter-user coupling, and the CAAC(-) ablation gives some evidence that the architecture matters. The baseline set (PPO-Lag, CPO, SCAOPO, greedy, equal-priority) is appropriate, and the parameter-size comparison in Table I is useful. The channel model and hyperparameters are described in enough detail that a patient reader could reimplement.\n\nSoft spots beyond the sign error: the feasible policy set Theta is never defined, so the surrogate problems (8)-(9) are formally underspecified; the policy distribution is not stated; the convergence argument delegates the core proof to [13, Lemma 3] without verifying that its assumptions transfer; and there are no confidence intervals despite averaging over 5 seeds. These are fixable but need to be stated.\n\nWho is this for? Researchers working on constrained RL for wireless resource allocation. It deserves a serious referee—the architecture idea is worth reviewing—but only if the authors correct the constraint sign, define Theta, and ideally release code. As submitted, I would not cite the numerical results.","headline":"The attention-enhanced critic is a genuinely useful idea, but a sign error makes the rate constraint for delay-tolerant users vacuous, so the headline QoS comparison does not test what it claims.","tokens_in":11460,"tokens_out":4844,"would_cite":false,"duration_ms":51372,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A constrained reinforcement learning algorithm, CAAC, schedules user priorities and power for WMMSE precoding, and in simulation achieves lower average power and fewer QoS violations than existing constrained-RL and heuristic methods.","keywords":["constrained reinforcement learning","WMMSE precoding","user priority scheduling","power allocation","QoS constraints","attention mechanism","successive convex approximation","6G wireless networks"],"falsifier":"Run the $K=8$ experiment with $B$ reduced from 200 to 50 (or $T_{\\mathrm{cri}}$ from 10 to 2) while keeping everything else fixed: if CAAC's QoS violation rises to the baselines' level or its power increases, the claimed advantage comes from the critic's sample budget rather than from the algorithm's design. Alternatively, repeat the simulation with 20 random seeds and check whether the 1.39 W and 2.98% figures are stable; if the variance is large, the reported advantage is not robust.","tokens_in":10360,"feed_emoji":"📡","tokens_out":12703,"duration_ms":111488,"temperature":0.7,"pith_summary":"This paper tries to establish that constrained reinforcement learning can solve the user-priority and power-scheduling problem for WMMSE precoding, a standard method for multi-antenna downlink transmission that normally uses fixed priorities and power. The proposed CAAC algorithm models the scheduling task as a constrained Markov decision process and learns a policy that chooses priority weights and transmit power to minimize average power subject to per-user QoS constraints. In simulated MIMO downlinks with $K=8$ users, CAAC reports 1.39 W average transmit power and a 2.98% QoS violation level, outperforming PPO-Lag, CPO, SCAOPO, and greedy baselines. If these results hold, WMMSE precoding can be made QoS-aware and energy-efficient in a model-free way, which matters for 6G networks with heterogeneous traffic demands.","feed_headline":"RL scheduling cuts power and QoS violations in WMMSE","feed_subtitle":"Simulations show 1.39 W average power and 2.98% QoS violations, beating CRL baselines.","key_machinery":"The central machinery is a two-module optimization loop. The Actor uses CSSCA to build convex quadratic surrogate functions $\\bar{f}_{k,i}(\\theta) = \\hat{f}_{k,i} + \\hat{g}_{k,i}^{\\top}(\\theta-\\theta_i) + \\zeta_k \\|\\theta-\\theta_i\\|^2$ from sample estimates of the objective and constraint values and gradients, then solves problem (8) for the next policy parameters, falling back to the relaxed feasible problem (9) when (8) is infeasible. The Critic trains attention-enhanced Q-networks with shared embedding and attention layers to estimate $Q^{\\pi_{\\theta_i}}_k(s,a)$ for the objective ($k=0$) and each user's QoS constraint ($k=1,\\dots,K$); the attention weight $\\alpha_{k,k',t} = \\mathrm{softmax}(k_{k,t}^{\\top} q_{k',t})$ lets each user's Q-estimate attend to the most relevant other users. These Q-estimates enter the policy-gradient estimates (4), so the accuracy of the Q-approximation directly controls whether the surrogate updates track the true gradient.","core_discovery":"The authors propose CAAC, a constrained actor-critic algorithm in which the actor updates a policy network by solving convex quadratic surrogates of the non-convex long-run average objective and QoS constraints (CSSCA), while the critic approximates the needed Q-functions with lightweight attention-enhanced Q-networks. The Q-networks share an embedding layer and an attention layer across users, reducing parameter count and letting each user's value estimate focus on the most relevant neighbors. Policy-gradient estimates (4) use these Q-networks in place of an environment model, so the algorithm is model-free. The paper shows on a simulated MIMO downlink that CAAC converges faster and to better operating points than PPO-Lag, CPO, and the earlier SCAOPO, reaching 1.39 W average power and 2.98% average QoS violation at $K=8$ users.","pith_inferences":["One extension the paper does not test is whether the same attention-based critic transfers to other per-user scheduling decisions, such as resource-block assignment or beamforming, where analogous per-user constraints hold.","The reported advantage is measured at one operating point ($B=200$, $T_{\\mathrm{cri}}=10$); the margin over baselines might shrink if channel dynamics become faster, since the Q-approximation would need more samples to stay accurate.","The shared embedding layer assumes users have homogeneous state structure; in a network with heterogeneous user hardware or mobility, the shared embedding might need to be split, which could reduce the claimed scalability."],"forward_implications":["If CAAC's reported performance holds, WMMSE precoding can be made QoS-aware without changing the underlying WMMSE solver: the algorithm sits above it, choosing priorities and power, so it can be retrofitted to existing systems.","The paper argues the attention-based critic scales better than independent Q-networks; if true, CRL-based scheduling becomes more practical as the number of users grows because parameter size and training time no longer grow as quickly.","Because CAAC is model-free, it can be deployed without a channel or traffic model, which matters in 6G where such models are uncertain.","The algorithm's structure allows new QoS types to be added by redefining the utility functions $U(n,k,t)$ without changing the learning procedure."],"supporting_citations":[{"why":"Supplies the WMMSE precoding scheme whose priorities and power are being scheduled.","marker":"[3]"},{"why":"Provides the greedy priority scheduling baseline that CAAC is compared against.","marker":"[6]"},{"why":"Defines the TRPO-Lag/PPO-Lag baselines that CAAC must beat.","marker":"[11]"},{"why":"Is the CPO baseline, an approximate-convex CRL method that cannot guarantee feasibility for non-convex constraints.","marker":"[12]"},{"why":"Provides the SCAOPO algorithm and the convergence lemma that CAAC's CSSCA argument relies on; also a baseline.","marker":"[13]"},{"why":"Justifies the temporal-difference updates that train the Q-networks.","marker":"[14]"},{"why":"Provides the channel model used in the simulations.","marker":"[15]"}],"fun_headline_variants":["CAAC: RL tunes WMMSE power and priorities for QoS","RL precoding scheduler cuts power, boosts QoS","Attention-critic RL beats baselines in WMMSE power","Constrained RL for WMMSE hits 1.39 W, 2.98% violations"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The results depend on the Q-networks being accurate enough after just 200 environment samples and 10 learning updates per iteration; if that accuracy is not achieved, the policy-update directions become biased and the convergence guarantee collapses.","fun_headline_variants_meta":{"raw":{"variants":["CAAC: RL tunes WMMSE power and priorities for QoS","RL precoding scheduler cuts power, boosts QoS","Attention-critic RL beats baselines in WMMSE power","Constrained RL for WMMSE hits 1.39 W, 2.98% violations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000203,"raw_usage":{"total_tokens":1377,"prompt_tokens":929,"completion_tokens":448,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":545,"completion_tokens_details":{"reasoning_tokens":370}},"tokens_in":545,"tokens_out":448,"duration_ms":4679,"temperature":1.0,"reasoning_tokens":370,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:45:08.958560+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the $K=8$ experiment with $B$ reduced from 200 to 50 (or $T_{\\mathrm{cri}}$ from 10 to 2) while keeping everything else fixed: if CAAC's QoS violation rises to the baselines' level or its power increases, the claimed advantage comes from the critic's sample budget rather than from the algorithm's design. Alternatively, repeat the simulation with 20 random seeds and check whether the 1.39 W and 2.98% figures are stable; if the variance is large, the reported advantage is not robust.","supporting_citations":[{"cited_title":"Rethinking WMMSE: Can its complexity scale linearly with the number of BS antennas?","cited_arxiv_id":null,"evidence_quote":"Supplies the WMMSE precoding scheme whose priorities and power are being scheduled."},{"cited_title":"Low complexity multiuser mimo scheduling for weighted sum rate maximization,","cited_arxiv_id":null,"evidence_quote":"Provides the greedy priority scheduling baseline that CAAC is compared against."},{"cited_title":"Constrained policy optimization,","cited_arxiv_id":null,"evidence_quote":"Is the CPO baseline, an approximate-convex CRL method that cannot guarantee feasibility for non-convex constraints."},{"cited_title":"Successive convex approx- imation based off-policy optimization for constrained reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Provides the SCAOPO algorithm and the convergence lemma that CAAC's CSSCA argument relies on; also a baseline."},{"cited_title":"A finite-time analysis of Q-learning with neural network function approximation,","cited_arxiv_id":null,"evidence_quote":"Justifies the temporal-difference updates that train the Q-networks."},{"cited_title":"Multi-agent deep reinforcement learning for dynamic power allocation in wireless networks,","cited_arxiv_id":null,"evidence_quote":"Provides the channel model used in the simulations."}],"review_version":1}