Pith. sign in

REVIEW 5 major objections 4 minor 15 references

Joint User Priority and Power Scheduling for QoS-Aware WMMSE Precoding: A Constrained-Actor Attentive-Critic Approach

T0 review · 5 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Pith's one-line read A constrained reinforcement learning algorithm, CAAC, schedules user priorities and power for WMMSE precoding, and in simulation achieves lower average power and fewer QoS violations than existing constrained-RL and heuristic methods.

desk verdict The attention-enhanced critic is a genuinely useful idea, but a sign error makes the rate constraint for delay-tolerant users vacuous, so the headline QoS comparison does not test what it claims. read the letter →

arxiv 2506.16074 v1 pith:TCIIJ3I5 submitted 2025-06-19 cs.LG

classification cs.LG
keywords constrainedreinforcementlearningWMMSEprecodinguserpriorityschedulingpowerallocationQoSconstraintsattentionmechanismsuccessiveconvexapproximation6Gwirelessnetworks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that constrained reinforcement learning can solve the user-priority and power-scheduling problem for WMMSE precoding, a standard method for multi-antenna downlink transmission that normally uses fixed priorities and power. The proposed CAAC algorithm models the scheduling task as a constrained Markov decision process and learns a policy that chooses priority weights and transmit power to minimize average power subject to per-user QoS constraints. In simulated MIMO downlinks with $K=8$ users, CAAC reports 1.39 W average transmit power and a 2.98% QoS violation level, outperforming PPO-Lag, CPO, SCAOPO, and greedy baselines. If these results hold, WMMSE precoding can be made QoS-aware and energy-efficient in a model-free way, which matters for 6G networks with heterogeneous traffic demands.

What carries the argument

The central machinery is a two-module optimization loop. The Actor uses CSSCA to build convex quadratic surrogate functions $\bar{f}_{k,i}(\theta) = \hat{f}_{k,i} + \hat{g}_{k,i}^{\top}(\theta-\theta_i) + \zeta_k \|\theta-\theta_i\|^2$ from sample estimates of the objective and constraint values and gradients, then solves problem (8) for the next policy parameters, falling back to the relaxed feasible problem (9) when (8) is infeasible. The Critic trains attention-enhanced Q-networks with shared embedding and attention layers to estimate $Q^{\pi_{\theta_i}}_k(s,a)$ for the objective ($k=0$) and each user's QoS constraint ($k=1,\dots,K$); the attention weight $\alpha_{k,k',t} = \mathrm{softmax}(k_{k,t}^{\top} q_{k',t})$ lets each user's Q-estimate attend to the most relevant other users. These Q-estimates enter the policy-gradient estimates (4), so the accuracy of the Q-approximation directly controls whether the surrogate updates track the true gradient.

What would settle it

Run the $K=8$ experiment with $B$ reduced from 200 to 50 (or $T_{\mathrm{cri}}$ from 10 to 2) while keeping everything else fixed: if CAAC's QoS violation rises to the baselines' level or its power increases, the claimed advantage comes from the critic's sample budget rather than from the algorithm's design. Alternatively, repeat the simulation with 20 random seeds and check whether the 1.39 W and 2.98% figures are stable; if the variance is large, the reported advantage is not robust.

Watch

Extended reading notes

Core claim

The authors propose CAAC, a constrained actor-critic algorithm in which the actor updates a policy network by solving convex quadratic surrogates of the non-convex long-run average objective and QoS constraints (CSSCA), while the critic approximates the needed Q-functions with lightweight attention-enhanced Q-networks. The Q-networks share an embedding layer and an attention layer across users, reducing parameter count and letting each user's value estimate focus on the most relevant neighbors. Policy-gradient estimates (4) use these Q-networks in place of an environment model, so the algorithm is model-free. The paper shows on a simulated MIMO downlink that CAAC converges faster and to better operating points than PPO-Lag, CPO, and the earlier SCAOPO, reaching 1.39 W average power and 2.98% average QoS violation at $K=8$ users.

Load-bearing premise

The results depend on the Q-networks being accurate enough after just 200 environment samples and 10 learning updates per iteration; if that accuracy is not achieved, the policy-update directions become biased and the convergence guarantee collapses.

Editorial extensions

If this is right

  • If CAAC's reported performance holds, WMMSE precoding can be made QoS-aware without changing the underlying WMMSE solver: the algorithm sits above it, choosing priorities and power, so it can be retrofitted to existing systems.
  • The paper argues the attention-based critic scales better than independent Q-networks; if true, CRL-based scheduling becomes more practical as the number of users grows because parameter size and training time no longer grow as quickly.
  • Because CAAC is model-free, it can be deployed without a channel or traffic model, which matters in 6G where such models are uncertain.
  • The algorithm's structure allows new QoS types to be added by redefining the utility functions $U(n,k,t)$ without changing the learning procedure.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One extension the paper does not test is whether the same attention-based critic transfers to other per-user scheduling decisions, such as resource-block assignment or beamforming, where analogous per-user constraints hold.
  • The reported advantage is measured at one operating point ($B=200$, $T_{\mathrm{cri}}=10$); the margin over baselines might shrink if channel dynamics become faster, since the Q-approximation would need more samples to stay accurate.
  • The shared embedding layer assumes users have homogeneous state structure; in a network with heterogeneous user hardware or mobility, the shared embedding might need to be split, which could reduce the claimed scalability.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes a constrained reinforcement learning algorithm, CAAC, for jointly scheduling user priority weights and transmit power for WMMSE precoding in a MIMO downlink. The actor uses a constrained stochastic successive convex approximation method, and the critic uses lightweight attention-enhanced Q-networks. The authors claim that CAAC achieves lower average transmit power and lower QoS violation than PPO-Lag, CPO, SCAOPO, greedy, and equal-priority baselines in simulations with 4 to 16 users. The paper also gives a brief convergence argument that relies on the authors' previous SCAOPO analysis.

Significance. If the performance claims hold, the paper would provide a practical model-free scheduler for QoS-aware WMMSE precoding, with a scalable critic architecture that addresses the growth of Q-network count with the number of users. The simulation setup is described in enough detail to be partially reproducible, and the comparison against multiple CRL baselines is useful. However, the theoretical novelty is limited because CSSCA is essentially a re-use of the authors' earlier SCAOPO method with an attention-augmented critic, and the convergence analysis is inherited from [13] rather than proved here. More importantly, the QoS constraint for delay-tolerant users is vacuous as written, which directly undermines the central claim of QoS-aware scheduling.

major comments (5)
  1. [Section II-B, Eq. (1) and Section III-A, Eq. (2)] The objective is stated inconsistently. Eq. (1) reads as a maximization of average transmit power, while the surrounding text says the goal is to minimize energy consumption. In Eq. (2) the reformulation is a minimization with C'_0(s_t,a_t)=R(s_t,a_t)=p_t, so the sign of the objective flips without explanation. Since the objective surrogate (7) and the reported power values in Section IV all presume minimization, the formal problem statement is self-contradictory and should be corrected, e.g., by replacing max with min in Eq. (1).
  2. [Section II-B and Section IV, QoS gap metric] The QoS constraint for delay-tolerant users is vacuous. With U(2,k,t)=-R_{k,t} and c_k=5 Mbps, the constraint lim (1/T) E[sum_t U(2,k,t)] <= c_k becomes -R <= 5, i.e., R >= -5 Mbps, which is always satisfied for positive rates. Correspondingly, the QoS gap metric in Section IV, [U(2,k,t)-c_k]_+ = [-R-5]_+, is identically zero for every delay-tolerant user. Thus the delay-tolerant half of the user population imposes no active constraint, and the reported '2.98% QoS violation' and the QoS comparisons in Figs. 3 and 4 do not test QoS-aware scheduling for rate-constrained users. The sign of c_k or of the inequality must be corrected, and the experiments must be rerun under a genuinely active rate constraint before the central claim can be evaluated.
  3. [Section III-B, Eqs. (8) and (9)] The feasible policy parameter set Theta is never defined. The surrogate problems (8) and (9) are solved over theta in Theta, and the convergence and feasibility statements in Section III-D depend on the properties of Theta, but the paper neither specifies Theta nor states assumptions on it. Please define Theta explicitly, including any boundedness or regularity conditions needed for the convergence argument.
  4. [Section III-D] The convergence claim is not established within this paper. The argument invokes the authors' previous [13, Lemma 3] without proof, and then asserts that TD updates make the Q-networks accurate enough for B=200 and T_cri=10. Since CSSCA is essentially SCAOPO with a different critic, the theoretical contribution is heavily self-referential. Please state the precise assumptions under which the Q-network approximation error is small, and either prove or cite a result that covers the finite-B, finite-T_cri regime used in the simulations. Without this, the bias in the policy-gradient estimates (4) is not controlled.
  5. [Section IV, Figs. 3 and 4] All simulation results are reported as averages over 5 random seeds, but no error bars, standard deviations, or per-seed curves are shown. For the near-tie comparisons among CRL baselines at small user counts, the claimed advantage of CAAC cannot be assessed statistically. Please report the spread across seeds, and ideally release code so that the results can be reproduced independently.
minor comments (4)
  1. [Section II-A] There is a typo: 'the obtrained precoder vector' should be 'the obtained precoder vector'.
  2. [Section III-D] The phrase 'unbias estimation' should be 'unbiased estimate'.
  3. [Section II-B] The symbol c_{k,b} is used in Section IV for the QoS threshold, but the problem formulation in Eq. (1) uses c_k. Please harmonize the notation.
  4. [Section III-C.2] The TD-update loop uses T_cri mini-batches but the notation 'tcri' is inconsistent; please use a clear subscript throughout, e.g., T_{cri} and t=1,...,T_{cri}.

Circularity Check

2 steps flagged · score 6.0 of 10

The QoS constraint for delay-tolerant users is vacuous as written (U=-R, c=+5 Mbps), making the reported QoS-violation numbers zero for that class by definition; convergence theory is imported wholesale from the authors' earlier SCAOPO paper.

  1. self definitional [Eq. (1), Section II-B, Section IV (QoS gap metric)]
    "2) If the k-th user is delay-tolerant, the utility function is defined as U(2, k, t) = −R_{k,t}; lim_{T→∞} 1/T E[Σ_{t=0}^{T−1} Σ_{n=1}^N δ(n−ϱ_k) U(n, k, t)] ≤ c_k,∀k; The remaining users are delay-tolerant, with a minimum transmission rate c_{k,b} = 5 Mbps; QoS gap(i) = 1/(KB(i+1)) Σ_{t=0}^{B(i+1)-1} Σ_{k=1}^K [Σ_{n=1}^N δ(n−ϱ_k)U(n,k,t)−c_k]_+"

    Substituting U(2,k,t) = −R_{k,t} into the constraint of Eq. (1) gives lim E[−Σ R_{k,t}] ≤ c_k; with c_{k,b} = 5 Mbps this is equivalent to E[R] ≥ −5 Mbps, an inequality that every nonnegative rate satisfies. In the QoS-gap metric, the delay-tolerant user term becomes [−R_{k,t} − 5]_+ ≡ 0 for every timeslot. Therefore the reported '2.98% QoS violation' and the QoS comparisons in Figs. 3–4 carry no information about the rate constraint for the delay-tolerant half of the user population. The central claim that CAAC satisfies QoS constraints for these users is true by construction of the utility and threshold, not by the learned policy, so the experimental evidence reduces to the sign choice in the definition.

  2. self citation load bearing [Section III-D (Convergence Analysis)]
    "According to our previous work [13, Lemma 3], the design of the step size η_i ensures that f̂_{k,i} asymptotically converges to f_k(θ_i). ... ĝ_{k,i} can also asymptotically converge to ∇J_k(θ_t), provided that the step size parameters satisfy κ1<κ2 [13, Lemma 3]."

    The paper's only convergence analysis for CAAC is inherited verbatim from [13, Lemma 3], a prior paper co-authored by An Liu, and the text does not prove or restate the lemma or verify its hypotheses for the new attention-enhanced Q-networks. The CAAC policy update (8)–(10) is the same convex-surrogate SCAOPO update, with Q-estimates replacing the Monte Carlo estimates of [13]. Thus the claimed convergence to a stationary point of the non-convex CMDP is not derived in this paper; it is carried by a load-bearing self-citation to the authors' own earlier result, without an independent check of assumptions such as boundedness of the feasible parameter set Θ or the Bellman approximation error of the attention critic.

full rationale

Two load-bearing reductions were found. First, for delay-tolerant users the QoS constraint and the QoS-gap metric are defined with U = −R and a positive threshold +5 Mbps, so the constraint reduces to R ≥ −5 Mbps and the per-user gap is identically zero. Consequently the headline QoS-violation numbers do not exercise the rate constraint for half the users, and the QoS-satisfaction claim for that class is guaranteed by definition rather than by the algorithm's behavior. This is a self-definitional circularity in the central experimental evidence. Second, the convergence guarantees in Section III-D are not derived in this manuscript; they are imported from the authors' own prior paper [13] via repeated 'Lemma 3' citations. Because the CSSCA policy update (8)–(10) is essentially the SCAOPO surrogate update with Q-network estimates, the theoretical contribution rests on a self-citation chain rather than a self-contained proof. The power-efficiency comparisons against PPO-Lag, CPO, SCAOPO, greedy, and equal-priority baselines are genuine external simulations, and no fitted parameter is renamed as a prediction, so the power-saving results are not fabricated by construction. I additionally note, as a non-circular correctness defect, that Eq. (1) displays 'max lim E[Σ p_t]' while the text says the goal is to minimize energy consumption, and Eq. (2) flips to 'min θ f0(θ)' with f0 based on p_t; this sign inconsistency is serious but is not itself a circular reduction. Overall, the paper exhibits partial circularity: the QoS evidence for one user class reduces by definition, and the theoretical convergence is load-bearing on the authors' prior self-citation.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on several hand-chosen hyperparameters (step sizes, surrogate constants, batch sizes, architecture sizes) and on convergence results borrowed from the authors' prior SCAOPO paper and from Q-learning theory. No new physical entities are introduced; the attention mechanism is an architectural component, not a postulated natural phenomenon.

free parameters (4)
  • Step-size exponents kappa1, kappa2, kappa3 = 0.6, 0.7, 0.3 (also reported as alpha_t=1/t^0.6, beta_t=1/t^0.7, gamma_t=1/t^0.3)
    Chosen by hand in Section IV with no sensitivity analysis; the convergence guarantee from [13] only requires kappa1 < kappa2, not these specific values.
  • Surrogate regularization constants zeta_k = not specified
    Appear in Eq (7) and Eq (8) as strictly positive constants defining the convex surrogates; no rule or value is given, so any implementation must pick them.
  • Batch size B and TD update count T_cri = B=200, T_cri=10
    Section IV specifies these values; the convergence argument requires them to be 'sufficiently large', which is assumed without verification.
  • Network dimensions (L1=2, L2=64, L3=32, hidden width 256) = as listed in Section IV
    Architecture hyperparameters chosen without ablation; the claimed efficiency gain of the attention design may depend on these specific sizes.
assumptions (5)
  • domain assumption Convergence of CSSCA policy updates follows from Lemmas in [13] (SCAOPO) when function and gradient estimates converge.
    Section III-D invokes [13, Lemma 3] to claim the objective reaches a stationary point and the iterates converge into the feasible region; this is borrowed, not proved, and its application to the WMMSE scheduling problem is not directly established.
  • domain assumption The TD updates with neural-network Q-functions converge to the true Q-functions when sample size and update count are large enough and representations are expressive.
    Section III-D cites [14] for Q-learning convergence; the simulation uses B=200 and T_cri=10, so the condition is assumed to hold approximately in practice.
  • standard math The policy gradient theorem applies, giving the gradient formula in Eq (4).
    Used in Section III-B for gradient estimation; standard in policy gradient methods, but requires regularity conditions on the policy and Q-functions that are not verified.
  • domain assumption The CMDP model with average-reward formulation and the QoS utility functions U(n,k,t) faithfully represent the scheduling problem.
    Section II defines delay-sensitive utility as L_{k,t}/a_k and delay-tolerant utility as -R_{k,t}; the constraint thresholds c_k are assumed to encode the required delay and rate QoS.
  • domain assumption The block fading channel model with Jakes correlation and the WMMSE rate expression describe the physical layer accurately.
    Section IV uses the model from [15]; all simulation conclusions are conditional on this model being representative of real deployments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Joint User Priority and Power Scheduling for QoS-Aware WMMSE Precoding: A Constrained-Actor Attentive-Critic Approach." pith.science (2026). https://pith.science/paper/TCIIJ3I5

@misc{pith2026250616074,
  author       = {Pith},
  title        = {Pith review of: Joint User Priority and Power Scheduling for QoS-Aware WMMSE Precoding: A Constrained-Actor Attentive-Critic Approach},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TCIIJ3I5}},
  note         = {Machine review of arXiv:2506.16074}
}
read the original abstract

6G wireless networks are expected to support diverse quality-of-service (QoS) demands while maintaining high energy efficiency. Weighted Minimum Mean Square Error (WMMSE) precoding with fixed user priorities and transmit power is widely recognized for enhancing overall system performance but lacks flexibility to adapt to user-specific QoS requirements and time-varying channel conditions. To address this, we propose a novel constrained reinforcement learning (CRL) algorithm, Constrained-Actor Attentive-Critic (CAAC), which uses a policy network to dynamically allocate user priorities and power for WMMSE precoding. Specifically, CAAC integrates a Constrained Stochastic Successive Convex Approximation (CSSCA) method to optimize the policy, enabling more effective handling of energy efficiency goals and satisfaction of stochastic non-convex QoS constraints compared to traditional and existing CRL methods. Moreover, CAAC employs lightweight attention-enhanced Q-networks to evaluate policy updates without prior environment model knowledge. The network architecture not only enhances representational capacity but also boosts learning efficiency. Simulation results show that CAAC outperforms baselines in both energy efficiency and QoS satisfaction.

Figures

Figures reproduced from arXiv: 2506.16074 by the authors.

Figure 1
Figure 1. Downlink transmission with heterogeneous QoS support. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The lightweight attention-enhanced Q-network architecture. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Performance versus the number of users. configured power control schemes with equal-priority (EP) scheduling, a greedy priority scheduling that adjusts [6] using QoSgap (i) as priorities, and three advanced Actor-Critic-based CRL algorithms (PPO-Lag, CPO, and SCAOPO) introduced in Section I. We also design a CAAC variant, denoted as CAAC (-), by replacing the lightweight attention-enhanced Q-networks with separate f… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 14 canonical work pages

  1. [13]

    Successive convex approx- imation based off-policy optimization for constrained reinforcement learning,

    C. Tian, A. Liu, G. Huang, and W. Luo, “Successive convex approx- imation based off-policy optimization for constrained reinforcement learning,”IEEE Trans. Signal Process., vol. 70, pp. 1609–1624, 2022

  2. [1]

    Multiagent deep-reinforcement- learning-based resource allocation for heterogeneous qos guarantees for vehicular networks,

    J. Tian, Q. Liu, H. Zhang, and D. Wu, “Multiagent deep-reinforcement- learning-based resource allocation for heterogeneous qos guarantees for vehicular networks,”IEEE Internet Things J., vol. 9, no. 3, pp. 1683– 1695, 2021

  3. [2]

    Energy-efficient star-ris-aided mu-mimo for next-generation urllc sys- tems,

    R. Deshpande, M. Katwe, K. Singh, M.-L. Ku, and D. Wing Kwan Ng, “Energy-efficient star-ris-aided mu-mimo for next-generation urllc sys- tems,”IEEE Trans. Wireless Commun., vol. 23, no. 11, pp. 17 807– 17 822, 2024

  4. [3]

    Rethinking WMMSE: Can its complexity scale linearly with the number of BS antennas?

    X. Zhao, S. Lu, Q. Shi, and Z.-Q. Luo, “Rethinking WMMSE: Can its complexity scale linearly with the number of BS antennas?”IEEE Trans. on Signal Process., vol. 71, pp. 433–446, 2023

  5. [4]

    Iterative algorithm induced deep-unfolding neural networks: Precoding design for multiuser MIMO systems,

    Q. Hu, Y . Cai, Q. Shi, K. Xu, G. Yu, and Z. Ding, “Iterative algorithm induced deep-unfolding neural networks: Precoding design for multiuser MIMO systems,”IEEE Trans. on Wireless Commun., vol. 20, no. 2, pp. 1394–1410, 2021

  6. [5]

    Robust wmmse-based precoder with practice- oriented design for massive mu-mimo,

    K. Wang and A. Liu, “Robust wmmse-based precoder with practice- oriented design for massive mu-mimo,”IEEE Wireless Commun. Lett., vol. 13, no. 7, pp. 1858–1862, 2024

  7. [6]

    Low complexity multiuser mimo scheduling for weighted sum rate maximization,

    G. Venkatraman, A. Tölli, J. Janhunen, and M. Juntti, “Low complexity multiuser mimo scheduling for weighted sum rate maximization,” in 2014 22nd European Signal Processing Conference, 2014, pp. 820–824

  8. [7]

    A survey on opportunistic scheduling in wireless communications,

    A. Asadi and V . Mancuso, “A survey on opportunistic scheduling in wireless communications,”IEEE Commun. Surv. Tutor ., vol. 15, no. 4, pp. 1671–1688, 2013

Show all 15 references
  1. [8]

    Generalizable resource scaling of 5G slices using constrained reinforce- ment learning,

    M. Sulaiman, M. Ahmadi, M. A. Salahuddin, R. Boutaba, and A. Saleh, “Generalizable resource scaling of 5G slices using constrained reinforce- ment learning,” inProc. NOMS IEEE/IFIP Netw. Operations Manage. Symp.IEEE, 2023, pp. 1–9

  2. [9]

    Clara: A constrained reinforcement learning based resource allocation framework for network slicing,

    Y . Liu, J. Ding, Z.-L. Zhang, and X. Liu, “Clara: A constrained reinforcement learning based resource allocation framework for network slicing,” in2021 IEEE International Conference on Big Data (Big Data). IEEE, 2021, pp. 1427–1437

  3. [10]

    Soft actor-critic- based multi-user multi-tti mimo precoding in multi-modal real-time broadband communications,

    Y . Huang, K. Chi, Q. Yang, Z. Yang, and Z. Zhang, “Soft actor-critic- based multi-user multi-tti mimo precoding in multi-modal real-time broadband communications,”IEEE Trans. Wireless Commun, vol. 23, no. 12, pp. 18 286–18 301, 2024

  4. [11]

    Benchmarking safe exploration in deep reinforcement learning,

    A. Ray, J. Achiam, and D. Amodei, “Benchmarking safe exploration in deep reinforcement learning,”arXiv preprint arXiv:1910.01708, vol. 7, no. 1, p. 2, 2019

  5. [12]

    Constrained policy optimization,

    J. Achiam, D. Held, A. Tamar, and P. Abbeel, “Constrained policy optimization,” inICML. PMLR, 2017, pp. 22–31

  6. [14]

    A finite-time analysis of Q-learning with neural network function approximation,

    P. Xu and Q. Gu, “A finite-time analysis of Q-learning with neural network function approximation,” inICML, 2020, pp. 10 555–10 565

  7. [15]

    Multi-agent deep reinforcement learning for dynamic power allocation in wireless networks,

    Y . S. Nasir and D. Guo, “Multi-agent deep reinforcement learning for dynamic power allocation in wireless networks,”IEEE J. Sel. Areas Commun., vol. 37, no. 10, pp. 2239–2250, 2019

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.