{"id":"c2be58b0-8316-4f36-aa56-1fe273a788c6","arxiv_id":"2504.18937","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A two-agent DRL framework with centralized training and decentralized execution outperforms single-agent DDPG, DQN, no-IRS, and random-IRS baselines for joint power and IRS mirror control in a simulated NOMA-VLC network.","lead":"This paper applies a two-agent deep reinforcement learning controller to jointly set power levels and mirror orientations in an indoor visible-light network that uses NOMA. The authors report gains in energy efficiency and fairness over single-agent DRL and no-IRS baselines in simulation.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"DRL action layer enforces only a simplex, not the NOMA ordering α1≥…≥αK, so the SIC rates in Eq. (9) and the headline sum-rate gains may not be achievable.","rationale":"The reader's concern about Eq. (22) is valid: SEE and Jain's index are global scalars, not per-user quantities, so the reward as written is undefined and does not equal the P1 objective J·SEE. That alone would warrant a conditional verdict. However, I find a more fundamental problem upstream of the reward: the NOMA SINR formula in Eq. (9) presupposes a decoding order tied to power coefficients that are inversely ordered by channel gain (Eq. (8)). The DRL power agent uses a Softmax output, which only guarantees a simplex, not the monotone ordering. If the policy ever outputs unordered α, the rates used for training and evaluation do not correspond to any feasible SIC decoding, so the reported sum-rate improvements are not meaningful. This affects all compared DRL algorithms equally, but it invalidates the absolute performance claims and the claim that P1 is being solved. The paper could be repaired by projecting α onto the ordered simplex before computing rates, or by deriving the correct SINR for the actual decoding order, and then retraining. But as submitted, the central numerical results are not supported. I therefore recommend REJECT rather than CONDITIONAL, since the flaw is in the physical model used to generate all results, not merely in the scalarization of the objective. The reader's reward concern is a related but separate issue that would also need to be addressed in any revision.","tokens_in":16592,"tokens_out":10865,"duration_ms":121252,"concrete_test":"Re-implement the described environment and, after training with the paper's exact setups, record the actor's α output at every evaluation step for the 7×7 IRS case at P=2W. Count violations of α1≥…≥αK (with users sorted by Eq. (7)). If any substantial fraction of steps violates the ordering, recompute the sum rate using the correct SIC procedure (decode in descending α order with the actual channel gains, or equivalently use the general SINR for that decoding order) and compare with Fig. 7's 40.5 Mbps. If the two-agent DRL's 16.5% gain over DDPG changes by more than a few percent, the reported performance claim is an artifact of the invalid SINR formula.","verdict_should_be":"REJECT","load_bearing_attack":"The central performance claim rests on sum rates computed from the NOMA SINR in Eq. (9). That SINR assumes users are indexed by increasing channel gain and that power coefficients satisfy α1≥α2≥...≥αK (Eq. (8)). However, P1's constraints (18.a–18.e) do not include this ordering, and the power-allocation agent's actor ends in a Softmax layer, which only ensures αk≥0 and Σαk=1. No projection, sorting step, or penalty enforces Eq. (8). Consequently, whenever the explored or deployed policy outputs a non-monotone α vector, Eq. (9) does not describe the actual SIC outcome: the 'strongest signal with highest power' need not belong to the weakest user, so the interference terms Σ_{j=k+1}^K α_j are not the remaining undecoded signals. The rates in Eq. (10), the reward in Eq. (22), and the reported 40.5 Mbps / 16.5% improvements are then computed with an invalid model. The paper gives no evidence—such as a layer that sorts α, a penalty for inversions, or a check on trained trajectories—that the ordering holds, so the main numerical claims are unverified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper models an indoor IRS-assisted NOMA-VLC downlink with mobile users, formulates an optimization problem P1 that maximizes the product of spectral/energy efficiency (SEE) and Jain's fairness index under QoS, power, and IRS-orientation constraints, and then reformulates this problem as a Markov decision process. The proposed solution is a two-agent deep deterministic policy gradient (DDPG) algorithm with centralized training and decentralized execution, where one agent allocates NOMA power coefficients and the other controls IRS mirror yaw and roll angles. Simulation results are reported for sum rate, SEE, and fairness against single-agent DDPG, DQN, fixed-IRS, random-IRS, and no-IRS baselines, including a headline sum rate of 40.5 Mbps with a 7x7 IRS array. The main claimed contributions are the joint SEE-fairness objective, the CTDE two-agent DRL design, and the demonstration of significant gains over the baselines in a dynamic indoor OWC scenario.","tokens_in":16887,"tokens_out":4485,"duration_ms":47119,"significance":"If the numerical claims are correct, the paper would provide a useful demonstration that multi-agent CTDE DDPG can jointly manage power allocation and IRS orientation in a dynamic NOMA-VLC system, and it would extend the small body of work on DRL-based IRS control in OWC. The system model is reasonably detailed, the MDP formulation is explicit, and the comparison set includes several meaningful baselines. However, the central performance claims currently rest on two unverified assumptions: that the reward in Eq. (22) is a valid scalarization of P1, and that the power-allocation agent always produces NOMA power coefficients satisfying the ordering required by the SIC rates in Eq. (9). Because these assumptions are load-bearing for the reported 16.5% and 66.7% improvements, the significance of the paper cannot be fully assessed until they are resolved.","major_comments":[{"comment":"The reward is defined as sum_k SEE_{k,t} + sum_k J_{k,t} - lambda1*rho1 - lambda2*rho2, but SEE in Eq. (16) is a single global quantity R_T/P_total and Jain's fairness index J in Eq. (17) is also a single global function of the rate vector. No per-user quantities SEE_{k,t} or J_{k,t} are defined anywhere in the paper. Consequently Eq. (22) is not a well-defined scalarization of the objective J*SEE in P1, and the trained policies may be optimizing a different objective from the one stated in Eq. (18). The authors should either define a clear per-user decomposition of SEE and J, or rewrite the reward directly in terms of the global SEE and J, and explain how that reward relates to P1.","section":"Section IV.A, Eq. (22)"},{"comment":"The SINR expression in Eq. (9) is only valid when users are ordered by increasing channel gain and the power coefficients satisfy alpha_1 >= alpha_2 >= ... >= alpha_K as stated in Eq. (8). The power-allocation actor uses a Softmax output layer, which guarantees only alpha_k >= 0 and sum_k alpha_k = 1; no sorting, projection, or penalty enforces the monotonicity in Eq. (8). If the policy produces a non-monotone alpha vector, the interference terms in Eq. (9) do not correspond to the actual SIC decoding order, so the rates in Eq. (10), the reward in Eq. (22), and the headline sum-rate gains in Figs. 6 and 7 are computed under an invalid model. The authors should enforce the ordering in the action space or provide empirical evidence that trained policies satisfy Eq. (8), and recompute the comparisons if violations occur.","section":"Sections II.B and IV.B/V.B, Eqs. (8)-(10)"},{"comment":"The text states that the simulation uses five active users, K = 5, while Table I lists 'Number of users 8'. Since the number of users directly affects the sum rate, fairness, SEE, and the dimensions of the state and action spaces, this discrepancy is material to the reported results. The authors must clarify the actual K used in each figure and update the text or table consistently; otherwise the numerical claims in Section V.C cannot be interpreted.","section":"Section V.A and Table I"},{"comment":"The penalty weights lambda_1 and lambda_2 in Eq. (22), which shape the reward and determine how strongly QoS and power constraints are enforced, are never reported in Table II or in the text. Similarly, the exploration noise standard deviation sigma used in the Gaussian exploration N_t ~ N(0, sigma^2) mentioned in Section IV.C is not given. Without these values the training procedure is not reproducible, and the reported constraint-satisfaction behavior and convergence results cannot be independently assessed.","section":"Tables I and II, Eq. (22)"}],"minor_comments":[{"comment":"The Fig. 7 caption describes baseline 2 as having a randomly selected IRS configuration, whereas Section V.C defines baseline 2 as a standard DDPG with each IRS mirror pointing to a specific direction; these descriptions should be aligned so the 44.1% comparison is unambiguous.","section":"Figure 7 caption and Section V.C"},{"comment":"Constraint (18.e) is written as -ϖ/2 <= ϑ_m <= π/2, which appears to contain a typo; the lower bound should likely be -π/2 to match the yaw constraint in (18.d).","section":"Section II.A, Eq. (18.e)"},{"comment":"There is a dangling 'Moreover,' immediately before the paragraph on the centralized critic update, and in Algorithm 1 the phrase 'soft update' is followed by 'donates' where 'denotes' is intended; these should be corrected.","section":"Section IV.C and Algorithm 1"},{"comment":"The complexity expressions such as O((2K+2M)+K) and O(3K+2M) appear to count network input/output dimensions rather than floating-point operations per update; the authors should clarify what complexity measure is being reported.","section":"Section IV.E"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely topic and the CTDE two-agent architecture is a reasonable design choice, but the two unverified assumptions identified above (reward scalarization and SIC ordering) directly affect the validity of the main numerical claims. These issues are fixable within the manuscript's scope, but they require more than minor editing, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid-idea paper with a serious gap in the implementation of the NOMA decoding order. The combination of a two-agent DRL controller for power allocation and IRS mirror orientation in dynamic NOMA-VLC is new, and the system model is built with care—RWP mobility, JT-CoMP, realistic power consumption, QoS constraints. The convergence plots and comparisons against DDPG, DQN, no-IRS, and random-IRS baselines are the right checks. If the paper were only about that, I'd call it a decent niche contribution.\n\nThe problem is that the central performance numbers are computed with a model that the policy may violate. Eq. (9) gives the SIC SINR under the ordering α1≥...≥αK after sorting users by channel gain (Eq. (8)). The power-allocation actor ends in a Softmax layer, which only enforces non-negativity and sum-to-one. There is no sorting, no projection, and no penalty that forces α to be monotone in the sorted user index. So whenever the policy outputs a non-monotone vector, Eq. (9) does not describe the actual SIC outcome, and the rates, rewards, and headline gains (40.5 Mbps, 16.5%, 66.7%) are computed on a false premise. The paper gives no evidence that trained trajectories respect the ordering. That is the load-bearing issue, not a cosmetic one.\n\nThere is also a reward mismatch in Eq. (22). SEE and Jain's index are global scalars, yet the reward sums them over K users as if they were per-user. And the penalty weights λ1, λ2 are never specified. These are fixable, but as written the reward is not a faithful scalarization of P1.\n\nMinor stuff: Section V.A says five active users, Table I says eight; no error bars; no code/data. Minor.\n\nIf the authors fix the reward, specify the penalties, and—more importantly—enforce the NOMA ordering in the action space or verify it holds on trained trajectories, this could become a credible engineering result. Right now I wouldn't cite it. But I would send it to peer review: the topic is relevant, the flaw is identifiable and fixable, and the system modeling has enough substance that a serious referee can give actionable feedback. For a reading group, maybe—good for discussing how DRL action spaces interact with physical-layer assumptions.","headline":"A well-built DRL-for-OWC paper whose central claim is not supported because the NOMA decoding order is never enforced in the action space.","tokens_in":17415,"tokens_out":4407,"would_cite":false,"duration_ms":44799,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Two-agent DRL lifts IRS-NOMA VLC sum rate 66.7% over no-IRS","keywords":["optical wireless communication","visible light communication","intelligent reflecting surface","non-orthogonal multiple access","deep reinforcement learning","multi-agent DDPG","energy efficiency","fairness"],"falsifier":"Take a small instance such as $K=2$ users and a $3\\times3$ IRS, compute the global SEE and Jain index from the resulting rates, and compare them with the reward terms in Eq. (22); if $\\sum_k\\mathrm{SEE}_k\\neq\\mathrm{SEE}$ or $\\sum_k J_k\\neq J$, retrain using the global objective and check whether the 40.5 Mbps result and the 16.5% and 66.7% gains persist. A cheaper check is to verify in every test episode that $\\sum_k\\alpha_k=1$ and that each user's rate stays above the 1 Mbps minimum.","tokens_in":16432,"feed_emoji":"📡","tokens_out":9224,"duration_ms":76372,"temperature":0.7,"pith_summary":"The paper tries to show that a two-agent deep reinforcement learning controller, built on centralized training and decentralized execution, can solve the joint problem of power allocation and IRS mirror orientation in an indoor NOMA-VLC network in real time. It models mobile users with time-varying demands and line-of-sight blockage, and maximizes the product of sum energy efficiency (SEE) and Jain's fairness index under QoS, power, and mirror-angle constraints. The proposed two-agent DDPG is reported to reach a sum rate of 40.5 Mbps with a $7\\times7$ mirror array, a 16.5% gain over single-agent DDPG and a 66.7% gain over no-IRS, with the fairness index converging near 0.97. If right, this makes IRS-assisted NOMA-VLC a practical real-time option for dynamic indoor optical wireless networks.","feed_headline":"Two-agent DRL lifts IRS-NOMA VLC sum rate 66.7% over no-IRS","feed_subtitle":"Centralized-training, decentralized-execution DDPG sets power and mirror angles in real time for mobile users.","key_machinery":"The central mechanism is a two-agent deep deterministic policy gradient (DDPG) architecture with centralized training and decentralized execution (CTDE). The power agent's actor outputs per-user NOMA coefficients through a softmax layer to respect $\\sum_k\\alpha_k=1$; the IRS agent's actor outputs mirror yaw and roll through a tanh layer to respect the $[-\\pi/2,\\pi/2]$ bounds. Each agent has its own actor, but the critics are centralized: each critic receives the global observation and both actions, so each agent learns its action-value while accounting for the other agent's decisions. Experience replay, target networks with soft updates, and Gaussian exploration noise stabilize training. This design is what carries the paper's claim that the joint continuous action space can be explored more effectively than with a single-agent DDPG.","core_discovery":"The paper's central claim is that coordinating two continuous-control DRL agents—one for NOMA power coefficients $\\alpha_k$, one for IRS mirror yaw and roll angles $(\\varphi_m,\\vartheta_m)$—through centralized critics that see both agents' observations and actions yields faster convergence and higher sum rate, energy efficiency, and fairness than a single global DDPG agent or fixed/random IRS configurations in a dynamic NOMA-VLC setting. The authors formulate the optimization as P1: maximize $J\\cdot\\mathrm{SEE}$ subject to per-user minimum rate, transmit power, coefficient-sum, and mirror-angle constraints, reformulate it as an MDP, and train with a reward $r_t=\\sum_k \\mathrm{SEE}_{k,t}+\\sum_k J_{k,t}-\\lambda_1\\rho_1-\\lambda_2\\rho_2$ plus penalties for QoS and power violations. In simulation they report that the two-agent algorithm reaches about 9.7 Mbits/Joule SEE, a $7\\times7$ IRS sum rate of 40.5 Mbps, and a fairness index around 0.97, beating single-agent DDPG by 16.5% in sum rate, randomly oriented IRS by 44.1%, and no-IRS by 66.7%.","pith_inferences":["The reward in Eq. (22) sums per-user SEE and per-user Jain terms, but the paper defines SEE and Jain as global quantities; checking whether that sum equals the global objective is a direct test of whether the reported 40.5 Mbps is optimizing P1.","The CTDE scheme should extend to larger mirror arrays or phase-shift metasurfaces, but the IRS agent's action dimension grows linearly with the number of elements, so scaling beyond the tested $7\\times7$ array would need verification.","A natural testable extension is to let an agent also choose NOMA user ordering or pairing, since the decoding order is fixed by channel gains in this work.","The comparison to baselines could be broadened to twin-delayed DDPG or multi-agent soft actor-critic, which often improve on vanilla DDPG in continuous control; the claimed margin over 'standard DRL' may depend on the baseline chosen."],"forward_implications":["If the reported gains hold, a pre-trained two-agent DDPG can serve as a real-time controller that adapts IRS mirror angles and user power coefficients on the fly, avoiding per-slot convex optimization.","The CTDE design separates the two control tasks during execution, so the power allocator and the IRS steering agent can run in parallel with low per-step inference cost.","The margin over randomly oriented IRS shows that continuous mirror steering, not just the presence of an IRS, drives the benefit in dynamic indoor optical channels.","The fairness index near 0.97 suggests that the joint reward can balance efficiency and fairness without a separate scheduling layer.","The stated polynomial complexity of successive convex approximation, versus the small feedforward networks used by the agents, is the paper's argument for real-time feasibility."],"supporting_citations":[{"why":"It supplies the mirror-array IRS channel model with specular reflections used for the NLoS gain in Eq. (2).","marker":"[5]"},{"why":"It provides the IRS-aided VLC resource-management setup and the mirror spacing assumptions adopted in the simulations.","marker":"[11]"},{"why":"It establishes DDPG for IRS-assisted secure VLC and is the continuous-control baseline the paper extends to two agents.","marker":"[18]"},{"why":"It uses DRL for IRS mirror optimization and beamforming in VLC, motivating the joint mirror-orientation control problem.","marker":"[29]"},{"why":"It motivates multi-agent deep reinforcement learning with centralized training and decentralized execution for wireless networks.","marker":"[31]"},{"why":"It defines the dimmable VLC capacity bound in Eq. (10), on which the rate, SEE, and fairness computations rest.","marker":"[37]"},{"why":"It defines the Jain fairness index used as the fairness objective in P1 and in the reward design.","marker":"[40]"}],"fun_headline_variants":["Coordinated DRL sets power and IRS angles, 66.7% sum-rate gain","Real-time two-agent DRL improves IRS-NOMA VLC energy efficiency and fairness","DRL agents jointly control power and mirrors, lifting VLC sum rate 66.7%","Two-agent deep RL beats single-agent DDPG by 16.5% in IRS-NOMA VLC","Faster adaption: two-agent DRL for dynamic IRS-NOMA VLC, 66.7% gain"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes that the per-user reward in Eq. (22) is a faithful scalarization of the constrained objective P1, so that maximizing the summed per-user SEE and Jain terms with penalties is equivalent to maximizing $J\\cdot\\mathrm{SEE}$; if that equivalence fails, the trained policies may be optimizing a different objective than the one reported.","fun_headline_variants_meta":{"raw":{"variants":["Coordinated DRL sets power and IRS angles, 66.7% sum-rate gain","Real-time two-agent DRL improves IRS-NOMA VLC energy efficiency and fairness","DRL agents jointly control power and mirrors, lifting VLC sum rate 66.7%","Two-agent deep RL beats single-agent DDPG by 16.5% in IRS-NOMA VLC","Faster adaption: two-agent DRL for dynamic IRS-NOMA VLC, 66.7% gain"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000414,"raw_usage":{"total_tokens":2185,"prompt_tokens":1040,"completion_tokens":1145,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":656,"completion_tokens_details":{"reasoning_tokens":1021}},"tokens_in":656,"tokens_out":1145,"duration_ms":9153,"temperature":1.0,"reasoning_tokens":1021,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:05:45.069528+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a small instance such as $K=2$ users and a $3\\times3$ IRS, compute the global SEE and Jain index from the resulting rates, and compare them with the reward terms in Eq. (22); if $\\sum_k\\mathrm{SEE}_k\\neq\\mathrm{SEE}$ or $\\sum_k J_k\\neq J$, retrain using the global objective and check whether the 40.5 Mbps result and the 16.5% and 66.7% gains persist. A cheaper check is to verify in every test episode that $\\sum_k\\alpha_k=1$ and that each user's rate stays above the 1 Mbps minimum.","supporting_citations":[{"cited_title":"Visible light communications via intelligent reflecting surfaces: Metasurfaces vs mirror arrays,","cited_arxiv_id":null,"evidence_quote":"It supplies the mirror-array IRS channel model with specular reflections used for the NLoS gain in Eq. (2)."},{"cited_title":"Joint resource management for intelligent reflecting surface–aided visible light communications,","cited_arxiv_id":null,"evidence_quote":"It provides the IRS-aided VLC resource-management setup and the mirror spacing assumptions adopted in the simulations."},{"cited_title":"Drl- based irs-assisted secure visible light communications,","cited_arxiv_id":null,"evidence_quote":"It establishes DDPG for IRS-assisted secure VLC and is the continuous-control baseline the paper extends to two agents."},{"cited_title":"Drl-based irs-assisted secure hybrid visible light and mmwave communications,","cited_arxiv_id":null,"evidence_quote":"It uses DRL for IRS mirror optimization and beamforming in VLC, motivating the joint mirror-orientation control problem."},{"cited_title":"A quantitative measure of fairness and discrimination,","cited_arxiv_id":null,"evidence_quote":"It defines the Jain fairness index used as the fairness objective in P1 and in the reward design."}],"review_version":1}