{"id":"95d2628d-02f4-49b1-93c0-2f62d455d5bf","arxiv_id":"2505.12902","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A GNN-augmented PPO agent that allocates D2D transmit power from channel and queue information reduces average packet delay in simulation and transfers to unseen network sizes.","lead":"This paper trains a central controller to set transmit powers in device-to-device wireless networks, using a graph neural network inside a reinforcement learning agent to reduce how long packets wait in queues. It is worth a skim as a concrete, incremental example of combining graph neural networks with reinforcement learning for a practical low-latency resource allocation problem.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported simulation evidence is internally inconsistent: Table II gives 1.096 ms for the default six-pair setting while Tables III/IV give 5.374 ms, and the claimed 56% gain over ITLinQ is only 24% from those tables; the empirical basis of the central claim is therefore not self-consistent.","rationale":"I considered the reader's weakest assumption, the Proposition 1 reward equivalence. That is a real gap: J_i[n] is undefined, Eq. (27) uses approximations, and terminal backlog is handwaved. But it is not the most load-bearing issue for the abstract's empirical claim. Even an approximate queue-length proxy can in practice reduce delay; the simulation section is what must demonstrate this. The numerical contradictions in the simulation section are direct, checkable, and cannot all be true under the stated default settings. They make the quantitative support for 'outperforms baseline methods' unreliable. This is consistent with the reader's CONDITIONAL verdict rather than a reason to reject the method outright, because the inconsistencies may be typographical or due to unreported configuration differences; a code release and rerun can settle them. I therefore keep the reader's verdict unchanged. The reward-equivalence proof should still be tightened as a secondary condition.","tokens_in":22588,"tokens_out":16301,"duration_ms":181573,"concrete_test":"Obtain or reconstruct the simulator and run the described default configuration (M=6, 500m×500m, λ=3 packets/ms, L=4000 bits, T_sum=300 slots, feature dimension 64, K=2, PPO over 2000 episodes) with at least 10 random seeds. Report the mean and standard deviation of steady-state average delay for the proposed method and ITLinQ. Then compare: if the value is ≈5.374 ms, Table II and the 56% claim are inconsistent with the reproduced result; if it is ≈1.096 ms, Tables III/IV are inconsistent. Either way the discrepancy is resolved only if the authors identify the distinct configuration behind each table (e.g., different λ or episode length) or release code demonstrating that one set of numbers is reproducible.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is empirical: the proposed method reduces average delay and outperforms baselines. The numbers supporting it do not cohere. With default parameters (M=6, λ=3, T_sum=300, feature dimension 64, K=2), Table II reports converged average delay 1.096 ms, whereas Tables III and IV report 5.374 ms for the same trained configuration. Section IV.C states 'approximately a 56% improvement over the ITLinQ method,' but the M=6 rows of Tables III/IV give 5.374 ms for the proposed method and 7.078 ms for ITLinQ, an improvement of about 24%. The duplicated M=6 rows in Tables III and IV also disagree on Max power (12.664 vs 14.660 ms). Figure 4's captions and text are swapped relative to the axes. Unless hidden setting differences explain these (none are stated), at most one of these reported numbers can be correct. Since the headline result is a quantitative comparison, this internal inconsistency undermines the load-bearing evidence for the claim; a reader cannot determine which reported delay actually corresponds to the described default experiment.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a centralized power-allocation method for D2D networks that combines proximal policy optimization (PPO) with graph neural network layers embedded in both the actor and critic. The state includes channel gains, packet delay, backlogged packets, and transmitted packets; the reward is the negative sum of buffer lengths, which Proposition 1 claims is asymptotically equivalent to minimizing average delay. The authors report simulation results showing lower average delay than ITLinQ, WMMSE, max power, random power, plain PPO, PPO+GraphSAGE, and TD3+ASAP, along with fairness improvements and scalability/generalization to unseen network sizes, densities, and user distributions.","tokens_in":22840,"tokens_out":5579,"duration_ms":60749,"significance":"If the reported results are correct, the paper offers a timely integration of GNNs inside an RL actor-critic loop for delay-aware power control, rather than using GNNs only as a preprocessing feature extractor. The evaluation is broader than many RL resource-allocation papers: it covers varying network sizes, densities, and user distributions, and the qualitative trends in Tables III–V and Figures 6–11 are consistent with the claimed advantages. However, the paper does not ship code or machine-checked proofs, the theoretical equivalence in Proposition 1 is explicitly approximate, and the empirical tables contain inconsistencies that currently prevent a reader from trusting the headline numbers. The central idea is defensible, but the numerical evidence needs substantial reconciliation before the claims can be accepted.","major_comments":[{"comment":"The default configuration described in Section IV.A (M=6, λ=3, T_sum=300, feature dimension 64, K=2) is reported with average delay 1.096 ms in Table II but 5.374 ms in Tables III, IV, and V. The same M=6 row also gives inconsistent Max power values (12.664 ms in Table III versus 14.660 ms in Table IV). Since Section IV.A states that all parameters take the Table I values unless a figure or table explicitly overrides them, and no such override is stated for these tables, at most one of these numbers can describe the default experiment. The authors must reconcile these figures or explicitly identify the setting differences that produce the discrepancy; without this, the numerical basis of the paper's central empirical claim is not self-consistent.","section":"§IV.A, Tables II–V"},{"comment":"The text states that the proposed method achieves 'approximately a 56% improvement over the ITLinQ method.' Using the M=6 rows of Tables III and IV, the proposed method gives 5.374 ms and ITLinQ gives 7.078 ms, which corresponds to about 24.1% reduction, not 56%. Please correct the percentage or provide the data underlying the 56% figure.","section":"§IV.C"},{"comment":"Proposition 1 and its proof in Eqs. (27)–(28) rely on two unstated approximations. First, the cumulative sum of buffer lengths is equated to the sum of queueing delays, which is only exact for stable lossless discrete-time queues with appropriate boundary handling; the proof does not address truncation at the end of the communication duration or the effect of packets still in buffers. Second, transmission delay is discarded because it is less than 1 ms, even though the objective in (6) explicitly includes transmission delay. Additionally, J_i[n] in Eq. (8) is never precisely defined relative to q_i[n] in Eq. (5) or Δ_i[n] in Eq. (14). Either justify these approximations formally or restate Proposition 1 as a heuristic reward-shaping argument rather than an equivalence claim.","section":"§III.C and Appendix A"},{"comment":"Tables III–V and Figures 6–11 report average-delay values without error bars, confidence intervals, or the number of random seeds used. Since channel realizations and Poisson arrivals create noticeable run-to-run variability, single point estimates per configuration are insufficient to establish the claimed quantitative superiority, scalability, and generalization. Please report means and variances over multiple independent seeds, especially for the M=6 default configuration that appears in three different tables.","section":"§IV.E and §IV.F"}],"minor_comments":[{"comment":"The captions of panels (a) and (b) appear swapped relative to their axes: panel (a) plots average delay versus episode but is captioned 'training speed,' while panel (b) plots return versus episode but is captioned 'average delay.' The main text also refers to these panels in a way that is inconsistent with the captions.","section":"Fig. 4"},{"comment":"The same figure number 'Fig. 6' is used for both the baseline performance comparison and the subsequent ablation study; please renumber so that each experiment refers to a distinct figure.","section":"§IV.C"},{"comment":"The sentence 'we compare in Fig. 11 the instantaneous rates of all D2D pairs ... in the same episode as Fig. 7' appears to reference the wrong figure; the episode is presumably the one shown in Fig. 10, which depicts remaining packets per buffer for the proposed method and WMMSE.","section":"§IV.D"},{"comment":"The notation for buffer occupancy is inconsistent: J_i[n], q_i[n], and Δ_i[n] are used for conceptually overlapping quantities, and the time indexing (beginning of slot, end of slot, before/after arrivals) is not stated. Please define each quantity exactly and use consistent notation throughout.","section":"§III.C, Eqs. (8), (13)–(15)"},{"comment":"The algorithm input refers to N_tra but the training loop uses N_t, and the number of agent update times K is not defined in the input list; also 'experience date' should be 'experience data.'","section":"Algorithm 1"},{"comment":"References [43]–[45] do not appear to be cited in the body of the manuscript; please check the citation list.","section":"References"},{"comment":"There are typographical errors such as 'emdeddings,' 'lagers,' and 'date' that should be corrected in a revision.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The central architectural idea and the qualitative trends are plausible, but the numerical inconsistencies in Tables II–V and the informal nature of Proposition 1 mean that the paper's current empirical and theoretical evidence does not yet support its headline claims. I recommend asking the authors to reconcile all reported numbers, add error bars over seeds, and either prove or clearly weaken the reward-equivalence statement before further consideration."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know. First, the proposed integration is sensible: PPO with GNN layers embedded in both actor and critic, a state that includes channel, queue length, delay, and transmitted packets, and a central controller. That is a reasonable engineering recipe, and the qualitative trends in the paper (fairer buffer occupancy, transfer to unseen network sizes and densities) are plausible. Second, the reported numbers do not cohere. Table II gives 1.096 ms average delay for the default six-pair, feature-dimension-64 setting, while Tables III and IV list 5.374 ms for the same trained six-pair configuration. The text says the method achieves about 56% improvement over ITLinQ, but the tables give roughly 24% (5.374 vs 7.078 ms). The duplicated max-power row also disagrees across Tables III and IV (12.664 vs 14.660 ms). At most one set of these numbers can describe the default experiment, and the discrepancy sits right on top of the central quantitative claim. I checked the stress-test note against the full text and it holds up.\n\nWhat is actually new: embedding GNN layers inside the PPO actor and critic and training them jointly, rather than using GNNs as a cascaded feature extractor, is a modest but legitimate extension over prior GNN+RL work. The delay and backlog augmented state is not new, nor is queue-length-based reward shaping, but the specific combination applied to D2D power allocation, with generalization tests across network sizes and densities, is a useful engineering data point. The paper also compares against a reasonable set of baselines, including ITLinQ, WMMSE, PPO, PPO+GraphSAGE, and TD3+ASAP, and the fairness analysis using per-buffer occupancy is illustrative.\n\nWhere it is soft: beyond the table inconsistency, there are no error bars or multiple-seed statistics, and no code is released, so the reported gains are hard to verify. Proposition 1, which connects the reward to average delay, uses ‘≈’ steps and implicitly assumes stable, lossless queues; J_i[n] is never defined precisely with respect to arrival/departure timing. The reward-delay equivalence is plausible but not established. Also, Figure 4’s captions and axes are swapped (a/b), which is minor but symptomatic of rushed reporting.\n\nWho this is for: engineers working on low-latency D2D or V2I power allocation who want a concrete GNN-RL design to build on. The paper is readable and the method is implementable. It deserves a serious referee because the idea is reasonable and the flaws look addressable, but the authors must reconcile the tables, add error bars, tighten Proposition 1, and fix the figure captions before the empirical claims can be taken at face value. If I were the editor, I would send it out, and I would expect the reviewers to insist on consistent, seed-averaged numbers.","headline":"Useful GNN+PPO recipe for D2D delay-aware power allocation, but the paper's own tables disagree about the headline delay numbers, so the empirical claims need repair before they can be trusted.","tokens_in":23392,"tokens_out":3352,"would_cite":false,"duration_ms":36358,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A graph-embedded PPO agent allocates D2D transmit powers to cut average delay by about 56% over the ITLinQ baseline in simulation.","keywords":["device-to-device communication","power allocation","graph neural networks","reinforcement learning","proximal policy optimization","average delay","user fairness","scalability"],"falsifier":"Run the identical training procedure with finite buffers that drop packets on overflow and with packet lengths large enough that transmission delay is not negligible; if the policy selected by the surrogate reward no longer minimizes measured average delay, or if the identity in equation (27) does not hold numerically, the reward-equivalence claim falls.","tokens_in":22392,"feed_emoji":"📡","tokens_out":6732,"duration_ms":63421,"temperature":0.7,"pith_summary":"The paper tries to establish that a centralized proximal policy optimization (PPO) agent, with graph neural network (GNN) layers embedded directly into both its actor and critic networks, can learn a transmit-power allocation policy for device-to-device (D2D) networks that minimizes average packet delay while keeping service fair across users. The state fed to the agent goes beyond channel gains to include per-buffer packet delay, backlogged packet count, and packets transmitted, and the reward is the negative of the total number of packets accumulated in all buffers. If the claims hold, this yields a delay-aware power controller that in simulation beats rate-focused baselines such as ITLinQ and WMMSE, improves fairness, and transfers to unseen network sizes, densities, and user distributions without retraining. The paper also argues that the surrogate reward is asymptotically equivalent to the average-delay objective and that the learned policy is permutation-equivariant.","feed_headline":"Graph-trained agent cuts D2D delay by 56% in simulation","feed_subtitle":"A PPO agent with graph layers uses queue states to beat rate-maximizing baselines on delay and fairness.","key_machinery":"The engine of the approach is a bidirectional GNN–PPO architecture. The D2D network is modeled as a directed graph whose vertices are the communication pairs and whose edges are interference links, with normalized channel gains as edge features and, per node, a proportional-fairness ratio, a normalized delay, a normalized backlog count, and a normalized transmitted-packet count as initial features. Message passing follows the ASAP local extremum operator, and the final node embeddings feed both the actor (which outputs a multivariate Gaussian over transmit powers, clipped to [0, Pmax]) and the critic. The reward is R_n = -Σ_i J_i[n], where J_i[n] is the number of accumulated packets in buffer i at slot n. Proposition 1 connects this reward to the average-delay objective via the identity that summing per-slot buffer occupancy over time equals the sum of per-packet queueing delays, while Proposition 2 guarantees permutation equivariance, the property that makes the policy transferable to arbitrary node orderings.","core_discovery":"On its own terms, the central discovery is that a delay-aware state representation plus a GNN-augmented PPO agent can solve the average-delay power allocation problem in a way that rate-maximizing methods cannot. In simulations with six D2D pairs in a 500 m by 500 m cell, the proposed agent achieves about a 56% reduction in average delay over the best conventional baseline (ITLinQ), transmits more packets (5261 vs 5144 for ITLinQ and 4639 for WMMSE over 300 slots), and leaves fewer packets in buffer (142 vs 271 and 834). The agent maintains low 95th-percentile delay, the metric associated with fairness, and keeps rates fluctuating across users rather than letting weak users starve. Two formal results underpin the method: Proposition 1 states that maximizing the cumulative reward (negative accumulated buffer packets) is asymptotically equivalent to minimizing average delay, and Proposition 2 proves that the GNN–RL policy is permutation-equivariant, which is what allows the trained policy to generalize to different numbers of nodes without retraining.","pith_inferences":["The paper positions its centralized controller as an upper bound; a natural next test is whether a decentralized variant with local message passing can approach the same delay–fairness trade-off under partial observability.","The reward equivalence in Proposition 1 is approximate; before deployment, the gap between surrogate reward and true delay should be measured under heavier traffic, finite buffers, and non-Poisson arrivals, which the simulations do not stress.","The same graph encoding of interference topology could be reused for other per-link decisions, such as sub-band or time-slot assignment, since the architecture only requires node and edge features.","The reported 56% improvement is over ITLinQ in one simulation configuration; the robustness claim would be stronger if the baselines were tuned for delay rather than rate, as WMMSE and ITLinQ were not designed for queue-aware control."],"forward_implications":["A single trained agent can be deployed on networks with 12, 24, and even 54 D2D pairs at similar density without retraining, with average delay climbing only mildly (5.374 ms trained at 6 pairs, 6.319 ms at 54 pairs).","Rate-maximizing resource allocation is the wrong objective for delay: WMMSE leaves 834 packets buffered after 300 slots and its average delay degrades steeply as networks densify.","Including buffer delay, backlog, and transmitted-packet counts in the state is what lets the agent adapt power to queue states, preventing the starvation that fixed or rate-maximizing policies produce.","The permutation-equivariance guarantee means deployment does not require re-indexing users when the network changes, removing a practical obstacle to transferring learned policies."],"supporting_citations":[{"why":"Supplies the GNN backbone, including the local extremum message-passing operator used in the layer update (equation 17).","marker":"[41]"},{"why":"Provides the proximal policy optimization algorithm that trains the actor and critic networks.","marker":"[42]"},{"why":"Sets the D2D simulation parameters and the proportional-fairness ratio and normalized dB edge features used in the node/edge construction.","marker":"[28]"},{"why":"Provides the state-augmented GNN radio resource management approach and simulation configuration that the paper builds on.","marker":"[34]"},{"why":"Invoked to justify the reward as the negative number of accumulated buffer packets.","marker":"[14]"},{"why":"ITLinQ is the strongest conventional baseline, and the paper measures its main 56% delay reduction against it.","marker":"[46]"},{"why":"WMMSE is the rate-maximizing sum-rate baseline whose buffer starvation and zero-rate periods are contrasted with the proposed method.","marker":"[47]"},{"why":"References the PPO+GraphSAGE ablation baseline that competes with the chosen GNN backbone.","marker":"[39]"},{"why":"Provides the TD3 algorithm used in the TD3+ASAP baseline for the actor-critic comparison.","marker":"[48]"}],"fun_headline_variants":["Graph RL cuts D2D delay 56% in simulation","GNN-PPO power allocation cuts D2D delay by 56%","Delay-aware graph RL beats rate-maximizing baselines","Graph RL agent reduces D2D delay 56% with fairness","GNN-augmented PPO cuts D2D delay 56% in simulations"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole method leans on the claim that punishing the total number of packets sitting in buffers is effectively the same as minimizing true average delay, but that equivalence is proved with approximations and silently assumes buffers never overflow and transmission time is negligible compared with waiting time.","fun_headline_variants_meta":{"raw":{"variants":["Graph RL cuts D2D delay 56% in simulation","GNN-PPO power allocation cuts D2D delay by 56%","Delay-aware graph RL beats rate-maximizing baselines","Graph RL agent reduces D2D delay 56% with fairness","GNN-augmented PPO cuts D2D delay 56% in simulations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000575,"raw_usage":{"total_tokens":2733,"prompt_tokens":984,"completion_tokens":1749,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":600,"completion_tokens_details":{"reasoning_tokens":1655}},"tokens_in":600,"tokens_out":1749,"duration_ms":12426,"temperature":1.0,"reasoning_tokens":1655,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:24:55.260295+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the identical training procedure with finite buffers that drop packets on overflow and with packet lengths large enough that transmission delay is not negligible; if the policy selected by the surrogate reward no longer minimizes measured average delay, or if the identity in equation (27) does not hold numerically, the reward-equivalence claim falls.","supporting_citations":[{"cited_title":"ASAP: Adaptive structure aware pooling for learning hierarchical graph representations,","cited_arxiv_id":null,"evidence_quote":"Supplies the GNN backbone, including the local extremum message-passing operator used in the layer update (equation 17)."},{"cited_title":"Learning resilient radio resource management policies with graph neural networks,","cited_arxiv_id":null,"evidence_quote":"Sets the D2D simulation parameters and the proportional-fairness ratio and normalized dB edge features used in the node/edge construction."},{"cited_title":"State-augmented learn- able algorithms for resource management in wireless networks,","cited_arxiv_id":null,"evidence_quote":"Provides the state-augmented GNN radio resource management approach and simulation configuration that the paper builds on."},{"cited_title":"Joint power control and rate allocation enabling ultra-reliability and energy efficiency in SIMO wireless networks,","cited_arxiv_id":null,"evidence_quote":"Invoked to justify the reward as the negative number of accumulated buffer packets."},{"cited_title":"ITLinQ: A new approach for spectrum sharing in device-to-device communication systems,","cited_arxiv_id":null,"evidence_quote":"ITLinQ is the strongest conventional baseline, and the paper measures its main 56% delay reduction against it."},{"cited_title":"Graph neural network meets multi- agent reinforcement learning: Fundamentals, applications, and future directions,","cited_arxiv_id":null,"evidence_quote":"References the PPO+GraphSAGE ablation baseline that competes with the chosen GNN backbone."},{"cited_title":"Addressing function approxi- mation error in actor-critic methods,","cited_arxiv_id":null,"evidence_quote":"Provides the TD3 algorithm used in the TD3+ASAP baseline for the actor-critic comparison."}],"review_version":1}