{"id":"46eb881c-28d8-407a-8f2a-4423a70407ae","arxiv_id":"2504.19499","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A graph reinforcement learning load balancer for multi-band O-RAN reduces QoS violations by 53% and increases the 5th percentile best-effort rate fourfold in simulation.","lead":"This paper trains a graph neural network and reinforcement learning agent to move users between cells in an Open RAN network, aiming to cut quality-of-service violations and improve throughput for best-effort traffic. In simulations, the method reduces QoS violations by 53% and quadruples the 5th percentile user rate versus simple signal-strength baselines.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reward proxy in Eq. (5) substitutes the minimum BE rate for the 5th-percentile coverage objective; with only ~9-17 BE UEs per deployment this is an extreme-order statistic whose alignment with cBE is unvalidated, leaving the 4x coverage claim not yet secured.","rationale":"The reader identified the same load-bearing assumption: the reward in Eq. (5) substitutes an extreme-order statistic for the coverage quantile in the optimization objective, without validating the alignment between the two. This is the most load-bearing concern because the second half of the central claim (the fourfold improvement in 5th-percentile BE goodput) depends directly on that substitution. The reported 53% QoS-dissatisfaction reduction is less affected by this issue, since the GBR reward terms in Eq. (5) are direct encodings of the QoS objective, not proxies. The suggested retraining experiment is a clean way to settle whether the coverage gain is an artifact of optimizing the minimum rate. I agree with the reader's conditional verdict: the proposal is coherent, the GNN-based dueling DQN formulation is plausible, and the simulation comparisons are meaningful, but the lack of validation of the reward proxy, combined with the absence of code, data, error bars, and comparisons to state-of-the-art DRL methods, means the evidence is not yet sufficient to accept the claims as robust.","tokens_in":10003,"tokens_out":4504,"duration_ms":53466,"concrete_test":"Retrain the agent under the same settings but replace r_min,BE in Eq. (5) with a smoothed empirical estimate of the 5th-percentile BE rate, computed by pooling per-UE average rates across a sufficiently large window of evaluation deployments (e.g., the last 100 episodes or all BE UEs in the current deployment over multiple TTI windows). Then re-run the Fig. 3 coverage comparison against the same max-SINR and max-RSRP baselines. If the fourfold 5th-percentile gain persists, the proxy concern does not alter the central claim. If the gain collapses or becomes substantially smaller, the reported coverage improvement is specific to optimizing the minimum BE rate rather than the claimed coverage objective.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Eq. (5), the reward term for BE traffic uses r_min,BE (the minimum average rate across BE UEs) in place of cBE, the 5th-percentile rate defined in Eq. (3), because the 5th percentile cannot be estimated from limited samples. With a 3:1 GBR:BE traffic ratio and U in {35, 42, ..., 70}, each deployment has only about 9-17 BE UEs. For such small samples, the minimum is the most extreme order statistic, not a robust quantile: a single UE with a temporarily poor average rate dominates the reward. The dueling DQN can therefore learn to offload that one outlier repeatedly, optimizing the minimum rather than the coverage distribution reported in Fig. 3. The paper asserts the substitution in Eq. (5) but provides no validation that maximizing the minimum average BE rate aligns with the 5th-percentile objective in Eq. (4a). Without such validation, the headline claim of a fourfold increase in the 5th-percentile BE goodput could be an artifact of optimizing a different, more brittle objective.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper develops a graph reinforcement learning (GRL) approach to quality-of-service (QoS)-aware load balancing in a multi-band Open RAN. The RAN is represented as a heterogeneous graph with UE and cell nodes, the load-balancing problem is cast as an MDP, and a GNN-based dueling DQN is trained to select handover actions for cell-edge UEs. The reward is constructed from the QoS objective for GBR traffic and a proxy for BE coverage. In a system-level simulator, the proposed method is compared against max-RSRP and max-SINR baselines, reporting a 53% reduction in QoS dissatisfaction rate and a fourfold increase in the 5th percentile BE goodput.","tokens_in":10153,"tokens_out":6923,"duration_ms":69506,"significance":"If validated, the paper would make a useful contribution by extending GNN-based DRL load balancing from BE-only scenarios to mixed GBR/BE traffic with per-UE QoS constraints, while exploiting permutation invariance and variable-size graph inputs. The problem formulation is coherent, the simulation environment is relatively detailed, and the algorithmic idea of processing subgraphs for scalability is sensible. However, the central performance claims currently rest on an unvalidated reward proxy and on an evaluation that lacks variance characterization and held-out test splits, so the significance is conditional on additional evidence.","major_comments":[{"comment":"The reward substitutes the minimum average BE rate, r_min,BE, for the coverage metric cBE defined in Eq. (3), with the justification that the 5th percentile cannot be estimated from limited samples. With U in {35, ..., 70} and a 3:1 GBR:BE traffic ratio, each deployment contains only about 9 to 17 BE UEs, so r_min,BE is the most extreme order statistic and can be dominated by a single UE with a poor average rate. The manuscript provides no evidence that maximizing this minimum aligns with the 5th-percentile objective in Eq. (4a); the fourfold coverage improvement in Fig. 3 could therefore be an artifact of optimizing a different, more brittle quantity. Please validate the proxy, for example by comparing against a reward based on a lower quantile estimated over a longer window, and report sensitivity of the learned policy to individual outlier BE UEs.","section":"Section III-B, Eq. (5) and Fig. 3"},{"comment":"All performance results are shown as single CDF curves without confidence intervals, multiple-seed statistics, or an explicit train/test separation. The text states that the model is trained over deployments with random UE counts, locations, and traffic parameters, but it does not report how many deployments were used, whether the evaluation deployments were held out from training, or how stable the reported averages (0.12 versus 0.29 and 0.26 in Fig. 2) are. The headline numbers in the abstract are point estimates; error bars and a clearly described evaluation protocol are needed to support them.","section":"Section IV, Figs. 2-4"},{"comment":"The comparison is limited to max-RSRP and max-SINR baselines. Because neither baseline is learning-based, the experiments do not isolate whether the gains come from the graph representation, the dueling DQN, or reinforcement learning in general. Adding a learning-based baseline, such as a DQN with a fixed-size MLP or the earlier GRL method in [12], would be necessary to support the stronger claim that the proposed architecture itself is responsible for the gains and to position the method relative to existing work.","section":"Section IV"},{"comment":"The description of the Q-network is incomplete and several training details are missing. After the mean pooling over cell nodes, the text does not specify how the advantage A(s,a) is computed for each feasible action; Eq. (12) assumes per-action advantages, but the action representation and the action branch of the network in Fig. 1 are not described. In addition, the value of alpha in the reward (5) is never reported, and the target-network update schedule ('after every 10 episodes') is not tied to a concrete number of gradient steps or environment interactions. These omissions prevent reproduction of the method and should be addressed.","section":"Section III-C and Section IV"}],"minor_comments":[{"comment":"The phrase 'QoS consideration are integrated' should be 'QoS considerations are integrated'.","section":"Abstract"},{"comment":"The set-difference notation 'U_GBR / U_ce' is ambiguous; use the standard backslash notation such as '\\mathcal{U}^{\\mathrm{GBR}} \\setminus \\mathcal{U}^{\\mathrm{ce}}'.","section":"Eq. (4a) and Eq. (5)"},{"comment":"The 3GPP reference is given as 'TS 23.50'; the correct specification number appears to be TS 23.501, and the year/version should be completed.","section":"Reference [15]"},{"comment":"The loop condition 'while episode ends do' is misleading; it should read 'while the episode has not ended do' or 'while not terminal do'.","section":"Algorithm 1, line 4"},{"comment":"The average bandwidth utilization per cell is reported as a single bar for each band; a fairness index or per-cell utilization distribution would make the load-balancing comparison more informative.","section":"Fig. 4"}],"recommendation":"major_revision","confidential_remarks":"To the editor: the reward-proxy issue in Section III-B is the main correctness risk; the coverage claim should not be accepted without either a direct validation of the proxy or a softening of the claim. The evaluation also needs multiple seeds, error bars, and a held-out test set. The paper is within scope and the proposed direction is plausible, but the current experimental evidence is not yet sufficient for a journal-level acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The main thing to know: this is a solid systems-oriented paper that extends the authors' earlier GRL load balancer to handle mixed GBR/BE traffic and multi-band O-RAN, with a coherent MDP formulation and a GNN-based dueling DQN. The headline gains (53% fewer QoS violations, 4x better 5th percentile BE rate) are plausible but not yet secured, because the eval lacks statistical rigor and because the reward uses an unvalidated proxy for the BE coverage metric.\n\nWhat's new and good: The paper is the first to combine GNN-based RL with per-UE GBR constraints, non-full buffer mixed traffic, and multi-band handovers. The state graph with UE/cell features and cell-to-cell edges is sensible, and the subgraph extraction algorithm (Alg. 2) is a practical step toward scaling to large RANs. Constraints (4b)-(4d) are reasonable for handover feasibility. The reward in (5) is a standard RL move: encode the objective you care about into a per-step difference (6). The reported numbers align with the figures, and the authors are open about the substitution of min average BE rate for the 5th percentile in (5).\n\nSoft spots, in order of importance. First, the reward proxy is load-bearing. With a 3:1 GBR:BE ratio and 35-70 UEs, you have only ~9-17 BE UEs per deployment. The minimum average rate is the most extreme order statistic; one UE with a temporarily bad average can dominate the reward. The paper does not validate that maximizing this min aligns with the 5th percentile objective in (4a). Without that check, the 4x coverage claim in Fig. 3 could be an artifact of optimizing a different objective. This is not a fatal flaw—it's fixable by reporting the correlation between the proxy and cBE, or by using a robust low-order statistic—but it's the first thing a referee should ask for.\n\nSecond, the evaluation is thin: no error bars or multiple seeds, no explicit train/test split, and no comparison with prior DRL-based QoS load balancing (e.g., [11]). The baselines are standard association rules, not state-of-the-art LB algorithms. Third, no code or data is provided, which limits reproducibility.\n\nThese are methodological weaknesses, not evidence that the approach is wrong. The central idea—using GRL to do QoS-aware load balancing with mixed traffic—is coherent and likely to work in practice. The paper reads like an honest engineering contribution from a team that knows the area.\n\nWho this is for: anyone building xApps for O-RAN RICs, or applying DRL/GNNs to RAN control. A serious referee could give it a fair shot, and the fixes are manageable. I wouldn't cite it as a definitive result until the author addresses the reward proxy and adds statistical rigor.\n\nRecommendation: send it to peer review, but flag the evaluation and the reward proxy as major required revisions.","headline":"Solid extension of the authors' GRL load balancer to mixed GBR/BE O-RAN, but the BE coverage reward proxy is unvalidated and the evaluation lacks statistical rigor.","tokens_in":10815,"tokens_out":3403,"would_cite":false,"duration_ms":32246,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a graph reinforcement learning load balancer, trained as a GNN-based dueling DQN, reduces QoS dissatisfaction by more than 53% and quadruples the 5th-percentile goodput of best-effort users in a simulated multi-band…","keywords":["graph reinforcement learning","load balancing","open radio access network","QoS","dueling deep Q network","graph neural network","GBR traffic","best effort traffic"],"falsifier":"Run the same training pipeline in the simulator but compute the reward from a quantile estimate of the BE rate (e.g., the 5th percentile over a long sliding window) instead of the minimum average rate, holding all other settings fixed; if the learned policy no longer delivers the reported fourfold increase in the 5th-percentile goodput, the claimed coverage gain depends on the reward proxy rather than on the graph RL mechanism.","tokens_in":9689,"feed_emoji":"📶","tokens_out":5479,"duration_ms":47021,"temperature":0.7,"pith_summary":"This paper tries to establish that graph reinforcement learning (GRL) can solve QoS-aware load balancing in Open RAN networks with mixed guaranteed-bit-rate (GBR) and best-effort (BE) traffic. It models the radio access network as a graph with UEs and cells as nodes, formulates load balancing as a Markov decision process, and trains a dueling deep Q-network whose Q-values are computed by graph convolutional layers. In a multi-band, non-full-buffer simulation, the learned policy cuts the average QoS dissatisfaction rate by more than 53% and raises the 5th-percentile BE goodput fourfold relative to max-SINR and max-RSRP handover baselines. If correct, the result suggests GNN-based DRL is a viable path for QoS-aware traffic steering in O-RAN without per-network retraining.","feed_headline":"Graph RL cuts QoS violations by 53% in O-RAN load balancing","feed_subtitle":"GNN-based dueling DQN also quadruples 5th-percentile goodput for best-effort users in multi-band simulations.","key_machinery":"The central object is the RAN graph representation $\\mathcal{G}=(\\mathcal{N},\\mathcal{E},X_u,X_c)$ with UEs and cells as heterogeneous nodes and two edge types (UE-to-cell access links and cell-to-cell load-balancing links). The argument is carried by a GNN-based dueling DQN: two graph convolutional layers propagate node embeddings over the graph, a mean pooling layer aggregates cell embeddings, and the dueling architecture splits the Q-value into state value $V(s)$ and advantage $A(s,a)$ via $Q(s,a)=V(s)+A(s,a)-\\frac{1}{|\\mathcal{A}|}\\sum_{a'}A(s,a')$. The reward is the improvement of a QoS-aware objective $f(\\mathcal{G})$ that sums GBR QoS metrics and the minimum average BE rate after each handover, and a subgraph extraction heuristic keeps the graph tractable for large deployments.","core_discovery":"The paper claims that a load balancing policy trained with graph reinforcement learning can jointly improve GBR QoS satisfaction and BE coverage under realistic interference, non-full-buffer traffic, and mixed bands. The central discovery is that representing the RAN state as a heterogeneous graph — with UE and cell features, UE-to-cell access links, and cell-to-cell load-balancing opportunity edges — and learning Q-values with a two-layer graph convolutional network inside a dueling DQN yields a policy that moves cell-edge UEs to underutilized cells across bands, reducing QoS violation rate by over 53% and multiplying the BE 5th percentile goodput by roughly four compared to max-RSRP and max-SINR. The learned policy is permutation invariant, scales to different network sizes via subgraph extraction, and accounts for spatial dependencies among cells.","pith_inferences":["Because the reward uses the minimum average BE rate rather than the true 5th percentile, the reported fourfold coverage gain is contingent on that proxy; a robust quantile estimate or long-horizon evaluation would test whether the gain reflects genuine edge-user improvement.","The same graph formulation could be extended to other O-RAN controls, such as cell sleep modes or spectrum allocation, by redefining actions and reward terms while reusing the GNN-based dueling DQN.","The subgraph extraction heuristic suggests a natural testable extension: compare full-graph vs. subgraph training on larger topologies to see whether the segmentation degrades policy quality.","Because the policy is trained on deployments with random UE counts and traffic parameters, a direct next step is out-of-distribution generalization testing with UE counts or band configurations outside the training range."],"forward_implications":["The learned LB policy can be hosted as an xApp and re-run as the RAN graph evolves, with no need to fix the input size beforehand.","The approach generalizes to different numbers of UEs and cells because GCN processing and mean pooling are permutation invariant and the subgraph extraction keeps complexity manageable.","Jointly optimizing GBR QoS and BE coverage is feasible in a single RL reward, without separating the two traffic classes into independent controllers.","The observed load distribution across bands indicates the GRL agent can move traffic to underused lower bands even when SINR favors higher bands, balancing utilization more evenly."],"supporting_citations":[{"why":"The prior GRL-based load balancing work this paper extends by adding QoS constraints, mixed GBR/BE traffic, and multi-band handovers.","marker":"[12]"},{"why":"Defines the QoS parameters (GFBR/MFBR, 5QI, delay budgets) that shape the QoS metric and the simulation settings.","marker":"[15]"},{"why":"Supplies the graph convolutional layer propagation rule used in the Q-network.","marker":"[16]"},{"why":"Introduces the dueling DQN advantage decomposition and identifiability fix used in Eq. (12).","marker":"[17]"}],"fun_headline_variants":["Graph RL load balancing cuts QoS violations by 53% in O-RAN","GNN-based RL quadruples BE 5th percentile rate, drops QoS fails 53%","O-RAN: Graph RL balances load, slashes QoS violations 53%, boosts BE 4x","QoS-aware load balancer with GRL: 53% fewer violations, 4x BE goodput"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The training reward replaces the coverage metric it aims to maximize — the 5th percentile best-effort rate — with the minimum average best-effort rate over the current window, and assumes that minimum faithfully tracks true edge-user coverage; if a single persistently unlucky user drives the minimum, the agent's learning signal becomes noise.","fun_headline_variants_meta":{"raw":{"variants":["Graph RL load balancing cuts QoS violations by 53% in O-RAN","GNN-based RL quadruples BE 5th percentile rate, drops QoS fails 53%","O-RAN: Graph RL balances load, slashes QoS violations 53%, boosts BE 4x","QoS-aware load balancer with GRL: 53% fewer violations, 4x BE goodput"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001088,"raw_usage":{"total_tokens":4574,"prompt_tokens":999,"completion_tokens":3575,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":615,"completion_tokens_details":{"reasoning_tokens":3475}},"tokens_in":615,"tokens_out":3575,"duration_ms":25866,"temperature":1.0,"reasoning_tokens":3475,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:51:07.615121+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same training pipeline in the simulator but compute the reward from a quantile estimate of the BE rate (e.g., the 5th percentile over a long sliding window) instead of the minimum average rate, holding all other settings fixed; if the learned policy no longer delivers the reported fourfold increase in the 5th-percentile goodput, the claimed coverage gain depends on the reward proxy rather than on the graph RL mechanism.","supporting_citations":[{"cited_title":"Connection Management xAPP for O-RAN RIC: A Graph Neural Network and Reinforcement Learning Approach,","cited_arxiv_id":null,"evidence_quote":"The prior GRL-based load balancing work this paper extends by adding QoS constraints, mixed GBR/BE traffic, and multi-band handovers."},{"cited_title":"System architecture for the 5G System (5GS),","cited_arxiv_id":null,"evidence_quote":"Defines the QoS parameters (GFBR/MFBR, 5QI, delay budgets) that shape the QoS metric and the simulation settings."},{"cited_title":"Semi-Supervised Classification with Graph Convolutional Networks,","cited_arxiv_id":null,"evidence_quote":"Supplies the graph convolutional layer propagation rule used in the Q-network."}],"review_version":1}