{"id":"5ae34cab-5b75-49fd-9f59-36399caa2ead","arxiv_id":"2502.05707","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"A one-hidden-layer DDQN agent learns to choose feasible IAB donor base stations for three network slices in a simulated seven-node 5G backhaul, reaching perfect held-out reward.","lead":"The paper trains a Double Deep Q-Network to pick which nearby 5G base station should wirelessly backhaul each network slice of a congested base station. It reports that a small one-hidden-layer network learns the selection rule in about 21 simulated days and then earns maximum reward on the held-out time slots.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported perfect test-phase reward (60/60) is likely an artifact of a chronological train/test split: the test phase (last 20% of the day) falls outside the congestion windows, so a policy that always selects the wired backhaul (BS1) already achieves the maximum test reward.","rationale":"The strongest claim singled out by the reader is the perfect test-phase score. That claim rests entirely on the composition of the test set. The paper does not state the split procedure, but the use of 'phases' and the numbers (201/27/60) are consistent with a chronological partition of the 288 samples in the single daily episode. Under that partition, the 60 test samples fall at the end of the day, after the DL congestion window ends at 18:30. In that period the wired backhaul of BS1 has spare capacity, so the correct action for every slice is to choose BS1. A trivial policy 'always BS1' attains the maximum reward of 60, so the test provides no information about whether the agent learned to select IAB donors. The reader's concern about orthogonal resources is valid but is an explicit modeling assumption (Section III.A.1), whereas the test-phase issue is an unacknowledged evaluation artifact. If the split were random, the test would still not be out-of-distribution because the environment is deterministic and the same states recur every episode; the agent collects experience from all time steps during training. Therefore the empirical support for the central claim is weaker than reported. The proposed test—measuring performance on the congestion windows and comparing with always-BS1—would settle whether the agent actually solves the IAB donor selection problem. I do not recommend rejecting the paper; the training curves suggest the method can learn the task, and the paper may be correct. But the evaluation must be redone before the 'never fails' claim is accepted. This reinforces the reader's CONDITIONAL verdict rather than changing it.","tokens_in":8936,"tokens_out":11261,"duration_ms":114824,"concrete_test":"Re-evaluate the trained 1x32 DDQN on a test set consisting only of the congestion windows (05:30-12:30 and 14:30-18:30), and separately compute the reward of a trivial baseline policy that always selects BS1 (wired) on the original 60-sample test phase. If the always-BS1 baseline also scores 60, the original test phase is non-discriminative. If the agent scores 60 on the congestion-only test set, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section V-F states that each episode contains 288 samples, split 70% training (201), 10% validation (27), and 20% testing (60). If the split is chronological, as the terms 'training phase', 'validation', and 'test phase' imply, the 60 test samples correspond to the last 20 time steps of the day, roughly 19:00-24:00. The congestion periods identified in Section V-C are 05:30-12:30 (UL) and 14:30-18:30 (DL). Outside these windows, BS1's 1 Gbps wired backhaul has sufficient capacity for all three slices, so the action 'select BS1' yields reward +1 at every test step. A policy that never exercises IAB donor selection therefore scores 60/60 on the reported test phase. The headline claim 'never fails during the test phase' is thus not evidence that the agent learned the backhaul selection task; it is consistent with a constant-wired-link policy. This is more load-bearing than the acknowledged orthogonal-resource assumption (Section III.A.1) because it directly undermines the reported evaluation. Even if the split were random, the test would still not be out-of-distribution: the environment is deterministic and the same daily traffic profile repeats every episode, so the agent encounters all time-of-day states during training. The paper provides no held-out episodes or shifted traffic profiles, so the 'test phase' does not demonstrate generalization.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a Double Deep Q-Network (DDQN) agent to select, every 15 minutes, a backhaul link (the wired BS1 or one of six IAB donor base stations) for each of three network slices served by a congested base station in an Integrated Access and Backhaul (IAB) network. The agent observes slice throughput requirements and the available bandwidth of candidate links, and receives a reward of +1 if the selected link provides the required QoS and 0 otherwise. Simulations with a custom SDN-based environment show that a single-hidden-layer network with 32 neurons reaches near-optimal episode reward after about 21 episodes and attains a perfect test reward of 60/60. The paper claims this demonstrates that the DRL agent performs the BS selection task successfully and can be implemented with minimal computational impact.","tokens_in":9272,"tokens_out":4706,"duration_ms":46482,"significance":"If the evaluation were sound, the paper would provide useful evidence that a very compact DDQN can solve a capacity-feasibility backhaul selection task in a deterministic simulated IAB scenario, which could have implications for lightweight and fast-retraining IAB control. The problem formulation is clear, the action/observation/reward scheme is straightforward, and the hyperparameter sweep across network sizes is a useful engineering exploration. However, the reported test result is undermined by a likely artifact of the chronological train/validation/test split, and the lack of baselines, multiple seeds, and out-of-distribution evaluation means the central claim is not yet supported. The paper is a plausible candidate after substantial revision and re-evaluation.","major_comments":[{"comment":"The reported perfect test-phase reward (60/60) does not substantiate the claim that the agent learned a valid IAB donor-selection policy. If the split is chronological, as the terms 'training phase', 'validation', and 'test phase' suggest, the 60 test samples correspond to the last 20% of the 288-sample day (roughly 19:00-24:00), which lies entirely outside the congestion windows identified in Section V-C (05:30-12:30 for UL and 14:30-18:30 for DL). A constant policy that always selects the wired backhaul (BS1) would receive +1 on every test sample because BS1's wired link has sufficient capacity in those hours. The paper must specify whether the split is chronological or random; if it is chronological, the test-phase metric is trivial and must be replaced by a test regime that includes congested periods, for example a held-out day with shifted traffic profiles or randomly sampled time slots across the whole day. The performance of the trivial always-BS1 policy should also be reported as a reference.","section":"Section V-F, Table IV and Section V-C"},{"comment":"The results are reported for a single run per architecture with no multiple seeds, confidence intervals, or significance testing. DDQN training is stochastic, so the claimed fast convergence in 21 episodes and the perfect test reward need variance estimates. Moreover, no baseline is compared, such as a greedy 'any feasible donor' policy, a least-loaded donor policy, or the trivial always-wired policy. Without such baselines, the paper cannot support the conclusion that DRL is the right tool for this task or that the learned policy is superior to simple heuristics. The authors should at least add the always-BS1 baseline and a greedy feasibility policy to the comparison in Table IV and report mean and standard deviation over multiple seeds.","section":"Table IV and Section V-A"},{"comment":"The early-stopping criterion is based on the moving average of the episode reward reaching 97.5% of the maximum achievable value over a training episode (0.975 * 201 = 195.975). Since the training portion is the first 70% of the day, which contains the congestion windows, this threshold could be achieved by a policy that simply selects BS1 during non-congested hours and any feasible donor during congested hours, without learning a sophisticated allocation. The paper should report the reward over the full episode (all 288 samples) for the trained agent, not only over the training/validation/test fractions, to demonstrate that the agent indeed handles the congested periods correctly.","section":"Section V-F, early stopping"}],"minor_comments":[{"comment":"The value '1,479 Gbps' appears to be a typo; given the 1 Gbps link capacities and the scale of the traffic profiles, the intended value is likely 1.479 Gbps (or 1479 Mbps).","section":"Section V-F, Fig. 8"},{"comment":"The assumption that wireless IAB links operate in orthogonal resources and therefore do not interfere with each other or with incumbent networks removes the central radio-resource coupling of in-band IAB. This limitation should be discussed more prominently, since in real deployments interference and shared spectrum are core to IAB operation and would affect both the state representation and the learned policy.","section":"Section III.A.1"},{"comment":"The action vector has seven options and the observation vector has 16 parameters, but the paper does not specify the exact ordering of the observation vector entries or the normalization method. A precise definition would improve reproducibility.","section":"Sections IV.A and IV.B"},{"comment":"The reward model gives +1 for any feasible link and 0 otherwise; this binary immediate reward does not account for future capacity consumption or the impact of sequential decisions on other slices. At least a brief discussion of why this reward is adequate is needed.","section":"Section IV.C"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope and the problem is relevant, but the central claim rests on a test-phase metric that is likely trivial under a chronological split. The authors need to redesign the evaluation with proper train/test separation (including congested periods), multiple seeds, and baselines before the manuscript can be considered for publication. I recommend major revision rather than rejection because the identified issues are fixable with additional experiments and analysis."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper applies a standard DDQN to a small simulated IAB backhaul-selection problem: at 15-minute intervals, decide which neighboring BS (or the wired link) should carry each slice's traffic from a congested BS. The genuinely useful part is the problem framing -- slice-level donor selection in IAB is a reasonable niche, and the authors give a clean action/observation/reward structure. The implementation is straightforward and the training curves show convergence in about 20 episodes, which is credible for such a low-dimensional task. So there is a narrow, real result: a tiny fully-connected network can learn a feasible donor-selection policy in this simulator.\n\nThe soft spots are serious. First, the model assumes wireless IAB links are orthogonal and never interfere (Sec. III.A.1). That reduces the decision to a capacity-feasibility lookup on the observation vector, and the paper never tests anything harder. Second, the evaluation is under-powered: no multiple seeds, no error bars, no baseline like \"always use the wired link\" or \"pick any feasible donor.\" Third, the headline claim of zero test failures looks like it may be an artifact of the split. Section V-F splits each 288-sample episode into 70% train / 10% validation / 20% test. If that split is chronological, as the wording suggests, the 60 test samples fall in the last 20% of the day (19:00-24:00), which is outside both congestion windows (05:30-12:30 UL, 14:30-18:30 DL). During those hours BS1's wired 1 Gbps backhaul has enough capacity for all three slices, so a policy that always selects the wired link scores 60/60. The paper does not state whether the split is chronological or random, nor does it provide a full-day test or a held-out traffic profile. Until that is clarified, the \"never fails\" claim is not evidence of learning the selection task.\n\nThe citation pattern looks fine -- related work is covered, and the novelty claim is modest (first DRL for slice-level IAB donor selection). The paper is honestly scoped, but the empirical case is thinner than the conclusion implies.\n\nI'd send it to review only after the authors demonstrate the agent succeeds on the congestion windows specifically, add simple baselines, and report multiple runs. As it stands, it is a promising position paper for a workshop, not a solid result.\n\nRecommendation: engage with it, but require the evaluation fix before accepting.","headline":"A small DDQN can learn a capacity-feasibility lookup in a simplified IAB scenario, but the reported perfect test score is probably an artifact of a chronological split that puts the test phase entirely outside congestion windows.","tokens_in":9811,"tokens_out":2135,"would_cite":false,"duration_ms":21395,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A Double Deep Q-Network with one hidden layer of 32 ReLU neurons can select which neighboring base station should wirelessly backhaul each network slice of a congested 5G base station, converging in about 21 episodes and earning the…","keywords":["integrated access and backhaul","deep reinforcement learning","network slicing","backhaul link selection","DDQN","5G","resource allocation","IAB donor selection"],"falsifier":"Run the trained DDQN agent on a scenario where IAB links share the same spectrum and interfere with each other's access traffic; if the test-phase reward drops below the maximum attainable 60, the orthogonal-resources assumption is load-bearing and the reported no-failure performance does not extend to interference-limited operation.","tokens_in":8774,"feed_emoji":"📡","tokens_out":4534,"duration_ms":41811,"temperature":0.7,"pith_summary":"The paper proposes using Integrated Access and Backhaul (IAB), the 5G feature that lets base stations with wired backhaul lend wireless capacity to neighboring base stations, to relieve a congested base station's network slices. Because traffic demands and slice combinations change over time, the authors model the choice of which neighboring base station should backhaul each slice as a deep reinforcement learning problem. They report that a Double Deep Q-Network with one hidden layer of 32 ReLU neurons selects feasible backhaul links successfully, converges in about 21 training episodes, and earns the maximum possible reward throughout the test phase in their simulator. The point is that a very small neural-network critic may be enough for slice-level backhaul selection, making the approach practical for deployment and retraining.","feed_headline":"A one-layer DDQN learns IAB donor selection in 21 episodes","feed_subtitle":"In simulation, a 32-neuron critic picks feasible 5G backhaul links and scores the maximum test reward.","key_machinery":"The central object is a Double Deep Q-Network (DDQN) agent, a value-based reinforcement-learning method that estimates the optimal state-action value $Q^*(s,a)$ with a fully connected neural network. The agent's input is a 23-dimensional vector that concatenates a 16-parameter observation (the slice's required DL/UL throughput plus the currently free DL/UL bandwidth of BS1 and BS2–BS7) with a 7-parameter one-hot action vector selecting one backhaul link; the output is the predicted reward for that slice-link pairing. The reward is +1 exactly when the selected directional link can carry the slice's required throughput to the core network, which makes the maximum achievable episode reward easy to compute and lets the authors measure convergence against a known ceiling.","core_discovery":"The central claim is that a DDQN agent, whose critic is a fully connected neural network with a single ReLU hidden layer of 32 neurons, can decide every 15 minutes which of the six neighboring base stations should wirelessly backhaul each of three slices of a congested base station, and can do so without a single failed allocation during testing. In the simulation, the agent receives a reward of +1 whenever the selected link provides the slice's required DL and UL throughput and 0 otherwise, so the maximum attainable per-episode reward is 288 and the maximum test-phase reward is 60. The authors report that the chosen network reaches 97.5 percent of the maximum training reward after 21 episodes and then collects the full 60-point test reward, meaning every tested slice allocation was feasible. They interpret the fast convergence and small model as evidence that the approach can be implemented with minimal computation on IAB base stations and can be retrained when traffic profiles change.","pith_inferences":["If the orthogonal-resources assumption is relaxed, the problem becomes one of managing interference, and the same state representation would need to include channel conditions or interference measurements; a natural test is to retrain the agent on a shared-spectrum scenario and compare the test reward.","A simpler non-learning policy could probably match this performance in the current setup, because the decision is a capacity-feasibility lookup; comparing the DDQN agent against such a greedy baseline would clarify whether learning adds value beyond the simulator's assumptions.","The action model decides each slice sequentially with one action at a time, so the paper leaves open the question of jointly optimizing all slices; extending to multi-agent or joint-action formulations would be needed for larger topologies, as the authors note for future work."],"forward_implications":["In the simulated scenario, a single hidden layer with 32 neurons is enough; deeper and wider critics do not systematically improve the test reward, so the learning task is simple enough for a small model.","Because training stops after about 21 episodes, the agent can be retrained quickly when surrounding traffic profiles change, which the authors identify as important for practical deployment.","The reward model gives an exact upper bound (288 per episode, 60 in the test split), so the reported convergence and no-failure test results are checkable quantities rather than relative improvements.","The approach extends the prior use of DRL in IAB from radio-resource allocation to network-slice-level backhaul selection, which is the paper's stated contribution."],"supporting_citations":[{"why":"Supplies the Double DQN algorithm that the agent is built on.","marker":"[15]"},{"why":"Introduces IAB as a 3GPP feature for wireless backhaul, the scenario the paper applies DRL to.","marker":"[1]"},{"why":"Motivates IAB use in 5G mmWave networks and its challenges, framing the target application.","marker":"[3]"},{"why":"Establishes the prior use of DRL for resource management in network slicing, which this paper extends to IAB backhaul selection.","marker":"[12]"}],"fun_headline_variants":["One-layer DDQN learns IAB backhaul link choice in 21 episodes","Single ReLU layer DDQN picks feasible backhaul links in 21 episodes","DDQN with 32-neuron hidden layer solves IAB backhaul choice in 21 episodes","21-episode DDQN training masters 5G IAB backhaul link selection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The model assumes that all wireless IAB links use orthogonal resources and never interfere with one another or with any incumbent network, which turns the selection problem into a pure capacity-feasibility check on the observed free bandwidth.","fun_headline_variants_meta":{"raw":{"variants":["One-layer DDQN learns IAB backhaul link choice in 21 episodes","Single ReLU layer DDQN picks feasible backhaul links in 21 episodes","DDQN with 32-neuron hidden layer solves IAB backhaul choice in 21 episodes","21-episode DDQN training masters 5G IAB backhaul link selection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000803,"raw_usage":{"total_tokens":3519,"prompt_tokens":923,"completion_tokens":2596,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":539,"completion_tokens_details":{"reasoning_tokens":2504}},"tokens_in":539,"tokens_out":2596,"duration_ms":17090,"temperature":1.0,"reasoning_tokens":2504,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T18:15:55.209089+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the trained DDQN agent on a scenario where IAB links share the same spectrum and interfere with each other's access traffic; if the test-phase reward drops below the maximum attainable 60, the orthogonal-resources assumption is load-bearing and the reported no-failure performance does not extend to interference-limited operation.","supporting_citations":[{"cited_title":"Deep reinforcement learning with double Q-learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the Double DQN algorithm that the agent is built on."},{"cited_title":"Integrated Access Backhauled Networks,","cited_arxiv_id":null,"evidence_quote":"Introduces IAB as a 3GPP feature for wireless backhaul, the scenario the paper applies DRL to."},{"cited_title":"Integrated access and backhaul in 5G mmWave networks: Potential and challenges,","cited_arxiv_id":null,"evidence_quote":"Motivates IAB use in 5G mmWave networks and its challenges, framing the target application."},{"cited_title":"Deep reinforcement learning for resource management in network slicing","cited_arxiv_id":null,"evidence_quote":"Establishes the prior use of DRL for resource management in network slicing, which this paper extends to IAB backhaul selection."}],"review_version":1}