{"id":"3d905397-d19a-4375-80ee-c293589cbf40","arxiv_id":"2508.08555","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A traffic-load-aware deep MARL strategy with confidence-weighted overheard information and NSGA-II action-space pruning improves simulated single-hop underwater network performance.","lead":"This paper proposes TARM, a deep multi-agent reinforcement learning strategy that schedules underwater acoustic links and adjusts transmission power and rate based on overheard neighbor traffic load. In simulation, it improves throughput, delay, and energy use for a single-hop underwater sensor network compared with simpler baselines.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Per-node NSGA-II action pruning (Eq. 16) excludes concurrent interference; the resulting fixed action set may lack the power/rate combinations needed for TARM's claimed high-load gains, and the paper provides no test of this.","rationale":"After reading the paper in full, the reader's weakest_assumption is the most technically precise place where the central performance claim could fail. The NSGA-II pruning in §4.2 defines the MARL action space and is solved per node without any stated treatment of multi-user interference. Because the reward (18) is based on actual receptions, the learning could in principle compensate by avoiding conflicting transmissions; the near-unity delivery ratios in §5.4 suggest this may be happening. But the paper provides no explicit check of whether the pruned set ever excludes an action that would be chosen under interference, and it evaluates only N=3. The action-space size inconsistency (seven output units vs. five modes, implying six transmission actions with no derivation from the Pareto front) reinforces that the pruning procedure is under-specified. I therefore agree with the reader that the concern is load-bearing and unresolved. The reported simulations are favorable and constitute real evidence, and the ablation in §5.5 supports the value of the traffic-load components, but the experiments do not isolate the interference-omission assumption; an enlarged-action-space rerun would settle it. This does not change the CONDITIONAL verdict, so verdict_should_be is UNCHANGED.","tokens_in":23157,"tokens_out":15194,"duration_ms":181975,"concrete_test":"Retrain TARM with an augmented action space: for each mode M in Table 2, add to Û the minimum power satisfying (16b) under a representative concurrent interferer (e.g., one other node at 4 km transmitting at 30 W), in addition to the current six actions; then re-run the §5.3 traffic-load sweep at λ_N=2.07 pkt/s and the §5.4 packet-length sweep with identical seeds, reporting throughput, delay, energy, delivery ratio, and channel utilization. If augmented-TARM does not materially outperform the published TARM (say, <5% relative throughput change), the interference-omission in the NSGA-II pruning is not the bottleneck and the concern does not land. If it does outperform, the fixed pruned set is provably suboptimal in exactly the regime the paper emphasizes, and the central claim that TARM adapts mode and power to achieve these gains is weakened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.2 builds the action set Û by solving (16), whose SINR constraint (16b) uses γ_i,m from Eq. (4) that includes interference from concurrent transmitters. The paper never states whether (16) is evaluated with the interference term set to zero; if it is, the Pareto front contains at most one solution per transmission mode (the minimum power meeting the threshold in isolation), and any higher-power version of a mode—exactly the action needed to survive a concurrent interferer—is dominated and discarded a priori. The DRQN in §4.4 then selects only from A={wait, Û}, so TARM cannot choose an interference-robust action even when the reward (18) would favor it. The claimed 92.8%–351.6% throughput gains at λ_N=2.07 pkt/s (§5.3) are largest precisely in the high-load regime where concurrent transmissions and interference are most likely, so this omission directly targets the headline result. The paper's near-unity delivery ratio in §5.4 suggests the learned policy may avoid conflicting overlaps in the tested 3-node geometry, which would make the interference-free front adequate empirically, but no analysis demonstrates that the learned schedules are conflict-free under all tested loads, and no experiment with more than three transmitters or an enlarged action space appears. The concern is therefore unresolved rather than refuted.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper addresses joint link scheduling and transmission parameter adaptation (mode and power) in single-hop underwater acoustic sensor networks. The authors formulate the problem as a Dec-POMDP (ERCMOP) and propose TARM, a deep MARL approach using DRQN with centralized training and decentralized execution. TARM has three components: (i) a traffic-load-aware observation mechanism in which each node attaches queue/load information to its packets and uses overheard neighbor information weighted by an information-confidence function based on the hyperbolic tangent of the information age; (ii) an NSGA-II-based solution space optimizer that prunes the per-node power/mode action set to a Pareto front; and (iii) a DRQN-based policy with a reward that encodes successful receptions, conflicts, and traffic normalization. Simulation results over a three-transmitter, one-sink acoustic network are reported for varying traffic load and packet length, with throughput, delay, energy, delivery ratio, and channel utilization metrics. Against baselines (two Slotted-Aloha variants, NF-TDMA, and DR-DLMA), TARM is reported to achieve throughput gains of 55.6%-220.2% at λ_N=0.75 pkt/s and 92.8%-351.6% at λ_N=2.07 pkt/s. A component ablation supports the value of local and neighbor load information.","tokens_in":23466,"tokens_out":9257,"duration_ms":83651,"significance":"If the reported performance holds, TARM would be a valuable contribution to distributed traffic-aware resource management in energy-constrained UWSNs: the overhear-based load estimation directly addresses partial observability under long propagation delays, and the NSGA-II action pruning is a sensible way to keep the MARL action space small. The ablation study in Section 5.5 is a strength, and the use of realistic acoustic modem modes (AquaSeNT OFDM) and the Urick/BELLHOP channel model gives the evaluation a grounding in practice. However, the significance is currently moderated by (i) an unresolved ambiguity in the action-space construction that could remove exactly the interference-robust actions needed at high load, (ii) the absence of any error bars or statistical tests supporting the headline percentage gains, and (iii) a baseline set that omits the traffic-load-aware schemes cited as the closest prior work. The paper does not provide code or a measurement of the claimed complexity reduction.","major_comments":[{"comment":"The per-node SSO problem (16) does not specify how the SINR constraint (16b) is evaluated. Since γ_i,m in Eq. (4) includes interference from concurrent transmitters, but the candidate set Û is generated before joint scheduling decisions are known, it is unclear whether (16) is solved with zero interference, worst-case interference, or some nominal value. If (16) is solved in isolation, then for each transmission mode the only non-dominated power is the minimum power that meets the threshold with no concurrent interference, so Û contains at most one power per mode and the agent cannot select a higher-power action to survive interference at the sink. The reported throughput gains at λ_N = 2.07 pkt/s (Section 5.3) are largest precisely in the regime where concurrent interference is most likely, so this ambiguity directly bears on the headline claim. Please state how (16) is evaluated and test the sensitivity of the results to the interference assumption, for example with an SSO variant that uses worst-case or sampled interference or with an enlarged action set.","section":"Section 4.2, Eq. (16)"},{"comment":"All curves are pointwise averages over 100 runs with no error bars, confidence intervals, or significance tests. Given the random Rayleigh fading, random mobility, and Poisson traffic, the reader cannot assess whether the reported differences (e.g., the 55.6%-220.2% and 92.8%-351.6% throughput gains in Section 5.3) are distinguishable from noise, particularly where curves are close at low λ_N. Please report standard deviations or confidence intervals and use matched-seed paired tests where curves are close.","section":"Section 5.1 and Figs. 6-7"},{"comment":"The related work identifies traffic-load-aware distributed schemes ([11], [29], [31], [33]) as the closest prior art, but the evaluation compares TARM only with Slotted-Aloha variants, NF-TDMA, and DR-DLMA. As a result, the claim that TARM improves on traffic-load-aware resource management is not directly evidenced. Please either include at least one such baseline (or a reasonable adaptation of it) in the comparison, or clearly state why these schemes cannot be reproduced in the current setup and soften the corresponding contribution claims.","section":"Sections 2 and 5.2"},{"comment":"The paper mentions VDN as an example of a CTDE mixing network but does not state whether it is actually used. Algorithm 3 initializes a single shared Q-network for all agents and contains no description of a value-decomposition or mixing network, nor of how the team reward (18)-(20) is decomposed into per-agent learning signals. Without this information the MARL component is not reproducible. Please specify the exact architecture (DRQN with shared parameters, any mixing network, GRU handling) and the hyperparameters used for the reported results.","section":"Section 4.4, Algorithm 3"},{"comment":"The paper does not state whether a separate TARM model is trained for each traffic value λ_N or whether one model trained over a range of λ_N is evaluated at each point. Since Eq. (19) normalizes the reward using λ_N, this distinction is important: separate training per λ would not demonstrate adaptability across traffic loads. Please clarify the training/evaluation protocol for each λ_N in Section 5.3.","section":"Section 5.3"},{"comment":"The paper claims that the NSGA-II based SSO reduces computational complexity, but no complexity metric, training time, inference time, or size of the candidate set Û is reported anywhere in the evaluation. Since action-space reduction is a stated contribution, the claim is currently unsupported. Please report at least the number of candidate solutions produced by NSGA-II and a comparison of training/execution time against a variant with the full action space.","section":"Contributions 3 and Section 5"}],"minor_comments":[{"comment":"The interference constraint (15b) is written as mt_recv ∈ {0,1}, but the text immediately after it says 'When mt_recv > 1, it implies that conflicts are occurring.' Please reconcile the binary constraint with the conflict condition.","section":"Section 3.5"},{"comment":"Algorithm 2 refers to Eq. (20) when computing the confidence CFj; the correct reference is Eq. (17).","section":"Algorithm 2"},{"comment":"Equation (11) uses the symbol '∥' where a logical OR (typically ∨) is intended, and the spacing in 't arrive_j' is inconsistent.","section":"Eq. (11)"},{"comment":"Section 5.1 states that the second FC layer has seven hidden units generating Q-values for each action, which implies |A|=7 (i.e., U=6), but the size of the NSGA-II candidate set is never reported. Please add the actual U or U-hat size.","section":"Section 5.1"},{"comment":"The notation 'U = |U-hat|' is confusing because U is also used for the full solution space; please use distinct symbols, e.g., U_full and U_cand.","section":"Section 4.4"},{"comment":"Section 2 contains a typo ('propagatioßn') and several other minor grammatical issues; a careful proofreading pass is recommended.","section":"Section 2"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the journal's scope and addresses a timely problem. The main technical risk is the action-space construction ambiguity in Section 4.2: if the NSGA-II pruning is done without modeling concurrent interference, the agent may lack the very actions needed at high load. This is potentially fixable with a clarified and re-tested design, so I recommend major revision rather than rejection. I would also encourage the editor to ask for the missing statistical details and a direct traffic-load-aware baseline before a second round."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid engineering paper with a fresh mechanism (confidence-weighted overheard load info feeding deep MARL), but the main experimental claim is only weakly supported, and there is an unresolved design issue around how the NSGA-II pre-pruning treats interference.\n\nWhat is genuinely good: the traffic-load-aware mechanism in Sec. 4.3 is a sensible way to use stale neighbor load information; replacing acquisition time with a hyperbolic-tangent confidence score is simple and defensible. Using NSGA-II to shrink the power/rate action space before RL is a practical engineering move, and it is clearly described. The simulation chain uses a real acoustic modem's modes, BELLHOP propagation, and a Urick channel—more realistic than most papers in this area. The problem framing as a Dec-POMDP with CTDE is standard but well-executed.\n\nWhere it gets soft. The biggest issue is exactly the one the stress-test identifies: Eq. (16) is solved per node, and the SINR constraint (16b) uses the channel model with interference from other transmitters. The paper never says whether the interference term is zeroed out during the NSGA-II sweep. If it is (and the per-node setup suggests it must be, since a node cannot know future concurrent transmissions), then for each mode the Pareto front contains only the minimum power that meets the threshold in isolation. Any higher-power version of that mode—precisely the action that would let a node survive interference from a concurrent transmission—is dominated and thrown away before RL ever sees it. That directly targets the headline throughput gains at high load. The near-unity delivery ratio in the 3-node test suggests the learned policy avoids overlaps, so the pruned action set might be enough in that geometry. But the paper does not analyze whether the learned schedules are conflict-free, and N=3 is a small sandbox. This is an unresolved gap, not a proven fatal flaw.\n\nTwo smaller things: no error bars or statistical tests anywhere, despite 100 runs; and the baselines are weak—Slotted-Aloha variants, TDMA, DR-DLMA. The closest traffic-aware competitors mentioned in Section 2 are never compared. The complexity-reduction claim is also never measured.\n\nBottom line: if I were an editor, I would send it to review—the core idea is worth discussing—but the authors need to show that the action set includes interference-robust options, add more transmitters, and compare against a real load-aware baseline. Worth engaging with, with a skeptical eye on the pruning step.","headline":"A credible UWSN resource-management paper with a genuinely new traffic-aware mechanism, but the NSGA-II action pruning likely ignores concurrent interference, and the evaluation is too narrow to settle whether the headline gains hold.","tokens_in":23937,"tokens_out":3141,"would_cite":false,"duration_ms":35055,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"In simulation, a traffic-aware MARL strategy that schedules links and adapts power/rate raises underwater network throughput by 92.8% to 351.6% over baselines.","keywords":["underwater wireless sensor networks","multi-agent reinforcement learning","link scheduling","traffic load-aware","transmission mode adaptation","NSGA-II","Dec-POMDP","acoustic communication"],"falsifier":"Run the same single-hop setup at the highest traffic load with a MARL agent whose action space is the full product of the five modem modes and the transmit-power range instead of the NSGA-II-pruned set, and measure throughput and delivery ratio; if the unpruned agent does not match TARM, the pruning step is not responsible for the gains, and if the unpruned agent substantially beats TARM, the pruning assumption is falsified. A second check is to add a hidden interferer whose transmissions are not in any node's observation; if TARM's delivery ratio or throughput degrades sharply while the unpruned agent adapts, the confidence mechanism is not bridging the observation gap.","tokens_in":22957,"feed_emoji":"🌊","tokens_out":9677,"duration_ms":100613,"temperature":0.7,"pith_summary":"TARM is proposed as a distributed way to manage scarce acoustic-channel resources: each underwater node decides for itself whether to transmit and, if so, at which transmission mode and power, instead of relying on a central scheduler. The paper's claim is that this jointly schedules links and adapts transmission parameters well enough to make communication efficient and reliable under limited energy and bandwidth. The authors formulate the problem as a decentralized partially observable Markov decision process and solve it with deep multi-agent reinforcement learning, using a confidence-weighted overheard-traffic mechanism to reduce the gap between local observations and the global state, and NSGA-II to shrink the action space. The reported simulations show TARM outperforming slotted-Aloha variants, NF-TDMA, and DR-DLMA on throughput, delay, energy, delivery ratio, and channel utilization, with a throughput improvement of 92.8% to 351.6% at the highest tested traffic load. The contribution is a practical, traffic-aware MARL design that operates within real acoustic modem constraints in a single-hop underwater network.","feed_headline":"Traffic-aware MARL boosts underwater network throughput up to 351.6%","feed_subtitle":"Distributed acoustic nodes schedule links and adapt rate and power from overheard traffic, beating fixed-slot methods in simulation.","key_machinery":"The load-bearing object is the reduced action set $A = \\{\\text{wait}, \\hat{U}\\}$: the NSGA-II filtering step takes the full product of transmission modes and transmit powers and keeps only the per-node Pareto-optimal tradeoffs between transmission delay and energy subject to the SINR threshold. This is what makes the deep MARL action space small enough to train. The second mechanism is the overhear information matrix $OI$, where each neighbor-load entry is weighted by a confidence $CF = \\tanh(\\Delta\\delta_j / a)$ that replaces the raw acquisition time, so stale overheard traffic still informs scheduling. The third mechanism is the DRQN policy trained under CTDE: recurrent Q-networks handle partial observability, and a team reward based on successful receptions minus conflicts at the sink guides cooperative link scheduling.","core_discovery":"The central claim is that TARM enables efficient and reliable communication in single-hop UWSNs by jointly optimizing link scheduling and per-node transmission parameters through deep MARL. Each agent's action comes from $A = \\{\\text{wait}, \\hat{U}\\}$, where $\\hat{U}$ is a Pareto front of $(\\text{mode}, \\text{power})$ pairs produced by NSGA-II from the tradeoff between transmission time and energy under an SINR threshold. Agents observe their own queue, position, and physical-layer status plus a neighbor-load table whose entries are weighted by a hyperbolic-tangent confidence that decays as information ages; a deep recurrent Q-network selects actions, and centralized training with decentralized execution uses a sink-side reward of successful receptions minus conflicts. The paper reports that this design delivers the stated throughput gains over baselines while keeping delivery ratio near one, and that removing either local load information or overheard neighbor information degrades delay and reliability under high traffic.","pith_inferences":["Editorial inference: if the per-node Pareto pruning is the bottleneck, the same TARM architecture could be retrained with an unpruned or adaptively expanded action set in dense networks, and the comparison would show how much of the reported gain comes from the pruning step versus the learned policy.","Editorial inference: the 10-byte traffic-metadata overhead means TARM's advantage may shrink for very short packets; a sweep of packet lengths below 190 bytes would show whether the overhead can offset the scheduling gains.","Editorial inference: because the simulation covers only single-hop networks, the natural next check is whether confidence-weighted overheard traffic remains useful in multi-hop settings where interference is no longer concentrated at one sink.","Editorial inference: the Poisson traffic assumption is testable; bursty or non-stationary sources would stress the linear traffic estimator and the confidence decay model, and TARM's adaptability under such traffic is not established by the paper."],"forward_implications":["If TARM's simulation results transfer to deployments, a UWSN can be run without a central scheduler or dedicated control-packet exchange, since nodes learn from traffic they already overhear.","The gains over fixed-slot or Aloha-based access are largest under high traffic loads, so TARM is most valuable exactly when the acoustic channel is congested.","Because TARM selects from an NSGA-II-pruned Pareto set, training and inference stay tractable as the number of modem modes and power levels grows.","The confidence-weighted overhear mechanism converts long propagation delays and stale neighbor information from a liability into a scheduling input, enabling concurrent transmissions and higher channel utilization.","TARM addresses throughput, delay, energy, delivery ratio, and channel utilization together rather than trading one objective against another, which is what the multi-objective ERCMOP formulation demands."],"supporting_citations":[{"why":"Supplies the sea-trial SINR thresholds used for each transmission mode, so rate and power choices map to real modem behavior.","marker":"[27]"},{"why":"Provides the underwater acoustic channel and SINR/noise model used in the simulations.","marker":"[35]"},{"why":"Ray-tracing tool that computes the propagation delays TARM exploits for concurrent scheduling.","marker":"[41]"},{"why":"Defines the five AquaSeNT OFDM transmission modes with rates and payloads that form the decision options.","marker":"[45]"},{"why":"Supplies NSGA-II, the Pareto-pruning algorithm that constructs the reduced action space.","marker":"[47]"},{"why":"Provides DRQN, the recurrent Q-network that handles partial observability during execution.","marker":"[50]"},{"why":"Gives the CTDE value-function basis for centralized training with decentralized execution.","marker":"[52]"},{"why":"Slotted-Aloha baseline whose scheduling and parameter choices TARM is compared against.","marker":"[58]"},{"why":"NF-TDMA baseline using near-far slot allocation that TARM must outperform.","marker":"[59]"},{"why":"DR-DLMA deep-RL baseline that exploits unused slots and is a key comparison for TARM.","marker":"[60]"}],"fun_headline_variants":["Traffic-aware MARL lifts underwater throughput 351.6%","Underwater net throughput up 351.6% with MARL and overheard traffic","MARL + traffic cues: 351.6% throughput gain in UWSNs","Acoustic sensors use neighbor load to boost throughput 351.6%","Deep MARL optimizes underwater resources, raising throughput 351.6%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The action set each agent is allowed to choose from is built by NSGA-II from a per-node optimization problem whose SINR constraint does not include interference from simultaneous transmissions by other nodes; if concurrent interference makes the best coordinated mode-and-power choices fall outside that pruned Pareto set, the agent cannot select them and the claimed performance and complexity advantages would not hold.","fun_headline_variants_meta":{"raw":{"variants":["Traffic-aware MARL lifts underwater throughput 351.6%","Underwater net throughput up 351.6% with MARL and overheard traffic","MARL + traffic cues: 351.6% throughput gain in UWSNs","Acoustic sensors use neighbor load to boost throughput 351.6%","Deep MARL optimizes underwater resources, raising throughput 351.6%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000831,"raw_usage":{"total_tokens":3636,"prompt_tokens":962,"completion_tokens":2674,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":578,"completion_tokens_details":{"reasoning_tokens":2572}},"tokens_in":578,"tokens_out":2674,"duration_ms":18824,"temperature":1.0,"reasoning_tokens":2572,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:35:46.510706+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same single-hop setup at the highest traffic load with a MARL agent whose action space is the full product of the five modem modes and the transmit-power range instead of the NSGA-II-pruned set, and measure throughput and delivery ratio; if the unpruned agent does not match TARM, the pruning step is not responsible for the gains, and if the unpruned agent substantially beats TARM, the pruning assumption is falsified. A second check is to add a hidden interferer whose transmissions are not in any node's observation; if TARM's delivery ratio or throughput degrades sharply while the unpruned agent adapts, the confidence mechanism is not bridging the observation gap.","supporting_citations":[{"cited_title":"Adaptive modulation and coding for underwater acoustic ofdm,","cited_arxiv_id":null,"evidence_quote":"Supplies the sea-trial SINR thresholds used for each transmission mode, so rate and power choices map to real modem behavior."},{"cited_title":"On the relationship between capacity and dis- tance in an underwater acoustic communication channel,","cited_arxiv_id":null,"evidence_quote":"Provides the underwater acoustic channel and SINR/noise model used in the simulations."},{"cited_title":"The bellhop manual and user’s guide: Preliminary draft,","cited_arxiv_id":null,"evidence_quote":"Ray-tracing tool that computes the propagation delays TARM exploits for concurrent scheduling."},{"cited_title":"Field tests of adaptive modulation and coding for underwater acoustic ofdm,","cited_arxiv_id":null,"evidence_quote":"Defines the five AquaSeNT OFDM transmission modes with rates and payloads that form the decision options."},{"cited_title":"Deep recurrent q-learning for par- tially observable mdps,","cited_arxiv_id":null,"evidence_quote":"Provides DRQN, the recurrent Q-network that handles partial observability during execution."},{"cited_title":"Optimal and ap- proximate q-value functions for decentralized pomdps,","cited_arxiv_id":null,"evidence_quote":"Gives the CTDE value-function basis for centralized training with decentralized execution."},{"cited_title":"Aloha-based mac protocols with collision avoidance for underwater acoustic net- works,","cited_arxiv_id":null,"evidence_quote":"Slotted-Aloha baseline whose scheduling and parameter choices TARM is compared against."},{"cited_title":"Lever- aging the near–far effect for improved spatial-reuse scheduling in underwater acoustic networks,","cited_arxiv_id":null,"evidence_quote":"NF-TDMA baseline using near-far slot allocation that TARM must outperform."},{"cited_title":"Deep reinforcement learning based mac protocol for underwater acoustic networks,","cited_arxiv_id":null,"evidence_quote":"DR-DLMA deep-RL baseline that exploits unused slots and is a key comparison for TARM."}],"review_version":2}