{"id":"4fe5e867-b885-41f2-9035-78c389fd2619","arxiv_id":"2507.10913","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"A MARL framework that uses an active-contour-inspired reward to train UAV swarms for collision avoidance without credit assignment or observation sharing.","lead":"This paper trains UAV swarms to avoid collisions by rewarding them for following contours on a virtual potential field where obstacles are peaks. The authors claim this enables large swarms to learn cooperative avoidance with simple independent learners, but the experiments raise questions about fairness and statistical robustness.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"PSO-adjusted positions used for reward may be unavailable at deployment, so the learned policy may optimize a different MDP and the reported safety/performance gains are not established.","rationale":"The PSO-adjusted reward is the most load-bearing concern because it attacks the internal validity of the learning signal. If the reward is computed on positions corrected by PSO, then the agent's policy gradient optimizes a return for a transition that is not the one resulting from its own action. At deployment the PSO correction is explicitly absent, so the learned Q-function and policy are evaluated under a different state-action distribution. This makes the reported superiority over baselines and the claimed safety guarantees unsupported, even if the baseline comparison were perfectly fair. The reader's weakest assumption identifies exactly this issue, and the paper's own statement that PSO is 'only needed in training' confirms that the correction is not applied at test time. The notation error in Eq. 8 (dU2O vs dU2U) further obscures what constraint PSO actually enforces, making it impossible to reproduce the training pipeline. A retraining experiment without PSO in the reward is a decisive check: if the performance collapses or safety thresholds are violated, the published results depend on oracle information not available to the deployed policy. This concern is concrete, testable, and central, so it does not change the reader's reject verdict.","tokens_in":10845,"tokens_out":6668,"duration_ms":77465,"concrete_test":"Retrain reMARL with r computed on the raw executed positions (remove the PSO adjustment step in Eq. 8 from the reward pipeline) in the 5U1O and 10U1O scenarios, then evaluate the final policy in the same test setups. If the average episodic reward or the safety metrics (min dU2O, min dU2U) drop below the published values, or if the trained policy violates dcol at deployment, the reported results rely on oracle positions unavailable at test time. As a complementary check, run the published trained model (if code is provided) with PSO disabled at evaluation and measure collision counts.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In 'Innovative Reward Design', Eq. 8 defines a PSO adjustment cost over UAV positions before (q_i) and after (q'_i) adjustment, and the text states 'the position of UAVs determine their contours ... must be adjusted together'. This implies the reward r = -f(S, Φ) + r_form·r_collide is computed on the adjusted positions q', not on the positions actually reached by the action a_i. Yet the Framework section states the PSO search is 'only needed in training' and is not used at deployment. Therefore the policy is trained to maximize returns in a state transition that never occurs at test time: at test, actions map to unadjusted positions, while the Q-function was fit on adjusted positions. The central claim that simple DDPG + this reward outperforms SOTA MARL is thus based on a train/test mismatch. Additionally, the safety assertion 'contours never go through peaks or intersect' applies to the adjusted contour curves, not to the executed trajectories; the paper itself admits in Results that UAVs 'gain the ability to adapt to complex environments where contours may be non-viable or non-existent', so the abstract's 'inherently avoided' is overclaimed. Eq. 8 also contains a notation error: f_thres is written with dU2O but the surrounding text says the constraint is on the minimum distance between any two UAVs (dU2U), making the actual constraint ambiguous.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes reMARL, a multi-agent reinforcement learning framework for cooperative collision avoidance of UAV swarms. The core idea is to construct a potential field in which obstacles and the swarm virtual center are peaks, then define a reward from active-contour theory so that each agent's trajectory approximates a contour on the field. Training uses independent DDPG agents that observe only self, swarm, and obstacle properties, with PSO used during training to adjust UAV positions collectively so that contour distances satisfy inter-UAV thresholds. The paper claims that this reward removes the need for credit assignment or observation sharing, enables training with swarm sizes up to 10, and outperforms VDN, QMIX, COMA, IQL, and MAPPO, while also achieving large reaction-time and energy-efficiency improvements over a meta-heuristic baseline. The empirical evidence, however, is undermined by a train/test mismatch in how the reward is computed, an asymmetric comparison against discretized baselines, and an energy metric that partly duplicates the training objective.","tokens_in":121,"tokens_out":3208,"duration_ms":74577,"significance":"If the central claims were valid, the paper would make a useful contribution: replacing complex MARL architectures with a domain-knowledge-derived individual reward and a simple DDPG learner is an appealing recipe for scalable UAV swarm training, and the observation design that avoids explicit observation sharing is interesting. The paper also addresses a practically relevant problem and tests multiple swarm/obstacle configurations. However, the current evidence does not support the headline claims. The reward is computed on positions that have been collectively adjusted by PSO during training but not necessarily at deployment, so the learned policy may optimize a different MDP than the one evaluated. The baseline comparison is also unfair because the competing MARL methods are restricted to 10 discrete actions while reMARL uses continuous DDPG. Finally, the energy-efficiency metric is essentially the smoothness term that is already minimized inside the training reward, making the energy claim circular. These issues are load-bearing for the paper's main assertions, so the contribution cannot be accepted in its present form.","major_comments":[{"comment":"The paper states that PSO adjusts UAV positions collectively and that this search is \"only needed in training\" (Framework), while the reward is computed on the adjusted positions q' via Eq. (8). If at deployment the PSO correction is not applied, the policy is trained on reward signals from adjusted trajectories that never occur under the executed action distribution. The Q-function and policy gradient therefore optimize a different MDP from the one deployed, so the reported gains over MARL baselines are not established. The authors should either apply the same PSO correction at deployment, or train and report using rewards evaluated on the positions actually produced by the agent's action.","section":"Framework / Innovative Reward Design (Eqs. 7-8)"},{"comment":"The comparison is asymmetric: reMARL uses DDPG with continuous actions, whereas VDN, QMIX, COMA, IQL, and MAPPO are restricted to 10 discrete actions sampled uniformly from [-π/4, π/4] for algorithms requiring a discrete action space. This confounds algorithm choice with action-space granularity; MAPPO and COMA can be implemented with continuous policies. The claim that reMARL outperforms state-of-the-art MARL therefore requires re-running baselines with comparable continuous action spaces or explicitly controlling for discretization error.","section":"Results"},{"comment":"The energy-efficiency metric is average curvature defined by Eq. (10), which is essentially the same smoothness term minimized inside the contour reward Eq. (4) (|S''|^2 vs. |S''|). Hence the reported 85.37% energy improvement over the meta-heuristic is partly a reflection of the training objective rather than an independent measure of energy consumption. An independent metric (e.g., simulated power draw from acceleration and turning) is needed to support the energy claim.","section":"Results (Table 1, Eq. 10)"},{"comment":"The abstract's claim that collisions are \"inherently avoided\" because contours never pass through peaks or intersect is not supported by the paper's own admission in Results that agents adapt to environments \"where contours may be non-viable or non-existent\" and navigate between obstacles. Safety in the actual experiments is enforced by the Swarming reward r_collide and by the threshold d_col, not by contour topology alone. Additionally, Eq. (8) defines f_thres using dU2O where the surrounding text says the hard constraint applies to the distance between any two UAVs (dU2U); the notation should be corrected and the actual constraint specified.","section":"Innovative Reward Design (Eq. 8) and Abstract"}],"minor_comments":[{"comment":"There are multiple typos and spacing issues: \"A voidance\" in the title, \"counters\" should be \"contours\" in the abstract, and \"UA V\" is inconsistently spaced throughout the manuscript.","section":"Abstract and Title"},{"comment":"The conclusion says \"Extensive experiments are conducted to proof that reMARL achieves...\" which should be \"prove\" or \"demonstrate.\"","section":"Conclusion"},{"comment":"The learning curves in Fig. 4 appear to be single runs without shaded confidence intervals or multiple-seed statistics, making it difficult to judge whether the reported differences are significant; a table of final mean and standard deviation across seeds would strengthen the empirical claims.","section":"Results (Fig. 4)"},{"comment":"The policy network outputs a single tanh unit, but the action is defined in the range [-π/4, π/4]; the scaling from the network output to the actual action range is not described, which hampers reproducibility.","section":"Agent"}],"recommendation":"reject","confidential_remarks":"The manuscript is under review at AAAI 2026. The central train/test mismatch involving PSO-adjusted reward positions is a fundamental correctness issue that cannot be patched by presentation changes alone, and the asymmetric baseline comparison further weakens the empirical contribution. If the authors can retrain with rewards on executed positions and provide fair continuous-action baselines, the idea might be worth a future submission, but in its current form the paper does not establish its main claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is actually neat. The authors convert the active-contour cost function from their earlier PSO-based work into an individual reward for independent DDPG learners. That lets them drop credit assignment and observation sharing entirely, and it's a plausible route to larger swarms. They're also honest about a real limit: when contours are non-viable or non-existent, the learned policy can still navigate, which the meta-heuristic version couldn't do.\n\nWhere the paper falls down is the evidence. The reward in Eq. 7 is defined after a PSO adjustment of UAV positions (Eq. 8), and the paper never clarifies whether the adjusted position is the one the executed action actually produces. The Framework section says PSO adjusts positions and the reward is then used to train; the Experiments section says the reward is computed on the actual trajectory after the action. If training rewards are computed on hypothetical adjusted positions rather than on the states actually reached, the policy optimizes a different MDP than the one at deployment, and the performance comparison is meaningless. This is a load-bearing ambiguity. There's also a notation slip in Eq. 8: f_thres is written with dU2O, but the surrounding text says the constraint is on the inter-UAV distance dU2U.\n\nThe baseline comparison is unfair. VDN, QMIX, COMA, and IQL are restricted to 10 discrete actions while reMARL uses continuous DDPG, so part of the gap in Fig. 4 is exactly what you'd expect. The energy metric in Table 1 is average curvature, which is the same smoothness term being minimized inside the reward, so that comparison is partly circular. And the error bars are huge—Energy Cost is reported as 19.72 ± 31.8, which makes the 85% improvement hard to trust. No code or data is provided.\n\nThe abstract overclaims: it says collisions are 'inherently avoided' because contours never pass through peaks, but the body admits contours may be non-viable and the learned policy sometimes departs from them. The safety claim should be scoped to the reward design's intention, not treated as a guarantee.\n\nThat said, the underlying idea is worth taking seriously. All the flaws are addressable in a revision: pin down the PSO/reward relationship, rerun baselines with continuous actions, report seeds and interquartile ranges, and separate the energy claim from the reward's smoothness term. I'd send this to a serious referee rather than desk-reject, because the reward-design insight could genuinely help UAV swarm settings if the train/test mismatch is resolved.","headline":"The reward-design idea is genuinely clever—turning an active-contour cost into an individual MARL reward to sidestep credit assignment—but the experiments as reported don't establish the claimed gains because of a train/test ambiguity in how the reward is computed plus unfair baselines.","tokens_in":11665,"tokens_out":3942,"would_cite":false,"duration_ms":42309,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"UAV swarms can learn cooperative collision avoidance without credit assignment or observation sharing, because a contour-following reward on a field with obstacle peaks makes each agent's individual maximization safe; the paper reports…","keywords":["multi-agent reinforcement learning","UAV swarm","collision avoidance","active contour model","domain-knowledge reward","potential field","DDPG","cooperative navigation"],"falsifier":"Replay a trained episode and recompute the reward from the executed, unadjusted trajectory; if it disagrees materially with the reward stored during training, the policy was optimized against a fictitious trajectory. A direct comparison between agents trained on PSO-adjusted positions and agents trained on the actually executed positions would settle whether the reported improvements are real.","tokens_in":10657,"feed_emoji":"🚁","tokens_out":9540,"duration_ms":102189,"temperature":0.7,"pith_summary":"reMARL, the framework proposed here, is an attempt to show that cooperative collision avoidance for UAV swarms does not need coordination machinery inside the learning algorithm. The environment is represented as a two-dimensional potential field with obstacles and the swarm's virtual center as peaks, and each drone's reward is built from the active-contour cost that is minimized by smooth curves following the field's edges. Because contours on such a field never pass through peaks and never intersect, a drone that maximizes its own reward is claimed to avoid collisions and fly smoothly. If the claim holds, large swarms can be trained with independent DDPG agents, sidestepping the credit-assignment and observation-sharing bottlenecks that limit existing MARL methods.","feed_headline":"Contour rewards train 10-drone swarms without credit assignment","feed_subtitle":"Treating drone paths as image contours avoids collisions and beats VDN, QMIX, COMA at larger swarm sizes.","key_machinery":"The load-bearing object is the active contour model from image processing, used here as a trajectory scorer rather than an image segmenter. By placing obstacles at field maxima and minimizing $f(S,\\Phi)=\\int\\left(\\frac{1}{2}|S''(\\rho)|^2-\\frac{1}{2}|\\nabla\\Phi(S(\\rho))|^2\\right)d\\rho$, trajectories are pulled onto smooth contour lines that, by construction, avoid peaks and other contours. The PSO correction of Eq. 8 enforces the hard inter-drone spacing during training, and a per-agent DDPG learner turns the resulting scalar reward into a policy.","core_discovery":"The paper's central claim is that the geometric property of contours—they never cross maxima—can be turned into a per-agent reward that makes cooperative collision avoidance an individual learning problem. Formally, the environment is mapped to a potential field $\\Phi(q)=\\Phi_s(q)+\\sum\\Phi_o(q)$, with the swarm's virtual center and every obstacle as maxima; a trajectory $S$ is scored by $f(S,\\Phi)=\\int\\left(\\frac{1}{2}|S''(\\rho)|^2-\\frac{1}{2}|\\nabla\\Phi(S(\\rho))|^2\\right)d\\rho$, the active-contour cost. The reward $r=-f(S,\\Phi)+r_{\\text{form}}\\cdot r_{\\text{collide}}$ adds a formation-preserving and safety-gating swarming term, while PSO adjusts the drones' positions during training so their contour levels respect a hard inter-drone distance. With this reward, each drone is trained by DDPG alone, and the reported experiments show reMARL outperforming VDN, QMIX, COMA, IQL, and MAPPO once the swarm exceeds three UAVs, scaling to ten, and cutting reaction time by 98.75% and energy cost by 85.37% relative to contour-based meta-heuristic search.","pith_inferences":["The paper leaves open whether the PSO-adjusted positions used to compute the reward match the positions the drone actually executed; if they do not, the learned policy is scored against a trajectory it did not fly, and the reported gains could be inflated.","The same reward construction should transfer to any multi-agent domain where safety can be encoded as 'never cross a peak' on a shared scalar field, such as surface vessels or ground robots navigating on a risk map.","A natural stronger evaluation would verify the learned policy's safety formally, since the hard distance constraints are enforced by PSO during training and are not guaranteed by the neural network at deployment.","The small-swarm regime's weaker performance suggests a broader design lesson: coordination machinery only pays off when the task's geometry makes independent locally optimal behavior unsafe; in trivial geometries it mostly adds noise."],"forward_implications":["Adding UAVs to the swarm only adds independent DDPG learners, so training cost scales linearly rather than exponentially in swarm size.","The fixed-size observation—self state, swarm virtual center, and detected obstacle states—removes communication and centralized critics from the network design.","The smoothness term in the reward bounds steering to $[-\\pi/4, \\pi/4]$, so actions are stable and energy-efficient by construction.","Because the agent maximizes the full reward, it can deviate from strict contours when contours are not viable, such as flying between two obstacles instead of taking a long detour.","A deployed policy is a single forward pass, giving a reaction time of about 0.006 seconds versus 0.48 seconds for the meta-heuristic baseline."],"supporting_citations":[{"why":"Supplies the potential-field and contour domain knowledge, and the meta-heuristic baseline reMARL is compared against for reaction time and energy cost.","marker":"Huang, Zhang, and Huang 2021"},{"why":"Defines the active contour model whose cost function Eq. (4) is adapted into the reward.","marker":"Kass, Witkin, and Terzopoulos 1988"},{"why":"Provides the VDN baseline, an implicit credit-assignment method that reMARL claims to eliminate.","marker":"Sunehag et al. 2017"},{"why":"Provides the QMIX baseline used in the comparison experiments.","marker":"Rashid et al. 2018"},{"why":"Provides the COMA baseline, an explicit credit-assignment method unsuitable for continuous action spaces.","marker":"Foerster et al. 2018"},{"why":"Provides the IQL baseline and the observation-sharing degradation issue the framework avoids.","marker":"Tan 1993"}],"fun_headline_variants":["Contour rewards teach UAV swarms collision avoidance","Drone swarm collision avoidance via contour rewards","No credit assignment, just contours: UAV swarm safety","Image-processing contours enable scalable drone swarm avoidance","Contour-based reward beats state-of-the-art in UAV swarms"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reward used for training is computed from positions that PSO has collectively shifted to keep drones apart, and the paper does not state whether those shifted positions are the positions the drone actually flew or whether the correction is applied at deployment.","fun_headline_variants_meta":{"raw":{"variants":["Contour rewards teach UAV swarms collision avoidance","Drone swarm collision avoidance via contour rewards","No credit assignment, just contours: UAV swarm safety","Image-processing contours enable scalable drone swarm avoidance","Contour-based reward beats state-of-the-art in UAV swarms"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000254,"raw_usage":{"total_tokens":1572,"prompt_tokens":952,"completion_tokens":620,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":568,"completion_tokens_details":{"reasoning_tokens":546}},"tokens_in":568,"tokens_out":620,"duration_ms":7412,"temperature":1.0,"reasoning_tokens":546,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T17:22:26.450515+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Replay a trained episode and recompute the reward from the executed, unadjusted trajectory; if it disagrees materially with the reward stored during training, the policy was optimized against a fictitious trajectory. A direct comparison between agents trained on PSO-adjusted positions and agents trained on the actually executed positions would settle whether the reported improvements are real.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the potential-field and contour domain knowledge, and the meta-heuristic baseline reMARL is compared against for reaction time and energy cost."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the active contour model whose cost function Eq. (4) is adapted into the reward."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the COMA baseline, an explicit credit-assignment method unsuitable for continuous action spaces."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the IQL baseline and the observation-sharing degradation issue the framework avoids."}],"review_version":1}