{"id":"0651ce00-9c35-4f40-9b86-20ad2bf5b9cc","arxiv_id":"2412.20075","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":8,"one_line_summary":"Adding a discrete 'communicate' action to a decentralized multi-robot exploration policy, with a reward that weights information gained by sharing, reduces exploration steps and overlap in simulated environments.","lead":"Decentralized robots trained with reinforcement learning get a new option in their action space: instead of only moving, they can choose to share their partial maps with teammates. The paper compares several reward designs and finds that one encourages useful communication and reduces the time to map an unknown area in simulation.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Gazebo evaluation does not support the central claim: 7 of 12 communication-policy runs achieve lower map coverage than the baseline, so the paper's own table contradicts the conclusion that communication improves coverage.","rationale":"The reader's verdict was CONDITIONAL, with the weakest assumption identified as the idealized communication channel. That is a legitimate and important limitation, but it is not the most load-bearing issue for the paper's central claim as written. The central claim is an empirical one: communication-trained policies improve exploration (less overlap, faster mapping). The paper's own Gazebo table (Table 5) directly undercuts that claim — 7 of 12 communication runs achieve lower map coverage than the no-communication baseline — yet the text in Sec 5.3.3 says the communication policies outperformed the baseline in all setups. This is an internal inconsistency, not merely an overreach relative to consensus. In my good-faith reading, the Gymnasium results (Case 4: 454 vs 557 steps, lower mean Jaccard) do provide suggestive support for the claim in the simplified environment. But the conclusion is stated unconditionally and the Gazebo validation is presented as confirmation; the reported Gazebo numbers do not confirm it. Because the paper asserts a conclusion contradicted by a table in the same paper, I recommend REJECT rather than CONDITIONAL. This is not a rejection of the underlying idea but of this paper's evidence for it: the claim needs either a corrected Gazebo evaluation (ideally multiple seeds, matched coverage targets, and a stated success criterion) or a substantially moderated conclusion that explicitly acknowledges the Gazebo results do not consistently favor communication. The reader's communication-channel concern is real and worth testing with a follow-up experiment, but the internal contradiction is more load-bearing because it strikes at whether the reported results support the stated finding at all.","tokens_in":15671,"tokens_out":1825,"duration_ms":15671,"concrete_test":"Recompute the comparison in Section 5.3.3 from Table 5: count, per study case, the number of setups where map coverage exceeds Study Case 1, and apply a paired sign test across the four setups. Also restore the Gymnasium success definition: rerun the 100 testing environments with 'success' defined as reaching p=90% of free cells, separate from episodes that terminated early, and report the step statistics (mean, median, IQR) separately for successful runs only. If the coverage advantage pattern remains inconsistent and the robustness definition is ambiguous, the stated conclusion should be moderated.","verdict_should_be":"REJECT","load_bearing_attack":"The paper's conclusion claims communication reduces both overlap and steps-to-explore. The Gymnasium metrics support the steps claim (Table 3, Case 4 mean 454 vs 557, median lower), and the Jaccard overlap is lower on average in Gymnasium. However, the 'robustness' metric of 86% vs 60% is defined as the ratio of successfully explored arenas, with no p=90% threshold stated for success; if 'success' is defined loosely, this headline statistic is uninterpretable. More importantly, the Gazebo results in Table 5 contradict the central claim: for map coverage (the only comparable metric there), Case 2 outperforms Case 1 in only 1 of 4 setups (37.7 vs 76.8, 34.4 vs 33.7, 15.5 vs 64.6, 75.8 vs 63.3), Case 3 in 2 of 4 (53.2 vs 76.8, 28.8 vs 33.7, 69.8 vs 64.6, 39.2 vs 63.3), and Case 4 in 2 of 4 (91.4 vs 76.8, 52.2 vs 33.7, 38.9 vs 64.6, 50.2 vs 63.3). Yet the text asserts communication-trained policies 'outperformed the first study case as shown by the map coverage ratio' in all setups. This is internally inconsistent. Since the conclusion is stated unconditionally and the Gazebo experiment has no statistical repetition, the central claim of improved mapping efficiency is not supported by the paper's own reported evidence.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes adding a discrete 'communicate' action to the action space of decentralized multi-agent reinforcement learning (D-MARL) for exploring unknown environments. Agents are trained with HAPPO and a shared critic, using three map-based observations (agent-specific, collaborative, transmission). Four reward functions are compared as study cases, with communication-aware rewards in Cases 2-4. Training is done in Gymnasium/PettingZoo, and evaluation is performed both in Gymnasium (100 random environments) and in a Gazebo simulation with four TurtleBot3 Burgers. The paper claims that communication-induced action spaces reduce exploration overlap and the number of steps needed to explore, with Study Case 4 performing best.","tokens_in":16067,"tokens_out":5245,"duration_ms":48347,"significance":"If the result holds, the contribution is a conceptually simple but potentially useful modification: making communication an explicit, learnable action with reward shaping that balances exploration and information sharing. The paper also provides a ROS2/Gazebo framework that bridges RL policies to simulated robots, which is a practical resource. The strengths are the explicit reward decomposition, the HAPPO algorithm pseudocode, and the full Gazebo validation setup. However, the central claim is currently not established: the Gymnasium results lack statistical support, and the Gazebo results in Table 5 contradict the paper's own validation sentence. No code, random seeds, or statistical significance tests are provided, and the robustness metric is not precisely defined. The significance of the work therefore depends on substantial additional evidence.","major_comments":[{"comment":"The sentence \"in all the setups the policies that were trained by rewarding inter-agent communication adequately, outperformed the first study case as shown by the map coverage ratio\" is contradicted by Table 5. In Setup 1 only Study Case 4 (91.4%) exceeds the Case 1 value (76.8%), while Case 2 (37.7%) and Case 3 (53.2%) are lower; in Setup 2 only Case 4 (52.2%) exceeds Case 1 (33.7%); in Setup 3 only Case 3 (69.8%) exceeds Case 1 (64.6%); in Setup 4 only Case 2 (75.8%) exceeds Case 1 (63.3%). Across the twelve communication-policy runs, exactly six have higher map coverage than the baseline. This internal inconsistency invalidates the validation claim in its current form.","section":"§5.3.3, Table 5"},{"comment":"The reported differences are not supported by any statistical significance test or confidence interval, and no random seeds are reported. For example, the number of steps for Case 1 is 557.00 ± 169.07 and for Case 4 is 454.33 ± 184.85, and the Jaccard index for Case 1 is 0.28 ± 0.07 versus 0.25 ± 0.05 for Cases 2-4; the overlapping standard deviations do not establish a robust reduction. The headline robustness values (0.86 versus 0.60) are single ratios with no uncertainty quantification, so the claim that communication improves robustness is not statistically supported.","section":"§5.2, Table 3"},{"comment":"The robustness metric is defined as the \"ratio of the successfully explored arena over the total number of testing environments,\" but the criterion for a \"successfully explored\" environment is not specified in the metric definition. The earlier text mentions a termination condition of p = 90% of free grids explored, which is not necessarily the same as a success threshold for the robustness calculation. Without an explicit success criterion, the 86% versus 60% comparison in Table 3 is uninterpretable.","section":"§5.1"},{"comment":"The conclusion states that \"the exploration task performed by agents trained with reward functions implementing inter-agent communication reduces both the overlap in the explored areas and the time steps needed to explore the environment,\" but this is not supported by the Gazebo data in Table 5. Several Jaccard coefficients for Study Case 4 are larger than the corresponding Case 1 values (e.g., Setup 1: Ji1,i2 = 0.295 vs 0.095, Ji1,i3 = 0.391 vs 0.262, Ji1,i4 = 0.294 vs 0.071), and in half of the communication-policy runs the map coverage is lower than the baseline. The conclusion overstates the evidence and should be revised or backed by additional experiments.","section":"§6"},{"comment":"The communication model is idealized: a 'communicate' action succeeds whenever agents are within range, with no latency, bandwidth limits, packet loss, or energy cost, even though the introduction (Section 1) motivates the work precisely by citing real-world data transmission constraints such as signal latency and bandwidth. The paper does not test how the learned policies behave under constrained or imperfect communication, so the claimed real-world applicability is not demonstrated. Either the simulation should model such constraints or the claims should be tempered accordingly.","section":"§2.4"}],"minor_comments":[{"comment":"The summation condition \"j>k\" is undefined because the outer sum uses the index ik; the notation should be clarified, for instance by summing over unordered pairs of agent indices.","section":"Eq. (14)"},{"comment":"The table headings \"na ne ns nb\" are not explained in the caption; they presumably denote number of agents, number of episodes, steps per episode, and batch size, but these should be stated explicitly.","section":"Table 2"},{"comment":"The training reward curves are presented as single traces without confidence intervals or multiple seeds, making it difficult to assess convergence variability across training runs.","section":"Figure 6"},{"comment":"The Data Availability statement says \"This manuscript does not report data generation or analysis,\" which is inconsistent with the simulation results reported in Tables 3 and 5; this statement should be corrected.","section":"Data Availability"},{"comment":"The heading contains a typo: \"T able 1\" should be \"Table 1.\"","section":"Table 1"},{"comment":"The claim that \"study cases 2 and 4 succeeded in lowering the median of the number of steps by more than 100 steps\" should report the actual median values, since Table 3 only lists means and standard deviations.","section":"§5.2"},{"comment":"The notation \"p = 90%\" is potentially confusing because p is also used as a coefficient in Eqs. (10)-(12); please use distinct notation for the exploration threshold.","section":"§5.1"}],"recommendation":"major_revision","confidential_remarks":"The paper has a plausible core idea but the reported evidence is internally inconsistent: the Gazebo results in Table 5 contradict the claim that communication-trained policies outperform the baseline in all setups, and the Gymnasium results lack statistical rigor. I would recommend major revision, with the authors either adding statistically sound Gazebo experiments that support the claim or substantially scaling back the conclusions to what the data actually show. The lack of code and random seeds is also a significant reproducibility concern for an empirical RL paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper's central claim — that adding a discrete 'communicate' action to a decentralized MARL explorer reduces exploration time and overlap — is plausible but only weakly supported, and the paper's own Gazebo table contradicts its headline assertion. The core idea is a modest extension of 'learning when to communicate' from MARL, which the paper doesn't cite.\n\nWhat it does well is build a concrete HAPPO-based exploration system with map-based observations, test four reward designs in 100 Gymnasium environments, and take the trouble to deploy the trained policies on simulated TurtleBots in ROS2/Gazebo. The Gymnasium results do show a step reduction for the best reward (Case 4: mean 454 vs 557 baseline), and the Jaccard overlap is slightly lower, though with overlapping standard deviations and no significance tests. That part is suggestive, not conclusive.\n\nWhere it falls short: no code, no random seeds, no statistical tests. The 'robustness' metric is a success ratio without a clear success threshold — the text says p=90% for exploration termination, but not what counts as 'successfully explored.' More seriously, Table 5 (Gazebo) shows SC2, SC3, and SC4 beating SC1 in only 1, 2, and 2 of 4 setups respectively, yet the text asserts communication-trained policies 'outperformed the first study case' in all setups. That is an internal inconsistency in the paper's own numbers. The communication model is also idealized: the comm action succeeds whenever agents are in range, with no latency, bandwidth, or energy cost, even though the introduction motivates the work with real-world transmission limits.\n\nThe right read is that this is a work-in-progress with a reasonable idea and a solid system-building effort, but the evidence as reported does not support the strength of the conclusion. A serious referee could push the authors to release code and seeds, run proper significance tests, fix the success metric, correct the Gazebo claim, and position the novelty against the existing communication-as-action literature. I'd engage with it as a maybe for a reading group, and I wouldn't cite it as-is. Send it to review if you have reviewers who will demand those revisions; otherwise desk-rejecting on evidence grounds is defensible, but the paper deserves a referee's time for the system integration and the reward-function comparison.","headline":"The Gymnasium step improvements are real but modest, the Gazebo table contradicts the headline claim, and the lack of stats/code makes the paper a conditional at best.","tokens_in":16618,"tokens_out":2376,"would_cite":false,"duration_ms":25209,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Giving each exploring robot a discrete 'communicate' action, trained with a reward that balances sharing and exploring, lets a decentralized team map an unknown environment faster and with less overlap.","keywords":["multi-agent reinforcement learning","decentralized exploration","communication-induced action space","HAPPO","reward shaping","occupancy grid mapping","multi-robot exploration","unknown environments"],"falsifier":"Run the same training and evaluation with a communication channel that drops a fraction of messages or charges an energy cost per transmission; if the Study Case 4 policy no longer beats the no-communication baseline in mean steps and robustness, the central claim would be falsified for realistic settings.","tokens_in":15498,"feed_emoji":"🤖","tokens_out":5944,"duration_ms":55781,"temperature":0.7,"pith_summary":"This paper tries to establish that a team of homogeneous mobile robots exploring an unknown environment maps it more efficiently when each robot's action space includes a discrete 'communicate' action, letting it choose when to share its locally built map with teammates instead of always moving. The agents are trained with the heterogeneous-agent proximal policy optimization algorithm, using reward functions that balance exploration progress against the value of shared information. Across 100 randomized test environments in Gymnasium, the best communication-aware reward (Study Case 4) reached 90% map coverage in a mean of 454 steps versus 557 for the no-communication baseline, and succeeded in 86% of environments versus 60% for the baseline, while also lowering overlap between agent maps. The paper argues this matters because real-world multi-robot exploration is constrained by communication limits, so letting agents decide when to transmit can reduce redundant coverage and speed up mapping.","feed_headline":"Robot teams that learn when to communicate map 100 steps faster","feed_subtitle":"Decentralized explorers trained to share maps at the right moments cut exploration time and overlap in unknown environments.","key_machinery":"The central mechanism is the communication-induced action space and the reward functions that make communication worth choosing. Each agent keeps three maps: an agent-specific map of its own discoveries, a collaborative map that merges information received from teammates, and a transmission map of agents currently in communication range. When an agent performs the 'communicate' action, its collaborative map is merged with those of all agents in the same communication network, per the identity $M_{i_k,\\mathrm{co}} = \\bigcup_{i_j \\in C} M_{i_j,\\mathrm{co}}$, so shared information propagates through chains of agents. The reward functions combine an exploration term proportional to newly discovered cells, a communication term proportional to the knowledge gained from the merged maps, and penalties for staying still or collisions; Study Case 4's history-dependent coefficient $p_k$ conditions the communication reward on how much the agent has discovered since its last communication, which is what makes the learned policy share data at informative moments.","core_discovery":"The paper's central claim is that adding a communication action to the action space, together with a reward term that rewards information gain from communication, improves decentralized multi-agent exploration. In the proposed formulation each agent's action set is {up, up-right, right, down-right, down, down-left, left, up-left, stay, communicate}, and selecting 'communicate' shares the agent's collaborative map with every agent in the same direct or indirect communication network, after which all participating maps are merged. The authors compare four reward functions; Study Case 4, which weights the communication reward by the average number of cells the agent has discovered since its last communication with each network member, achieves the best results: lower mean steps to 90% coverage (454 vs 557), higher robustness (86% vs 60%), and reduced overlap as measured by the Jaccard index. The conclusion states that agents trained with communication-based rewards reduce both overlap in explored areas and the steps needed to explore the environment.","pith_inferences":["The simulations assume an ideal communication channel with no latency, bandwidth, or energy cost, so the reported gains likely represent an upper bound; a natural extension would run the same reward scheme under packet-loss or power-budget models.","The history-dependent coefficient in Study Case 4 suggests a general principle: communication rewards should be gated by what the transmitting agent has learned since the last exchange, which could transfer to other cooperative MARL tasks beyond mapping.","A sensitivity sweep over the communication range $r_c$ would reveal how dependent the benefit is on connectivity; the paper fixes $r_c = 5$ and does not test sparser or denser networks.","Because the policy broadcasts entire collaborative maps, a message-content variant that shares only frontier or high-uncertainty cells might achieve similar gains with far less data, aligning with the paper's real-world communication constraints."],"forward_implications":["A decentralized multi-robot team can learn an implicit division of labor — when to move and when to share — without a central coordinator, purely from the reward signal.","Rewarding communication in proportion to newly shared information reduces duplicated coverage between agents, as measured by lower Jaccard overlap.","The best-tuned communication reward (Study Case 4) improves robustness across unseen environments, from 60% to 86% success.","The policy transfers from the Gymnasium grid world to a Gazebo simulation with TurtleBot3 Burger robots and laser-SLAM occupancy grids, though performance depends on the starting configuration."],"supporting_citations":[{"why":"Supplies the Heterogeneous-Agent Proximal Policy Optimisation (HAPPO) algorithm used to train all agent policies.","marker":"[27]"},{"why":"Provides the PettingZoo multi-agent API used to implement the Gymnasium training environments.","marker":"[28]"},{"why":"Provides the exploration evaluation metrics (steps to coverage, robustness) that the paper adopts.","marker":"[29]"},{"why":"Defines the Jaccard similarity coefficient used to measure overlap between agent-specific maps.","marker":"[30]"},{"why":"Provides the SLAM Toolbox used in Gazebo to generate occupancy grids from laser scans for the robot policies.","marker":"[34]"},{"why":"Provides the Gazebo simulator used for the multi-robot validation experiments.","marker":"[32]"}],"fun_headline_variants":["Communication choice cuts robot mapping time by 100 steps","Letting robots decide when to talk speeds up map building","Decentralized agents learn communication to reduce exploration steps","Robots that choose to share maps explore with less overlap","Communication reward boosts robot map coverage by 18%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a 'communicate' action succeeds reliably whenever agents are within range, with no latency, bandwidth, packet loss, or energy cost; under realistic communication imperfections the learned balance between moving and sharing could change.","fun_headline_variants_meta":{"raw":{"variants":["Communication choice cuts robot mapping time by 100 steps","Letting robots decide when to talk speeds up map building","Decentralized agents learn communication to reduce exploration steps","Robots that choose to share maps explore with less overlap","Communication reward boosts robot map coverage by 18%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000376,"raw_usage":{"total_tokens":1988,"prompt_tokens":916,"completion_tokens":1072,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":532,"completion_tokens_details":{"reasoning_tokens":994}},"tokens_in":532,"tokens_out":1072,"duration_ms":10307,"temperature":1.0,"reasoning_tokens":994,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:35:01.119366+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same training and evaluation with a communication channel that drops a fraction of messages or charges an energy cost per transmission; if the Study Case 4 policy no longer beats the no-communication baseline in mean steps and robustness, the central claim would be falsified for realistic settings.","supporting_citations":[{"cited_title":"Journal of Machine Learning Research25(1-67), 1 (2024)","cited_arxiv_id":null,"evidence_quote":"Supplies the Heterogeneous-Agent Proximal Policy Optimisation (HAPPO) algorithm used to train all agent policies."},{"cited_title":": Pettingzoo: Gym for multi-agent reinforcement learning","cited_arxiv_id":null,"evidence_quote":"Provides the PettingZoo multi-agent API used to implement the Gymnasium training environments."},{"cited_title":"Explore-Bench: Data Sets, Metrics and Evaluations for Frontier-based and Deep-reinforcement-learning-based Autonomous Exploration","cited_arxiv_id":"2202.11931","evidence_quote":"Provides the exploration evaluation metrics (steps to coverage, robustness) that the paper adopts."},{"cited_title":"Journal of Open Source Software 6(61), 2783 (2021) 28","cited_arxiv_id":null,"evidence_quote":"Provides the SLAM Toolbox used in Gazebo to generate occupancy grids from laser scans for the robot policies."},{"cited_title":"In: 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat","cited_arxiv_id":null,"evidence_quote":"Provides the Gazebo simulator used for the multi-robot validation experiments."}],"review_version":1}