{"id":"15f18d48-dbc4-4c9e-86b0-f843cd20f3c5","arxiv_id":"2501.13200","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Shared recurrent memory with global broadcast improves coordination in decentralized multi-agent pathfinding and generalizes to longer corridors better than private-memory baselines.","lead":"This paper introduces SRMT, a multi-agent neural network where each agent writes its internal memory into a shared block that all agents can read. It outperforms several reinforcement learning baselines on bottleneck navigation and is competitive with recent planners on POGEMA pathfinding benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Shared-memory attribution is not clean: the paper's own RATE-gen result shows initialization is a strong confound, RMT's initialization is unreported, and the abstract overstates Sparse long-corridor results.","rationale":"The reader's CONDITIONAL verdict correctly identifies the memory-initialization confound as the central threat to the paper's causal claim. The paper itself provides direct evidence for the confound via the RATE-gen experiment, and it does not document whether the key RMT ablation controls for initialization. This is not a rejection-level issue: the architecture is well-motivated, the evaluation is broad, and the code is released. But the headline attribution to shared memory cannot be verified without an init-matched comparison between SRMT and RMT. The abstract's \"consistently outperforms\" claim is also stronger than the reported Sparse-reward long-corridor results, where SRMT drops below RMT beyond length 400; this should be qualified in revision. Since these concerns are addressable and the reader already recommends CONDITIONAL, I do not change the verdict.","tokens_in":12883,"tokens_out":6082,"duration_ms":60998,"concrete_test":"Run a two-factor ablation on the Bottleneck task with Sparse and Moving Negative rewards, for corridor lengths 30, 100, 400, and 1000: (a) SRMT with first-observation-generated memory initialization; (b) RMT with the same first-observation-generated initialization; (c) RMT with fixed/random initialization; (d) SRMT with fixed/random initialization. Use the same hyperparameters, training steps, and at least 5 seeds per condition; report CSR, ISR, and SoC with confidence intervals. If (a) beats (b) by the same margin as in Figure 3, shared broadcast is confirmed. If (b) matches (a), the reported SRMT advantage is due to initialization, not shared memory. Also report the exact RMT initialization used in the current paper and code; if it equals SRMT's, the missing detail is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that SRMT's gains come from globally broadcasting shared memory is not cleanly established. In Section 4.1 and Figure 4, the authors report that re-initializing RATE with SRMT-style memory initialization (RATE-gen) substantially raises its Moving-Negative performance, and they conclude that this shows \"the importance of the proper memory initialization procedure.\" That admission makes memory initialization a plausible alternative explanation for SRMT's advantage over the externally defined baselines (ATM, RATE, RRNN). The potentially decisive control is SRMT against RMT, which the paper describes as an ablation without shared memory; however, the text never states whether RMT uses the same first-observation-generated initialization as SRMT or a fixed/random vector. If RMT uses fixed/random initialization, the SRMT-vs-RMT gap in the Sparse setting (Figure 3) is confounded in exactly the same way. Even if RMT is initialized like SRMT, the paper omits this detail and reports no explicit init-matched comparison across all architectures, so the reader cannot verify that shared broadcast, rather than initialization, drives the headline result. Separately, the abstract's \"consistently outperforms... especially under sparse rewards\" is internally overstated: for Sparse reward, Figure 4 and the text state that SRMT leads only up to corridor length 400 and then drops below RMT for collective performance, so the claim should be qualified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Shared Recurrent Memory Transformer (SRMT), a multi-agent architecture in which each agent's recurrent memory vectors are pooled into a shared memory and globally broadcast through cross-attention. The authors evaluate SRMT on a two-agent Bottleneck navigation task under several reward schemes and on the POGEMA lifelong MAPF benchmark. They report that SRMT outperforms existing MARL baselines and ablation variants in the Bottleneck task, especially under sparse rewards, generalizes to corridors longer than those seen in training, and is competitive with recent methods on POGEMA maps. The paper also includes an appendix with additional reward variants, training details, and an analysis of the learned memory representations.","tokens_in":13229,"tokens_out":3383,"duration_ms":37524,"significance":"If the central claim is established, SRMT would be a useful addition to decentralized MARL for pathfinding: it proposes a simple mechanism for implicit information exchange without an explicit communication protocol, and the paper provides code, confidence intervals, and multi-seed evaluations. The authors also provide a useful negative result in the RATE-gen variant, showing that memory initialization alone changes performance substantially. However, the paper's headline claims currently overreach the evidence: the sparse-reward generalization claim is contradicted by Figure 4, and the attribution of SRMT's gains to shared memory is not cleanly isolated from memory-initialization differences. The lifelong MAPF comparison is also weakened by the use of planning-derived dense rewards for SRMT without a stated equivalent for the baselines. These issues are fixable, but they are load-bearing for the paper's main message.","major_comments":[{"comment":"The abstract claims that SRMT 'consistently outperforms a variety of reinforcement learning baselines, especially under sparse rewards, and generalizes effectively to longer corridors.' This is internally contradicted by the manuscript's own Figure 4 and the text in Section 4.1, which state that for the Sparse reward, SRMT leads only up to corridor length 400 and then drops below RMT for collective performance. The central claim should be qualified to specify the metric, the corridor-length range, and the distinction between individual and cooperative success rates.","section":"Abstract and Section 4.1 (Figure 4)"},{"comment":"The RATE-gen experiment demonstrates that memory initialization is a strong confound: re-initializing RATE's memory from the initial observation substantially improves its Moving Negative performance, which the authors attribute to 'the importance of the proper memory initialization procedure.' The paper treats RMT as the key ablation without shared memory, but it never states whether RMT uses the same first-observation-generated initialization as SRMT or a fixed/random vector. If RMT uses a different initialization, the SRMT-vs-RMT gap in the Sparse setting is confounded in exactly the same way. The authors should report RMT's initialization and ideally add an initialization-matched ablation across all architectures, or provide evidence that initialization is not the main driver of the reported gains.","section":"Section 4.1, Figure 4, and Section 3"},{"comment":"In the lifelong experiments, SRMT is trained with a dense reward for following a planned path computed by A* or the Follower heuristic, as described in the paragraph after Figure 5. The comparison with MAMBA and QPLEX is only meaningful if those baselines receive an equivalent reward signal; otherwise, SRMT benefits from auxiliary planning information that the baselines do not have. The paper should state the reward functions used for each baseline and, ideally, include a controlled variant of SRMT trained without the planning-derived reward, or apply the same reward to the baselines.","section":"Section 4.2 (Lifelong MAPF) and Figure 5"}],"minor_comments":[{"comment":"The phrase 'collective performance' in the statement about SRMT dropping below RMT is ambiguous; specify whether this refers to CSR, ISR, or SoC.","section":"Section 4.1, Figure 4 caption"},{"comment":"The text says the input sequence includes 'the historical sequence of the agent's observations from the past h=8 time steps' but it is not clear whether this history is a fixed window of raw observations or processed hidden states; please clarify in the architecture description.","section":"Section 3, Figure 1"},{"comment":"The scalability figure reports shaded confidence intervals, but the number of evaluation runs and seeds is not stated; please provide this detail for reproducibility.","section":"Appendix A.2, Figure 10"},{"comment":"The caption lists 'Empty, RNN, Attention' as ablations, but the main text also mentions a GRU-based RNN; please align the naming so that the reader can map each curve to the architecture described in the text.","section":"Section 4.1, Figure 3"}],"recommendation":"major_revision","confidential_remarks":"The paper has a solid empirical setup with code release and confidence intervals, but the central attribution claim needs strengthening. The memory-initialization confound is especially important because the authors' own RATE-gen result shows how large an effect initialization can have. The abstract's 'consistently outperforms' wording should be checked against Figure 4, where RMT surpasses SRMT on the collective metric for sparse-reward corridors beyond length 400. I recommend major revision rather than rejection: the issues are addressable with additional experiments or careful re-scoping of the claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"SRMT is worth a look if you care about communication-free coordination in MARL. The core idea is simple: instead of each agent keeping a private recurrent memory (as in RMT, ATM, RATE), pool all agents' memory vectors into a shared buffer and let every agent cross-attend to that buffer. That is a genuinely new mechanism relative to the memory transformers it cites. The Bottleneck experiments are thorough, with multiple reward functions, corridor-length generalization up to 1000, confidence intervals, and public code. The lifelong POGEMA results are honestly described as competitive, and the limitations section is clear.\n\nThe problem is attribution. The paper reports that re-initializing RATE with SRMT-style memory initialization (\"RATE-gen\") substantially improves its Moving-Negative performance, and it credits the proper initialization procedure. That admission means the advantage over RATE, ATM, and RRNN is at least partly initialization, not shared broadcast. The naive control that should isolate shared memory is RMT — SRMT without the shared buffer — but the paper never states how RMT's memory is initialized. Given that SRMT initializes memory from the first observation, RMT could be using fixed or random vectors, which would confound the Sparse-setting gap between SRMT and RMT in exactly the same way. That's a load-bearing gap in the evidence, not a stylistic quibble.\n\nThe abstract's \"consistently outperforms\" is also stronger than the paper's own Figure 4: under Sparse reward, SRMT's cooperative success rate drops below RMT's beyond corridor length 400. The text in Section 4.1 says this, but the abstract doesn't. That matters for any reader who only skims.\n\nMinor but real: hyperparameter reporting is thin for the baselines. The paper describes a grid search for SRMT's entropy and learning rate but doesn't say whether the baselines were tuned with matching budgets. That's fixable but should be stated.\n\nMy bottom line: send it to review. If you work on memory-augmented RL or communication-free coordination, the architecture is worth engaging with. The empirical attribution needs one clean control experiment. I'd want the authors to add init-matched comparisons and soften the abstract, then it's a solid contribution.","headline":"A neat shared-memory mechanism for decentralized MARL, but the paper's own RATE-gen control shows initialization is a confound and the abstract overstates the sparse-reward results.","tokens_in":13717,"tokens_out":3597,"would_cite":true,"duration_ms":34909,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A shared memory broadcast lets agents coordinate without explicit messages.","keywords":["shared memory","multi-agent reinforcement learning","multi-agent pathfinding","memory transformer","recurrent memory","decentralized coordination","sparse reward","POGEMA"],"falsifier":"Train the RMT baseline (recurrent memory without sharing) using SRMT's memory-initialization scheme and compare it with SRMT under Sparse and Moving Negative rewards; if RMT-gen matches SRMT's success rates and corridor-length scaling, the shared-memory cross-attention is not the source of the reported gains.","tokens_in":12686,"feed_emoji":"🧠","tokens_out":4641,"duration_ms":42381,"temperature":0.7,"pith_summary":"This paper claims that agents in a decentralized multi-agent system can coordinate simply by pooling their individual recurrent memory states into a shared, globally readable buffer. The proposed Shared Recurrent Memory Transformer (SRMT) gives each agent a personal memory vector, broadcasts all agents' memory vectors as a common sequence each step, and lets every agent cross-attend to that sequence before acting. In a two-agent bottleneck navigation task, SRMT outperforms reinforcement learning baselines, especially when rewards are sparse, and its policies generalize to corridors of length 1000 after training on corridors of length 3 to 30. On the POGEMA benchmark, SRMT is competitive with recent MARL, hybrid, and planning-based methods. A sympathetic reader would care because this suggests that implicit information exchange through a global workspace can replace explicit communication protocols in cooperative multi-agent systems.","feed_headline":"A shared memory lets agents coordinate without talking","feed_subtitle":"New MARL architecture pools every agent's memory and beats baselines on bottleneck navigation and POGEMA maps.","key_machinery":"The load-bearing mechanism is the shared memory sequence: at every time step, the memories $\\text{mem}_{i,t}$ of all agents $i$ are concatenated into one ordered list, and each agent's transformer performs cross-attention between its own hidden state and that global list. This gives every agent a direct read on the hidden state of every other agent without any explicit message passing, while a memory head rewrites each agent's private memory for the next step. The design extends the Recurrent Memory Transformer to the multi-agent setting, where the shared memory acts as a global workspace consistent with the paper's stated inspiration from global workspace theory.","core_discovery":"The central discovery claimed is that a shared recurrent memory, implemented as a globally broadcast sequence of per-agent memory vectors, enables implicit coordination in decentralized multi-agent pathfinding. Each agent's policy is a memory transformer: it self-attends over its own recent observations and personal memory, then cross-attends to the shared memory containing every agent's current memory, and updates its personal memory via a memory head. This mechanism is fully decentralized at execution time, requires no hand-designed communication protocol, and maintains a single shared policy across homogeneous agents. The paper reports that SRMT solves the Bottleneck task where other memory and communication baselines fail under sparse reward, and that it remains effective on POGEMA maps including Mazes, Random, MovingAI, and Warehouse when coupled with the Follower heuristic planner.","pith_inferences":["The authors' RATE-gen result implies that memory initialization—not shared broadcasting—may be the dominant factor in the Bottleneck gains; a systematic sweep of initialization schemes across all baselines would settle the mechanism.","If the shared memory truly carries coordination information, the cosine-distance analysis could be turned into a causal test: permuting or masking the shared memory during evaluation should degrade coordination performance proportionally to the amount of information removed.","The shared-memory design may transfer to other cooperative MARL domains beyond pathfinding, such as formation control or task allocation, wherever agents need a cheap global summary of each other's state.","Since memory is pooled into a single ordered sequence, the method's scalability depends on the attention cost of reading $n$ memory vectors; the authors' experiments with 64–128 agents suggest an opportunity for a sparse or hierarchical shared memory in larger populations."],"forward_implications":["If SRMT's claims hold, decentralized MARL can achieve coordination through a global memory broadcast rather than learned communication channels, which may simplify training and execution in settings where message protocols are hard to design.","SRMT policies trained on short corridors (3–30 cells) would remain effective on corridors up to 1000 cells, suggesting the shared memory encodes reusable coordination heuristics rather than overfitting to a specific corridor length.","On the POGEMA benchmark, SRMT would be a competitive option against planning-based centralized methods on throughput, while retaining full decentralization.","Combining SRMT with the Follower heuristic planner would yield the best congestion performance in warehouse-like maps, indicating that learned shared memory and classical planning can complement each other."],"supporting_citations":[{"why":"Introduced Memory Transformer with trainable memory tokens, the foundation for the memory mechanism SRMT extends.","marker":"Burtsev et al. (2020)"},{"why":"Recurrent Memory Transformer, which turns memory tokens into a recurrent state and is the base architecture SRMT generalizes to multiple agents.","marker":"Bulatov et al. (2022)"},{"why":"Agent Transformer Memory, a transformer-based working-memory MARL baseline that maintains individual per-agent memory states.","marker":"Yang et al. (2022)"},{"why":"Recurrent Action Transformer with Memory, a baseline whose memory-initialization scheme SRMT compares against and which provides the RATE-gen ablation.","marker":"Cherepanov et al. (2024)"},{"why":"POGEMA benchmark, the environment framework used for all training and evaluation in the paper.","marker":"Skrynnik et al. (2024a)"},{"why":"Follower, whose training procedure and heuristic path-planning approach are adopted for the lifelong MAPF experiments and SRMT-FlwrPlan variant.","marker":"Skrynnik et al. (2024b)"}],"fun_headline_variants":["Shared memory lets agents coordinate without explicit talk","Broadcast agent memories beat baselines in bottleneck navigation","One shared memory pool enables implicit multi-agent coordination","SRMT: pooling memories to coordinate many agents on pathfinding","Memory sharing improves multi-agent pathfinding on POGEMA maps"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's headline advantage is attributed to the shared-memory broadcast, but its own ablation shows that simply giving baselines the same memory-initialization scheme as SRMT (as in RATE-gen) closes much of the gap, so the unique contribution of the shared cross-attention is not cleanly isolated.","fun_headline_variants_meta":{"raw":{"variants":["Shared memory lets agents coordinate without explicit talk","Broadcast agent memories beat baselines in bottleneck navigation","One shared memory pool enables implicit multi-agent coordination","SRMT: pooling memories to coordinate many agents on pathfinding","Memory sharing improves multi-agent pathfinding on POGEMA maps"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000294,"raw_usage":{"total_tokens":1693,"prompt_tokens":912,"completion_tokens":781,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":528,"completion_tokens_details":{"reasoning_tokens":704}},"tokens_in":528,"tokens_out":781,"duration_ms":8164,"temperature":1.0,"reasoning_tokens":704,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T16:21:58.727682+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the RMT baseline (recurrent memory without sharing) using SRMT's memory-initialization scheme and compare it with SRMT under Sparse and Moving Negative rewards; if RMT-gen matches SRMT's success rates and corridor-length scaling, the shared-memory cross-attention is not the source of the reported gains.","supporting_citations":[],"review_version":1}