Pith. sign in

REVIEW 3 major objections 6 minor 31 references

Scaling Multi Agent Reinforcement Learning for Underwater Acoustic Tracking via Autonomous Vehicles

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a GPU-vectorized approximation of a high-fidelity underwater vehicle simulator can accelerate multi-agent reinforcement learning by up to 30,000x while still producing policies that transfer back to the simulator…

desk verdict A genuine engineering advance—30,000x speedup and real Gazebo transfer for tracking error—but 'seamless transfer' overreaches once you look at the collision mismatch. read the letter →

arxiv 2505.08222 v3 pith:OTUK6HWZ submitted 2025-05-13 cs.RO cs.AIcs.DCcs.PF

Matteo Gallici , Ivan Masmitja , Mario Martín This is my paper · ORCID
classification cs.ROcs.AIcs.DCcs.PF
keywords underwateracoustictrackingmulti-agentreinforcementlearningGPU-vectorizedsimulationcurriculumtransformerpoliciesautonomousvehiclesrange-onlysingle-beaconlocalizationsim-to-realtransfer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that multi-agent reinforcement learning for underwater acoustic tracking can be made practical by replacing the slow, high-fidelity simulator with a GPU-vectorized approximation that runs up to thirty thousand times faster while reproducing the simulator's vehicle dynamics. It introduces TransfMAPPO, a Transformer-based version of the multi-agent PPO algorithm in which agents and targets are treated as tokens, so a single policy is defined for any number of vehicles and targets and can be scaled through curriculum learning from one agent chasing one fast target up to five agents following five targets. If the claim holds, a training run that would take months in the high-fidelity simulator takes minutes to hours on one GPU, and the resulting policies still work in the realistic simulator with average tracking errors below five metres. The paper reports evaluations in the high-fidelity simulator showing transfer of the GPU-trained policies with comparable tracking error and target-loss rates.

What carries the argument

The load-bearing machinery is a simplified trajectory model $p_{t+1}=p_t+v\,\delta t\,(\cos(\psi_t+\delta\psi),\sin(\psi_t+\delta\psi))$ with heading change $\delta\psi=\theta(\gamma)$ modelled as a linear function of rudder angle $\gamma$, fitted to simulator trajectories with mean absolute error below 0.015 radians and $R^2=0.99$, plus injected Gaussian noise with standard deviation 0.02 radians. This model lets thousands of environments be vectorized on a GPU while keeping vehicle dynamics close to the high-fidelity simulator. The second piece of machinery is a Transformer actor and critic in which agents and targets are input tokens; self-attention learns a latent coordination graph whose size is independent of the number of entities, which is what makes the policy invariant to fleet and target counts and enables the curriculum procedure.

What would settle it

Run the final curriculum-trained policy in the high-fidelity simulator for the full 1000-step (8-hour) mission and compare average tracking error and target-loss probability with the JaxLrauv training values; if the error diverges beyond the reported five-metre range or target loss rises sharply, the linear heading-change model has failed to preserve long-horizon dynamics. A more direct check is to record yaw drift over 1000 steps and compare it with the 0.02-radian Gaussian noise envelope assumed during training.

Watch

Extended reading notes

Core claim

The central discovery is a training-to-evaluation pipeline in which the expensive simulator is used only for data collection and final testing, while reinforcement learning happens in a simplified environment whose dynamics are aligned to the simulator. The simplified environment models each vehicle's next position from current heading and speed, with heading change treated as a linear function of rudder angle fitted to simulator trajectories plus Gaussian noise; sensor noise, communication dropouts, and partial observability are reproduced from simulator data. On top of this environment, TransfMAPPO uses self-attention over agent and target tokens to produce decentralized actor policies and a centralized critic that are permutation-invariant and hence defined for arbitrary fleet sizes and target counts. Curriculum training starts with a single agent tracking a fast target, extends the horizon, then fine-tunes for cooperative multi-agent and multi-target tasks. The paper claims this yields a policy that tracks up to five targets with five vehicles and, in the high-fidelity simulator, maintains average tracking errors near three metres for very fast targets and below five metres in multi-target configurations, with the same control interface usable on real vehicles.

Load-bearing premise

The load-bearing premise is that a vehicle's heading change over a 30-second step is well approximated by a linear function of rudder angle plus small Gaussian noise, so a policy trained in that cheap model remains valid over hundreds of steps in the high-fidelity simulator and ultimately at sea.

Editorial extensions

If this is right

  • A single-target tracking policy can be trained in about ten minutes on one GPU, a run the authors estimate would take months in the high-fidelity simulator, making iterative reinforcement-learning experiments practical.
  • The same final policy can be deployed with one to five agents tracking one to five targets, so operators no longer need a separate trained model for each fleet configuration.
  • Policies trained only in the simplified environment transfer to the high-fidelity simulator with comparable tracking errors, supporting the idea that sim-to-real transfer can be addressed without massive simulator compute.
  • With five vehicles the method tracks five simultaneously moving targets, whereas the authors report earlier multi-agent reinforcement learning approaches needed up to twelve vehicles to track four targets.
  • Because the Python control interface works with both the simulator and physical long-range autonomous underwater vehicles, the same trained policy is positioned for direct field deployment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the only fidelity-critical component is the fitted heading-change function, the same pipeline should transfer to other underwater vehicles or simulators by refitting that function from logged trajectories, leaving the architecture and curriculum unchanged; the paper does not test this portability.
  • The authors note that the linear heading model cannot capture long-term dependencies and diverges when used autoregressively, which suggests a cheap stress test: compare predicted and simulated trajectories over the full 1000-step mission and check whether divergence stays inside the injected 0.02-radian noise envelope.
  • The higher collision probability observed in the high-fidelity simulator for multi-agent runs implies that a collision-avoidance layer or intra-step safety check would be needed before field deployment; the paper flags this as future work, and the reported numbers already motivate it.
  • The token-based policy representation may extend naturally to heterogeneous fleets, such as vehicles with different speeds or sensors, by adding per-vehicle features to the token embeddings; the paper does not investigate this, but it follows from the architecture's permutation-invariant design.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a training pipeline for multi-agent reinforcement learning (MARL) in underwater acoustic tracking. It introduces PyLrauv, a ROS2-based Python interface to the Gazebo LRAUV high-fidelity simulator; JaxLrauv, a JAX-vectorized simplified environment whose per-step heading change is a linear function of rudder angle fit to PyLrauv trajectories; and TransfMAPPO, a transformer-based MAPPO variant with curriculum learning for progressively larger fleets. The authors report up to 30,000x speedup of JaxLrauv over Gazebo, training curves showing curriculum superiority over from-scratch MAPPO, and Gazebo evaluations in Table II with average tracking errors mostly between 3 and 6 m. The central claims are that JaxLrauv preserves LRAUV Sim dynamics and that policies trained in it transfer seamlessly to Gazebo, while being invariant to the number of agents and targets.

Significance. If the fidelity and transfer claims held, this would be a practical and timely advance: MARL training that would take months in Gazebo could run in minutes to hours on a single H100 GPU. The vectorized particle filter, the direct JaxLrauv-versus-Gazebo comparisons, and the curriculum from single-agent to multi-agent scenarios are useful contributions. However, the load-bearing 'preserving dynamics' and 'seamless transfer' claims are only partially supported: the simplified heading model is acknowledged to diverge when used autoregressively, and Table II shows nontrivial collision-probability gaps between JaxLrauv and Gazebo. The invariance-to-fleet-size claim is also tested only on configurations seen in training, except for single-target agent counts. The paper is promising, but the central claims currently outrun the evidence.

major comments (3)
  1. [IV-C and Table II] Section IV-C states that the simplified heading model 'cannot capture long-term dependencies and diverges when used autoregressively for long trajectories,' yet the abstract claims that JaxLrauv preserves Gazebo dynamics and enables seamless transfer. Table II provides a direct check and shows a safety-relevant mismatch: collision probability is 0.0% in JaxLrauv versus 15.32% in PyLrauv for the 3-agents/1-very-fast-target configuration, and 2.1% versus 10.0% for the 5-agents/5-targets configuration. Since collision avoidance is encoded in the reward through the crash penalty, this gap is not peripheral; it shows that the simplified environment is not dynamically equivalent in multi-agent close encounters. The authors should either provide additional analysis showing that this mismatch does not affect the tracking-error conclusions, or substantially temper the 'preserving dynamics' and 'seamless transfer' claims.
  2. [V-C and Table II] The Gazebo transfer evaluation in Table II is based on 50 episodes per configuration and reports only means with large standard deviations; for example, the 1-agent/1-fast-target tracking error is 17.40 +/- 21.5 in JaxLrauv and 20.33 +/- 28.38 in PyLrauv. Without confidence intervals, significance tests, or per-episode distribution comparisons, the phrase 'comparable evaluation measures' and the abstract's 'seamless transfer' claim are not established over the full 300-1000 step mission horizons. The authors should report more episodes, confidence intervals, or additional statistical comparisons to support the transfer claim.
  3. [V and Figure 6] The abstract claims that TransfMAPPO learns policies 'invariant to fleet size and number of targets.' Figure 6 varies the number of agents from 1 to 7 for a single target, which supports agent-count invariance in that setting, but no experiment varies the number of targets independently of the number of agents. The multi-target evaluations use only the symmetric configurations seen during curriculum training (2x2, 3x3, 5x5), so invariance to the number of targets is not demonstrated. Adding cross-configuration tests such as 4 agents with 5 targets or 6 agents with 3 targets would be needed to support the claim; otherwise, the claim should be revised to something like 'policies trained on multiple symmetric configurations.'
minor comments (6)
  1. [IV-B] In the reward function description, 'nεmax' appears to be a typo for 'and εmax'.
  2. [IV-E] The curriculum description contains unclear numerical expressions: '1 10 timesteps' and '1 8 timesteps' appear to have missing superscripts, and 'for a maximum 29 timesteps' is ambiguous; these should be specified precisely for reproducibility.
  3. [V-C] Please clarify whether the '50 episodes' are per configuration or total across all five configurations listed in Table II, and report episode counts alongside each row.
  4. [IV-A and I] The paper calls PyLrauv an 'open-source Python package' but provides no repository URL; please include the code or a clear availability statement.
  5. [Figure 6] The caption says the highlighted area 'represents the values seen in training,' but it is not clear which training configurations this area corresponds to; please define it in the caption or text.
  6. [Throughout] There are several typographical and formatting issues, such as 'ros gz1' and 'f.i.' instead of 'e.g.'; a careful proofreading pass would improve readability.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the fitted JaxLrauv surrogate is disclosed, and the Gazebo transfer results are independent measurements rather than consequences of the fit.

full rationale

The load-bearing claim is that policies trained in JaxLrauv transfer to Gazebo with low tracking error. Section IV-C explicitly states that the simplified heading dynamics are obtained by collecting trajectories in PyLrauv and fitting a linear model: "we collect trajectories in PyLrauv and train a supervised model that predicts δψ from θ(γ)" with "mean absolute error below 0.015 radians and a global R2 score of 0.99." This is a disclosed calibration of a surrogate, not a first-principles derivation, so the subsequent Gazebo evaluation in Table II is an independent check rather than a consequence of the fit by construction. The paper's own limitation — "such a simple model cannot capture long-term dependencies and diverges when used autoregressively for long trajectories" — and the higher Gazebo collision rates in Table II (e.g., 15.32% vs 0.0% for 3 agents and 1 very fast target) are correctness and robustness concerns, not circularity. Self-citations to TransfQMix [15] and to [31] are architectural inspiration and a LayerNorm implementation detail, respectively; neither injects an unverified premise that defines the tracking-error outcome. The speedup numbers are direct measurements of the implemented environments. Thus no load-bearing step reduces to its inputs by definition or through a self-citation chain.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The paper's central claims depend on hand-set reward thresholds, a fitted heading-change model, and several unverified modeling assumptions about the LRAUV simulator and the transformer's generalization. None of these is derived from first principles; they are either fitted to Gazebo trajectories or chosen by hand, and the reported tracking errors therefore inherit these choices. The paper introduces no new physical entity, force, or mechanism, so the invented-entities ledger is empty.

free parameters (6)
  • Heading-change linear model coefficients theta(gamma) for each (speed, dt) pair = not reported, R^2=0.99, MAE below 0.015 rad
    Fit to PyLrauv/Gazebo trajectories in Section IV-C; used as the core simplified dynamics of JaxLrauv.
  • Trajectory perturbation Gaussian noise standard deviation = 0.02 rad
    Added by hand to the heading-change model in Section IV-C to force robustness; affects transfer behavior.
  • Reward threshold epsilon_min = 10 m
    Hand-set ideal-error threshold in tracking reward, Eq. (1), Section IV-A.
  • Reward threshold epsilon_max = 50 m
    Hand-set maximum rewarding error in tracking reward, Eq. (1).
  • Follow threshold d_min = 50 m, and 100 m for fast targets
    Hand-set threshold in follow reward, Eq. (2), defining a successfully followed target.
  • Collision-safe distance d_safe = not explicitly stated
    Used in the crash penalty and chosen by hand in Section IV-A, part b.
assumptions (5)
  • domain assumption A 2D piecewise-constant velocity model with heading rotation is an adequate representation of LRAUV motion over the action interval.
    Section IV-C uses p_{t+1}=p_t+v*dt*(cos(psi+delta_psi), sin(psi+delta_psi)) and ignores 3D dynamics, currents, and yaw coupling.
  • domain assumption A linear map from rudder angle to heading change fitted once in Gazebo, plus Gaussian noise, preserves enough dynamics for policy transfer.
    Section IV-C; the paper admits long-term divergence and relies on recurrent mechanisms instead of long-horizon validation.
  • domain assumption The modeled sensor noise, communication dropouts, and partial observability in JaxLrauv match PyLrauv and Gazebo.
    Section IV-C states these are implemented manually based on data collected in Gazebo, but the noise distributions and drop-out model parameters are not provided.
  • standard math PPO and MAPPO with a centralized critic and decentralized execution form a sound backbone for the cooperative tracking task.
    Used throughout Sections III-B and IV-D; standard in the literature and not derived in this paper.
  • domain assumption The transformer's permutation-invariant token representation over agents and targets yields policies invariant to fleet size and target count.
    Section IV-D claims this property from architecture; it is supported only empirically in Figures 6-8, not proven.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Scaling Multi Agent Reinforcement Learning for Underwater Acoustic Tracking via Autonomous Vehicles." pith.science (2026). https://pith.science/paper/OTUK6HWZ

@misc{pith2026250508222,
  author       = {Pith},
  title        = {Pith review of: Scaling Multi Agent Reinforcement Learning for Underwater Acoustic Tracking via Autonomous Vehicles},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OTUK6HWZ}},
  note         = {Machine review of arXiv:2505.08222}
}
read the original abstract

Autonomous vehicles (AVs) offer a cost-effective solution for scientific missions such as underwater tracking. Reinforcement learning (RL) has emerged as a powerful method for controlling AVs, but scaling to fleets (essential for multi-target tracking or rapidly moving targets) is challenging. Multi-Agent RL (MARL) is notoriously sample-inefficient, and while high-fidelity simulators like Gazebo's LRAUV provide up to 100x faster-than-real-time single-robot simulations, they offer little speedup in multi-vehicle scenarios, making MARL training impractical. Yet, high-fidelity simulation is crucial to test complex policies and close the sim-to-real gap. To address these limitations, we develop a GPU-accelerated environment that achieves up to 30,000x speedup over Gazebo while preserving its dynamics. This enables fast, end-to-end GPU training and seamless transfer to Gazebo for evaluation. We also introduce a Transformer-based architecture (TransfMAPPO) that learns policies invariant to fleet size and number of targets, enabling curriculum learning to train larger fleets on increasingly complex scenarios. After large-scale GPU training, we perform extensive evaluations in Gazebo, showing our method maintains tracking errors below 5m even with multiple fast-moving targets.

Figures

Figures reproduced from arXiv: 2505.08222 by the authors.

Figure 1
Figure 1. Overview of our training and evaluation pipeline. PyLrauv (𝑎) is a new Python package to control multiple robots in the C++ high-fidelity LRAUV simulator. JaxLrauv (𝑏) is a GPU-accelerated, simplified environment that supports massive parallelization while preserving the dynamics of the LRAUV simulator. TransfMAPPO (𝑐) employs transformers to train progressively larger fleets of vehicles to coordinate via curriculum… view at source ↗
Figure 2
Figure 2. Five agents trained in the GPU simplified environment [PITH_FULL_IMAGE:figures/full_fig_p001_2.png] view at source ↗
Figure 3
Figure 3. Overview of TransfMAPPO architecture. E. Curriculum Learning Taking advantage of our Transformer-based architecture, we employ curriculum learning to progressively learn Multi￾Agent policies starting from a Single-Agent policy (see [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (7 more)
Figure 5
Figure 5. Figure 5: Training multiple agents to track a very fast target. [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Multi-Robot Tracking Evaluation. The highlighted area [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Training Multiple Agents to Follow Fast Targets. [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 11
Figure 11. Figure 11: Coordination when tracking a very fast target. Notice how agents react to a rapid change of direction of the target by waiting for each other in order not to lose communication and then rapidly moving together. 400 600 800 X 100 300 500 700 Y Step 90 100 m 300 500 700…
Figure 12
Figure 12. Figure 12: Coordination in a multi-target setting. Notice how blue and orange agents resolve the ”traffic” problem by making circles to wait for other agents’ passing, and one target is dynamically reassigned from one agent to another. References [1] Nikolaos D Zarokanellos et a…
Figure 9
Figure 9. Figure 9: (a): Single-agent tracking, target velocity 0.66× of agent velocity. Video. (b): Multi-agent tracking, target velocity 0.88x of agents’ velocity. Video. 300 200 100 X 800 900 1000 1100 1200 Y Target Velocity: 0.3x of Agent 100 m 800 900 1000 1100 1300 X 1600 1800 1900 …
Figure 10
Figure 10. Figure 10: Agents’ trajectory according to target speed. Notice the change in the curvature of the agent’s trajectory to optimally track targets moving at different speeds. ACKNOWLEDGMENT This work acknowledges the Spanish Ministerio de Ciencia, Innovacion y Uni- ´ versidades (B…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 25 canonical work pages

  1. [1]

    Frontal dynamics in the Alboran sea: 1. Coherent 3D pathways at the Almeria- Oran front using underwater glider observations

    Nikolaos D Zarokanellos et al. “Frontal dynamics in the Alboran sea: 1. Coherent 3D pathways at the Almeria- Oran front using underwater glider observations”. In:Journal of Geophysical Research: Oceans127.3 (2022), e2021JC017405

    work page 2022

  2. [2]

    Mobile robotic platforms for the acoustic tracking of deep-sea demersal fishery resources

    Ivan Masmitja et al. “Mobile robotic platforms for the acoustic tracking of deep-sea demersal fishery resources”. In:Science Robotics5.48 (2020), eabc3701

    work page 2020

  3. [3]

    A system of coordinated au- tonomous robots for Lagrangian studies of microbes in the oceanic deep chlorophyll maximum

    Yanwu Zhang et al. “A system of coordinated au- tonomous robots for Lagrangian studies of microbes in the oceanic deep chlorophyll maximum”. In:Science Robotics6.50 (2021), eabb9138.doi:10 . 1126 / scirobotics.abb9138

    work page 2021

  4. [4]

    Spatial ecology of Norway lobster Nephrops norvegicus in Mediterranean deep-water en- vironments: implications for designing no-take marine reserves

    Maria Vigo et al. “Spatial ecology of Norway lobster Nephrops norvegicus in Mediterranean deep-water en- vironments: implications for designing no-take marine reserves”. In:Marine Ecology Progress Series674 (2021), pp. 173–188

    work page 2021

  5. [5]

    Underwater sensor networks: applications, advances and challenges

    John Heidemann, Milica Stojanovic, and Michele Zorzi. “Underwater sensor networks: applications, advances and challenges”. In:Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engi- neering Sciences370.1958 (2012), pp. 158–175

    work page 2012

  6. [6]

    Characterizing snow crab (Chionoe- cetes opilio) movements in the Sydney Bight (Nova Sco- tia, Canada): a collaborative approach using multiscale acoustic telemetry

    David Cote et al. “Characterizing snow crab (Chionoe- cetes opilio) movements in the Sydney Bight (Nova Sco- tia, Canada): a collaborative approach using multiscale acoustic telemetry”. In:Canadian Journal of Fisheries and Aquatic Sciences76.2 (2019), pp. 334–346

    work page 2019

  7. [7]

    Dynamic robotic tracking of under- water targets using reinforcement learning

    I. Masmitja et al. “Dynamic robotic tracking of under- water targets using reinforcement learning”. In:Science Robotics8.80 (2023), eade7811.doi:10 . 1126 / scirobotics.ade7811

    work page 2023

  8. [8]

    Multi-AUV cooperative underwa- ter multi-target tracking based on dynamic-switching- enabled multi-agent reinforcement learning

    Shengbo Wang et al. “Multi-AUV cooperative underwa- ter multi-target tracking based on dynamic-switching- enabled multi-agent reinforcement learning”. In:IEEE Transactions on Mobile Computing(2024)

    work page 2024

Show all 31 references
  1. [9]

    Secure and cooperative target tracking via AUV swarm: A reinforcement learning approach

    Zhaoqi Yang et al. “Secure and cooperative target tracking via AUV swarm: A reinforcement learning approach”. In:2021 IEEE Global Communications Conference (GLOBECOM). IEEE. 2021, pp. 1–6

    2021

  2. [10]

    From Concept to Field Tests: Accelerated Development of Multi-AUV Mis- sions Using a High-Fidelity Faster-than-Real-Time Simulator

    Timothy R. Player et al. “From Concept to Field Tests: Accelerated Development of Multi-AUV Mis- sions Using a High-Fidelity Faster-than-Real-Time Simulator”. In:2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, May 2023, pp. 3102–3108.doi:10 . 1109 /...

    2023

  3. [11]

    Robot Operating System 2: Design, architecture, and uses in the wild

    Steven Macenski et al. “Robot Operating System 2: Design, architecture, and uses in the wild”. In:Science Robotics7.66 (2022), eabm6074.doi:10 . 1126 / scirobotics.abm6074

    2022

  4. [12]

    JaxMARL: Multi- Agent RL Environments in JAX

    Alexander Rutherford et al. “JaxMARL: Multi- Agent RL Environments in JAX”. In:arXiv preprint arXiv:2311.10090(2023)

    2023 arXiv

  5. [13]

    The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games

    Chao Yu et al. “The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games”. In: (2022). arXiv: 2103.01955 [cs.LG].url:https://arxiv. org/abs/2103.01955

    2022 arXiv

  6. [14]

    Synchronization of Multiagent Systems Using Event-Triggered and Self-Triggered Broadcasts

    Jo ˜ao Almeida, Carlos Silvestre, and Ant ´onio Pas- coal. “Synchronization of Multiagent Systems Using Event-Triggered and Self-Triggered Broadcasts”. In: IEEE Transactions on Automatic Control62.9 (2017), pp. 4741–4746.doi:10 . 1109 / TAC . 2017 . 2671029

    2017

  7. [15]

    TransfQMix: Transformers for Leveraging the Graph Structure of Multi-Agent Reinforcement Learning Problems

    Matteo Gallici, Mario Martin, and Ivan Masmitja. “TransfQMix: Transformers for Leveraging the Graph Structure of Multi-Agent Reinforcement Learning Problems”. In:Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems. AAMAS ’23. London, Un...

    2023

  8. [16]

    Scalable multi-agent rein- forcement learning through intelligent information ag- gregation

    Siddharth Nayak et al. “Scalable multi-agent rein- forcement learning through intelligent information ag- gregation”. In:International conference on machine learning. PMLR. 2023, pp. 25817–25833

    2023

  9. [17]

    Graph neural network- based multi-agent reinforcement learning for resilient distributed coordination of multi-robot systems

    Anthony Goeckner et al. “Graph neural network- based multi-agent reinforcement learning for resilient distributed coordination of multi-robot systems”. In: 2024 IEEE/RSJ International Conference on Intel- ligent Robots and Systems (IROS). IEEE. 2024, pp. 5732–5739

    2024

  10. [18]

    Transformer-based multi-agent rein- forcement learning for generalization of heterogeneous multi-robot cooperation

    Yuxin Cai et al. “Transformer-based multi-agent rein- forcement learning for generalization of heterogeneous multi-robot cooperation”. In:2024 IEEE/RSJ Interna- tional Conference on Intelligent Robots and Systems (IROS). IEEE. 2024, pp. 13695–13702

    2024

  11. [19]

    UPDeT: Universal Multi-agent RL via Policy Decoupling with Transformers

    Siyi Hu et al. “UPDeT: Universal Multi-agent RL via Policy Decoupling with Transformers”. In:Interna- tional Conference on Learning Representations. 2020

    2020

  12. [20]

    Multi-agent reinforcement learning is a sequence modeling problem

    Muning Wen et al. “Multi-agent reinforcement learning is a sequence modeling problem”. In:Advances in Neural Information Processing Systems35 (2022), pp. 16509–16521

    2022

  13. [21]

    Automatic curriculum learning for large-scale cooperative multiagent systems

    Tianle Zhang et al. “Automatic curriculum learning for large-scale cooperative multiagent systems”. In:IEEE Transactions on Emerging Topics in Computational Intelligence7.3 (2022), pp. 912–930

    2022

  14. [22]

    From few to more: Large-scale dynamic multiagent curriculum learning

    Weixun Wang et al. “From few to more: Large-scale dynamic multiagent curriculum learning”. In:Proceed- ings of the AAAI conference on artificial intelligence. Vol. 34. 05. 2020, pp. 7293–7300

    2020

  15. [23]

    Robust Range-Only Beacon Localization

    Edwin Olson, John J. Leonard, and Seth Teller. “Robust Range-Only Beacon Localization”. In:IEEE Journal of Oceanic Engineering31.4 (2006), pp. 949–958

    2006

  16. [24]

    Observ- ability based control in range-only underwater vehicle localization

    Jake D. Quenzer and Kristi A. Morgansen. “Observ- ability based control in range-only underwater vehicle localization”. In:2014 American Control Conference. 2014, pp. 4702–4707.doi:10.1109/ACC.2014. 6859032

    2014 doi

  17. [25]

    Positioning and navigation sys- tems for robotic underwater vehicles

    Alex Alcocer Penas. “Positioning and navigation sys- tems for robotic underwater vehicles”. In:Doctor thesis, Instituto Superior Tcnico(2009)

    2009

  18. [26]

    Observ- ability based control in range-only underwater vehicle localization

    Jake D Quenzer and Kristi A Morgansen. “Observ- ability based control in range-only underwater vehicle localization”. In:2014 American control conference. IEEE. 2014, pp. 4702–4707

    2014

  19. [27]

    Robust range-only beacon localization

    Edwin Olson, John J Leonard, and Seth Teller. “Robust range-only beacon localization”. In:IEEE Journal of Oceanic Engineering31.4 (2006), pp. 949–958

    2006

  20. [28]

    Springer, 2016

    Frans A Oliehoek and Christopher Amato.A concise in- troduction to decentralized POMDPs. Springer, 2016

    2016

  21. [29]

    Tabish Rashid et al.QMIX: Monotonic Value Func- tion Factorisation for Deep Multi-Agent Reinforcement Learning. 2018. arXiv:1803.11485 [cs.LG].url: https://arxiv.org/abs/1803.11485

    2018 arXiv

  22. [30]

    John Schulman et al.Proximal Policy Optimization Algorithms. 2017. arXiv:1707 . 06347 [cs.LG]. url:https://arxiv.org/abs/1707.06347

    2017 arXiv

  23. [31]

    Matteo Gallici et al.Simplifying Deep Temporal Differ- ence Learning. 2024. arXiv:2407.04811 [cs.LG]. url:https://arxiv.org/abs/2407.04811

    2024 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.