Pith. sign in

REVIEW 4 major objections 6 minor 30 references

Towards Learning Scalable Agile Dynamic Motion Planning for Robosoccer Teams with Policy Optimization

T0 review · 4 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a policy-gradient motion planner trained on one static agent generalizes to small dynamic Robosoccer teams, trading some collisions for target coverage.

desk verdict A small, honest RL demo on a new robosoccer-like testbed; the transfer claim is asserted more than tested. read the letter →

arxiv 2502.05526 v1 pith:2WOQ3BYF submitted 2025-02-08 cs.RO cs.AIcs.LGcs.MA

Brandon Ho , Batuhan Altundas , Matthew Gombolay This is my paper · ORCID
classification cs.ROcs.AIcs.LGcs.MA
keywords motionplanningmulti-agentsystemspolicyoptimizationreinforcementlearningdynamicobstacleavoidanceRobosoccerdecentralizednavigationscalable
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that a simple end-to-end trainable neural policy performs continuous-space motion planning for a multi-agent Robosoccer-like game, steering agents to ordered targets while avoiding static and moving obstacles. Its central claim is that a policy trained with policy-gradient optimization on a single agent in a static environment—three targets and ten fixed obstacles—transfers to decentralized use by several agents in dynamic domains with mobile obstacles. If that transfer holds, an agile collision-avoiding controller can be learned without multi-agent training or replanning, then copied to each agent at runtime. The authors also state plainly that scalability is not yet solved and is left to future graph-based work.

What carries the argument

The load-bearing object is a reward-shaped policy-gradient controller. The reward is $R = -\alpha D(u,v) + \sum_i g(D(u,o_i), r_{o_i})$, where $D$ is Euclidean distance, $\alpha = 10$, and the obstacle term is asymmetric: $g(D,r) = \beta_1(D-r)$ outside the keep-out radius and $\beta_2(D-r)$ inside it, with $\beta_1 = 1$ and $\beta_2 = 100$. That asymmetry makes collisions dominate the learning signal only when an obstacle is inside the keep-out radius, while target distance keeps pulling the agent forward. Each agent samples a velocity offset $\langle \delta x, \delta y \rangle$ from a small neural policy, and the policy is trained with a policy-gradient objective in which the advantage is the reward for a sampled action minus the batch mean reward. The training MDP is fully observable, but deployment is decentralized: each agent sees only its own target sequence and the ten nearest obstacles.

What would settle it

Run the trained policy in a dynamic three-agent domain while increasing the number of obstacles beyond ten, so the nearest-ten observation necessarily hides some threatening obstacle; if the learned policy's collision rate then reaches or exceeds the straight-line baseline's rate, or its target completion falls below baseline in problems the baseline can finish, the static-to-dynamic transfer claim fails outside the tested range.

Watch

Extended reading notes

Core claim

The paper's core discovery is that one policy-gradient model, receiving the agent's location, its current target, and the positions, radii, and distances of the ten nearest obstacles, can learn to travel through a sequence of targets in continuous space without collisions, and the same weights work when copied to each of three agents facing moving obstacles. This is demonstrated in a static single-agent domain, a dynamic multi-agent domain with three agents, three targets, and ten mobile obstacles, and a larger dynamic domain with three agents and ten targets. In the two smaller domains the learned policy reaches targets at a rate near the no-obstacle-knowledge baseline and collisions decrease over training; in the larger domain the paper reports a performance drop that it attributes to observing only the nearest ten obstacles.

Load-bearing premise

The transfer claim rests on the assumption that a policy learned by one agent moving among three static targets and ten static obstacles keeps working when the same weights are used by three agents moving among mobile obstacles and, in the large case, ten targets.

Editorial extensions

If this is right

  • A single trained policy can be duplicated across multiple heterogeneous agents and executed decentralized, with no per-agent retraining.
  • Because the model outputs continuous displacement deltas each time step, it avoids the per-step cost of rerunning a classical planner such as RRT* or A*.
  • Because training happens in a simple static single-agent environment, acquiring the dynamic collision-avoidance behavior is cheaper than training directly in the multi-agent dynamic setting.
  • The reported drop in the larger dynamic domain implies that the nearest-ten-obstacle observation, not the policy itself, is the current scaling bottleneck.
  • The method can be paired with a higher-level scheduler that supplies target sequences, so task allocation and motion planning remain separable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the transfer claim is right, it suggests a curriculum: dynamic multi-agent skills can be bootstrapped from static single-agent practice, and adding modest domain randomization during that practice should close most of the remaining gap to the baseline.
  • The asymmetric obstacle penalty is a plausible mechanism for the behavior; an ablation that sets $\beta_1 = \beta_2$ would reveal whether the sharp inside-radius penalty is what teaches avoidance.
  • The reported large-domain drop predicts that any observation representation invariant to obstacle count, such as a set- or graph-based encoder, should recover the performance loss in the ten-target domain.
  • An adversarial variant in which moving obstacles actively pursue the agent, rather than moving in a straight line at constant speed, would test whether the transferred policy's avoidance behavior persists under harder dynamics.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a policy-gradient-based motion planning policy for decentralized multi-agent navigation in continuous Robosoccer-like domains. The model is a simple neural network mapping agent, target, and up to the N closest obstacle observations to a velocity action, and it is trained on a single-agent static environment with 3 targets and 10 obstacles using a hand-designed reward. The authors evaluate the trained policy on three domains: a static single-agent domain, a dynamic multi-agent domain with 3 agents, 3 targets, and 10 moving obstacles, and a larger dynamic domain with 3 agents and 10 targets. They compare against a straight-line target-to-target baseline using collisions, targets reached, and a weighted score, reporting results graphically. The central claim is that static-environment training transfers to dynamic multi-agent settings. The paper also discusses limitations and proposes GNN-based extensions for scalability.

Significance. If the transfer claim were established, the contribution would be a simple, decentralized, end-to-end trainable policy for continuous-space navigation with moving obstacles, together with a reusable environment for Robosoccer-style multi-agent motion planning. Strengths of the paper are the clean problem formulation as an MDP, the use of a pure policy-gradient method rather than a learned heuristic for a classical planner, and an honest discussion of the observability limitation. However, the experimental evaluation as presented does not support the transfer claim: there are no matched static controls, no numerical summaries or error bars, no significance tests, and no comparison with the classical planners named in the introduction. The paper is a reasonable starting point, but its central claim is currently under-supported.

major comments (4)
  1. [IV, Multi-Agent Dynamic Domain] The sentence 'training learned on simple path planning in static environments is transferable to dynamic domains' (Section IV) is an assertion rather than a demonstrated result: the three test domains differ simultaneously in number of agents, number of targets, and obstacle motion, so the dynamic-domain results cannot be attributed to transfer of static obstacle-avoidance behavior. A matched static control with the same numbers of agents and targets but static obstacles, and ideally a policy trained directly on the dynamic task, are needed to separate transfer from task difficulty.
  2. [IV, Figs. 2-4 and metrics] The evaluation reports no numerical means, standard deviations, or significance tests across the three seeds and 100 test problems per domain; the figures show only graphical curves and the x-axis is not labeled. Because the straight-line baseline reaches every target by construction, the target-reach metric is trivially favorable to the baseline, and the collision and weighted-score curves alone do not establish the model's advantage. The introduction motivates the method by speed and smoothness relative to RRT* and A*, yet no comparison to those planners or to the learning-based methods in Table I is provided.
  3. [Eq. (4) vs. Eq. (1)] The evaluation score in Eq. (4) uses the same constant alpha=10 as the reward in Eq. (1), so the weighted score is partly aligned with the training objective by construction and is not an independent evaluation measure. The authors should report the raw metrics separately and either vary alpha in evaluation or use a separately chosen score weight; otherwise the weighted-score comparisons are hard to interpret.
  4. [IV, Multi-Agent Dynamic Domain (Large)] The attribution of the large-domain performance drop to 'lack of full-observability' is a plausible hypothesis but is not tested: no experiment varies the observation window N while holding the number of obstacles fixed, and the fixed N=10 window is a representation-capacity choice rather than evidence about transfer. An ablation over N, or over obstacle count with N matched, is needed before this explanation is used.
minor comments (6)
  1. [Abstract] The phrase 'avoiding moving targets' appears to be a typo for 'avoiding moving obstacles'; as written, it conflicts with the stated objective of reaching targets.
  2. [III-B / III-C] The problem is called a fully observable MDP in Section III-B, but the observation consists only of the N closest obstacles (Section III-C); this inconsistency, and the conclusion's claim that the model 'is able to perform under partial observability,' should be clarified because partial observability is neither formalized nor explicitly tested.
  3. [III-D, Eq. (2)] The piecewise condition in Eq. (2) should specify the second branch as the case D-r < 0; 'otherwise' is ambiguous, and the sign convention for beta2 should be stated explicitly.
  4. [III-D, Eq. (3)] Equation (3) uses an advantage term A^pi_t that is not formally defined; the text says it is computed by subtracting the mean batch reward, but the notation and the policy-gradient estimator should be stated more precisely.
  5. [IV] The x-axes of Figures 2-4 are not labeled; specify whether they show training steps, evaluation episodes, or environment time steps, and add axis labels and legends for the baseline and model curves.
  6. [Throughout] There are repeated typographical errors, including 'heterogenous' for 'heterogeneous', 'optimally' for 'optimality' in the abstract, and 'unmatured' in the introduction; a careful proofread is needed.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the RL pipeline is trained on a hand-specified reward and evaluated on held-out test problems; the only mild overlap is the shared α=10 in the reward and the weighted score, which does not force any result.

full rationale

The paper's derivation chain is a standard policy-gradient training loop: a neural network maps observations to (δx, δy) actions, is trained against the dense reward in Eq. 1, and is evaluated on three held-out problem suites using collisions, targets reached, and Eq. 4. No predicted quantity is refit from the evaluation data, and no 'prediction' is equivalent to an input by construction. The same constant α=10 appears in the reward (Eq. 1) and the weighted score (Eq. 4), making the metric somewhat aligned with the training objective, but the score is a separate evaluation statistic and does not determine the reported comparisons. Self-citations ([4], [26]-[28]) are contextual or future-work references and are not load-bearing; no uniqueness theorem or prior result by the same authors is invoked to force the model choice. The static-to-dynamic transfer sentence in Section IV ('training learned on simple path planning in static environments is transferable to dynamic domains') is an empirical assertion without a matched static control or an explicit transfer mechanism, but that is an experimental-support limitation, not circularity. The baseline's always-reaching-targets property weakens the target-reach comparison, but this is a benchmark-design issue, not a circular argument. The paper also acknowledges its scalability and partial-observability limitations. Therefore no circular step meets the quoted-reduction evidence bar.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central demonstration depends on hand-set reward weights, a truncated observation window, and several domain assumptions about collision and target reaching. None of these are varied or validated, and no public artifacts are provided.

free parameters (4)
  • reward weights (alpha, beta1, beta2) = 10, 1, 100
    Hand-set constants in Eqs. 1 and 2; no sensitivity analysis is reported.
  • observation limit N closest obstacles = 10
    Model input is truncated to the 10 nearest obstacles; this cap is the stated reason for scalability limits in the large domain.
  • evaluation weight alpha in weighted score = 10
    Eq. 4 uses the same alpha as the reward; choosing this weight shapes the reported metric.
  • MDP hyperparameters gamma, batch size, learning rate, weight decay = 0.99, 8, 8e-3, 1e-4
    Reported in Section IV; chosen without ablation.
assumptions (4)
  • domain assumption The motion planning problem can be modeled as a fully observable MDP with state including all agent, target, and obstacle information.
    Section III-B states the MDP formulation, but the implemented model observes only the ten closest obstacles and treats unassigned targets as obstacles, so full observability is not actually used.
  • domain assumption A target is reached whenever the agent arrives at the target location, independent of path, and the straight-line baseline always reaches all targets.
    Section III-C and Section IV: the baseline 'always able to reach the provided targets due to taking the shortest path... in the continuous domain'; this ignores obstacle-induced collision constraints on reaching.
  • standard math The policy gradient advantage can be estimated by subtracting the mean reward of a batch from each reward.
    Equation 3 follows the standard REINFORCE-with-baseline form; the choice of mean reward baseline is standard but not motivated.
  • domain assumption Obstacles and agents are represented as circles with keep-out radii, with collision defined by center distance below radius.
    Equation 2 defines the threat and collision condition; this abstracts away robot shape and dynamics.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Learning Scalable Agile Dynamic Motion Planning for Robosoccer Teams with Policy Optimization." pith.science (2026). https://pith.science/paper/2WOQ3BYF

@misc{pith2026250205526,
  author       = {Pith},
  title        = {Pith review of: Towards Learning Scalable Agile Dynamic Motion Planning for Robosoccer Teams with Policy Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2WOQ3BYF}},
  note         = {Machine review of arXiv:2502.05526}
}
read the original abstract

In fast-paced, ever-changing environments, dynamic Motion Planning for Multi-Agent Systems in the presence of obstacles is a universal and unsolved problem. Be it from path planning around obstacles to the movement of robotic arms, or in planning navigation of robot teams in settings such as Robosoccer, dynamic motion planning is needed to avoid collisions while reaching the targeted destination when multiple agents occupy the same area. In continuous domains where the world changes quickly, existing classical Motion Planning algorithms such as RRT* and A* become computationally expensive to rerun at every time step. Many variations of classical and well-formulated non-learning path-planning methods have been proposed to solve this universal problem but fall short due to their limitations of speed, smoothness, optimally, etc. Deep Learning models overcome their challenges due to their ability to adapt to varying environments based on past experience. However, current learning motion planning models use discretized environments, do not account for heterogeneous agents or replanning, and build up to improve the classical motion planners' efficiency, leading to issues with scalability. To prevent collisions between heterogenous team members and collision to obstacles while trying to reach the target location, we present a learning-based dynamic navigation model and show our model working on a simple environment in the concept of a simple Robosoccer Game.

Figures

Figures reproduced from arXiv: 2502.05526 by the authors.

Figure 1
Figure 1. Sample policy of the Trained Model blue agent [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Single-Agent Static Domain with 3 Targets, and 10 Obstacles Performance for Collisions (lower is better), Number of [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Multi-Agent Dynamic Domain with 3 Agents, 3 Targets, and 10 Obstacles Performance for Collisions (lower is better), [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Multi-Agent Dynamic Domain (Large) with 3 Agents, 10 Targets, and 10 Obstacles Performance for Collisions (lower [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 23 canonical work pages

  1. [1]

    A Review of Mo- tion Planning Techniques for Automated Vehicles,

    D. Gonz ´alez, J. P´erez, V . Milan´es, and F. Nashashibi, “A Review of Mo- tion Planning Techniques for Automated Vehicles,” IEEE Transactions on Intelligent Transportation Systems , vol. 17, no. 4, pp. 1135–1145, Apr. 2016

    work page 2016

  2. [2]

    A Survey of Deep Learning Techniques for Autonomous Driving

    S. Grigorescu, B. Trasnea, T. Cocias, and G. Macesanu, “A Survey of Deep Learning Techniques for Autonomous Driving,” Journal of Field Robotics, vol. 37, no. 3, pp. 362–386, Apr. 2020, arXiv:1910.07738 [cs]. [Online]. Available: http://arxiv.org/abs/1910.07738

    work page Pith review arXiv 2020

  3. [3]

    A Survey of Motion Planning and Control Techniques for Self-Driving Urban Vehicles,

    B. Paden, M. ˇC´ap, S. Z. Yong, D. Yershov, and E. Frazzoli, “A Survey of Motion Planning and Control Techniques for Self-Driving Urban Vehicles,” IEEE Transactions on Intelligent V ehicles , vol. 1, no. 1, pp. 33–55, Mar. 2016

    work page 2016

  4. [4]

    Athletic Mobile Manipulator System for Robotic Wheelchair Tennis

    Z. Zaidi, D. Martin, N. Belles, V . Zakharov, A. Krishna, K. M. Lee, P. Wagstaff, S. Naik, M. Sklar, S. Choi, Y . Kakehi, R. Patil, D. Mallemadugula, F. Pesce, P. Wilson, W. Hom, M. Diamond, B. Zhao, N. Moorman, R. Paleja, L. Chen, E. Seraj, and M. Gombolay, “Athletic Mobile Manipulator System for Robotic Wheelchair Tennis,” Feb. 2023, arXiv:2210.02517 [c...

    work page Pith review arXiv 2023

  5. [5]

    The RoboCup synthetic agent challenge 97,

    H. Kitano, M. Tambe, P. Stone, M. Veloso, S. Coradeschi, E. Osawa, H. Matsubara, I. Noda, and M. Asada, “The RoboCup synthetic agent challenge 97,” in RoboCup-97: Robot Soccer World Cup I , ser. Lecture Notes in Computer Science, H. Kitano, Ed. Berlin, Heidelberg: Springer, 1998, pp. 62–73

    work page 1998

  6. [6]

    Sampling-based Algorithms for Optimal Motion Planning,

    S. Karaman and E. Frazzoli, “Sampling-based Algorithms for Optimal Motion Planning,” May 2011, arXiv:1105.1186 [cs]. [Online]. Available: http://arxiv.org/abs/1105.1186

    arXiv 2011

  7. [7]

    Path Planning with Modified a Star Algorithm for a Mobile Robot,

    F. Ducho ˇn, A. Babinec, M. Kajan, P. Be ˇno, M. Florek, T. Fico, and L. Juri ˇsica, “Path Planning with Modified a Star Algorithm for a Mobile Robot,” Procedia Engineering , vol. 96, pp. 59–69,

    work page

  8. [8]

    Rapidly-exploring random trees : a new tool for path planning

    S. M. LaValle, “Rapidly-exploring random trees : a new tool for path planning.” [Online]. Available: http://msl.cs.uiuc.edu/ ∼lavalle/papers/ Lav98c.pdf

    work page

Show all 30 references
  1. [9]

    Incremental Sampling-based Algorithms for Optimal Motion Planning,

    S. Karaman and E. Frazzoli, “Incremental Sampling-based Algorithms for Optimal Motion Planning,” 2010. [Online]. Available: https: //arxiv.org/pdf/1005.0416.pdf

    2010 arXiv

  2. [10]

    Motion Planning for Autonomous Driving: The State of the Art and Future Perspectives,

    S. Teng, X. Hu, P. Deng, B. Li, Y . Li, Y . Ai, D. Yang, L. Li, Z. Xuanyuan, F. Zhu, and L. Chen, “Motion Planning for Autonomous Driving: The State of the Art and Future Perspectives,” IEEE Transactions on Intelligent V ehicles, vol. 8, no. 6, pp. 3692–3711, Jun. 2023

    2023

  3. [11]

    Autonomous Driving with Deep Learning: A Survey of State-of-Art Technologies,

    Y . Huang and Y . Chen, “Autonomous Driving with Deep Learning: A Survey of State-of-Art Technologies,” Jul. 2020, arXiv:2006.06091 [cs]. [Online]. Available: http://arxiv.org/abs/2006.06091

    2020 arXiv

  4. [12]

    Neural RRT*: Learning-Based Optimal Path Planning,

    J. Wang, W. Chi, C. Li, C. Wang, and M. Q.-H. Meng, “Neural RRT*: Learning-Based Optimal Path Planning,” 2020. [Online]. Available: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9037111

    2020

  5. [13]

    Path Planning using Neural A* Search,

    R. Yonetani, T. Taniai, M. Barekatain, M. Nishimura, and A. Kanezaki, “Path Planning using Neural A* Search,” Jul. 2021, arXiv:2009.07476 [cs, stat]. [Online]. Available: http://arxiv.org/abs/2009.07476

    2021 arXiv

  6. [14]

    Reinforcement Learning with A* and a Deep Heuristic,

    A. Keselman, S. Ten, A. Ghazali, and M. Jubeh, “Reinforcement Learning with A* and a Deep Heuristic,” Nov. 2018, arXiv:1811.07745 [cs, stat]. [Online]. Available: http://arxiv.org/abs/1811.07745

    2018 arXiv

  7. [15]

    Path Planning via an Improved DQN-Based Learning Policy,

    L. Lv, S. Zhang, D. Ding, and Y . Wang, “Path Planning via an Improved DQN-Based Learning Policy,” IEEE Access , vol. 7, pp. 67 319–67 330, 2019

    2019

  8. [16]

    Learning Sampling Distributions for Robot Motion Planning,

    B. Ichter, J. Harrison, and M. Pavone, “Learning Sampling Distributions for Robot Motion Planning,” in 2018 IEEE International Conference on Robotics and Automation (ICRA) , May 2018, pp. 7087–7094, iSSN: 2577-087X

    2018

  9. [17]

    Graph Neural Networks for Motion Planning,

    A. Khan, A. Ribeiro, V . Kumar, and A. G. Francis, “Graph Neural Networks for Motion Planning,” Dec. 2020, arXiv:2006.06248 [cs]. [Online]. Available: http://arxiv.org/abs/2006.06248

    2020 arXiv

  10. [18]

    Reducing Collision Checking for Sampling-Based Motion Planning Using Graph Neural Networks,

    C. Yu and S. Gao, “Reducing Collision Checking for Sampling-Based Motion Planning Using Graph Neural Networks,” in Advances in Neural Information Processing Systems, vol. 34. Curran Associates, Inc., 2021, pp. 4274–4289

    2021

  11. [19]

    Graph Neural Networks for Decentralized Multi-Robot Path Planning,

    Q. Li, F. Gama, A. Ribeiro, and A. Prorok, “Graph Neural Networks for Decentralized Multi-Robot Path Planning,” in 2020 IEEE/RSJ Interna- tional Conference on Intelligent Robots and Systems (IROS) , Oct. 2020, pp. 11 785–11 792, iSSN: 2153-0866

    2020

  12. [20]

    MAPPER: Multi-Agent Path Planning with Evolutionary Reinforce- ment Learning in Mixed Dynamic Environments,

    Z. Liu, B. Chen, H. Zhou, G. Koushik, M. Hebert, and D. Zhao, “MAPPER: Multi-Agent Path Planning with Evolutionary Reinforce- ment Learning in Mixed Dynamic Environments,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , Oct. 2020, pp. 11 74...

    2020

  13. [21]

    Learning-based Motion Planning in Dynamic Environments Using GNNs and Temporal Encoding,

    R. Zhang, C. Yu, J. Chen, C. Fan, and S. Gao, “Learning-based Motion Planning in Dynamic Environments Using GNNs and Temporal Encoding,” Oct. 2022, arXiv:2210.08408 [cs]. [Online]. Available: http://arxiv.org/abs/2210.08408

    2022 arXiv

  14. [22]

    GRSTAPS: Graphically Recursive Simultaneous Task Allocation, Planning, and Scheduling,

    A. Messing, G. Neville, S. Chernova, S. Hutchinson, and H. Ravichan- dar, “GRSTAPS: Graphically Recursive Simultaneous Task Allocation, Planning, and Scheduling,” The International Journal of Robotics Research, vol. 41, no. 2, pp. 232–256, Feb. 2022. [Online]. Available: http:...

    2022 doi

  15. [23]

    Deep Reinforcement Learning in Parameterized Action Space,

    M. Hausknecht and P. Stone, “Deep Reinforcement Learning in Parameterized Action Space,” Feb. 2016, arXiv:1511.04143 [cs]. [Online]. Available: http://arxiv.org/abs/1511.04143

    2016 arXiv

  16. [24]

    Policy gradient methods,

    J. Peters, “Policy gradient methods,” Scholarpedia, vol. 5, no. 11, p. 3698, Nov. 2010. [Online]. Available: http://www.scholarpedia.org/ article/Policy gradient methods

    2010

  17. [25]

    Adam: A Method for Stochastic Optimization,

    D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization,” Jan. 2017, arXiv:1412.6980 [cs]. [Online]. Available: http://arxiv.org/abs/1412.6980

    2017 arXiv

  18. [26]

    Human-Robot Teaming: Grand Challenges,

    M. Natarajan, E. Seraj, B. Altundas, R. Paleja, S. Ye, L. Chen, R. Jensen, K. C. Chang, and M. Gombolay, “Human-Robot Teaming: Grand Challenges,” Current Robotics Reports , vol. 4, no. 3, pp. 81–100, Aug. 2023. [Online]. Available: https://link.springer.com/10. 1007/s43154-023-00103-1

    2023

  19. [27]

    Learning Coordination Policies over Heterogeneous Graphs for Human-Robot Teams via Recurrent Neural Schedule Propagation,

    B. Altundas, Z. Wang, J. Bishop, and M. Gombolay, “Learning Coordination Policies over Heterogeneous Graphs for Human-Robot Teams via Recurrent Neural Schedule Propagation,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , Oct. 2022, pp. 11 ...

    2022 arXiv

  20. [28]

    Heterogeneous graph attention networks for scalable multi-robot scheduling with temporospatial constraints,

    Z. Wang, C. Liu, and M. Gombolay, “Heterogeneous graph attention networks for scalable multi-robot scheduling with temporospatial constraints,” Autonomous Robots , vol. 46, no. 1, pp. 249–268, Jan. 2022. [Online]. Available: https://link.springer.com/10.1007/s10514-021-09997-2

    2022 doi

  21. [29]

    Coordination in Adversarial Sequential Team Games via Multi-Agent Deep Reinforcement Learning,

    A. Celli, M. Ciccone, R. Bongo, and N. Gatti, “Coordination in Adversarial Sequential Team Games via Multi-Agent Deep Reinforcement Learning,” Dec. 2019, arXiv:1912.07712 [cs]. [Online]. Available: http://arxiv.org/abs/1912.07712

    2019 arXiv

  22. [2014]

    Available: https://linkinghub.elsevier.com/retrieve/pii/ S187770581403149X

    [Online]. Available: https://linkinghub.elsevier.com/retrieve/pii/ S187770581403149X

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.