Thinking Beyond Visibility: A Near-Optimal Policy Framework for Locally Interdependent Multi-Agent MDPs

Alex DeWeese; Guannan Qu

REVIEW 3 major objections 4 minor 20 references

Thinking Beyond Visibility: A Near-Optimal Policy Framework for Locally Interdependent Multi-Agent MDPs

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper establishes a class of decentralized policies—built by solving an extended-visibility cutoff MDP and extracting a belief-based execution policy—whose performance gap to the fully observable joint optimum shrinks exponentially…

desk verdict A genuinely new policy class and an honest but nontrivial gap: the main theorem rests on four 'Consistent Performance Policy' inequalities that are asserted, not proved. read the letter →

arxiv 2506.04215 v1 pith:Z4MXWFCP submitted 2025-06-04 cs.MA cs.AIcs.LGmath.OC

Alex DeWeese , Guannan Qu This is my paper · ORCID

classification cs.MAcs.AIcs.LGmath.OC MSC 90C4068T4293A16

keywords LocallyInterdependentMulti-AgentMDPDec-POMDPpartialobservabilitydecentralizedpoliciescutoffpolicyexponentialnear-optimalityPenaltyJitteringcooperativenavigation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper addresses decentralized multi-agent problems such as cooperative navigation, obstacle avoidance, and formation control, where agents only see a local region and local interactions matter. It proposes the Extended Cutoff Policy Class: solve a 'cutoff' version of the problem with an enlarged computational visibility, then execute the first step of that solution using a belief about agents beyond the physical visibility. The main theorem states that every policy in this class has value within $\beta \gamma^{\lfloor (V_{\mathrm{exec}}-R)/2\rfloor -1}/(1-\gamma)$ of the fully observable joint optimum, so larger visibility buys exponential closeness to optimality; the same guarantee is proved when transitions and rewards also depend locally on other agents. The class contains and unifies three earlier closed-form policies and includes a memory-based extraction method that fixes a 'Penalty Jittering' failure mode and, in deterministic settings with all agents initially in view, converges to joint optimality. A sympathetic reader would care because this turns a NEXP-complete general problem into a tractable family of closed-form policies with provable performance and a tunable trade-off between computation and belief radius.

What carries the argument

The load-bearing construction is the Extended Cutoff Policy Class. To build a policy, one chooses an extended computational visibility $V_{\mathrm{comp}}=V_{\mathrm{exec}}+\xi$ and horizon $c+\eta$ with $c=\lfloor (V_{\mathrm{exec}}-R)/2\rfloor$; the corresponding Cutoff Multi-Agent MDP, where disconnected groups never reconnect, is solved to that horizon, and the first-step policy is executed through an extraction method $\rho$ that converts the current observation history into a belief state over out-of-view agents. Three extraction methods are described: Trivial, Aggregate, and Simple Memory Based. The proof that the policy class is near optimal rests on the Dependence Time Lemma, which uses the speed limit (agents move at most one unit per step) and $V>R$ to guarantee that agents in different visibility groups cannot affect each other's rewards or transitions for $c$ steps; this buffer lets local value comparisons be made with only exponentially small error. The paper also isolates four Consistent Performance Policy inequalities that allow the value of the computation phase to transfer to the execution phase, and asserts that the optimal finite-horizon cutoff policy satisfies them.

What would settle it

Enumerate the optimal finite-horizon cutoff policy for a small two-agent grid (for example $V=R+2$, $\gamma=0.9$) and directly compute the four value differences in Appendix D: Constructive, Deconstructive, 1-Step Displaced, and 1-Step Contracted Improvement. A single violation of any of them would falsify the claimed exponential bound, because the proof of Theorem 1 is exactly the chain that converts those inequalities into the bound.

Watch

Extended reading notes

Core claim

The central claim is Theorem 1: for any (Generalized) Locally Interdependent Multi-Agent MDP with execution visibility $V_{\mathrm{exec}}$, any policy in the Extended Cutoff Policy Class satisfies $V^*(s)-V^\pi(s) \le \beta \gamma^{\lfloor (V_{\mathrm{exec}}-R)/2\rfloor-1}/(1-\gamma)$, where $\beta$ is a constant depending on $\gamma$, the horizon parameter $\eta$, and the gap between computation and execution visibility (and on $n$, the number of agents, in the generalized setting). The proof routes through Theorem 2: for any Consistent Performance Policy, the value of the extended cutoff computation and the value of the extracted execution policy differ by the same exponential-in-visibility term. The paper shows that the optimal finite-horizon extended cutoff policy is such a policy, and that the bound matches the known lower bound up to constants. The proposed framework therefore gives, as the authors state, the first non-trivial class of near-optimal closed-form partially observable policies for every Locally Interdependent Multi-Agent MDP, while subsuming the Amalgam, Cutoff, and First-Step policies as special cases.

Load-bearing premise

The proof depends on the four Consistent Performance Policy inequalities in Appendix D, which the paper asserts are trivially satisfied by the optimal finite-horizon cutoff policy without providing a proof; if any one of those inequalities fails, Lemma 6 no longer holds and the exponential closeness guarantee has no support.

Editorial extensions

If this is right

Every policy in the Extended Cutoff Policy Class is exponentially close to optimal: the gap $V^*(s)-V^\pi(s)$ decays like $\gamma^{\lfloor (V_{\mathrm{exec}}-R)/2\rfloor}$, matching the existing lower bound up to constants.
The class unifies the Amalgam, Cutoff, and First-Step Finite Horizon Optimal policies as special cases of Trivial Extraction with different $\xi$ and $\eta$.
In the generalized setting with local transition dependence and extended reward dependence, the same exponential guarantee holds, with the constant depending on the number of agents.
Simple Memory Based Extraction resolves Penalty Jittering in the fixed-visibility regime; when the environment is deterministic and all agents start in view, increasing $\xi$ and $\eta$ makes the partially observable policy attain the fully observable joint optimum.
Once the extended cutoff solution is computed, any valid extraction method yields a new near-optimal policy without additional computation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial extension: The trade-off suggests a practical tuning rule for deployment—choose $\xi$ and $\eta$ from a computational budget rather than the physical sensing range, since the theorem quantifies exactly how much suboptimality each unit of belief radius removes.
Editorial extension: Because only the Dependence Time buffer and the four consistency inequalities are used, the same exponential-transfer argument should carry over to other local-interaction models, such as factor graphs or communication-limited planners, provided a similar $V>R$ buffer exists.
Editorial extension: The memory-based extraction algorithm's failure cases in stochastic and out-of-view-agent simulations point to a testable improvement—keeping confidence-weighted or particle memories and dropping low-confidence estimates—and the paper notes that such variants remain valid extraction methods.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

I read the paper and I largely agree with the stress-test note. The central theorem currently rests on an unproved assertion, and that is the thing to fix before this can be taken as established.

What is genuinely new: the Extended Cutoff Policy Class is a real generalization. It subsumes the three prior policies, adds the memory-based extraction mechanism, and gives a clean story of why remembering out-of-view agents can help in small-visibility settings. The generalized MDP with transition and reward dependence is also a meaningful extension, and the proof structure there is a faithful analogue of the standard case. The simulations, while informal, are honest: they include adversarially chosen cases where Simple Memory Based Extraction underperforms, and the 100-agent navigation result without collisions is a useful existence check.

Where the soft spot is: Appendix D defines four Consistent Performance Policy inequalities and says they are “trivially satisfied” by the optimal finite-horizon cutoff policy. I do not think that is trivial, and I think the stress-test note is right that the horizon-comparison inequalities are the suspicious ones. The 1-Step Displaced and 1-Step Contracted conditions compare value functions at different starting times and different horizons; optimality of a finite-horizon policy does not by itself say that its value from time 0 dominates a shifted tail, because the immediate reward at time 0 and the extra terminal reward at time c+eta+1 are not comparable without an argument. Lemma 6 and Lemma 8 use these conditions directly, and Theorem 2—hence Theorem 1—collapses if they fail. I am not claiming the inequalities are false; I am claiming they are nontrivial and unproved.

The rest of the proof, as far as I can see, is structurally coherent. The Dependence Time Lemma is sound, the decomposition in Theorem 2 is a reasonable strategy, and the generalized constants are consistent with the looser assumptions. The lack of error bars on single-rollout simulations is a minor issue compared to the Appendix D gap, but it means the empirical claims should be read as illustrative.

Who this is for: someone working on theoretically grounded decentralized multi-agent control would want to engage with this, but only after the gap is closed. The paper deserves a serious referee; the right outcome of peer review would be a demand for a proof of the Consistent Performance Policy conditions, or a revision that states them as assumptions rather than consequences. As posted, the abstract overstates what is actually supported.

Referee Report

3 major / 4 minor

Summary. The paper proposes an Extended Cutoff Policy Class for Locally Interdependent Multi-Agent MDPs, consisting of policies obtained by solving a Cutoff Multi-Agent MDP with an enlarged computational visibility and then extracting a group-decentralized policy through a belief-like extraction method. The central claim is Theorem 1: every policy in this class is within β γ^{c-1}/(1-γ) of the fully observable joint optimum, where c = floor((V_exec - R)/2), so performance improves exponentially as the execution visibility grows. The proof route is Theorem 2, a more general near-optimality bound for any 'Consistent Performance Policy', applied to the optimal finite-horizon cutoff policy π^{ξ,η}_comp. The paper also proposes a Generalized Locally Interdependent Multi-Agent MDP, claims the same bounds there, and presents simulations intended to show that Simple Memory Based Extraction improves behavior at small fixed visibility and can achieve full observability in deterministic settings.

Significance. If Theorem 1 is correct, the paper would establish a broad and useful class of closed-form partially observable policies that unify the earlier Amalgam, Cutoff, and First-Step policies, and it would extend the theory to a generalized model with transition and reward dependence. The explicit bound, the identification of the Extended Cutoff Policy Class, and the large-scale 100-agent simulation without deep learning are genuine strengths, as is the transparent effort to include negative empirical examples against the proposed method. However, the main theoretical result is currently conditional on an unproved set of inequalities, so the significance cannot be assessed until that gap is closed.

major comments (3)

[Appendix D and Lemma 6] The four 'Consistent Performance Policy' inequalities are asserted, not proved. The text in Appendix D states only that 'these conditions are trivially satisfied by the optimal finite horizon policies π^{ξ,η}_comp', with no derivation. These inequalities are load-bearing: Lemma 6 uses the Constructive/Deconstructive inequalities in Equation (10), Corollary 7 and Lemma 8 use the 1-Step Displaced and 1-Step Contracted inequalities, and Theorem 2's Step 2 bounds the key Δ_t terms through Lemma 6. Since Theorem 1 applies Theorem 2 to the optimal policy π^{ξ,η}_comp, the exponential bound in Theorem 1 collapses if any of the four inequalities fails. The inequalities are not immediate consequences of finite-horizon optimality: for example, 1-Step Displaced Improvement compares the tail of an extended policy π′ (with an appended stationary policy π_{c+η+1}) against the full value of π starting one step earlier, and optimality of π for horizon c+η does not by itself constrain the value of the appended tail. Similarly, Deconstructive Improvement compares a policy that coordinates under a coarser partition than the environment actually uses, and it needs an argument rather than a 'trivial' assertion. A complete proof of the CPP conditions for π^{ξ,η}_comp, or a revised theorem that does not depend on them, is required before Theorem 1 is established.
[Section 3.2.2, Proposition 3] Proposition 3 claims that Simple Memory Based Extraction converges to the fully observable joint optimal solution as ξ, η → ∞ in any Locally Interdependent Multi-Agent MDP where all agents start within view. The paragraph preceding the proposition gives intuition about Algorithm 1, but no proof is supplied. A convergence claim of this form requires controlling both the error from the finite horizon c+η and the error from the memory-based belief extraction as ξ and η grow; neither is analyzed. Since this proposition underlies contribution (iii) and the Aisle Walk / Long Journey claims, it needs either a rigorous proof or a clear downgrade to an empirical observation.
[Appendix A (simulations)] The empirical claims about resolving Penalty Jittering and improving small-visibility performance are based on single rollouts without error bars, confidence intervals, or multiple random seeds, even in the stochastic example of Appendix A.9. The appendix does include adversarial examples where Simple Memory Based Extraction underperforms Trivial Extraction, which is honest, but the general claim that the class 'resolves' Penalty Jittering is stronger than what single trajectories can support. I view this as a presentation/evidence issue rather than a fatal flaw, but the wording should be softened or the experiments should be repeated.

minor comments (4)

[Algorithm 1, Appendix B] The notation 'N one' appears in Algorithm 1 and its explanation; this appears to be a rendering of 'None' or 'null'. Also, the line 's_belief_next = argmax_{s'} P(s'|[s_belief]_z_belief, a_belief)' chooses a single most likely next state, which is not the same as the expected transition; this choice should be stated explicitly as a heuristic, since it affects the memory update.
[Section 3.2.1, Theorem 1 statement] The bound is stated with β defined in the theorem, but the proof of Theorem 1 in Appendix E.1.1 writes Equation (3) with an additional γ^{c′−c+1} term that comes from Theorem 10. This is internally consistent only if one notes c′ = floor((V_comp − R)/2); the presentation would benefit from explicitly writing the substitution, since a reader can otherwise mistake the constant factor.
[Appendix E.1.2, Equation (18)] In Theorem 10, the equality of the first c′ rewards between the original trajectory and the cutoff trajectory is stated without a detailed proof in the standard (transition-independent) case; the generalized proof supplies the needed equivalence argument. I suggest moving that argument or citing it in the standard case for completeness.
[Appendix D, notation] The definition of a trajectory π|P′_{s,P} uses P′ both as a partition and as the policy's assumed initial partition, while the actual partition is P; this is clear after careful reading, but the notation would be less confusing if the assumed partition were named differently, e.g., P_assume.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular reduction found: Theorem 1 is derived from the Consistent Performance Policy framework and Dependence-Time bounds, not from its own conclusion; the unproved Appendix D consistency assertion is a proof gap, and self-citations to DeWeese and Qu (2024) are not load-bearing.

full rationale

The central derivation is not circular. Theorem 1 reduces the desired bound to Theorem 2, the optimality of pi^{xi,eta}_comp, and Theorem 10; Theorem 2 is proved for arbitrary Consistent Performance Policies using Lemma 6, Corollary 7, Lemma 8, and Lemma 9, whose exponential-in-c bounds come from the Dependence Time Lemma and the distance argument V > R, not from the conclusion being assumed. The Extended Cutoff Policy Class and the extraction methods are defined constructively, and no parameter is fitted to V* or to the extracted value, so no prediction is a renamed input. The paper leans heavily on DeWeese and Qu [2024] for the underlying MDP model, the lower bound, and the proper-cutoff optimality theorem; these citations are to prior work with stated assumptions and are not the mechanism that produces the exponential guarantee, so they are at most a minor self-citation issue rather than load-bearing circularity. The important caveat, flagged per the reviewing rule, is an omitted proof: Appendix D asserts 'Notice that these conditions are trivially satisfied by the optimal finite horizon policies pi^{xi,eta}_comp' without deriving the four inequalities, and the 1-Step Displaced and 1-Step Contracted inequalities are not immediate consequences of optimality. This is a correctness and rigor gap in the chain from Appendix D to Theorem 1, not a circular equation-to-equation reduction, so it does not by itself raise the circularity score above 2. Overall: no significant circularity; the score reflects inherited self-citation and an unproved but non-circular linchpin.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The theory uses no fitted constants. The load-bearing assumptions are the LI-MDP structural axioms (V>R, speed limit, local rewards) and the unproved Consistent Performance Policy property. The simulation results depend on hand-chosen visibility/horizon parameters and heuristic rules.

free parameters (3)

Vcomp (computational visibility) = varied across simulations (e.g., 3, 4, 9, 11, 30)
Chosen by hand in each simulation; no principled tuning or fitting. The near-optimality bound holds for any Vcomp >= Vexec, but empirical performance depends on its value.
eta (extra computation horizon) = varied (0 to large)
Controls the horizon c+eta in the extended cutoff MDP; larger eta improves lookahead but is chosen by hand; the bound's beta term includes gamma^{eta+1}.
Heuristic parameters in Random Navigation simulation = sampling probability 0.8, 'large number of samples'
Hand-chosen heuristic for groups of size >=4; not part of the theory.

assumptions (4)

domain assumption V > R and unit speed limit: d(s_i, s_i') <= 1 per step
Used in the Dependence Time Lemma (Lemma 4) to guarantee c = floor((V-R)/2) steps of buffer between visibility disconnection and reward dependence. This is the structural assumption of the LI-MDP model.
ad hoc to paper Consistent Performance Policy conditions hold for the optimal finite-horizon cutoff policy
Asserted in Appendix D without proof; the proof of Lemma 6, Corollary 7, Lemma 9, and Theorem 2 depends on these four inequalities.
domain assumption Proposition 3 conditions: deterministic transitions and all agents initially within view
The convergence to fully observable joint optimal behavior is only established under these conditions (Section 3.2.2); the algorithm's memory updates track deterministic positions.
domain assumption Valid extraction condition: z in Zexec(sp) and [sp]_z = sz(t) for all belief states
Required in Definition 3 and Lemma 9 to ensure the belief state agrees with current observations; restricts the policy class but is stated as part of the construction.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Thinking Beyond Visibility: A Near-Optimal Policy Framework for Locally Interdependent Multi-Agent MDPs." pith.science (2026). https://pith.science/paper/Z4MXWFCP

@misc{pith2026250604215,
  author       = {Pith},
  title        = {Pith review of: Thinking Beyond Visibility: A Near-Optimal Policy Framework for Locally Interdependent Multi-Agent MDPs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z4MXWFCP}},
  note         = {Machine review of arXiv:2506.04215}
}

read the original abstract

Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs) are known to be NEXP-Complete and intractable to solve. However, for problems such as cooperative navigation, obstacle avoidance, and formation control, basic assumptions can be made about local visibility and local dependencies. The work DeWeese and Qu 2024 formalized these assumptions in the construction of the Locally Interdependent Multi-Agent MDP. In this setting, it establishes three closed-form policies that are tractable to compute in various situations and are exponentially close to optimal with respect to visibility. However, it is also shown that these solutions can have poor performance when the visibility is small and fixed, often getting stuck during simulations due to the so called "Penalty Jittering" phenomenon. In this work, we establish the Extended Cutoff Policy Class which is, to the best of our knowledge, the first non-trivial class of near optimal closed-form partially observable policies that are exponentially close to optimal with respect to the visibility for any Locally Interdependent Multi-Agent MDP. These policies are able to remember agents beyond their visibilities which allows them to perform significantly better in many small and fixed visibility settings, resolve Penalty Jittering occurrences, and under certain circumstances guarantee fully observable joint optimal behavior despite the partial observability. We also propose a generalized form of the Locally Interdependent Multi-Agent MDP that allows for transition dependence and extended reward dependence, then replicate our theoretical results in this setting.

Figures

Figures reproduced from arXiv: 2506.04215 by the authors.

**Figure 3.** (Aisle Walk Problem) In red and purple is the roll out for the fully observable joint [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗

**Figure 4.** (Bullseye Problem) In red is the fully observable joint optimal policy with discounted [PITH_FULL_IMAGE:figures/full_fig_p017_4.png] view at source ↗

Figures from the paper (14 more)

**Figure 5.** Figure 5: (Modified Bullseye Problem) Again, in red is the rollout for the fully observable joint [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]

**Figure 6.** Figure 6: (Highway Problem) In red is the rollout for the fully observable joint optimal policy with [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]

**Figure 7.** Figure 7: (Modified Highway Problem) In red is the rollout for the fully observable joint optimal [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]

**Figure 8.** Figure 8: (Penalty Jittering) In red is the rollout for the fully observable joint optimal policy with [PITH_FULL_IMAGE:figures/full_fig_p021_8.png]

**Figure 9.** Figure 9: (Long Journey) In red and purple is the trajectory for the fully observable joint optimal [PITH_FULL_IMAGE:figures/full_fig_p023_9.png]

**Figure 10.** Figure 10: A table of performance outcomes in the Long Journey simulation for Trivial Extraction [PITH_FULL_IMAGE:figures/full_fig_p025_10.png]

**Figure 11.** Figure 11: (Random Navigation With Many Agents) The environment and the initial placement [PITH_FULL_IMAGE:figures/full_fig_p026_11.png]

**Figure 12.** Figure 12: (Random Navigation With Many Agents) Snapshot at time step 50. Trajectories are [PITH_FULL_IMAGE:figures/full_fig_p028_12.png]

**Figure 13.** Figure 13: (Random Navigation With Many Agents) Snapshot at time step 150. Trajectories are [PITH_FULL_IMAGE:figures/full_fig_p029_13.png]

**Figure 14.** Figure 14: (Random Navigation With Many Agents) Snapshot at time step 500. Trajectories are [PITH_FULL_IMAGE:figures/full_fig_p030_14.png]

**Figure 15.** Figure 15: (Random Navigation With Many Agents) Snapshot at time step 1000. Trajectories are [PITH_FULL_IMAGE:figures/full_fig_p031_15.png]

**Figure 16.** Figure 16: (Stochastic Transitions) In red is the rollout for the fully observable joint optimal policy [PITH_FULL_IMAGE:figures/full_fig_p032_16.png]

**Figure 17.** Figure 17: (Unanticipated Reward Dependence) In red, blue, and green are rollouts for the trajec [PITH_FULL_IMAGE:figures/full_fig_p033_17.png]

**Figure 18.** Figure 18: (Out of View Coordination) In red is the trajectory for the fully observable joint optimal [PITH_FULL_IMAGE:figures/full_fig_p034_18.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 20 canonical work pages

[1]

Allen and S

M. Allen and S. Zilberstein. Complexity of decentralized control: Special cases . Advances in neural information processing systems, 22, 2009

work page 2009
[2]

S. Aradi. Survey of deep reinforcement learning for motion planning of autonomous vehicles . IEEE Transactions on Intelligent Transportation Systems, 23 0 (2): 0 740--759, 2020

work page 2020
[3]

Baldazo, J

D. Baldazo, J. Parras, and S. Zazo. Decentralized multi-agent deep reinforcement learning in swarms of drones for flood monitoring . In 2019 27th European Signal Processing Conference (EUSIPCO), pages 1--5. IEEE, 2019

work page 2019
[4]

Batra, Z

S. Batra, Z. Huang, A. Petrenko, T. Kumar, A. Molchanov, and G. S. Sukhatme. Decentralized control of quadrotor swarms with end-to-end deep reinforcement learning . In Conference on Robot Learning, pages 576--586. PMLR, 2022

work page 2022
[5]

D. S. Bernstein, R. Givan, N. Immerman, and S. Zilberstein. The complexity of decentralized control of Markov decision processes . Mathematics of operations research, 27 0 (4): 0 819--840, 2002

work page 2002
[6]

DeWeese and G

A. DeWeese and G. Qu. Locally Interdependent Multi-Agent MDP: Theoretical Framework for Decentralized Agents with Dynamic Dependencies . In Forty-first International Conference on Machine Learning, 2024

work page 2024
[7]

C. V. Goldman and S. Zilberstein. Decentralized control of cooperative systems: Categorization and complexity analysis . Journal of artificial intelligence research, 22: 0 143--174, 2004

work page 2004
[8]

R. Han, S. Chen, and Q. Hao. Cooperative multi-robot navigation in dynamic environment with deep reinforcement learning . In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 448--454. IEEE, 2020

work page 2020

Show all 20 references

[9]

P. Long, T. Fan, X. Liao, W. Liu, H. Zhang, and J. Pan. Towards optimally decentralized multi-robot collision avoidance via deep reinforcement learning . In 2018 IEEE international conference on robotics and automation (ICRA), pages 6252--6259. IEEE, 2018

2018
[10]

R. Lowe, Y. I. Wu, A. Tamar, J. Harb, O. Pieter Abbeel, and I. Mordatch. Multi-agent actor-critic for mixed cooperative-competitive environments . Advances in neural information processing systems, 30, 2017

2017
[11]

F. S. Melo and M. Veloso. Learning of coordination: Exploiting sparse interactions in multiagent systems . In Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems-Volume 2, pages 773--780. Citeseer, 2009

2009
[12]

R. Nair, P. Varakantham, M. Tambe, and M. Yokoo. Networked distributed POMDPs: A synthesis of distributed constraint optimization and POMDPs . In AAAI, volume 5, pages 133--139, 2005

2005
[13]

F. A. Oliehoek. Decentralized POMDPS . In Reinforcement Learning: State-of-the-Art, pages 471--503. Springer, 2012

2012
[14]

F. A. Oliehoek, C. Amato, et al. A Concise Introduction to Decentralized POMDPs , volume 1. Springer, 2016

2016
[15]

Palanisamy

P. Palanisamy. Multi-agent connected autonomous driving using deep reinforcement learning . In 2020 International Joint Conference on Neural Networks (IJCNN), pages 1--7. IEEE, 2020

2020
[16]

G. Qu, Y. Lin, A. Wierman, and N. Li. Scalable Multi-Agent Reinforcement Learning for Networked Systems with Average Reward . Advances in Neural Information Processing Systems, 33, 2020 a

2020
[17]

G. Qu, A. Wierman, and N. Li. Scalable reinforcement learning of localized policies for multi-agent networked systems . In Learning for Dynamics and Control, pages 256--266. PMLR, 2020 b

2020
[18]

G. Qu, A. Wierman, and N. Li. Scalable reinforcement learning for multiagent networked systems . Operations Research, 70 0 (6): 0 3601--3628, 2022

2022
[19]

M. T. Spaan and F. S. Melo. Interaction-driven Markov games for decentralized multiagent planning under uncertainty . In Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems-Volume 1, pages 525--532. Citeseer, 2008

2008
[20]

Zhang and V

C. Zhang and V. Lesser. Coordinated multi-agent reinforcement learning in networked distributed POMDPs . In Proceedings of the AAAI Conference on Artificial Intelligence, volume 25, pages 764--770, 2011

2011

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Allen and S

M. Allen and S. Zilberstein. Complexity of decentralized control: Special cases . Advances in neural information processing systems, 22, 2009

work page 2009

[2] [2]

S. Aradi. Survey of deep reinforcement learning for motion planning of autonomous vehicles . IEEE Transactions on Intelligent Transportation Systems, 23 0 (2): 0 740--759, 2020

work page 2020

[3] [3]

Baldazo, J

D. Baldazo, J. Parras, and S. Zazo. Decentralized multi-agent deep reinforcement learning in swarms of drones for flood monitoring . In 2019 27th European Signal Processing Conference (EUSIPCO), pages 1--5. IEEE, 2019

work page 2019

[4] [4]

Batra, Z

S. Batra, Z. Huang, A. Petrenko, T. Kumar, A. Molchanov, and G. S. Sukhatme. Decentralized control of quadrotor swarms with end-to-end deep reinforcement learning . In Conference on Robot Learning, pages 576--586. PMLR, 2022

work page 2022

[5] [5]

D. S. Bernstein, R. Givan, N. Immerman, and S. Zilberstein. The complexity of decentralized control of Markov decision processes . Mathematics of operations research, 27 0 (4): 0 819--840, 2002

work page 2002

[6] [6]

DeWeese and G

A. DeWeese and G. Qu. Locally Interdependent Multi-Agent MDP: Theoretical Framework for Decentralized Agents with Dynamic Dependencies . In Forty-first International Conference on Machine Learning, 2024

work page 2024

[7] [7]

C. V. Goldman and S. Zilberstein. Decentralized control of cooperative systems: Categorization and complexity analysis . Journal of artificial intelligence research, 22: 0 143--174, 2004

work page 2004

[8] [8]

R. Han, S. Chen, and Q. Hao. Cooperative multi-robot navigation in dynamic environment with deep reinforcement learning . In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 448--454. IEEE, 2020

work page 2020