REVIEW 44 cited by
Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?
read the original abstract
Most recently developed approaches to cooperative multi-agent reinforcement learning in the \emph{centralized training with decentralized execution} setting involve estimating a centralized, joint value function. In this paper, we demonstrate that, despite its various theoretical shortcomings, Independent PPO (IPPO), a form of independent learning in which each agent simply estimates its local value function, can perform just as well as or better than state-of-the-art joint learning approaches on popular multi-agent benchmark suite SMAC with little hyperparameter tuning. We also compare IPPO to several variants; the results suggest that IPPO's strong performance may be due to its robustness to some forms of environment non-stationarity.
Forward citations
Cited by 44 Pith papers
-
Structural Equivalence and Learning Dynamics in Delayed MARL
Observation and action delays are formally equivalent in cooperative Dec-POMDPs, yielding identical optimal solutions and enabling zero-shot transfer, though learning dynamics differ due to credit assignment and opera...
-
Aggregate in the Advantage, Not the Ratio: A Canonical-Form Analysis of Cooperative Multi-Agent Policy Optimization
For cooperative PPO, the expected gradient at the on-policy point depends on advantage and ratio aggregation supports only through their matrix product, and the variance-optimal design keeps the ratio per-agent and ag...
-
HiComm: Hierarchical Communication for Multi-agent Reinforcement Learning
HiComm proposes a plug-in hierarchical communication protocol for cooperative MARL that performs structured information retrieval over observation hierarchies using receiver queries and three-stage decoding, matching ...
-
Benchmarking Open-Ended Multi-Agent Coordination in Language Agents
ALEM benchmark reveals LLM agents achieve only ~6% normalized return in open-ended multi-agent settings, with communication as the main driver of coordination and individual task competence not implying coordination c...
-
ARMS: Automatic Reward Shaping for Sparse-Reward Multi-Agent Reinforcement Learning
ARMS is an automatic reward-shaping framework for sparse-reward MARL that uses trajectory ranking and conditional best-response reasoning to preserve Nash equilibria while improving sampling efficiency in pathfinding tasks.
-
Randomness is sometimes necessary for coordination
Structured per-agent randomness via ranked masking in attention allows symmetric agents to break ties and coordinate, achieving perfect success on symmetric tasks where deterministic policies fail and enabling zero-sh...
-
Quality-Aware Exploration Budget Allocation for Cooperative Multi-Agent Reinforcement Learning
A quality-aware exploration method using return-conditioned sigmoid scheduling and per-agent RSQ metrics achieves top-tier returns on seven cooperative MARL benchmarks.
-
One Step is Enough: Multi-Agent Reinforcement Learning based on One-Step Policy Optimization for Order Dispatch on Ride-Sharing Platforms
OSPO trains optimal order dispatch policies for homogeneous AV fleets using only one-step group rewards, outperforming GRPO on a real ride-hailing dataset.
-
Dreamer-CPC: Message Learning with World Models for Decentralized Multi-agent Reinforcement Learning
Dreamer-CPC has each decentralized agent send messages drawn from its learned world-model memory, outperforming current-observation messaging baselines, especially when key observations are temporarily missing.
-
Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents
A tabular UCB controller trained on task success improves LLM-agent memory use over fixed heuristics, without extra LLM calls.
-
Action-Factored Multi-Agent Reinforcement Learning for Scalable Quantum Device Tuning
Online action-space factorization via Kalman-refined cross-capacitance lets shared multi-agent policies zero-shot tune larger quantum-dot arrays with near-constant steps.
-
Queue-Aware Graph Reinforcement Learning for UAV-ISAC-Assisted Maritime Data Collection
A queue-weighted graph-MARL framework with masked sequential b-matching for UAV-buoy associations improves cumulative collection utility by 106% over rate-driven baselines in maritime ISAC simulations.
-
HiComm: Hierarchical Communication for Multi-agent Reinforcement Learning
HiComm converts MARL communication into structured three-stage hierarchical retrieval over sender observations, matching or exceeding baselines with up to 23x lower communication volume per receiver per episode.
-
Contract-Based Compositional Shielding for Safe Multi-Agent Reinforcement Learning
Certified local LTL_safe contracts, jointly fixed-point checked and selected by a bandit, recover coordinated safe team policies under decentralised multi-agent RL execution.
-
Uncertainty-Aware Motion Planning for Autonomous Driving in Mixed Traffic Environment
UAMP adds a proximity-aware uncertainty estimator and Uncertainty-Calibrated Value Learning to reinforcement learning motion planning so autonomous vehicles can handle uncertain human intents in mixed traffic.
-
One Policy, Infinite NPCs: Persona-Traceable Shared RL Policies for Scalable Game Agents
pcsp is a shared RL policy using LLM persona embeddings, low-rank projection, and PPO+InfoNCE+KL training that delivers 17x above-chance zero-shot persona identification and 22x faster inference on a 300-persona benchmark.
-
GAE Falls Short in Imperfect-Information Self-Play Reinforcement Learning
GAE suffers from amplified variance in imperfect-info self-play RL; VRPO with Q-boosting and multi-step Expected SARSA(λ) reduces it and improves performance on mid-to-large games.
-
Robust Instruction Compliance in Cooperative Multi-Agent Reinforcement Learning
MAVIC corrects Bellman backups at instruction boundaries by adjusting the incoming objective and restoring continuation value, enabling consistent estimation under stochastic instruction switching in a unified policy.
-
Robust Instruction Compliance in Cooperative Multi-Agent Reinforcement Learning
MAVIC corrects Bellman backups at instruction boundaries by adjusting the incoming objective and restoring continuation value, enabling consistent estimation under stochastic instruction switching in cooperative MARL.
-
Shaping Zero-Shot Coordination via State Blocking
SBC generates virtual environments via state blocking to expose agents to diverse suboptimal partner policies, yielding superior zero-shot coordination performance including with humans.
-
Priority-Driven Control and Communication in Decentralized Multi-Agent Systems via Reinforcement Learning
A model-free priority-driven RL method learns joint control and communication in decentralized multi-agent systems and outperforms baselines on benchmarks.
-
Rethinking Ratio-Based Trust Regions for Policy Optimization in Multi-Agent Reinforcement Learning
MARS replaces additive clipping and soft penalties in multi-agent trust-region methods with a symmetric geometric barrier, matching or exceeding MAPPO and MASPO performance across 47 tasks in eight environments.
-
SACHI: Structured Agent Coordination via Holistic Information Integration in Multi-Agent Reinforcement Learning
SACHI enriches agent representations via graph transformer convolutions over inter-agent graphs to enable holistic information integration, outperforming baselines across five cooperative tasks with statistical significance.
-
SACHI: Structured Agent Coordination via Holistic Information Integration in Multi-Agent Reinforcement Learning
SACHI uses graph transformer convolutions on inter-agent coordination graphs to enrich partial-observation agents with content-dependent teammate information, yielding statistically significant gains over baselines in...
-
Hierarchical Multiagent Reinforcement Learning for Multi-Group Tax Game
A bi-level MARL framework with curriculum learning and closed-loop sequential updates learns stable tax policies in multi-group hierarchical games, extending effective game duration by 60.92% and cutting GDP dispariti...
-
Hierarchical Multiagent Reinforcement Learning for Multi-Group Tax Game
A bilevel MARL framework with curriculum learning and closed-loop sequential updates learns stable tax policies in multi-group taxation simulations, extending effective game duration by 60.92% and reducing GDP dispari...
-
Dynamics of Cognitive Heterogeneity: Investigating Behavioral Biases in Multi-Stage Supply Chains with LLM-Based Simulation
Heterogeneous LLM agents in supply chain simulations exhibit myopic self-interested behaviors that worsen inefficiencies, but information sharing mitigates these effects.
-
TABX: A High-Throughput Sandbox Battle Simulator for Multi-Agent Reinforcement Learning
Presents TABX, a modular JAX-accelerated sandbox simulator enabling customizable multi-agent tasks and high-throughput evaluation for cooperative MARL.
-
TABX: A High-Throughput Sandbox Battle Simulator for Multi-Agent Reinforcement Learning
TABX is a JAX-based, GPU-accelerated, configurable multi-agent battle simulator that lets researchers vary units, terrain, and physics to benchmark cooperative MARL algorithms.
-
Learning Decentralized LLM Collaboration with Multi-Agent Actor Critic
Multi-agent actor-critic methods with a centralized critic improve decentralized LLM collaboration over Monte Carlo baselines in long-horizon and sparse-reward settings.
-
Automata-Conditioned Cooperative Multi-Agent Reinforcement Learning
ACC-MARL trains decentralized multi-agent policies that solve many automaton-specified cooperative tasks at once, with a proof of optimality for the Markovian reformulation and value-based task assignment.
-
Multi-Agent Reinforcement Learning for Safe Autonomous Driving Under Pedestrian Behavioral Uncertainty
Co-training an SDC and 12 pedestrians with MAPPO in a MARL setup yields 78% goal success and 14% collisions versus 35% goals and 33% for the best rule-based baseline, with jaywalking linked to 62% of collisions despit...
-
Priority-Driven Control and Communication in Decentralized Multi-Agent Systems via Reinforcement Learning
A priority-driven RL algorithm learns joint communication priorities and control policies for decentralized multi-agent systems in a model-free way and outperforms baselines on benchmark tasks.
-
A Survey of Multi-Agent Deep Reinforcement Learning with Graph Neural Network-Based Communication
A survey of MARL with GNN-based communication that proposes a generalized process to organize and clarify existing methods.
-
Diffusing to Coordinate: Efficient Online Multi-Agent Diffusion Policies
A diffusion-policy multi-agent RL framework substitutes an ELBO for intractable joint entropy and reports 2.5–5× sample-efficiency gains on 10 MPE/MAMuJoCo tasks.
-
Learning Decentralized LLM Collaboration with Multi-Agent Actor Critic
Centralized-critic actor-critic training (CoLLM-CC) improves sample efficiency and stability over Monte-Carlo multi-agent RL for training decentralized LLM collaboration.
-
CoRL-MPPI: Enhancing MPPI With Learnable Behaviours For Efficient And Provably-Safe Multi-Robot Collision Avoidance
CoRL-MPPI injects a learned cooperative policy into MPPI's sampling distribution to speed up and make safer multi-robot navigation in dense simulations.
-
Centralized Adaptive Sampling for Reliable Co-Training of Independent Multi-Agent Policies
CoSER adaptively samples joint actions in CTDE MARL to reduce sampling error relative to the joint on-policy distribution, empirically improving reliability of independent policy gradient convergence.
-
Low Variance Trust Region Optimization with Independent Actors and Sequential Updates in Cooperative Multi-agent Reinforcement Learning
Proposes a clipping objective for sequential trust-region updates in independent-actor cooperative MARL that yields a monotonic improvement bound and sub-linear convergence to epsilon-Nash equilibria while reducing ad...
-
$\alpha$-fair heterogeneous agent reinforcement learning
Introduces α-fair HATRPO and HAPPO algorithms that integrate α-fairness into HATRL via a weighted advantage function while claiming to preserve convergence to Nash equilibria.
-
Multi-Agent Reinforcement Learning for Safe Autonomous Driving Under Pedestrian Behavioral Uncertainty
Co-training an SDC and pedestrians with MAPPO yields 78% goal success and 14% collisions versus 35%/33% for rule-based baselines, with jaywalking causing 62% of collisions and evidence of poor anticipation via speed d...
-
Reinforcement Learning for LLM-based Multi-Agent Systems through Orchestration Traces
This survey organizes RL for LLM multi-agent systems into reward families, credit units, and five orchestration sub-decisions, notes the absence of explicit stopping-decision training in its paper pool, and releases a...
-
Dynamics of Cognitive Heterogeneity: Investigating Behavioral Biases in Multi-Stage Supply Chains with LLM-Based Simulation
LLM-based agents simulating supply chain tiers exhibit known behavioral biases, and information sharing mitigates resulting inefficiencies.
-
GLo-MAPPO: Multi-Agent Deep Reinforcement Learning for Energy-Efficient UAV-Assisted LoRa Networks
GLo-MAPPO applies centralized-training decentralized-execution MAPPO with a gain-based association scheme to jointly optimize LoRa parameters and UAV paths, yielding higher weighted energy efficiency than prior MARL b...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.