Pith. sign in

REVIEW 6 cited by

A Closer Look at Invalid Action Masking in Policy Gradient Algorithms

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2006.14171 v3 pith:VZTQIUWA submitted 2020-06-25 cs.LG cs.AIstat.ML

Shengyi Huang , Santiago Ontañón This is my paper
classification cs.LGcs.AIstat.ML
keywords actioninvalidmaskingactionsalgorithmspolicygamesgradient
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

In recent years, Deep Reinforcement Learning (DRL) algorithms have achieved state-of-the-art performance in many challenging strategy games. Because these games have complicated rules, an action sampled from the full discrete action distribution predicted by the learned policy is likely to be invalid according to the game rules (e.g., walking into a wall). The usual approach to deal with this problem in policy gradient algorithms is to "mask out" invalid actions and just sample from the set of valid actions. The implications of this process, however, remain under-investigated. In this paper, we 1) show theoretical justification for such a practice, 2) empirically demonstrate its importance as the space of invalid actions grows, and 3) provide further insights by evaluating different action masking regimes, such as removing masking after an agent has been trained using masking. The source code can be found at https://github.com/vwxyzjn/invalid-action-masking

Discussion (0). Sign in to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Towards Scalable O-RAN Resource Management: Graph-Augmented Proximal Policy Optimization

    cs.NI 2025-09 conditional novelty 5.0 of 10

    Graph-Augmented Proximal Policy Optimization (GPPO) jointly optimizes functional split and baseband unit placement in O-RAN, achieving lower deployment cost and better reliability than generic RL baselines in simulation.

  2. Dynamic Collaborative Material Distribution System for Intelligent Robots In Smart Manufacturing

    cs.RO 2025-06 reject novelty 5.0 of 10

    A single shared PPO policy with action masking and a potential reward based on the robots' centroid is trained to solve dynamic multi-robot rendezvous navigation in factory grids.

  3. Optimal Reward Shaping: Autonomous Car Parking Case Study

    cs.LG 2026-07 conditional novelty 4.0 of 10

    Co-optimizing a parameterized parking reward with DQN hyperparameters via Bayesian search raises mean parking score from ~27 to 95.6 and removes paralysis and over-caution failure modes.

  4. Learning to Assemble the Soma Cube with Legal-Action Masked DQN and Safe ZYZ Regrasp on a Doosan M0609

    cs.RO 2025-08 reject novelty 4.0 of 10

    A Doosan M0609 cobot learns to assemble a Soma cube with a legal-action-masked DQN and ZYZ singularity guard, achieving 75% real-robot assembly success in 300 trials.

  5. Novel Multi-Agent Action Masked Deep Reinforcement Learning for General Industrial Assembly Lines Balancing Problems

    cs.AI 2025-07 conditional novelty 4.0 of 10

    A multi-agent, action-masked deep RL framework for general assembly line balancing that claims faster convergence to optimal schedules and polynomial action-space growth on small benchmark instances.

  6. Data-Driven Policy Mapping for Safe RL-based Energy Management Systems

    cs.LG 2025-06 conditional novelty 4.0 of 10

    A three-stage framework (load-profile clustering, LSTM forecasting, masked PPO) reduces simulated building energy costs by up to 15% and maps new buildings to pre-trained policies.

Pith tools