REVIEW 4 cited by
Discovering Policies with DOMiNO: Diversity Optimization Maintaining Near Optimality
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Finding different solutions to the same problem is a key aspect of intelligence associated with creativity and adaptation to novel situations. In reinforcement learning, a set of diverse policies can be useful for exploration, transfer, hierarchy, and robustness. We propose DOMiNO, a method for Diversity Optimization Maintaining Near Optimality. We formalize the problem as a Constrained Markov Decision Process where the objective is to find diverse policies, measured by the distance between the state occupancies of the policies in the set, while remaining near-optimal with respect to the extrinsic reward. We demonstrate that the method can discover diverse and meaningful behaviors in various domains, such as different locomotion patterns in the DeepMind Control Suite. We perform extensive analysis of our approach, compare it with other multi-objective baselines, demonstrate that we can control both the quality and the diversity of the set via interpretable hyperparameters, and show that the discovered set is robust to perturbations.
Forward citations
Cited by 4 Pith papers
-
Solving Zero-Sum Convex Markov Games
Independent policy-gradient algorithms provably compute approximate Nash equilibria in two-player zero-sum convex Markov games.
-
From Tabula Rasa to Emergent Abilities: Discovering Robot Skills via Real-World Unsupervised Quality-Diversity
URSA extends quality-diversity actor-critic with learned skill spaces, safety constraints, and world-model training, enabling real-world unsupervised skill discovery on a quadruped.
-
The impact of intrinsic rewards on exploration in Reinforcement Learning
An empirical MiniGrid study shows state-counting is best for low-dimensional observations, maximum entropy is more robust with images, and DIAYN skill learning does not aid exploration.
-
Enhancing Diversity in Parallel Agents: A Maximum State Entropy Exploration Story
A centralized policy gradient for parallel state entropy maximization improves state coverage on small gridworlds, but the paper's concentration-rate proof is invalid.
Discussion (0). Continue with ORCID to comment.