S-MEME iteratively fine-tunes a diffusion model using its own score as the exploration reward, provably converging to the maximum-entropy distribution on the learned manifold.
Geometric Entropic Exploration
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Exploration is essential for solving complex Reinforcement Learning (RL) tasks. Maximum State-Visitation Entropy (MSVE) formulates the exploration problem as a well-defined policy optimization problem whose solution aims at visiting all states as uniformly as possible. This is in contrast to standard uncertainty-based approaches where exploration is transient and eventually vanishes. However, existing approaches to MSVE are theoretically justified only for discrete state-spaces as they are oblivious to the geometry of continuous domains. We address this challenge by introducing Geometric Entropy Maximisation (GEM), a new algorithm that maximises the geometry-aware Shannon entropy of state-visits in both discrete and continuous domains. Our key theoretical contribution is casting geometry-aware MSVE exploration as a tractable problem of optimising a simple and novel noise-contrastive objective function. In our experiments, we show the efficiency of GEM in solving several RL problems with sparse rewards, compared against other deep RL exploration approaches.
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Provable Maximum Entropy Manifold Exploration via Diffusion Models
S-MEME iteratively fine-tunes a diffusion model using its own score as the exploration reward, provably converging to the maximum-entropy distribution on the learned manifold.