A new advantage-weighted Bellman operator is claimed to speed up Q-learning convergence, but the proofs are flawed and experiments lack error bars.
The Value Function Polytope in Reinforcement Learning
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
We establish geometric and topological properties of the space of value functions in finite state-action Markov decision processes. Our main contribution is the characterization of the nature of its shape: a general polytope (Aigner et al., 2010). To demonstrate this result, we exhibit several properties of the structural relationship between policies and value functions including the line theorem, which shows that the value functions of policies constrained on all but one state describe a line segment. Finally, we use this novel perspective to introduce visualizations to enhance the understanding of the dynamics of reinforcement learning algorithms.
fields
cs.LG 1years
2025 1verdicts
REJECT 1representative citing papers
citing papers explorer
-
Bellman operator convergence enhancements in reinforcement learning algorithms
A new advantage-weighted Bellman operator is claimed to speed up Q-learning convergence, but the proofs are flawed and experiments lack error bars.