REVIEW 2 cited by
Reinforcement Learning with Dynamic Boltzmann Softmax Updates
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Value function estimation is an important task in reinforcement learning, i.e., prediction. The Boltzmann softmax operator is a natural value estimator and can provide several benefits. However, it does not satisfy the non-expansion property, and its direct use may fail to converge even in value iteration. In this paper, we propose to update the value function with dynamic Boltzmann softmax (DBS) operator, which has good convergence property in the setting of planning and learning. Experimental results on GridWorld show that the DBS operator enables better estimation of the value function, which rectifies the convergence issue of the softmax operator. Finally, we propose the DBS-DQN algorithm by applying dynamic Boltzmann softmax updates in deep Q-network, which outperforms DQN substantially in 40 out of 49 Atari games.
Forward citations
Cited by 2 Pith papers
-
The Courage to Stop: Overcoming Sunk Cost Fallacy in Deep Reinforcement Learning
Introduces LEAST, an adaptive early-episode-stopping rule for off-policy deep RL that improves learning efficiency on MuJoCo and DeepMind Control benchmarks.
-
Causality-informed Anomaly Detection in Partially Observable Sensor Networks: Moving beyond Correlations
A deep Q-network that mixes causal statistics and a causality-weighted entropy term is proposed for placing sensors in partially observable anomaly detection.
Discussion (0). Sign in to comment.