Pith. sign in

REVIEW 3 major objections 6 minor 45 references

EVaDE : Event-Based Variational Thompson Sampling for Model-Based Reinforcement Learning

T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that adding three event-designed variational layers to SimPLe's reward model improves Atari 100K exploration, lifting mean human-normalized score by 30 percent over vanilla SimPLe and 79 percent over CURL.

desk verdict Solid empirical paper: three new dropout-based exploration layers give a controlled 30% gain over a same-schedule SimPLe(30) baseline on Atari 100K, though the PSRL framing is loose. read the letter →

arxiv 2501.09611 v1 pith:X762V5T5 submitted 2025-01-16 cs.LG

Siddharth Aravindan , Dixant Mittal , Wee Sun Lee This is my paper · ORCID
classification cs.LG
keywords event-basedvariationaldistributionsThompsonsamplingmodel-basedreinforcementlearningGaussiandropoutexplorationSimPLeAtari100Kobject-baseddomains
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces Event-based Variational Distributions for Exploration (EVaDE), a set of three Gaussian-dropout convolutional layers inserted into the reward network of the model-based agent SimPLe. These layers make the reward function a sample from an approximate posterior, implementing a practical form of Thompson sampling for exploration. On the 100K Atari benchmark, the resulting agent, EVaDE-SimPLe, achieves a mean human-normalized score of 0.682, 79% higher than CURL's 0.381 and 30% higher than vanilla SimPLe(30)'s 0.525. The paper argues that perturbing object interactions, event importances, and object translations in object-based domains generates exploratory trajectories that help the agent escape local optima.

What carries the argument

The three EVaDE layers are convolutional layers whose weights are perturbed multiplicatively by Gaussian noise, following $\tilde{\theta}^i_{env} \leftarrow \theta^i_{env}(1 + \sigma^i_{env}\epsilon^i)$ with $\epsilon^i \sim \mathcal{N}(0,1)$, with variances trained jointly with the model. The noisy event interaction layer combines local patches across channels to perturb object interactions; the noisy event weighting layer scales each channel independently to up- or down-weight events; the noisy event translation layer computes noisy weighted sums over neighboring pixels to translate detected objects. Inserted between the deconvolutional layers of SimPLe's reward network (with weights shared with the transition network), they turn the reward function into a sample from an approximate posterior. A theorem shows each layer can represent the identity, so the augmented network retains the original's representable functions.

What would settle it

Replace the three EVaDE layers with generic Gaussian multiplicative dropout applied to ordinary fully populated 1x1 and 3x3 convolutions at the same positions in the network, keeping the parameter count matched, and compare mean human-normalized scores on the 26-game Atari 100K suite; if the control matches or exceeds EVaDE-SimPLe's 0.682 HNS, the event-based filter structure is not the causal factor.

Watch

Extended reading notes

Core claim

The central claim is that in object-based domains, exploration in model-based reinforcement learning can be improved by designing the variational distribution to induce event-level perturbations: altering interactions between objects, up- or down-weighting the reward associated with detected events, and translating detected objects or events. The paper operationalizes this with three noisy convolutional layers—noisy event interaction, noisy event weighting, and noisy event translation—that apply Gaussian multiplicative dropout to their parameters and are inserted into the reward network of SimPLe. At each iteration, the policy is optimized against a reward function sampled from this event-aware variational family. EVaDE-SimPLe reports a mean HNS of 0.682 on Atari 100K, outperforming CURL (0.381) and SimPLe(30) (0.525), with a paired t-test p-value of 3e-3 and an IQM of 0.339 versus 0.202. The paper also proves that adding these layers does not reduce the representational capacity of the original network, since each EVaDE layer can represent the identity transformation.

Load-bearing premise

The design rests on the premise that intermediate convolutional channels in this Atari reward model actually capture object positions or events, so that channel-wise or pixel-wise Gaussian perturbation corresponds to perturbing object interactions, event importance, and translation; if that premise is false, the measured gains could come from added capacity or generic dropout regularization rather than event-directed exploration.

Editorial extensions

If this is right

  • Any model-based RL agent with a convolutional reward model can insert the EVaDE layers to obtain an exploration mechanism without changing the training algorithm.
  • The three perturbation types provide distinct exploration behaviors, and their combination is more effective than any single layer in the reported ablation.
  • Adding the layers does not remove the network's ability to represent the true reward function, so observed gains are not explained by a loss of representational capacity.
  • The reported improvement over SimPLe(30) is statistically significant under a per-game paired t-test and holds under the outlier-robust inter-quartile mean metric.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct control experiment using generic Gaussian multiplicative dropout layers with the same parameter count and placement would test whether the structured event-based perturbations, rather than added capacity or simple regularization, cause the measured improvement.
  • The paper's premise that intermediate convolutional channels capture object positions or events could be verified on simulated manipulation or navigation domains with known object states; such verification would also inform where the layers should be inserted.
  • The design principle—choosing a variational distribution for the purpose of exploration rather than posterior fidelity—may extend to other model-based planners, including search-based agents that can sample reward models during simulated rollouts.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes Event-based Variational Distributions for Exploration (EVaDE), a set of three noisy convolutional layers (event interaction, event weighting, event translation) that can be inserted into the reward model of a model-based reinforcement learning agent. The authors instantiate EVaDE inside SimPLe, forming EVaDE-SimPLe, and evaluate it on the 26-game Atari 100K benchmark. They report a mean human-normalized score (HNS) of 0.682 for EVaDE-SimPLe, compared to 0.525 for a 30-iteration SimPLe baseline (SimPLe(30)) run in the same codebase, and also compare against CURL and other published baselines. An ablation on 12 games shows that each layer individually improves over SimPLe(30), and all three together perform best. The paper also presents a representability theorem, visualizations of the learned layer behavior, and a statistical analysis (IQM and paired t-test) supporting the internal comparison.

Significance. If the internal results hold, the paper offers a practical and inexpensive plug-in module for improving the data efficiency of model-based RL in object-based domains, with a reproducible and statistically careful internal evaluation (five runs, IQM, paired t-test). The three-layer design is interpretable, and the visualizations give some evidence that the layers perform the intended operations. The main limitations are conceptual: the claimed connection to posterior sampling for RL (PSRL) is not derived, and the ablation does not isolate the event-based structure from generic dropout or added noise. The paper's secondary external comparisons rest on literature point estimates without variance, so the headline gains over CURL should be treated as indicative rather than established.

major comments (3)
  1. [Section 3.5 and Algorithm 1 (Appendix A)] The paper repeatedly claims that EVaDE-SimPLe 'approximates PSRL' by maintaining an approximate posterior over reward functions, but no derivation or formal argument connects the procedure in Algorithm 1 to posterior sampling. In PSRL, a model is sampled from the posterior at the start of an episode, whereas Algorithm 1 samples one perturbed reward model per SimPLe iteration and then trains a policy against that single sample for 28.8M simulated interactions. This is closer to randomized reward functions or parameter-space noise than to posterior sampling. Because the title and motivation rest on the PSRL framing, the authors should either provide a formal connection (e.g., showing the sampled reward model corresponds to a sample from a well-defined posterior over reward parameters) or substantially weaken the claim to 'inspired by PSRL.'
  2. [Section 4.4 (Ablation Studies)] The ablation shows that each EVaDE layer improves over SimPLe(30) and that their combination is best, but it does not include a control with generic multiplicative Gaussian dropout applied to the same reward network, nor a control with an unstructured additional layer of comparable parameter count. The authors' parameter-count argument (translation layer adds 4K parameters to a ~10M-parameter model) rules out a pure capacity effect, but it does not rule out the possibility that any form of multiplicative noise on the reward model helps exploration. Without such a control, the paper's central explanatory claim that the specifically structured event-based inductive bias is responsible for the gains is not empirically isolated.
  3. [Section 4.3, Table 1, and abstract] The abstract's headline '79% higher than CURL' is based on a single point estimate from the CURL paper, with no error bars or statistical test comparing the authors' five runs to that baseline. The internal EVaDE-SimPLe vs SimPLe(30) comparison is well supported by the reported five runs, paired t-test (p = 3e-3), and IQM, but the external comparisons to CURL, OTRainbow, and Eff. Rainbow should be presented as literature point estimates and should not be part of the paper's central claims without a proper uncertainty quantification.
minor comments (6)
  1. [Abstract] The abstract says the 0.525 mean HNS is achieved by 'vanilla SimPLe agents'; since Section 4.2 later defines SimPLe(30) separately from the original 15-iteration SimPLe from [45] (which has mean HNS 0.443), the abstract should say 'SimPLe(30)' to avoid ambiguity.
  2. [Section 4.3] There is a typo in the sentence 'This affirms that the improvements obtained due to the addition of the EVaDE layers are robust...': the text reads 'Simple(30)' instead of 'SimPLe(30)'.
  3. [Appendix E.1] Reference [39] is cited as the source of the SimPLe implementation used in the experiments, but [39] is 'Tensor2Tensor for Neural Machine Translation' by Vaswani et al. This citation appears to be incorrect; the authors should cite the actual repository or the original SimPLe paper's code.
  4. [Appendix B.3, Claim 2] In the proof of Claim 2, the sentence 'As stated in Equation 3, the event interaction layer computes every output yk_i,j' should read 'the event weighting layer' instead of 'event interaction layer'.
  5. [Figure 4 caption] The caption says 'Learning curves of EVaDE-SimPLe agents, SimPLe(30) agents and agents which only add one of the EVaDE layers' but the figure itself shows curves for SimPLe(30), the three single-layer variants, and EVaDE-SimPLe; the caption should list all five curves explicitly, as the main text does.
  6. [Section 4.4] The random selection of the 12-game subset uses numpy without reporting a seed; since the selected games are not listed in the main text (only in Table 2), the seed or an explicit game list should be given for full reproducibility.

Circularity Check

0 steps flagged · score 2.0 of 10

Empirical benchmark claim is self-contained; only a minor non-load-bearing self-citation is present.

full rationale

The central claim is an empirical result (mean HNS 0.682 vs 0.525 for SimPLe(30)) obtained by running EVaDE-SimPLe and SimPLe(30) under the same codebase, schedule, and hyperparameters, with the CURL and original SimPLe baseline scores taken from external papers rather than fitted. No target number is used to define the method, and the ablations compare one-layer variants against the same SimPLe(30) control, so the improvements do not reduce by construction. The variational-posterior interpretation of Gaussian dropout is inherited from external results, principally Gal & Ghahramani and Kingma et al., and is a framing assumption rather than a derivation from the paper's own equations. Theorem 1 is a genuine identity-construction representability proof, not a renamed input assumption. The only self-citation is reference [3], used as one of several related-work pointers; it is not load-bearing for the empirical claim. The head-to-head comparison with the original 15-iteration SimPLe may be confounded by the 30-iteration schedule, but that is a benchmarking/correctness concern, not circularity.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

No new physical entities are introduced and no hand-set constants are fitted to produce a target result. The central risks are the domain assumptions about channel semantics and about dropout-as-posterior, plus the learned dropout variances that drive the perturbation magnitude.

free parameters (1)
  • Gaussian dropout variances (sigma_env^i)^2 = Learned jointly with model parameters; values not reported
    These variational parameters control the magnitude of reward perturbations in Equation 1 and are optimized on Atari reward prediction data. They are part of the exploration mechanism, though they are learned rather than hand-set constants.
assumptions (4)
  • domain assumption Gaussian multiplicative dropout induces variational distributions that approximate the posterior (Gal and Ghahramani).
    Invoked in Section 3 and Appendix C to justify calling the perturbed reward model a sample from an approximate posterior. It is imported from prior work and not established for this specific architecture.
  • domain assumption Intermediate convolutional channels in Atari reward models capture object positions or events.
    Stated as a hypothesis in Section 3. The three layer designs in Equations 2 through 4 are only meaningful if this holds.
  • domain assumption Perturbing object interactions, event weights, and event translations in the reward function yields useful exploratory trajectories in object-based domains.
    Section 3 and Section 5 use this mechanism to explain the claimed exploration benefit; it is a behavioral assumption about the environments rather than a proven theorem.
  • ad hoc to paper Optimizing a policy against one sampled perturbed reward model for an entire SimPLe iteration approximates posterior sampling for reinforcement learning.
    Section 3.5 and Algorithm 1 sample only reward-model parameters; the transition model is deterministic and no posterior over it is sampled. This assumption is specific to the paper's PSRL framing.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EVaDE : Event-Based Variational Thompson Sampling for Model-Based Reinforcement Learning." pith.science (2026). https://pith.science/paper/X762V5T5

@misc{pith2026250109611,
  author       = {Pith},
  title        = {Pith review of: EVaDE : Event-Based Variational Thompson Sampling for Model-Based Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/X762V5T5}},
  note         = {Machine review of arXiv:2501.09611}
}
read the original abstract

Posterior Sampling for Reinforcement Learning (PSRL) is a well-known algorithm that augments model-based reinforcement learning (MBRL) algorithms with Thompson sampling. PSRL maintains posterior distributions of the environment transition dynamics and the reward function, which are intractable for tasks with high-dimensional state and action spaces. Recent works show that dropout, used in conjunction with neural networks, induces variational distributions that can approximate these posteriors. In this paper, we propose Event-based Variational Distributions for Exploration (EVaDE), which are variational distributions that are useful for MBRL, especially when the underlying domain is object-based. We leverage the general domain knowledge of object-based domains to design three types of event-based convolutional layers to direct exploration. These layers rely on Gaussian dropouts and are inserted between the layers of the deep neural network model to help facilitate variational Thompson sampling. We empirically show the effectiveness of EVaDE-equipped Simulated Policy Learning (EVaDE-SimPLe) on the 100K Atari game suite.

Figures

Figures reproduced from arXiv: 2501.09611 by the authors.

Figure 1
Figure 1. Rewards in Breakout, a popular Atari game. (a) shows an interaction between [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. (a) This image shows one noisy event interaction filter acting on an input with [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. The network architecture of the environment model used to train EVaDE-SimPLe. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Learning curves of EVaDE-SimPLe agents, SimPLe(30) agents and agents which [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: This figure shows the output map that captures interactions between two input [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: This figure shows an output map (channel) that up-weights the corresponding [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: This figure shows an output map (channel) that down-weights the corresponding [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: This figure shows the function of the noisy translation layer. The output map [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Learning curves of EVaDE-SimPLe agents, SimPLe(30) agents and agents which [PITH_FULL_IMAGE:figures/full_fig_p031_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 36 canonical work pages

  1. [1]

    Deep Reinforcement Learning at the Edge of the Statistical Precipice

    Rishabh Agarwal, Max Schwarzer, Pablo Samuel Castro, Aaron C Courville, and Marc Bellemare. Deep Reinforcement Learning at the Edge of the Statistical Precipice. Advances in Neural Information Processing Systems , 34, 2021

    work page 2021

  2. [2]

    Optimistic Posterior Sampling for Reinforcement Learning: Worst-Case Regret Bounds

    Shipra Agrawal and Randy Jia. Optimistic Posterior Sampling for Reinforcement Learning: Worst-Case Regret Bounds. In I. Guyon, U. V. Luxburg, S. Bengio, H. W allach, R. F ergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 30. Curran Associates, Inc., 2017. URL https://proceedings.neurips.cc/paper/20...

    work page 2017

  3. [3]

    State-Aware Variational Thompson Sampling for Deep Q-Networks

    Siddharth Aravindan and W ee Sun Lee. State-Aware Variational Thompson Sampling for Deep Q-Networks. In 20th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2021) , pages 124–132, 2021

    work page 2021

  4. [4]

    Efficient Exploration through Bayesian Deep Q- Networks

    Kamyar Azizzadenesheli, Emma Brunskill, and Animashree Anandkumar. Efficient Exploration through Bayesian Deep Q- Networks. In 2018 Information Theory and Applications Workshop (ITA) , pages 1–9. IEEE, 2018

    work page 2018

  5. [5]

    Campbell, and Sergey Levine

    Mohammad Babaeizadeh, Chelsea Finn, Dumitru Erhan, Roy H. Campbell, and Sergey Levine. Stochastic Variational Video Prediction. In International Conference on Learning Representations , 2018. URL https://openreview.net/forum?id=rk49Mg-CW

    work page 2018

  6. [6]

    Unifying Count-Based Explo- ration and Intrinsic Motivation

    Marc Bellemare, Sriram Srinivasan, Georg Ostrovski, T om Schaul, David Saxton, and Remi Munos. Unifying Count-Based Explo- ration and Intrinsic Motivation. In Advances in Neural Information Processing Systems , pages 1471–1479, 2016

    work page 2016

  7. [8]

    Path Integral Guided Policy Search

    Y evgen Chebotar, Mrinal Kalakrishnan, Ali Y ahya, Adrian Li, Stefan Schaal, and Sergey Levine. Path Integral Guided Policy Search. In 2017 IEEE international conference on robotics and automation (ICRA) , pages 3381–3388. IEEE, 2017

    work page 2017

  8. [9]

    Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search

    R´ emi Coulom. Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search. In International conference on computers and games , pages 72–83. Springer, 2006

    work page 2006

Show all 45 references
  1. [10]

    Efficient Model-Based Reinforcement Learning through Optimistic Policy Search and Planning

    Sebastian Curi, F elix Berkenkamp, and Andreas Krause. Efficient Model-Based Reinforcement Learning through Optimistic Policy Search and Planning. Advances in Neural Information Processing Systems , 33, 2020

    2020

  2. [11]

    Noisy Networks for Exploration

    Meire F ortunato, Mohammad Azar, Bilal Piot, Jacob Menick, Matteo Hessel, Ian Osband, Alex Graves, V olodymyr Mnih, Remi Munos, Demis Hassabis, Olivier Pietquin, Charles Blundell, and Shane Legg. Noisy Networks for Exploration. In International Conference on Learning Represent...

    2018

  3. [12]

    Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning

    Y arin Gal and Zoubin Ghahramani. Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. In international conference on machine learning , pages 1050–1059. PMLR, 2016

    2016

  4. [13]

    T emporal Difference Variational Auto-Encoder

    Karol Gregor, George Papamakarios, F rederic Besse, Lars Buesing, and Theophane W eber. T emporal Difference Variational Auto-Encoder. In International Conference on Learning Representations , 2019. URL https://openreview.net/forum?id=S1x4ghC9tQ

    2019

  5. [14]

    Recurrent World Models Facilitate Policy Evolution

    David Ha and J¨ urgen Schmidhuber. Recurrent World Models Facilitate Policy Evolution. In Proceedings of the 32nd International Conference on Neural Information Processing Systems , pages 2455–2467, 2018

    2018

  6. [15]

    Learning Latent Dynamics for Planning from Pixels

    Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, and James Davidson. Learning Latent Dynamics for Planning from Pixels. In International Conference on Machine Learning , pages 2555–2565. PMLR, 2019

    2019

  7. [16]

    Harris, K

    Charles R. Harris, K. Jarrod Millman, St´ efan J. van der W alt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian T aylor, Sebastian Berg, Nathaniel J. Smith, Robert Kern, Matti Picus, Stephan Hoyer, Marten H. van Kerkwijk, Matthew Brett, Allan Haldane, Jaim...

    2020

  8. [17]

    Deep reinforcement learning that matters

    Peter Henderson, Riashat Islam, Philip Bachman, Joelle Pineau, Doina Precup, and David Meger. Deep reinforcement learning that matters. In Proceedings of the AAAI conference on artificial intelligence , volume 32, 2018

    2018

  9. [18]

    Near-Optimal Regret Bounds for Reinforcement Learning

    Thomas Jaksch, Ronald Ortner, and Peter Auer. Near-Optimal Regret Bounds for Reinforcement Learning. Journal of Machine Learning Research, 11(Apr):1563–1600, 2010

    2010

  10. [19]

    Importance of Using Appropriate Baselines for Evaluation of Data-Efficiency in Deep Reinforcement Learning for Atari

    Kacper Kielak. Importance of Using Appropriate Baselines for Evaluation of Data-Efficiency in Deep Reinforcement Learning for Atari. CoRR, abs/2003.10181, 2020. URL https://arxiv.org/abs/2003.10181

    2003 arXiv

  11. [20]

    V ariational Dropout and the Local Reparameterization Trick

    Diederik P Kingma, Tim Salimans, and Max W elling. V ariational Dropout and the Local Reparameterization Trick. In Proceedings of the 28th International Conference on Neural Information Processing Systems-Volume 2 , pages 2575–2583, 2015

    2015

  12. [21]

    CURL: Contrastive Unsupervised Representations for Reinforcement Learn- ing

    Michael Laskin, Aravind Srinivas, and Pieter Abbeel. CURL: Contrastive Unsupervised Representations for Reinforcement Learn- ing. In International Conference on Machine Learning , pages 5639–5650. PMLR, 2020

    2020

  13. [22]

    Guided Policy Search

    Sergey Levine and Vladlen Koltun. Guided Policy Search. In International conference on machine learning , pages 1–9. PMLR, 2013

    2013

  14. [23]

    F earing, and Sergey Levine

    Anusha Nagabandi, Gregory Kahn, Ronald S. F earing, and Sergey Levine. Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning. CoRR, abs/1708.02596, 2017. URL http://arxiv.org/abs/1708.02596

    2017 arXiv

  15. [24]

    Action-Conditional Video Prediction Using Deep Networks in Atari Games

    Junhyuk Oh, Xiaoxiao Guo, Honglak Lee, Richard Lewis, and Satinder Singh. Action-Conditional Video Prediction Using Deep Networks in Atari Games. In Proceedings of the 28th International Conference on Neural Information Processing Systems-Volume 2 , pages 2863–2871, 2015. EV a...

    2015

  16. [25]

    Bootstrapped Thompson Sampling and Deep Exploration

    Ian Osband and Benjamin V an Roy . Bootstrapped Thompson Sampling and Deep Exploration. arXiv preprint arXiv:1507.00300 , 2015

    2015 arXiv

  17. [26]

    Why is Posterior Sampling Better than Optimism for Reinforcement Learning? InInternational Conference on Machine Learning , pages 2701–2710, 2017

    Ian Osband and Benjamin V an Roy . Why is Posterior Sampling Better than Optimism for Reinforcement Learning? InInternational Conference on Machine Learning , pages 2701–2710, 2017

    2017

  18. [27]

    (More) Efficient Reinforcement Learning via Posterior Sampling

    Ian Osband, Daniel Russo, and Benjamin V an Roy . (More) Efficient Reinforcement Learning via Posterior Sampling. In Advances in Neural Information Processing Systems , pages 3003–3011, 2013

    2013

  19. [28]

    Deep Exploration via Bootstrapped DQN

    Ian Osband, Charles Blundell, Alexander Pritzel, and Benjamin V an Roy . Deep Exploration via Bootstrapped DQN. In Advances in Neural Information Processing Systems , pages 4026–4034, 2016

    2016

  20. [29]

    Generalization and Exploration via Randomized Value Functions

    Ian Osband, Benjamin V an Roy , and Zheng W en. Generalization and Exploration via Randomized Value Functions. InProceedings of the 33rd International Conference on Machine Learning-Volume 48 , pages 2377–2386. JMLR. org, 2016

    2016

  21. [30]

    Chen, Xi Chen, T amim Asfour, Pieter Abbeel, and Marcin Andrychowicz

    Matthias Plappert, Rein Houthooft, Prafulla Dhariwal, Szymon Sidor, Richard Y. Chen, Xi Chen, T amim Asfour, Pieter Abbeel, and Marcin Andrychowicz. Parameter Space Noise for Exploration. In International Conference on Learning Representations, 2018. URL https://openreview.net...

    2018

  22. [31]

    Evolution Strategies as a Scalable Alternative to Reinforcement Learning

    Tim Salimans, Jonathan Ho, Xi Chen, Szymon Sidor, and Ilya Sutskever. Evolution Strategies as a Scalable Alternative to Reinforcement Learning. arXiv preprint arXiv:1703.03864 , 2017

    2017 arXiv

  23. [32]

    Proximal Policy Optimization Algorithms

    John Schulman, Filip W olski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal Policy Optimization Algorithms. CoRR, abs/1707.06347, 2017. URL http://arxiv.org/abs/1707.06347

    2017 arXiv

  24. [33]

    Dropout: A Simple Way to Prevent Neural Networks from Overfitting

    Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky , Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. Journal of Machine Learning Research , 15(56):1929–1958, 2014. URL http://jmlr.org/ papers/v15/srivastava14a.html

    1929

  25. [34]

    A Bayesian Framework for Reinforcement Learning

    Malcolm Strens. A Bayesian Framework for Reinforcement Learning. In International Conference on Machine Learning , volume 2000, pages 943–950, 2000

    2000

  26. [35]

    Dyna, an Integrated Architecture for Learning, Planning, and Reacting

    Richard S Sutton. Dyna, an Integrated Architecture for Learning, Planning, and Reacting. ACM Sigart Bulletin , 2(4):160–163, 1991

    1991

  27. [36]

    On the Likelihood that One Unknown Probability Exceeds Another in View of the Evidence of Two Samples

    William R Thompson. On the Likelihood that One Unknown Probability Exceeds Another in View of the Evidence of Two Samples. Biometrika, 25(3/4):285–294, 1933

    1933

  28. [37]

    V ariational Inference for the Multi-Armed Contextual Bandit

    I˜ nigo Urteaga and Chris Wiggins. V ariational Inference for the Multi-Armed Contextual Bandit. In International Conference on Artificial Intelligence and Statistics , pages 698–706. PMLR, 2018

    2018

  29. [38]

    When to Use Parametric Models in Reinforcement Learning? In NeurIPS, pages 14322–14333, 2019

    Hado van Hasselt, Matteo Hessel, and John Aslanides. When to Use Parametric Models in Reinforcement Learning? In NeurIPS, pages 14322–14333, 2019

    2019

  30. [39]

    Gomez, Stephan Gouws, Llion Jones, Lukasz Kaiser, Nal Kalchbrenner, Niki Parmar, Ryan Sepassi, Noam Shazeer, and Jakob Uszkoreit

    Ashish V aswani, Samy Bengio, Eugene Brevdo, F rancois Chollet, Aidan N. Gomez, Stephan Gouws, Llion Jones, Lukasz Kaiser, Nal Kalchbrenner, Niki Parmar, Ryan Sepassi, Noam Shazeer, and Jakob Uszkoreit. Tensor2Tensor for Neural Machine Translation. CoRR, abs/1803.07416, 2018. ...

    2018 arXiv

  31. [40]

    Thompson Sampling via Local Uncertainty

    Zhendong W ang and Mingyuan Zhou. Thompson Sampling via Local Uncertainty . InProceedings of the 37th International Conference on Machine Learning , volume 119 of Proceedings of Machine Learning Research, pages 10115–10125. PMLR, 13–18 Jul 2020. URL http://proceedings.mlr.pres...

    2020

  32. [41]

    Model Predictive Path Integral control Using Covariance Variable Importance Sampling

    Grady Williams, Andrew Aldrich, and Evangelos Theodorou. Model Predictive Path Integral control Using Covariance Variable Importance Sampling. arXiv preprint arXiv:1509.01149 , 2015

    2015 arXiv

  33. [42]

    NADPEx: An On-Policy Temporally Consistent Exploration Method for Deep Reinforcement Learning

    Sirui Xie, Junning Huang, Lanxin Lei, Chunxiao Liu, Zheng Ma, W ei Zhang, and Liang Lin. NADPEx: An On-Policy Temporally Consistent Exploration Method for Deep Reinforcement Learning. In International Conference on Learning Representations , 2019. URL https://openreview.net/fo...

    2019

  34. [43]

    Mastering atari games with limited data

    W eirui Y e, Shaohuai Liu, Thanard Kurutach, Pieter Abbeel, and Y ang Gao. Mastering atari games with limited data. Advances in Neural Information Processing Systems , 34, 2021

    2021

  35. [44]

    Scalable Thompson Sampling via Optimal Transport

    Ruiyi Zhang, Zheng W en, Changyou Chen, Chen F ang, T ong Y u, and Lawrence Carin. Scalable Thompson Sampling via Optimal Transport. In The 22nd International Conference on Artificial Intelligence and Statistics , pages 87–96. PMLR, 2019

    2019

  36. [45]

    Model Based Reinforcement Learning for Atari

    Kaiser Lukasz, Babaeizadeh Mohammad, Mi los Piotr, Osi´ nski B la ˙ zej, Campbell Roy , H, Czechowski Konrad, Erhan Dumitru, Finn Chelsea, Kozakowski Piotr, Levine Sergey , Mohiuddin Afroz, Sepassi Ryan, T ucker George, and Michalewski Henryk. Model Based Reinforcement Learnin...

    2020

  37. [2019]

    URL http://arxiv.org/abs/1905.12726

    1905 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.