Pith. sign in

REVIEW 4 major objections 4 minor 32 references

Minding Motivation: The Effect of Intrinsic Motivation on Agent Behaviors

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Intrinsic motivation changes how game agents play, not just the rewards they earn.

desk verdict An honest, reproducible empirical study of intrinsic motivation effects on MiniGrid behavior, but the quantitative support for GRM's mitigation claim rests on a confounded metric and needs a controlled re-analysis before acceptance. read the letter →

arxiv 2507.19725 v1 pith:YZXSIO3U submitted 2025-07-26 cs.AI

Leonardo Villalobos-Arias , Grant Forbes , Jianxun Wang , David L Roberts , Arnav Jhala This is my paper
classification cs.AI
keywords intrinsicmotivationreinforcementlearningrewardhackingpolicydivergenceshapingMiniGridexplorationinvariance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks a question most reinforcement-learning evaluations skip: beyond changing scores, does intrinsic motivation change how an agent actually behaves? The authors train agents with three intrinsic motivation methods (State Counting, Max Entropy, and the Intrinsic Curiosity Model) on five maps from MiniGrid, a suite of small grid-world games, each with and without Generalized Reward Matching (GRM), a shaping method meant to keep the optimal policy unchanged. They find that intrinsic motivation does more than speed up reward acquisition: it visibly shifts the strategies the agent settles into, and different methods produce recognizable behavioral styles. GRM reduces divergence from the no-intrinsic-motivation baseline in several scenarios and curbs some reward hacking (agents chasing the extra reward instead of the game's objective), but its optimality guarantee does not fully show up within the training budget used, so deviations remain. The upshot is that judging intrinsic motivation by reward curves alone misses a real dimension of its effect, and theoretical guarantees need empirical checks on finite horizons.

What carries the argument

The central measuring instrument is policy divergence, defined as the mean absolute difference in position visitation frequency between an IM-trained agent and the no-IM baseline. After training, the authors simulate each agent for 5,000 steps on each of ten randomly selected map instances, record how often the agent visits every grid position, and compute $\frac{1}{N}\sum_{i,j}|S(i,j)-S'(i,j)|$, where $N=5000$ total steps and $S$, $S'$ are the visitation distributions of the IM agent and baseline. A value of 0 means identical policies and 2 means maximally different. Position heatmaps taken at 5% and 100% of training supplement the metric by showing where each agent spends its time. The design also depends on choosing environments that no-IM agents can learn without intrinsic rewards, so that a meaningful baseline policy exists to diverge from.

What would settle it

Re-run the same protocol with a sequence-aware behavior metric, such as the distribution of full action trajectories or state-action occupancy, on maps where the no-IM baseline learns reliably in every run; if the IM agents then look identical to the baseline, the claim that intrinsic motivation alters behavior would be refuted. A second check is to train GRM agents for five to ten times longer: if their policies converge exactly to the no-IM baseline, the reported GRM deviation is a finite-budget artifact rather than a genuine behavioral effect.

Watch

Extended reading notes

Core claim

The central finding is that intrinsic motivation changes policy behavior, not just reward curves. On MiniGrid, State Count encourages early exploration that later turns into over-exploration, Max Entropy produces risk-averse policies that stick to familiar regions, and ICM behaves as a milder middle ground. Combining these rewards with GRM (specifically D-GRM) lowers policy divergence from the no-IM baseline for non-State Count methods on several maps and mitigated reward hacking in some scenarios, yet GRM agents still deviated from the baseline at 20.48 million frames, so the theoretical policy-invariance guarantee does not translate into practical invariance on this training budget. The authors also report cases where intrinsic motivation produced policies they judged closer to optimal than the baseline, especially under uncertainty, suggesting the side effects of IM rewards are not always harmful.

Load-bearing premise

The comparison treats the no-intrinsic-motivation policy as the reference for 'normal' behavior, and treats how often an agent occupies each grid position as a complete account of how it plays, even though the order of actions is ignored and the baseline sometimes does not learn the task at all.

Editorial extensions

If this is right

  • Reward curves alone understate the effect of intrinsic motivation, so IM evaluations should include behavioral measures such as policy divergence or visitation heatmaps.
  • State Count finds sparse rewards earliest but keeps over-exploring; Max Entropy settles into narrow, risk-averse paths; ICM sits between the two.
  • GRM reduces policy divergence and visible reward hacking for Max Entropy and ICM on some maps, but its policy-invariance guarantee needs longer training to become observable.
  • On harder maps such as DoorKey-16x16, only State Count-based agents learn the task within the budget, so the other methods cannot be compared against a meaningful baseline there.
  • Combining State Count for early exploration with Max Entropy for later refinement is a concrete recipe the paper suggests for more reliable IM training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension would replace occupancy with action-order statistics; the paper admits heatmaps disregard the order of operations, so sequence-level divergence could reveal even larger behavioral differences than reported.
  • If these archetypes generalize, then the common instinct to treat reward hacking as uniformly harmful should be revisited: under uncertainty, the IM policies in this study often looked closer to optimal than the no-IM baseline.
  • Because GRM's guarantee is asymptotic, the right question is not whether GRM preserves the policy, but how quickly divergence decays with training; a scaling study across budgets would settle its practical value.
  • The policy-divergence numbers on maps where the baseline fails conflate task progress with behavioral change; recomputing the metric only over runs where both agents solve the task would isolate the pure policy effect.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper reports an empirical study of three intrinsic motivation (IM) methods—State Count, Max Entropy, and ICM—with and without Generalized Reward Matching (GRM), on five MiniGrid environments. It measures episodic return and position coverage during training, and uses position-visitation heatmaps and a policy-divergence statistic to assess behavioral differences from a no-IM baseline. The central claims are that IM changes not only reward curves but also agent behavior, and that GRM partially mitigates reward hacking in some scenarios.

Significance. If the behavioral findings are robust, the study would be a useful first step toward characterizing how IM shapes policies, and it would provide a benchmark of GRM beyond Montezuma's Revenge. Strengths include the use of learnable environments with no-IM baselines, ten runs per condition, public code, transparent reporting of hyperparameters and beta values, and an explicit threats-to-validity section. The analysis is primarily empirical and descriptive, with no derivation; the paper is appropriately cautious in its conclusions and acknowledges several limitations. However, the quantitative support for the GRM mitigation claim is currently incomplete, and the central comparison is confounded by independent tuning of the intrinsic reward coefficient.

major comments (4)
  1. [Table 3] The policy-divergence numbers are reported as single means with no variance or significance test. With ten map instances per condition, differences such as FourRooms GRM+ME 0.67 versus Max Entropy 0.84, or RedBlueDoors GRM+ME 0.55 versus Max Entropy 0.99, cannot be separated from seed-to-seed noise. The claim that 'GRM mitigated some of the issues of reward hacking' (FourRooms Policy Divergence paragraph) rests on this table, so the quantitative basis for the paper's second central claim is not yet established.
  2. [Evaluation section; Results] The policy-divergence baseline is a no-IM policy that fails to learn on several maps: 2/10 runs on DoorKey-8x8 and RedBlueDoors-8x8, and all runs on DoorKey-16x16 (Results; Threats to Validity). Averaging divergence over runs that mix successful and failed baselines does not isolate the behavioral effect of IM; on DoorKey-16x16, where no-IM never learns, any comparison is dominated by whether the IM agent learned at all. A per-seed analysis restricted to runs where the baseline reaches a competent policy is required before the divergence values can support the mitigation claim.
  3. [Table 2] The intrinsic reward coefficient beta is tuned independently per method and map, and in several conditions the GRM and non-GRM variants use different beta values (e.g., FourRooms State Count beta=1 versus GRM+SC beta=0.05; RedBlueDoors SC beta=1 versus GRM+SC beta=0.1). The GRM-versus-non-GRM comparisons therefore conflate the effect of GRM with a change in the magnitude of the intrinsic reward. The authors should either match beta across paired conditions or explicitly justify the tuning as part of the method's configuration and discuss its impact on the comparison.
  4. [Evaluation section; Conclusions and Discussion] The policy-divergence metric is based solely on position-visitation frequency and ignores the temporal order of actions, as the paper itself concedes ('disregards the order of operations'). Since the abstract claims IM alters 'the way the agent plays,' the current metric can only support a claim about spatial occupancy, not about action sequences or decision processes. Additionally, 'reward hacking' is not operationally defined: it is inferred from heatmap patterns in the FourRooms and RedBlueDoors Policy Divergence paragraphs, but no quantitative criterion (e.g., time spent away from task-relevant regions or deviation from optimal trajectories) is provided. Without such a definition, the claim that GRM 'mitigated reward hacking' is difficult to evaluate independently.
minor comments (4)
  1. [Experimental Design; Threats to Validity] The training budget is stated as '20.48 million frames (1,000 rollouts)' in Experimental Design but '10 million frames total' in Threats to Validity; with 16 parallel environments and 128 frames per rollout, 1,000 rollouts would be 2.048 million frames. Please reconcile these numbers.
  2. [Results] There are typos in the Results section: 'reward metics' should be 'reward metrics', and 'it is likely intrinsic reward distracted the agent' should be 'it is likely that the intrinsic reward distracted the agent'.
  3. [Conclusions and Discussion] The terms 'the protocol study' and 'the baseline study' are used interchangeably to refer to Kayal, Pignatelli, and Toni (2025); for clarity, use one consistent term throughout.
  4. [Results, DoorKey-8x8 Policy Divergence] In the DoorKey-8x8 Policy Divergence paragraph, the sentence 'The IM model resulted on a better policy' is ambiguous: it appears to refer to non-GRM Max Entropy, but the preceding sentence discusses GRM+ME showing higher divergence. Please clarify which model is being compared.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: all central claims are direct empirical measurements, and the same-group citation for GRM's optimality guarantee is background, not load-bearing.

full rationale

This paper does not derive predictions from assumptions or fit parameters and then relabel them as predictions. The central claims — that intrinsic motivation changes return curves and position-visitation behavior, and that GRM reduces policy divergence in some maps — are supported by direct measurements (Figures 2-3, heatmaps, Table 3). The policy divergence metric is defined as the L1 difference in occupancy against a no-IM baseline; while this operationalization is coarse and can be distorted by failed no-IM baselines (e.g., DoorKey-16x16), that is a construct-validity and statistical-support concern, not a circular reduction: the divergence values are not equal to the inputs by construction. The paper also concedes in the Conclusions that heatmaps 'disregard the order of operations,' which limits the strength of 'altering the way the agent plays' but does not make the claim tautological. The only notable self-citation is the selection of GRM/D-GRM from Forbes et al. 2024b, whose optimality guarantee is cited as background for why GRM is a policy-invariant method. That citation is not load-bearing for the paper's own empirical findings, which would stand or fall on the reported runs; moreover the paper explicitly observes that GRM still deviates from baseline in short horizons, undercutting any appeal to the guarantee as the source of the result. No equation reduces to another equation, no fitted coefficient is relabeled as a prediction, and no alternative is ruled out by an imported uniqueness claim. The empirical benchmark is therefore self-contained with respect to circularity.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new theoretical constructs. Its empirical claims rest on tuned β values, the transferred PPO setup, the borrowed GRM guarantee, and the position-frequency behavioral metric.

free parameters (1)
  • Intrinsic reward coefficient β = Per map and method, see Table 2 (e.g., DoorKey-8x8 StateCount β=1, MaxEntropy β=0.001, ICM β=0.1)
    Manually tuned per map and method over a 13-value range; comparisons between GRM and non-GRM are confounded when β differs (e.g., FourRooms SC β=1 vs GRM+SC β=0.05).
assumptions (4)
  • domain assumption PPO is an appropriate base learner and its hyperparameters from Kayal et al. transfer to these environments
    Used without re-tuning; stated in Experimental Design.
  • domain assumption GRM is policy-invariant as proven in Forbes et al. 2024b
    Relied upon for the claim that GRM guarantees optimality; not re-derived here. The paper's own results show empirical deviations, suggesting asymptotic nature.
  • domain assumption MiniGrid is representative of game-like environments for behavior analysis
    Stated in Generalizability: 'Grid-live environments of around the same complexity'.
  • ad hoc to paper Position visitation frequency is a sufficient behavioral summary
    The policy divergence metric uses only position frequencies, ignoring action order; authors acknowledge this in Conclusions: 'disregards the order of operations'.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Minding Motivation: The Effect of Intrinsic Motivation on Agent Behaviors." pith.science (2026). https://pith.science/paper/YZXSIO3U

@misc{pith2026250719725,
  author       = {Pith},
  title        = {Pith review of: Minding Motivation: The Effect of Intrinsic Motivation on Agent Behaviors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YZXSIO3U}},
  note         = {Machine review of arXiv:2507.19725}
}
read the original abstract

Games are challenging for Reinforcement Learning~(RL) agents due to their reward-sparsity, as rewards are only obtainable after long sequences of deliberate actions. Intrinsic Motivation~(IM) methods -- which introduce exploration rewards -- are an effective solution to reward-sparsity. However, IM also causes an issue known as `reward hacking' where the agent optimizes for the new reward at the expense of properly playing the game. The larger problem is that reward hacking itself is largely unknown; there is no answer to whether, and to what extent, IM rewards change the behavior of RL agents. This study takes a first step by empirically evaluating the impact on behavior of three IM techniques on the MiniGrid game-like environment. We compare these IM models with Generalized Reward Matching~(GRM), a method that can be used with any intrinsic reward function to guarantee optimality. Our results suggest that IM causes noticeable change by increasing the initial rewards, but also altering the way the agent plays; and that GRM mitigated reward hacking in some scenarios.

Figures

Figures reproduced from arXiv: 2507.19725 by the authors.

Figure 1
Figure 1. MiniGrid maps used in this experiment. In contrast with the protocol study, we only use the stan￾dard grid partial observation space (7 × 7 × 3 tensor). The actions available to an agent in MiniGrid are: turn left, turn right, move forward, pickup, drop, and toggle. Model Architecture Following the protocol study, we use Proximal Policy Op￾timization (Schulman et al. 2017) as the base learning al￾gorithm. In contras… view at source ↗
Figure 2
Figure 2. Episodic rewards per iteration of the all trained models. Columns group results by map and rows by type of IM: 1) [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Position (tiles in grid) coverage per iteration of the all trained models. Columns group results by map and rows by [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Heatmaps with the position visitation frequency of the seven trained agents through early training and final policy on [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Heatmaps with the position visitation frequency of the seven trained agents through early training and final policy on [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Heatmaps with the position visitation frequency of the seven trained agents through early training and final policy on [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Heatmaps with the position visitation frequency of the seven trained agents through early training and final policy on [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

32 extracted references · 21 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Andres, A.; Villar-Rodriguez, E.; and Del Ser, J. 2022. An evaluation study of intrinsic motivation techniques applied to reinforcement learning over hard exploration environments. In International Cross-Domain Conference for Machine Learning and Knowledge Extraction, 201--220. Springer

    work page 2022

  4. [4]

    P.; Sprechmann, P.; Vitvitskyi, A.; Guo, D.; Piot, B.; Kapturowski, S.; Tieleman, O.; Arjovsky, M.; Pritzel, A.; Bolt, A.; and Blundell, C

    Badia, A. P.; Sprechmann, P.; Vitvitskyi, A.; Guo, D.; Piot, B.; Kapturowski, S.; Tieleman, O.; Arjovsky, M.; Pritzel, A.; Bolt, A.; and Blundell, C. 2020. Never Give Up: Learning Directed Exploration Strategies. In International Conference on Learning Representations

    work page 2020

  5. [5]

    E.; Harutyunyan, A.; and Bowling, M

    Behboudian, P.; Satsangi, Y.; Taylor, M. E.; Harutyunyan, A.; and Bowling, M. 2022. Policy invariant explicit shaping: an efficient alternative to reward shaping. Neural Computing and Applications, 1--14

    work page 2022

  6. [6]

    Burda, Y.; Edwards, H.; Pathak, D.; Storkey, A.; Darrell, T.; and Efros, A. A. 2018 a . Large-scale study of curiosity-driven learning. arXiv preprint arXiv:1808.04355

    arXiv 2018

  7. [7]

    Burda, Y.; Edwards, H.; Storkey, A.; and Klimov, O. 2018 b . Exploration by random network distillation. arXiv preprint arXiv:1810.12894

    arXiv 2018

  8. [8]

    Chen, E.; Hong, Z.-W.; Pajarinen, J.; and Agrawal, P. 2022. Redeeming intrinsic rewards via constrained optimization. In Koyejo, S.; Mohamed, S.; Agarwal, A.; Belgrave, D.; Cho, K.; and Oh, A., eds., Advances in Neural Information Processing Systems, volume 35, 4996--5008. Curran Associates, Inc

    work page 2022

Show all 32 references
  1. [9]

    S.; and Terry, J

    Chevalier-Boisvert, M.; Dai, B.; Towers, M.; Perez-Vicente, R.; Willems, L.; Lahlou, S.; Pal, S.; Castro, P. S.; and Terry, J. 2023. Minigrid & Miniworld : Modular & Customizable Reinforcement Learning Environments for Goal - Oriented Tasks . Advances in Neural Information Pro...

    2023

  2. [10]

    Colas, C.; Karch, T.; Sigaud, O.; and Oudeyer, P.-Y. 2022. Autotelic agents with intrinsically motivated goal-conditioned reinforcement learning: a short survey. Journal of Artificial Intelligence Research, 74: 1159--1199

    2022

  3. [11]

    C.; Gupta, N.; Villalobos-Arias, L.; Potts, C

    Forbes, G. C.; Gupta, N.; Villalobos-Arias, L.; Potts, C. M.; Jhala, A.; and Roberts, D. L. 2024 a . Potential-Based Reward Shaping for Intrinsic Motivation. In Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems, 589--597

    2024

  4. [12]

    C.; Villalobos-Arias, L.; Wang, J.; Jhala, A.; and Roberts, D

    Forbes, G. C.; Villalobos-Arias, L.; Wang, J.; Jhala, A.; and Roberts, D. L. 2024 b . Potential-Based Intrinsic Motivation: Preserving Optimality With Complex, Non-Markovian Shaping Rewards. arXiv preprint arXiv:2410.12197

    2024 arXiv

  5. [13]

    C.; Wang, J.; Villalobos-Arias, L.; Jhala, A.; and Roberts, D

    Forbes, G. C.; Wang, J.; Villalobos-Arias, L.; Jhala, A.; and Roberts, D. L. 2025. Action-Dependent Optimality-Preserving Reward Shaping. arXiv preprint arXiv:2505.12611

    2025 arXiv

  6. [14]

    Huang, S.; and Ontañón, S. 2020. Action Guidance : Getting the Best of Sparse Rewards and Shaped Rewards for Real -time Strategy Games . ArXiv:2010.03956 [cs]

    2020 arXiv

  7. [15]

    M.; White, A.; Silva, B

    Jordan, S. M.; White, A.; Silva, B. C. D.; White, M.; and Thomas, P. S. 2024. Position: Benchmarking is Limited in Reinforcement Learning Research. In Salakhutdinov, R.; Kolter, Z.; Heller, K.; Weller, A.; Oliver, N.; Scarlett, J.; and Berkenkamp, F., eds., Proceedings of the ...

    2024

  8. [16]

    Kayal, A.; Pignatelli, E.; and Toni, L. 2025. The impact of intrinsic rewards on exploration in Reinforcement Learning . ArXiv:2501.11533 [cs]

    2025 arXiv

  9. [17]

    Laskin, M.; Yarats, D.; Liu, H.; Lee, K.; Zhan, A.; Lu, K.; Cang, C.; Pinto, L.; and Abbeel, P. 2021. URLB : Unsupervised Reinforcement Learning Benchmark. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2)

    2021

  10. [18]

    Le, H.; Do, K.; Nguyen, D.; and Venkatesh, S. 2024. Beyond Surprise : Improving Exploration Through Surprise Novelty . In Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems , 1084--1092

    2024

  11. [19]

    Liu, J.; Gu, X.; and Liu, S. 2019. Policy optimization reinforcement learning with entropy regularization. arXiv preprint arXiv:1912.01557

    2019 arXiv

  12. [20]

    Mnih, V.; Kavukcuoglu, K.; Silver, D.; Graves, A.; Antonoglou, I.; Wierstra, D.; and Riedmiller, M. 2013. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602

    2013 arXiv

  13. [21]

    A.; Veness, J.; Bellemare, M

    Mnih, V.; Kavukcuoglu, K.; Silver, D.; Rusu, A. A.; Veness, J.; Bellemare, M. G.; Graves, A.; Riedmiller, M.; Fidjeland, A. K.; Ostrovski, G.; et al. 2015. Human-level control through deep reinforcement learning. nature, 518(7540): 529--533

    2015

  14. [22]

    Mohamed, S.; and Jimenez Rezende, D. 2015. Variational information maximisation for intrinsically motivated reinforcement learning. Advances in neural information processing systems, 28

    2015

  15. [23]

    A.; Silva, C

    Onta \ n \'o n, S.; Barriga, N. A.; Silva, C. R.; Moraes, R. O.; and Lelis, L. H. 2018. The first microrts artificial intelligence competition. AI Magazine, 39(1): 75--83

    2018

  16. [24]

    Oudeyer, P.-Y.; and Kaplan, F. 2007. What is intrinsic motivation? A typology of computational approaches. Frontiers in neurorobotics, 1: 108

    2007

  17. [25]

    A.; and Darrell, T

    Pathak, D.; Agrawal, P.; Efros, A. A.; and Darrell, T. 2017. Curiosity-driven exploration by self-supervised prediction. In International conference on machine learning, 2778--2787. PMLR

    2017

  18. [26]

    Raileanu, R.; and Rocktäschel, T. 2020. RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environments. In International Conference on Learning Representations

    2020

  19. [27]

    Schulman, J.; Wolski, F.; Dhariwal, P.; Radford, A.; and Klimov, O. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347

    2017 arXiv

  20. [28]

    L.; and Littman, M

    Strehl, A. L.; and Littman, M. L. 2008. An analysis of model-based interval estimation for Markov decision processes. Journal of Computer and System Sciences, 74(8): 1309--1331

    2008

  21. [29]

    A.; Fedus, W.; Machado, M

    Taiga, A. A.; Fedus, W.; Machado, M. C.; Courville, A.; and Bellemare, M. G. 2020. On Bonus Based Exploration Methods In The Arcade Learning Environment. In International Conference on Learning Representations

    2020

  22. [30]

    M.; Mathieu, M.; Dudzik, A.; Chung, J.; Choi, D

    Vinyals, O.; Babuschkin, I.; Czarnecki, W. M.; Mathieu, M.; Dudzik, A.; Chung, J.; Choi, D. H.; Powell, R.; Ewalds, T.; Georgiev, P.; et al. 2019. Grandmaster level in StarCraft II using multi-agent reinforcement learning. nature, 575(7782): 350--354

    2019

  23. [31]

    Wan, S.; Tang, Y.; Tian, Y.; and Kaneko, T. 2023. DEIR: Efficient and Robust Exploration through Discriminative-Model-Based Episodic Intrinsic Rewards. In Elkind, E., ed., Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI-23 , 42...

    2023

  24. [32]

    E.; and Tian, Y

    Zhang, T.; Xu, H.; Wang, X.; Wu, Y.; Keutzer, K.; Gonzalez, J. E.; and Tian, Y. 2021. Noveld: A simple yet effective exploration criterion. Advances in Neural Information Processing Systems, 34: 25217--25230

    2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.