Pith. sign in

REVIEW 5 major objections 6 minor 47 references

Deep Reinforcement Learning with Hybrid Intrinsic Reward Model

T0 review · 5 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that cycling through intrinsic rewards one at a time is the most robust way to combine exploration bonuses, beating sum, product, and max fusion on most hard-exploration tasks, and beating single bonuses in unsupervised…

desk verdict Useful empirical comparison of fusion strategies, but the NGU baseline in Eq. (4) is not the original NGU, and since NGU drives most top candidates, the cycle result needs to be verified against a faithful implementation. read the letter →

arxiv 2501.12627 v1 pith:ZZOEOWFP submitted 2025-01-22 cs.LG

Mingqi Yuan , Bo Li , Xin Jin , Wenjun Zeng This is my paper · ORCID
classification cs.LG
keywords hybridintrinsicrewardsmotivationexplorationrewardfusionstrategiesunsupervisedreinforcementlearningdeepMiniGridProcgen
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Intrinsic reward shaping adds a built-in signal — curiosity, novelty, or state entropy — to push a reinforcement-learning agent toward unexplored behavior when the environment's own rewards are sparse or absent. This paper argues that one such signal is rarely enough, and that how you combine several matters as much as which ones you pick. It introduces HIRE (Hybrid Intrinsic REward), a modular framework that fuses any number of intrinsic rewards through four strategies — summation, product, cycle, and maximum — and sweeps all combinations of four representative rewards (ICM, NGU, RE3, E3B) across sixteen hard-exploration tasks in MiniGrid and Procgen, plus unsupervised pre-training on Atari. The central finding is that the newly proposed cycle rule, which activates one intrinsic reward at a time on a rotating schedule, is the most robust fusion strategy on most tasks, and that hybrid rewards outperform every single-reward baseline in the unsupervised setting. If the paper is right, practitioners get a simple, low-tuning default for sparse-reward problems: combine two or three complementary bonuses and let them take turns.

What carries the argument

The carrying object is the time-multiplexing fusion rule itself. HIRE defines four ways to collapse a set of intrinsic reward functions $\{I^1_t,\dots,I^n_t\}$ into one scalar: summation $I_t = \sum_{i=1}^n w^i_t I^i_t$, product $I_t = \prod_{i=1}^n I^i_t$, cycle $I_t = I^i_t$ with $i = (t \bmod n)$, and maximum $I_t = \max_i I^i_t$. The cycle rule, newly proposed here, does the explanatory work: by attending to one motivation at a time, it sidesteps the scale-mismatch and conflicting-signal problems the authors attribute to summation and product, and the greedy over-exploration they attribute to maximum. The four rewards being fused span three families — ICM (prediction error of a learned dynamics model), NGU (lifelong RND novelty modulated by episodic pseudo-counts), RE3 (k-nearest-neighbor estimate of state-entropy), and E3B (ellipsoidal episodic bonus in a learned embedding) — and the whole module sits in front of a fixed PPO loop with all PPO hyperparameters held constant, so observed differences are attributed to the rewards and their fusion.

What would settle it

Re-run the same sixteen MiniGrid and Procgen tasks with independently reproduced implementations of ICM, NGU, RE3, and E3B from the original papers' released code, using the same PPO backbone, seeds, environment steps, and the same four fusion rules; if the cycle strategy does not stay at or near the top on most tasks, or if a single intrinsic reward already matches the best hybrids, the central claim fails. A cheaper probe is to hold one normalization scheme fixed across all four fusion rules and check whether the cycle advantage survives, since scale mismatch is the paper's own stated reason summation and product struggle.

Watch

Extended reading notes

Core claim

The discovery is that the fusion rule is a first-class design choice in intrinsically motivated RL. Across more than four thousand runs, the cycle strategy $I_t = I^i_t$ with $i = (t \bmod n)$ — using one intrinsic reward at a time and rotating through them — achieved the best performance on most of the sixteen MiniGrid and Procgen tasks and showed the greatest robustness, capturing 75% of top-1 candidates on MiniGrid and 50% on Procgen, while the product rule was the least stable on sequential navigation tasks like KeyCorridor. The reward NGU, itself a hybrid of lifelong and episodic novelty, appeared in twelve of the sixteen best candidates, and the authors single out the two-reward pair (NGU, RE3) as the best balance of exploration coverage and computational cost. In the ALE-5 unsupervised RL setting, hybrid intrinsic rewards produced significantly better fine-tuned scores than any single-reward pre-training. The quantity effect saturates: three rewards helped on navigation-heavy MiniGrid levels, two were generally best on procedurally generated Procgen levels, and frames-per-second falls steadily as rewards are added, so up to three rewards marks the practical efficiency frontier.

Load-bearing premise

The whole comparison assumes that the four intrinsic-reward implementations (ICM, NGU, RE3, E3B), taken from the authors' own toolkits with fixed recommended hyperparameters (Appendix B.1), faithfully represent how those algorithms perform; if those implementations differ from canonical versions or carry hidden tuning that happens to suit the cycle rule, the strategy ranking could be a property of the toolkit rather than of hybrid fusion.

Editorial extensions

If this is right

  • A default recipe for sparse-reward tasks: take two complementary bonuses such as NGU and RE3, cycle between them, and expect stronger and more robust scores than either bonus alone, with no fusion-weight tuning.
  • The quantity effect is finite — adding a third reward helps on sequential navigation tasks in MiniGrid, but on dynamically generated Procgen levels more rewards tend to hurt, so practitioners should test two and three rather than assume more is better.
  • Hybrid intrinsic rewards transfer to unsupervised RL: pre-training with combined bonuses improves downstream fine-tuning on ALE-5 compared with pre-training on any single bonus.
  • The product rule is the least reliable general choice for sequential tasks, although it can win in dynamic environments such as Chaser and Miner where satisfying several motivations simultaneously matters.
  • Computational cost grows with reward count, so two or three rewards define a practical efficiency frontier for the framework.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Cycle fusion may work as a form of temporal ensembling that averages out noise and scale quirks of individual reward channels; a direct test is to compare the fixed rotating schedule against random or adversarially chosen orders to see whether periodicity itself is what helps.
  • The results suggest reward fusion is a hyperparameter space of its own, and learned arbitration — letting the agent choose which bonus is active based on state or recent progress — is a natural successor to the fixed cycle rule that this paper implicitly benchmarks.
  • Because only the two best-performing combinations (NGU, RE3) and (NGU, RE3, ICM) were carried into the ALE-5 study, the claim that hybrids beat singles in unsupervised RL rests on a narrow slice; extending to more combinations and more games would test whether the advantage belongs to hybrids generally or to NGU's presence.
  • The recommendation of (NGU, RE3) is partly a compute argument, since RE3 needs no learned representation, so the same four fusion rules could be re-tested with representation-learning-free or skill-discovery rewards to see whether the cycle advantage generalizes beyond the curiosity and novelty families.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper introduces HIRE, a framework for combining multiple intrinsic reward signals through four fusion strategies (summation, product, cycle, and maximum), and evaluates the resulting hybrid rewards on eight MiniGrid tasks, eight Procgen tasks, and five ALE games in an unsupervised pretraining/fine-tuning setup. Using PPO as a fixed backbone and the authors' RLLTE/RLEXPLORE implementations of ICM, NGU, RE3, and E3B, the authors report over four thousand runs with five seeds, aggregate results with IQM and bootstrap confidence intervals, and rank all strategy/candidate combinations. The main empirical findings are that the cycle fusion strategy is the most robust and best-performing on most tasks, that NGU and RE3 are the most frequent contributors to the best candidates, that the (NGU, RE3) combination is recommended, and that hybrid rewards improve fine-tuned ALE-5 returns relative to single intrinsic rewards.

Significance. If the empirical claims hold, the paper offers a simple, algorithm-agnostic recipe — time-multiplexing intrinsic rewards — that could be a reliable default for hard-exploration RL, and it provides one of the more systematic comparisons of fusion strategies and reward combinations to date. The strengths are the breadth of the study (16 environments plus ALE-5, 45 reward candidates per fusion strategy), the use of IQM and bootstrap confidence intervals, the fixed PPO hyperparameters across conditions, and the modularity of the framework. The main weaknesses are that the central claims about exploration diversity and skill acquisition are not measured directly, and that the results depend on the fidelity of the authors' own implementations of the four intrinsic reward baselines. With those caveats addressed, this would be a useful systematic empirical contribution to intrinsically motivated RL.

major comments (5)
  1. [Abstract; §5.2 Q1–Q5] The abstract and conclusion claim that HIRE 'significantly enhance[s] exploration efficiency and diversity, as well as skill acquisition', but no direct measure of exploration diversity or skill acquisition is reported anywhere in the paper. All evidence is in the form of episode returns on MiniGrid/Procgen and fine-tuned returns on ALE-5. Please either add direct metrics (for example, state coverage, state entropy, or number/coverage of learned skills) or revise the claims to refer to return/performance rather than diversity. This matters because diversity is one of the two advertised benefits and is used to explain why the cycle strategy is superior.
  2. [§5.2 Q1; Tables 5–6] The claim that 'the cycle strategy demonstrates superior robustness and achieves the best performance on most tasks' is based on top-1 counts (75% on MiniGrid, 50% on Procgen) and aggregate IQM, but no paired significance test across environments or seeds is provided. With five seeds per environment, the top-1 proportions may be within sampling noise, and overlapping bootstrap intervals in Figure 3 do not by themselves establish a significant difference. Please report paired bootstrap confidence intervals (for example, the probability of improvement) or a nonparametric paired test across the 16 environments, and calibrate the abstract and §5.2 wording accordingly.
  3. [§5.2 Q5; Table 8; Figure 12] The ALE-5 evaluation is internally inconsistent: the text says that only (NGU, RE3) and (NGU, RE3, ICM) were selected as HIRE-2 and HIRE-3 candidates, yet Table 8 lists C(NGU, E3B, RE3, ICM) as the best candidate for BattleZone and Figure 12 includes HIRE-4 candidates in its rankings. Please clarify the exact candidate set used in the ALE-5 experiments. In addition, because the ALE-5 candidates were selected as the best performers on MiniGrid/Procgen, the URL comparison is a transfer/selection study rather than an unbiased evaluation of the framework; this limitation should be stated explicitly, or all hybrid combinations should be evaluated in the URL setting.
  4. [Appendix A Eq. (4); Appendix B.1] The fidelity of the NGU baseline is load-bearing: NGU appears in 12 of the 16 best reward candidates and in the recommended (NGU, RE3) combination, yet the implementation is taken from the authors' own RLLTE/RLEXPLORE toolkits rather than an independent reference implementation. The compressed formula in Eq. (4) is incomplete as written — 'min{max{α_t}, C}' lacks the lower clipping value — and the computation and normalization of α_t are not specified. Please report the exact α_t definition, the value of the decay rate κ in Eq. (2), the summation weights w_i in Table 1, and provide a sanity check by reproducing published single-reward baseline results with these implementations, so that the strategy rankings cannot be attributed to toolkit-specific artifacts.
  5. [§2.2; §5] The paper does not compare HIRE against established hybrid intrinsic reward methods such as RIDE, NovelD, AGAC, or even the original NGU product combination, which is itself a hybrid of episodic and lifelong novelty. Since the paper's contribution is specifically about combining intrinsic rewards, the lack of any existing hybrid baseline makes it difficult to judge whether the proposed fusion strategies offer practical advantages over prior hybrid designs. A comparison with at least one or two such baselines on the same environments would substantially strengthen the empirical claims.
minor comments (6)
  1. [Appendix A, Eq. (4)] The notation 'min{max{α_t}, C}' is malformed; it should be written as, for example, min{max{α_t, 1}, C}, and the definition of α_t should be given in the same appendix rather than referring only to 'the RND method'.
  2. [Eq. (2); Appendix B] The exploration coefficient β_t depends on a decay rate κ, but the value of κ is never reported in Table 4 or Appendix B. Please report κ for each benchmark, since it is a free parameter that affects the entire training curve.
  3. [Table 1] The statement that cycle and maximum 'can be viewed as special cases of the summation method' is inaccurate for maximum: maximum selects the largest realized reward and cannot be expressed as a fixed non-zero weight vector of the summation strategy.
  4. [§5.2 Q5; Figure 6] Figure 6 uses the legend HIRE-1 through HIRE-4, but the text says only HIRE-2 and HIRE-3 were the selected reward candidates; please reconcile the figure, the text, and Table 8 so that the ALE-5 candidate set is unambiguous.
  5. [§5.2 Q5] The description of the ALE-5 protocol says the fine-tuning phase uses 'extrinsic rewards only', but it is not stated whether β_t is set to zero or whether the intrinsic reward module is disabled; please clarify the exact protocol.
  6. [Abstract] The phrase 'significantly enhance exploration efficiency and diversity' is stronger than the evidence reported in the paper; consider replacing 'diversity' with 'downstream task performance' or adding the missing diversity metrics.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: HIRE's fusion-strategy rankings are empirical observations, not results that reduce to their own inputs.

full rationale

The paper's claims are empirical observations of a modular fusion framework, not derivations whose conclusions return as premises. The fusion strategies in Table 1 are definitions (summation, product, cycle, maximum), and the cycle strategy's reported advantage on MiniGrid and Procgen comes from measured PPO returns, not from these formulas. The only fitted quantity, beta_0, is tuned by grid search in Appendix B.1 and is never presented as a prediction; it is a standard scaling constant applied uniformly across all compared strategies. The ALE-5 candidates are chosen from the best MiniGrid and Procgen candidates, but ALE-5 is a separate held-out benchmark, so this is post-hoc selection on an independent evaluation set rather than circularity. The reliance on the authors' RLLTE and RLEXPLORE implementations (Appendix B.1) is a reproducibility and implementation-fidelity concern, not a circular reduction: those implementations are inputs to the experiment, not consequences of the conclusions. Similarly, the NGU variant in Eq. (4) may deviate from the original NGU algorithm, but that would be an implementation-validity issue; it does not make the strategy ranking equivalent to its inputs. The paper also candidly lists limitations in Section 6, including the restricted reward set and computational constraints, which further indicates that the claims are empirical and bounded rather than circular. No prediction or first-principles result reduces by construction to a fitted parameter, a self-citation, or a definitional identity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities. Its empirical claims rest on two tuned or implicit parameters (beta0 and the undefined summation weights) and on fiducial assumptions about the benchmark implementations and environment representativeness. All four axioms are domain assumptions or standard background; none is ad hoc to the paper's theoretical machinery.

free parameters (2)
  • beta0 (initial exploration coefficient) = 0.25 for MiniGrid, 0.1 for Procgen, 0.1 for ALE-5
    Chosen by grid search over [0.1, 0.25, 0.5, 1.0] per benchmark, as stated in Appendix B.1. This value scales the entire intrinsic reward term and directly affects all experimental results.
  • summation weights w_i = unspecified, presumably equal
    Table 1 defines the summation strategy with per-reward weights w_i, but the paper never states how these weights are set. The reported HIRE-S results depend on this unstated choice, making it an implicit free parameter.
assumptions (4)
  • domain assumption The four intrinsic reward implementations (ICM, NGU, RE3, E3B) from RLLTE and RLEXPLORE are faithful to the originals and mutually comparable.
    All baseline results and hybrid combinations are built on these implementations (Appendix B.1). If any implementation is unfaithful or has hidden configuration issues, the strategy rankings could be artifacts.
  • domain assumption The benchmark environments (MiniGrid, Procgen, ALE-5) are representative settings for evaluating exploration in RL.
    The paper selects eight environments per benchmark and assumes conclusions generalize; this is standard practice but an unverified domain assumption.
  • domain assumption ALE-5 is representative of the 57-game Arcade Learning Environment.
    The paper relies on Aitchison et al. for the claim that ALE-5 median scores are within 10% of full ALE values, and uses ALE-5 to support conclusions about unsupervised RL.
  • standard math Standard RL assumptions: the environment is a Markov decision process and PPO with fixed hyperparameters is an adequate backbone for comparing intrinsic rewards.
    The MDP formulation in Section 3 and the fixed PPO hyperparameters in Table 4 are standard background assumptions of the empirical analysis.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Deep Reinforcement Learning with Hybrid Intrinsic Reward Model." pith.science (2026). https://pith.science/paper/ZZOEOWFP

@misc{pith2026250112627,
  author       = {Pith},
  title        = {Pith review of: Deep Reinforcement Learning with Hybrid Intrinsic Reward Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZZOEOWFP}},
  note         = {Machine review of arXiv:2501.12627}
}
read the original abstract

Intrinsic reward shaping has emerged as a prevalent approach to solving hard-exploration and sparse-rewards environments in reinforcement learning (RL). While single intrinsic rewards, such as curiosity-driven or novelty-based methods, have shown effectiveness, they often limit the diversity and efficiency of exploration. Moreover, the potential and principle of combining multiple intrinsic rewards remains insufficiently explored. To address this gap, we introduce HIRE (Hybrid Intrinsic REward), a flexible and elegant framework for creating hybrid intrinsic rewards through deliberate fusion strategies. With HIRE, we conduct a systematic analysis of the application of hybrid intrinsic rewards in both general and unsupervised RL across multiple benchmarks. Extensive experiments demonstrate that HIRE can significantly enhance exploration efficiency and diversity, as well as skill acquisition in complex and dynamic settings.

Figures

Figures reproduced from arXiv: 2501.12627 by the authors.

Figure 1
Figure 1. The overview of the HIRE framework. (a) Four reward fusion strategies implemented in HIRE. (b) HIRE is designed to be fully [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Screenshots of the experiment environments. (a) From left [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Strategy-level performance comparison on the MiniGrid and Procgen benchmarks. Here, each strategy corresponds to eleven reward [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Aggregated performance ranking of all the reward candidates on the MiniGrid (top) and Procgen (bottom) benchmarks. For [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Cumulative distribution function of the performance from HIRE-1 to HIRE-4 on the MiniGrid (left) and Procgen (right) benchmarks. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Quantity-level performance comparison on the ALE-5 benchmark. Here, each strategy corresponds to four reward candidates. The [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Computational efficiency from HIRE-1 to HIRE-4 on the [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Performance ranking on KeyCorridorS8R5, KeyCorridorS9R6, KeyCorridorS10R7, and MultiRoom-N7-S8. The mean and standard error are computed using five random seeds [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: Performance ranking on MultiRoom-N10-S10, MultiRoom-N12-S10, LockedRoom, and Dynamic-Obstacles-16×16. The mean and standard error are computed using five random seeds [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Performance ranking on CaveFlyer, Chaser, Dodgeball, and Heist. The mean and standard error are computed using five random seeds [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Performance ranking on Jumper, Maze, Miner, and Plunder. The mean and standard error are computed using five random seeds [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 12
Figure 12. Figure 12: Performance ranking on BattleZone, DoubleDunk, NameThisGame, Phoenix, and Q*bert. The mean and standard error are computed using five random seeds [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]
Figure 13
Figure 13. Figure 13: Quantity-level performance comparison on the MiniGrid benchmark. [PITH_FULL_IMAGE:figures/full_fig_p018_13.png]
Figure 14
Figure 14. Figure 14: Quantity-level performance comparison on the Procgen benchmark. [PITH_FULL_IMAGE:figures/full_fig_p018_14.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 37 canonical work pages

  1. [1]

    Deep reinforcement learning at the edge of the statistical precipice

    Rishabh Agarwal, Max Schwarzer, Pablo Samuel Castro, Aaron C Courville, and Marc Bellemare. Deep reinforcement learning at the edge of the statistical precipice. Advances in neural information processing systems , 34:29304--29320, 2021

    work page 2021

  2. [2]

    Atari-5: Distilling the arcade learning environment down to five games

    Matthew Aitchison, Penny Sweetser, and Marcus Hutter. Atari-5: Distilling the arcade learning environment down to five games. In International Conference on Machine Learning , pages 421--438. PMLR, 2023

    work page 2023

  3. [3]

    Existence, relatedness, and growth: Human needs in organizational settings

    Clayton P Alderfer. Existence, relatedness, and growth: Human needs in organizational settings. The Free Press google schola , 2:1--39, 1972

    work page 1972

  4. [4]

    Using confidence bounds for exploitation-exploration trade-offs

    Peter Auer. Using confidence bounds for exploitation-exploration trade-offs. Journal of Machine Learning Research , 3(Nov):397--422, 2002

    work page 2002

  5. [5]

    Never give up: Learning directed exploration strategies

    Adri \`a Puigdom \`e nech Badia, Pablo Sprechmann, Alex Vitvitskyi, Daniel Guo, Bilal Piot, Steven Kapturowski, Olivier Tieleman, Martin Arjovsky, Alexander Pritzel, Andrew Bolt, and Charles Blundell. Never give up: Learning directed exploration strategies. In International Conference on Learning Representations , 2020

    work page 2020

  6. [6]

    The arcade learning environment: An evaluation platform for general agents

    Marc G Bellemare, Yavar Naddaf, Joel Veness, and Michael Bowling. The arcade learning environment: An evaluation platform for general agents. Journal of Artificial Intelligence Research , 47:253--279, 2013

    work page 2013

  7. [7]

    Unifying count-based exploration and intrinsic motivation

    Marc Bellemare, Sriram Srinivasan, Georg Ostrovski, Tom Schaul, David Saxton, and Remi Munos. Unifying count-based exploration and intrinsic motivation. Proceedings of Advances in Neural Information Processing Systems , 29:1471--1479, 2016

    work page 2016

  8. [8]

    A markovian decision process

    Richard Bellman. A markovian decision process. Journal of mathematics and mechanics , pages 679--684, 1957

    work page 1957

Show all 47 references
  1. [9]

    Exploration by random network distillation

    Yuri Burda, Harrison Edwards, Amos Storkey, and Oleg Klimov. Exploration by random network distillation. Proceedings of the 7th International Conference on Learning Representations , pages 1--17, 2019

    2019

  2. [10]

    Explore, discover and learn: Unsupervised discovery of state-covering skills

    V \' ctor Campos, Alexander Trott, Caiming Xiong, Richard Socher, Xavier Gir \'o -i Nieto, and Jordi Torres. Explore, discover and learn: Unsupervised discovery of state-covering skills. In International Conference on Machine Learning , pages 1317--1327. PMLR, 2020

    2020

  3. [11]

    Minigrid & miniworld: Modular & customizable reinforcement learning environments for goal-oriented tasks

    Maxime Chevalier - Boisvert, Bolun Dai, Mark Towers, Rodrigo Perez - Vicente, Lucas Willems, Salem Lahlou, Suman Pal, Pablo Samuel Castro, and Jordan Terry. Minigrid & miniworld: Modular & customizable reinforcement learning environments for goal-oriented tasks. In Advances in...

    2023

  4. [12]

    Leveraging procedural generation to benchmark reinforcement learning

    Karl Cobbe, Chris Hesse, Jacob Hilton, and John Schulman. Leveraging procedural generation to benchmark reinforcement learning. In International conference on machine learning , pages 2048--2056. PMLR, 2020

    2020

  5. [13]

    Stochastic linear optimization under bandit feedback

    Varsha Dani, Thomas P Hayes, and Sham M Kakade. Stochastic linear optimization under bandit feedback. In COLT , volume 2, page 3, 2008

    2008

  6. [14]

    Diversity is all you need: Learning skills without a reward function

    Benjamin Eysenbach, Abhishek Gupta, Julian Ibarz, and Sergey Levine. Diversity is all you need: Learning skills without a reward function. In International Conference on Learning Representations , 2018

    2018

  7. [15]

    Adversarially guided actor-critic

    Yannis Flet-Berliac, Johan Ferret, Olivier Pietquin, Philippe Preux, and Matthieu Geist. Adversarially guided actor-critic. In International Conference on Learning Representations , 2021

    2021

  8. [16]

    Variational intrinsic control

    Karol Gregor, Danilo Jimenez Rezende, and Daan Wierstra. Variational intrinsic control. arXiv preprint arXiv:1611.07507 , 2016

    2016 arXiv

  9. [17]

    Fast task inference with variational intrinsic successor features

    Steven Hansen, Will Dabney, Andre Barreto, David Warde-Farley, Tom Van de Wiele, and Volodymyr Mnih. Fast task inference with variational intrinsic successor features. In International Conference on Learning Representations , 2020

    2020

  10. [18]

    Provably efficient maximum entropy exploration

    Elad Hazan, Sham Kakade, Karan Singh, and Abby Van Soest. Provably efficient maximum entropy exploration. In Proceedings of the International Conference on Machine Learning , pages 2681--2691, 2019

    2019

  11. [19]

    Exploration via elliptical episodic bonuses

    Mikael Henaff, Roberta Raileanu, Minqi Jiang, and Tim Rockt \"a schel. Exploration via elliptical episodic bonuses. Advances in Neural Information Processing Systems , 35:37631--37646, 2022

    2022

  12. [20]

    A study of global and episodic bonuses for exploration in contextual mdps

    Mikael Henaff, Minqi Jiang, and Roberta Raileanu. A study of global and episodic bonuses for exploration in contextual mdps. arXiv preprint arXiv:2306.03236 , 2023

    2023 arXiv

  13. [21]

    Planning and acting in partially observable stochastic domains

    Leslie Pack Kaelbling, Michael L Littman, and Anthony R Cassandra. Planning and acting in partially observable stochastic domains. Artificial intelligence , 101(1-2):99--134, 1998

    1998

  14. [22]

    Curl: Contrastive unsupervised representations for reinforcement learning

    Michael Laskin, Aravind Srinivas, and Pieter Abbeel. Curl: Contrastive unsupervised representations for reinforcement learning. In International conference on machine learning , pages 5639--5650. PMLR, 2020

    2020

  15. [23]

    Cic: Contrastive intrinsic control for unsupervised skill discovery

    Michael Laskin, Hao Liu, Xue Bin Peng, Denis Yarats, Aravind Rajeswaran, and Pieter Abbeel. Cic: Contrastive intrinsic control for unsupervised skill discovery. In Deep RL Workshop NeurIPS 2021 , 2021

    2021

  16. [24]

    Urlb: Unsupervised reinforcement learning benchmark

    Misha Laskin, Denis Yarats, Hao Liu, Kimin Lee, Albert Zhan, Kevin Lu, Catherine Cang, Lerrel Pinto, and Pieter Abbeel. Urlb: Unsupervised reinforcement learning benchmark. In J. Vanschoren and S. Yeung, editors, Proceedings of the Neural Information Processing Systems Track o...

    2021

  17. [25]

    A contextual-bandit approach to personalized news article recommendation

    Lihong Li, Wei Chu, John Langford, and Robert E Schapire. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th international conference on World wide web , pages 661--670, 2010

    2010

  18. [26]

    Aps: Active pretraining with successor features

    Hao Liu and Pieter Abbeel. Aps: Active pretraining with successor features. In International Conference on Machine Learning , pages 6736--6747. PMLR, 2021

    2021

  19. [27]

    Count-based exploration with the successor representation

    Marlos C Machado, Marc G Bellemare, and Michael Bowling. Count-based exploration with the successor representation. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 34, pages 5125--5133, 2020

    2020

  20. [28]

    A dynamic theory of human motivation

    Abraham H Maslow. A dynamic theory of human motivation. 1958

    1958

  21. [29]

    Improving intrinsic exploration with language abstractions

    Jesse Mu, Victor Zhong, Roberta Raileanu, Minqi Jiang, Noah Goodman, Tim Rockt \"a schel, and Edward Grefenstette. Improving intrinsic exploration with language abstractions. Advances in Neural Information Processing Systems , 35:33947--33960, 2022

    2022

  22. [30]

    Count-based exploration with neural density models

    Georg Ostrovski, Marc G Bellemare, A \"a ron Oord, and R \'e mi Munos. Count-based exploration with neural density models. In Proceedings of the International Conference on Machine Learning , pages 2721--2730, 2017

    2017

  23. [31]

    Lipschitz-constrained unsupervised skill discovery

    Seohong Park, Jongwook Choi, Jaekyeom Kim, Honglak Lee, and Gunhee Kim. Lipschitz-constrained unsupervised skill discovery. arXiv preprint arXiv:2202.00914 , 2022

    2022 arXiv

  24. [32]

    Curiosity-driven exploration by self-supervised prediction

    Deepak Pathak, Pulkit Agrawal, Alexei A Efros, and Trevor Darrell. Curiosity-driven exploration by self-supervised prediction. In International conference on machine learning , pages 2778--2787. PMLR, 2017

    2017

  25. [33]

    Self-supervised exploration via disagreement

    Deepak Pathak, Dhiraj Gandhi, and Abhinav Gupta. Self-supervised exploration via disagreement. In International conference on machine learning , pages 5062--5071. PMLR, 2019

    2019

  26. [34]

    Ride: Rewarding impact-driven exploration for procedurally-generated environments

    Roberta Raileanu and Tim Rockt \"a schel. Ride: Rewarding impact-driven exploration for procedurally-generated environments. In International Conference on Learning Representations , 2020

    2020

  27. [35]

    Minihack the planet: A sandbox for open-ended reinforcement learning research

    Mikayel Samvelyan, Robert Kirk, Vitaly Kurin, Jack Parker-Holder, Minqi Jiang, Eric Hambro, Fabio Petroni, Heinrich Kuttler, Edward Grefenstette, and Tim Rockt \"a schel. Minihack the planet: A sandbox for open-ended reinforcement learning research. In Thirty-fifth Conference ...

    2021

  28. [36]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 , 2017

    2017 arXiv

  29. [37]

    State entropy maximization with random encoders for efficient exploration

    Younggyo Seo, Lili Chen, Jinwoo Shin, Honglak Lee, Pieter Abbeel, and Kimin Lee. State entropy maximization with random encoders for efficient exploration. In Proceedings of the 38th International Conference on Machine Learning , pages 9443--9454, 2021

    2021

  30. [38]

    Incentivizing exploration in reinforcement learning with deep predictive models

    Bradly C Stadie, Sergey Levine, and Pieter Abbeel. Incentivizing exploration in reinforcement learning with deep predictive models. arXiv preprint arXiv:1507.00814 , 2015

    2015 arXiv

  31. [39]

    Reinforcement learning: An introduction

    Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction . MIT press, 2018

    2018

  32. [40]

    \# exploration: A study of count-based exploration for deep reinforcement learning

    Haoran Tang, Rein Houthooft, Davis Foote, Adam Stooke, OpenAI Xi Chen, Yan Duan, John Schulman, Filip DeTurck, and Pieter Abbeel. \# exploration: A study of count-based exploration for deep reinforcement learning. Advances in neural information processing systems , 30, 2017

    2017

  33. [41]

    Reinforcement learning with prototypical representations

    Denis Yarats, Rob Fergus, Alessandro Lazaric, and Lerrel Pinto. Reinforcement learning with prototypical representations. In International Conference on Machine Learning , pages 11920--11931. PMLR, 2021

    2021

  34. [42]

    Rewarding episodic visitation discrepancy for exploration in reinforcement learning

    Mingqi Yuan, Bo Li, Xin Jin, and Wenjun Zeng. Rewarding episodic visitation discrepancy for exploration in reinforcement learning. In Deep Reinforcement Learning Workshop NeurIPS 2022 , 2022

    2022

  35. [43]

    R \'e nyi state entropy maximization for exploration acceleration in reinforcement learning

    Mingqi Yuan, Man-On Pun, and Dong Wang. R \'e nyi state entropy maximization for exploration acceleration in reinforcement learning. IEEE Transactions on Artificial Intelligence , 2022

    2022

  36. [44]

    Rlexplore: Accelerating research in intrinsically-motivated reinforcement learning

    Mingqi Yuan, Roger Creus Castanyer, Bo Li, Xin Jin, Glen Berseth, and Wenjun Zeng. Rlexplore: Accelerating research in intrinsically-motivated reinforcement learning. arXiv preprint arXiv:2405.19548 , 2024

    2024 arXiv

  37. [45]

    Rllte: Long-term evolution project of reinforcement learning

    Mingqi Yuan, Zequn Zhang, Yang Xu, Shihao Luo, Bo Li, Xin Jin, and Wenjun Zeng. Rllte: Long-term evolution project of reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence , 2025

    2025

  38. [46]

    Noveld: A simple yet effective exploration criterion

    Tianjun Zhang, Huazhe Xu, Xiaolong Wang, Yi Wu, Kurt Keutzer, Joseph E Gonzalez, and Yuandong Tian. Noveld: A simple yet effective exploration criterion. Advances in Neural Information Processing Systems , 34:25217--25230, 2021

    2021

  39. [47]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence '...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.