Explore Beyond the Boundary Using Entropic Information

Bumgeun Park; Donghwan Lee

REVIEW 5 major objections 4 minor 32 references

Explore Beyond the Boundary Using Entropic Information

T0 review · 5 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Entropy-based intrinsic rewards push RL agents past explored boundaries

desk verdict Entropic boundary exploration is a genuinely new idea with a good matched-novelty comparison, but Theorem 1 is an unquantified approximation and the empirical reporting is too thin to fully trust. read the letter →

arxiv 2607.29419 v1 pith:24FB2JDI submitted 2026-07-31 cs.LG cs.AI

Bumgeun Park , Donghwan Lee This is my paper · ORCID

classification cs.LGcs.AI

keywords reinforcementlearningexplorationintrinsicrewardentropystatenoveltyboundary-basedsparserewardsdelayed

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ENTINEX is a reinforcement-learning exploration method that identifies states near the boundary between explored and unexplored territory and gives the agent an intrinsic reward for being there. Instead of comparing novelty between consecutive states, as earlier boundary methods do, it computes the entropy of a Boltzmann distribution over actions that favor novel transitions: near the boundary, actions split into those that stay inside and those that lead out, making the distribution sharply peaked and its negative entropy large. The paper reports that this entropic signal consistently outperforms established exploration baselines in MuJoCo tasks with sparse or delayed rewards, and that the improvement persists even when the novelty function is matched with the baseline. The broader value is a parameter-light way to push agents into genuinely new regions without reward shaping or dense feedback.

What carries the argument

The novelty action probability distribution (NAPD): a Boltzmann distribution over actions, $\psi(a|s) \propto \exp(\phi'(s,a))$, where $\phi'$ is the expected novelty of the next state. Its sample-based negative log-probability $r_{\mathrm{int}}(s) = \log \psi(a_\psi|s)$ is the intrinsic reward. The NAPD converts the scalar novelty landscape into an action-space distribution whose shape (peaked vs. uniform) marks the boundary; Theorem 1 connects its negative entropy to the normalized consecutive-state novelty difference, while Theorem 2 bounds the discrepancy from sampling actions from a different distribution. For continuous action spaces, the Boltzmann distribution is projected onto a parameterized Gaussian via KL minimization (Eq. 10), and a learned latent dynamics model supplies $\phi'(s,a_\psi)$ for actions the agent never executed.

What would settle it

In a small tabular MDP with known transition probabilities, compute both the negative entropy of the NAPD exactly and the quantity on the right-hand side of Theorem 1 for every state; if the two diverge for states that are not 'near the boundary', or if replacing the entropy reward with the exact normalized novelty difference achieves the same performance, then the entropic computation is not the operative cause.

Watch

Extended reading notes

Core claim

The central claim is that the entropy of the novelty action probability distribution (NAPD) — the Boltzmann distribution over actions that favors actions leading to novel states — encodes whether a state lies on the state-novelty boundary, and that using its negative log-probability as an intrinsic reward steers a policy to cross that boundary. Theorem 1 states that near the boundary the negative entropy of the NAPD is approximately the normalized consecutive-state novelty difference, the same quantity earlier boundary-based methods compute directly from visited transitions. ENTINEX separates the roles: the NAPD tells you which actions would be novel, and the entropy tells you whether that novelty is concentrated or spread out. The paper reports consistent wins over ICM, RND, RE3, and NovelD across sparse-reward and delayed-reward MuJoCo environments, and shows that the entropic boundary estimate, rather than the novelty measure, drives the improvement against NovelD.

Load-bearing premise

Near a state-novelty boundary the novelty action probability distribution is sharply peaked enough that its negative entropy tracks the normalized novelty difference between consecutive states, and the Gaussian projection of Eq. (10) preserves that peak.

Editorial extensions

If this is right

Agents trained with ENTINEX's entropy-based intrinsic reward learn faster and reach higher returns than prediction-error (ICM, RND), state-entropy (RE3), and consecutive-novelty (NovelD) baselines in sparse- and delayed-reward continuous-control tasks.
The boundary signal comes from the entropy of the action distribution, not from the specific novelty estimator, so ENTINEX works with both random-encoder entropic novelty and prediction-error novelty.
Because the intrinsic reward uses actions sampled from the NAPD rather than from the behavior policy, it guides exploration toward novel transitions without requiring the agent to actually take those actions.
Unsupervised pre-training with only ENTINEX's intrinsic rewards creates a policy that, when fine-tuned on task rewards, learns with fewer environment interactions.
When the novelty function is fixed, ENTINEX's entropic boundary estimate still beats NovelD's consecutive-state novelty difference, isolating the boundary-identification mechanism as the source of improvement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If Theorem 1 could be strengthened with an explicit error bound (the paper gives none), the NAPD entropy would become a provable proxy for boundary distance, potentially enabling convergence guarantees for exploration bonuses.
The method's dependence on a learned dynamics model to score never-taken actions suggests a natural stress test: in environments where the latent dynamics model is inaccurate, the intrinsic reward may mis-rank actions and exploration could degrade—an untested edge case.
The entropy-of-induced-distribution idea is portable beyond RL: the same NAPD construction could serve as an acquisition function in active learning or as a curiosity signal in unsupervised skill discovery, wherever one wants to detect decision boundaries from a novelty field.
A simpler variant that replaces the Gaussian projection with a direct entropy estimate (e.g., via sampling) could test whether the projection step, rather than the entropy itself, carries the empirical gains.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know. First, the core idea is genuinely new: instead of measuring boundary proximity through novelty differences between consecutive states, they build a Boltzmann distribution over actions weighted by expected next-state novelty and use its entropy as the exploration signal. That is a real departure from NovelD, BeBold, and RIDE. Second, the theoretical bridge in Theorem 1 is thinner than it looks, and the experimental appendix is too sparse to bank on. That combination, not the idea itself, is what needs scrutiny.

What the paper does well: the matched-novelty comparison against NovelD in Figure 4 is the right move. It isolates the boundary-detection mechanism from the choice of novelty estimator, which is exactly where prior comparisons get muddy. The unsupervised pre-training experiment is a reasonable bonus, and the Discussion is honest that rigorous theoretical characterization remains open.

Now the soft spots, in order of severity.

Theoretical: Theorem 1 states that near the boundary the negative NAPD entropy equals a normalized consecutive-state novelty difference. Algebraically that is the claim log Z ≈ max_a φ'(s,a), with no error bound and no nearness criterion. So the theory does not establish that −H(ψ) tracks boundary proximity; it is a heuristic dressed as a theorem. Worse, the practical method replaces the exact Boltzmann NAPD with a reverse-KL Gaussian projection (Eq. 10), which is mode-seeking: for a multimodal NAPD it can lock onto one mode, and its entropy then reflects local curvature, not the novelty gap. And the implemented reward (Eq. 8) is a single sample of log ψ, whose mean is −H only in expectation. The load-bearing premise—that the Gaussian-projected reward remains monotonically related to boundary distance—is asserted, not proved or measured.

Experimental: all results are learning curves with 5 seeds, no numeric tables, no significance tests, and no code release. One number (SparseWalker2d final returns) is given, but I cannot verify the "consistently outperforms" claim from shaded curves alone. There is also an undefined variable in Algorithm 1: the update buffer is written with a_r while the sampling loop uses a_i. Likely a typo, but it signals the manuscript was not cleaned.

On the circularity worry: the reader's stress test correctly notes that NAPD entropy is a deterministic function of novelty and reduces to NovelD's signal under the Theorem 1 approximation. That is fair. But the implemented method—learned Gaussian projection, dynamics model, sample-based log-density—does behave differently experimentally, and the matched-novelty comparison is exactly the right way to claim a mechanism. So I do not see it as fatal; I see it as an unresolved need to characterize when the entropic signal actually diverges from the novelty-difference signal.

Verdict: this paper deserves a serious referee. The idea is worth engaging, and the experiments are plausible but under-reported. A referee should require code and hyperparameters, numeric tables with variance, and a rewriting of Theorem 1 as a conjecture or with a proven bound. I would send it to peer review, expecting major revision.

Referee Report

5 major / 4 minor

Summary. The paper proposes ENTINEX, an intrinsic-reward exploration method for sparse and delayed-reward RL. The method defines a novelty action probability distribution (NAPD) as a Boltzmann distribution over actions induced by expected consecutive-state novelty, and assigns intrinsic rewards based on the negative entropy of this distribution, intended to identify states near the boundary of the state-novelty distribution. The practical implementation replaces the NAPD with a reverse-KL Gaussian projection, uses a learned latent dynamics model to evaluate the novelty of actions not actually executed, and combines the intrinsic reward with SAC as the base algorithm. Experiments on MuJoCo environments with sparse and delayed rewards compare ENTINEX against ICM, RND, RE3, and NovelD, including a matched-novelty comparison with NovelD, an ablation over novelty functions, and an unsupervised pre-training study. The central theoretical claim is Theorem 1, which asserts an unquantified approximation relating the negative NAPD entropy to a normalized consecutive-state novelty difference near the SND boundary.

Significance. The core idea of using the entropy of a novelty-induced action distribution as a boundary indicator is novel and could provide a useful alternative to direct novelty-difference methods, which rely on particular action distributions. The matched-novelty comparison with NovelD is a well-designed control experiment, and the ablations on novelty functions and unsupervised pre-training broaden the empirical scope. If the empirical improvements are real, the method's practical contribution is meaningful. However, the theoretical support is currently not load-bearing: Theorem 1 is an unquantified approximation, Theorem 2 does not relate entropy to boundary proximity, and the implemented reward is a single-sample log-density whose connection to the theory is only in expectation. The experimental evidence is also reported only as learning curves without numeric tables or significance tests, making the central claim impossible to verify from the manuscript.

major comments (5)

[Section 4.1, Theorem 1 (Eq. 4)] Theorem 1 is the load-bearing link between the proposed entropy reward and boundary proximity, but it is an unquantified approximation. Algebraically, Eq. (4) is equivalent to the statement log Z ≈ max_{a'} φ'(s,a'), with no error bound, no rate of convergence, and no formal definition of "near the SND boundary." The theorem therefore does not establish that −H(ψ(·|s)) reliably tracks the consecutive-state novelty difference or the distance to the exploration boundary. Moreover, the implemented method in Section 4.2 replaces the exact NAPD with a reverse-KL Gaussian projection (Eq. 10); reverse KL is mode-seeking, so for a multimodal Boltzmann distribution the projected Gaussian can concentrate on one mode, and its entropy can be controlled by the local curvature rather than by the novelty gap at the boundary. The actual reward in Eq. (8) is a single-sample log-density whose expectation equals −H only if the projection is exact and whose per-sample variance is unspecified. The paper needs a concrete error bound for Theorem 1 and an empirical or theoretical test of the monotonic relationship between r_int and boundary proximity, for instance a correlation analysis in the sparse-reward environments.
[Section 4.1, Theorem 2 (Eq. 5)] Theorem 2 provides a total-variation bound on the difference between expected novelty differences under two action distributions, but this bound does not involve the entropy of ψ and does not imply that entropy equals or even correlates with novelty differences. The theorem is not used in the derivation of the intrinsic reward or in any experimental analysis, so as written it does not support the paper's central claim. If the intended role of Theorem 2 is to justify replacing one action distribution with another, the bound should be connected to a quantity that actually appears in the algorithm, such as the error introduced by sampling from the Gaussian projection rather than the exact NAPD.
[Section 4.2, Algorithm 1, line 14] The policy update uses the tuple {(s_i, a_r, s'_i, r_total_i)}, but the variable a_r is undefined anywhere in the algorithm or the surrounding text. It appears to be a typo for a_i, the action stored in the replay buffer, but as written the algorithm is not reproducible. Please correct the variable and specify whether the policy update uses the executed action a_i or some other action.
[Section 5.2, Figures 3–6] The central empirical claim of "consistent improvement" over baselines is supported only by learning-curve plots. The manuscript does not provide numerical tables of mean and standard deviation at the final training step or at any intermediate points, and no significance tests or effect sizes are reported. The caption of Fig. 3 mentions 5 random seeds, but the shaded area only shows 0.5σ, which underrepresents the variance. For example, the claim that baselines "fail to learn effectively" in SparseWalker2d is not backed by final scores or error bars, so the reader cannot assess whether the differences are reliable. Please add tables with mean ± std over all seeds for every method and environment, and ideally report the number of seeds and a statistical comparison or confidence intervals.
[Section 5.2, Figure 4] The matched-novelty comparison with NovelD is intended to isolate the effect of the boundary-estimation method, but the description does not specify the exact hyperparameters, network sizes, or the underlying novelty estimator used for both methods in this comparison. Since the default novelty functions are different (RE3 for ENTINEX and RND for NovelD), the claim that any performance gap primarily arises from the entropic approach requires that all other components be tightly controlled. Please provide the precise experimental configuration for Fig. 4 and discuss any remaining differences that could affect the comparison.

minor comments (4)

[Section 2.2, Eq. (3) and Section 4.2, Eq. (10)] The symbol ψ is used for both the exact Boltzmann NAPD in Eq. (3) and the parameterized Gaussian projection in Eq. (10), which is confusing. Using a different symbol for the projected distribution (e.g., ψ_param) would make the derivation of Eq. (11) clearer.
[Section 4.2, Eq. (11)] In the stated loss, the same notation ψ appears inside the expectation and inside the log-density; after the projection, the sampling distribution and the evaluated density should be the same parameterized Gaussian, but the current notation obscures this. Please rewrite the loss using explicit parameterization.
[Section 4.1, State-Novelty Distribution] The term "state-novelty distribution (SND)" is used throughout but never formally defined; the definition of its boundary is also informal. A precise definition would make the premise of Theorem 1 testable and would help readers understand exactly what the boundary is.
[Section 6, Discussion] The discussion mentions "training instability and additional computational overhead" but does not quantify the overhead of the dynamics model and the additional components relative to the baselines. A brief cost comparison (wall-clock time or parameter count) would strengthen the practical assessment.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the derivation chain is self-contained and the central empirical claim is benchmarked against external methods.

full rationale

ENTINEX defines the novelty action probability distribution from the novelty function (Eq. 3) and computes the intrinsic reward as a sample log-density (Eq. 8); there is no fitted parameter that is later renamed as a prediction. Theorem 1 is a stated approximation (log-sum-exp ≈ max) that relates negative NAPD entropy to a normalized novelty difference, but the paper uses an explicit approximate equality, and its practical reward is not asserted to equal that difference exactly. The matched-novelty comparison against NovelD (Fig. 4) is an external control that prevents the improvement from being attributed solely to the shared novelty measure. No load-bearing uniqueness theorem or self-citation chain is invoked; the only citations to prior work are for standard components (RE3 encoder, latent dynamics, SAC). The unquantified near-boundary assumption in Theorem 1 is a robustness/correctness concern about whether the Gaussian-projected reward tracks boundary proximity, but it is not a reduction of the result to its own inputs. Therefore the paper is not circular.

Assumptions & free parameters 1 free parameters · 4 assumptions · 1 invented entities

The method introduces one new construct (NAPD) and depends on a peaked-action premise, an unquantified log-sum-exp approximation, an accurate learned dynamics model, and the quality of the RE3 novelty function. The intrinsic reward weight alpha is an unreported free parameter.

free parameters (1)

Intrinsic reward weight alpha = not reported
Controls how much the intrinsic reward contributes to the total reward in Eq. (9). No value or sensitivity analysis is given, so the reported results may depend on per-environment tuning.

assumptions (4)

ad hoc to paper Near the SND boundary, the NAPD is sharply peaked because some actions yield much higher expected novelty than others.
Central premise for using negative NAPD entropy as a boundary indicator; asserted in Section 4.1 without formal proof.
domain assumption The log-sum-exp approximation log Z(s) approximately equals max_a phi'(s,a) is accurate enough for Theorem 1 (Eq. 4) to hold.
Used implicitly in Theorem 1; no error bound is provided.
domain assumption The learned dynamics model P_hat predicts next states accurately enough to compute phi'(s,a_psi) for actions not executed in the environment.
The NAPD update in Eq. (11) depends on dynamics model predictions; errors are not bounded.
domain assumption The RE3 random encoder provides a valid state novelty estimate phi(s).
Borrowed from RE3 as a fixed novelty function; the method inherits its strengths and biases.

invented entities (1)

Novelty Action Probability Distribution (NAPD)
purpose: A Boltzmann distribution over actions induced by expected novelty, whose entropy is used as the intrinsic reward.
An internal construct defined from the novelty function; no external falsifiable prediction is attached.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Explore Beyond the Boundary Using Entropic Information." pith.science (2026). https://pith.science/paper/24FB2JDI

@misc{pith2026260729419,
  author       = {Pith},
  title        = {Pith review of: Explore Beyond the Boundary Using Entropic Information},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/24FB2JDI}},
  note         = {Machine review of arXiv:2607.29419}
}

read the original abstract

In reinforcement learning, exploration with sparse and delayed rewards presents a significant challenge due to the limited feedback available for guiding the learning process. Addressing this issue requires extensive exploration in the state space to discover valuable reward signals. In this paper, we propose Entropic Information for Exploration (ENTINEX), a novel method that enhances exploration by incentivizing agents to explore beyond the boundaries of the state distribution. ENTINEX achieves this by assigning intrinsic rewards to these boundaries, leveraging entropic information to identify them effectively. Through extensive experimentation, we demonstrate that ENTINEX consistently improves exploration performance in environments characterized by sparse and delayed rewards. Our experimental results show that ENTINEX outperforms existing exploration methods, highlighting its effectiveness in both sparse and delayed reward scenarios.

Figures

Figures reproduced from arXiv: 2607.29419 by the authors.

**Figure 2.** Illustration of SND boundary and NAPD. The SND boundary represents [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Performance on MuJoCo environments with sparse and delayed reward. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Performance comparison with another boundary-based method, NovelD, [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

**Figure 5.** Figure 5: Performance evaluation of ENTINEX with two different novelty functions. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

**Figure 6.** Figure 6: Performance evaluation to validate the effectiveness of unsupervised pre [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 20 canonical work pages

[1]

Advances in neural in- formation processing systems29(2016)

Bellemare, M., Srinivasan, S., Ostrovski, G., Schaul, T., Saxton, D., Munos, R.: Unifying count-based exploration and intrinsic motivation. Advances in neural in- formation processing systems29(2016)

work page 2016
[2]

arXiv preprint arXiv:1606.01540 (2016)

Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., Zaremba, W.: Openai gym. arXiv preprint arXiv:1606.01540 (2016)

arXiv 2016
[3]

arXiv preprint arXiv:1810.12894 (2018)

Burda, Y., Edwards, H., Storkey, A., Klimov, O.: Exploration by random network distillation. arXiv preprint arXiv:1810.12894 (2018)

arXiv 2018
[4]

Au- tonomous Robots42(2), 235–256 (2018)

Carrillo, H., Dames, P., Kumar, V., Castellanos, J.A.: Autonomous robotic ex- ploration using a utility function based on rényi’s general theory of entropy. Au- tonomous Robots42(2), 235–256 (2018)

work page 2018
[5]

Dai, Y., Wang, G., Muhammad, K., Liu, S.: A closed-loop healthcare processing approachbasedondeepreinforcementlearning.MultimediaToolsandApplications 81(3), 3107–3129 (2022)

work page 2022
[6]

Machine learning8(3), 279–292 (1992)

Dayan, P., Watkins, C.: Q-learning. Machine learning8(3), 279–292 (1992)

work page 1992
[7]

In: International conference on machine learning

Florensa, C., Held, D., Geng, X., Abbeel, P.: Automatic goal generation for rein- forcement learning agents. In: International conference on machine learning. pp. 1515–1528. PMLR (2018)

work page 2018
[8]

In: International Conference on Machine Learning

Fu, Y., Peng, R., Lee, H.: Go beyond imagination: maximizing episodic reachability with world models. In: International Conference on Machine Learning. pp. 10405– 10420. PMLR (2023)

work page 2023

Show all 32 references

[9]

In: International conference on machine learning

Haarnoja, T., Zhou, A., Abbeel, P., Levine, S.: Soft actor-critic: Off-policy maxi- mum entropy deep reinforcement learning with a stochastic actor. In: International conference on machine learning. pp. 1861–1870. PMLR (2018)

2018
[10]

PMLR (2022)

Han, B., Ren, Z., Wu, Z., Zhou, Y., Peng, J.: Off-policy reinforcement learning with delayedrewards.In:InternationalConferenceonMachineLearning.pp.8280–8303. PMLR (2022)

2022
[11]

Ad- vances in Neural Information Processing Systems34, 25732–25745 (2021)

Han, S., Sung, Y.: A max-min entropy framework for reinforcement learning. Ad- vances in Neural Information Processing Systems34, 25732–25745 (2021)

2021
[12]

arXiv preprint arXiv:2203.04955 (2022)

Hansen, N., Wang, X., Su, H.: Temporal difference learning for model predictive control. arXiv preprint arXiv:2203.04955 (2022)

2022 arXiv
[13]

Advances in neural information processing systems31(2018)

Hong, Z.W., Shann, T.Y., Su, S.Y., Chang, Y.H., Fu, T.J., Lee, C.Y.: Diversity- driven exploration strategy for deep reinforcement learning. Advances in neural information processing systems31(2018)

2018
[14]

In: 2019 international conference on robotics and automation (ICRA)

Johannink, T., Bahl, S., Nair, A., Luo, J., Kumar, A., Loskyll, M., Ojea, J.A., Solowjow, E., Levine, S.: Residual reinforcement learning for robot control. In: 2019 international conference on robotics and automation (ICRA). pp. 6023–6029. IEEE (2019)

2019
[15]

In: 2009 ieee conference on computer vision and pattern recognition

Joshi, A.J., Porikli, F., Papanikolopoulos, N.: Multi-class active learning for image classification. In: 2009 ieee conference on computer vision and pattern recognition. pp. 2372–2379. IEEE (2009)

2009
[16]

Artificial Intelligence316, 103829 (2023) 12 B

Knox, W.B., Allievi, A., Banzhaf, H., Schmitt, F., Stone, P.: Reward (mis) design for autonomous driving. Artificial Intelligence316, 103829 (2023) 12 B. Park and D. Lee

2023
[17]

In: Proceed- ings of the 26th annual international conference on machine learning

Kolter, J.Z., Ng, A.Y.: Near-bayesian exploration in polynomial time. In: Proceed- ings of the 26th annual international conference on machine learning. pp. 513–520 (2009)

2009
[18]

IEEE/CAA Journal of Automatica Sinica3(3), 247–254 (2016)

Li, L., Lv, Y., Wang, F.Y.: Traffic signal timing via deep reinforcement learning. IEEE/CAA Journal of Automatica Sinica3(3), 247–254 (2016)

2016
[19]

In: Distributed Ar- tificial Intelligence: Second International Conference, DAI 2020, Nanjing, China, October 24–27, 2020, Proceedings 2

Li, Z., Chen, X.H.: Efficient exploration by novelty-pursuit. In: Distributed Ar- tificial Intelligence: Second International Conference, DAI 2020, Nanjing, China, October 24–27, 2020, Proceedings 2. pp. 85–102. Springer (2020)

2020
[20]

arXiv preprint arXiv:1509.02971 (2015)

Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., Wierstra, D.: Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971 (2015)

2015 arXiv
[21]

In: Conference on Robot Learning

Mazoure, B., Doan, T., Durand, A., Pineau, J., Hjelm, R.D.: Leveraging explo- ration in off-policy algorithms via normalizing flows. In: Conference on Robot Learning. pp. 430–444. PMLR (2020)

2020
[22]

arXiv preprint arXiv:1312.5602 (2013)

Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M.: Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013)

2013 arXiv
[23]

Advances in neural information processing sys- tems35, 27730–27744 (2022)

Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al.: Training language models to follow instructions with human feedback. Advances in neural information processing sys- tems35, 27730–27744 (2022)

2022
[24]

In: International conference on machine learning

Pathak, D., Agrawal, P., Efros, A.A., Darrell, T.: Curiosity-driven exploration by self-supervised prediction. In: International conference on machine learning. pp. 2778–2787. PMLR (2017)

2017
[25]

arXiv preprint arXiv:2002.12292 (2020)

Raileanu, R., Rocktäschel, T.: Ride: Rewarding impact-driven exploration for procedurally-generated environments. arXiv preprint arXiv:2002.12292 (2020)

2020 arXiv
[26]

arXiv preprint arXiv:1707.05246 (2017)

Ruder, S., Plank, B.: Learning to select data for transfer learning with bayesian optimization. arXiv preprint arXiv:1707.05246 (2017)

2017 arXiv
[27]

In: International Conference on Machine Learning

Seo, Y., Chen, L., Shin, J., Lee, H., Abbeel, P., Lee, K.: State entropy maximization with random encoders for efficient exploration. In: International Conference on Machine Learning. pp. 9443–9454. PMLR (2021)

2021
[28]

Advances in neural information processing systems 30(2017)

Tang, H., Houthooft, R., Foote, D., Stooke, A., Xi Chen, O., Duan, Y., Schulman, J., DeTurck, F., Abbeel, P.: # exploration: A study of count-based exploration for deep reinforcement learning. Advances in neural information processing systems 30(2017)

2017
[29]

In: 2012 IEEE/RSJ international conference on intelligent robots and systems

Todorov, E., Erez, T., Tassa, Y.: Mujoco: A physics engine for model-based control. In: 2012 IEEE/RSJ international conference on intelligent robots and systems. pp. 5026–5033. IEEE (2012)

2012
[30]

In: Proceedings of the AAAI conference on artificial intelligence

Van Hasselt, H., Guez, A., Silver, D.: Deep reinforcement learning with double q- learning. In: Proceedings of the AAAI conference on artificial intelligence. vol. 30 (2016)

2016
[31]

arXiv preprint arXiv:2012.08621 (2020)

Zhang, T., Xu, H., Wang, X., Wu, Y., Keutzer, K., Gonzalez, J.E., Tian, Y.: Bebold: Exploration beyond the boundary of explored regions. arXiv preprint arXiv:2012.08621 (2020)

2020 arXiv
[32]

Advances in Neural Information Processing Systems34, 25217–25230 (2021)

Zhang, T., Xu, H., Wang, X., Wu, Y., Keutzer, K., Gonzalez, J.E., Tian, Y.: Nov- eld: A simple yet effective exploration criterion. Advances in Neural Information Processing Systems34, 25217–25230 (2021)

2021

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Advances in neural in- formation processing systems29(2016)

Bellemare, M., Srinivasan, S., Ostrovski, G., Schaul, T., Saxton, D., Munos, R.: Unifying count-based exploration and intrinsic motivation. Advances in neural in- formation processing systems29(2016)

work page 2016

[2] [2]

arXiv preprint arXiv:1606.01540 (2016)

Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., Zaremba, W.: Openai gym. arXiv preprint arXiv:1606.01540 (2016)

arXiv 2016

[3] [3]

arXiv preprint arXiv:1810.12894 (2018)

Burda, Y., Edwards, H., Storkey, A., Klimov, O.: Exploration by random network distillation. arXiv preprint arXiv:1810.12894 (2018)

arXiv 2018

[4] [4]

Au- tonomous Robots42(2), 235–256 (2018)

Carrillo, H., Dames, P., Kumar, V., Castellanos, J.A.: Autonomous robotic ex- ploration using a utility function based on rényi’s general theory of entropy. Au- tonomous Robots42(2), 235–256 (2018)

work page 2018

[5] [5]

Dai, Y., Wang, G., Muhammad, K., Liu, S.: A closed-loop healthcare processing approachbasedondeepreinforcementlearning.MultimediaToolsandApplications 81(3), 3107–3129 (2022)

work page 2022

[6] [6]

Machine learning8(3), 279–292 (1992)

Dayan, P., Watkins, C.: Q-learning. Machine learning8(3), 279–292 (1992)

work page 1992

[7] [7]

In: International conference on machine learning

Florensa, C., Held, D., Geng, X., Abbeel, P.: Automatic goal generation for rein- forcement learning agents. In: International conference on machine learning. pp. 1515–1528. PMLR (2018)

work page 2018

[8] [8]

In: International Conference on Machine Learning

Fu, Y., Peng, R., Lee, H.: Go beyond imagination: maximizing episodic reachability with world models. In: International Conference on Machine Learning. pp. 10405– 10420. PMLR (2023)

work page 2023