Pith. sign in

REVIEW 4 major objections 6 minor 3 cited by

Reinforcement Learning via Implicit Imitation Guidance

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper argues that expert demonstrations accelerate online reinforcement learning most effectively when they shape the agent's exploration noise, not its policy objective, and reports up to 2-3x gains over offline-data RL baselines on…

desk verdict A clean exploration-prior idea with solid empirical gains, but the missing shuffled-data control leaves the central mechanism underdetermined. read the letter →

arxiv 2506.07505 v1 pith:OHNZ2SPD submitted 2025-06-09 cs.LG cs.AI

Perry Dong , Alec M. Lessing , Annie S. Chen , Chelsea Finn This is my paper
classification cs.LGcs.AI
keywords reinforcementlearningimitationexplorationsampleefficiencydemonstrationdatacontinuouscontrolsparserewardsdata-guidednoise
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that demonstrations are most valuable in deep RL as a guide for exploration, not as a target for imitation. It proposes Data-Guided Noise (DGN), which learns a state-dependent Gaussian noise distribution from the difference between demonstrated actions and the agent's current policy actions, and adds this noise to actions during rollouts. The claim is that this implicit imitation signal accelerates learning in sparse-reward continuous control, achieving up to 2-3x improvement over prior offline-to-online RL methods across seven simulated tasks, and matching or beating existing demonstration-augmented approaches. If right, it means imitation guidance can be injected without imitation losses, reference policies, or behavior-cloning constraints, avoiding the long-term performance degradation those methods suffer.

What carries the argument

The load-bearing object is the learned, state-dependent covariance matrix $\Sigma_\phi(s) = A_\phi(s) A_\phi(s)^T$, produced by an MLP that outputs the Cholesky factor. It is trained by minimizing the negative log-likelihood of the sampling distribution $\mathcal{N}(\mu_\theta(s), \Sigma_\phi(s))$ over the demonstration states and actions, and the parameters are re-fit periodically against the latest RL policy. This covariance converts 'actions that worked in the data' into a direction and scale of exploration, while the policy mean continues to be learned purely by RL.

What would settle it

Measure DGN's return with the learned covariance evaluated only on states inside the demonstration-state support, for example by zeroing the noise whenever the current state is far from all demonstration states in nearest-neighbor distance. If the gains over RLPD vanish, the benefit depends on the covariance transferring across distribution shift; if they persist, the mechanism is robust to off-demonstration states.

Watch

Extended reading notes

Core claim

The central claim is that prior data such as demonstrations should be used to set the variance of the agent's action distribution rather than its mean. DGN learns a state-conditioned covariance matrix by fitting a Gaussian to the difference between demonstrated actions and the current RL policy's mean actions on demonstration states, then samples exploratory actions from that Gaussian. The RL update remains untouched, so the policy is free to exceed the demonstrations, and the noise is annealed or shut off once the policy reaches a success threshold. Empirically, DGN matches or outperforms existing demonstration-augmented methods on seven tasks and reports up to 2-3x improvement over offline-data RL methods, with the largest gap on the hardest tasks.

Load-bearing premise

The central bet is that the covariance fit on demonstration states tells the agent which exploratory directions are useful even at states the demonstrations never visited, and that this guidance remains valid as the policy's own mean moves.

Editorial extensions

If this is right

  • DGN can be dropped into an existing online RL loop with only a covariance-fitting network added, so imitation guidance no longer requires pretraining a separate imitation policy or tuning an imitation-loss weight.
  • Because the noise is annealed or shut off once the policy reaches a success threshold, DGN is designed to avoid the final-performance ceiling that behavior-cloning regularization can impose.
  • On tasks where a reference-policy method like IBRL fails because the imitation policy is weak or the demonstrations are multimodal, DGN retains most of its benefit.
  • State-conditioning the noise is load-bearing: replacing the covariance with a single global matrix substantially hurts performance.
  • A full residual-policy variant performs about as well as the zero-mean covariance variant, so the benefit comes from the structure of the exploration distribution, not from the learned mean.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the mechanism is noise shaping, DGN should also accelerate on-policy algorithms or model-based RL, since it never touches the policy update; testing this would sharpen the claim that the benefit is purely exploratory.
  • The paper's annealing and shutdown heuristics could be replaced by a distribution-shift diagnostic that measures how far current policy actions are from demonstrated actions and scales the noise accordingly, making the method less sensitive to schedule hyperparameters.
  • The covariance-fitting objective is a local Gaussian approximation; a non-Gaussian or diffusion-based noise model could capture multimodal demonstration structure, which the paper notes as future work but does not test.
  • The KL-divergence result suggests the right design principle is to let the policy drift away from demonstrations as it improves, meaning methods that maintain a fixed imitation budget may be structurally limited.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Data-Guided Noise (DGN), a method for using expert demonstrations in online RL without behavior-cloning losses or reference IL policies. DGN keeps the RL policy mean μθ(s) as the action mean and learns a state-dependent covariance Σφ(s) by maximum-likelihood fitting of a Gaussian N(μθ(s), Σφ(s)) to residuals between dataset actions and current policy actions at dataset states. During rollouts, actions are sampled from this distribution; an annealing schedule (Adroit) or a success-rate-based shutdown (Robomimic) tapers the noise. Experiments compare DGN to RLPD, RFT, IQL, and IBRL on seven Adroit/Robomimic sparse-reward tasks, with ablations over the residual mean, state conditioning, number of demonstrations, and MLP size. The main reported findings are up to 2–3× improvement over RLPD on the hardest tasks and parity or better performance than IBRL when IBRL has a strong BC policy, with greater robustness when the BC policy is weak or the dataset is multimodal.

Significance. The contribution is potentially useful: if demonstrations can be converted into a state-dependent exploration covariance and used only as sampling noise, DGN would be a simple, low-overhead recipe that sidesteps the known pitfalls of imitation regularization and reference-policy switching. The paper ships a reasonably complete empirical study: seven environments, three seeds with standard error, comparisons against several baselines, and ablations isolating the residual mean, state conditioning, demonstration count, and model capacity. These are real strengths. However, the causal claim that demonstrations identify what to explore is not yet established, because no control removes the content of the demonstrations while keeping the state-dependent noise structure. The IBRL comparison also covers only two of seven tasks, so the 'matches or outperforms state-of-the-art demonstration-based methods' claim is broader than the evidence. The absence of code and data further limits verification. If the missing control and broader comparison are supplied, the contribution would be a solid empirical advance.

major comments (4)
  1. [§4.1, Eq. (1); §5.3, Figs. 7–8] The central claim that demonstrations tell the agent 'which actions to explore' is not supported without a control that removes the demonstration content from the learned covariance. In the zero-mean instantiation, expert information enters only through Σφ(s), and the ablation in Fig. 7 shows that adding the learned mean µφ(s) does not improve over covariance-only DGN. The state-conditioning ablation (Fig. 8) demonstrates that state-dependent variance helps, but it does not show that the variance must be learned from demonstrations. I request a control in which the same state-dependent covariance MLP is fit to random or permuted state-action pairs (or to a non-expert dataset) and used with the same update and annealing schedule. If that control matches DGN, the 2–3× gains over RLPD reported in Fig. 4 would be attributable to added state-dependent exploration noise rather than to implicit imitation guidance.
  2. [§4.1, Eq. (1); §4.2] The covariance is fit at demonstration states but used at all states visited during online rollouts; the paper does not validate that the expert-residual directions transfer under this distribution shift. The residuals (a − μθ(s)) at dataset states change as μθ is updated, and Σφ is refreshed only every N environment steps. The annealing and shutdown mechanisms reduce the magnitude of the noise but do not address whether its direction remains informative on off-demonstration states. Please provide a quantitative diagnosis—for instance, compare the learned noise scale/direction on demonstration states versus states visited by the online policy, or retrain the covariance on online states—to justify the transfer assumption.
  3. [§5.2, Figs. 5–6] The abstract claims DGN 'matches or outperforms existing state-of-the-art approaches that use demonstration data,' but the strongest reference-policy baseline, IBRL, is evaluated only on Can and Square. On the other five tasks the comparison set is limited to RLPD, RFT, and IQL. Please extend the IBRL comparison to the remaining tasks, or restrict the claim to the tasks where it was tested.
  4. [§5, Table 2] The exploration schedule is environment-specific (τ = 30000 annealing on Adroit; shutdown at m = 0.5 over n = 10 episodes on Robomimic), and DGN introduces several additional hyperparameters (update interval N, dropout, warm-up episodes). No sensitivity analysis is reported for τ, m, n, or N. Since the method's benefit over RLPD may depend on these choices, a coarse sweep on at least one Adroit and one Robomimic task, or a robustness discussion, is needed to establish that the reported gains are not an artifact of the selected schedule.
minor comments (6)
  1. [§3, Eq. (1)] The Gaussian notation is missing a closing parenthesis in 'N (µθ(s), Σϕ(s)', and the objective in Algorithm 1 would benefit from an explicit 'with respect to φ'.
  2. [Figure 4] 'Normalized Returns' is not defined; please state the normalization (e.g., success rate or return relative to expert).
  3. [Appendix A] 'Soft-IBRL' with β = 10 is not described; please provide the definition or citation.
  4. [Table 2] The ' . . . ' entries should be replaced by '—' or 'N/A' for readability.
  5. [References] 'V olodymyr' in the Riedmiller et al. entry is a typo; also the paper would benefit from a code/data availability statement.
  6. [Table 4] 'Warm-Up Episodes' is not described in the algorithm or text; please clarify whether these are random episodes before DGN noise is active.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DGN's covariance fit (Eq. 1) is an algorithmic mechanism; performance claims are empirical against external baselines, not derived from the fitted values.

full rationale

All load-bearing claims are empirical, not derived from fitted constants. Equation (1) is a maximum-likelihood training objective for the covariance Sigma_phi(s) on demonstration residuals; Algorithm 1 then uses the resulting sampling distribution during rollouts. This is the algorithm's intended mechanism, not a prediction that reduces to the fit. The paper does not rename a fit as a prediction: performance is measured against RLPD, RFT, IQL, and IBRL on seven benchmarks, and conclusions are based on normalized return curves and ablations. The residual-mean ablation shows covariance-only DGN and full-residual DGN perform comparably, which is an empirical finding about which mechanism matters, not a definitional equivalence. Self-citations (Mark et al. 2023, Nakamoto et al. 2023) occur only in related-work context and are not load-bearing for the DGN derivation. The skeptical concern about a missing control (e.g., covariance learned from random data) is a correctness/experimental-design issue, not a circularity issue: no equation or fitted parameter is asserted to be its own prediction.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities. Its central claim rests on the standard MDP/Gaussian-policy setup, the availability of successful demonstrations, and a specific ad hoc modeling choice for exploration noise. The free parameters are hyperparameters of the DGN update and annealing schedule, which are tuned per benchmark and affect the reported results.

free parameters (5)
  • DGN update interval N = 1000 (Robomimic), 2000 (Adroit)
    Frequency of refitting the covariance matrix; chosen per benchmark and reported in Table 2.
  • Annealing timescale τ = 30000 (Adroit only)
    Controls exponential decay of exploration noise; not used for Robomimic, reflecting per-domain tuning.
  • Noise shutoff success threshold m and window n = m=0.5, n=10 (Robomimic only)
    Turns off guided noise once the policy reaches 50% success; a heuristic that requires tuning.
  • Dropout = 0.5 (only in IBRL comparison section)
    Applied to the actor for comparison consistency with IBRL; noted as an orthogonal design choice.
  • Warm-up episodes = 20/40/50/50/0/0/0 per task
    Number of episodes before online RL begins; task-specific and potentially influential for early learning.
assumptions (4)
  • standard math MDP with Gaussian policy parameterization N(µθ(s), Σ)
    Preliminaries Section 3: the policy class is restricted to Gaussian distributions, standard in continuous control.
  • domain assumption Demonstration data consists of successful behaviors relevant to the task
    The method assumes prior data such as expert demonstrations provide useful exploratory directions; stated in Section 1.
  • domain assumption Sparse binary reward is sufficient signal for RL
    All environments use sparse binary rewards to indicate task completion; described in Section 5.
  • ad hoc to paper The residual (a_demo - µθ(s)) is a useful exploration direction
    Core mechanism formalized in Eq. 1; introduced as a design choice without first-principles justification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Reinforcement Learning via Implicit Imitation Guidance." pith.science (2026). https://pith.science/paper/OHNZ2SPD

@misc{pith2026250607505,
  author       = {Pith},
  title        = {Pith review of: Reinforcement Learning via Implicit Imitation Guidance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OHNZ2SPD}},
  note         = {Machine review of arXiv:2506.07505}
}
read the original abstract

We study the problem of sample efficient reinforcement learning, where prior data such as demonstrations are provided for initialization in lieu of a dense reward signal. A natural approach is to incorporate an imitation learning objective, either as regularization during training or to acquire a reference policy. However, imitation learning objectives can ultimately degrade long-term performance, as it does not directly align with reward maximization. In this work, we propose to use prior data solely for guiding exploration via noise added to the policy, sidestepping the need for explicit behavior cloning constraints. The key insight in our framework, Data-Guided Noise (DGN), is that demonstrations are most useful for identifying which actions should be explored, rather than forcing the policy to take certain actions. Our approach achieves up to 2-3x improvement over prior reinforcement learning from offline data methods across seven simulated continuous control tasks.

Figures

Figures reproduced from arXiv: 2506.07505 by the authors.

Figure 1
Figure 1. Data-Guided Noise (DGN). We propose to guide exploration by learning a state-conditioned noise distribution that uses the difference between expert actions and the current RL policy to provide implicit imitation signals for exploration. directions in action space that have led to successful outcomes. As a practical instantiation of DGN, we train a state-dependent covariance matrix on dataset-policy action difference… view at source ↗
Figure 2
Figure 2. Behavior of Online RL with Expert Data. Prior work has proposed several strategies for sparse￾reward RL that leverage expert data. Initializing the replay buffer with expert data does not directly use the expert information to maximally accelerate learning. In IL-regularized RL, the agent is constrained to mimic expert actions, which may limit the agent from finding more optimal solutions. IL + RL frameworks that us… view at source ↗
Figure 3
Figure 3. Visualizations of the seven environments on which we evaluate [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Average Normalized Returns. for Robomimic and Adroit tasks comparing with standard uncon￾strained RL and imitation-regularized RL methods. Across all tasks, DGN consistently exceeds or matches the performance of the best baseline—even as the best baseline method varies…
Figure 5
Figure 5. Figure 5: Comparison to IBRL. Average normalized returns on can and square, comparing to the reference policy-based approach of IBRL. IBRL’s performance strongly depends on having a well-trained IL policy, and its performance can degrade substantially without it, while DGN’s doe…
Figure 6
Figure 6. Figure 6: Comparison to IBRL with a multi￾modal dataset. DGN outperforms IBRL, showing DGN is less sensitive to dataset quality and multi￾modality. To better understand the importance of different components of DGN for policy performance, we ablate over two key components that c…
Figure 7
Figure 7. Figure 7: Ablation on Learning Full Residual Policy via Imitation Learning. Learning a full residual policy via imitation performs similarly to only learning the covariance. 8 [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Ablation on State-Conditioning DGN Distribution. The ablation of DGN without state-conditioning of the learned covariance matrix performs worse than DGN on Robomimic tasks. tool hang, where vanilla RLPD outperforms the ablation with no state conditioning, indicating th…
Figure 9
Figure 9. Figure 9: KL Divergence from BC Policy over Training. We plot the KL divergence between each method’s policy and a BC policy trained on expert demonstrations, evaluated on a fixed set of demon￾stration states. All methods initially reduce their di￾vergence, reflecting early-stag…
Figure 10
Figure 10. Figure 10: Ablation over Number of Demos. We evaluate how changing the number of demonstrations changes the performance of DGN and vanilla RLPD on the Lift and Square tasks. Adding more demos generally increases the performance of both DGN and RLPD, with DGN matching or outperfo…
Figure 11
Figure 11. Figure 11: Ablation over the hidden size of the MLP of the state-dependent covariance model for the Lift and Square tasks. The performance on both tasks is strong across all tested model sizes, though slightly worse for larger MLP sizes, possibly indicating overfitting. 14 [PIT…

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Do You Really Need to Pretrain Q-Functions for Online RL Fine-Tuning?

    cs.LG 2026-07 conditional novelty 6.0 of 10

    On six robot-manipulation tasks, offline Q-pretraining does not accelerate online RL fine-tuning from a pretrained policy, while seeding the replay buffer with rollouts from an ensemble of policies (IPE) improves fina...

  2. Incremental Residual Reinforcement Learning Toward Real-World Learning for Social Navigation

    cs.RO 2026-04 unverdicted novelty 5.5 of 10

    IRRL lets robots learn social navigation in the real world by incrementally updating only the differences from a base policy, matching replay-buffer methods in simulation and adapting to new settings on physical robots.

  3. Value Flows

    cs.LG 2025-10 reject novelty 5.0 of 10

    Value Flows fits the full return distribution in RL with a flow-matching critic and reweights its learning objective by estimated return variance; the central theoretical guarantee does not follow from the stated equations.

Reference graph

Works this paper leans on

23 extracted references · 7 canonical work pages · cited by 3 Pith papers

  1. [1]

    Efficient online reinforcement learning with offline data

    Philip J Ball, Laura Smith, Ilya Kostrikov, and Sergey Levine. Efficient online reinforcement learning with offline data. arXiv preprint arXiv:2302.02948,

    arXiv

  2. [2]

    epochs per update

    The “epochs per update" hyperparameter is the number of epochs for which the DGN learned covariance matrix is trained per DGN update. For the IBRL baselines, we use the same hyperparameters as in the original IBRL paper [Hu et al., 2023] for the state-based Robomimic tasks. In particular we use dropout of 0.5 for the actor and use the “Soft-IBRL" variant with β =

    work page 2023

  3. [4]

    worse", which are successful demonstrations collected by inexperienced operators to incorporate additional diversity. Note that even though it is labeled

    Hyperparameter Robomimic Adroit DGN Update Interval (N) 1000 2000 Optimizer AdamW MLP Hidden Layers 2 Dropout 0.5 Batch Size 128 MLP Hidden Size 128 256 Weight Decay 3e-2 Epochs Per Update 2 10 Annealing Timescale (τ) . . . 30000 Shutoff Success Rate Threshold (m) 0.5 . . . Epochs to Measure Success Rate for Shutoff (n) 10 . . . Table 2: DGN Hyperparamete...

    work page 2000

  4. [5]

    Imitation bootstrapped reinforcement learning

    Hengyuan Hu, Suvir Mirchandani, and Dorsa Sadigh. Imitation bootstrapped reinforcement learning. arXiv preprint arXiv:2311.02198,

    arXiv

  5. [6]

    Offline reinforcement learning with implicit q-learning

    Ilya Kostrikov, Ashvin Nair, and Sergey Levine. Offline reinforcement learning with implicit q-learning. arXiv preprint arXiv:2110.06169,

    arXiv

  6. [8]

    Offline retraining for online rl: Decoupled policy learning to mitigate exploration bias

    Max Sobol Mark, Archit Sharma, Fahim Tajwar, Rafael Rafailov, Sergey Levine, and Chelsea Finn. Offline retraining for online rl: Decoupled policy learning to mitigate exploration bias. arXiv preprint arXiv:2310.08558,

    arXiv

  7. [9]

    Over- coming exploration in reinforcement learning with demonstrations

    Ashvin Nair, Bob McGrew, Marcin Andrychowicz, Wojciech Zaremba, and Pieter Abbeel. Over- coming exploration in reinforcement learning with demonstrations. In 2018 IEEE international conference on robotics and automation (ICRA), pages 6292–6299. IEEE,

    work page 2018

  8. [11]

    Computational Theories of Curiosity-Driven Learning

    10 Pierre-Yves Oudeyer. Computational theories of curiosity-driven learning. arXiv:1802.10546,

    work page Pith review arXiv

Show all 23 references
  1. [14]

    Learning complex dexterous manipulation with deep reinforcement learning and demonstrations

    Aravind Rajeswaran, Vikash Kumar, Abhishek Gupta, Giulia Vezzani, John Schulman, Emanuel Todorov, and Sergey Levine. Learning complex dexterous manipulation with deep reinforcement learning and demonstrations. arXiv preprint arXiv:1709.10087,

    arXiv

  2. [16]

    Schmidhuber

    J. Schmidhuber. Formal theory of creativity, fun, and intrinsic motivation (1990–2010). IEEE Transactions on Autonomous Mental Development, 2(3),

    1990

  3. [17]

    Parrot: Data-driven behavioral priors for reinforcement learning

    Avi Singh, Huihan Liu, Gaoyue Zhou, Albert Yu, Nicholas Rhinehart, and Sergey Levine. Parrot: Data-driven behavioral priors for reinforcement learning. arXiv preprint arXiv:2011.10024,

    2011 arXiv

  4. [18]

    Hybrid rl: Using both offline and online data can make rl efficient.arXiv preprint arXiv:2210.06718,

    Yuda Song, Yifei Zhou, Ayush Sekhari, J Andrew Bagnell, Akshay Krishnamurthy, and Wen Sun. Hybrid rl: Using both offline and online data can make rl efficient.arXiv preprint arXiv:2210.06718,

    arXiv

  5. [19]

    Leveraging demonstrations for deep reinforcement learning on robotics problems with sparse rewards.arXiv preprint arXiv:1707.08817,

    Mel Vecerik, Todd Hester, Jonathan Scholz, Fumin Wang, Olivier Pietquin, Bilal Piot, Nicolas Heess, Thomas Rothörl, Thomas Lampe, and Martin Riedmiller. Leveraging demonstrations for deep reinforcement learning on robotics problems with sparse rewards.arXiv preprint arXiv:1707.08817,

    arXiv

  6. [20]

    Learning latent state representa- tion for speeding up exploration

    Giulia Vezzani, Abhishek Gupta, Lorenzo Natale, and Pieter Abbeel. Learning latent state representa- tion for speeding up exploration. arXiv preprint arXiv:1905.12621,

    1905 arXiv

  7. [21]

    Policy expansion for bridging offline-to-online reinforcement learning

    Haichao Zhang, We Xu, and Haonan Yu. Policy expansion for bridging offline-to-online reinforcement learning. arXiv preprint arXiv:2302.00935,

    arXiv

  8. [2016]

    Exploration by random network distillation

    Yuri Burda, Harrison Edwards, Amos Storkey, and Oleg Klimov. Exploration by random network distillation. arXiv preprint arXiv:1810.12894,

    arXiv

  9. [2017]

    Learning by playing- solving sparse reward tasks from scratch

    Martin Riedmiller, Roland Hafner, Thomas Lampe, Michael Neunert, Jonas Degrave, Tom Van de Wiele, V olodymyr Mnih, Nicolas Heess, and Jost Tobias Springenberg. Learning by playing- solving sparse reward tasks from scratch. arXiv:1802.10567,

    arXiv

  10. [2018]

    Awac: Accelerating online reinforcement learning with offline datasets

    Ashvin Nair, Abhishek Gupta, Murtaza Dalal, and Sergey Levine. Awac: Accelerating online reinforcement learning with offline datasets. arXiv preprint arXiv:2006.09359,

    2006 arXiv

  11. [2019]

    Self-supervised exploration via disagreement

    Deepak Pathak, Dhiraj Gandhi, and Abhinav Gupta. Self-supervised exploration via disagreement. arXiv:1906.04161,

    1906 arXiv

  12. [2020]

    Go-explore: a new approach for hard-exploration problems

    Adrien Ecoffet, Joost Huizinga, Joel Lehman, Kenneth O Stanley, and Jeff Clune. Go-explore: a new approach for hard-exploration problems. arXiv:1901.10995,

    1901 arXiv

  13. [2021]

    Efficient exploration via state marginal matching

    Lisa Lee, Benjamin Eysenbach, Emilio Parisotto, Eric Xing, Sergey Levine, and Ruslan Salakhutdinov. Efficient exploration via state marginal matching. arXiv:1906.05274,

    1906 arXiv

  14. [2022]

    Making efficient use of demonstrations to solve hard exploration problems

    Tom Le Paine, Caglar Gulcehre, Bobak Shahriari, Misha Denil, Matt Hoffman, Hubert Soyer, Richard Tanburn, Steven Kapturowski, Neil Rabinowitz, Duncan Williams, et al. Making efficient use of demonstrations to solve hard exploration problems. arXiv preprint arXiv:1909.01387,

    1909 arXiv

  15. [2023]

    Modem: Accelerating visual model-based reinforcement learning with demonstrations

    Nicklas Hansen, Yixin Lin, Hao Su, Xiaolong Wang, Vikash Kumar, and Aravind Rajeswaran. Modem: Accelerating visual model-based reinforcement learning with demonstrations. arXiv preprint arXiv:2212.05698,

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.