Pith. sign in

REVIEW 5 major objections 5 minor 38 references

Dream to Generalize: Zero-Shot Model-Based Reinforcement Learning for Unseen Visual Distractions

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Dream to Generalize makes model-based RL transfer zero-shot to unseen visual distractions by replacing pixel reconstruction with dual contrastive learning and a recurrent inverse-dynamics objective.

desk verdict Useful empirical recipe for visual RL generalization, but the headline gains are selective and the predictive-dreaming mechanism is not actually pinned down by the equations. read the letter →

arxiv 2506.05419 v1 pith:2ODOBNJ3 submitted 2025-06-05 cs.CV cs.AI

Jeongsoo Ha , Kyungsoo Kim , Yusung Kim This is my paper
classification cs.CVcs.AI
keywords model-basedreinforcementlearningzero-shotgeneralizationcontrastiveworldmodelsvisualdistractionsDreamerrecurrentstateinversedynamicsself-supervised
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a model-based reinforcement learning agent trained on simple, clean backgrounds can still control its environment when the background changes to unseen natural videos, clouds, shadows, or random overlays, without any fine-tuning at test time. The authors replace the usual pixel-reconstruction objective of the Dreamer-style world model with two contrastive losses: one that pulls together latent states of differently augmented views of the same observation, and one that pulls imagined latent states from the world model toward encoded realities. A recurrent inverse-dynamics loss, which infers the executed action from consecutive imagined latent states, ties the imagined rollouts to real action dynamics. The result, measured on six DeepMind Control tasks and four Robosuite manipulation tasks, is a reported average performance gain of 117% and 14% over prior model-free and model-based baselines in those two benchmarks.

What carries the argument

The load-bearing components are Dual Contrastive Learning (DCL) and Recurrent State Inverse Dynamics (RSID) built on the recurrent state-space model (RSSM) used by Dreamer. DCL applies two InfoNCE contrastive losses: reality-reality, which aligns latent states from soft (random-shift) and hard (random-overlay) augmentations of the same observation through a momentum-updated target encoder; and dream-reality, which aligns imagined latent states from the world model with encoded hard-augmented realities. RSID is a small predictor that takes two successive imagined latent states and outputs the action that must have been taken, trained with mean-squared error against the real action. The joint objective $J_{\mathrm{Dr.G}} = \sum_t \mathbb{E}[J^t_E + J^t_{\mathrm{RSSM}} + J^t_{\mathrm{RSID}} + J^t_R - J^t_{KL}]$ trains the world model and encoder so that the imagined rollouts used for actor-critic learning stay faithful in task-relevant dynamics without ever reconstructing pixels.

What would settle it

A direct test is to run the trained agent on a background whose distractor is correlated with the agent's motion (for example, a shadow that follows the robot) rather than on a static video: if performance collapses or the learned internal representations separate by background rather than by behavior, then the encoder has latched onto background correlations instead of discarding them.

Watch

Extended reading notes

Core claim

The paper's central claim is that a world model trained without any image reconstruction can generalise zero-shot to visual distractions it has never seen, as long as its latent states are made invariant to appearance changes while remaining predictive of dynamics. The encoder is trained so that soft-augmented and hard-augmented views of the same observation produce the same latent state; the world model is trained so that its dreamed next state matches the encoded reality under hard augmentation; and the recurrent inverse-dynamics head recovers the action that connects two successive imagined states. These three objectives replace the reconstruction term of Dreamer, and the paper reports that the resulting agent outperforms prior model-based and model-free methods by 117% on DeepMind Control video-background settings and by 14% on Robosuite randomising environments. The ablation in which the dream-reality objective is removed shows catastrophic degradation, which the paper takes as evidence that the tie between imagined and encoded states is what keeps the latent imagination faithful.

Load-bearing premise

The load-bearing premise is that the contrastive and inverse-dynamics objectives, with no pixel reconstruction, keep the imagined latent states accurate enough for policy learning; if the dreamed dynamics drift from the real task-relevant state evolution, the whole policy is built on a faulty simulator.

Editorial extensions

If this is right

  • Training on simple backgrounds and deploying on unseen video backgrounds without any test-time adaptation is sufficient to outperform prior baselines on 10 of 12 DeepMind Control tasks and on most Robosuite tasks.
  • Removing the dream-reality contrastive objective causes catastrophic degradation, so the tie between imagined latent states and encoded realities is load-bearing for the method.
  • A reconstruction-free world model trained with DCL and RSID supports actor-critic learning through latent imagination, so image reconstruction is not required for control.
  • The reported Robosuite results indicate that the same recipe works across both locomotion and multi-arm manipulation, not just single-body control.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the same dual-contrastive objective should transfer to other latent-dynamics models, not just the RSSM/Dreamer backbone, because the losses are defined on latent states rather than on Dreamer-specific components.
  • A testable consequence of the authors' mechanism is that the harder the visual distraction, the larger the relative advantage over reconstruction-based methods, since reconstruction spends capacity on backgrounds; this could be checked by sweeping the overlay interpolation coefficient alpha.
  • The method depends on a 1.8M-scene external image dataset for hard augmentation; an implicit question is whether procedurally generated textures or a much smaller corpus would give the same invariance, which would matter for real-world deployment.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes Dr. G, a model-based reinforcement learning method that replaces Dreamer's reconstruction objective with dual contrastive learning (reality-reality and dream-reality) and a recurrent state inverse dynamics (RSID) loss, with the aim of achieving zero-shot generalization to unseen visual distractions. The method is trained on simple backgrounds and evaluated on six DeepMind Control tasks under video easy and video hard distractions and on four Robosuite tasks under easy, hard, and extreme distractors. The authors report a 117% average improvement over prior methods on DMControl and a 14% improvement on Robosuite, and they provide ablation studies and open-source code.

Significance. If the claimed mechanism holds, the paper would make a useful contribution by showing that a reconstruction-free, contrastive world model can discard task-irrelevant visual information and support zero-shot generalization in visual MBRL. The open-source code, the inclusion of multiple ablations, and the evaluation across two benchmarks are strengths. However, the central mechanistic claim that the actor-critic is trained on faithful predictive 'dreams' is not yet verified, and the headline improvement numbers are partly based on selective task exclusion, so the significance is currently below what the abstract suggests.

major comments (5)
  1. [§4.2 (Dual Contrastive Learning), Eq. (9); Algorithm 1] The dream state in Eq. (9) is defined as x_t^h = RSSM_theta(z_t^h, x_{t-1}^h, a_{t-1}), i.e., it is a function of the current observation encoding z_t^h. Therefore the dream-reality contrastive loss aligns the RSSM output with the current encoder output; it does not train the transition to predict the next latent from the previous latent and action alone. The RSID loss in Eq. (10) operates on x_t^h and x_{t+1}^h, both of which are posterior states conditioned on observations, so it also does not enforce forward prediction. The paper never specifies whether the imagination rollouts used to train the actor and critic use the prior transition (no observations) or the posterior transition (with observations). In the first case, the world model has never been trained to predict forward; in the second case, the policy is not trained on a predictive model at all. Please specify the imagination procedure and provide an empirical check (e.g., open-loop latent prediction error or a loss that trains the prior to match the posterior) to support the claim that the actor-critic is trained on faithful dreamed rollouts.
  2. [Abstract; Table 1] The headline '117% improvement over prior works' is computed on 10 of the 12 DMControl tasks, as the Table 1 caption states, because SODA outperforms Dr. G on Ball in cup Catch (875 vs 701) and Cartpole Swingup (758 vs 572) in the video easy setting. The abstract should either report the average over all 12 tasks or explicitly state the exclusion; the current wording overstates the result.
  3. [Table 2; Conclusion] On Robosuite, SODA outperforms Dr. G on Nut assembly in all three settings (easy, hard, and extreme), and the text acknowledges this only parenthetically. The conclusion that Dr. G 'demonstrates the best zero-shot generalization performance' and the 14% aggregate should be qualified with per-task win/loss counts, since a single aggregate improvement can mask a systematic failure on one task.
  4. [Experiments - DeepMind Control Suite] The text states that Dr. G shows 'approximately 65% improvement in generalization ability over the prior best-performing baseline' in video easy and 'increased by 152% over the state-of-the-art algorithms' in video hard. These percentages are not derivable from Table 1: averaging over the six video easy tasks gives roughly 14% over SODA and about 7% over SECANT on the four available SECANT tasks, and the video hard average over SODA is about 109%. Please reconcile the numbers or state exactly which baseline and aggregation were used.
  5. [Ablation Studies, Fig. 6; Supplementary Fig. 10] The ablation labeled 'w/o dream-reality' shows catastrophic degradation, but this does not by itself establish that the world model predicts future latents: removing the dream-reality objective also removes the only term that ties the RSSM posterior to the encoder, so the collapse could reflect a misaligned representation rather than a non-predictive transition. In addition, the choice of hard over soft augmentation for the dream-reality loss was made based on the best zero-shot generalization performance shown in Supplementary Figure 10, which is a selection on the test distribution; please disclose this and, if possible, select the augmentation type on a held-out validation set.
minor comments (5)
  1. [Figure 2] The diagram is difficult to read because the line styles for reality-reality, dream-reality, and RSID are easily confused; please enlarge and label the arrows more explicitly.
  2. [Algorithm 1] There are typos ('updata setpc', 'E B[JDr.G(z, x, a|θ)]') and the target encoder E_bar_theta is used in Eq. (7) but its initialization and its update via Eq. (8) are not described in the algorithm box; please fix.
  3. [Eq. (6)] The random-overlay mixing coefficient alpha is used but its value or schedule is not given in the main text; specify it or refer to the supplementary with a concrete number.
  4. [Title page] The copyright line 'Copyright © 2023, Association for the Advancement of Artificial Intelligence' appears to be a leftover template; update to the correct year and venue.
  5. [Figure 4] The t-SNE visualization is qualitative; please add a quantitative measure of representation clustering (e.g., k-NN accuracy or linear probe) to support the claim that Dr. G embeds behaviorally comparable data closest.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the reported 117%/14% gains are externally measured, not derived from fitted constants; the only self-citation is motivational, and the observation-conditioned dream state of Eq. 9 is a mechanism-level caveat rather than a circular reduction.

full rationale

Dr. G is an empirical method paper. Its headline claims are measured by training on simple backgrounds and evaluating on held-out video/Robosuite distractors, with code released; the numbers are not produced by plugging a fitted parameter back into the defining equation. The only overlapping-author citation (Kim, Ha, and Kim 2022) appears in a motivational sentence ('generalization ... is crucial for solving real-world problems') and is not load-bearing for any derivation. Two caveats are noted but do not constitute circularity. First, Eq. (9) defines the matched 'dream' state as x_t^h = RSSM_theta(z_t^h, x_{t-1}^h, a_{t-1}), i.e., it is conditioned on the current encoded observation, so the dream-reality contrastive loss mainly aligns current-state encodings; the paper's wording that this makes the world model 'predict the next latent state' is overstrong, and the actual predictive pressure must come from the retained KL term and RSID. This is a mechanism/specification gap, not a derivation that reduces to its inputs, because the generalization results are still externally measured. Second, the paper states that dream-reality uses hard augmentation 'only because it shows the best zero-shot generalization empirically,' which is a disclosed test-set-based design choice and should temper the zero-shot claim, but it is a selection issue rather than a circular argument. Overall, no fitted constant is renamed as a prediction and no claimed result is equivalent to its own input by construction.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method borrows its backbone from Dreamer/RSSM, its augmentations from the SODA/PAD line, and its contrastive machinery from MoCo/CURL, so most of the machinery is pulled from the prior literature. The load-bearing choices the paper itself makes are the loss weighting (all fixed to 1), the augmentation-type selection for dream-reality (tuned on test performance), and the assumption that a reconstruction-free world model retains enough dynamics for planning.

free parameters (4)
  • dream-reality augmentation type (hard vs soft) = hard augmentation only
    Selected based on empirical zero-shot generalization performance (supplementary Figure 10); this is a test-set-informed design choice.
  • loss weighting coefficients in Eq. 12 = all set to 1
    The relative weights of J_E, J_RSSM, J_RSID, J_R, and J_KL are all 1; no ablation or tuning is reported for these weights.
  • EMA momentum tau (Eq. 8) = not reported in main text
    A standard hyperparameter in momentum contrastive learning, but the value and schedule are not given in the main text.
  • random-overlay mixing coefficient alpha (Eq. 6) = not reported
    Controls the strength of hard augmentation; presumably inherited from SODA but not stated.
assumptions (4)
  • domain assumption The RSSM world model trained without image reconstruction preserves sufficient task-relevant dynamics for policy planning.
    The actor and critic are trained entirely on rollouts imagined by this world model (Algorithm 1, Equation 12); the ablation 'w/o dream-reality' shows performance collapse, indicating the assumption is load-bearing.
  • standard math InfoNCE contrastive loss with an EMA target encoder yields invariant, task-relevant representations.
    The paper relies on the standard contrastive-learning result from MoCo and CURL that InfoNCE aligns positive pairs and repels negatives; this is not proved and is a background method assumption.
  • domain assumption Random-Overlay hard augmentation with a 1.8M-scene natural image dataset is a valid proxy for the unseen video backgrounds used at test time.
    Test videos come from Kinetics400 and Realestate10k, which are distributionally similar to the natural scenes used in training augmentation; this assumption underlies the 'zero-shot' framing.
  • domain assumption Reward reconstruction and KL divergence terms from Dreamer remain appropriate when the image reconstruction objective is removed.
    The objective keeps J_R and J_KL from Dreamer (Equation 12) while dropping image reconstruction; the paper does not analyze whether these terms interact adversely with the new contrastive losses.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dream to Generalize: Zero-Shot Model-Based Reinforcement Learning for Unseen Visual Distractions." pith.science (2026). https://pith.science/paper/2ODOBNJ3

@misc{pith2026250605419,
  author       = {Pith},
  title        = {Pith review of: Dream to Generalize: Zero-Shot Model-Based Reinforcement Learning for Unseen Visual Distractions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2ODOBNJ3}},
  note         = {Machine review of arXiv:2506.05419}
}
read the original abstract

Model-based reinforcement learning (MBRL) has been used to efficiently solve vision-based control tasks in highdimensional image observations. Although recent MBRL algorithms perform well in trained observations, they fail when faced with visual distractions in observations. These task-irrelevant distractions (e.g., clouds, shadows, and light) may be constantly present in real-world scenarios. In this study, we propose a novel self-supervised method, Dream to Generalize (Dr. G), for zero-shot MBRL. Dr. G trains its encoder and world model with dual contrastive learning which efficiently captures task-relevant features among multi-view data augmentations. We also introduce a recurrent state inverse dynamics model that helps the world model to better understand the temporal structure. The proposed methods can enhance the robustness of the world model against visual distractions. To evaluate the generalization performance, we first train Dr. G on simple backgrounds and then test it on complex natural video backgrounds in the DeepMind Control suite, and the randomizing environments in Robosuite. Dr. G yields a performance improvement of 117% and 14% over prior works, respectively. Our code is open-sourced and available at https://github.com/JeongsooHa/DrG.git

Figures

Figures reproduced from arXiv: 2506.05419 by the authors.

Figure 1
Figure 1. The agent is trained in a simple background envi [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 1
Figure 1. Dr. G yields a performance improvement of 117% over existing model-based and model-free RL algorithms on the DeepMind Control suite and 14% over existing algo￾rithms on the Robosuit. The key contributions of this study are as follows: • We introduce a zero-shot MBRL method, Dr. G, to train both the encoder and world model in a self-supervised manner. Using DCL and RSID, Dr. G can achieve robust representations and p… view at source ↗
Figure 2
Figure 2. Our Framework Overview: Dr. G trains the encoder and world model through two-way augmentation, Dual Con￾trastive Learning (green and blue dashed line), and Recurrent State Inverse Dynamics (orange dash line) with sequential data. iterations. A policy model aims to maximize a value esti￾mate, whereas a value model aims to match the value esti￾mate to a behavioral model. Within the imagined trajectory, the actor and c… view at source ↗
Figures from the paper (4 more)
Figure 3
Figure 3. Figure 3: Random shift and random overlay are used as soft [PITH_FULL_IMAGE:figures/full_fig_p004_3.png]
Figure 4
Figure 4. Figure 4: Results of t-SNE of representations learned by [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Our agent is trained in a clean environment (first [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Ablation study on effects of each module in [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

38 extracted references · 13 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    M.; Baker, B.; Chociej, M.; Jozefowicz, R.; McGrew, B.; Pachocki, J.; Petron, A.; Plappert, M.; Powell, G.; Ray, A.; et al

    Andrychowicz, O. M.; Baker, B.; Chociej, M.; Jozefowicz, R.; McGrew, B.; Pachocki, J.; Petron, A.; Plappert, M.; Powell, G.; Ray, A.; et al. 2020. Learning dexterous in-hand manipulation. The International Journal of Robotics Research, 39(1): 3--20

    work page 2020

  4. [4]

    Bishop, C. M. 2006. Pattern recognition. Machine learning, 128(9)

    work page 2006

  5. [5]

    Chen, T.; Kornblith, S.; Norouzi, M.; and Hinton, G. 2020. A simple framework for contrastive learning of visual representations. In International conference on machine learning, 1597--1607. PMLR

    2020

  6. [6]

    Cho, K.; Van Merri \"e nboer, B.; Bahdanau, D.; and Bengio, Y. 2014. On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259

    arXiv 2014

  7. [7]

    Deng, F.; Jang, I.; and Ahn, S. 2021. DreamerPro: Reconstruction-Free Model-Based Reinforcement Learning with Prototypical Representations. arXiv preprint arXiv:2110.14565

    work page Pith review arXiv 2021

  8. [8]

    Fan, L.; Wang, G.; Huang, D.-A.; Yu, Z.; Fei-Fei, L.; Zhu, Y.; and Anandkumar, A. 2021. SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policies. arXiv preprint arXiv:2106.09678

    arXiv 2021

Show all 38 references
  1. [9]

    H.; Buchatskaya, E.; Doersch, C.; Pires, B

    Grill, J.-B.; Strub, F.; Altch \'e , F.; Tallec, C.; Richemond, P. H.; Buchatskaya, E.; Doersch, C.; Pires, B. A.; Guo, Z. D.; Azar, M. G.; et al. 2020. Bootstrap your own latent: A new approach to self-supervised learning. arXiv preprint arXiv:2006.07733

    2020 arXiv

  2. [10]

    Haarnoja, T.; Zhou, A.; Hartikainen, K.; Tucker, G.; Ha, S.; Tan, J.; Kumar, V.; Zhu, H.; Gupta, A.; Abbeel, P.; et al. 2018. Soft actor-critic algorithms and applications. arXiv preprint arXiv:1812.05905

    2018 arXiv

  3. [11]

    Hadsell, R.; Chopra, S.; and LeCun, Y. 2006. Dimensionality Reduction by Learning an Invariant Mapping. In CVPR (2)

    2006

  4. [12]

    Hafner, D.; Lillicrap, T.; Ba, J.; and Norouzi, M. 2019 a . Dream to control: Learning behaviors by latent imagination. arXiv preprint arXiv:1912.01603

    2019 arXiv

  5. [13]

    Hafner, D.; Lillicrap, T.; Fischer, I.; Villegas, R.; Ha, D.; Lee, H.; and Davidson, J. 2019 b . Learning latent dynamics for planning from pixels. In International Conference on Machine Learning, 2555--2565. PMLR

    2019

  6. [14]

    Hafner, D.; Lillicrap, T.; Norouzi, M.; and Ba, J. 2020. Mastering atari with discrete world models. arXiv preprint arXiv:2010.02193

    2020 arXiv

  7. [15]

    A.; Pinto, L.; and Wang, X

    Hansen, N.; Jangir, R.; Sun, Y.; Aleny \`a , G.; Abbeel, P.; Efros, A. A.; Pinto, L.; and Wang, X. 2020. Self-supervised policy adaptation during deployment. arXiv preprint arXiv:2007.04309

    2020 arXiv

  8. [16]

    Hansen, N.; and Wang, X. 2021. Generalization in reinforcement learning by soft data augmentation. In 2021 IEEE International Conference on Robotics and Automation (ICRA), 13611--13617. IEEE

    2021

  9. [17]

    He, K.; Fan, H.; Wu, Y.; Xie, S.; and Girshick, R. 2020. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9729--9738

    2020

  10. [18]

    Henaff, O. 2020. Data-efficient image recognition with contrastive predictive coding. In International conference on machine learning, 4182--4192. PMLR

    2020

  11. [19]

    A.; Wood, F.; and Whiteson, S

    Igl, M.; Zintgraf, L.; Le, T. A.; Wood, F.; and Whiteson, S. 2018. Deep variational reinforcement learning for POMDPs. In International Conference on Machine Learning, 2117--2126. PMLR

    2018

  12. [20]

    Kalashnikov, D.; Irpan, A.; Pastor, P.; Ibarz, J.; Herzog, A.; Jang, E.; Quillen, D.; Holly, E.; Kalakrishnan, M.; Vanhoucke, V.; et al. 2018. Qt-opt: Scalable deep reinforcement learning for vision-based robotic manipulation. arXiv preprint arXiv:1806.10293

    2018 arXiv

  13. [21]

    Kay, W.; Carreira, J.; Simonyan, K.; Zhang, B.; Hillier, C.; Vijayanarasimhan, S.; Viola, F.; Green, T.; Back, T.; Natsev, P.; et al. 2017. The kinetics human action video dataset. arXiv preprint arXiv:1705.06950

    2017 arXiv

  14. [22]

    Kim, K.; Ha, J.; and Kim, Y. 2022. Self-Predictive Dynamics for Generalization of Vision-based Reinforcement Learning. In International Joint Conference on Artificial Intelligence, 3150--3156

    2022

  15. [23]

    P.; and Welling, M

    Kingma, D. P.; and Welling, M. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114

    2013 arXiv

  16. [24]

    Kostrikov, I.; Yarats, D.; and Fergus, R. 2020. Image augmentation is all you need: Regularizing deep reinforcement learning from pixels. arXiv preprint arXiv:2004.13649

    2020 arXiv

  17. [25]

    Laskin, M.; Srinivas, A.; and Abbeel, P. 2020. Curl: Contrastive unsupervised representations for reinforcement learning. In International Conference on Machine Learning, 5639--5650. PMLR

    2020

  18. [26]

    Levine, S.; Finn, C.; Darrell, T.; and Abbeel, P. 2016. End-to-end training of deep visuomotor policies. The Journal of Machine Learning Research, 17(1): 1334--1373

    2016

  19. [27]

    P.; Hunt, J

    Lillicrap, T. P.; Hunt, J. J.; Pritzel, A.; Heess, N.; Erez, T.; Tassa, Y.; Silver, D.; and Wierstra, D. 2015. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971

    2015 arXiv

  20. [28]

    Ma, X.; Chen, S.; Hsu, D.; and Lee, W. S. 2020. Contrastive Variational Reinforcement Learning for Complex Observations. arXiv preprint arXiv:2008.02430

    2020 arXiv

  21. [29]

    Mnih, V.; Kavukcuoglu, K.; Silver, D.; Graves, A.; Antonoglou, I.; Wierstra, D.; and Riedmiller, M. 2013. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602

    2013 arXiv

  22. [30]

    Nair, A.; Pong, V.; Dalal, M.; Bahl, S.; Lin, S.; and Levine, S. 2018. Visual reinforcement learning with imagined goals. arXiv preprint arXiv:1807.04742

    2018 arXiv

  23. [31]

    Nguyen, T.; Shu, R.; Pham, T.; Bui, H.; and Ermon, S. 2021. Temporal Predictive Coding For Model-Based Planning In Latent Space. arXiv preprint arXiv:2106.07156

    2021 arXiv

  24. [32]

    Oord, A. v. d.; Li, Y.; and Vinyals, O. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748

    2018 arXiv

  25. [33]

    Schulman, J.; Moritz, P.; Levine, S.; Jordan, M.; and Abbeel, P. 2015. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438

    2015 arXiv

  26. [34]

    S.; and Barto, A

    Sutton, R. S.; and Barto, A. G. 2018. Reinforcement learning: An introduction. MIT press

    2018

  27. [35]

    Tassa, Y.; Doron, Y.; Muldal, A.; Erez, T.; Li, Y.; Casas, D. d. L.; Budden, D.; Abdolmaleki, A.; Merel, J.; Lefrancq, A.; et al. 2018. Deepmind control suite. arXiv preprint arXiv:1801.00690

    2018 arXiv

  28. [36]

    Van der Maaten, L.; and Hinton, G. 2008. Visualizing data using t-SNE. Journal of machine learning research, 9(11)

    2008

  29. [37]

    X.; and Lin, D

    Wu, Z.; Xiong, Y.; Yu, S. X.; and Lin, D. 2018. Unsupervised feature learning via non-parametric instance discrimination. In Proceedings of the IEEE conference on computer vision and pattern recognition, 3733--3742

    2018

  30. [38]

    Zhu, Y.; Wong, J.; Mandlekar, A.; and Mart \' n-Mart \' n, R. 2020. robosuite: A modular simulation framework and benchmark for robot learning. arXiv preprint arXiv:2009.12293

    2020 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.