Pith. sign in

REVIEW 3 major objections 4 minor 25 references

Self-Predictive Dynamics for Generalization of Vision-based Reinforcement Learning

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Self-Predictive Dynamics, a self-supervised objective chaining inverse and forward dynamics across weak and strong augmentations, makes vision-based RL generalize to unseen distracting backgrounds and outperform prior methods on DeepMind…

desk verdict SPD shows a strong, consistent generalization gain on DMControl and CARLA, but the abstract overstates the win over baselines and the dynamics-chaining loss needs a closer look before the mechanism is accepted. read the letter →

arxiv 2506.05418 v1 pith:DHCSDDV3 submitted 2025-06-05 cs.CV cs.AIcs.LG

Kyungsoo Kim , Jeongsoo Ha , Yusung Kim This is my paper
classification cs.CVcs.AIcs.LG
keywords vision-basedreinforcementlearningrepresentationgeneralizationdataaugmentationdynamicschainingDeepMindControlsuiteCARLAself-supervised
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SPD is a representation-learning side objective that attaches to any vision-based RL algorithm without changing the RL update. It processes each observation twice, once with a weak augmentation and once with a strong one, and trains a shared encoder so that the two latent views carry the same task-relevant content. The training signal comes from three parts: a relativistic discriminator that aligns the two views, inverse dynamics that infer the actual action from cross-view transition pairs, and forward dynamics that predict the next latent from that inferred action. On the DeepMind Control suite with distracting backgrounds and in the CARLA driving simulator, the authors report that SPD improves data efficiency in complex observations and, above all, generalization to observations whose distracting elements were never seen during training. If the claim holds, it shows that a training-time, self-supervised objective can replace deployment-time adaptation for many visual-distractor shifts.

What carries the argument

The load-bearing object is the SPD objective itself, a weighted sum of a relativistic-GAN discriminator and a chained inverse-plus-forward dynamics model in latent space. Weak augmentation is Random-shift; strong augmentation is Random-shift plus a randomly chosen texture or color distortion (grayscale, random convolution, color-jitter, or cutout-color). The discriminator uses a relativistic GAN loss to push the encoder to assign similar scores to $z^w$ and $z^s$, aligning the two views. Dynamics chaining then asks the encoder and the small MLP predictors $I$ and $F$ to solve a consistency problem: the action that moves a weak view forward should also move the corresponding strong view forward, and the forward-predicted next latents should coincide with the encoded next observations. This couples invariance, from the discriminator, to predictive information, from dynamics, which is what lets the policy act on background-invariant latents.

What would settle it

Measure Eq. 3's action-reconstruction error separately on cross-augmentation pairs, $(z^w_t,z^s_{t+1})$ and $(z^s_t,z^w_{t+1})$, versus same-augmentation pairs, $(z^w_t,z^w_{t+1})$ and $(z^s_t,z^s_{t+1})$, on a held-out set. If cross-pair error is systematically larger while the generalization gains remain, the dynamics losses are not enforcing the claimed consistency; if cross-pair error tracks same-pair error, the discriminator is doing the alignment the argument requires.

Watch

Extended reading notes

Core claim

On its own terms, the central claim is that dynamics chaining between weak and strong augmented views is the missing piece for distractor-robust representations in vision-based RL. With a shared encoder fixed, SPD optimizes $J(\psi,\phi,D)=\lambda_\psi(J(I)+J(F))+\lambda_A J(\phi,D)$, where $J(I)$ forces the action inferred from $(z^w_t,z^s_{t+1})$ and from $(z^s_t,z^w_{t+1})$ to match the real action $a_t$, $J(F)$ forces forward predictions $\tilde z^s_{t+1}=F(z^s_t,\tilde a_t)$ and $\bar z^w_{t+1}=F(z^w_t,\bar a_t)$ to match the actually encoded next states, and the relativistic discriminator $J(\phi,D)$ makes $z^s$ indistinguishable from $z^w$. The paper argues that this combination makes the encoder keep exactly the information needed to predict transitions while discarding background, color, and texture changes. The reported result is that on complex and unseen observations, SPD outperforms prior representation-learning RL baselines such as DrQ, CURL, SODA, and PAD, and achieves best or near-best performance on all nine DMControl tasks and in CARLA, while requiring no pre-training or test-time fine-tuning.

Load-bearing premise

The load-bearing premise is that a weak and a strong augmentation of the same observation still represent the same state, so the action that carried the original frame pair also carries the cross-augmentation pairs used in Equations 3 and 4.

Editorial extensions

If this is right

  • SPD can be layered on top of any RL algorithm; the paper shows gains with SAC, TD3, and PPO, with no changes to the RL updates.
  • Training on a simple distractor background transfers to an unseen natural-video background across all nine DMControl tasks, on some tasks exceeding even test-time-adapted PAD.
  • In heavily distracting environments, SPD's data-efficiency edge grows with distraction level: it wins 3 of 5 main-table tasks on Simple Distractor but 5 of 5 on Natural Video.
  • In CARLA, SPD reaches the highest average return and the smallest latent distance between observations that are behaviorally identical but visually different.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The cross-augmentation inverse-dynamics loss implicitly assumes the two views of the same state map nearby in latent space; a natural stress test we would add is to weight Eq. 3 by view agreement or add a stop-gradient, which the paper does not explore.
  • If the mechanism is as described, the method should transfer to other distractor families that the strong augmentations approximate, such as shadows or weather, but not to shifts outside that coverage such as changes in camera intrinsics or viewpoint; this is our inference, not a paper claim.
  • The CARLA representation-distance metric suggests a cheap diagnostic for other domains: a well-generalized encoder should keep representation distance low between same-behavior observations, so the metric could serve as an early-stopping or selection criterion.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes Self-Predictive Dynamics (SPD), a self-supervised auxiliary objective for vision-based reinforcement learning. SPD applies weak and strong augmentations to the same observation, trains an encoder with a relativistic discriminator to align the two views, and adds an inverse-plus-forward dynamics chaining loss that predicts actions and next latents across the two augmented levels. The method is evaluated on nine DeepMind Control suite tasks and on CARLA, reporting strong data-efficiency and generalization results, including generalization from Simple Distractor to unseen Natural Video backgrounds, with ablations and hyperparameter sensitivity in the supplementary material.

Significance. If the reported results are sound, the paper makes a useful empirical contribution: it shows that a single-stage, training-time representation objective can improve robustness to unseen distractors in vision-based RL, without test-time fine-tuning. The evaluation is unusually comprehensive for this line of work: nine DMControl tasks, three seeds, two background protocols, CARLA, ablations for each component, integration with TD3 and PPO, and a hyperparameter sensitivity study. The code release is also a concrete asset. The main weakness is that the forward-dynamics component of the central mechanism is not validated against a plausible degenerate solution, and the key generalization baseline from the related work (DBC) is not included in the comparisons.

major comments (3)
  1. [§3.4, Eq. (4)] The forward dynamics loss is vulnerable to target leakage. In the first branch, the action \tilde{a}_t = I(z^w_t, z^s_{t+1}) is computed from the very target z^s_{t+1} that F is then asked to predict, and the loss penalizes the distance between F(z^s_t, \tilde{a}_t) and z^s_{t+1}. F can therefore minimize this loss by learning to decode information about z^s_{t+1} from \tilde{a}_t, rather than by modeling the true transition. The same holds for the second branch. Because of this, the improvement of full SPD over "Discriminator + Inverse" in Figure 4 (left) cannot be attributed to predictive dynamics. Please report whether the inverse model achieves comparable error on same-level and cross-level pairs, and whether the forward model predicts the next latent better from the recorded action a_t than from the inferred action; if leakage is confirmed, the forward loss should be modified, for example by stopping gradients through the inferred action or by evaluating F on the true action.
  2. [§3.4, Eq. (3)] The inverse dynamics loss presupposes that the cross-level pairs (z^w_t, z^s_{t+1}) and (z^s_t, z^w_{t+1}) are valid transitions generated by the same action a_t as the original frame pair. This holds only if the encoder already maps weak and strong views of the same state to nearby latents; the discriminator encourages this but does not guarantee it. If this premise fails, Equation (3) fits the action predictor to inconsistent labels and Equation (4) inherits the error. The paper provides no diagnostic of this assumption. Please add an experiment that tracks inverse-dynamics error on cross-level pairs versus same-level pairs over training, or an equivalent check that the cross-level transitions are actually consistent.
  3. [§4.2 and Table 3] DBC, cited in §2.2 as the main invariance-based generalization baseline, is not included in any of the experiments. Since the paper's headline claim is about generalization to unseen observations, omitting DBC makes the statement that SPD "significantly outperforms previous studies" stronger than the evidence presented. Either add DBC to the DMControl generalization comparison or explain concretely why it cannot be included under the same protocol.
minor comments (4)
  1. [Abstract and Table 1] The abstract states that SPD "outperforms previous studies in complex observations," but in the Simple Distractor protocol CURL is better on Walker Walk (917.4 vs. 895.0) and Walker Run (488.1 vs. 406.1); the claim should be qualified to the Natural Video setting or to "most tasks."
  2. [§3.3 and Eq. (6)] Equation (6) uses the notation J(\phi, D), but the individual objectives are defined as J(\phi) and J(D) in Equations (1) and (2); please clarify how the two are combined in J(\phi, D).
  3. [Algorithm 1 and §3.4] The text says the training phase is divided into two steps, but Algorithm 1 alternates self-supervision and RL updates within each iteration; this wording is confusing and should be revised.
  4. [Supplementary Material C] The modification of CURL, SODA, and PAD from Random Crop plus Center Crop to Random-shift plus No augmentation is important for interpreting the baselines; this should be stated in the main text rather than only in the supplementary material.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the generalization claim is tested on held-out backgrounds, and no prediction reduces to a fitted input or a self-citation chain; Eq. 4's target-derived action is a training-loss caveat, not a load-bearing circular step.

full rationale

SPD is an empirical representation-learning method rather than a theorem-driven derivation. The central generalization claim (Table 1, bottom row) is evaluated on held-out Natural Video backgrounds after training on Simple Distractor, so the reported returns are not constructed from the training objective or from the parameters fitted during training. There are no load-bearing self-citations: the author list includes no prior-work citations by the same authors, and the method relies on standard self-supervised components (weak/strong augmentation, relativistic GAN discriminator, inverse and forward dynamics) rather than on an imported uniqueness theorem or an ansatz smuggled in via citation. The one self-referential element is the forward-dynamics loss in Eq. 4, where the action fed to F, tilde a_t = I(z^w_t, z^s_{t+1}), is inferred from the very target latent z^s_{t+1} that F is asked to predict; this can allow a shortcut in the auxiliary loss and is a legitimate learning-dynamics or identifiability concern. However, the paper's headline result is downstream RL return on unseen observations, not the forward-loss value, and the ablation attributing gains to dynamics chaining is an internal mechanism claim rather than the paper's primary prediction. The supplementary note that CURL/SODA/PAD center-crop baselines were replaced with random-shift is a baseline-fairness caveat, not a circularity. Overall, the core empirical claim is self-contained and externally evaluated, so no circular step is identified; the score of 2 reflects only the minor self-referential training objective without making it load-bearing.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The central claim rests on a set of empirical design choices and assumptions about the training setup. The two weighting hyperparameters are manually chosen, although sensitivity is reported. The most load-bearing assumptions are that strong augmentations preserve task-relevant content and that the cross-level transition pairs used for inverse dynamics are valid, since these directly determine the auxiliary losses that drive representation learning.

free parameters (2)
  • lambda_psi (dynamics chaining weight) = 0.1
    Chosen hyperparameter in the SPD objective (Eq. 6). Hyperparameter sensitivity is shown in Figure 13, but the selection was made by the authors and could influence the balance between dynamics and adversarial losses.
  • lambda_A (adversarial weight) = 0.001
    Chosen hyperparameter in the SPD objective (Eq. 6). Sensitivity is reported, but the value affects the strength of the invariance alignment relative to the dynamics losses.
assumptions (6)
  • domain assumption Shared encoder trained jointly by RL and SPD losses does not suffer catastrophic interference
    Section 3.1: the encoder phi is updated with gradients from both RL and SPD; the paper does not analyze potential conflicts between these gradient signals.
  • domain assumption Strong augmentations preserve task-relevant semantics
    Section 3.2: grayscale, random convolution, color-jitter, cutout-color and random-shift are assumed to distort distractors while leaving control-relevant content identifiable; if an augmentation destroys task-relevant features, the inverse dynamics labels become invalid.
  • ad hoc to paper Cross-level transition pairs are valid
    Section 3.4, Eq. 3: the inverse dynamics loss treats (z_w_t, z_s_{t+1}) and (z_s_t, z_w_{t+1}) as transitions with action a_t, even though these are not real transitions observed in the replay buffer. This is a modeling assumption central to the dynamics chaining.
  • domain assumption Relativistic GAN training converges without destabilizing the policy
    Section 3.3: the discriminator and encoder are optimized alternately with SAC; no convergence analysis is given, only empirical results.
  • domain assumption Evaluation protocol for baselines is fair
    Supplementary C: Random Crop and Center Crop are replaced with Random-shift and no augmentation for CURL, SODA, and PAD, assuming this does not systematically disadvantage these baselines.
  • domain assumption Visual invariance learned from a fixed augmentation set transfers to unseen natural video backgrounds
    Section 4.2 Generalization: the paper evaluates on Natural Video after training on Simple Distractor, assuming the discriminator and dynamics losses build invariance that extrapolates beyond the augmentation family.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Self-Predictive Dynamics for Generalization of Vision-based Reinforcement Learning." pith.science (2026). https://pith.science/paper/DHCSDDV3

@misc{pith2026250605418,
  author       = {Pith},
  title        = {Pith review of: Self-Predictive Dynamics for Generalization of Vision-based Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DHCSDDV3}},
  note         = {Machine review of arXiv:2506.05418}
}
read the original abstract

Vision-based reinforcement learning requires efficient and robust representations of image-based observations, especially when the images contain distracting (task-irrelevant) elements such as shadows, clouds, and light. It becomes more important if those distractions are not exposed during training. We design a Self-Predictive Dynamics (SPD) method to extract task-relevant features efficiently, even in unseen observations after training. SPD uses weak and strong augmentations in parallel, and learns representations by predicting inverse and forward transitions across the two-way augmented versions. In a set of MuJoCo visual control tasks and an autonomous driving task (CARLA), SPD outperforms previous studies in complex observations, and significantly improves the generalization performance for unseen observations. Our code is available at https://github.com/unigary/SPD.

Figures

Figures reproduced from arXiv: 2506.05418 by the authors.

Figure 1
Figure 1. Our Framework Overview: we use a shared encoder for RL and Self-Predictive Dynamics (SPD). An observation is augmented [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. We use three different background types. There are exam [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. t-SNE of representations learned by SPD, SODA, PAD and SAC. Even if the background is dramatically different, SPD can encode [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: (left) Ablation studies for SPD. We test the effects of dis [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Scenes in CARLA simulations classified as Highway (left [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 7
Figure 7. Figure 7: Data augmentations used in our framework: (From the [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 9
Figure 9. Figure 9: (top) Data Efficiency evaluation, (bottom) Generalization [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: Results of data efficiency evaluation for SPD and baselines on [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]
Figure 11
Figure 11. Figure 11: Results of data efficiency evaluation for SPD and baselines on [PITH_FULL_IMAGE:figures/full_fig_p010_11.png]
Figure 12
Figure 12. Figure 12: Results of generalization evaluation for SPD and baselines. We show the learning curves of each tasks on three different seeds [PITH_FULL_IMAGE:figures/full_fig_p011_12.png]
Figure 13
Figure 13. Figure 13: Hyper-parameter sensitivity of SPD’s objective weights. [PITH_FULL_IMAGE:figures/full_fig_p012_13.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

25 extracted references · 10 canonical work pages

  1. [1]

    A simple framework for contrastive learning of visual representations

    [Chenet al., 2020 ] Ting Chen, Simon Kornblith, Moham- mad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. InIn- ternational conference on machine learning, pages 1597–

    2020

  2. [7]

    Learning latent dynamics for planning from pixels

    [Hafneret al., 2019 ] Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, and James Davidson. Learning latent dynamics for planning from pixels. InInternational Conference on Machine Learning, pages 2555–2565. PMLR,

    work page 2019

  3. [8]

    Generalization in reinforcement learning by soft data augmentation

    [Hansen and Wang, 2021] Nicklas Hansen and Xiaolong Wang. Generalization in reinforcement learning by soft data augmentation. In2021 IEEE International Confer- ence on Robotics and Automation (ICRA), pages 13611– 13617. IEEE,

    work page 2021

  4. [9]

    Self-supervised policy adaptation during deployment.arXiv preprint arXiv:2007.04309,

    [Hansenet al., 2020 ] Nicklas Hansen, Rishabh Jangir, Yu Sun, Guillem Aleny `a, Pieter Abbeel, Alexei A Efros, Lerrel Pinto, and Xiaolong Wang. Self-supervised policy adaptation during deployment.arXiv preprint arXiv:2007.04309,

    arXiv 2020

  5. [10]

    The relativistic discriminator: a key element missing from standard gan.arXiv preprint arXiv:1807.00734,

    [Jolicoeur-Martineau, 2018] Alexia Jolicoeur-Martineau. The relativistic discriminator: a key element missing from standard gan.arXiv preprint arXiv:1807.00734,

    arXiv 2018

  6. [11]

    The kinetics human action video dataset

    [Kayet al., 2017 ] Will Kay, Joao Carreira, Karen Si- monyan, Brian Zhang, Chloe Hillier, Sudheendra Vijaya- narasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natsev, et al. The kinetics human action video dataset. arXiv preprint arXiv:1705.06950,

    arXiv 2017

  7. [12]

    Image augmentation is all you need: Reg- ularizing deep reinforcement learning from pixels.arXiv preprint arXiv:2004.13649,

    [Kostrikovet al., 2020 ] Ilya Kostrikov, Denis Yarats, and Rob Fergus. Image augmentation is all you need: Reg- ularizing deep reinforcement learning from pixels.arXiv preprint arXiv:2004.13649,

    arXiv 2020

  8. [13]

    Reinforcement learning with augmented data.arXiv preprint arXiv:2004.14990,

    [Laskinet al., 2020 ] Michael Laskin, Kimin Lee, Adam Stooke, Lerrel Pinto, Pieter Abbeel, and Aravind Srini- vas. Reinforcement learning with augmented data.arXiv preprint arXiv:2004.14990,

    arXiv 2020

Show all 25 references
  1. [14]

    Stochastic latent actor-critic: Deep reinforcement learning with a latent variable model

    [Leeet al., 2019a ] Alex X Lee, Anusha Nagabandi, Pieter Abbeel, and Sergey Levine. Stochastic latent actor-critic: Deep reinforcement learning with a latent variable model. arXiv preprint arXiv:1907.00953,

    1907 arXiv

  2. [15]

    Network randomization: A simple tech- nique for generalization in deep reinforcement learning

    [Leeet al., 2019b ] Kimin Lee, Kibok Lee, Jinwoo Shin, and Honglak Lee. Network randomization: A simple tech- nique for generalization in deep reinforcement learning. arXiv preprint arXiv:1910.05396,

    1910 arXiv

  3. [16]

    Representation learning with contrastive predic- tive coding.arXiv preprint arXiv:1807.03748,

    [Oordet al., 2018 ] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding.arXiv preprint arXiv:1807.03748,

    2018 arXiv

  4. [17]

    Curiosity-driven exploration by self-supervised prediction

    [Pathaket al., 2017 ] Deepak Pathak, Pulkit Agrawal, Alexei A Efros, and Trevor Darrell. Curiosity-driven exploration by self-supervised prediction. InInternational conference on machine learning, pages 2778–2787. PMLR,

    2017

  5. [18]

    Prox- imal policy optimization algorithms.arXiv preprint arXiv:1707.06347,

    [Schulmanet al., 2017 ] John Schulman, Filip Wolski, Pra- fulla Dhariwal, Alec Radford, and Oleg Klimov. Prox- imal policy optimization algorithms.arXiv preprint arXiv:1707.06347,

    2017 arXiv

  6. [19]

    Data-efficient reinforcement learn- ing with self-predictive representations.arXiv preprint arXiv:2007.05929,

    [Schwarzeret al., 2020 ] Max Schwarzer, Ankesh Anand, Rishab Goel, R Devon Hjelm, Aaron Courville, and Philip Bachman. Data-efficient reinforcement learn- ing with self-predictive representations.arXiv preprint arXiv:2007.05929,

    2020 arXiv

  7. [20]

    Curl: Contrastive unsupervised repre- sentations for reinforcement learning.arXiv preprint arXiv:2004.04136,

    [Srinivaset al., 2020 ] Aravind Srinivas, Michael Laskin, and Pieter Abbeel. Curl: Contrastive unsupervised repre- sentations for reinforcement learning.arXiv preprint arXiv:2004.04136,

    2020 arXiv

  8. [21]

    Deepmind control suite.arXiv preprint arXiv:1801.00690,

    [Tassaet al., 2018 ] Yuval Tassa, Yotam Doron, Alistair Muldal, Tom Erez, Yazhe Li, Diego de Las Casas, David Budden, Abbas Abdolmaleki, Josh Merel, Andrew Lefrancq, et al. Deepmind control suite.arXiv preprint arXiv:1801.00690,

    2018 arXiv

  9. [22]

    Improving sample efficiency in model-free reinforcement learning from images.arXiv preprint arXiv:1910.01741,

    [Yaratset al., 2019 ] Denis Yarats, Amy Zhang, Ilya Kostrikov, Brandon Amos, Joelle Pineau, and Rob Fergus. Improving sample efficiency in model-free reinforcement learning from images.arXiv preprint arXiv:1910.01741,

    2019 arXiv

  10. [23]

    Learn- ing invariant representations for reinforcement learning without reconstruction.arXiv preprint arXiv:2006.10742,

    [Zhanget al., 2020 ] Amy Zhang, Rowan McAllister, Roberto Calandra, Yarin Gal, and Sergey Levine. Learn- ing invariant representations for reinforcement learning without reconstruction.arXiv preprint arXiv:2006.10742,

    2020 arXiv

  11. [24]

    We show the learning curves of each experiments on three different seeds with 1.0 standard error shaded

    Supplementary Material A Data Augmentation ablations Random shiftGrayscaleRandomConvolutionCutout-colorColor-jitterOriginal Figure 7: Data augmentations used in our framework: (From the left) Random-shift, Grayscale, Random Convolution, Cutout-color, and Color-jitter 0 1 2 3 4...

    2018

  12. [255]

    We apply augmentation to images sampled from the buffer or a recent trajectory only during training procedure, not environment interaction procedure

    Data augmentations described in Figure 7 is applied to the normalized image. We apply augmentation to images sampled from the buffer or a recent trajectory only during training procedure, not environment interaction procedure. In the DeepMind Control suite, when Random shift i...

    2019

  13. [2017]

    Secant: Self-expert cloning for zero-shot generaliza- tion of visual policies.arXiv preprint arXiv:2106.09678,

    [Fanet al., 2021 ] Linxi Fan, Guanzhi Wang, De-An Huang, Zhiding Yu, Li Fei-Fei, Yuke Zhu, and Anima Anandku- mar. Secant: Self-expert cloning for zero-shot generaliza- tion of visual policies.arXiv preprint arXiv:2106.09678,

    2021 arXiv

  14. [2018]

    Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor

    [Haarnojaet al., 2018 ] Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. InInternational conference on ma- chine learning, pages 1861–1870. PMLR,

    2018

  15. [2019]

    Carla: An open urban driving simulator

    [Dosovitskiyet al., 2017 ] Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. Carla: An open urban driving simulator. InConference on robot learning, pages 1–16. PMLR,

    2017

  16. [2020]

    Quantifying general- ization in reinforcement learning

    [Cobbeet al., 2019 ] Karl Cobbe, Oleg Klimov, Chris Hesse, Taehoon Kim, and John Schulman. Quantifying general- ization in reinforcement learning. InInternational Con- ference on Machine Learning, pages 1282–1289. PMLR,

    2019

  17. [2021]

    Addressing function approximation error in actor-critic methods

    [Fujimotoet al., 2018 ] Scott Fujimoto, Herke Hoof, and David Meger. Addressing function approximation error in actor-critic methods. InInternational Conference on Machine Learning, pages 1587–1596. PMLR,

    2018

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.