Pith. sign in

REVIEW 4 major objections 5 minor 41 references

DeGuV: Depth-Guided Visual Reinforcement Learning for Generalization and Interpretability in Manipulation

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Depth-stable masks, not the depth data itself, drive the generalization gains of DeGuV.

desk verdict Depth-guided masking is a real idea with big benchmark gains, but the missing depth-control ablation leaves the core attribution unproven. read the letter →

arxiv 2509.04970 v1 pith:GELUCV62 submitted 2025-09-05 cs.RO cs.AI

Tien Pham , Xinyun Chi , Khang Nguyen , Manfred Huber , Angelo Cangelosi This is my paper
classification cs.ROcs.AI
keywords visualreinforcementlearningdepth-guidedmaskinggeneralizationsim-to-realtransferroboticmanipulationcontrastivesampleefficiencyinterpretability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the generalization bottleneck in vision-based reinforcement learning is the variance that data augmentation injects into task-irrelevant pixels. DeGuV uses the depth channel, which is invariant to color, texture, and brightness shifts, to train a masker that zeros out those pixels before the RGB image reaches the policy. With the remaining variance reduced, the agent can learn a visual perturbation-invariant representation, and the paper adds contrastive learning and stabilized Q-value estimation to keep that learning efficient and stable. On four Franka manipulation tasks in the RL-ViGen benchmark, DeGuV reports an average episode return of 311.44 against 200.23 for the next-best baseline, retains 82.5% of training performance across unseen visual conditions, and transfers zero-shot to a real Franka robot on the Lift task. If the claim holds, depth is not just extra sensor data but a source of stable supervision for what to look at.

What carries the argument

The load-bearing object is the depth-guided masker M_theta: a fully differentiable convolutional network with ReLU hidden activations and a final Hardtanh, which takes only the depth image and outputs a scalar mask in [0,1] that is multiplied element-wise into the RGB frame (DGMask, Eq. 4). Its stability comes from the invariance of depth to the color/texture/brightness perturbations used in augmentation. It is updated solely through the critic's Bellman loss, leveraging the fact that the optimal Q-function ignores distractors, with target networks and a stop-grad operator on the augmented Q-estimation branch shielding it from non-stationary gradients. The auxiliary machinery is an InfoNCE c

What would settle it

Run DeGuV's exact setup with the masker removed and depth simply concatenated to the RGB input (or fused by a trivial encoder) before the actor-critic, keeping the contrastive loss and SVEA intact. If that agent matches DeGuV's average episode return (~311) or retention (~82.5%), the claim that masking causes the improvement is falsified. Conversely, replacing the learned mask with a random [0,1] mask or a fixed center crop should produce a large performance drop if the mask's causal role is real.

Watch

Extended reading notes

Core claim

The paper's central claim is that the generalization gap in visual RL comes from variance that data augmentation injects into task-irrelevant pixels. It decomposes augmented RGB variance into a task-relevant part and a distraction part, and argues that if the distraction part is zeroed out by masking, the learned representation can satisfy the visual perturbation-invariance condition z* = f(s) = f(s~) without paying augmentation's usual sample-efficiency cost. Because the optimal Q-function is by definition invariant to distractions, training the masker with the same Bellman residual that trains the critic is claimed to make the mask converge to the truly task-relevant pixels, while the dept

Load-bearing premise

The load-bearing premise is that a masker trained only by the critic's reward-based loss learns to erase exactly the pixels whose absence still leaves the RGB image sufficient for optimal control - and that this masking, not the extra depth channel, is what creates the improvement; the paper includes no depth-without-masking baseline to test that premise.

Editorial extensions

If this is right

  • On the paper's numbers, policies trained with DeGuV keep about four-fifths of their training performance when the environment shifts to easy, medium, and hard visual perturbations, where baseline retention falls to roughly 0.23-0.46.
  • The reported average episode return of 311.44 across four tasks and three evaluation modes implies the method can be a direct replacement for augmentation-only visual RL in robotic manipulation.
  • The masked images being nearly identical across easy and hard evaluations (16.08% of pixels revealed in both) means the masker can be read as an explanation of which scene elements the policy actually uses.
  • The zero-shot Franka Lift deployment indicates policies trained in robosuite can be wrapped in ROS2 and run on a real robot without real-world fine-tuning, as long as depth post-processing filters handle sensor noise.
  • Because the mask is produced from depth alone, it does not depend on the RGB encoder's current features, so it can be computed at deployment time even under heavy visual perturbation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The core mechanism is not cleanly isolated: no baseline receives depth without the masker, so the paper's interpretation that masking - and not the extra depth modality - causes the improvement remains an inference the authors did not directly test. A depth-fused, unmasked control would settle it.
  • The masker only ever sees the critic's reward-derived loss during training, so its masks may encode task-specific saliency rather than general object relevance; evaluating on held-out tasks with different distractors would reveal whether the mask module transfers.
  • Since the paper itself flags depth noise as a real-world problem, a natural extension is to learn the mask from denoised or predicted depth at deployment, or to fuse multiple stable cues when depth is unreliable.
  • The masker could be detached and reused as a fixed attention module for other RGB-only policies or for interpretability dashboards, because it maps depth to saliency independently of the RGB encoder.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes DeGuV, a visual RL method for robotic manipulation that uses a learnable depth-guided masker to erase task-irrelevant RGB pixels, combined with contrastive (InfoNCE) representation learning and Stabilized Q-Value Estimation (SVEA) on top of SAC. The method is evaluated on four robosuite/RL-ViGen manipulation tasks (Lift, Door, NutAssemblyRound, TwoArmPegInHole) under train/easy/medium/hard visual generalization splits, compared against DrQv2, CURL, SGQN, SVEA, and MaDi. The reported results show higher average episode returns and higher training-performance retention, plus a qualitative real-robot zero-shot sim-to-real demonstration on the Lift task. The central claim is that depth-guided masking, not merely the availability of depth or the additional contrastive/SVEA components, drives the improvements.

Significance. If substantiated, DeGuV would provide a practical way to leverage depth for visual generalization and interpretability in manipulation, with a straightforward integration into SAC. The paper includes a public implementation link, evaluations on a recognized benchmark, and a real-robot demonstration, which are useful for reproducibility. However, the central causal attribution is not yet supported because the experimental design lacks a depth-aware baseline or a masker-disabled ablation; in addition, the theoretical motivation contains unproven variance decompositions and a Bellman residual that is not standard SAC. The contribution is potentially significant but requires the missing controlled experiments and theory cleanup.

major comments (4)
  1. The paper's central claim is that depth-guided masking, not depth availability itself, causes the performance gains. Yet all baselines (DrQv2, CURL, SGQN, SVEA, MaDi) use RGB only, and no ablation removes the masker while keeping depth as an input (e.g., depth concatenation) or keeps depth but uses an all-ones mask. Since depth is known to be informative for manipulation, DeGuV's improvement could be entirely due to the extra modality, or to the combination of contrastive learning and SVEA, rather than to masking. The claim in Section V-B that 'thanks to depth-guided masking' DeGuV generalizes is therefore unsupported. Please add ablations: (a) DeGuV with masker output fixed to all ones; (b) DeGuV with depth concatenated to RGB but no mask; (c) DeGuV without contrastive loss; and report mask statistics (e.g., fraction of retained pixels, variance across seeds). Without these, Section IV-
  2. Eq. (1) defines the Bellman residual for SAC as R = (r + gamma max_{a'} Q_tgt(s',a') - Q(s,a)). This max-based target is not the SAC target: SAC's continuous-action critic should target the expectation over the stochastic policy, typically including the entropy bonus, not a hard max over actions. As written, the critic objective does not correspond to SAC, and the subsequent claim that minimizing this residual simultaneously learns the optimal policy and masks distractions inherits this problem. Please correct the equation to the actual SAC target used in the implementation, or explain if a DDPG-style critic is used; the paper's method and algorithm assume SAC throughout.
  3. Eq. (3) states Var(P_RGB_aug) = Var(P_RGB_relevant) + Var(P_RGB_distraction). Partitioning pixels into two subsets does not by itself make the variance of the union additive; this decomposition requires independence between the pixel subsets or a more careful definition of the distributions over pixels. The equation is the formal motivation for why masking reduces learning complexity, so it is load-bearing. Please provide a derivation with explicit definitions of these pixel distributions, or replace this with a clearly stated heuristic claim.
  4. The empirical evidence is limited to four tasks with three seeds and no significance tests. Many reported standard deviations are very large (e.g., Lift easy: 293.80±160.07; Door medium: 333.32±203.53), and the aggregate average (311.44 vs. 200.23) may be driven by a subset of tasks. Table II's 'performance retention' is an aggregate ratio that presumably mixes different reward scales; its computation should be described and accompanied by confidence intervals. Add per-task significance tests (or at least bootstrap intervals) and report the number of seeds per cell. Without this, the strong cross-task generalization claim is disproportionate to the evidence.
minor comments (5)
  1. The InfoNCE loss sum is written as sum_{i=0}^M, but negative keys are conventionally indexed from 1 to M; also the notation for the positive key and the temperature hyperparameter should be defined where first used.
  2. Notation is overloaded: P denotes both transition function and pixel space; S is used both as a generic state space and as the stacked RGB-D state space. Please use distinct symbols.
  3. Line 11 uses '//' to indicate stop-gradient, but the notation is not defined until later in the text. Also, target network updates are described only for Q, while Algorithm 1 includes f_tgt; clarify whether the encoder target is updated and how.
  4. In the text and Table I, CURL is cited as [22], but reference [22] is the contrastive behavioral similarity embedding paper, not CURL (which is reference [23]). Please check the citation mapping.
  5. Hyperparameters alpha, beta, and tau, as well as the masker architecture and training schedule, are not reported; these affect the balance between masking and Q-learning. The real-robot experiment is qualitative for one task only; reporting a success rate over several trials would be more informative.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DeGuV's method is an empirical combination of independently cited components; the central claim is under-supported by a missing ablation but not circular.

full rationale

The paper's claimed derivation runs from a definition of visual perturbation-invariant representations (Eq. 2), through a variance decomposition (Eq. 3), to a depth-guided masker trained by the Bellman residual (Eq. 7). Each objective is stated as a standard RL or contrastive loss; the masker is not fitted to the evaluation benchmark or to any quantity that is later reported as a 'prediction.' No parameter is tuned on the RL-ViGen test environments, and no load-bearing claim is justified by self-citation: the cited baselines (DrQv2, CURL, SGQN, SVEA, MaDi) and benchmarks are all external to the present authors. The closest issue is that the attribution 'thanks to depth-guided masking' (Section V-B) is not controlled by a depth-without-masking baseline, so the improvement might stem from the extra depth modality or from the combination with contrastive learning and SVEA; however, that is an experimental confound, not a circular derivation. The manuscript even includes a limitation sentence (Section VII) about depth noise, which further indicates the authors do not presuppose the conclusion. Therefore the circularity score is 0.

Assumptions & free parameters 3 free parameters · 2 assumptions · 0 invented entities

The method relies on the stability of depth under perturbation, the decomposability of image variance, and several hyperparameters whose values are not reported. No new physical entities are introduced.

free parameters (3)
  • alpha, beta (loss coefficients) = not reported
    Eq. 7 balances the original Bellman residual and the augmented residual; values are chosen by hand and not stated, affecting the central empirical result.
  • InfoNCE temperature tau = not reported
    Eq. 6 uses tau as a temperature hyperparameter; not specified in the paper.
  • masker architecture hyperparameters = not reported
    Number of convolutional layers, channels, and hardtanh threshold are not given; a re-implementation would need to guess.
assumptions (2)
  • domain assumption Depth input is invariant to visual perturbations such as color and brightness changes.
    Used in Section IV-B to justify using depth as the source for masking; real sensors have depth noise, which the paper acknowledges in Section VII.
  • domain assumption The variance of the augmented RGB distribution decomposes into the sum of variances of relevant and distraction pixels (Eq. 3), implying zero covariance between the two pixel sets.
    Eq. 3 in Section IV-B; not justified, and independence is unlikely to hold for natural images.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DeGuV: Depth-Guided Visual Reinforcement Learning for Generalization and Interpretability in Manipulation." pith.science (2026). https://pith.science/paper/GELUCV62

@misc{pith2026250904970,
  author       = {Pith},
  title        = {Pith review of: DeGuV: Depth-Guided Visual Reinforcement Learning for Generalization and Interpretability in Manipulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GELUCV62}},
  note         = {Machine review of arXiv:2509.04970}
}
read the original abstract

Reinforcement learning (RL) agents can learn to solve complex tasks from visual inputs, but generalizing these learned skills to new environments remains a major challenge in RL application, especially robotics. While data augmentation can improve generalization, it often compromises sample efficiency and training stability. This paper introduces DeGuV, an RL framework that enhances both generalization and sample efficiency. In specific, we leverage a learnable masker network that produces a mask from the depth input, preserving only critical visual information while discarding irrelevant pixels. Through this, we ensure that our RL agents focus on essential features, improving robustness under data augmentation. In addition, we incorporate contrastive learning and stabilize Q-value estimation under augmentation to further enhance sample efficiency and training stability. We evaluate our proposed method on the RL-ViGen benchmark using the Franka Emika robot and demonstrate its effectiveness in zero-shot sim-to-real transfer. Our results show that DeGuV outperforms state-of-the-art methods in both generalization and sample efficiency while also improving interpretability by highlighting the most relevant regions in the visual input

Figures

Figures reproduced from arXiv: 2509.04970 by the authors.

Figure 1
Figure 1. (a) DeGuV is trained with robotic manipulation tasks in robosuite environment, (b) is evaluated on the RL-ViGen benchmark for generalization and interpretability verfication, and (c) is deployed on a Franka Emika robot to demonstrate its zero-shot sim-to-real transferability. The RGB-D observations and the masked image produced by DeGuV are presented from left to right in each case. Abstract— Reinforcement learning … view at source ↗
Figure 2
Figure 2. Illustrations of (a) generalization problem in visual RL posed by distribution shifting, (b) visual perturbation-invariant space between training and evaluation representations, (c) changes of variances of RGB and depth distributions under data augmentation. • We introduce an RL framework, leveraging depth input to reduce visual observation’s variation internally during training and evaluation, enhancing the trained… view at source ↗
Figure 3
Figure 3. The training pipeline of DeGuV consists three main components: Depth-Guided Masking module takes in depth inputs to create the mask that erases non-relevant pixels in RGB image, VPIR Learning module employs contrastive loss in Eq. 6 to facilitate the RL agent to extract invariant representations, and Stablized Q-Value Estimation is applied to stabilize the optimal policy training by avoiding non-stationary updates u… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Comparisons of episode returns of DeGuV compared to other baselines in four robotic tasks: Lift, Door, NutAssemblyRound, and TwoArmPegInHole, showing the our proposed method achieves better or similar data efficiency through the training process. Observation MaDi DeGuV…
Figure 5
Figure 5. Figure 5: Qualitative results of masked observations between MaDi and DeGuV from easy and hard of the Lift task. DeGuV [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Experiment set-up and results of original and masked observations for the Lift task on a real-robot system. C. Interpretability Given that the mask serves as a tool for interpreting the agent’s decision-making rationale in response to an observation, we conduct analyse…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

41 extracted references · 23 canonical work pages

  1. [1]

    Human-level atari 200x faster,

    S. Kapturowski, V . Campos, R. Jiang, N. Raki ´cevi´c, H. van Hasselt, C. Blundell, and A. P. Badia, “Human-level atari 200x faster,”arXiv preprint arXiv:2209.07550, 2022

    arXiv 2022

  2. [2]

    Fast and Data-Efficient Training of Rainbow: an Experimental Study on Atari

    D. Schmidt and T. Schmied, “Fast and data-efficient training of rainbow: an experimental study on atari,”arXiv preprint arXiv:2111.10247, 2021

    work page Pith review arXiv 2021

  3. [3]

    Agent57: Outperforming the atari human benchmark,

    A. P. Badia, B. Piot, S. Kapturowski, P. Sprechmann, A. Vitvitskyi, Z. D. Guo, and C. Blundell, “Agent57: Outperforming the atari human benchmark,” inInternational conference on machine learning. PMLR, 2020, pp. 507–517

    work page 2020

  4. [4]

    Learning to navigate in complex environments,

    P. Mirowski, R. Pascanu, F. Viola, H. Soyer, A. J. Ballard, A. Banino, M. Denil, R. Goroshin, L. Sifre, K. Kavukcuogluet al., “Learning to navigate in complex environments,”arXiv preprint arXiv:1611.03673, 2016

    arXiv 2016

  5. [5]

    Target-driven visual navigation in indoor scenes using deep reinforcement learning,

    Y . Zhu, R. Mottaghi, E. Kolve, J. J. Lim, A. Gupta, L. Fei-Fei, and A. Farhadi, “Target-driven visual navigation in indoor scenes using deep reinforcement learning,” in2017 IEEE international conference on robotics and automation (ICRA). IEEE, 2017, pp. 3357–3364

    2017

  6. [6]

    End-to- end urban driving by imitating a reinforcement learning coach,

    Z. Zhang, A. Liniger, D. Dai, F. Yu, and L. Van Gool, “End-to- end urban driving by imitating a reinforcement learning coach,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 15 222–15 232

    work page 2021

  7. [7]

    Solving rubik’s cube with a robot hand,

    I. Akkaya, M. Andrychowicz, M. Chociej, M. Litwin, B. McGrew, A. Petron, A. Paino, M. Plappert, G. Powell, R. Ribaset al., “Solving rubik’s cube with a robot hand,”arXiv preprint arXiv:1910.07113, 2019

    arXiv 1910

  8. [8]

    Learning agile soccer skills for a bipedal robot with deep reinforcement learning,

    T. Haarnoja, B. Moran, G. Lever, S. H. Huang, D. Tirumala, J. Hump- lik, M. Wulfmeier, S. Tunyasuvunakool, N. Y . Siegel, R. Hafner et al., “Learning agile soccer skills for a bipedal robot with deep reinforcement learning,”Science Robotics, vol. 9, no. 89, p. eadi8022, 2024

    work page 2024

Show all 41 references
  1. [9]

    Look closer: Bridging egocentric and third-person views with transformers for robotic manipulation,

    R. Jangir, N. Hansen, S. Ghosal, M. Jain, and X. Wang, “Look closer: Bridging egocentric and third-person views with transformers for robotic manipulation,”IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 3046–3053, 2022

    2022

  2. [10]

    A study on overfit- ting in deep reinforcement learning,

    C. Zhang, O. Vinyals, R. Munos, and S. Bengio, “A study on overfit- ting in deep reinforcement learning,”arXiv preprint arXiv:1804.06893, 2018

    2018 arXiv

  3. [11]

    MaDi: Learning to Mask Distractions for Generalization in Visual Deep Reinforcement Learning,

    B. Grooten, T. Tomilin, G. Vasan, M. E. Taylor, A. R. Mahmood, M. Fang, M. Pechenizkiy, and D. C. Mocanu, “MaDi: Learning to Mask Distractions for Generalization in Visual Deep Reinforcement Learning,”The 23rd International Conference on Autonomous Agents and Multiagent System...

    2024 arXiv

  4. [12]

    Stabilizing deep q-learning with convnets and vision transformers under data augmentation,

    N. Hansen, H. Su, and X. Wang, “Stabilizing deep q-learning with convnets and vision transformers under data augmentation,” inCon- ference on Neural Information Processing Systems, 2021

    2021

  5. [13]

    Dream to generalize: Zero-shot model- based reinforcement learning for unseen visual distractions,

    J. Ha, K. Kim, and Y . Kim, “Dream to generalize: Zero-shot model- based reinforcement learning for unseen visual distractions,” inPro- ceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 6, 2023, pp. 7802–7810

    2023

  6. [14]

    Look where you look! saliency-guided q-networks for generalization in visual reinforcement learning,

    D. Bertoin, A. Zouitine, M. Zouitine, and E. Rachelson, “Look where you look! saliency-guided q-networks for generalization in visual reinforcement learning,”Advances in Neural Information Processing Systems, vol. 35, pp. 30 693–30 706, 2022

    2022

  7. [15]

    Learn- ing to manipulate anywhere: A visual generalizable framework for reinforcement learning,

    Z. Yuan, T. Wei, S. Cheng, G. Zhang, Y . Chen, and H. Xu, “Learn- ing to manipulate anywhere: A visual generalizable framework for reinforcement learning,”arXiv preprint arXiv:2407.15815, 2024

    2024 arXiv

  8. [16]

    Au- tomatic data augmentation for generalization in reinforcement learn- ing,

    R. Raileanu, M. Goldstein, D. Yarats, I. Kostrikov, and R. Fergus, “Au- tomatic data augmentation for generalization in reinforcement learn- ing,”Advances in Neural Information Processing Systems, vol. 34, pp. 5402–5415, 2021

    2021

  9. [17]

    Generalization in reinforcement learning by soft data augmentation,

    N. Hansen and X. Wang, “Generalization in reinforcement learning by soft data augmentation,” in2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021, pp. 13 611–13 617

    2021

  10. [18]

    Visarl: Visual reinforcement learning guided by human saliency,

    A. Liang, J. Thomason, and E. Bıyık, “Visarl: Visual reinforcement learning guided by human saliency,”arXiv preprint arXiv:2403.10940, 2024

    2024 arXiv

  11. [19]

    Reinforcement learning with augmented data,

    M. Laskin, K. Lee, A. Stooke, L. Pinto, P. Abbeel, and A. Srinivas, “Reinforcement learning with augmented data,”Advances in neural information processing systems, vol. 33, pp. 19 884–19 895, 2020

    2020

  12. [20]

    Image augmentation is all you need: Regularizing deep reinforcement learning from pixels,

    I. Kostrikov, D. Yarats, and R. Fergus, “Image augmentation is all you need: Regularizing deep reinforcement learning from pixels,”arXiv preprint arXiv:2004.13649, 2020

    2004 arXiv

  13. [21]

    Rl-vigen: A reinforcement learning benchmark for visual generalization,

    Z. Yuan, S. Yang, P. Hua, C. Chang, K. Hu, and H. Xu, “Rl-vigen: A reinforcement learning benchmark for visual generalization,”Advances in Neural Information Processing Systems, vol. 36, 2024

    2024

  14. [22]

    Contrastive behavioral similarity embeddings for generalization in reinforcement learning,

    R. Agarwal, M. C. Machado, P. S. Castro, and M. G. Bellemare, “Contrastive behavioral similarity embeddings for generalization in reinforcement learning,”arXiv preprint arXiv:2101.05265, 2021

    2021 arXiv

  15. [23]

    Curl: Contrastive unsupervised representations for reinforcement learning,

    M. Laskin, A. Srinivas, and P. Abbeel, “Curl: Contrastive unsupervised representations for reinforcement learning,” inInternational confer- ence on machine learning. PMLR, 2020, pp. 5639–5650

    2020

  16. [24]

    Decomposing the generalization gap in imitation learning for visual robotic manipulation,

    A. Xie, L. Lee, T. Xiao, and C. Finn, “Decomposing the generalization gap in imitation learning for visual robotic manipulation,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 3153–3160

    2024

  17. [25]

    Domain adaptation in reinforcement learning via latent unified state representation,

    J. Xing, T. Nagata, K. Chen, X. Zou, E. Neftci, and J. L. Krichmar, “Domain adaptation in reinforcement learning via latent unified state representation,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 12, 2021, pp. 10 452–10 459

    2021

  18. [26]

    Pre- trained image encoder for generalizable visual reinforcement learning,

    Z. Yuan, Z. Xue, B. Yuan, X. Wang, Y . Wu, Y . Gao, and H. Xu, “Pre- trained image encoder for generalizable visual reinforcement learning,” Advances in Neural Information Processing Systems, vol. 35, pp. 13 022–13 037, 2022

    2022

  19. [27]

    Focus-then-decide: segmentation-assisted reinforcement learning,

    C. Chen, J. Xu, W. Liao, H. Ding, Z. Zhang, Y . Yu, and R. Zhao, “Focus-then-decide: segmentation-assisted reinforcement learning,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 10, 2024, pp. 11 240–11 248

    2024

  20. [28]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo, P. Doll ´ar, and R. Gir- shick, “Segment anything,”arXiv:2304.02643, 2023

    2023 arXiv

  21. [29]

    Mask-based latent reconstruction for reinforcement learning,

    T. Yu, Z. Zhang, C. Lan, Y . Lu, and Z. Chen, “Mask-based latent reconstruction for reinforcement learning,”Advances in Neural Infor- mation Processing Systems, vol. 35, pp. 25 117–25 131, 2022

    2022

  22. [30]

    Ignorance is bliss: Robust control via information gating,

    M. Tomar, R. Islam, M. Taylor, S. Levine, and P. Bachman, “Ignorance is bliss: Robust control via information gating,”Advances in Neural Information Processing Systems, vol. 36, 2024

    2024

  23. [31]

    An a* curriculum approach to reinforcement learning for rgbd indoor robot navigation,

    K. Balakrishnan, P. Chakravarty, and S. Shrivastava, “An a* curriculum approach to reinforcement learning for rgbd indoor robot navigation,” arXiv preprint arXiv:2101.01774, 2021

    2021 arXiv

  24. [32]

    Robotic grasping using deep reinforcement learning,

    S. Joshi, S. Kumra, and F. Sahin, “Robotic grasping using deep reinforcement learning,” in2020 IEEE 16th International Conference on Automation Science and Engineering (CASE). IEEE, 2020, pp. 1461–1466

    2020

  25. [33]

    A dqn-based autonomous car-following framework using rgb-d frames,

    H. Friji, H. Ghazzai, H. Besbes, and Y . Massoud, “A dqn-based autonomous car-following framework using rgb-d frames,” in2020 IEEE Global Conference on Artificial Intelligence and Internet of Things (GCAIoT). IEEE, 2020, pp. 1–6

    2020

  26. [34]

    Q-attention: Enabling efficient learning for vision-based robotic manipulation,

    S. James and A. J. Davison, “Q-attention: Enabling efficient learning for vision-based robotic manipulation,”IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 1612–1619, 2022

    2022

  27. [35]

    Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,

    T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,” inInternational conference on machine learning. Pmlr, 2018, pp. 1861–1870

    2018

  28. [36]

    Learning to predict by the methods of temporal differences,

    R. S. Sutton, “Learning to predict by the methods of temporal differences,”Machine learning, vol. 3, pp. 9–44, 1988

    1988

  29. [37]

    Mastering visual continuous control: Improved data-augmented reinforcement learning,

    D. Yarats, R. Fergus, A. Lazaric, and L. Pinto, “Mastering visual continuous control: Improved data-augmented reinforcement learning,” arXiv preprint arXiv:2107.09645, 2021

    2021 arXiv

  30. [38]

    Representation learning with contrastive predictive coding,

    A. v. d. Oord, Y . Li, and O. Vinyals, “Representation learning with contrastive predictive coding,”arXiv preprint arXiv:1807.03748, 2018

    2018 arXiv

  31. [39]

    robosuite: A modular simulation framework and benchmark for robot learning,

    Y . Zhu, J. Wong, A. Mandlekar, R. Mart ´ın-Mart´ın, A. Joshi, S. Nasiriany, Y . Zhu, and K. Lin, “robosuite: A modular simulation framework and benchmark for robot learning,” inarXiv preprint arXiv:2009.12293, 2020

    2009 arXiv

  32. [40]

    Simple copy-paste is a strong data augmentation method for instance segmentation,

    G. Ghiasi, Y . Cui, A. Srinivas, R. Qian, T.-Y . Lin, E. D. Cubuk, Q. V . Le, and B. Zoph, “Simple copy-paste is a strong data augmentation method for instance segmentation,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 2918–2928

    2021

  33. [41]

    Taming the panda with python: A powerful duo for seamless robotics programming and integration,

    J. Elsner, “Taming the panda with python: A powerful duo for seamless robotics programming and integration,”SoftwareX, vol. 24, p. 101532, 2023. [Online]. Available: https://www.sciencedirect.com/ science/article/pii/S2352711023002285

    2023

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.