Pith. sign in

REVIEW 4 major objections 5 minor 47 references

Equivariant Goal Conditioned Contrastive Reinforcement Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Encoding planar rotation symmetry into both the critic and the actor makes goal-conditioned contrastive RL learn faster and generalize to unseen goal positions.

desk verdict Solid but imperfect method paper for goal-conditioned CRL with a real symmetry-specification hole around orientation variables and weaker empirical claims than the abstract admits. read the letter →

arxiv 2507.16139 v1 pith:GC7E7KZ3 submitted 2025-07-22 cs.RO cs.LG

Arsh Tangri , Nichols Crawford Taylor , Haojie Huang , Robert Platt This is my paper
classification cs.ROcs.LG
keywords ReinforcementLearningContrastiveEquivarianceGoal-ConditionedRLRobotManipulationRotationInvarianceSampleEfficiencyRepresentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that rotation symmetry can be built directly into contrastive reinforcement learning for goal-conditioned robot manipulation, and that doing so buys sample efficiency and spatial generalization. It introduces Goal-Conditioned Group-Invariant MDPs, a formal setting where rotating the state, goal, and action together leaves the optimal value unchanged and rotates the optimal policy accordingly. On top of this, Equivariant Contrastive RL (ECRL) uses C_N-equivariant encoders whose regular-representation outputs make the critic's similarity score rotation-invariant while keeping the actor rotation-equivariant. The reported experiments claim ECRL consistently beats standard Contrastive RL and a hindsight-relabeling baseline across state-based and image-based simulated manipulation tasks, including offline settings with very few demonstrations. A sympathetic reader would care because it points to a recipe for transferring geometric structure in a task into learning speed without requiring reward design.

What carries the argument

The load-bearing mechanism is the regular representation of the cyclic group $C_N$. A $C_N$-equivariant encoder maps state-action pairs and goals to vectors whose $N$ elements are indexed by group elements; rotating the input by one step cyclically shifts the vector. When both encoders share this structure, the inner product (or $\ell^2$ distance) between the state-action embedding and the goal embedding is untouched by simultaneous rotation, making the critic rotation-invariant without discarding orientation. The actor is a second $C_N$-equivariant map that outputs mixed trivial- and standard-representation action features, so planar components rotate while gripper height, rotation angle, and open/close state stay fixed. The Goal-Conditioned Group-Invariant MDP formalizes which environments admit this construction.

What would settle it

One concrete test is to measure the success-rate gap between ECRL and CRL as the environment's rotational symmetry is progressively broken, for example by adding a fixed obstacle or tilting the camera further from top-down; if the gap does not shrink as the perturbation grows, the claim that symmetry awareness causes the gains is falsified. A second check is to evaluate a trained ECRL critic on many random rotations and compare f(gs, ga, gg) with f(s, a, g); a large mismatch would show the assumed invariance is not actually enforced.

Watch

Extended reading notes

Core claim

The paper's central claim is that goal-conditioned manipulation has a rotational symmetry that can be made algorithmic. It defines a Goal-Conditioned Group-Invariant MDP and proves (Proposition 1) that in such an MDP the optimal Q-function is group-invariant, $Q^*(gs, ga, gg) = Q^*(s, a, g)$, and the optimal policy is group-equivariant, $\pi^*(gs, gg) = g\pi^*(s, g)$. On this basis it constructs Equivariant Contrastive RL: a critic whose state-action and goal encoders are $C_N$-equivariant networks outputting regular-representation feature vectors, so rotating both inputs by the same group element permutes the two embeddings in the same way and leaves inner-product or $\ell^2$ similarity unchanged, paired with a $C_N$-equivariant actor whose planar action components rotate with the input. The experiments claim this architecture consistently outperforms non-equivariant Contrastive RL and a hindsight-relabeling baseline on state-based and image-based manipulation tasks, and also in offline RL with small datasets.

Load-bearing premise

The entire method assumes that the robot task is genuinely unchanged when state, goal, and action are rotated together, and that rotating the camera image by the same amount accurately encodes that symmetry; the authors note the camera is 'slightly skewed rather than strictly top-down,' so the symmetry is only partially present in image observations.

Editorial extensions

If this is right

  • In state-based experiments, ECRL reaches higher success rates with fewer environment interactions than non-equivariant Contrastive RL and the TD3+HER baseline on all of the manipulation tasks considered.
  • In image-based experiments, the same pattern holds even though the camera is slightly skewed rather than strictly top-down, so the encoded symmetry is only partially present in the observations.
  • When training goals are restricted to one quadrant and evaluation goals are drawn from the full workspace, ECRL generalizes better and shows lower variance than CRL.
  • In offline RL with 5 to 50 demonstrations, ECRL outperforms CRL on every task and dataset size tested.
  • An ablation that replaces equivariant representations with global pooling shows the equivariant critic is much more data-efficient, indicating that preserving orientation information matters.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An extension the paper leaves implicit: the same critic construction should transfer to other discrete groups, such as the dihedral group for rotations plus reflections or discretized 3D rotations, and the paper's own appendix shows preliminary evidence for both.
  • A testable prediction that goes beyond the paper: because image experiments succeed despite a slightly skewed camera, ECRL should retain part of its advantage in real-world setups where symmetry is only approximate, with the advantage shrinking as the actual transform moves further from a true rotation.
  • The ablation against global pooling suggests the orientation information in the embeddings is doing real work, so one could probe this further by testing a task where goal orientation is irrelevant; the equivariant critic's advantage should shrink or disappear in that case.
  • The performance gains appear orthogonal to the choice of contrastive loss, so the equivariant critic may also combine with other representation-learning objectives that rely on elementwise operations across embeddings.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Equivariant Contrastive RL (ECRL), which injects cyclic-group equivariance into Contrastive RL for goal-conditioned manipulation. It formalizes a Goal-Conditioned Group-Invariant MDP, proves in Proposition 1 that the optimal Q-function is group-invariant and the optimal policy is group-equivariant, and builds a rotation-invariant critic from C_N-equivariant encoders operating on regular representations, paired with a rotation-equivariant actor. The empirical section evaluates ECRL against CRL and TD3+HER on state-based and image-based tasks from the Fetch, Meta-World, and BulletArm suites, reports gains in sample efficiency and final success rate, extends the method to offline RL, and includes ablations over similarity metric, group pooling, group order, and number of regular representations, as well as an unseen-goal generalization study.

Significance. If the claims hold, the paper makes a useful and well-motivated contribution: it connects equivariance to contrastive RL and provides a simple, algebraically clean construction of an invariant critic. The invariance argument itself is correct — aligned cyclic permutations of regular-representation vectors make both inner-product and L2 similarities exactly invariant — and Proposition 1 follows from existing equivariant-MDP theory, so the theoretical scaffolding is sound. The empirical breadth is also a strength: multiple benchmarks, state and image observations, online and offline settings, an InfoNCE variant, and a generalization-to-unseen-goals study. However, the formal guarantees transfer to the experiments only if the group action encoded in the networks is a true symmetry of the actual MDPs, and that is not established for tasks with orientation coordinates. In addition, the headline empirical claim currently lacks uncertainty quantification, and at least one experimental protocol (BlockPick) is under-specified. These issues are load-bearing for the central claims but appear addressable.

major comments (4)
  1. [Sec. 4.2 / Eq. (5) and Sec. 4.3] The group action defined for the action vector (x,y,z,θ,w) leaves the gripper rotation angle θ invariant, and the factored state space implicitly treats orientation coordinates as ρ0-invariant. For a physical planar rotation about the vertical axis, the gripper yaw and any object orientation/rotation coordinates transform by the same rotation angle; they are not invariant scalars. Unless the action and state coordinates are expressed in a relative or body frame in which these quantities are genuinely invariant — which the manuscript does not state — the encoded group action is not a symmetry of the actual Fetch/BulletArm MDPs, and Proposition 1 does not imply that the learned critic is Q* or that the learned policy is π* for those MDPs. Please either specify which state and goal components are assigned to S_inv and S_equi, include orientation in the group action (e.g., θ -> θ + 2πi/N for C_N), or restrict the formal claims to tasks without orientation degrees of freedom. The camera-skew caveat in Sec. 5 addresses only the image observation transform (Eq. 4) and does not resolve this mismatch.
  2. [Figs. 3-6 and 10-12] All learning curves are reported as means over 3 or 4 seeds without error bars, confidence intervals, or statistical significance tests. The central claim that ECRL 'consistently outperforms' strong baselines rests on curve separation that is visually plausible in some tasks (e.g., FetchPush, FetchPickAndPlace) but is small or even reversed in others (e.g., SawyerBin in Fig. 4, where CRL with L2 is reported best). Please add per-seed bands or 95% confidence intervals to the learning curves and report a simple significance test over seeds (for example, on final success rate or area under the learning curve), with the number of seeds stated for each figure.
  3. [Sec. 5, BulletArm BlockPick paragraph] The text states that for BlockPick 'we provide five expert demonstrations and include a behavioral cloning loss on the actor.' It is not stated whether the CRL and TD3+HER baselines also receive these demonstrations and the behavioral-cloning loss. If only ECRL receives this additional supervision, the comparison is confounded. Please clarify the protocol and, if the baselines did not receive the demonstrations, add an ablated CRL baseline trained with the same demonstration and BC-loss treatment.
  4. [Appendix B.1 / Table 3] In the offline experiments, CRL and ECRL are trained with different hyperparameters: batch size 1024 vs. 512, hidden layer sizes (1024,1024) vs. (256,256), and representation dimension 16 for both. While Table 3 notes these are modified from recommended settings, the comparison does not isolate equivariance because capacity and batch size differ between the two methods. Please justify the protocol or run a matched-capacity comparison (for example, same hidden sizes and batch size for both methods), so that the offline gains can be attributed to the equivariant structure rather than to hyperparameter choice.
minor comments (5)
  1. [Sec. 4.3 title] The section title contains a typo: 'Equivariant Contrative RL' should be 'Equivariant Contrastive RL'; also 'formulaton' in Sec. 4.2 should be 'formulation'.
  2. [Appendix A, Eqs. (7)-(8)] The augmented MDP's transition and reward functions are written as p(hat{s}'|hat{s},a) and r(hat{s},a) without being formally defined in terms of the original goal-conditioned MDP; please define them explicitly, e.g., p_hat((s',g')|(s,g),a) = p(s'|s,a) delta(g'=g) and r_hat((s,g),a) = r_g(s,a).
  3. [Sec. 3 and 4] The notation G is used both for the symmetry group and for the goal space; the paper acknowledges this but the dual use remains confusing in Proposition 1 and the GCGI-MDP definition. Consider using a different symbol for the goal space.
  4. [Fig. 6 ablation] The pooled-invariant critic has N times fewer output scalars than the equivariant critic for the same K, so the ablation conflates representation type with capacity; a matched-capacity pooled-invariant baseline would strengthen the conclusion that pooling, not parameter count, is responsible for the performance drop.
  5. [Appendix F.2] The sentence 'this drop is statistically insignificant' is not supported by any test; either add a significance test over seeds or remove the claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Proposition 1 follows from an external theorem and the invariant critic is an algebraic construction, while the sample-efficiency gains are empirical.

full rationale

The paper's derivation chain is not circular. Proposition 1 is not a fitted quantity renamed as a prediction: Appendix A reduces it to the standard G-invariant-MDP optimality theorem of Wang et al. [9] by augmenting the state with the goal, and verifies the required transition and reward invariance (Eqs. 7-8). The cited theorem is parameter-free, is stated for ordinary G-invariant MDPs, and does not assume the goal-conditioned conclusion, so even though [9] shares an author with the present paper, it constitutes independent mathematical evidence rather than a circular self-citation. The rotation-invariant critic is likewise constructed algebraically: Eq. 6 is satisfied because regular-representation feature vectors are cyclically permuted under the group action, and both inner-product and l2 similarities are unchanged under aligned permutations (Sec. 4.3, Figure 1); no parameter is fitted to force this invariance. The claimed sample-efficiency and generalization improvements are empirical results (Figures 3-6, 9-10), not derivations from the model, so there is no prediction that reduces to an input by construction. The paper's explicit limitations - the Sec. 5 note that the image camera is 'slightly skewed rather than strictly top-down' and the Sec. 6 observation that benefits diminish in SawyerBin - are symmetry-validity and task-suitability caveats, not circular steps; likewise, the ground-truth-reward HER relabeling described in Appendix B.2 is a baseline fairness choice. No load-bearing step equates the paper's conclusions with its inputs by definition.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claims rest on the assumed rotation symmetry of the environments, on the standard equivariant RL theorem, and on the CRL formulation from prior work. Architecture choices such as N and K are tuned rather than derived, and the l2 similarity has two learned scalar parameters. No new physical entities are introduced.

free parameters (4)
  • L2 similarity scale a = not reported (learned)
    Used in the l2 variant f = -a * ||phi-psi||^2 + b; jointly optimized with the critic, adds capacity but is not required for the invariance property.
  • L2 similarity offset b = not reported (learned)
    Shift term in the l2 similarity metric; learned during critic training.
  • Cyclic group order N = 8
    Chosen as a discrete approximation of SO(2); ablation in Appendix F.2 shows N=4, 8, 16 perform comparably.
  • Number of regular representations K = 64
    Embedding width selected by ablation in Appendix F.1; K=32 underperforms and K=128 gives only marginal gains.
assumptions (4)
  • domain assumption The environment transition and reward are invariant under C8 planar rotations applied jointly to state, action, and goal.
    Section 4.2 assumes tabletop manipulation tasks can be modeled as goal-conditioned C8-invariant MDPs; image observations are only approximately aligned with this symmetry because the camera is skewed.
  • standard math Optimal Q and policy of an invariant MDP are invariant and equivariant, respectively (Proposition 4.1 of Wang et al. [9]).
    Used in Appendix A to prove Proposition 1; accepted prior theorem in the equivariant RL literature.
  • domain assumption The contrastive NCE critic f approximates the goal-conditioned Q-function.
    Taken from Eysenbach et al. [5]; the method trains f with binary NCE loss and the actor maximizes f as a proxy for Q.
  • domain assumption The action parametrization decomposes into equivariant planar components and invariant components, with the gripper angle treated as invariant.
    Equation 5 in Section 4.2; if the gripper angle is an absolute orientation rather than a relative displacement, this symmetry map could be misspecified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Equivariant Goal Conditioned Contrastive Reinforcement Learning." pith.science (2026). https://pith.science/paper/GC7E7KZ3

@misc{pith2026250716139,
  author       = {Pith},
  title        = {Pith review of: Equivariant Goal Conditioned Contrastive Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GC7E7KZ3}},
  note         = {Machine review of arXiv:2507.16139}
}
read the original abstract

Contrastive Reinforcement Learning (CRL) provides a promising framework for extracting useful structured representations from unlabeled interactions. By pulling together state-action pairs and their corresponding future states, while pushing apart negative pairs, CRL enables learning nontrivial policies without manually designed rewards. In this work, we propose Equivariant CRL (ECRL), which further structures the latent space using equivariant constraints. By leveraging inherent symmetries in goal-conditioned manipulation tasks, our method improves both sample efficiency and spatial generalization. Specifically, we formally define Goal-Conditioned Group-Invariant MDPs to characterize rotation-symmetric robotic manipulation tasks, and build on this by introducing a novel rotation-invariant critic representation paired with a rotation-equivariant actor for Contrastive RL. Our approach consistently outperforms strong baselines across a range of simulated tasks in both state-based and image-based settings. Finally, we extend our method to the offline RL setting, demonstrating its effectiveness across multiple tasks.

Figures

Figures reproduced from arXiv: 2507.16139 by the authors.

Figure 1
Figure 1. Rotation-invariant Critic The above figure illustrates our rotation-invariant critic, using the C4 group and two channels for simplicity. Each network outputs a stack of two 4-dimensional regular representations of the C4 group. Rotating both the state-action pair (s, a) and the goal g induces aligned cyclic permutations of each regular-representation vector in their embeddings, but the inner product remains unchang… view at source ↗
Figure 2
Figure 2. Simulation environments: (a) Sawyer Bin, (b) Sawyer Push, (c) Fetch Pick And Place, (d) [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Comparison of Equivariant and Non-Equivariant Goal-Conditioned Contrastive RL on [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Comparison of Equivariant and Non-Equivariant Goal-Conditioned Contrastive RL on [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Equivariant contrastive learning consistently outperforms standard contrastive learning on [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Achieving invariance through equivariant representations has much higher data efficiency [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Comparison of TD3+HER, CRL, and ECRL on Image-Based Tasks. Both CRL and ECRL [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Comparison of TD3+HER with the restricted relabeling strategy on state-based tasks. The [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Generalization to unseen goals sampled outside the training quadrant. ECRL general [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: Comparison of ECRL and CRL using the InfoNCE loss across various tasks. ECRL [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: Ablation on the number of regular representations [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: Ablation on the cyclic group order N in ECRL. N = 4, N = 8, and N = 16 perform comparably, with N = 8 used in the main paper. Results are averaged over 3 random seeds [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]
Figure 13
Figure 13. Figure 13: Demonstration of ECRL with different symmetry groups. We show results for [PITH_FULL_IMAGE:figures/full_fig_p017_13.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

47 extracted references · 24 canonical work pages

  1. [1]

    Eysenbach, A

    B. Eysenbach, A. Gupta, J. Ibarz, and S. Levine. Diversity is All You Need: Learning Skills without a Reward Function. CoRR, abs/1802.06070, 2018. URL http://arxiv.org/abs/ 1802.06070. arXiv: 1802.06070

    arXiv 2018

  2. [2]

    Y . Kim, W. Nam, H. Kim, J.-H. Kim, and G. Kim. Curiosity-Bottleneck: Exploration By Distilling Task-Specific Novelty. In K. Chaudhuri and R. Salakhutdinov, editors, Proceed- ings of the 36th International Conference on Machine Learning , volume 97 of Proceed- ings of Machine Learning Research , pages 3379–3388. PMLR, June 2019. URL https: //proceedings.m...

    work page 2019

  3. [3]

    S. Park, O. Rybkin, and S. Levine. METRA: Scalable Unsupervised RL with Metric-Aware Abstraction. In The Twelfth International Conference on Learning Representations , 2024. URL https://openreview.net/forum?id=c5pwL0Soay

    work page 2024

  4. [4]

    Zheng, J

    C. Zheng, J. Tuyls, J. Peng, and B. Eysenbach. Can a MISL Fly? Analysis and Ingredients for Mutual Information Skill Learning. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=xoIeVdFO7U

    work page 2025

  5. [5]

    Eysenbach, T

    B. Eysenbach, T. Zhang, S. Levine, and R. R. Salakhutdinov. Contrastive learning as goal- conditioned reinforcement learning. Advances in Neural Information Processing Systems , 35: 35603–35620, 2022

    work page 2022

  6. [6]

    M. Jia, D. Wang, G. Su, D. Klee, X. Zhu, R. Walters, and R. Platt. SEIL: Simulation-augmented Equivariant Imitation Learning. In 2023 IEEE International Conference on Robotics and Au- tomation (ICRA), pages 1845–1851. IEEE, 2023

    work page 2023

  7. [7]

    Huang, D

    H. Huang, D. Wang, A. Tangri, R. Walters, and R. Platt. Leveraging symmetries in pick and place. The International Journal of Robotics Research , 43(4):550–571, 2024

    work page 2024

  8. [8]

    D. Wang, S. Hart, D. Surovik, T. Kelestemur, H. Huang, H. Zhao, M. Yeatman, J. Wang, R. Walters, and R. Platt. Equivariant Diffusion Policy. In 8th Annual Conference on Robot Learning, 2024. URL https://openreview.net/forum?id=wD2kUVLT1g

    work page 2024

Show all 47 references
  1. [9]

    D. Wang, R. Walters, and R. Platt. \mathrmSO(2)-Equivariant Reinforcement Learning. In International Conference on Learning Representations , 2022. URL https://openreview. net/forum?id=7F9cOhdvfk_

    2022

  2. [10]

    M. Liu, M. Zhu, and W. Zhang. Goal-conditioned reinforcement learning: Problems and solutions. arXiv preprint arXiv:2201.08299, 2022

    2022 arXiv

  3. [11]

    Chane-Sane, C

    E. Chane-Sane, C. Schmid, and I. Laptev. Goal-Conditioned Reinforcement Learning with Imagined Subgoals. In M. Meila and T. Zhang, editors, Proceedings of the 38th Interna- tional Conference on Machine Learning , volume 139 of Proceedings of Machine Learning Research, pages 14...

    2021

  4. [12]

    Steccanella and A

    L. Steccanella and A. Jonsson. State representation learning for goal-conditioned reinforce- ment learning. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 84–99. Springer, 2022

    2022

  5. [13]

    Andrychowicz, F

    M. Andrychowicz, F. Wolski, A. Ray, J. Schneider, R. Fong, P. Welinder, B. McGrew, J. Tobin, P. Abbeel, and W. Zaremba. Hindsight Experience Replay. CoRR, abs/1707.01495, 2017. URL http://arxiv.org/abs/1707.01495. arXiv: 1707.01495

    2017 arXiv

  6. [14]

    X. Lin, H. S. Baweja, and D. Held. Reinforcement Learning without Ground-Truth State. CoRR, abs/1905.07866, 2019. URL http://arxiv.org/abs/1905.07866. arXiv: 1905.07866

    1905 arXiv

  7. [15]

    Y . Ding, C. Florensa, M. Phielipp, and P. Abbeel. Goal-conditioned Imitation Learning.CoRR, abs/1906.05838, 2019. URL http://arxiv.org/abs/1906.05838. arXiv: 1906.05838. 9

    1906 arXiv

  8. [16]

    Lynch, M

    C. Lynch, M. Khansari, T. Xiao, V . Kumar, J. Tompson, S. Levine, and P. Sermanet. Learning Latent Plans from Play. CoRR, abs/1903.01973, 2019. URL http://arxiv.org/abs/1903. 01973. arXiv: 1903.01973

    1903 arXiv

  9. [17]

    Ghosh, A

    D. Ghosh, A. Gupta, J. Fu, A. Reddy, C. Devin, B. Eysenbach, and S. Levine. Learning To Reach Goals Without Reinforcement Learning. CoRR, abs/1912.06088, 2019. URL http: //arxiv.org/abs/1912.06088. arXiv: 1912.06088

    1912 arXiv

  10. [18]

    R. K. Srivastava, P. Shyam, F. Mutz, W. Ja´skowski, and J. Schmidhuber. Training agents using upside-down reinforcement learning. arXiv preprint arXiv:1912.02877, 2019

    1912 arXiv

  11. [19]

    Eysenbach, R

    B. Eysenbach, R. Salakhutdinov, and S. Levine. C-Learning: Learning to Achieve Goals via Recursive Classification. CoRR, abs/2011.08909, 2020. URL https://arxiv.org/abs/ 2011.08909. arXiv: 2011.08909

    2011 arXiv

  12. [20]

    Zheng, B

    C. Zheng, B. Eysenbach, H. Walke, P. Yin, K. Fang, R. Salakhutdinov, and S. Levine. Sta- bilizing contrastive rl: Techniques for robotic goal reaching from offline data. arXiv preprint arXiv:2306.03346, 2023

    2023 arXiv

  13. [21]

    Zheng, R

    C. Zheng, R. Salakhutdinov, and B. Eysenbach. Contrastive difference predictive coding.arXiv preprint arXiv:2310.20141, 2023

    2023

  14. [22]

    G. Liu, M. Tang, and B. Eysenbach. A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoals, 2024. URL https://arxiv.org/abs/2408.05804. eprint: 2408.05804

    2024 arXiv

  15. [23]

    Bortkiewicz, W

    M. Bortkiewicz, W. Pałucki, V . Myers, T. Dziarmaga, T. Arczewski, Ł. Kuci´nski, and B. Ey- senbach. Accelerating Goal-Conditioned RL Algorithms and Research. arXiv preprint arXiv:2408.11052, 2024

    2024

  16. [24]

    D. Wang, R. Walters, X. Zhu, and R. Platt. Equivariant \Q\ Learning in Spatial Action Spaces. In 5th Annual Conference on Robot Learning , 2021. URL https://openreview. net/forum?id=IScz42A3iCI

    2021

  17. [25]

    D. Wang, J. Y . Park, N. Sortur, L. L. Wong, R. Walters, and R. Platt. The surprising effective- ness of equivariant models in domains with latent symmetry.arXiv preprint arXiv:2211.09231, 2022

    2022 arXiv

  18. [26]

    Weiler and G

    M. Weiler and G. Cesa. General E(2)-Equivariant Steerable CNNs. In Conference on Neural Information Processing Systems (NeurIPS), 2019

    2019

  19. [27]

    Jenner and M

    E. Jenner and M. Weiler. Steerable Partial Differential Operators for Equivariant Neural Networks. In International Conference on Learning Representations , 2022. URL https: //openreview.net/forum?id=N9W24a4zU

    2022

  20. [28]

    X. Zhu, D. Wang, G. Su, O. Biza, R. Walters, and R. Platt. On Robot Grasp Learning Using Equivariant Models. Autonomous Robots, 2023

    2023

  21. [29]

    Tangri, O

    A. Tangri, O. Biza, D. Wang, D. Klee, O. Howell, and R. Platt. Equivariant offline reinforce- ment learning. arXiv preprint arXiv:2406.13961, 2024

    2024 arXiv

  22. [30]

    Y . J. Ma, S. Sodhani, D. Jayaraman, O. Bastani, V . Kumar, and A. Zhang. Vip: Towards universal visual reward and representation via value-implicit pre-training. arXiv preprint arXiv:2210.00030, 2022

    2022 arXiv

  23. [31]

    S. Park, T. Kreiman, and S. Levine. Foundation policies with hilbert representations. arXiv preprint arXiv:2402.15567, 2024

    2024 arXiv

  24. [32]

    S. Nair, A. Rajeswaran, V . Kumar, C. Finn, and A. Gupta. R3m: A universal visual represen- tation for robot manipulation. arXiv preprint arXiv:2203.12601, 2022

    2022 arXiv

  25. [33]

    J. Y . Park, O. Biza, L. Zhao, J. W. van de Meent, and R. Walters. Learning Symmetric Rep- resentations for Equivariant World Model. In International Conference on Machine Learning ,

  26. [34]

    A. v. d. Oord, Y . Li, and O. Vinyals. Representation Learning with Contrastive Predictive Coding. CoRR, abs/1807.03748, 2018. URL http://arxiv.org/abs/1807.03748. arXiv: 1807.03748

    2018 arXiv

  27. [35]

    R. D. Hjelm, A. Fedorov, S. Lavoie-Marchildon, K. Grewal, P. Bachman, A. Trischler, and Y . Bengio. Learning deep representations by mutual information estimation and maximization,

  28. [36]

    Ma and M

    Z. Ma and M. Collins. Noise Contrastive Estimation and Negative Sampling for Conditional Models: Consistency and Statistical Efficiency. CoRR, abs/1809.01812, 2018. URL http: //arxiv.org/abs/1809.01812. arXiv: 1809.01812

    2018 arXiv

  29. [37]

    Haarnoja, A

    T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine. Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. CoRR, abs/1801.01290, 2018. URL http://arxiv.org/abs/1801.01290. arXiv: 1801.01290

    2018 arXiv

  30. [38]

    Fujimoto and S

    S. Fujimoto and S. S. Gu. A minimalist approach to offline reinforcement learning. Advances in neural information processing systems , 34:20132–20145, 2021

    2021

  31. [39]

    D. Wang, C. Kohler, X. Zhu, M. Jia, and R. Platt. Bulletarm: An open-source robotic ma- nipulation benchmark and learning framework. In The International Symposium of Robotics Research, pages 335–350. Springer, 2022

    2022

  32. [40]

    Todorov, T

    E. Todorov, T. Erez, and Y . Tassa. MuJoCo: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems , pages 5026–

    2012

  33. [41]

    T. Yu, D. Quillen, Z. He, R. Julian, K. Hausman, C. Finn, and S. Levine. Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning. CoRR, abs/1910.10897, 2019. URL http://arxiv.org/abs/1910.10897. arXiv: 1910.10897

    1910 arXiv

  34. [42]

    J. Park, J. Lee, I.-J. Kim, and K. Sohn. Probabilistic representations for video contrastive learn- ing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 14711–14721, 2022

    2022

  35. [43]

    T. Wang, A. Torralba, P. Isola, and A. Zhang. Optimal goal-reaching reinforcement learning via quasimetric learning. In International Conference on Machine Learning, pages 36411–36430. PMLR, 2023

    2023

  36. [44]

    G. Cesa, L. Lang, and M. Weiler. A Program to Build E(N)-Equivariant Steerable CNNs. In International Conference on Learning Representations , 2022. URL https://openreview. net/forum?id=WE4qe9xlnQw. 11 7 Appendix A Proof First, we define a new MDP, ˆM, with state space ˆS := S...

    2022

  37. [2019]

    eprint: 1808.06670

    URL https://arxiv.org/abs/1808.06670. eprint: 1808.06670

    arXiv

  38. [2022]

    URL https://arxiv.org/abs/2204.11371. 10

    arXiv

  39. [5033]

    doi:10.1109/IROS.2012.6386109

    IEEE, 2012. doi:10.1109/IROS.2012.6386109

    2012

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.