{"id":"7e9a6a65-2d12-47ca-849d-6eec107c073f","arxiv_id":"2507.16139","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Equivariant Contrastive RL imposes C8 rotation symmetry on the critic and actor, improving sample efficiency and goal generalization in simulated manipulation.","lead":"This paper adds rotation symmetry to contrastive reinforcement learning, so a robot's value function stays the same and its actions rotate when the scene rotates. In simulations, this makes goal-reaching and manipulation policies learn faster and generalize to new goal directions.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed invariance is w.r.t. a possibly mis-specified group action: orientation coordinates (gripper yaw θ, object quaternions) are treated as invariant in Sec. 4.2, so Proposition 1 may not apply to the actual Fetch/BulletArm MDPs.","rationale":"The reader's weakest assumption was partial symmetry in image observations (slightly skewed camera). My concern is adjacent but more specific and more load-bearing: even in state-based tasks, the prescribed group action may be wrong for orientation-valued state/action coordinates. The paper explicitly states that θ remains trivial under rotation, which is physically incorrect if θ is a gripper yaw angle. Since the invariant critic is constructed by enforcing Eq. 6 under this prescribed action, a wrong group action would mean the theoretical guarantee does not apply to the environments studied, and the reported gains are not explained by the stated mechanism. This is not an ad hominem or a disagreement with equivariant RL in general; it is a checkable technical condition. I do not think the paper should be rejected on this basis, because the authors may have reduced states to positions-only vectors or may handle θ differently in code, and the empirical results could still hold. But the claim as written is under-specified at exactly the point where correctness is load-bearing. The reader's CONDITIONAL verdict already requires additional evidence, and my concern strengthens that requirement without changing the verdict category.","tokens_in":16135,"tokens_out":12788,"duration_ms":156988,"concrete_test":"Obtain or reconstruct the exact feature-field assignment from the implementation: for every component of state and action in FetchPickAndPlace and BulletArm BlockPick, record whether it is fed to the C8-equivariant network as ρ0 or ρ1. Then, with a trained critic, take a random (s,a,g), apply a 45° ground-truth physical rotation to all coordinates including object quaternion and gripper yaw, and compare f(s,a,g) with f(gs,ga,gg). If the relative difference exceeds a small tolerance, the critic is not invariant under the true symmetry and the Proposition 1 justification fails for those tasks. A complementary test is to re-train with orientation components correctly placed in equivariant fields and check whether ECRL's margin over CRL persists.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central construction assumes a group action on the factored state/action space in which only the planar position components transform under ρ1 and all other components—including, per Sec. 4.2, the gripper rotation angle θ in (x,y,z,θ,w)—are ρ0-invariant. For a true planar rotation about the vertical axis, the yaw of the gripper and the yaw/quaternion of a manipulated object transform nontrivially (they shift by the rotation angle). Therefore the group action used to build the C8-equivariant encoders may not be the symmetry of the actual MDP, and the invariant-critic property (Eq. 6) plus Proposition 1 would not guarantee an invariant Q* for the real Fetch/BulletArm state spaces, which include orientation coordinates. The empirical gains on tasks where orientation matters (FetchPickAndPlace, BulletArm BlockPick) could then stem from capacity or from accidentally ignoring a nuisance variable rather than from exploiting the true symmetry. The camera-skew caveat in Sec. 5 is the paper's only explicit symmetry caveat, but the state/action feature-field assignment is a more fundamental place where the encoded symmetry can fail, and it is not discussed. If the implemented state/action vectors actually exclude orientation variables, this concern is vacuous; the check below determines which is the case.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Equivariant Contrastive RL (ECRL), which injects cyclic-group equivariance into Contrastive RL for goal-conditioned manipulation. It formalizes a Goal-Conditioned Group-Invariant MDP, proves in Proposition 1 that the optimal Q-function is group-invariant and the optimal policy is group-equivariant, and builds a rotation-invariant critic from C_N-equivariant encoders operating on regular representations, paired with a rotation-equivariant actor. The empirical section evaluates ECRL against CRL and TD3+HER on state-based and image-based tasks from the Fetch, Meta-World, and BulletArm suites, reports gains in sample efficiency and final success rate, extends the method to offline RL, and includes ablations over similarity metric, group pooling, group order, and number of regular representations, as well as an unseen-goal generalization study.","tokens_in":16381,"tokens_out":11233,"duration_ms":136914,"significance":"If the claims hold, the paper makes a useful and well-motivated contribution: it connects equivariance to contrastive RL and provides a simple, algebraically clean construction of an invariant critic. The invariance argument itself is correct — aligned cyclic permutations of regular-representation vectors make both inner-product and L2 similarities exactly invariant — and Proposition 1 follows from existing equivariant-MDP theory, so the theoretical scaffolding is sound. The empirical breadth is also a strength: multiple benchmarks, state and image observations, online and offline settings, an InfoNCE variant, and a generalization-to-unseen-goals study. However, the formal guarantees transfer to the experiments only if the group action encoded in the networks is a true symmetry of the actual MDPs, and that is not established for tasks with orientation coordinates. In addition, the headline empirical claim currently lacks uncertainty quantification, and at least one experimental protocol (BlockPick) is under-specified. These issues are load-bearing for the central claims but appear addressable.","major_comments":[{"comment":"The group action defined for the action vector (x,y,z,θ,w) leaves the gripper rotation angle θ invariant, and the factored state space implicitly treats orientation coordinates as ρ0-invariant. For a physical planar rotation about the vertical axis, the gripper yaw and any object orientation/rotation coordinates transform by the same rotation angle; they are not invariant scalars. Unless the action and state coordinates are expressed in a relative or body frame in which these quantities are genuinely invariant — which the manuscript does not state — the encoded group action is not a symmetry of the actual Fetch/BulletArm MDPs, and Proposition 1 does not imply that the learned critic is Q* or that the learned policy is π* for those MDPs. Please either specify which state and goal components are assigned to S_inv and S_equi, include orientation in the group action (e.g., θ -> θ + 2πi/N for C_N), or restrict the formal claims to tasks without orientation degrees of freedom. The camera-skew caveat in Sec. 5 addresses only the image observation transform (Eq. 4) and does not resolve this mismatch.","section":"Sec. 4.2 / Eq. (5) and Sec. 4.3"},{"comment":"All learning curves are reported as means over 3 or 4 seeds without error bars, confidence intervals, or statistical significance tests. The central claim that ECRL 'consistently outperforms' strong baselines rests on curve separation that is visually plausible in some tasks (e.g., FetchPush, FetchPickAndPlace) but is small or even reversed in others (e.g., SawyerBin in Fig. 4, where CRL with L2 is reported best). Please add per-seed bands or 95% confidence intervals to the learning curves and report a simple significance test over seeds (for example, on final success rate or area under the learning curve), with the number of seeds stated for each figure.","section":"Figs. 3-6 and 10-12"},{"comment":"The text states that for BlockPick 'we provide five expert demonstrations and include a behavioral cloning loss on the actor.' It is not stated whether the CRL and TD3+HER baselines also receive these demonstrations and the behavioral-cloning loss. If only ECRL receives this additional supervision, the comparison is confounded. Please clarify the protocol and, if the baselines did not receive the demonstrations, add an ablated CRL baseline trained with the same demonstration and BC-loss treatment.","section":"Sec. 5, BulletArm BlockPick paragraph"},{"comment":"In the offline experiments, CRL and ECRL are trained with different hyperparameters: batch size 1024 vs. 512, hidden layer sizes (1024,1024) vs. (256,256), and representation dimension 16 for both. While Table 3 notes these are modified from recommended settings, the comparison does not isolate equivariance because capacity and batch size differ between the two methods. Please justify the protocol or run a matched-capacity comparison (for example, same hidden sizes and batch size for both methods), so that the offline gains can be attributed to the equivariant structure rather than to hyperparameter choice.","section":"Appendix B.1 / Table 3"}],"minor_comments":[{"comment":"The section title contains a typo: 'Equivariant Contrative RL' should be 'Equivariant Contrastive RL'; also 'formulaton' in Sec. 4.2 should be 'formulation'.","section":"Sec. 4.3 title"},{"comment":"The augmented MDP's transition and reward functions are written as p(hat{s}'|hat{s},a) and r(hat{s},a) without being formally defined in terms of the original goal-conditioned MDP; please define them explicitly, e.g., p_hat((s',g')|(s,g),a) = p(s'|s,a) delta(g'=g) and r_hat((s,g),a) = r_g(s,a).","section":"Appendix A, Eqs. (7)-(8)"},{"comment":"The notation G is used both for the symmetry group and for the goal space; the paper acknowledges this but the dual use remains confusing in Proposition 1 and the GCGI-MDP definition. Consider using a different symbol for the goal space.","section":"Sec. 3 and 4"},{"comment":"The pooled-invariant critic has N times fewer output scalars than the equivariant critic for the same K, so the ablation conflates representation type with capacity; a matched-capacity pooled-invariant baseline would strengthen the conclusion that pooling, not parameter count, is responsible for the performance drop.","section":"Fig. 6 ablation"},{"comment":"The sentence 'this drop is statistically insignificant' is not supported by any test; either add a significance test over seeds or remove the claim.","section":"Appendix F.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the journal's scope and the core construction is likely publishable if the symmetry-action mismatch is resolved and the empirical protocol is tightened. The orientation-coordinate issue is the most serious: if the authors cannot justify the ρ0 treatment of gripper yaw/object orientation, the formal framework and the experiments would be disconnected. I would also encourage the editor to require the authors to release code/data for reproducibility, although this is not a formal condition for my recommendation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe short version: this is a competent and useful method paper for the goal-conditioned RL subfield. The new piece is a goal-conditioned group-invariant MDP formalization plus a rotation-invariant critic built from C_N-equivariant encoders, with a rotation-equivariant actor. The invariant-critic construction is clean—aligned permutations of regular-representation features make the inner-product and L2 similarities invariant by construction. The ablation showing that equivariant representations beat global pooling is genuinely informative. Proposition 1 is a lift of an external result from Wang et al.; the proof in the appendix works.\n\nThe soft spots are manageable but real. The biggest one is the group action specification in Sec 4.2. The paper explicitly treats the gripper rotation angle θ and, implicitly, object orientation as ρ0-invariant under planar rotation. But a 90° rotation of the scene changes both. So the equivariance encoded in the network is not a symmetry of the actual Fetch/BulletArm MDPs when orientations are in the state. The paper never discusses this, except the camera-skew caveat for images. If the implemented state vectors actually drop orientation (some versions of Fetch do), then the concern is vacuous—but the text doesn't say that. This needs clarification, and the theory needs to state the exact factored state/action decomposition used.\n\nSecond, the empirical reporting is weaker than the abstract. No code, no error bars or significance tests on the main curves, and the claim of 'consistently outperforms' is contradicted by their own SawyerBin result where plain CRL with L2 similarity wins. There's also no matched-capacity baseline; ECRL's final embedding is 64×8 regular-representation features, while the baseline representation is 64-dimensional, so the comparison may stack parameters in ECRL's favor.\n\nThe stress-test note you have is right. The camera-skew caveat is not the only symmetry caveat that matters; the state/action orientation assignment is more fundamental.\n\nWho gets value: anyone working on equivariance in RL or on sample-efficient goal-conditioned robotic manipulation. It deserves a serious refereeing cycle, but the authors need to fix the orientation issue (or state that orientations are excluded), release code, add error bars, and run a capacity-matched comparison. I'd recommend 'revise and resubmit' rather than accept as is.\n\nCandidly yours,\nA senior colleague","headline":"Solid but imperfect method paper for goal-conditioned CRL with a real symmetry-specification hole around orientation variables and weaker empirical claims than the abstract admits.","tokens_in":16941,"tokens_out":3563,"would_cite":true,"duration_ms":37064,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Encoding planar rotation symmetry into both the critic and the actor makes goal-conditioned contrastive RL learn faster and generalize to unseen goal positions.","keywords":["Reinforcement Learning","Contrastive Learning","Equivariance","Goal-Conditioned RL","Robot Manipulation","Rotation Invariance","Sample Efficiency","Representation Learning"],"falsifier":"One concrete test is to measure the success-rate gap between ECRL and CRL as the environment's rotational symmetry is progressively broken, for example by adding a fixed obstacle or tilting the camera further from top-down; if the gap does not shrink as the perturbation grows, the claim that symmetry awareness causes the gains is falsified. A second check is to evaluate a trained ECRL critic on many random rotations and compare f(gs, ga, gg) with f(s, a, g); a large mismatch would show the assumed invariance is not actually enforced.","tokens_in":15903,"feed_emoji":"🤖","tokens_out":6960,"duration_ms":71356,"temperature":0.7,"pith_summary":"The paper tries to establish that rotation symmetry can be built directly into contrastive reinforcement learning for goal-conditioned robot manipulation, and that doing so buys sample efficiency and spatial generalization. It introduces Goal-Conditioned Group-Invariant MDPs, a formal setting where rotating the state, goal, and action together leaves the optimal value unchanged and rotates the optimal policy accordingly. On top of this, Equivariant Contrastive RL (ECRL) uses C_N-equivariant encoders whose regular-representation outputs make the critic's similarity score rotation-invariant while keeping the actor rotation-equivariant. The reported experiments claim ECRL consistently beats standard Contrastive RL and a hindsight-relabeling baseline across state-based and image-based simulated manipulation tasks, including offline settings with very few demonstrations. A sympathetic reader would care because it points to a recipe for transferring geometric structure in a task into learning speed without requiring reward design.","feed_headline":"Built-in rotation symmetry speeds up goal-conditioned robot learning","feed_subtitle":"Equivariant critics and actors let simulated robot arms master push and pick tasks with far fewer interactions.","key_machinery":"The load-bearing mechanism is the regular representation of the cyclic group $C_N$. A $C_N$-equivariant encoder maps state-action pairs and goals to vectors whose $N$ elements are indexed by group elements; rotating the input by one step cyclically shifts the vector. When both encoders share this structure, the inner product (or $\\ell^2$ distance) between the state-action embedding and the goal embedding is untouched by simultaneous rotation, making the critic rotation-invariant without discarding orientation. The actor is a second $C_N$-equivariant map that outputs mixed trivial- and standard-representation action features, so planar components rotate while gripper height, rotation angle, and open/close state stay fixed. The Goal-Conditioned Group-Invariant MDP formalizes which environments admit this construction.","core_discovery":"The paper's central claim is that goal-conditioned manipulation has a rotational symmetry that can be made algorithmic. It defines a Goal-Conditioned Group-Invariant MDP and proves (Proposition 1) that in such an MDP the optimal Q-function is group-invariant, $Q^*(gs, ga, gg) = Q^*(s, a, g)$, and the optimal policy is group-equivariant, $\\pi^*(gs, gg) = g\\pi^*(s, g)$. On this basis it constructs Equivariant Contrastive RL: a critic whose state-action and goal encoders are $C_N$-equivariant networks outputting regular-representation feature vectors, so rotating both inputs by the same group element permutes the two embeddings in the same way and leaves inner-product or $\\ell^2$ similarity unchanged, paired with a $C_N$-equivariant actor whose planar action components rotate with the input. The experiments claim this architecture consistently outperforms non-equivariant Contrastive RL and a hindsight-relabeling baseline on state-based and image-based manipulation tasks, and also in offline RL with small datasets.","pith_inferences":["An extension the paper leaves implicit: the same critic construction should transfer to other discrete groups, such as the dihedral group for rotations plus reflections or discretized 3D rotations, and the paper's own appendix shows preliminary evidence for both.","A testable prediction that goes beyond the paper: because image experiments succeed despite a slightly skewed camera, ECRL should retain part of its advantage in real-world setups where symmetry is only approximate, with the advantage shrinking as the actual transform moves further from a true rotation.","The ablation against global pooling suggests the orientation information in the embeddings is doing real work, so one could probe this further by testing a task where goal orientation is irrelevant; the equivariant critic's advantage should shrink or disappear in that case.","The performance gains appear orthogonal to the choice of contrastive loss, so the equivariant critic may also combine with other representation-learning objectives that rely on elementwise operations across embeddings."],"forward_implications":["In state-based experiments, ECRL reaches higher success rates with fewer environment interactions than non-equivariant Contrastive RL and the TD3+HER baseline on all of the manipulation tasks considered.","In image-based experiments, the same pattern holds even though the camera is slightly skewed rather than strictly top-down, so the encoded symmetry is only partially present in the observations.","When training goals are restricted to one quadrant and evaluation goals are drawn from the full workspace, ECRL generalizes better and shows lower variance than CRL.","In offline RL with 5 to 50 demonstrations, ECRL outperforms CRL on every task and dataset size tested.","An ablation that replaces equivariant representations with global pooling shows the equivariant critic is much more data-efficient, indicating that preserving orientation information matters."],"supporting_citations":[{"why":"Defines Contrastive RL and supplies the contrastive objective and representation-learning setup that ECRL modifies with equivariant constraints.","marker":"[5]"},{"why":"Establishes group-invariant MDP optimality and equivariant network conventions that Proposition 1 and the actor/critic design build on.","marker":"[9]"},{"why":"Provides TD3+HER and the hindsight relabeling objective used as the privileged-reward baseline in the comparison.","marker":"[13]"},{"why":"Supplies the l2-distance variant of Contrastive RL used as both an alternative similarity metric and a baseline.","marker":"[23]"},{"why":"Supplies the behavioral-cloning regularizer that adapts the policy objective to the offline RL experiments.","marker":"[38]"},{"why":"Provides the equivariant network construction used to implement the C_N-equivariant encoders and actor.","marker":"[26]"}],"fun_headline_variants":["Rotation symmetry yields faster goal-conditioned robot learning","Symmetry-aware critic and actor speed up robot manipulation tasks","Equivariant contrastive RL uses rotation symmetry for faster learning","Group-invariant MDPs enable sample-efficient goal-conditioned RL","Rotation-equivariant agents master manipulation tasks with fewer data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire method assumes that the robot task is genuinely unchanged when state, goal, and action are rotated together, and that rotating the camera image by the same amount accurately encodes that symmetry; the authors note the camera is 'slightly skewed rather than strictly top-down,' so the symmetry is only partially present in image observations.","fun_headline_variants_meta":{"raw":{"variants":["Rotation symmetry yields faster goal-conditioned robot learning","Symmetry-aware critic and actor speed up robot manipulation tasks","Equivariant contrastive RL uses rotation symmetry for faster learning","Group-invariant MDPs enable sample-efficient goal-conditioned RL","Rotation-equivariant agents master manipulation tasks with fewer data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000918,"raw_usage":{"total_tokens":3930,"prompt_tokens":927,"completion_tokens":3003,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":543,"completion_tokens_details":{"reasoning_tokens":2922}},"tokens_in":543,"tokens_out":3003,"duration_ms":24402,"temperature":1.0,"reasoning_tokens":2922,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T15:17:47.573797+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"One concrete test is to measure the success-rate gap between ECRL and CRL as the environment's rotational symmetry is progressively broken, for example by adding a fixed obstacle or tilting the camera further from top-down; if the gap does not shrink as the perturbation grows, the claim that symmetry awareness causes the gains is falsified. A second check is to evaluate a trained ECRL critic on many random rotations and compare f(gs, ga, gg) with f(s, a, g); a large mismatch would show the assumed invariance is not actually enforced.","supporting_citations":[{"cited_title":"Eysenbach, T","cited_arxiv_id":null,"evidence_quote":"Defines Contrastive RL and supplies the contrastive objective and representation-learning setup that ECRL modifies with equivariant constraints."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes group-invariant MDP optimality and equivariant network conventions that Proposition 1 and the actor/critic design build on."},{"cited_title":"Fujimoto and S","cited_arxiv_id":null,"evidence_quote":"Supplies the behavioral-cloning regularizer that adapts the policy objective to the offline RL experiments."},{"cited_title":"Weiler and G","cited_arxiv_id":null,"evidence_quote":"Provides the equivariant network construction used to implement the C_N-equivariant encoders and actor."}],"review_version":1}