Pith. sign in

REVIEW 3 major objections 5 minor 3 cited by

EquAct: An SE(3)-Equivariant Multi-Task Transformer for Open-Loop Robotic Manipulation

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read EquAct builds a transformer that is provably SE(3)-equivariant and claims state-of-the-art results on 18 RLBench manipulation tasks.

desk verdict A promising architecture and strong new benchmark, but the paper's exact-equivariance guarantee is broken by the finite query set and a wrong invariance condition in Section 4.4. read the letter →

arxiv 2505.21351 v1 pith:G6KXWO7P submitted 2025-05-27 cs.RO

Xupeng Zhu , Yu Qi , Yizhe Zhu , Robin Walters , Robert Platt This is my paper
classification cs.RO
keywords SE(3)equivarianceroboticmanipulationmulti-taskpolicykeyframeactionsphericalharmonicspointcloudtransformernaturallanguageconditioningRLBench
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a transformer-based manipulation policy can be made continuously SE(3)-equivariant, so that rotating or translating the observed scene rotates or translates the predicted gripper keyframe by the same rigid motion, while still conditioning on natural language. To achieve this it introduces EquAct, built from an SE(3)-equivariant point-cloud U-Net with spherical Fourier features, SE(3)-invariant FiLM layers for language, and an equivariant field network that scores candidate keyframe actions. The payoff would be spatial generalization: a policy that automatically respects scene geometry from fewer demonstrations instead of learning it from data. On 18 RLBench tasks and 4 physical tasks, the authors report that EquAct outperforms the compared baselines, with the largest margin in the hardest SE(3)-randomized setting.

What carries the argument

The carrying mechanism is the spherical Fourier representation of point-cloud features, on which rotations act linearly via Wigner D-matrices. This makes the pooling, upsampling, and attention layers equivariant by construction; the equivariant field network propagates these features to arbitrary query poses, returning a type-0 invariant feature for translation scoring and a spherical convolution with a learned filter for rotation scoring.

What would settle it

Take a trained EquAct policy, fix a language instruction, and record the predicted keyframe action a for a given scene. Apply a rigid rotation g to the entire scene and predict again; equivariance requires the new prediction to equal g·a. Test this for rotations that are not multiples of the query sampling angles, since the proof does not cover how finite sampling of candidate actions affects the argmax.

Watch

Extended reading notes

Core claim

The central claim is that the keyframe action policy π(o, n) = a is SE(3)-equivariant in the observation–action mapping and SE(3)-invariant to the language instruction, i.e., π(g·o, n) = g·a for every rigid transformation g. EquAct achieves this by composing three equivariant or invariant pieces: the EPTU encoder, which operates on spherical Fourier coefficients so that rotations act by Wigner D-matrices; the iFiLM layers, which modulate type-0 features only, keeping language conditioning invariant; and an equivariant field network that propagates features to arbitrary query poses and uses a spherical convolution to score rotational actions. The authors prove the equivariance of each layer by induction and report state-of-the-art average success rates on 18 RLBench tasks under SE(2) and SE(3) object initializations, plus 65% average success on 4 physical tasks.

Load-bearing premise

The guarantee rests on the action-scoring network behaving correctly when scene and query action are rotated together, and on the finite list of candidate actions used at inference; the paper proves a related but different invariance and does not analyze the discrete argmax.

Editorial extensions

If this is right

  • A policy trained on demonstrations from a few object poses should transfer to arbitrarily rotated versions of those scenes without extra data, because the architecture enforces the symmetry rather than learning it.
  • Rotating or translating the scene during evaluation will transform the predicted keyframe pose identically, so rigid perturbations cannot break action selection up to sampling and numerical error.
  • The language instruction acts as an invariant condition: changing the instruction changes which task-specific keyframe is selected, but never injects a preferred frame into the geometry.
  • The reported gains over baselines grow in the SE(3)-randomized 10-demo setting (15.4% average improvement), suggesting the equivariance is most valuable when object poses vary in full 3D.
  • Translation, rotation, and gripper-open actions are evaluated in a single forward pass, avoiding iterative denoising, which the authors claim keeps inference time comparable to non-equivariant baselines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the equivariance guarantee holds, EquAct-type architectures could reduce the augmentation burden in imitation learning: instead of sampling many scene rotations, a trainer could rely on the network's built-in symmetry, though the paper still uses augmentation to reduce numerical error.
  • The same invariant-FiLM conditioning idea could transfer to other equivariant backbones that use scalar (type-0) features to carry semantic information, suggesting a general recipe for injecting language into any equivariant network.
  • The unresolved interplay between finite action sampling and continuous equivariance suggests a testable extension: an adaptive or steerable action sampling scheme that makes the argmax exactly commute with rotations, which could turn the asymptotic guarantee into a hard one.
  • EquAct's limitation to open-loop keyframes means the equivariance guarantee applies to goal-pose prediction; extending the same Fourier-based design to closed-loop, full-trajectory diffusion policies is a natural next step that would face the challenge of equivariant denoising over many time steps.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes EquAct, an SE(3)-equivariant transformer for language-conditioned, open-loop keyframe manipulation. The architecture combines an equivariant point-cloud U-Net (EPTU) with spherical Fourier features, invariant FiLM (iFiLM) layers for language conditioning, and equivariant field networks that evaluate sampled translational, rotational, and gripper-open action queries. The authors claim a theoretical guarantee that EquAct is SE(3)-equivariant in the observation-action mapping and SE(3)-invariant to language. They evaluate on 18 RLBench tasks under SE(2) and SE(3) initializations and on 4 physical tasks, reporting state-of-the-art success rates over SAM2ACT and 3DDA, with additional ablations on data augmentation, FiLM, spherical harmonic degree, and breaking equivariance.

Significance. If the central claim of exact SE(3) equivariance were established, this would be a valuable contribution: the paper targets a realistic multi-task, language-conditioned setting that most prior equivariant manipulation work does not cover, and the empirical gains on the SE(3)-perturbed RLBench setting (15.4% over baselines) are notable. The ablations in Table 3 are informative, especially the large drop when a single equivariant layer is replaced (equ.→no equ.), which supports the practical importance of the architectural symmetry. The physical experiments, while small, add credibility to the approach. However, the theoretical guarantee as stated is not proven: the proof omits the field network, and the finite action-query sampling is incompatible with exact continuous equivariance. These issues are fixable by either correcting the proofs or rephrasing the claims as approximate equivariance, but they are load-bearing for the paper's central contribution.

major comments (3)
  1. [Sec. 4.4 (translational field network)] The paper states that the translation action-value satisfies qt(at, h) = qt(at, g·h) for g∈SO(3). The property needed for an SE(3)-equivariant translation policy is qt(g·at, g·h) = qt(at, h), i.e., the value should be equivariant jointly in the query and the observation, not invariant under rotating the observation with a fixed query. As written, the printed equation is a different invariance and does not imply that the argmax translation transforms as g·a*. This is a load-bearing discrepancy: Proposition 4.1's proof rests on the field network, and the section does not establish the correct equivariance relation.
  2. [Sec. 4 (inference) and Prop. 4.1] Exact SE(3) equivariance of the full policy is impossible with the finite query sampling described in the method. Table 5 reports #at train/test = 450/3000 sampled translational queries, and the policy selects the argmax over this finite set. A finite subset of R^3 cannot be closed under the continuous translation subgroup T(3), so for a generic g∈SE(3), the transformed winning action g·a* will not belong to the query set, and the argmax cannot be guaranteed to transform correctly. Proposition 4.1 therefore overclaims. The authors need either an equivariant continuous argmax or sampling scheme, or an explicit statement that the guarantee is approximate with an error bound depending on query density.
  3. [Appendix A.1 (proof of Prop. 4.1)] The proof of Proposition 4.1 is an induction that cites equivariance of Equiformer layers [36] and spherical CNNs [5], plus the pooling/upsampling propositions, but it does not cover the equivariant field network of Section 4.4 or the effect of the argmax over sampled query actions. The field network aggregates features from the latent point cloud h to a query point at via a k-NN graph and an EquiformerV2 attention block; transla- tion equivariance of this construction and the rotation behavior of the spherical output must be proved explicitly, and the finite-query argmax must be addressed. Without these steps, Proposition 4.1 is not established.
minor comments (5)
  1. [Sec. 4.4] The text says EquAct 'evaluates actions in the entire pose action space AT ⊂ SE(3)', but the method actually samples a finite set of query actions (Table 5). This wording is misleading and should be revised to describe the action query set explicitly.
  2. [Appendix A.2, A.3, A.4] There are several typos in the proofs: 'winger-D' should be 'Wigner-D', 'manganite' should be 'magnitude', 'Shur's lemma' should be 'Schur's lemma' (also appearing in Section 4.3), and 'trans-rotal' in Section 4.4 should be 'translational'. These should be corrected.
  3. [Abstract and Section 1] The phrase 'continuous SE(3)-equivariance' is used while the action selection is over a discrete finite query set; this is at least imprecise and should be qualified as approximate or as equivariance of the underlying value function rather than of the argmax policy.
  4. [Table 1] The table layout is dense and several numbers appear to run together (e.g., in the 'open drawer' and 'screw bulb' columns). Please reformat the table to make each method's success rate unambiguous, and double-check the alignment of the per-task entries.
  5. [Section 5.3, Table 3] For the ablation 'iFiLM→FiLM', the paper says performance is similar to the full model in most cases but the table shows a large drop on 'place wine' (45 vs. 68) and 'place cups' (62 vs. 24); the text should more accurately summarize these numbers, or clarify whether the differences are within variance.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: EquAct's equivariance claims are structural consequences of the architecture and are benchmarked against external baselines; the main weakness is an under-proved exactness claim, not a derivation that reduces to its inputs.

full rationale

The paper's central claim—that EquAct is SE(3)-equivariant in observation-action mapping and language-invariant—is not obtained by fitting a parameter to data or by renaming an input. The architecture-level equivariance of the pooling (Prop. 4.2), upsampling (Prop. 4.3), iFiLM (Prop. 4.4), and attention blocks is proved from Wigner-D orthogonality and Schur's lemma, with EquiformerV2 and Spherical CNN cases cited to external prior work ([36], [5]), not to unverified results of the present authors. The empirical evaluation compares against external baselines (SAM2ACT, 3DDA) and physical tasks, so the performance claims are not forced by construction. There is heavy self-citation from the authors' own equivariant-policy line, but those citations justify the motivating equivariance assumption in Eq. 2 and prior SE(2) results; they do not carry the proof of the proposed architecture's equivariance. The genuine weaknesses are correctness gaps rather than circularity: Appendix A.1's induction covers pooling, upsampling, Equiformer, and Spherical CNN layers but omits the field network, and Table 5's finite query set (#a_t = 450/3000) plus argmax cannot be exactly invariant under continuous translations; Section 4.4 also prints qt(at,h)=qt(at,g·h), which is the wrong invariance for a translation-action policy. These would affect whether Proposition 4.1 is exactly true, but they do not make the claim equivalent to its own inputs or to a fitted parameter. Score 2 reflects the presence of many self-citations in the surrounding motivation while the central derivation retains independent content.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The central equivariance claim rests on standard group-theoretic facts, prior equivariant architectures (EquiformerV2, Spherical CNN), the keyframe assumption, and hand-designed choices for the SE(3) benchmark. No new physical entities are introduced. The proofs cite prior work for core building blocks, so the paper's theoretical contribution is its composition.

free parameters (3)
  • SO(3) perturbation ranges per RLBench task = varies per task (e.g., [-0.5, 0.5] rad for turn tap)
    Hand-chosen initialization ranges in Table 4 define the SE(3) benchmark; they affect task difficulty and the reported gains.
  • Data augmentation rotation range = [+/-5 deg, +/-5 deg, +/-45 deg] around x, y, z
    Chosen by hand in Section 4; augments training with rotated point clouds and actions.
  • Query action sample count = 450 train / 3000 test translation queries
    From Table 5; the coarse-to-fine sampling resolution affects action selection accuracy.
assumptions (6)
  • standard math Wigner D-matrices are orthogonal; norms of spherical Fourier coefficients are SO(3)-invariant
    Used in proof of Proposition 4.2 in Appendix A.2.
  • standard math Schur's lemma implies scalar weighting of irreducible representations is equivariant
    Invoked in proofs of Propositions 4.3 and 4.4 (Appendix A.3, A.4). The invocation is imprecise because scalar weighting commutes with any linear map; the conclusion is nonetheless true.
  • domain assumption EquiformerV2 graph attention blocks are SE(3)-equivariant
    Proposition 4.1 proof refers to [36] for equivariance of these blocks without restating conditions.
  • domain assumption Spherical CNN convolution is SO(3)-equivariant
    Proposition 4.1 proof refers to [5] for spherical CNN equivariance.
  • domain assumption Keyframe tasks can be solved by a sequence of discrete gripper poses
    The keyframe action formulation (Section 2, Conclusion) assumes open-loop keyframes suffice; the authors acknowledge this fails for closed-loop tasks.
  • domain assumption Language instruction is SE(3)-invariant, i.e., text is unaffected by scene rigid transforms
    Section 4.1 and iFiLM design treat CLIP/Transformer text embeddings as type-0 features.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EquAct: An SE(3)-Equivariant Multi-Task Transformer for Open-Loop Robotic Manipulation." pith.science (2026). https://pith.science/paper/G6KXWO7P

@misc{pith2026250521351,
  author       = {Pith},
  title        = {Pith review of: EquAct: An SE(3)-Equivariant Multi-Task Transformer for Open-Loop Robotic Manipulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/G6KXWO7P}},
  note         = {Machine review of arXiv:2505.21351}
}
read the original abstract

Transformer architectures can effectively learn language-conditioned, multi-task 3D open-loop manipulation policies from demonstrations by jointly processing natural language instructions and 3D observations. However, although both the robot policy and language instructions inherently encode rich 3D geometric structures, standard transformers lack built-in guarantees of geometric consistency, often resulting in unpredictable behavior under SE(3) transformations of the scene. In this paper, we leverage SE(3) equivariance as a key structural property shared by both policy and language, and propose EquAct-a novel SE(3)-equivariant multi-task transformer. EquAct is theoretically guaranteed to be SE(3) equivariant and consists of two key components: (1) an efficient SE(3)-equivariant point cloud-based U-net with spherical Fourier features for policy reasoning, and (2) SE(3)-invariant Feature-wise Linear Modulation (iFiLM) layers for language conditioning. To evaluate its spatial generalization ability, we benchmark EquAct on 18 RLBench simulation tasks with both SE(3) and SE(2) scene perturbations, and on 4 physical tasks. EquAct performs state-of-the-art across these simulation and physical tasks.

Figures

Figures reproduced from arXiv: 2505.21351 by the authors.

Figure 1
Figure 1. Overview of EquAct. EquAct first encodes the observation o = {s, e} into latent spherical features h using a SE(3)-equivariant U-Net, enco, while conditioning the natural language instruction n through invariant iFiLM layers. Based on the encoded features h, EquAct then samples and refines translational query actions and gripper open actions using an equivariant field network, resulting in action value functions Qt … view at source ↗
Figure 2
Figure 2. The equivariance and invariance of the multi-task keyframe policy. Under the equivariance assumption, when the obser￾vation is transformed to g · o, the predicted action transforms accordingly to g · a. Under the invariance assumption, given a fixed natu￾ral language instruction n, the action transfor￾mation depends solely on the transformation applied to the observation. EquAct assumes that the keyframe action poli… view at source ↗
Figure 3
Figure 3. SE(3)-Equivariant Point Transformer U-net (EPTU). Spherical Fourier maxpooling. Analogous to the maxpooling operation in convolutional neural networks [33], the spherical Fourier maxpooling layer ( [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Simulation and physical experiments. First row: 18 standard RLBench tasks[52, 25]. Second row: 18 RLBench tasks with SE(3) randomization. Third row: 4 physical experiments. A language instruction specifies each variant of the task [PITH_FULL_IMAGE:figures/full_fig_p00…
Figure 5
Figure 5. Figure 5: 4 Physical tasks [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]
Figure 6
Figure 6. Figure 6: Real world experimental setup Number of keyframe actions: 4. Variations: "top flower", "middle flower", "bottom flower". Objects: Three artificial flowers and one vase. Success Metric: The robot must accurately grab the designated flower and pluck it. 17 [PITH_FULL_IM…

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. RayViT: Ray-Conditioned Visual Representations for Viewpoint-Robust Imitation Learning

    cs.RO 2026-07 conditional novelty 6.0 of 10

    Conditioning a pretrained ViT on per-pixel Plücker camera rays — via a gated-cross-attention class token and patch-level ray embeddings — makes imitation-learned manipulation policies substantially more robust to came...

  2. SE(3)-Equivariant Diffusion Policy in Spherical Fourier Space

    cs.RO 2025-07 conditional novelty 6.0 of 10

    Continuous SE(3) equivariance is embedded in the policy by representing states, actions, and denoising steps in spherical Fourier space, improving generalization to novel 3D arrangements.

  3. Robotic Manipulation via Imitation Learning: Taxonomy, Evolution, Benchmark, and Challenges

    cs.RO 2025-08 conditional novelty 4.0 of 10

    A survey that taxonomizes robotic manipulation policies trained by imitation learning, traces their evolution, and compiles benchmark comparisons.

Reference graph

Works this paper leans on

79 extracted references · 45 canonical work pages · cited by 3 Pith papers

  1. [36]

    Equiformerv2: Improved equivariant transformer for scaling to higher-degree representations

    Yi-Lun Liao, Brandon M Wood, Abhishek Das, and Tess Smidt. Equiformerv2: Improved equivariant transformer for scaling to higher-degree representations. InThe Twelfth International Conference on Learning Representations, 2024

    work page 2024

  2. [5]

    Spherical cnns

    Taco S Cohen, Mario Geiger, Jonas Köhler, and Max Welling. Spherical cnns. InInternational Conference on Learning Representations, 2018

    work page 2018

  3. [1]

    A program to build e (n)-equivariant steerable cnns

    Gabriele Cesa, Leon Lang, and Maurice Weiler. A program to build e (n)-equivariant steerable cnns. InInternational conference on learning representations, 2022

    work page 2022

  4. [2]

    Se(3)-equivariant attention networks for shape reconstruction in function space

    Evangelos Chatzipantazis, Stefanos Pertigkiozoglou, Edgar Dobriban, and Kostas Daniilidis. Se(3)-equivariant attention networks for shape reconstruction in function space. InThe Eleventh International Conference on Learning Representations, 2023

    work page 2023

  5. [3]

    Diffusion policy: Visuomotor policy learning via action diffusion

    Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, page 02783649241273668, 2023

    2023

  6. [4]

    Group equivariant convolutional networks

    Taco Cohen and Max Welling. Group equivariant convolutional networks. InInternational conference on machine learning, pages 2990–2999. PMLR, 2016

    2016

  7. [6]

    Vector neurons: A general framework for so (3)-equivariant networks

    Congyue Deng, Or Litany, Yueqi Duan, Adrien Poulenard, Andrea Tagliasacchi, and Leonidas J Guibas. Vector neurons: A general framework for so (3)-equivariant networks. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 12200–12209, 2021

    work page 2021

  8. [7]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, G Heigold, S Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. InInternational Conference on Learning Representations, 2020

    2020

Show all 79 references
  1. [8]

    Sam2act: Integrating visual foundation model with a memory architecture for robotic manipulation, 2025

    Haoquan Fang, Markus Grotz, Wilbert Pumacay, Yi Ru Wang, Dieter Fox, Ranjay Krishna, and Jiafei Duan. Sam2act: Integrating visual foundation model with a memory architecture for robotic manipulation, 2025. URLhttps://arxiv.org/abs/2501.18564

    2025 arXiv

  2. [9]

    Se (3)-transformers: 3d roto-translation equivariant attention networks.Advances in neural information processing systems, 33:1970–1981, 2020

    Fabian Fuchs, Daniel Worrall, V olker Fischer, and Max Welling. Se (3)-transformers: 3d roto-translation equivariant attention networks.Advances in neural information processing systems, 33:1970–1981, 2020

    1970

  3. [10]

    Riemann: Near real-time se (3)-equivariant robot manipulation without point cloud segmentation

    Chongkai Gao, Zhengrong Xue, Shuying Deng, Tianhai Liang, Siqi Yang, Lin Shao, and Huazhe Xu. Riemann: Near real-time se (3)-equivariant robot manipulation without point cloud segmentation. In8th Annual Conference on Robot Learning, 2024

    2024

  4. [11]

    Towards generalizable vision-language robotic manipulation: A benchmark and llm-guided 3d policy

    Ricardo Garcia, Shizhe Chen, and Cordelia Schmid. Towards generalizable vision-language robotic manipulation: A benchmark and llm-guided 3d policy. InInternational Conference on Robotics and Automation (ICRA), 2025

    2025

  5. [12]

    e3nn: Euclidean neural networks.arXiv preprint arXiv:2207.09453, 2022

    Mario Geiger and Tess Smidt. e3nn: Euclidean neural networks.arXiv preprint arXiv:2207.09453, 2022

    2022 arXiv

  6. [13]

    Act3d: 3d feature field transformers for multi-task robotic manipulation

    Theophile Gervet, Zhou Xian, Nikolaos Gkanatsios, and Katerina Fragkiadaki. Act3d: 3d feature field transformers for multi-task robotic manipulation. InConference on Robot Learning, pages 3949–3965. PMLR, 2023

    2023

  7. [14]

    Rvt: Robotic view transformer for 3d object manipulation.CoRL, 2023

    Ankit Goyal, Jie Xu, Yijie Guo, Valts Blukis, Yu-Wei Chao, and Dieter Fox. Rvt: Robotic view transformer for 3d object manipulation.CoRL, 2023

    2023

  8. [15]

    Rvt2: Learning precise manipulation from few demonstrations.RSS, 2024

    Ankit Goyal, Valts Blukis, Jie Xu, Yijie Guo, Yu-Wei Chao, and Dieter Fox. Rvt2: Learning precise manipulation from few demonstrations.RSS, 2024

    2024

  9. [16]

    Equivariant single view pose prediction via induced and restriction representations.Advances in Neural Information Processing Systems, 36:47251–47263, 2023

    Owen Howell, David Klee, Ondrej Biza, Linfeng Zhao, and Robin Walters. Equivariant single view pose prediction via induced and restriction representations.Advances in Neural Information Processing Systems, 36:47251–47263, 2023

    2023

  10. [17]

    Orbitgrasp: Se (3)-equivariant grasp learning

    Boce Hu, Xupeng Zhu, Dian Wang, Zihao Dong, Haojie Huang, Chenghao Wang, Robin Walters, and Robert Platt. Orbitgrasp: Se (3)-equivariant grasp learning. In8th Annual Conference on Robot Learning. 10

  11. [18]

    Push-grasp policy learning using equivariant models and grasp score optimization, 2025

    Boce Hu, Heng Tian, Dian Wang, Haojie Huang, Xupeng Zhu, Robin Walters, and Robert Platt. Push-grasp policy learning using equivariant models and grasp score optimization, 2025. URL https://arxiv.org/abs/2504.03053

    2025 arXiv

  12. [19]

    Fourier transporter: Bi-equivariant robotic manipulation in 3d

    Haojie Huang, Owen Lewis Howell, Dian Wang, Xupeng Zhu, Robert Platt, and Robin Walters. Fourier transporter: Bi-equivariant robotic manipulation in 3d. InThe Twelfth International Conference on Learning Representations,

  13. [20]

    Imagination policy: Using generative point cloud models for learning manipulation policies

    Haojie Huang, Karl Schmeckpeper, Dian Wang, Ondrej Biza, Yaoyao Qian, Haotian Liu, Mingxi Jia, Robert Platt, and Robin Walters. Imagination policy: Using generative point cloud models for learning manipulation policies. In8th Annual Conference on Robot Learning,

  14. [21]

    Equivariant Transporter Network

    Haojie Huang, Dian Wang, Robin Walters, and Robert Platt. Equivariant Transporter Network. InProceedings of Robotics: Science and Systems, New York City, NY , USA, June 2022. doi: 10.15607/RSS.2022.XVIII.007

    2022 doi

  15. [22]

    Leveraging symme- tries in pick and place.arXiv preprint arXiv:2308.07948, 2023

    Haojie Huang, Dian Wang, Arsh Tangri, Robin Walters, and Robert Platt. Leveraging symme- tries in pick and place.arXiv preprint arXiv:2308.07948, 2023

    2023 arXiv

  16. [23]

    Edge grasp network: A graph-based se (3)-invariant approach to grasp detection

    Haojie Huang, Dian Wang, Xupeng Zhu, Robin Walters, and Robert Platt. Edge grasp network: A graph-based se (3)-invariant approach to grasp detection. In2023 IEEE International Conference on Robotics and Automation (ICRA), pages 3882–3888. IEEE, 2023

    2023

  17. [24]

    Q-attention: Enabling efficient learning for vision-based robotic manipulation.IEEE Robotics and Automation Letters, 7(2):1612–1619, 2022

    Stephen James and Andrew J Davison. Q-attention: Enabling efficient learning for vision-based robotic manipulation.IEEE Robotics and Automation Letters, 7(2):1612–1619, 2022

    2022

  18. [25]

    Rlbench: The robot learning benchmark & learning environment.IEEE Robotics and Automation Letters, 5(2): 3019–3026, 2020

    Stephen James, Zicong Ma, David Rovick Arrojo, and Andrew J Davison. Rlbench: The robot learning benchmark & learning environment.IEEE Robotics and Automation Letters, 5(2): 3019–3026, 2020

    2020

  19. [26]

    Coarse-to-fine q- attention: Efficient learning for visual robotic manipulation via discretisation

    Stephen James, Kentaro Wada, Tristan Laidlow, and Andrew J Davison. Coarse-to-fine q- attention: Efficient learning for visual robotic manipulation via discretisation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13739–13748, 2022

    2022

  20. [27]

    Seil: Simulation-augmented equivariant imitation learning

    Mingxi Jia, Dian Wang, Guanang Su, David Klee, Xupeng Zhu, Robin Walters, and Robert Platt. Seil: Simulation-augmented equivariant imitation learning. In2023 IEEE International Conference on Robotics and Automation (ICRA), pages 1845–1851. IEEE, 2023

    2023

  21. [28]

    Open-vocabulary pick and place via patch-level semantic maps.arXiv preprint arXiv:2406.15677, 2024

    Mingxi Jia, Haojie Huang, Zhewen Zhang, Chenghao Wang, Linfeng Zhao, Dian Wang, Jason Xinyu Liu, Robin Walters, Robert Platt, and Stefanie Tellex. Open-vocabulary pick and place via patch-level semantic maps.arXiv preprint arXiv:2406.15677, 2024

    2024 arXiv

  22. [29]

    3d diffuser actor: Policy diffusion with 3d scene representations

    Tsung-Wei Ke, Nikolaos Gkanatsios, and Katerina Fragkiadaki. 3d diffuser actor: Policy diffusion with 3d scene representations. In8th Annual Conference on Robot Learning

  23. [30]

    Image to sphere: Learning equivariant features for efficient pose prediction

    David Klee, Ondrej Biza, Robert Platt, and Robin Walters. Image to sphere: Learning equivariant features for efficient pose prediction. InThe Eleventh International Conference on Learning Representations, 2023

    2023

  24. [31]

    Symmetric models for visual force policy learning

    Colin Kohler, Anuj Shrivatsav Srikanth, Eshan Arora, and Robert Platt. Symmetric models for visual force policy learning. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 3101–3107. IEEE, 2024

    2024

  25. [32]

    Reinforcement learning with augmented data.Advances in neural information processing systems, 33:19884–19895, 2020

    Misha Laskin, Kimin Lee, Adam Stooke, Lerrel Pinto, Pieter Abbeel, and Aravind Srinivas. Reinforcement learning with augmented data.Advances in neural information processing systems, 33:19884–19895, 2020

    2020

  26. [33]

    Gradient-based learning applied to document recognition.Proceedings of the IEEE, 86(11):2278–2324, 1998

    Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition.Proceedings of the IEEE, 86(11):2278–2324, 1998

    1998

  27. [34]

    Large language-geometry model: When llm meets equivariance.arXiv preprint arXiv:2502.11149, 2025

    Zongzhao Li, Jiacheng Cen, Bing Su, Wenbing Huang, Tingyang Xu, Yu Rong, and Deli Zhao. Large language-geometry model: When llm meets equivariance.arXiv preprint arXiv:2502.11149, 2025. 11

    2025 arXiv

  28. [35]

    Equiformer: Equivariant graph attention transformer for 3d atomistic graphs

    Yi-Lun Liao and Tess Smidt. Equiformer: Equivariant graph attention transformer for 3d atomistic graphs. InThe Eleventh International Conference on Learning Representations, 2023

    2023

  29. [37]

    Continual vision-based reinforcement learning with group symmetries

    Shiqi Liu, Mengdi Xu, Peide Huang, Xilun Zhang, Yongkang Liu, Kentaro Oguchi, and Ding Zhao. Continual vision-based reinforcement learning with group symmetries. InConference on Robot Learning, pages 222–240. PMLR, 2023

    2023

  30. [38]

    Relevance of rotationally equivariant convolutions for predicting molecular properties.arXiv preprint arXiv:2008.08461, 2020

    Benjamin Kurt Miller, Mario Geiger, Tess E Smidt, and Frank Noé. Relevance of rotationally equivariant convolutions for predicting molecular properties.arXiv preprint arXiv:2008.08461, 2020

    2008 arXiv

  31. [39]

    Equivariant reinforcement learning under partial observability

    Hai Huu Nguyen, Andrea Baisero, David Klee, Dian Wang, Robert Platt, and Christopher Amato. Equivariant reinforcement learning under partial observability. In7th Annual Conference on Robot Learning, 2023. URLhttps://openreview.net/forum?id=AnDDMQgM7-

    2023

  32. [40]

    Learning symmetric representations for equivariant world model

    Jung Yeon Park, Ondrej Biza, Linfeng Zhao, Jan Willem van de Meent, and Robin Walters. Learning symmetric representations for equivariant world model. InInternational Conference on Machine Learning, 2022. URLhttps://arxiv.org/abs/2204.11371

    2022 arXiv

  33. [41]

    Reducing so (3) convolutions to so (2) for efficient equivariant gnns

    Saro Passaro and C Lawrence Zitnick. Reducing so (3) convolutions to so (2) for efficient equivariant gnns. InInternational conference on machine learning, pages 27420–27438. PMLR, 2023

    2023

  34. [42]

    Film: Visual reasoning with a general conditioning layer

    Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. Film: Visual reasoning with a general conditioning layer. InProceedings of the AAAI conference on artificial intelligence, volume 32, 2018

    2018

  35. [43]

    Two by two: Learning multi-task pairwise objects assembly for generalizable robot manipulation.arXiv preprint arXiv:2504.06961, 2025

    Yu Qi, Yuanchen Ju, Tianming Wei, Chi Chu, Lawson LS Wong, and Huazhe Xu. Two by two: Learning multi-task pairwise objects assembly for generalizable robot manipulation.arXiv preprint arXiv:2504.06961, 2025

    2025 arXiv

  36. [44]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pag...

    2021

  37. [45]

    Sam 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024

    2024 arXiv

  38. [46]

    Rahmatullah Roche, Bernard Moussad, Md Hossain Shuvo, Sumit Tarafder, and Debswapna Bhattacharya. Equipnas: improved protein–nucleic acid binding site prediction using protein- language-model-informed equivariant deep graph neural networks.Nucleic Acids Research, 52 (5):e27–e27, 2024

    2024

  39. [47]

    U-net: Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. InMedical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part I...

    2015

  40. [48]

    Equivariant descriptor fields: Se (3)-equivariant energy-based models for end-to-end visual robotic manipulation learning

    Hyunwoo Ryu, Hong-in Lee, Jeong-Hoon Lee, and Jongeun Choi. Equivariant descriptor fields: Se (3)-equivariant energy-based models for end-to-end visual robotic manipulation learning. In The Eleventh International Conference on Learning Representations

  41. [49]

    Diffusion-edfs: Bi-equivariant denoising generative modeling on se (3) for visual robotic manipulation

    Hyunwoo Ryu, Jiwoo Kim, Hyunseok An, Junwoo Chang, Joohwan Seo, Taehan Kim, Yubin Kim, Chaewon Hwang, Jongeun Choi, and Roberto Horowitz. Diffusion-edfs: Bi-equivariant denoising generative modeling on se (3) for visual robotic manipulation. InProceedings of the IEEE/CVF Confe...

    2024

  42. [50]

    Neue begründung der theorie der gruppencharaktere

    Issai Schur. Neue begründung der theorie der gruppencharaktere. InSitzungsberichte der Königlich Preußischen Akademie der Wissenschaften zu Berlin: Jahrgang 1905; Erster Halb- band Januar bis Juni, pages 406–432. Verlag der Königlichen Akademie der Wissenschaften, 1905

    1905

  43. [51]

    Clip-fields: Weakly supervised semantic fields for robotic memory.arXiv preprint arXiv:2210.05663, 2022

    Nur Muhammad Mahi Shafiullah, Chris Paxton, Lerrel Pinto, Soumith Chintala, and Arthur Szlam. Clip-fields: Weakly supervised semantic fields for robotic memory.arXiv preprint arXiv:2210.05663, 2022

    2022 arXiv

  44. [52]

    Perceiver-actor: A multi-task transformer for robotic manipulation

    Mohit Shridhar, Lucas Manuelli, and Dieter Fox. Perceiver-actor: A multi-task transformer for robotic manipulation. InConference on Robot Learning, pages 785–799. PMLR, 2023

    2023

  45. [53]

    Neural descriptor fields: Se (3)-equivariant object representations for manipulation

    Anthony Simeonov, Yilun Du, Andrea Tagliasacchi, Joshua B Tenenbaum, Alberto Rodriguez, Pulkit Agrawal, and Vincent Sitzmann. Neural descriptor fields: Se (3)-equivariant object representations for manipulation. In2022 International Conference on Robotics and Automation (ICRA)...

    2022

  46. [54]

    Roformer: Enhanced transformer with rotary position embedding.Neurocomputing, 568:127063, 2024

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding.Neurocomputing, 568:127063, 2024

    2024

  47. [55]

    Equivariant offline reinforcement learning.arXiv preprint arXiv:2406.13961, 2024

    Arsh Tangri, Ondrej Biza, Dian Wang, David Klee, Owen Howell, and Robert Platt. Equivariant offline reinforcement learning.arXiv preprint arXiv:2406.13961, 2024

    2024 arXiv

  48. [56]

    Tensor field networks: Rotation-and translation-equivariant neural networks for 3d point clouds.arXiv preprint arXiv:1802.08219, 2018

    Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation-and translation-equivariant neural networks for 3d point clouds.arXiv preprint arXiv:1802.08219, 2018

    2018 arXiv

  49. [57]

    Et- seed: Efficient trajectory-level se (3) equivariant diffusion policy

    Chenrui Tie, Yue Chen, Ruihai Wu, Boxuan Dong, Zeyi Li, Chongkai Gao, and Hao Dong. Et- seed: Efficient trajectory-level se (3) equivariant diffusion policy. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  50. [58]

    Mdp homomorphic networks: Group symmetries in reinforcement learning.Advances in Neural Information Processing Systems, 33:4199–4210, 2020

    Elise Van der Pol, Daniel Worrall, Herke van Hoof, Frans Oliehoek, and Max Welling. Mdp homomorphic networks: Group symmetries in reinforcement learning.Advances in Neural Information Processing Systems, 33:4199–4210, 2020

    2020

  51. [59]

    Attention is all you need.Advances in neural information processing systems, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017

    2017

  52. [60]

    Equivariant $q$ learning in spatial action spaces

    Dian Wang, Robin Walters, Xupeng Zhu, and Robert Platt. Equivariant $q$ learning in spatial action spaces. In5th Annual Conference on Robot Learning, 2021. URL https: //openreview.net/forum?id=IScz42A3iCI

    2021

  53. [61]

    On-robot learning with equivariant models

    Dian Wang, Mingxi Jia, Xupeng Zhu, Robin Walters, and Robert Platt. On-robot learning with equivariant models. In6th Annual Conference on Robot Learning, 2022. URL https: //openreview.net/forum?id=K8W6ObPZQyh

    2022

  54. [62]

    SO(2)-equivariant reinforcement learning

    Dian Wang, Robin Walters, and Robert Platt. SO(2)-equivariant reinforcement learning. In International Conference on Learning Representations, 2022. URL https://openreview. net/forum?id=7F9cOhdvfk_

    2022

  55. [63]

    Equivariant q learning in spatial action spaces

    Dian Wang, Robin Walters, Xupeng Zhu, and Robert Platt. Equivariant q learning in spatial action spaces. In Aleksandra Faust, David Hsu, and Gerhard Neumann, editors,Proceedings of the 5th Conference on Robot Learning, volume 164 ofProceedings of Machine Learning Research, pag...

    2022

  56. [64]

    Wong, Robin Walters, and Robert Platt

    Dian Wang, Jung Yeon Park, Neel Sortur, Lawson L.S. Wong, Robin Walters, and Robert Platt. The surprising effectiveness of equivariant models in domains with latent symmetry. In International Conference on Learning Representations, 2023. URL https://openreview. net/forum?id=P4...

    2023

  57. [65]

    Equivariant diffusion policy

    Dian Wang, Stephen Hart, David Surovik, Tarik Kelestemur, Haojie Huang, Haibo Zhao, Mark Yeatman, Jiuguang Wang, Robin Walters, and Robert Platt. Equivariant diffusion policy. In8th Annual Conference on Robot Learning, 2024. URL https://openreview.net/forum?id= wD2kUVLT1g

    2024

  58. [66]

    Hager, and Liangjun Zhang

    Weiyao Wang, Yutian Lei, Shiyu Jin, Gregory D. Hager, and Liangjun Zhang. Vihe: Virtual in- hand eye transformer for 3d robotic manipulation. In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 403–410, 2024. doi: 10.1109/IROS58592. 2024.10802366

    2024

  59. [67]

    General e (2)-equivariant steerable cnns.Advances in neural information processing systems, 32, 2019

    Maurice Weiler and Gabriele Cesa. General e (2)-equivariant steerable cnns.Advances in neural information processing systems, 32, 2019

    2019

  60. [68]

    Chaineddiffuser: Unifying trajectory diffusion and keypose prediction for robotic manipulation

    Zhou Xian and Nikolaos Gkanatsios. Chaineddiffuser: Unifying trajectory diffusion and keypose prediction for robotic manipulation. InConference on Robot Learning/Proceedings of Machine Learning Research. Proceedings of Machine Learning Research, 2023

    2023

  61. [69]

    $SE(3)$ equivariant ray embeddings for implicit multi-view depth estimation

    Yinshuang Xu, Dian Chen, Katherine Liu, Sergey Zakharov, Rares Andrei Ambrus, Kostas Daniilidis, and Vitor Campagnolo Guizilini. $SE(3)$ equivariant ray embeddings for implicit multi-view depth estimation. InThe Thirty-eighth Annual Conference on Neural Information Processing ...

    2024

  62. [70]

    Equibot: Sim (3)-equivariant diffusion policy for generalizable and data efficient learning

    Jingyun Yang, Ziang Cao, Congyue Deng, Rika Antonova, Shuran Song, and Jeannette Bohg. Equibot: Sim (3)-equivariant diffusion policy for generalizable and data efficient learning. In 8th Annual Conference on Robot Learning

  63. [71]

    Equivact: Sim (3)-equivariant visuomotor policies beyond rigid object manipulation

    Jingyun Yang, Congyue Deng, Jimmy Wu, Rika Antonova, Leonidas Guibas, and Jeannette Bohg. Equivact: Sim (3)-equivariant visuomotor policies beyond rigid object manipulation. In2024 IEEE international conference on robotics and automation (ICRA), pages 9249–9255. IEEE, 2024

    2024

  64. [72]

    Learning synergies between pushing and grasping with self-supervised deep reinforcement learning

    Andy Zeng, Shuran Song, Stefan Welker, Johnny Lee, Alberto Rodriguez, and Thomas Funkhouser. Learning synergies between pushing and grasping with self-supervised deep reinforcement learning. In2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pag...

    2018

  65. [73]

    Autore- gressive action sequence learning for robotic manipulation.IEEE Robotics and Automation Letters, 2025

    Xinyu Zhang, Yuhan Liu, Haonan Chang, Liam Schramm, and Abdeslam Boularias. Autore- gressive action sequence learning for robotic manipulation.IEEE Robotics and Automation Letters, 2025

    2025

  66. [74]

    Point transformer

    Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip HS Torr, and Vladlen Koltun. Point transformer. In Proceedings of the IEEE/CVF international conference on computer vision, pages 16259–16268, 2021

    2021

  67. [75]

    Integrating symmetry into differentiable planning with steerable convolutions

    Linfeng Zhao, Xupeng Zhu, Lingzhi Kong, Robin Walters, and Lawson LS Wong. Integrating symmetry into differentiable planning with steerable convolutions. InInternational Conference on Learning Representations. International Conference on Learning Representations, 2023

    2023

  68. [76]

    On the continuity of rotation representations in neural networks

    Yi Zhou, Connelly Barnes, Jingwan Lu, Jimei Yang, and Hao Li. On the continuity of rotation representations in neural networks. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5745–5753, 2019

    2019

  69. [77]

    Sample efficient grasp learning using equivariant models.Proceedings of Robotics: Science and Systems (RSS), 2022

    Xupeng Zhu, Dian Wang, Ondrej Biza, Guanang Su, Robin Walters, and Robert Platt. Sample efficient grasp learning using equivariant models.Proceedings of Robotics: Science and Systems (RSS), 2022

    2022

  70. [78]

    On robot grasp learning using equivariant models.Autonomous Robots, 2023

    Xupeng Zhu, Dian Wang, Guanang Su, Ondrej Biza, Robin Walters, and Robert Platt. On robot grasp learning using equivariant models.Autonomous Robots, 2023

    2023

  71. [79]

    open the drawer

    Xupeng Zhu, David Klee, Dian Wang, Boce Hu, Haojie Huang, Arsh Tangri, Robin Walters, and Robert Platt. Coarse-to-fine 3d keyframe transporter, 2025. URL https://arxiv.org/ abs/2502.01773. 14 A Proofs A.1 Proof of proposition 4.1: Proof. To prove the equivariance of EquAct wit...

    2025 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.