{"id":"c79c21d9-3d8f-4e06-8847-401f97343a0a","arxiv_id":"2502.07005","paper_version":6,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A heterogeneous SE(3)-equivariant graph policy (HEPi) outperforms Transformer and homogeneous equivariant baselines on a new seven-task manipulation reinforcement learning benchmark.","lead":"This paper presents a robot-control method that treats manipulation tasks as graphs with separate nodes for grippers and objects, and uses symmetry-aware message passing to learn policies for inserting, pushing, and hanging objects. It also introduces a new simulation benchmark with seven tasks covering rigid and deformable objects.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The scalar target-distance feature in Section 3.2 and Appendix B makes the policy partially observable for orientation-dependent tasks, undermining the interpretability of the benchmark results.","rationale":"The reader's weakest_assumption is exactly the concern I identify: the policy observations may not fully specify the target because only a scalar distance is provided. This is the most load-bearing issue because it undercuts the validity of the benchmark itself. If the observation is invariant under target rotation, then no policy can consistently solve the intended task, and the numerical returns become measures of performance on an ill-posed POMDP rather than on the stated manipulation problems. The concern is not about tuning or baseline strength; it is about whether the evaluation measures what the paper claims. The paper's own Figure 1 and Appendix B strongly support the scalar-distance reading, and the value-function divergence in Cloth-Hanging (using absolute target coordinates for the critic but not the policy) suggests the authors were aware of limitations in the distance feature for learning. I considered the alternative concern that SE(3) equivariance is not a true symmetry of gravity-affected tasks, but that issue affects all equivariant baselines roughly equally and would not overturn the relative ordering; the target-observation issue directly contaminates the reported absolute returns and the interpretability of all seven tasks. The reader's verdict of CONDITIONAL is appropriate: the concern is addressable by a code check and a small ablation, but until then the central claim is not fully verified. I therefore recommend no change to the reader's verdict.","tokens_in":24582,"tokens_out":9945,"duration_ms":97123,"concrete_test":"Obtain the code or the precise feature specification from the authors. If dtarget is the Euclidean distance to a target position reference point, construct two evaluation states for a task such as Cloth-Hanging or Rigid-Insertion-Two-Agents: keep all object node positions and velocities identical and rotate the target by 180 degrees about its centroid so that every dtarget is unchanged. Confirm that the rewards and optimal actions differ under the task's reward definition. Then retrain HEPi with an additional target node encoding full target pose (position plus orientation) or with per-node relative target vectors; if the augmented policy achieves substantially higher returns or visibly solves the task, the original benchmark is partially observable and the published results must be reinterpreted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is that HEPi outperforms baselines across the proposed manipulation benchmark. This claim is only interpretable if each task is a well-posed MDP, i.e., the policy observation fully specifies the state relevant to the task. The paper's own description suggests it does not. Section 3.2 states that for object nodes 'the feature vector also includes the relative distance to the target, d_{v,target}, embedding target information directly without the need for an additional target node,' and Figure 1 explicitly says 'The target distance is absorbed into the feature representation rather than treated as a separate node type.' Appendix B consistently lists only a scalar 'distance to target dtarget' as the target-related node feature in every task, including Rigid-Sliding, Rigid-Insertion, Rigid-Insertion-Two-Agents, Rope-Closing, Rope-Shaping, and Cloth-Hanging. For tasks in which the target has an orientation (all rigid tasks with target yaw, Cloth-Hanging with target roll/pitch/yaw, Rope-Shaping with target shape orientation), the optimal action depends on that orientation. However, if dtarget is the Euclidean distance from each object node to a target reference point, then rotating the target about that reference point leaves every dtarget unchanged while changing the required action and the reward (which the paper defines in terms of orientation differences, e.g., Rrotation and |cos(θalign)-1|). Consequently, multiple distinct MDP states map to the same policy input, the observation is not Markovian, and the reported returns cannot be interpreted as measuring success on the intended manipulation task. This threatens the validity of the entire empirical comparison, not merely one baseline. The concern is concrete and testable: it does not depend on tuning or implementation choices once the exact definition of dtarget is known.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces HEPi, an SE(3)-equivariant heterogeneous graph policy for manipulation of rigid and deformable objects. It represents actuators and objects as disjoint node sets with separate message-passing kernels per edge type, uses a PONITA-style EMPN backbone, and Trust Region Projection Layers for stable on-policy RL. The authors propose a seven-task IsaacLab benchmark covering rigid insertion/sliding/pushing, rope closing/shaping, and cloth hanging, and report IQM returns with 95% confidence intervals over 10 seeds. They claim HEPi outperforms Transformer and homogeneous EMPN baselines in final return, sample efficiency, and generalization to unseen geometries, with ablations on attention, number of message-passing steps, k-NN connectivity, orientation discretization, sample-space size, and PPO vs. TRPL.","tokens_in":24862,"tokens_out":7035,"duration_ms":66537,"significance":"If the results hold, the benchmark and architecture are a useful contribution: the tasks address a real gap (geometry-aware RL for varying and deformable shapes), the evaluation is more careful than typical (10 seeds, IQM and confidence intervals, held-out objects, noise robustness, several ablations), and the heterogeneous graph design is a sensible way to combine equivariance with global information flow to actuators. However, the paper's central empirical claim is only interpretable if each task is a well-posed MDP; the scalar target feature problem discussed below casts doubt on this for orientation-dependent tasks. The relative ranking may survive because all policies see the same partial observations, but the benchmark's ability to support conclusions about precise orientation alignment is reduced.","major_comments":[{"comment":"The observation for object nodes is stated to include 'the relative distance to the target, d_{v,target}' in Section 3.2, and Appendix B lists only the scalar 'distance to target dtarget' as the target-related feature for every task. For all rigid tasks, Rope-Shaping, and Cloth-Hanging, the target orientation is uniformly sampled and the reward explicitly depends on orientation (e.g., Rrotation, |cos(θalign)-1|, and the Dshape descriptor). A scalar distance from each object node to a target reference point is invariant under rotations of the target about that point, so distinct MDP states with different optimal actions and different rewards map to the same policy observation. Thus these tasks are partially observable as specified, and the reported returns cannot be interpreted as evidence that any method solves the intended orientation-sensitive manipulation problem. Please either include the target pose (e.g., a target vector or quaternion as a node or global feature) in the policy observation, restrict target sampling to position-only tasks, or explicitly reformulate the problem as a POMDP and make the memory and observation assumptions clear. This affects the benchmark's validity and should be addressed before the main comparative claim is accepted.","section":"Section 3.2 and Appendix B"},{"comment":"The theoretical justification for the heterogeneous full-connectivity design is not fully established. The proof gives an explicit Jacobian for HEPi (Eq. 5) but only asserts, without derivation, that for MPNN+VNLocal the Jacobian becomes independent of u when u and v are 'more than 2 hops' apart. The notion of hop distance is not formalized for the mixed edge types (obj-obj, act-act, obj-act), and no equation is provided for the MPNN+VNLocal Jacobian. In a residual message-passing network, the Jacobian can depend on u through longer paths even when the direct connection is absent, so the claim needs a precise argument or a counterexample. Since Proposition 3.1 is presented as a contribution and used to justify the local-vs-global connection design, this gap should be closed.","section":"Appendix A, Proposition 3.1"}],"minor_comments":[{"comment":"The generalization bar plots are reported without confidence intervals or an explicit number of seeds; please add error bars and seed counts for consistency with Figure 3.","section":"Section 4.2, Figure 5 (right)"},{"comment":"There are several typos, including 'NIVIDA IsaacLab' in Section 4.1 and 'rigid-slding' in Section 4.2; please proofread.","section":"Throughout"},{"comment":"The phrase 'one-hot scalar-vector' is ambiguous; clarify whether the node type is a scalar one-hot encoding or a concatenated vector.","section":"Section 3.2"},{"comment":"The paper reports only composite IQM returns, not task success rates. Adding a success-rate metric (e.g., thresholded on Rgoal/Rrotation or hole-hanger alignment) would make the benchmark more directly interpretable and strengthen the comparative claim.","section":"Experimental evaluation"},{"comment":"The value function is given absolute target and initial coordinates while the policy receives only scalar distances; please explain why this asymmetry does not undermine the intended equivariance inductive bias or describe how the value function handles coordinate-frame dependence.","section":"Appendix B.7, Cloth-Hanging"}],"recommendation":"major_revision","confidential_remarks":"The main risk is the scalar target feature issue: it is load-bearing for the benchmark's interpretability but appears fixable by changing the observation specification and rerunning experiments. The use of TRPL and PONITA from overlapping author groups is transparently cited and does not constitute circularity. If the authors can show that d_target is actually a target vector or that target orientation is otherwise available to the policy, the paper may become acceptable after revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth reading. The paper introduces HEPi, a heterogeneous SE(3)-equivariant policy using PONITA-style message passing, and a seven-task manipulation benchmark spanning rigid insertion and deformable rope/cloth tasks. The main comparison is careful: IQM returns with 95% CIs over 10 seeds, and HEPi consistently beats Transformer and homogeneous EMPN baselines, especially in 3D tasks. The ablations on attention, KNN structure, message-passing depth, and orientation discretization are genuinely informative and support the design choices. The TRPL comparison is also honest: PPO can match it in 2D when properly tuned, but TRPL is more stable in 3D.\n\nThe soft spot that matters is the target representation. Section 3.2 and Figure 1 say target information is embedded as a scalar distance, and Appendix B lists 'distance to target dtarget' in every task. For tasks with target orientation (rigid insertion with yaw, cloth-hanging with roll/pitch/yaw, rope-shaping with a desired shape orientation), a scalar distance to a reference point cannot specify the orientation. The rewards explicitly penalize orientation error, so the policy is operating in a partially observable MDP. This doesn't necessarily invalidate the relative rankings, since all baselines see the same observations, but it does mean the benchmark may not measure what the tasks are intended to measure. If the authors actually use a relative position vector or provide the target pose through a global feature, the text and appendix need to say so. As written, this is a genuine ambiguity, not a nitpick.\n\nOther minor issues: no code release, no task success rates, and Proposition 3.1 is more of a qualitative argument about information flow than a real theorem. The paper also skips comparisons to prior equivariant RL work like SO(2)-equivariant or subequivariant graph policies. These are addressable.\n\nOverall: I'd engage with it. The empirical contribution is substantial and the central relative claim likely holds. But I'd want the target-observation question resolved before trusting the benchmark as a reproducible testbed. It deserves a serious referee, and the fix is a clarification, not a redesign.","headline":"Solid empirical RL paper with a new benchmark, but the scalar target-distance feature likely makes the tasks partially observable and needs clarification.","tokens_in":25528,"tokens_out":3637,"would_cite":false,"duration_ms":32159,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T40","68T07","68T05"],"pacs":[],"model":"deepseek-v4-flash","headline":"A heterogeneous graph policy whose actuator and object nodes are updated separately and constrained to be SE(3)-equivariant outperforms Transformer-based and homogeneous equivariant policies across seven rigid and deformable manipulation…","keywords":["heterogeneous graph","equivariant policy","SE(3) equivariance","reinforcement learning","deformable object manipulation","rigid insertion","graph neural network","trust region"],"falsifier":"Train and evaluate HEPi on Rigid-Insertion-Two-Agents with object features restricted to the scalar distance to target (and no global target vector or orientation feature) and with target direction varied while distance is held fixed; if the policy's success tracks the unobserved direction, the observation is not Markovian and the reported returns would not reflect a policy that knows where the target is. The comparison that would settle it: add the full target vector to the object features and show the gap in success rate closes.","tokens_in":24360,"feed_emoji":"🦾","tokens_out":7002,"duration_ms":58040,"temperature":0.7,"pith_summary":"The paper is trying to establish that robotic manipulation of varied rigid shapes and deformable objects—insertion, pushing, rope shaping, cloth hanging—is best learned by an explicitly heterogeneous, SE(3)-equivariant graph policy. It claims that its Heterogeneous Equivariant Policy (HEPi) outperforms Transformer-based and homogeneous equivariant policies on a new seven-task benchmark in final returns, sample efficiency, and generalization to unseen objects. The authors also argue, with a proof sketch, that treating actuator nodes as global virtual nodes connected to every object node lets any actuator and any object exchange information in a single message-passing layer, whereas k-nearest-neighbour local connections can make the actuator's output independent of distant object nodes. If the claim holds, geometry-aware inductive biases plus explicit role separation are a practical recipe for sample-efficient reinforcement learning in high-dimensional manipulation.","feed_headline":"Robot and object graph policy beats Transformer on seven tasks","feed_subtitle":"SE(3)-equivariant message passing plus explicit role separation boosts returns, sample efficiency, and generalization.","key_machinery":"The central object is HEPi's heterogeneous graph: nodes split into actuator nodes and object nodes, with intra-edges inside each set and directed fully connected inter-edges from objects to actuators, and with separate kernel parameters per edge type. The backbone is an efficient approximately SE(3)-equivariant message-passing network, based on lifting positions to position-orientation space and factorized kernels; the paper's Proposition 3.1 argues that because actuators connect to all object nodes, any actuator-object pair can exchange information in one layer, which the ablations link to avoiding oversquashing. Target information is absorbed into object-node features as a scalar distance to target, and the value function is a permutation-invariant DeepSets readout. Training uses a trust-region projection layer rather than clipped PPO to keep policy updates inside a KL constraint.","core_discovery":"The paper claims that manipulation tasks with diverse geometries and deformable bodies can be cast as a single heterogeneous graph with disjoint actuator and object node sets, and that a policy built from that graph—HEPi—is more expressive than a homogeneous equivariant message-passing network and more sample-efficient than a Transformer. On its new GPU-simulated benchmark of seven tasks, HEPi reports higher average returns and better generalization to unseen objects, with the largest gains in tasks with large 3D search spaces: two-agent insertion, pushing, and cloth hanging. The authors attribute the gains to three design choices acting together: SE(3)-equivariant message passing, separate parameters for object-object, actuator-actuator, and object-actuator interactions, and a trust-region projection layer for stable on-policy updates. They support the graph design with Proposition 3.1, which shows that fully connected object-to-actuator edges let any actuator respond to any object node in one hop, while k-nearest-neighbour local connectivity can make that response independent of distant object nodes. They also report that adding attention to the graph does not improve performance and roughly doubles training time.","pith_inferences":["Editorial: in the rigid-insertion and cloth-hanging tasks where the target pose is sampled, a scalar distance-to-target object feature is not Markovian by itself; the paper's reported performance implicitly requires target direction or orientation to be recoverable from other inputs or the scalar to be interpreted as a vector, and this should be stated explicitly when reusing the benchmark.","Editorial: the same heterogeneous-graph recipe could apply to multi-arm assembly or human-robot collaboration where each robot is an actuator sub-graph, since the paper's formulation already allows multiple actuators with complete intra-actuator edges.","Editorial: because the EMPN backbone is only approximately equivariant (orientations are sampled on a grid over the sphere), a direct test of the equivalence claim would be to compare performance as the orientation grid is refined; the paper reports gains up to 24 samples, so even finer grids are a natural stress test.","Editorial: the benchmark's use of keypoint coordinates as observations presumes a perception front-end; a testable extension is to plug learned keypoint extractors into the same policy to measure how much of the equivariance advantage survives visual noise."],"forward_implications":["If the benchmark results hold, explicit heterogeneity—at least one parameter set per edge type—is a larger win than adding attention in on-policy graph RL; the paper's attention ablation shows no gain and about double training time.","The actuator-as-virtual-node design means one message-passing step suffices for global information flow, so deeper stacks are unnecessary and can hurt through oversquashing—a direct corollary of Proposition 3.1 supported by the paper's ablations.","SE(3) equivariance is the main driver of sample efficiency in large 3D search spaces; the cloth-hanging ablations show heterogeneous but non-equivariant GNNs need more samples to reach similar final performance.","The policy transfers across object resolutions: a model trained on roughly 20-node meshes evaluates on roughly 1200-node meshes with only mild degradation, and tolerates added position and velocity noise."],"supporting_citations":[{"why":"Supplies the efficient equivariant message-passing backbone that HEPi uses, including the position-orientation lifting and factorized kernel.","marker":"Bekkers et al., 2024"},{"why":"Supplies the trust-region projection layer used for stable on-policy policy updates.","marker":"Otto et al., 2021"},{"why":"Provides the MPNN + virtual-node interpretation that underpins Proposition 3.1 and the actuator-as-virtual-node design.","marker":"Southern et al., 2024"},{"why":"Supplies the MDP homomorphism theory that justifies exploiting symmetry in reinforcement learning.","marker":"Van der Pol et al., 2020"},{"why":"Defines the PPO baseline that HEPi must beat and that TRPL is compared against.","marker":"Schulman et al., 2017"},{"why":"Defines the Transformer architecture used as a primary baseline and as a fully connected graph model.","marker":"Vaswani et al., 2017"},{"why":"Inspires the rigid manipulation tasks, particularly the transporter-style insertion and sliding settings.","marker":"Zeng et al., 2020"},{"why":"Motivates velocity-vector equivariant policies for dynamic and deformable manipulation tasks.","marker":"Yang et al., 2024"},{"why":"Supplies the DeepSets architecture used for the permutation-invariant value function.","marker":"Zaheer et al., 2017"},{"why":"Provides the GPU-based simulation environment in which the seven-task benchmark is implemented.","marker":"Mittal et al., 2023"}],"fun_headline_variants":["Heterogeneous graph policy beats Transformer on seven robot tasks","Graph policy leverages SE(3) symmetry for deformable object control","Heterogeneous equivariant policy outperforms on unseen object shapes","Robot policy with heterogeneous graph improves generalization"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the policy's observations fully specify the manipulation target: target information enters object nodes as a scalar distance to target, with absolute target coordinates only in the value function for cloth hanging, so for tasks with sampled target positions or orientations the agent must effectively have enough information elsewhere to reconstruct the target; if it does not, the benchmark results are not interpretable as solving the intended task.","fun_headline_variants_meta":{"raw":{"variants":["Heterogeneous graph policy beats Transformer on seven robot tasks","Graph policy leverages SE(3) symmetry for deformable object control","Heterogeneous equivariant policy outperforms on unseen object shapes","Robot policy with heterogeneous graph improves generalization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000862,"raw_usage":{"total_tokens":3772,"prompt_tokens":1009,"completion_tokens":2763,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":625,"completion_tokens_details":{"reasoning_tokens":2709}},"tokens_in":625,"tokens_out":2763,"duration_ms":17880,"temperature":1.0,"reasoning_tokens":2709,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T14:04:45.291373+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train and evaluate HEPi on Rigid-Insertion-Two-Agents with object features restricted to the scalar distance to target (and no global target vector or orientation feature) and with target direction varied while distance is held fixed; if the policy's success tracks the unobserved direction, the observation is not Markovian and the reported returns would not reflect a policy that knows where the target is. The comparison that would settle it: add the full target vector to the object features and show the gap in success rate closes.","supporting_citations":[],"review_version":1}