Robotics · arXiv:2504.17784 v1 · submitted 24 Apr 2025 · 49 references
Gripper Keypose and Object Pointflow as Interfaces for Bimanual Robotic Manipulation
The paper says
“Bimanual manipulation is a challenging yet crucial robotic capability, demanding precise spatial localization and versatile motion trajectories, which pose significant challenges to existing approaches.”
more of the abstract
Existing approaches fall into two categories: keyframe-based strategies, which predict gripper poses in keyframes and execute them via motion planners, and continuous control methods, which estimate actions sequentially at each timestep. The keyframe-based method lacks inter-frame supervision, struggling to perform consistently or execute curved motions, while the continuous method suffers from weaker spatial perception. To address these issues, this paper introduces an end-to-end framework PPI (keyPose and Pointflow Interface), which integrates the prediction of target gripper poses and object pointflow with the continuous actions estimation. These interfaces enable the model to effectively attend to the target manipulation area, while the overall framework guides diverse and collision-free trajectories. By combining interface predictions with continuous actions estimation, PPI demonstrates superior performance in diverse bimanual manipulation tasks, providing enhanced spatial localization and satisfying flexibility in handling movement restrictions. In extensive evaluations, PPI significantly outperforms prior methods in both simulated and real-world experiments, achieving state-of-the-art performance with a +16.1% improvement on the RLBench2 simulation benchmark and an average of +27.5% gain across four challenging real-world tasks. Notably, PPI exhibits strong stability, high precision, and remarkable generalization capabilities in real-world scenarios. Project page: https://yuyinyang3y.github.io/PPI/
from the abstract
Pith says
Machine review · deepseek-v4-flash · 16 Aug 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. The interface idea is sensible and the ablations line up, but the headline numbers have enough comparison and consistency problems that the quantitative claims need verification before they can be trusted. This line is wrong
3 major objections · 4 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 49 references say.
The paper says, at V-A and Appendix E
The review did not quote this passage.
Pith says: Objection 1 of 3 · major
The RLBench2 comparison is not shown to be apples-to-apples. In Section V-A the authors state that they regenerate the training data because the official dataset has robot shadows present during training but absent during evaluation, and because it lacks pointflow meta-information; PPI is trained on this regenerated set.
more of objection 1
However, Appendix E says that ACT and PerAct2 are reproduced with the official RLBench2 code and that DP3 and 3D Diffuser Actor scores are taken from their respective papers, without stating that any baseline was retrained on the regenerated dataset. If the baselines were trained on the original shadow-corrupted data while PPI used corrected data, the reported +16.1% average improvement in Table VI could be partly an artifact of dataset cleanup rather than of the proposed interfaces.
The paper says, at IV-A and Appendix G
The review did not quote this passage.
Pith says: Objection 2 of 3 · major
The real-world success metric is under-specified. The text says each method is evaluated across 10 settings with 3 repeated trials per setting, but Tables XI-XVII present exactly 10 binary rows per method, with SR values of 0.0 or 1.0 per row.
more of objection 2
It is unclear whether SR is a per-trial success rate (which would require 30 trials per method), a per-setting success rate with an aggregation rule such as 'any trial succeeded' or 'all trials succeeded', or a record of a single rollout per setting; the normalized score also needs a stated aggregation rule across the three trials. Because the abstract's average +27.5% real-world gain rests on these numbers, the authors should report exact trial-level counts and the aggregation rule for SR, Loc-SR, and normalized score.
The paper says, at Table I and Appendix G
The review did not quote this passage.
Pith says: Objection 3 of 3 · major
The Wipe the Plate Loc-SR for PPI is inconsistent between the main table and the detailed results. Table I reports 70.0 for Loc-SR, but Table XIII shows that in 8 of the 10 settings both the sponge and the plate are picked (cases 1, 2, 4, 5, 6, 8, 9, 10), which is 80.0.
more of objection 3
The reported overall average Loc-SR of 92.5 is also consistent with 80.0 for this task, since (100 + 100 + 80 + 90) / 4 = 92.5. Please correct the table and re-audit the other entries against the appendix.
4 minor objections
The paper says, at Throughout
Pith says: minor objection 1 of 4
There are several typographical errors, including 'Robik's Cubes' (Section IV-C), 'Muti-plate' (Table IV), 'performace' and 'excute' (Appendix C), and 'incoporates' (Section III-D).
The paper says, at Figure 2
Pith says: minor objection 2 of 4
The figure contains unexpanded placeholder labels such as 'Scene ??', 'Noisy [KP] ??', and '?,?' in the prediction module, which should be replaced with the actual token definitions or meaningful captions.
show 2 more
The paper says, at Appendix H, Table XVIII
Pith says: minor objection 3 of 4
The label 'Lighting' appears where 'Flickering' is intended in the 3D Diffuser Actor row of the Carry the Tray generalization table; this should be corrected to avoid ambiguity.
The paper says, at V-A
Pith says: minor objection 4 of 4
The sentence 'we regenerate the training data' should explicitly state whether the same regenerated data were used for all internal ablations in Table VII and for the final PPI evaluation, so readers can interpret the ablation relative to the baseline comparison.
What would settle it
Request the regenerated dataset and training configurations, then retrain ACT and PerAct2 (and ideally DP3 and 3D Diffuser Actor) on that exact corrected dataset with the same episode splits, training steps, and checkpoint-averaging protocol, and evaluate 100 rollouts per task as in Table VI. If averaged success of ACT and PerAct2 rises materially (e.g., PerAct2 from 40.0 toward 3D Diffuser Actor's 64.7), the claimed +16.1% margin is at least partly a data artifact. A simpler immediate check: ask the authors whether Appendix E's ACT/PerAct2 runs used the regenerated dataset; the text currently does not say so.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 16 Aug 2026
- Reader holds up, with conditions
- Skeptic agrees
- Editor wrote the verdict above
- Referee major revision: 3 major objections, 4 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
The core story is reasonable: add two auxiliary interfaces to a continuous diffusion policy—predicted gripper keyposes and object pointflow—and let action tokens attend to them. The ablations in Table VII are the most convincing part: adding either interface lifts the continuous baseline, and both together give the best result. That is a real, transferable recipe, and the unidirectional attention design is concrete. Credit is also due for the real-world generalization experiments (new objects, lighting, distractors) and the per-case appendix tables.
The soft spots are where the numbers meet the world. The simulation comparison is the big one. Section V-A says they regenerated the RLBench2 training data to remove robot shadows and add pointflow metadata, and PPI is trained on that set. Appendix E says the baseline numbers for ACT and PerAct2 come from reproducing official code 'in RLBench2', without saying they were retrained on the corrected data; DP3 and 3D Diffuser Actor are taken from their papers. If the baselines were trained on shadow-corrupted data and PPI on clean data, the +16.1% margin is not apples-to-apples. The paper needs to either retrain all baselines on the same regenerated set or show the shadow artifact is performance-neutral.
The real-world tables also have internal inconsistencies. Table I gives Ours Scan the Bottle Loc-SR as 80, but the appendix per-case table (Table XIV) yields 9/10 = 90 for that metric. Table VIII lists Simple PPI Carry SR as 50, while the per-case Table XI gives 4/10 = 40. Table VIII lists PPI Wipe Loc-SR as 70, while the per-case Table XIII gives 8/10 = 80. These are not rounding differences. A reader cannot tell which number is correct.
One more omission: the related work cites BiKC, a keypose-conditioned bimanual policy, but BiKC is not included in the RLBench2 comparison. If it has public results, it should be there.
My take: the architecture and interface idea are worth engaging, and the ablations suggest the mechanism is real. But the empirical claims as written are not yet solid enough to quote. This is exactly the kind of paper that should go to peer review with a specific request for controlled baselines, corrected tables, and ideally code release.
The strongest objection
The central claim is that PPI's architecture achieves state-of-the-art performance, with a +16.1% improvement on RLBench2 (80.8% vs. 64.7% for the best baseline). For this claim to hold, all methods must be trained and evaluated on the same data. Section V-A says the authors 'regenerate the training data' because the official RLBench2 set has robot shadows present in training but absent during evaluation, and because it lacks metadata for pointflow. PPI is trained on this regenerated set. However, Appendix E says: 'In the simulation benchmark, we report the scores for DP3, ACT, PerAct2, and 3D Diffuser Actor from their respective papers. For ACT and PerAct2, we reproduce the results using the official code in RLBench2...' It never states that these baselines were retrained on the corrected, shadow-free regenerated dataset. If ACT and PerAct2 were trained on the original shadow-corrupted data while PPI used the cleaned data, the +16.1% improvement could reflect dataset cleanup rather than the keypose/pointflow interfaces. The same ambiguity applies to DP3 and 3D Diffuser Actor, whose training data is not explicitly identified. The paper's internal ablations (Table VII) do support the value of both interfaces, so the concern is not that the method lacks merit; it is that the headline quantitative comparison is not demonstrably controlled.
The reader identified the same weakest assumption, and I agree. The simulation benchmark is the quantitative basis for the 'state-of-the-art' and '+16.1%' headline claims. The paper's own wording creates a direct ambiguity: Section V-A justifies regenerating the data, while Appendix E describes baseline results from official code without noting retraining on that regenerated data. Since shadows appear in training but not evaluation, point-cloud-based baselines are especially disadvantaged. The real-world results are less central: they show consistent qualitative gains but rely on small trial counts and contain minor arithmetic inconsistencies (e.g., Table I normalized-score averages off by 0.1 for two rows), which would not reverse the direction of the comparison. In contrast, the simulation margin could shrink or disappear if baselines are retrained on the corrected dataset. The ablations in Table VII are independent evidence that the interfaces contribute, so the concern targets the headline comparison rather than the method's internal validity. A conditional verdict with a request for baseline retraining or explicit clarification is appropriate; hence the reader's verdict should remain unchanged.
The referee's report
If the controlled comparisons are confirmed, the contribution is solid and timely: the paper integrates two auxiliary prediction targets (gripper keypose and object pointflow) into a continuous diffusion-policy framework, with internal ablations in Table VII showing that each interface separately improves success rate and that combining them yields the best performance. The method is clearly specified, including all losses, attention structure, hyperparameters, and training/inference procedures, and the appendix is unusually complete with per-case real-world results. The central caveat is that the headline quantitative claims are not yet demonstrably controlled: the RLBench2 comparison may mix training datasets between PPI and the baselines, and the real-world success-rate definition is ambiguous. The internal ablations and the attention visualizations are valuable evidence for the mechanism, but the abstract's quantitative claims need a controlled comparison to be fully credible.
Pith's reading of the paper (a machine's summary, not the authors')
The paper argues that the two dominant strategies for bimanual manipulation—keyframe-based policies, which predict sparse gripper poses and execute them with motion planners, and continuous-control policies, which predict actions at every timestep—can be combined in one end-to-end diffusion policy rather than treated as alternatives. The proposed model, PPI, predicts two intermediate interfaces at keyframe timesteps: target gripper poses and the 3D pointflow of the manipulated object (where a set of points sampled on the object moves over the next few keyframes), and conditions continuous action generation on both. The claim is that these interfaces give a continuous policy the spatial localization of keyframe methods while preserving its ability to produce curved, collision-free trajectories. On the RLBench2 bimanual benchmark the method reports an average success rate of 80.8%, a +16.1% improvement over prior baselines, and on four real-world tasks an average +27.5% gain in success rate, with robustness to unseen objects, lighting changes, and distractor objects. If correct, the result shows that keyframe information is best used as conditioning for a continuous policy, not as an execution plan.
What carries the argument. The load-bearing object is the interface-conditioned diffusion transformer: a denoising network whose tokens carry explicit 3D positions and attend in a fixed causal order, with relative 3D offsets encoded through rotary embeddings. Continuous-action tokens attend to everything, keyframe-action tokens attend to pointflow and scene tokens, pointflow tokens attend to scene and language tokens, and all tokens attend to language. Three supporting mechanisms carry the argument. First, a 3D semantic neural field built from multi-view RGBD images, fusing self-supervised visual features into downsampled scene points via a hierarchical point encoder, gives every scene token spatial and semantic identity. Second, initial query points $F_0$ are sampled from the object using open-vocabulary detection and segmentation, anchoring the pointflow prediction to the object rather than to world coordinates; this converts the learning problem from estimating $p(F)$ to estimating $p(F \mid F_0)$. Third, a heuristic keyframe splitter identifies turning points in the demonstration and provides the supervision targets for the keypose interface. Together these components let the continuous policy see where the grippers should be and where the object is going, without ever committing to a motion-planned keyframe trajectory.
Core claim. The central discovery is that a single continuous-action diffusion transformer can localize as precisely as keyframe methods by predicting two auxiliary interfaces at keyframe timesteps: future gripper poses and the future positions of object query points (the pointflow). The model is trained end-to-end with three losses—denoising noise for continuous actions, denoising noise for keyframe actions, and direct regression of the pointflow—and uses a unidirectional attention pattern in which continuous-action tokens attend to language, scene, keypose, and pointflow tokens, while keypose tokens attend to pointflow and scene, and pointflow tokens attend to scene and language. The pointflow interface deliberately estimates the conditional distribution $p(F \mid F_0)$, the future positions of query points given their initial positions on the object, rather than absolute coordinates, which the paper argues is why the policy generalizes to unseen objects and distractors. Empirically, the paper reports 80.8% average success on seven RLBench2 tasks (best checkpoint 82.6%) versus 64.7% for the strongest prior method, and an average 62.5% success across four real-world tasks versus at most 35.0% for the baselines, with the largest margins appearing in localization success and stage-wise scores.
The premise it rests on. The reported simulation margin assumes the comparison is apples-to-apples: the paper regenerates the training data for the seven selected tasks to remove robot shadows and add pointflow metadata, but the baselines are reproduced with official code without explicitly stating they were retrained on that same corrected data, so part of the +16.1% could come from the data cleanup rather than from the proposed interfaces.
If the paper is right (Pith's inferences, not the authors')
Keyframe-based and continuous control need not be mutually exclusive: keyframe information can be injected as conditioning tokens in a continuous diffusion policy, preserving curved and collision-free trajectories.
The object pointflow is the larger source of gain: conditioning continuous actions on pointflow alone raises average success from 47.6% to 74.3%, while keypose alone raises it to 53.6%; combining both reaches 80.8% on RLBench2.
Because pointflow is predicted relative to initial object-anchored query points, the policy can be redirected to a novel object by changing the open-vocabulary prompt at the first timestep, enabling zero-shot manipulation of unseen rigid objects.
The method remains stable under lighting changes and object interference, which the paper attributes to interfaces that combine positional and semantic features rather than raw color.
The approach works with small expert datasets (20 demonstrations per real-world task) and can be sped up roughly four times with modest accuracy loss by reducing scene tokens and denoising steps, yielding a control policy running at about 10.5 Hz.
A natural extension is to replace the pose-tracking-based pointflow labels with point tracks or optical flow at data-collection time; if the conditional formulation is what drives generalization, weaker supervision should still yield most of the benefit.
The unidirectional attention design suggests the interfaces act as a learned compression of the scene for the action head; one testable consequence is that pruning scene tokens should degrade performance more gracefully when interfaces are present than when they are absent, and the reported 6x scene-token reduction in the fast variant is consistent with that.
If the reported margins survive a fair re-benchmark on identical training data, the two-interface recipe could become a standard add-on for behavior-cloning policies: predict a sparse goal and an object-motion field, then denoise dense actions conditioned on both.
The claim that pointflow is robust to distractors implies a concrete stress test: place a visually salient but task-irrelevant object between the camera and the target and measure whether attention weights and success rates remain target-focused, since the paper's visualizations suggest pointflow tokens concentrate on the manipulated object's edges.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Bimanual manipulation is a challenging yet crucial robotic capability, demanding precise spatial localization and versatile motion trajectories, which pose significant challenges to existing approaches. Existing approaches fall into two categories: keyframe-based strategies, which predict gripper poses in keyframes and execute them via motion planners, and continuous control methods, which estimate actions sequentially at each timestep. The keyframe-based method lacks inter-frame supervision, struggling to perform consistently or execute curved motions, while the continuous method suffers from weaker spatial perception. To address these issues, this paper introduces an end-to-end framework PPI (keyPose and Pointflow Interface), which integrates the prediction of target gripper poses and object pointflow with the continuous actions estimation. These interfaces enable the model to effectively attend to the target manipulation area, while the overall framework guides diverse and collision-free trajectories. By combining interface predictions with continuous actions estimation, PPI demonstrates superior performance in diverse bimanual manipulation tasks, providing enhanced spatial localization and satisfying flexibility in handling movement restrictions. In extensive evaluations, PPI significantly outperforms prior methods in both simulated and real-world experiments, achieving state-of-the-art performance with a +16.1% improvement on the RLBench2 simulation benchmark and an average of +27.5% gain across four challenging real-world tasks. Notably, PPI exhibits strong stability, high precision, and remarkable generalization capabilities in real-world scenarios. Project page: https://yuyinyang3y.github.io/PPI/
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-09-23
The authors' main claim, as the classifier picked it out 0.61
“In extensive evaluations, PPI significantly outperforms prior methods in both simulated and real-world experiments, achieving state-of-the-art performance with a +16.1% improvement on the RLBench2 simulation benchmark and an average of +27.5% gain across four challenging real-world tasks.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.06 · states limitation 0.07 · human participants 0.03 · controlled comparison 0.73 · randomized assignment 0.01 · states quantitative result 0.99 · compares with prior results 0.98 · mentions code or data availability 0.28
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Bimanual manipulation is a challenging yet crucial robotic capability, demanding precise spatial localization and versatile motion trajectories, which pose significant challenges to existing approaches. | background 1.00 | 0.05 | 0.03 | asserted 1.00 |
| Existing approaches fall into two categories: keyframe-based strategies, which predict gripper poses in keyframes and execute them via motion planners, and continuous control methods, which estimate actions sequentially at each timestep. | background 1.00 | 0.05 | 0.02 | asserted 1.00 |
| The keyframe-based method lacks inter-frame supervision, struggling to perform consistently or execute curved motions, while the continuous method suffers from weaker spatial perception. | gap 0.58 | 0.08 | 0.02 | asserted 1.00 |
| To address these issues, this paper introduces an end-to-end framework PPI (keyPose and Pointflow Interface), which integrates the prediction of target gripper poses and object pointflow with the continuous actions estimation. | contribution 0.89 | 0.89 | 0.08 | asserted 1.00 |
| These interfaces enable the model to effectively attend to the target manipulation area, while the overall framework guides diverse and collision-free trajectories. | method 0.64 | 0.57 | 0.08 | asserted 0.98 |
| By combining interface predictions with continuous actions estimation, PPI demonstrates superior performance in diverse bimanual manipulation tasks, providing enhanced spatial localization and satisfying flexibility in handling movement restrictions. | result 0.82 | 0.90 | 0.56 | asserted 1.00 |
| In extensive evaluations, PPI significantly outperforms prior methods in both simulated and real-world experiments, achieving state-of-the-art performance with a +16.1% improvement on the RLBench2 simulation benchmark and an average of +27.5% gain across four challenging real-world tasks. | result 1.00 | 0.95 | 0.43 | asserted 1.00 |
| Notably, PPI exhibits strong stability, high precision, and remarkable generalization capabilities in real-world scenarios. | result 0.68 | 0.87 | 0.13 | asserted 1.00 |
| Project page: https://yuyinyang3y.github.io/PPI/ | future work 0.42 | 0.08 | 0.02 | unclear 0.97 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.96
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 7,017 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (15)
Showing 13 of the paper's 15 figures. Figures are the authors', reproduced from the paper.
References (49)
- [1] Homanga Bharadhwaj, Roozbeh Mottaghi, Abhinav Gupta, and Shubham Tulsiani. Track2act: Predicting point tracks from internet videos enables generalizable robot manipulation. In Arxiv, 2024 on Pith
- [2] Wen Bowen, Yang Wei, Kautz Jan, and Birchfield Stan. FoundationPose: Unified 6d pose estimation and tracking of novel objects. In CVPR, 2024 on Pith
- [3] Anthony Brohan, Noah Brown, Justice Carbajal, Yev- gen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al. Rt-1: Robotics transformer for real-world control at scale. In Arxiv, 2022 on Pith
- [4] Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. In RSS, 2023 on Pith
- [5] Shivin Devgon, Jeffrey Ichnowski, Ashwin Balakrishna, Harry Zhang, and Ken Goldberg. Orienting novel 3d objects using self-supervised learning of rotation trans- forms. In CASE, 2020 on Pith
- [6] Danny Driess, Fei Xia, Mehdi SM Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, et al. Palm- e: An embodied multimodal language model. In Arxiv, 2023 on Pith
- [7] Ben Eisner, Harry Zhang, and David Held. Flowbot3d: Learning 3d articulation flow to manipulate articulated objects. In RSS, 2022 on Pith
- [8] Hao-Shu Fang, Chenxi Wang, Hongjie Fang, Minghao Gou, Jirong Liu, Hengxu Yan, Wenhai Liu, Yichen Xie, and Cewu Lu. Anygrasp: Robust and efficient grasp perception in spatial and temporal domains. In TRO, 2023 on Pith
- [9] Markus Grotz, Mohit Shridhar, Tamim Asfour, and Dieter Fox. Peract2: A perceiver actor framework for bimanual manipulation tasks. In Arxiv, 2024 on Pith
- [10] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurIPS, 2020 on Pith
- [11] Stephen James and Andrew J. Davison. Q-attention: Enabling efficient learning for vision-based robotic ma- nipulation. In IEEE Robotics and Automation Letters , 2022 on Pith
- [12] Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Cotracker: It is better to track together. In ECCV, 2024 on Pith
- [13] Tsung-Wei Ke, Nikolaos Gkanatsios, and Katerina Fragkiadaki. 3d diffuser actor: Policy diffusion with 3d scene representations. In CoRL, 2024 on Pith
- [14] Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source vision-language-action model. In CoRL, 2024 on Pith
- [15] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross Girshick. Segment anything. In Arxiv, 2023 on Pith
- [16] I Liu, Chun Arthur, Sicheng He, Daniel Seita, and Gaurav Sukhatme. V oxact-b: V oxel-based acting and stabilizing policy for bimanual manipulation. In CoRL, 2024 on Pith
- [17] Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In Arxiv, 2023 on Pith
- [18] Songming Liu, Lingxuan Wu, Bangguo Li, Hengkai Tan, Huayu Chen, Zhengyi Wang, Ke Xu, Hang Su, and Jun Zhu. Rdt-1b: a diffusion foundation model for bimanual manipulation. In Arxiv, 2024 on Pith
- [19] Maxime Oquab, Timoth ´ee Darcet, Theo Moutakanni, Huy V . V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Russell Howes, Po-Yao Huang, Hu Xu, Vasu Sharma, Shang-Wen Li, Wojciech Galuba, Mike Rabbat, Mido Assran, Nicol... on Pith
- [20] Chuer Pan, Brian Okorn, Harry Zhang, Ben Eisner, and David Held. Tax-pose: Task-specific cross-pose estimation for robot manipulation. In CoRL, 2023 on Pith
show 29 more
- [21] Ethan Perez, Florian Strub, Harm de Vries, Vincent Dumoulin, and Aaron Courville. Film: Visual reasoning with a general conditioning layer. In Proceedings of the AAAI Conference on Artificial Intelligence , 2017 on Pith
- [22] Carl Qi, Yilin Wu, Lifan Yu, Haoyue Liu, Bowen Jiang, Xingyu Lin, and David Held. Learning generalizable tool-use skills through trajectory generation. In IROS, 2024 on Pith
- [23] Charles R. Qi, Li Yi, Hao Su, and Leonidas J. Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In Arxiv, 2017 on Pith
- [24] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Arxiv, 2021 on Pith
- [25] Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Ro- man R ¨adle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. In Arxiv, 2024 on Pith
- [26] Daniel Seita, Yufei Wang, Sarthak J Shetty, Edward Yao Li, Zackory Erickson, and David Held. Toolflownet: Robotic manipulation with tools via predicting tool flow from point clouds. In CoRL, 2023 on Pith
- [27] Mohit Shridhar, Lucas Manuelli, and Dieter Fox. Perceiver-actor: A multi-task transformer for robotic ma- nipulation. In CoRL, 2022 on Pith
- [28] Jiaming Song, Chenlin Meng, and Stefano Ermon. De- noising diffusion implicit models. In Arxiv, 2020 on Pith
- [29] Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Yunfeng Liu. Roformer: Enhanced trans- former with rotary position embedding. In Arxiv, 2023 on Pith
- [30] Ioan A Sucan, Mark Moll, and Lydia E Kavraki. The open motion planning library. In IEEE Robotics & Automation Magazine, 2012 on Pith
- [31] Balakumar Sundaralingam, Siva Kumar Sastry Hari, Adam Fishman, Caelan Garrett, Karl Van Wyk, Valts Blukis, Alexander Millane, Helen Oleynikova, Ankur Handa, Fabio Ramos, et al. Curobo: Parallelized collision-free robot motion generation. In ICRA, 2023 on Pith
- [32] Yang Tian, Jiyao Zhang, Zekai Yin, and Hao Dong. Robot structure prior guided temporal attention for camera-to-robot pose estimation from image sequence. In CVPR, 2023 on Pith
- [33] Yang Tian, Jiyao Zhang, Guowei Huang, Bin Wang, Ping Wang, Jiangmiao Pang, and Hao Dong. Robokeygen: Robot pose and joint angles estimation via diffusion- based 3d keypoint generation. In ICRA, 2024 on Pith
- [34] Yang Tian, Sizhe Yang, Jia Zeng, Ping Wang, Dahua Lin, Hao Dong, and Jiangmiao Pang. Predictive inverse dynamics models are scalable learners for robotic ma- nipulation. In ICLR, 2025 on Pith
- [35] Chenxi Wang, Hongjie Fang, Hao-Shu Fang, and Cewu Lu. Rise: 3d perception makes real-world robot imitation simple and effective. In IROS, 2024 on Pith
- [36] Hanqing Wang, Jiahe Chen, Wensi Huang, Qingwei Ben, Tai Wang, Boyu Mi, Tao Huang, Siheng Zhao, Yilun Chen, Sizhe Yang, et al. Grutopia: Dream general robots in a city at scale. In Arxiv, 2024 on Pith
- [37] Yixuan Wang, Guang Yin, Binghao Huang, Tarik Ke- lestemur, Jiuguang Wang, and Yunzhu Li. Gendp: 3d semantic fields for category-level generalizable diffusion policy. In CoRL, 2024 on Pith
- [38] Yixuan Wang, Mingtong Zhang, Zhuoran Li, Tarik Ke- lestemur, Katherine Driggs-Campbell, Jiajun Wu, Li Fei- Fei, and Yunzhu Li. D 3fields: Dynamic 3d descriptor fields for zero-shot generalizable rearrangement. In CoRL, 2024 on Pith
- [39] Bowen Wen, Jonathan Tremblay, Valts Blukis, Stephen Tyree, Thomas M ¨uller, Alex Evans, Dieter Fox, Jan Kautz, and Stan Birchfield. Bundlesdf: Neural 6-dof tracking and 3d reconstruction of unknown objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patte... on Pith
- [40] Chuan Wen, Xingyu Lin, John So, Kai Chen, Qi Dou, Yang Gao, and Pieter Abbeel. Any-point trajectory modeling for policy learning. In RSS, 2024 on Pith
- [41] Thomas Weng, Sujay Man Bajracharya, Yufei Wang, Khush Agrawal, and David Held. Fabricflownet: Bi- manual cloth manipulation with a flow-based policy. In CoRL, 2022 on Pith
- [42] Philipp Wu, Yide Shentu, Zhongke Yi, Xingyu Lin, and Pieter Abbeel. Gello: A general, low-cost, and intuitive teleoperation framework for robot manipulators. In IROS, 2024 on Pith
- [43] Mengda Xu, Zhenjia Xu, Yinghao Xu, Cheng Chi, Gor- don Wetzstein, Manuela Veloso, and Shuran Song. Flow as the cross-domain manipulation interface. In CoRL, 2024 on Pith
- [44] Dongjie Yu, Hang Xu, Yizhou Chen, Yi Ren, and Jia Pan. Bikc: Keypose-conditioned consistency policy for bimanual robotic manipulation. In Arxiv, 2024 on Pith
- [45] Chengbo Yuan, Chuan Wen, Tong Zhang, and Yang Gao. General flow as foundation affordance for scalable robot learning. In CoRL, 2024 on Pith
- [46] Yanjie Ze, Gu Zhang, Kangning Zhang, Chenyuan Hu, Muhan Wang, and Huazhe Xu. 3d diffusion policy. In RSS, 2024 on Pith
- [47] Harry Zhang, Ben Eisner, and David Held. Flowbot++: Learning generalized articulated objects manipulation via articulation projection. In CoRL, 2023 on Pith
- [48] Tony Z Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-grained bimanual manipulation with low-cost hardware. In RSS, 2023 on Pith
- [49] Yan Zhao, Ruihai Wu, Zhehuan Chen, Yourong Zhang, Qingnan Fan, Kaichun Mo, and Hao Dong. Dualafford: Learning collaborative visual affordance for dual-gripper manipulation. In ICLR, 2023. APPENDIX A. Foundation Models PPI leverages foundation models in various aspects, either ... on Pith
Cited by 2 Pith papers
Reviewed papers on Pith that cite this one. OpenAlex reports about 14 citations worldwide. The full citation record
How the literature uses it
7 arXiv papers cite this in 9 sentences
Papers citing it · 5 sentences without a confident label
Cites it as background 3 papers
-
While some studies utilize 2D flow to capture motion trends on the image plane [ref] , others further lift it to 3D point flow for more accurate spatial modeling [ref] .
X-Imitator: Spatial-Aware Imitation Learning via Bidirectional Action-Pose Interaction · Flow Representations. · 0.91
-
The second family instead predicts more compact structured cues, such as textual rationales or chains of thought [ref] , keyposes and object pointflow [ref] , reconstructive or gaze-region targets [ref] , and implicit 3D/spatial representation alignment [ref] .
AffordanceVLA: A Vision-Language-Action Model Empowering Action Generation through Affordance-Aware Understanding · Vision-Language-Action Models · 0.93
-
Meanwhile, some works are focused on bimanual manipulation [ref] .
BiCoord: A Bimanual Manipulation Benchmark towards Long-Horizon Spatial-Temporal Coordination · Related Works · 0.99
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment