Pith. sign in

REVIEW 3 major objections 6 minor 6 cited by

G3Flow: Generative 3D Semantic Flow for Pose-aware and Generalizable Object Manipulation

T0 review · 3 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read G3Flow constructs a real-time, object-centric 3D semantic flow from a generated digital twin and uses pose tracking to keep it aligned, giving diffusion policies part-level and orientation awareness for precise, generalizable manipulation.

desk verdict The core idea—build a complete semantic field from a generated digital twin before manipulating, then track it with FoundationPose—is new and works in simulation, but the experiments never directly verify the twin and tracker, so the headline gains can't yet be attributed to the full pipeline. read the letter →

arxiv 2411.18369 v3 pith:3NRW3AOD submitted 2024-11-27 cs.RO cs.AIcs.CVcs.SYeess.SY

Tianxing Chen , Yao Mu , Zhixuan Liang , Zanxin Chen , Shijia Peng , Qiangyu Chen , Mingkun Xu , Ruizhen Hu
show 3 more authors
Hongyuan Zhang Xuelong Li Ping Luo
This is my paper · ORCID
classification cs.ROcs.AIcs.CVcs.SYeess.SY
keywords semanticflow3Dgenerativemodeldigitaltwindiffusionpolicyposetrackingobject-centricrepresentationimitationlearningroboticmanipulation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

G3Flow claims that a robot can manipulate objects more precisely and generalize to unseen instances when its imitation-learning policy is fed a complete, object-centric 3D semantic representation that is built once from a generated digital twin and then kept aligned by pose tracking. This representation, called semantic flow, is assembled from DINOv2 features extracted in virtual space and transported to the physical object through FoundationPose tracking, so the policy sees semantically labeled parts even under occlusion. Across five simulated manipulation tasks, the G3Flow-conditioned diffusion policy reaches 68.3% average success on terminal-constrained tasks and 50.1% on cross-object generalization, versus 46.2% and 31.7% for the strongest baselines. The claim matters because it offers a path from geometry-only 3D imitation learning to part-aware, pose-aware manipulation without manual keypoint annotations.

What carries the argument

The central object is semantic flow: a time-varying, object-centric 3D point cloud in which each point carries a PCA-compressed DINOv2 semantic feature. It is generated once in virtual space from a digital twin produced by a 3D generative model, then kept aligned with the physical object by FoundationPose, which transforms the cloud at each timestep according to the estimated pose change. This machinery does the work: it converts a one-time virtual reconstruction into an online, occlusion-resistant semantic observation that the diffusion policy consumes together with real point-cloud geometry and robot joint states.

What would settle it

Use a real shoe with a novel texture and a partly occluded toe, build the digital twin from the multi-view exploration, then align the virtual semantic cloud to the real point cloud and compare the PCA-compressed DINOv2 features in the occluded region against a manual part label; if the toe and heel are swapped or FoundationPose's estimated pose drifts by more than a few degrees over a five-second manipulation, the core claim fails.

Watch

Extended reading notes

Core claim

G3Flow's core claim is that semantic understanding for manipulation can be computed once in a virtual space and then maintained online by rigid pose tracking, instead of being recomputed every timestep from partial real observations. Multi-view RGB views from an active exploration phase let a 3D generative model reconstruct a digital twin; DINOv2 features rendered from virtual views of the twin are PCA-compressed into a semantic point cloud; and FoundationPose tracks the object's 6-DOF pose so the cloud is transformed and stays aligned with the physical object. This semantic flow is fed into a diffusion policy alongside the real point cloud and robot state. The result is consistent gains in pose-aware tasks such as shoe placement, bottle orientation, and tool adjustment, as well as in generalization to geometrically distinct unseen objects and unseen tool categories.

Load-bearing premise

The load-bearing premise is that the digital twin generated from multi-view observations is a faithful, complete stand-in for the physical object, so semantic features rendered from virtual views and pose estimates computed against the twin remain correct on unseen objects and under occlusion; all reported experiments are in simulation.

Editorial extensions

If this is right

  • Feeding semantic flow into a diffusion policy pushes terminal-constrained manipulation success to 68.3% average, against 46.2% for the best baseline.
  • The same flow lifts cross-object generalization to 50.1%, including unseen shoes, bottle sizes, and tool categories.
  • Because semantic features come from a complete virtual twin and are transformed by pose tracking, the representation stays complete under occlusion rather than degrading with partial views.
  • The pipeline removes manual keypoint selection and runs at 34.04 Hz decision frequency, making it compatible with closed-loop control.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial: the rigid single-object tracking assumption means the recipe would need per-part or articulation tracking to extend to articulated or deformable objects; the paper does not claim that extension.
  • Editorial: the reported margins come from simulation; real depth noise and tracking drift would likely shrink the gap, since twin fidelity is the load-bearing premise.
  • Editorial: because DINOv2 is frozen, its part-level semantics are fixed; a stress test on object categories far outside DINOv2's training distribution would reveal how much of the generalization is genuinely semantic.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes G3Flow, an imitation-learning framework that constructs a 3D semantic flow for a manipulated object. The pipeline first performs active multi-view exploration, reconstructs a digital twin with a 3D generative model (Rodin), extracts DINOv2 features from virtual multi-view renderings and compresses them with PCA, and then uses FoundationPose to track the object pose so that the canonical semantic point cloud is transformed into the current frame via Eq. (1). The semantic flow is combined with real point-cloud observations and robot state in a diffusion policy. Experiments on five RoboTwin simulation tasks compare DP, DP3, and DP3-with-color, with additional ablations on semantic-field quality, efficiency, and choice of vision foundation model. The paper reports substantial gains, e.g., 68.3% versus 46.2% average success on terminal-constrained tasks and 50.1% versus 31.7% on cross-object generalization.

Significance. If the reported results hold, the paper offers a practical recipe for injecting object-centric semantic understanding into 3D diffusion policies without manual keypoint annotation, and the reported inference speed is valuable for closed-loop control. The framework is clearly specified, the claims are falsifiable, and standard deviations over three seeds are reported. The main limitation is that the experiments, as described, do not establish that the two novel perceptual components—the generated digital twin and FoundationPose tracking—are actually exercised at test time, because the RoboTwin simulator provides ground-truth mesh and pose. Until this is clarified or measured, the headline gains cannot be cleanly attributed to the complete G3Flow pipeline rather than to the addition of a virtual semantic field to DP3.

major comments (3)
  1. [§4.1 and §3.3] The experimental section never states whether test-time evaluation uses the generated digital twin and the FoundationPose estimates, or the simulator's ground-truth mesh and pose. Since RoboTwin makes both available, the current description is compatible with an oracle setting in which Eq. (1) is evaluated with ground-truth poses and the 3D generative model is bypassed. This would validate only a semantic-field-enhanced DP3, not the proposed 'complete under occlusions' pipeline. Please state explicitly which modules are used at test time, report digital-twin fidelity (e.g., Chamfer distance or appearance metrics) and pose-tracking error, and add an ablation that switches each module between the proposed method and the ground-truth oracle.
  2. [Tables 1 and 2] The most directly related semantic-field baselines, GenDP and D3Fields, appear only in the two-task ablation of Table 4 and are absent from the main success-rate tables. The abstract and introduction claim that G3Flow 'consistently outperforms existing methods,' but the main comparison is only against DP, DP3, and DP3-with-color. The authors should include GenDP and D3Fields in the main tables across all five tasks under the same protocol, or explicitly restrict the comparison claim to geometry-only baselines.
  3. [§4.4, Table 4] The semantic-field ablation covers only Shoe Place (T) and Dual Shoes Place (T), and it does not isolate the contributions of the digital twin, the pose tracker, and the virtual semantic field. Consequently, the paper does not demonstrate which component is responsible for the reported gains, and the cross-object generalization claims are not ablated at all. Please add component ablations (e.g., G3Flow without tracking, G3Flow with ground-truth mesh, G3Flow without the semantic flow) and at least one cross-object generalization task.
minor comments (6)
  1. [§4.3] The sentence 'Our method achieves an average success rate 18.4% higher than strongest baseline' should read '18.4 percentage points higher,' since Table 2 reports 50.1% versus 31.7%.
  2. [§4.4, Table 3] The efficiency table does not define how 'Field-Gen Freq.' and 'Decision-making Freq.' are measured, nor which stages of each baseline are included; please add the measurement protocol and hardware details.
  3. [Tables 1 and 2] The baseline naming is inconsistent: '3D DP' is used in the tables while 'DP3' is used in the text and in Section 4.1; please unify the notation.
  4. [References] Reference [2] is a duplicate of [1], and references [40] and [41] are duplicates of the same DP3 paper; these should be merged.
  5. [§3.4, Eq. (2)] The equation contains the notation 'γk,k,' which appears to be a typo; it should likely be 'γk' or a clearly defined scheduler-dependent scalar.
  6. [§4.4, Table 4] The sentence describing D3Fields as benefiting from 'goal image priors' (C1) is not backed by a citation or a precise definition of what the prior is; please clarify.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the empirical gains are measured by simulator success on independently defined tasks and the semantic-flow derivation reduces to no fitted input or self-citation chain.

full rationale

I walked the paper's derivation chain. The semantic flow is constructed by (i) active multi-view exploration, (ii) Rodin-based digital-twin generation, (iii) DINOv2 feature extraction plus PCA, and (iv) FoundationPose-based rigid alignment via Eq. (1). Each of these modules is external or independently trained; none is defined in terms of the downstream success metric. The policy is trained with the standard diffusion noise-prediction loss in Eq. (3) on 100 expert demonstrations and evaluated by the simulator's success criteria on terminal-constrained and cross-object generalization splits. I found no equation-level reduction: Eq. (1) is simply a rigid transform of a fixed feature cloud, and Eq. (3) is a standard MSE objective. The use of the RoboTwin benchmark [20], which shares authors with this paper, is a code-released simulation testbed with randomized object designs and simulator-defined success; it is not an unverified theorem invoked to force the result, so it does not constitute load-bearing circularity under the hard rules. The same applies to the citation of [18] for multi-view reconstruction necessity: it motivates a design choice but does not define the reported quantities. The legitimate skeptical concern is a validation gap rather than circularity: Sections 3.2 and 3.3 assert that the generated digital twin is faithful and that FoundationPose maintains accurate pose under occlusion, yet Section 4 never reports twin-fidelity or pose-error metrics, and no oracle ablation isolates whether the simulator's ground-truth mesh/pose could be silently replacing the proposed perception stack. That gap affects whether the experiments exercise the full G3Flow pipeline, not whether any claimed derivation is equivalent to its own inputs. Accordingly, the paper has no demonstrable circular step; the score of 1 reflects only the presence of minor non-load-bearing self-citations in the benchmark and motivation, with the central empirical claim retaining independent content.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim relies on the reliability of pre-trained foundation models and the RoboTwin simulation. The only hand-chosen numbers are representation hyperparameters, not fits to the success rates. No new physical entities are introduced; 'semantic flow' is a data representation.

free parameters (4)
  • PCA feature dimension = 5
    Chosen by hand to reduce 384-dim DINOv2 features; affects compactness of semantic flow and is not swept.
  • Feature and real point cloud size = 1024
    Farthest Point Sampling downsamples both real and virtual clouds to 1024 points; chosen by hand.
  • Number of exploration viewpoints = not specified
    Number of wrist-camera views used to build the digital twin; impacts reconstruction quality but no value is given.
  • DINOv2 transform size = 420x420
    Images resized to 420 by 420 pixels for ViT-S/14; chosen by hand.
assumptions (4)
  • domain assumption DINOv2 features from virtual RGB views capture part-level semantics useful for manipulation.
    The semantic flow's value rests on this; Section 3.2.
  • domain assumption FoundationPose estimates and tracks object pose accurately under occlusion given the digital twin.
    Equation (1) transforms the semantic field based on this pose; Section 3.3.
  • domain assumption The 3D generative model reconstructs faithful digital twins from multi-view observations, including plausible hidden parts.
    Section 3.2, 'Object 3D Model Generation'; generation failure corrupts semantic understanding.
  • domain assumption The RoboTwin simulator is a valid proxy for real-world robotic manipulation.
    All experiments are in simulation; Section 4.

how reviews work

0 comments
Cite this review

Pith. "Pith review of G3Flow: Generative 3D Semantic Flow for Pose-aware and Generalizable Object Manipulation." pith.science (2026). https://pith.science/paper/3NRW3AOD

@misc{pith2026241118369,
  author       = {Pith},
  title        = {Pith review of: G3Flow: Generative 3D Semantic Flow for Pose-aware and Generalizable Object Manipulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3NRW3AOD}},
  note         = {Machine review of arXiv:2411.18369}
}
read the original abstract

Recent advances in imitation learning for 3D robotic manipulation have shown promising results with diffusion-based policies. However, achieving human-level dexterity requires seamless integration of geometric precision and semantic understanding. We present G3Flow, a novel framework that constructs real-time semantic flow, a dynamic, object-centric 3D semantic representation by leveraging foundation models. Our approach uniquely combines 3D generative models for digital twin creation, vision foundation models for semantic feature extraction, and robust pose tracking for continuous semantic flow updates. This integration enables complete semantic understanding even under occlusions while eliminating manual annotation requirements. By incorporating semantic flow into diffusion policies, we demonstrate significant improvements in both terminal-constrained manipulation and cross-object generalization. Extensive experiments across five simulation tasks show that G3Flow consistently outperforms existing approaches, achieving up to 68.3% and 50.1% average success rates on terminal-constrained manipulation and cross-object generalization tasks respectively. Our results demonstrate the effectiveness of G3Flow in enhancing real-time dynamic semantic feature understanding for robotic manipulation policies.

Figures

Figures reproduced from arXiv: 2411.18369 by the authors.

Figure 1
Figure 1. Motivation of G3Flow. Our approach leverages 3D generative model and language-guided detection model to gen￾erate 3D semantic flow (top). Through continuous field track￾ing, G3Flow enables pose-aware and generalizable manipulation, demonstrating superior performance across terminal constraint control and cross-object generalization tasks over multiple base￾lines (DP, DP3, and DP3 w/ color) (bottom). methods utilizin… view at source ↗
Figure 2
Figure 2. Pipeline of G3Flow. Our framework consists of (top) an initialization phase that generates comprehensive 3D representation (surface normals, wireframe, and geometry) through object-centric exploration and digital twin generation, which enables rich semantic field extraction, and (bottom) a control execution phase where real-time pose tracking maintains dynamic semantic fields to guide diffusion￾based manipulation ac… view at source ↗
Figure 3
Figure 3. Failure mode of single-view 3D generation. When using a single view for 3D generation, certain geometric details may be inaccurately reconstructed due to occlusion, even if the result appears plausible from a commonsense perspective. structed twins serve dual purposes: providing a basis for comprehensive semantic feature extraction and enabling ac￾curate pose tracking during dynamic interactions. Virtual Semantic Fl… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Spatial alignment via object tracking. We achieve alignment between the semantic flow and the physical object in real world by synchronizing the relative transformations of the ob￾ject coordinate system to the world coordinate system. pendent of real-world observation …
Figure 7
Figure 7. Figure 7: Seen and unseen object sets for four tasks with high terminal constraint requirements. employ PCA to reduce the feature dimensions of DINOv2 to 5, and downsample both the original point cloud and the virtual point cloud to 1024 points. Baselines: We use the 3D Diffusio…
Figure 8
Figure 8. Figure 8: Cross-object generalization settings. Seen and unseen object sets for four tasks with high generalization requirements. baseline methods in achieving pose-aware requirements across all four tasks. Our method achieves over 25% higher success rates in the Shoe Place (T) …
Figure 9
Figure 9. Figure 9: Feature Quality Visualization. A: raw RGB, B: G3Flow, C and D: Scene-level DINOv2 feature. Shoe Place Dual Shoes Place Tool Adjust Bottle Adjust Diverse Bottles Pick [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: Visualization of G3Flow in the 5 evaluation tasks. Shoe Place (T) Dual Shoes Place (T) DP w/ Scene-Level Feature 67.7±1.5 17.0±1.7 DP w/ D3Fields (GenDP) 73.7±2.5 20.3±6.8 DP w/ G3Flow 83.0±3.6 24.0±3.6 [PITH_FULL_IMAGE:figures/full_fig_p008_10.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ControlVLA: Few-shot Object-centric Adaptation for Pre-trained Vision-Language-Action Models

    cs.RO 2025-06 conditional novelty 6.0 of 10

    ControlVLA adapts a DROID-pretrained diffusion VLA policy to new manipulation tasks with 10 to 20 demos by injecting object-centric features through zero-initialized cross-attention layers, achieving 76.7% success acr...

  2. RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins

    cs.RO 2025-04 conditional novelty 6.0 of 10

    Pre-training on RoboTwin's generative digital twins and fine-tuning on 20 real demonstrations raises dual-arm task success from about 20% to 62% and single-arm success from about 1% to 72%.

  3. CordViP: Correspondence-based Visuomotor Policy for Dexterous Manipulation in Real-World

    cs.RO 2025-02 conditional novelty 6.0 of 10

    CordViP achieves strong real-world dexterous manipulation by feeding a diffusion policy with pose-tracked 3D object models and hand point clouds, pretrained on contact maps and arm-hand coordination.

  4. AffordDP: Generalizable Diffusion Policy with Transferable Affordance

    cs.RO 2024-12 conditional novelty 6.0 of 10

    A diffusion-based manipulation policy conditioned on transferred 3D contact points and post-contact trajectories, with adaptive affordance-guided sampling, generalizes to unseen object instances and categories.

  5. DexHandDiff: Interaction-aware Diffusion Planning for Adaptive Dexterous Manipulation

    cs.RO 2024-11 conditional novelty 6.0 of 10

    A dual-phase diffusion planner with dynamics-consistency and LLM-written guidance achieves strong success on goal-adaptive dexterous manipulation in simulation.

  6. Grounding Language Models with Semantic Digital Twins for Robotic Planning

    cs.RO 2025-06 reject novelty 4.0 of 10

    The system grounds an LLM's action plans in hand-built semantic rules about a simulated home and reports success on all 14 selected ALFRED tasks.

Reference graph

Works this paper leans on

41 extracted references · 16 canonical work pages · cited by 6 Pith papers

  1. [2]

    Diffusion policy: Visuomotor policy learning via action dif- fusion

    Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action dif- fusion. The International Journal of Robotics Research, page 02783649241273668, 2023

    work page 2023

  2. [3]

    Get3d: A genera- tive model of high quality 3d textured shapes learned from images

    Jun Gao, Shuyang Song, Luming Tang, Xiaohui Wang, Arie Zhang, Sanja Fidler, and Ming-Yu Liu. Get3d: A genera- tive model of high quality 3d textured shapes learned from images. arXiv preprint arXiv:2209.11163, 2022

    arXiv 2022

  3. [4]

    Act3d: 3d feature field transformers for multi-task robotic manipulation

    Theophile Gervet, Zhou Xian, Nikolaos Gkanatsios, and Ka- terina Fragkiadaki. Act3d: 3d feature field transformers for multi-task robotic manipulation. In 7th Annual Conference on Robot Learning, 2023

    work page 2023

  4. [5]

    Rvt-2: Learning precise manipulation from few demonstrations

    Ankit Goyal, Valts Blukis, Jie Xu, Yijie Guo, Yu-Wei Chao, and Dieter Fox. Rvt-2: Learning precise manipulation from few demonstrations. arXiv preprint arXiv:2406.08545, 2024

    arXiv 2024

  5. [6]

    Peract2: Benchmarking and learn- ing for robotic bimanual manipulation tasks

    Markus Grotz, Mohit Shridhar, Yu-Wei Chao, Tamim As- four, and Dieter Fox. Peract2: Benchmarking and learn- ing for robotic bimanual manipulation tasks. In CoRL 2024 Workshop on Whole-body Control and Bimanual Manipula- tion: Applications in Humanoids and Beyond, 2024

    work page 2024

  6. [7]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. CoRR, abs/2006.11239, 2020

    arXiv 2006

  7. [8]

    Planning with diffusion for flexible behavior syn- thesis

    Michael Janner, Yilun Du, Joshua Tenenbaum, and Sergey Levine. Planning with diffusion for flexible behavior syn- thesis. In International Conference on Machine Learning , pages 9902–9915. PMLR, 2022

    work page 2022

  8. [9]

    Shap-e: Generat- ing conditional 3d implicit functions

    Heewoo Jun and Alex Nichol. Shap-e: Generat- ing conditional 3d implicit functions. arXiv preprint arXiv:2305.02463, 2023

    arXiv 2023

Show all 41 references
  1. [10]

    3d diffuser actor: Policy diffusion with 3d scene rep- resentations

    Tsung-Wei Ke, Nikolaos Gkanatsios, and Katerina Fragki- adaki. 3d diffuser actor: Policy diffusion with 3d scene rep- resentations. arXiv preprint arXiv:2402.10885, 2024

    2024 arXiv

  2. [11]

    Principal component analysis (pca)

    Takio Kurita. Principal component analysis (pca). Computer vision: a reference guide, pages 1–4, 2019

    2019

  3. [12]

    Adaptdiffuser: Diffusion mod- els as adaptive self-evolving planners

    Zhixuan Liang, Yao Mu, Mingyu Ding, Fei Ni, Masayoshi Tomizuka, and Ping Luo. Adaptdiffuser: Diffusion mod- els as adaptive self-evolving planners. In International Con- ference on Machine Learning, pages 20725–20745. PMLR, 2023

    2023

  4. [13]

    Skilldiffuser: Interpretable hi- erarchical planning via skill abstractions in diffusion-based task execution

    Zhixuan Liang, Yao Mu, Hengbo Ma, Masayoshi Tomizuka, Mingyu Ding, and Ping Luo. Skilldiffuser: Interpretable hi- erarchical planning via skill abstractions in diffusion-based task execution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,...

    2024

  5. [14]

    Dexdiffuser: Interaction-aware dif- fusion planning for adaptive dexterous manipulation

    Zhixuan Liang, Yao Mu, Yixiao Wang, Fei Ni, Tianxing Chen, Wenqi Shao, Wei Zhan, Masayoshi Tomizuka, Ping Luo, and Mingyu Ding. Dexdiffuser: Interaction-aware dif- fusion planning for adaptive dexterous manipulation. arXiv preprint arXiv:2411.18562, 2024

    2024 arXiv

  6. [15]

    Magic3d: High- resolution text-to-3d content creation

    Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fi- dler, Ming-Yu Liu, and Tsung-Yi Lin. Magic3d: High- resolution text-to-3d content creation. arXiv preprint arXiv:2211.10440, 2023

    2023 arXiv

  7. [16]

    Data scaling laws in imi- tation learning for robotic manipulation

    Fanqi Lin, Yingdong Hu, Pingyue Sheng, Chuan Wen, Ji- acheng You, and Yang Gao. Data scaling laws in imi- tation learning for robotic manipulation. arXiv preprint arXiv:2410.18647, 2024

    2024 arXiv

  8. [17]

    Rdt-1b: a diffusion foundation model for bimanual manipu- lation

    Songming Liu, Lingxuan Wu, Bangguo Li, Hengkai Tan, Huayu Chen, Zhengyi Wang, Ke Xu, Hang Su, and Jun Zhu. Rdt-1b: a diffusion foundation model for bimanual manipu- lation. arXiv preprint arXiv:2410.07864, 2024

    2024 arXiv

  9. [18]

    Avr: Active vision-driven robotic preci- sion manipulation with viewpoint and focal length optimiza- tion, 2025

    Yushan Liu, Shilong Mu, Xintao Chao, Zizhen Li, Yao Mu, Tianxing Chen, Shoujie Li, Chuqiao Lyu, Xiao ping Zhang, and Wenbo Ding. Avr: Active vision-driven robotic preci- sion manipulation with viewpoint and focal length optimiza- tion, 2025

    2025

  10. [19]

    Manicm: Real-time 3d diffu- sion policy via consistency model for robotic manipulation

    Guanxing Lu, Zifeng Gao, Tianxing Chen, Wenxun Dai, Zi- wei Wang, and Yansong Tang. Manicm: Real-time 3d diffu- sion policy via consistency model for robotic manipulation. arXiv preprint arXiv:2406.01586, 2024

    2024 arXiv

  11. [20]

    Robotwin: Dual-arm robot benchmark with generative digi- tal twins (early version)

    Yao Mu, Tianxing Chen, Shijia Peng, Zanxin Chen, Zeyu Gao, Yude Zou, Lunkai Lin, Zhiqiang Xie, and Ping Luo. Robotwin: Dual-arm robot benchmark with generative digi- tal twins (early version). arXiv preprint arXiv:2409.02920, 2024

    2024 arXiv

  12. [21]

    Metadiffuser: Diffusion model as conditional planner for offline meta-rl

    Fei Ni, Jianye Hao, Yao Mu, Yifu Yuan, Yan Zheng, Bin Wang, and Zhixuan Liang. Metadiffuser: Diffusion model as conditional planner for offline meta-rl. In International Con- ference on Machine Learning, pages 26087–26105. PMLR, 2023

    2023

  13. [22]

    Point-e: A system for generat- ing 3d point clouds from complex prompts

    Alex Nichol, Heewoo Jun, Prafulla Dhariwal, Pamela Mishkin, and Mark Chen. Point-e: A system for generat- ing 3d point clouds from complex prompts. arXiv preprint arXiv:2212.08751, 2022

    2022 arXiv

  14. [23]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023

    2023 arXiv

  15. [24]

    Imitating human behaviour with diffusion models, 2023

    Tim Pearce, Tabish Rashid, Anssi Kanervisto, Dave Bignell, Mingfei Sun, Raluca Georgescu, Sergio Valcarcel Macua, Shan Zheng Tan, Ida Momennejad, Katja Hofmann, and Sam Devlin. Imitating human behaviour with diffusion models, 2023

    2023

  16. [25]

    Dreamfusion: Text-to-3d using 2d diffusion

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.14988, 2022

    2022 arXiv

  17. [26]

    Open-vocabulary mobile manipulation in un- seen dynamic environments with 3d semantic maps

    Dicong Qiu, Wenzong Ma, Zhenfu Pan, Hui Xiong, and Jun- wei Liang. Open-vocabulary mobile manipulation in un- seen dynamic environments with 3d semantic maps. arXiv preprint arXiv:2406.18115, 2024

    2024 arXiv

  18. [27]

    Grounded sam: Assembling open-world models for diverse visual tasks

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, et al. Grounded sam: Assembling open-world models for diverse visual tasks. arXiv preprint arXiv:2401.14159 , 2024

    2024 arXiv

  19. [28]

    Goal-conditioned imitation learning using score- based diffusion policies

    Moritz Reuss, Maximilian Li, Xiaogang Jia, and Rudolf Li- outikov. Goal-conditioned imitation learning using score- based diffusion policies. arXiv preprint arXiv:2304.02532, 2023

    2023 arXiv

  20. [29]

    Should EBMs model the energy or the score? In Energy Based Models Workshop - ICLR 2021, 2021

    Tim Salimans and Jonathan Ho. Should EBMs model the energy or the score? In Energy Based Models Workshop - ICLR 2021, 2021

    2021

  21. [30]

    Distilled feature fields enable few-shot language-guided manipulation

    William Shen et al. Distilled feature fields enable few-shot language-guided manipulation. arXiv preprint arXiv:2307.12345, 2023

    2023 arXiv

  22. [31]

    Perceiver- actor: A multi-task transformer for robotic manipulation

    Mohit Shridhar, Lucas Manuelli, and Dieter Fox. Perceiver- actor: A multi-task transformer for robotic manipulation. In Conference on Robot Learning , pages 785–799. PMLR, 2023

    2023

  23. [32]

    Revisiting energy based models as policies: Ranking noise contrastive estimation and interpolating energy models, 2023

    Sumeet Singh, Stephen Tu, and Vikas Sindhwani. Revisiting energy based models as policies: Ranking noise contrastive estimation and interpolating energy models, 2023

    2023

  24. [33]

    Weiss, Niru Mah- eswaranathan, and Surya Ganguli

    Jascha Sohl-Dickstein, Eric A. Weiss, Niru Mah- eswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics, 2015

    2015

  25. [34]

    Ar- ticulated object manipulation using online axis estimation with sam2-based tracking

    Xi Wang, Tianxing Chen, Qiaojun Yu, Tianling Xu, Zanxin Chen, Yiting Fu, Cewu Lu, Yao Mu, and Ping Luo. Ar- ticulated object manipulation using online axis estimation with sam2-based tracking. arXiv preprint arXiv:2409.16287, 2024

    2024 arXiv

  26. [35]

    Gendp: 3d semantic fields for category-level generalizable diffusion policy.arXiv preprint arXiv:2410.17488, 2024

    Yixuan Wang, Guang Yin, Binghao Huang, Tarik Keleste- mur, Jiuguang Wang, and Yunzhu Li. Gendp: 3d semantic fields for category-level generalizable diffusion policy.arXiv preprint arXiv:2410.17488, 2024

    2024 arXiv

  27. [36]

    D3fields: Dynamic 3d descriptor fields for zero- shot generalizable rearrangement

    Yixuan Wang, Mingtong Zhang, Zhuoran Li, Tarik Keleste- mur, Katherine Driggs-Campbell, Jiajun Wu, Li Fei-Fei, and Yunzhu Li. D3fields: Dynamic 3d descriptor fields for zero- shot generalizable rearrangement. In 8th Annual Conference on Robot Learning, 2024

    2024

  28. [37]

    Foundationpose: Unified 6d pose estimation and tracking of novel objects

    Bowen Wen, Wei Yang, Jan Kautz, and Stan Birchfield. Foundationpose: Unified 6d pose estimation and tracking of novel objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17868– 17879, 2024

    2024

  29. [38]

    Chaineddiffuser: Uni- fying trajectory diffusion and keypose prediction for robotic manipulation

    Zhou Xian, Nikolaos Gkanatsios, Theophile Gervet, Tsung- Wei Ke, and Katerina Fragkiadaki. Chaineddiffuser: Uni- fying trajectory diffusion and keypose prediction for robotic manipulation. In 7th Annual Conference on Robot Learning, 2023

    2023

  30. [39]

    Rodin: A generative model for sculpting 3d digital avatars using diffusion

    Can Xu, Xiaodong Zhang, Yu Sun, Xia Wang, Ziyu Zhang, Yichang Wang, Chang Wang, Wei Wang, Zhen Liu, Li Wang, et al. Rodin: A generative model for sculpting 3d digital avatars using diffusion. arXiv preprint arXiv:2303.04137 , 2023

    2023 arXiv

  31. [40]

    3d diffusion policy: Gen- eralizable visuomotor policy learning via simple 3d repre- sentations

    Yanjie Ze, Gu Zhang, Kangning Zhang, Chenyuan Hu, Muhan Wang, and Huazhe Xu. 3d diffusion policy: Gen- eralizable visuomotor policy learning via simple 3d repre- sentations. In Proceedings of Robotics: Science and Systems (RSS), 2024

    2024

  32. [41]

    3d diffusion policy

    Yanjie Ze, Gu Zhang, Kangning Zhang, Chenyuan Hu, Muhan Wang, and Huazhe Xu. 3d diffusion policy. arXiv preprint arXiv:2403.03954, 2024

    2024 arXiv

  33. [42]

    Learning fine-grained bimanual manipulation with low-cost hardware

    Tony Z Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-grained bimanual manipulation with low-cost hardware. arXiv preprint arXiv:2304.13705, 2023. G3Flow: Generative 3D Semantic Flow for Pose-aware and Generalizable Object Manipulation Supplementary Materia...

    2023 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.