Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Adaptive Articulated Object Manipulation On The Fly with Foundation Model Reasoning and Part Grounding

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read AdaRPG claims that segmenting an articulated object into functional parts and scoring each part's surface with an affordance network makes manipulation skills transfer to novel object categories.

desk verdict Category overlap undermines the headline generalization claim, but the method is a solid modular recipe with credible results on genuinely held-out categories. read the letter →

arxiv 2507.18276 v1 pith:2LSXEFZZ submitted 2025-07-24 cs.RO cs.CV

Xiaojie Zhang , Yuanfei Wang , Ruihai Wu , Kunqi Xu , Yu Li , Liuyu Xiang , Hao Dong , Zhaofeng He This is my paper · ORCID
classification cs.ROcs.CV
keywords articulatedobjectmanipulationaffordancelearningfoundationmodelspartgroundingcross-categorygeneralizationadaptivecode-as-policiespointcloud
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AdaRPG argues that the main obstacle to manipulating never-seen articulated objects is not the object as a whole but its individual functional parts. It builds a pipeline where GPT-4o describes and grounds the relevant part, GroundingDINO and SAM segment it, and a part-level affordance model trained only on detached part point clouds chooses where to grasp and act. Then GPT-4o writes Python control code that calls primitive skill functions such as grasp, pull, push, and rotate in an adaptive retry loop. The paper reports success rates of 0.70 to 1.00 across seven simulated categories and 8 to 10 out of 10 on four real-world objects, outperforming methods that train on all test categories or reason over whole objects. The central claim is that part geometry is similar enough across categories to make affordance transfer work, so a single reusable recipe can replace per-category policies.

What carries the argument

The load-bearing object is the part-level affordance model and its dataset \(\mathcal{D}_{\text{pa}} = \{(O_i, p_i, r_i)\}\), where \(O_i\) is a part point cloud, \(p_i\) is a point, and \(r_i\) is a binary actionability label. The model is a PointNet++ encoder followed by an MLP and sigmoid, trained with binary cross-entropy loss \(\mathcal{L}_V = \mathrm{BCELoss}_{\mathcal{D}_{\text{pa}}}(r_i, V(O_i, p_i))\). At inference, the pipeline is a grounding chain: GPT-4o writes a detailed description of the actionable part, GroundingDINO converts that description to a bounding box, SAM segments the mask, the mask is back-projected onto depth to build a part point cloud, and the affordance model emits per-point scores that select a grasp pose. The six atomic skill functions operate in the end-effector frame under impedance control and return success flags, which the LLM-generated Python code uses in adaptive retry loops.

What would settle it

Run the same AdaRPG pipeline on articulated objects whose functional parts come from categories absent from Table 1 (for example, a folding ladder, an umbrella, or a sliding shower door) and compare success rates; alternatively, compute the geometric nearest-neighbor distance between each AdaManip test part and the PartNet-Mobility training parts and check whether success falls as that distance grows.

Watch

Extended reading notes

Core claim

The paper's central claim is that adaptive manipulation of novel articulated objects can be decomposed into three reusable pieces: grounding the object's functional part with frozen foundation models, scoring that part's surface with an affordance network trained only on detached part point clouds, and having an LLM compose primitive functions into an adaptive program. It reports that this decomposition lifts simulation success rates to 0.84, 0.73, 1.00, 0.80, 0.84, 0.78, and 0.70 across bottle, pen, pressure cooker, coffee machine, window, door, and lamp in the AdaManip environment, and 9/10, 10/10, 9/10, and 8/10 on a real pressure cooker, microwave, bottle, and lamp. The paper attributes the gains to the fact that parts share greater local geometric similarity than whole objects, so a part-level affordance model generalizes across categories that a whole-object model cannot.

Load-bearing premise

The evaluation treats the test objects as novel even though several of the affordance-model training categories are the same as the test categories; if those test objects closely resemble training parts, the reported gains could come from category similarity rather than true part-level generalization.

Editorial extensions

If this is right

  • A single part-affordance model, trained once on detached part clouds, transfers across categories and environments without retraining, so new object types can be absorbed by updating only the grounding and code-generation prompts.
  • Because the ablation without GPT-4o part descriptions drops success to near zero, reliable open-vocabulary part grounding is a first-order requirement for the whole pipeline, not a cosmetic addition.
  • The affordance-guided contact selection adds roughly 15 percentage points of average success over using the part center, confirming that part-level actionability scoring is doing the work.
  • LLM-generated control loops that retry rotation and periodically test a pull can express hidden-state adaptation without task-specific scripts, allowing the same prompt to drive all seven simulated categories.
  • The method outperforms an imitation policy trained on all seven test categories while itself never seeing test objects, suggesting part-level priors can beat category-level demonstrations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: a cleaner test of the paper's premise would train the affordance model on part categories disjoint from the test categories in Table 4; the current overlap between Tables 1 and 4 leaves open whether the gains are part-level or category-level.
  • Editorial inference: the generated code's pull attempt becomes more likely as rotation count increases, which is a stochastic proxy rather than a direct observation of lock state, so replacing it with a learned state estimator could improve mechanisms with multiple hidden states.
  • Editorial inference: because the affordance dataset stores parts without full-object context, it can be reused for other downstream skills such as pressing, twisting, or sliding by re-annotating the affordance surface plane, giving the dataset a life beyond the six primitive functions.
  • Editorial inference: all perception and reasoning models are frozen, so AdaRPG's ceiling is set by pretrained segmentation and code-generation quality rather than by the amount of robot experience collected, which means the approach should improve automatically as those foundation models improve.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes AdaRPG, a framework for adaptive articulated-object manipulation that combines foundation-model-based part grounding (GPT-4o, GroundingDINO, SAM) with a part-level affordance model trained on a newly annotated PartNet-Mobility-derived dataset, and GPT-4o-generated Python control code that invokes primitive manipulation skills. The central claim is that the part-level representation enables cross-category generalization to 'entirely novel' articulated object categories. The paper reports simulation success rates across seven categories (Tab. 4) and real-world success rates across four object types (Tab. 5), consistently outperforming baselines such as SAGE, CoPa, and AdaManip, as well as two ablations.

Significance. If the claims are correct, the paper makes a useful contribution: it shows that a modular pipeline of frozen foundation models for part grounding, a lightweight part-affordance network, and LLM-generated adaptive control code can solve long-horizon manipulation tasks without training on the test objects. The part-level affordance dataset is also a potentially reusable resource. The strongest evidence is the performance on pressure cooker and lamp, which are not among the training categories; both simulation and real-world results on those categories are markedly better than the baselines. However, the breadth of the cross-category generalization claim is not supported by the current evaluation because five of the seven simulation categories and one of the four real-world categories overlap with the training categories. The paper also omits statistical detail (trial counts, seeds, variance) for the simulation experiments, making it difficult to assess the reliability of the reported margins.

major comments (4)
  1. [Sec. 4.1, Tab. 1, Tab. 4] The statement in Sec. 4.1 that the framework 'is tested on entirely novel object categories' is directly contradicted by the paper's own tables. Table 1 lists Window, Bottle, CM, Door, Pen, Safe, Faucet, Pot, Microwave, Switch, and Toaster as the source categories of the part-affordance training data, while Table 4 evaluates on Bottle, Pen, PC, CM, Window, Door, and Lamp. Five of the seven simulation test categories (Bottle, Pen, CM, Window, Door) are also training categories, and the real-world test set in Table 5 includes Microwave, another training category. Since the central claim is cross-category generalization, this overlap is load-bearing. The authors should either (a) provide object-level evidence that the test instances are disjoint from all training instances and that no part geometry from the same categories is shared, (b) re-run the evaluation with training categories fully disjoint from test categories, or (c) explicitly restrict the 'novel category' claim to PC and Lamp and present the overlapping categories as within-category or near-category generalization. As written, the reported gains on the overlapping categories may be inflated by category-level part similarity rather than true part-level generalization.
  2. [Tabs. 3 and 4] The simulation results are reported as single point estimates with no number of trials, no seeds, and no measure of variance (standard deviation, confidence interval, or per-seed range). For a policy whose generated code contains stochastic elements (e.g., the np.random.rand() condition in Fig. 4) and whose perception pipeline itself may be stochastic, this makes it impossible to determine whether the reported margins over baselines and ablations are statistically meaningful. The paper should report the number of episodes per category, the number of random seeds, and the variance across seeds for success rate, affordance F1, and segmentation IoU.
  3. [Sec. 3.1 and Sec. 4.1] The evaluation environment AdaManip [49] is authored by the same group, and the part-affordance training data are extracted from PartNet-Mobility, the same underlying dataset family used to construct many articulated-object benchmarks. The paper states that 'our affordance model is not trained on the AdaManip dataset,' but it does not establish that the AdaManip test objects are distinct object instances from the PartNet-Mobility objects used to extract training parts. If the test objects or their parts are drawn from the same source models, the affordance model could be memorizing part geometry rather than generalizing. The authors should specify the exact source of each AdaManip test object and provide a formal train/test split at the object-instance level, not merely at the category level.
  4. [Sec. 4.4, Tab. 5] The real-world results, while encouraging, are reported as counts out of 10 with no trial protocol details (e.g., number of distinct physical objects per category, initial configurations, whether successes are all-or-nothing task completion or partial progress). This makes it hard to compare against the simulation numbers and to assess how much of the 'real-world performance surpasses simulation' observation is due to task difficulty differences rather than domain-gap effects. The authors should describe the real-world task definitions, the number of unique objects per category, and the success criteria in enough detail that the experiments could be reproduced.
minor comments (5)
  1. [Sec. 4.1, Tab. 2] The segmentation IoU values in Table 2 are reported without any indication of the number of images or objects evaluated, nor any per-instance variance. Adding this information would strengthen the claim that grounding is robust.
  2. [Sec. 3.4, Fig. 4] The probability condition 'np.random.rand() < 0.1*step' in the generated code is a hand-tuned-looking schedule; the paper should clarify whether this schedule is produced autonomously by GPT-4o from the general prompt, or whether the prompt or post-processing enforces this specific form. This matters because the adaptive behavior is partly a property of this schedule.
  3. [Sec. 3.1] The annotation procedure says 'high-affordance points are automatically computed based on the part's center and bounding box,' but the exact rule is not specified. A precise definition (e.g., radius, kernel, or threshold) would make the dataset construction reproducible.
  4. [Sec. 2.1] The related-work discussion of AdaManip states that it 'introduces five distinct adaptive mechanisms,' but the experiments here use seven categories; the relationship between the mechanisms and the chosen categories is not explained, and it would help the reader to know which mechanisms appear in which test category.
  5. [Sec. 4.4] The real-world section says the pipeline selects 'the point with the highest affordance score,' while the simulation section (Sec. 3.3) uses a threshold epsilon and averages over selected points. The discrepancy should be acknowledged or reconciled.

Circularity Check

1 steps flagged · score 4.0 of 10

Five of seven simulated test categories overlap the part-affordance training categories, so the 'entirely novel object categories' claim is not structurally satisfied; the central cross-category result is partially fitted-input-called-prediction, though PC and Lamp still give independent evidence.

  1. fitted input called prediction [Sec. 3.1 (Table 1) vs Sec. 4.1 and Sec. 4.3.3 (Tables 3–4)]
    ""our part affordance training set derived from PartNet-Mobility lacks object-level annotations. This ensures that our framework is tested on entirely novel object categories, making it an unbiased assessment of its generalization capability" ... "We select 11 categories of object from PartNet-Mobility and extract key functional parts, such as handles, buttons, and knobs.""

    The affordance model is trained on PartNet-Mobility part point clouds from 11 categories, including Window, Bottle, CM (coffee machine), Door, and Pen (Table 1). The simulation evaluation then reports success on Bottle, Pen, PC, CM, Window, Door, and Lamp (Table 4). Five of seven test categories are therefore also training categories. The paper argues novelty from the absence of full-object context and from not training on AdaManip meshes, but the part-level input distribution still includes the same categorical part types. Bottle caps, door handles, window handles, and pen tips from the same categories share substantial geometry, so the model can exploit category-level similarity rather than purely part-level transfer.

full rationale

The paper's core method is not circular in the formal sense: the affordance loss (Eq. 1), part grounding pipeline, and GPT-4o code generation are each defined independently, and no result is a direct algebraic restatement of its inputs. The main circularity risk is empirical. Section 4.1 states the framework 'is tested on entirely novel object categories,' yet Section 3.1's training categories (Window, Bottle, CM, Door, Pen) overlap five of the seven simulation test categories (Bottle, Pen, PC, CM, Window, Door, Lamp). Because the trained component is the part-affordance model, and parts within those categories are geometrically similar, the favorable scores on overlapping categories can be explained by category-level familiarity rather than genuine part-level generalization. The disjoint categories PC and Lamp (success 1.00 and 0.70 in simulation; 10/10 and 8/10 in the real world) still support a weaker cross-category claim. The AdaManip test environment is from overlapping authors (ref [49]), but that is a minor self-citation concern because the environment is a published benchmark and the comparison includes other baselines; it does not by itself force the result. The rotation-to-pull probability schedule in the generated control code is hand-stated rather than learned, but the paper does not claim that schedule is learned, so it is a design choice, not a circular prediction. Overall, the central claim retains independent content in the genuinely novel categories, but the stated seven-category 'entirely novel' evaluation is inflated by training/test category overlap, warranting a partial circularity score of 4 rather than a clean 0-2.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim relies on a small number of hand-set control parameters (threshold, rotation schedule, step sizes) and several domain assumptions about part-geometry generalization, GPT-4o prompt quality, and the novelty of the test objects. No new physical entities are introduced.

free parameters (3)
  • Affordance threshold epsilon = not reported
    In Sec 3.3, points with affordance score above epsilon are selected as interaction targets; the value is hand-set and not stated.
  • Rotation-to-pull probability coefficient = 0.1 per step
    The GPT-4o prompt includes a fixed schedule (np.random.rand() < 0.1*step) that controls when the robot switches from rotating to pulling; this is hand-designed, not learned.
  • Primitive skill step sizes = not reported
    Translation and rotation actions use 'a fixed step' and 'controlled rotations' (Sec 3.3), with no values or tuning procedure given.
assumptions (4)
  • domain assumption Parts have greater local geometric similarity than whole objects across categories.
    Stated in Sec 1 as the core motivation for part-level affordance; it is plausible but not proven or quantified.
  • domain assumption The AdaManip test objects are novel with respect to the PartNet-Mobility part training set.
    Sec 4.1 claims novelty, but category names overlap between Tab 1 and Tab 4, so this assumption is questionable.
  • domain assumption Detailed part descriptions from GPT-4o improve GroundingDINO localization enough for reliable segmentation.
    Sec 3.2 says 'extensive experimental trials confirm' but no comparison numbers are given for prompt variants.
  • domain assumption Impedance control with the described correction mechanism can execute generated code on diverse objects without per-object tuning.
    Sec 3.3 asserts robustness, but no stability or repeatability data are reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Adaptive Articulated Object Manipulation On The Fly with Foundation Model Reasoning and Part Grounding." pith.science (2026). https://pith.science/paper/2LSXEFZZ

@misc{pith2026250718276,
  author       = {Pith},
  title        = {Pith review of: Adaptive Articulated Object Manipulation On The Fly with Foundation Model Reasoning and Part Grounding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2LSXEFZZ}},
  note         = {Machine review of arXiv:2507.18276}
}
read the original abstract

Articulated objects pose diverse manipulation challenges for robots. Since their internal structures are not directly observable, robots must adaptively explore and refine actions to generate successful manipulation trajectories. While existing works have attempted cross-category generalization in adaptive articulated object manipulation, two major challenges persist: (1) the geometric diversity of real-world articulated objects complicates visual perception and understanding, and (2) variations in object functions and mechanisms hinder the development of a unified adaptive manipulation strategy. To address these challenges, we propose AdaRPG, a novel framework that leverages foundation models to extract object parts, which exhibit greater local geometric similarity than entire objects, thereby enhancing visual affordance generalization for functional primitive skills. To support this, we construct a part-level affordance annotation dataset to train the affordance model. Additionally, AdaRPG utilizes the common knowledge embedded in foundation models to reason about complex mechanisms and generate high-level control codes that invoke primitive skill functions based on part affordance inference. Simulation and real-world experiments demonstrate AdaRPG's strong generalization ability across novel articulated object categories.

Figures

Figures reproduced from arXiv: 2507.18276 by the authors.

Figure 1
Figure 1. Overview of AdaRPG. We construct a part-level affordance dataset to train the affordance model, which takes a part point cloud as input. During inference, AdaRPG first leverages foundation vision and language models to segment the part point cloud and predict part affordance, guiding primitive skills such as grasping and rotating. AdaRPG then utilizes GPT-4o to reason about manipulation mechanisms and generate Pytho… view at source ↗
Figure 2
Figure 2. Visualization of several representative parts and affor [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Foundation Models Guided Part Grounding and Seg [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Foundation Model Reasoning and High-Level Python [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Qualitative Simulation Result. The intermediate re￾sults of part segmentation and affordance prediction demonstrate the effectiveness of AdaRPG. data. While GraspNet excels at generic object grasping, our tasks require functional grasping, where the robot must in￾terac…
Figure 6
Figure 6. Figure 6: Qualitative Real-World Results. We visualize the segmented handle, knob, and cap parts, along with their corresponding affordance score predictions. The high-level control codes leverage affordance-guided primitive actions to successfully complete the tasks. tion. Addi…

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Try Once, Then Optimal: De-Redundified Procedure Memory for Cross-Episode Exploration Amortization

    cs.RO 2026-07 conditional novelty 6.0 of 10

    Object-centric procedure memory amortizes hidden-state exploration across encounters, cutting robot manipulation operations 16–30% at non-regressing success.

Reference graph

Works this paper leans on

55 extracted references · 25 canonical work pages · cited by 1 Pith paper

  1. [49]

    Adamanip: Adaptive articulated object manipulation envi- ronments and policy learning

    Yuanfei Wang, Xiaojie Zhang, Ruihai Wu, Yu Li, Yan Shen, Mingdong Wu, Zhaofeng He, Yizhou Wang, and Hao Dong. Adamanip: Adaptive articulated object manipulation envi- ronments and policy learning. In International Conference on Learning Representations, 2025. 2, 5, 6, 7

    work page 2025

  2. [1]

    Do as i can, not as i say: Grounding language in robotic affordances

    Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Cheb- otar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, et al. Do as i can, not as i say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.01691, 2022. 3

    arXiv 2022

  3. [2]

    Learning reward func- tions for robotic manipulation by observing humans

    Minttu Alakuijala, Gabriel Dulac-Arnold, Julien Mairal, Jean Ponce, and Cordelia Schmid. Learning reward func- tions for robotic manipulation by observing humans. In2023 IEEE International Conference on Robotics and Automation (ICRA), pages 5006–5012. IEEE, 2023. 2

    work page 2023

  4. [3]

    pi 0: A vision-language- action flow model for general robot control

    Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. pi 0: A vision-language- action flow model for general robot control. arXiv preprint arXiv:2410.24164, 2024. 2

    arXiv

  5. [4]

    Rt-1: Robotics transformer for real-world control at scale

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakr- ishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al. Rt-1: Robotics transformer for real-world control at scale. arXiv preprint arXiv:2212.06817, 2022. 2

    arXiv 2022

  6. [5]

    Chang, Thomas A

    Angel X. Chang, Thomas A. Funkhouser, Leonidas J. Guibas, Pat Hanrahan, Qi-Xing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, L. Yi, and Fisher Yu. Shapenet: An information-rich 3d model repository. ArXiv, abs/1512.03012, 2015. 2

    arXiv 2015

  7. [6]

    Cousineau, Benjamin Burchfiel, and Shuran Song

    Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric A. Cousineau, Benjamin Burchfiel, and Shuran Song. Diffu- sion policy: Visuomotor policy learning via action diffusion. ArXiv, abs/2303.04137, 2023. 2

    arXiv 2023

  8. [7]

    Can foundation models perform zero-shot task specification for robot manipulation? In Learning for dynamics and control conference , pages 893–

    Yuchen Cui, Scott Niekum, Abhinav Gupta, Vikash Kumar, and Aravind Rajeswaran. Can foundation models perform zero-shot task specification for robot manipulation? In Learning for dynamics and control conference , pages 893–

    work page

Show all 55 references
  1. [8]

    Palm-e: An embodied multimodal language model

    Danny Driess, Fei Xia, Mehdi SM Sajjadi, Corey Lynch, Aakanksha Chowdhery, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, et al. Palm-e: An embodied multimodal language model. arXiv preprint arXiv:2303.03778, 2023. 2

    2023 arXiv

  2. [9]

    Flowbot3d: Learning 3d articulation flow to manipulate articulated ob- jects

    Ben Eisner, Harry Zhang, and David Held. Flowbot3d: Learning 3d articulation flow to manipulate articulated ob- jects. arXiv preprint arXiv:2205.04382, 2022. 2

    2022 arXiv

  3. [10]

    Graspnet-1billion: A large-scale benchmark for general ob- ject grasping

    Hao-Shu Fang, Chenxi Wang, Minghao Gou, and Cewu Lu. Graspnet-1billion: A large-scale benchmark for general ob- ject grasping. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 11444– 11453, 2020. 6

    2020

  4. [11]

    Rh20t: A comprehensive robotic dataset for learning diverse skills in one-shot

    Hao-Shu Fang, Hongjie Fang, Zhenyu Tang, Jirong Liu, Chenxi Wang, Junbo Wang, Haoyi Zhu, and Cewu Lu. Rh20t: A comprehensive robotic dataset for learning diverse skills in one-shot. arXiv preprint arXiv:2307.00595, 2023. 2

    2023 arXiv

  5. [12]

    Partmanip: Learning cross-category generalizable part manipulation policy from point cloud ob- servations

    Haoran Geng, Ziming Li, Yiran Geng, Jiayi Chen, Hao Dong, and He Wang. Partmanip: Learning cross-category generalizable part manipulation policy from point cloud ob- servations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2978– ...

    2023

  6. [13]

    Sage: Bridging semantic and actionable parts for generalizable articulated-object manipu- lation under language instructions, 2023

    Haoran Geng, Songlin Wei, Congyue Deng, Bokui Shen, He Wang, and Leonidas Guibas. Sage: Bridging semantic and actionable parts for generalizable articulated-object manipu- lation under language instructions, 2023. 2, 3, 6, 7

    2023

  7. [14]

    Gapartnet: Cross-category domain-generalizable object perception and manipulation via generalizable and actionable parts

    Haoran Geng, Helin Xu, Chengyang Zhao, Chao Xu, Li Yi, Siyuan Huang, and He Wang. Gapartnet: Cross-category domain-generalizable object perception and manipulation via generalizable and actionable parts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern ...

    2023

  8. [15]

    End-to-end affordance learn- ing for robotic manipulation

    Yiran Geng, Boshi An, Haoran Geng, Yuanpei Chen, Yaodong Yang, and Hao Dong. End-to-end affordance learn- ing for robotic manipulation. ICRA, 2023. 2

    2023

  9. [16]

    Arnold: A benchmark for language-grounded task learning with con- tinuous states in realistic 3d scenes

    Ran Gong, Jiangyong Huang, Yizhou Zhao, Haoran Geng, Xiaofeng Gao, Qingyang Wu, Wensi Ai, Ziheng Zhou, Demetri Terzopoulos, Song-Chun Zhu, et al. Arnold: A benchmark for language-grounded task learning with con- tinuous states in realistic 3d scenes. In Proceedings of the IEEE...

    2023

  10. [17]

    Copa: General robotic manipulation through spa- tial constraints of parts with foundation models

    Haoxu Huang, Fanqi Lin, Yingdong Hu, Shengjie Wang, and Yang Gao. Copa: General robotic manipulation through spa- tial constraints of parts with foundation models. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 9488–9495. IEEE, 2024. 3, 6, 7

    2024

  11. [18]

    Language models as zero-shot planners: Extract- ing actionable knowledge for embodied agents

    Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch. Language models as zero-shot planners: Extract- ing actionable knowledge for embodied agents. In Interna- tional conference on machine learning , pages 9118–9147. PMLR, 2022. 3

    2022

  12. [19]

    Inner monologue: Em- bodied reasoning through planning with language models

    Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tompson, Igor Mordatch, Yevgen Chebotar, et al. Inner monologue: Em- bodied reasoning through planning with language models. arXiv preprint arXiv:2207.05608, 2022. 3

    2022 arXiv

  13. [20]

    V oxposer: Composable 3d value maps for robotic manipulation with language models

    Wenlong Huang, Chen Wang, Ruohan Zhang, Yunzhu Li, Jiajun Wu, and Li Fei-Fei. V oxposer: Composable 3d value maps for robotic manipulation with language models. arXiv preprint arXiv:2307.05973, 2023. 3

    2023 arXiv

  14. [21]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perel- man, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Weli- hinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024. 3

    2024 arXiv

  15. [22]

    Openvla: An open-source vision-language-action model

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source vision-language-action model. arXiv preprint arXiv:2406.09246, 2024. 2

    2024 arXiv

  16. [23]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In Proceedings of the IEEE/CVF international confer- ence on computer vision, pages 4015–4026, 2023. 4

    2023

  17. [24]

    Cogact: A foundational vision- language-action model for synergizing cognition and action in robotic manipulation

    Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, et al. Cogact: A foundational vision- language-action model for synergizing cognition and action in robotic manipulation. arXiv preprint arXiv:2411.19650 ,

    arXiv

  18. [25]

    Flowbothd: History-aware diffuser handling ambiguities in articulated objects manipulation

    Yishu Li, Wen Hui Leng, Yiming Fang, Ben Eisner, and David Held. Flowbothd: History-aware diffuser handling ambiguities in articulated objects manipulation. arXiv preprint arXiv:2410.07078, 2024. 2

    2024 arXiv

  19. [26]

    Unidoormanip: Learn- ing universal door manipulation policy over large-scale and diverse door manipulation environments

    Yu Li, Xiaojie Zhang, Ruihai Wu, Zilong Zhang, Yiran Geng, Hao Dong, and Zhaofeng He. Unidoormanip: Learn- ing universal door manipulation policy over large-scale and diverse door manipulation environments. arXiv preprint arXiv:2403.02604, 2024. 2

    2024 arXiv

  20. [27]

    Code as policies: Language model programs for embodied control

    Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code as policies: Language model programs for embodied control. In 2023 IEEE International Conference on Robotics and Au- tomation (ICRA), pages 9493–9500. IEEE, 2023. 3

    2023

  21. [28]

    Articulated object manipulation with coarse-to-fine affordance for miti- gating the effect of point cloud noise

    Suhan Ling, Yian Wang, Shiguang Wu, Yuzheng Zhuang, Tianyi Xu, Yu Li, Chang Liu, and Hao Dong. Articulated object manipulation with coarse-to-fine affordance for miti- gating the effect of point cloud noise. ICRA, 2024. 2

    2024

  22. [29]

    Robomamba: Efficient vision-language-action model for robotic reasoning and ma- nipulation

    Jiaming Liu, Mengzhen Liu, Zhenyu Wang, Pengju An, Xi- aoqi Li, Kaichen Zhou, Senqiao Yang, Renrui Zhang, Yan- dong Guo, and Shanghang Zhang. Robomamba: Efficient vision-language-action model for robotic reasoning and ma- nipulation. Advances in Neural Information Processing S...

    2024

  23. [30]

    Akb-48: A real-world articulated object knowledge base

    Liu Liu, Wenqiang Xu, Haoyuan Fu, Sucheng Qian, Qiao- jun Yu, Yang Han, and Cewu Lu. Akb-48: A real-world articulated object knowledge base. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14809–14818, 2022. 2

    2022

  24. [31]

    Rdt-1b: a diffusion foundation model for bimanual manipu- lation

    Songming Liu, Lingxuan Wu, Bangguo Li, Hengkai Tan, Huayu Chen, Zhengyi Wang, Ke Xu, Hang Su, and Jun Zhu. Rdt-1b: a diffusion foundation model for bimanual manipu- lation. arXiv preprint arXiv:2410.07864, 2024. 2

    2024 arXiv

  25. [32]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European Conference on Computer Vision , pages 38–55. Springe...

    2024

  26. [33]

    Vip: Towards universal visual reward and representation via value-implicit pre-training

    Yecheng Jason Ma, Shagun Sodhani, Dinesh Jayaraman, Os- bert Bastani, Vikash Kumar, and Amy Zhang. Vip: Towards universal visual reward and representation via value-implicit pre-training. arXiv preprint arXiv:2210.00030, 2022. 2

    arXiv

  27. [34]

    Eureka: Human-level reward design via coding large language models

    Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Eureka: Human-level reward design via coding large language models. arXiv preprint arXiv:2310.12931, 2023

    arXiv

  28. [35]

    Zero-shot reward specification via grounded natural lan- guage

    Parsa Mahmoudieh, Deepak Pathak, and Trevor Darrell. Zero-shot reward specification via grounded natural lan- guage. In International Conference on Machine Learning , pages 14743–14752. PMLR, 2022. 2

    2022

  29. [36]

    Isaac gym: High performance gpu-based physics simulation for robot learning

    Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, et al. Isaac gym: High performance gpu-based physics simulation for robot learning. arXiv preprint arXiv:2108.10470, 2021. 5

    2021 arXiv

  30. [37]

    Chang, Li Yi, Subarna Tripathi, Leonidas J

    Kaichun Mo, Shilin Zhu, Angel X. Chang, Li Yi, Subarna Tripathi, Leonidas J. Guibas, and Hao Su. PartNet: A large- scale benchmark for fine-grained and hierarchical part-level 3D object understanding. In The IEEE Conference on Com- puter Vision and Pattern Recognition (CVPR), ...

    2019

  31. [38]

    Guibas, Mustafa Mukadam, Abhi- nav Gupta, and Shubham Tulsiani

    Kaichun Mo, Leonidas J. Guibas, Mustafa Mukadam, Abhi- nav Gupta, and Shubham Tulsiani. Where2act: From pix- els to actions for articulated 3d objects. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 6813–6823, 2021. 2, 3

    2021

  32. [39]

    Where2explore: Few-shot affordance learning for unseen novel categories of articulated objects

    Chuanruo Ning, Ruihai Wu, Haoran Lu, Kaichun Mo, and Hao Dong. Where2explore: Few-shot affordance learning for unseen novel categories of articulated objects. In Ad- vances in Neural Information Processing Systems (NeurIPS),

  33. [40]

    Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0

    Abby O’Neill, Abdul Rehman, Abhiram Maddukuri, Ab- hishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Poo- ley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, et al. Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0. In 2024 IEEE Inte...

    2024

  34. [41]

    Pointnet++: Deep hierarchical feature learning on point sets in a metric space

    Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems, 30, 2017. 3

    2017

  35. [42]

    Sam 2: Segment anything in images and videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman R¨adle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714, 2024. 4

    2024 arXiv

  36. [43]

    Grounded sam: Assembling open-world models for diverse visual tasks

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, et al. Grounded sam: Assembling open-world models for diverse visual tasks. arXiv preprint arXiv:2401.14159 ,

    arXiv

  37. [44]

    Progprompt: Generating situated robot task plans using large language models

    Ishika Singh, Valts Blukis, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, Jesse Thomason, and Animesh Garg. Progprompt: Generating situated robot task plans using large language models. In 2023 IEEE In- ternational Conference on Robotics and Automat...

    2023

  38. [45]

    Octo: An open-source generalist robot policy

    Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, et al. Octo: An open-source generalist robot policy. arXiv preprint arXiv:2405.12213, 2024. 2

    2024 arXiv

  39. [46]

    Doorgym: A scalable door opening environment and baseline agent.arXiv preprint arXiv:1908.01887, 2019

    Yusuke Urakami, Alec Hodgkinson, Casey Carlin, Randall Leu, Luca Rigazio, and Pieter Abbeel. Doorgym: A scalable door opening environment and baseline agent.arXiv preprint arXiv:1908.01887, 2019. 2

    1908 arXiv

  40. [47]

    Bridgedata v2: A dataset for robot learning at scale

    Homer Rich Walke, Kevin Black, Tony Z Zhao, Quan Vuong, Chongyi Zheng, Philippe Hansen-Estruch, An- dre Wang He, Vivek Myers, Moo Jin Kim, Max Du, et al. Bridgedata v2: A dataset for robot learning at scale. In Con- ference on Robot Learning, pages 1723–1736. PMLR, 2023. 2

    2023

  41. [48]

    Adaafford: Learning to adapt manipulation affordance for 3d articulated objects via few-shot interactions

    Yian Wang, Ruihai Wu, Kaichun Mo, Jiaqi Ke, Qingnan Fan, Leonidas Guibas, and Hao Dong. Adaafford: Learning to adapt manipulation affordance for 3d articulated objects via few-shot interactions. European conference on computer vi- sion (ECCV 2022), 2022. 2

    2022

  42. [50]

    V AT-mart: Learning visual action trajectory proposals for manipulating 3d ARTiculated ob- jects

    Ruihai Wu, Yan Zhao, Kaichun Mo, Zizheng Guo, Yian Wang, Tianhao Wu, Qingnan Fan, Xuelin Chen, Leonidas Guibas, and Hao Dong. V AT-mart: Learning visual action trajectory proposals for manipulating 3d ARTiculated ob- jects. In International Conference on Learning Represen- tat...

    2022

  43. [51]

    Learning environment-aware affor- dance for 3d articulated object manipulation under occlu- sions

    Ruihai Wu, Kai Cheng, Yan Zhao, Chuanruo Ning, Guanqi Zhan, and Hao Dong. Learning environment-aware affor- dance for 3d articulated object manipulation under occlu- sions. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. 2

    2023

  44. [52]

    Sapien: A simulated part-based interactive environment

    Fanbo Xiang, Yuzhe Qin, Kaichun Mo, Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang, Yifu Yuan, He Wang, et al. Sapien: A simulated part-based interactive environment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 110...

    2020

  45. [53]

    3d diffusion policy

    Yanjie Ze, Gu Zhang, Kangning Zhang, Chenyuan Hu, Muhan Wang, and Huazhe Xu. 3d diffusion policy. ArXiv, abs/2403.03954, 2024. 2

    2024 arXiv

  46. [54]

    Learning reward for robot skills using large language models via self-alignment

    Yuwei Zeng, Yao Mu, and Lin Shao. Learning reward for robot skills using large language models via self-alignment. arXiv preprint arXiv:2405.07162, 2024. 2

    2024 arXiv

  47. [55]

    Flowbot++: Learning generalized articulated objects manipulation via articulation projection

    Harry Zhang, Ben Eisner, and David Held. Flowbot++: Learning generalized articulated objects manipulation via articulation projection. arXiv preprint arXiv:2306.12893 ,

    arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.