Pith. sign in

REVIEW 5 major objections 6 minor 34 references

CheckManual: A New Challenge and Benchmark for Manual-based Appliance Manipulation

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read CheckManual is the first benchmark for manual-based appliance manipulation: 1,107 human-verified manuals for 369 articulated appliances, three challenge tracks, and a baseline model ManualPlan that reaches only 20.70% planning success on…

desk verdict A genuine first for the field—manual-grounded appliance manipulation in simulation—with a substantial dataset, though the synthetic-manual premise and same-model confound need real validation. read the letter →

arxiv 2506.09343 v1 pith:ILUGBM7J submitted 2025-06-11 cs.CV cs.RO

Yuxing Long , Jiyao Zhang , Mingjie Pan , Tianshu Wu , Taewhan Kim , Hao Dong This is my paper
classification cs.CVcs.RO
keywords manual-basedmanipulationappliancebenchmarkrobottaskplanningmultimodallargelanguagemodelarticulatedobjectslong-horizonsimulationhuman-verifieddataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that using a household appliance the right way is a skill a robot must learn from the manufacturer's manual, not from common sense, and that this skill can be measured in simulation. To make the test possible, it builds CheckManual: 1,107 human-verified manuals for 369 appliances across 11 categories, with 2,211 labeled movable parts and 1,464 step-annotated manipulation tasks. Its companion planner ManualPlan, the first manual-based manipulation planning model, reaches 20.70% planning success on the easiest track and only 2.39-2.87% task success on the execution tracks, while removing the manual cuts planning success to 7.99%, which the paper takes as evidence that the manual is doing the work. If the benchmark is faithful, it gives the field a concrete way to measure a capability that current large-model planners largely lack.

What carries the argument

The load-bearing object is the synthetic manual created by a large-model-assisted, human-revised pipeline from articulated CAD appliance models. Each manual is a multi-page PDF in which every movable part receives a function name and a set of function states, tasks are expressed as resolvable step sequences, and figures are drawn in the point-line annotation, close-up, motion-decomposition, and trajectory styles observed in real manuals. The companion ManualPlan model carries the argument by splitting the problem into manual resolution, manipulation planning, and part alignment, so that each of the three challenge tracks can be scored on the sub-capability it isolates.

What would settle it

Replace the synthetic manuals in Track 1 with real manufacturer manuals for the same appliance categories, have human experts annotate the same task steps, and compare planning success; a large drop would show the synthetic manuals omit information or ambiguities that real manuals contain.

Watch

Extended reading notes

Core claim

The paper's central claim is that manual-based appliance manipulation is a distinct capability, worth benchmarking separately from common-sense tool use and from manual question answering. Each manual in CheckManual is grounded in an articulated CAD appliance, so every named part has a real movable geometry and every task step can be executed and checked in simulation. The proposed ManualPlan baseline shows where the difficulty lies: 20.70% planning success on Track 1, 2.39% task success on Track 2, and 2.87% on Track 3, with the no-manual ablation dropping Track 1 planning to 7.99%. The paper interprets these numbers as showing that reading the manual is the key source of correctness and that all three sub-problems, manual comprehension, part alignment, and multi-step execution, must improve substantially before manual-based appliance manipulation is solved.

Load-bearing premise

The benchmark's usefulness rests on the assumption that the LLM-generated, human-revised manuals carry the same information and the same ambiguities as real manufacturer manuals, so success on CheckManual predicts success with real manuals.

Editorial extensions

If this is right

  • A model that succeeds on CheckManual must solve manual reading, part grounding, planning, and physical execution together, so the benchmark yields a single comparable score for the full capability.
  • The gap between the 20.70% planning success on Track 1 and the 2.39% task success on Track 2 quantifies how much is lost to part-alignment and low-level execution errors.
  • Removing the manual drops Track 1 planning success from 20.70% to 7.99%, isolating the manual as the main source of the benchmark's achievable correctness.
  • The 1,464 tasks include sequences of up to 18 steps, and success is checked after every step, which makes long-horizon error accumulation a measurable bottleneck.
  • The simulator environments and metrics let future manual-following manipulation models be compared against these baselines on the same three tracks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the synthetic manuals faithfully reflect real manufacturer manuals, then benchmark scores may predict real-world manual-following ability; the paper does not test that transfer, so it remains open.
  • A Track 1 variant with perfect perception, scoring only manual comprehension and planning, would isolate reading ability from execution and make error attribution cleaner.
  • Adding deliberately ambiguous, erroneous, or incomplete manuals could turn CheckManual into a robustness test of whether a model knows when the manual is not enough.
  • Since human verification is the quality gate for the ground truth, reporting inter-annotator agreement or a residual-error audit would make the benchmark's reliability itself measurable.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper introduces CheckManual, which it claims is the first benchmark for manual-based appliance manipulation. The authors design an LLM-assisted, human-revised pipeline that starts from PartNet-Mobility CAD models, generates part function annotations, manipulation tasks, and manual figures, and then compiles LaTeX-based PDF manuals. The benchmark provides 1107 manuals for 369 appliances across 11 categories, with 2211 manipulable parts and 1464 manipulation tasks, plus three challenge tracks (manual-CAD-appliance aligned planning, manual-and-CAD-based manipulation, and pure manual-based manipulation) and metrics implemented in the SAPIEN simulator. The authors also propose ManualPlan, a GPT-4o-based high-level planner, and combine it with CAD-assisted primitive actions or VoxPoser for low-level execution. Simulation results show low overall success rates (e.g., 20.70% planning success on Track 1 and 2.39% task success on Track 2), and a short real-robot section reports no quantitative results.

Significance. If the synthetic manuals are a valid proxy for real manufacturer manuals, CheckManual would be a useful and reusable asset: it is the first benchmark of its kind, it is built on articulated CAD models, it provides a large number of human-verified annotations, and it defines concrete evaluation tracks with a simulator environment. The authors also provide an initial baseline family that quantifies the difficulty of the proposed tasks, and the w/o-manual ablations support the intuition that the manual contributes to planning and execution. However, the significance of the benchmark hinges on construct validity: the manuals used for evaluation are generated by the same model family that is evaluated on them, and the paper provides no quantitative evidence that the generated manuals capture the ambiguity, implicit knowledge, and formatting variability of real manufacturer manuals. These issues must be addressed before the headline numbers can be interpreted as evidence of real-world manual-based manipulation ability.

major comments (5)
  1. [§3.1.1–§3.1.5, Fig. 2] The central claim of the benchmark is that success on CheckManual indicates an ability to use real appliance manuals, but the representativeness of the generated manuals is asserted rather than demonstrated. The analysis of 110 real manuals in §3.1.1 is summarized only in qualitative terms (point-line annotations, four guidance formats, task layout styles), and no quantitative distributional comparison is reported between the real manuals and the 1107 generated manuals. Since the entire benchmark, and the real-world transfer claim in §5.3, depends on the synthetic manuals being a valid proxy, the authors should provide a quantitative comparison of structural and linguistic properties (e.g., annotation style frequencies, task-step distributions, figure types, ambiguity patterns) between real and generated manuals, or otherwise justify the proxy assumption.
  2. [§3.1.2–§3.1.4] The paper reports that humans verified more than 2200 part annotations, 1400+ tasks, and 1800+ figures, and the introduction states that this verification 'guarantee[s] correctness.' However, no inter-annotator agreement, double-verification protocol, or residual error rate is reported. Without such data, human verification is a described process, not a measured guarantee. The authors should report the verification protocol, the number of annotators, agreement statistics, and the rate of corrected errors; this is especially important for tasks and part states, which directly define the ground truth for the evaluation metrics in §3.3.
  3. [§3.1.2, §3.1.3, §3.1.5 vs. §4.1, §5.1] GPT-4o is used on both sides of the evaluation: it generates part function names, tasks, and LaTeX manuals in the dataset creation pipeline, and it implements ManualPlan's manual resolution, manipulation planning, and part alignment modules. This is a confound: planning success may reflect GPT-4o parsing its own stylistic and content conventions rather than robust, general manual comprehension. The authors should break this circularity, for example by evaluating ManualPlan with a different planner model on the same manuals, or by evaluating GPT-4o-based planners on a sample of real manufacturer manuals, and comparing the result distributions.
  4. [Table 2, §5.2] All results in Table 2 are point estimates with no variance, number of trials, or number of seeds reported. Given that the key numbers are low (20.70% planning success on Track 1, 2.39% task success on Track 2), and that the w/o-manual ablations show small absolute differences in several categories, the difficulty claim and the importance of the manual need to be supported by repeated runs with standard deviations or confidence intervals. The authors should also specify how many tasks per appliance category and how many independent evaluation runs were used to compute each entry.
  5. [§3.3, Evaluation Metrics] The step-level success criteria introduce several thresholds — L ± 30° for revolute parts, >25% L movement for buttons and sliders, and the motion-distance ratio >50% — but no sensitivity analysis or justification is provided for these choices. Since these thresholds directly determine the reported success rates, the authors should report how the results in Table 2 change under reasonable variations of these parameters, or at least justify the thresholds against the part sizes and joint limits in the dataset.
minor comments (6)
  1. [Table 1] Table 1 contains a duplicated row for 'Track 2: Manual & CAD based Manipulation' and no row for Track 3, which makes the table confusing; the duplication should be removed and the Track 3 row added.
  2. [§3.3, Table 1] The text uses 'Planing' several times (e.g., 'Track 1: CAD-Appliance Aligned Planing'); this should be corrected to 'Planning'.
  3. [§2.1] In the related-work section, 'score a group of kills' appears to be a typo for 'skills' and should be corrected.
  4. [§3.1.1] The footnote uses 'ChcekManual' instead of 'CheckManual'; the typo should be fixed.
  5. [§5.1] The name 'groundingDINO' is written without a space in the implementation details; it should be 'Grounding-DINO' for consistency with the reference list.
  6. [§5.3] The real-robot deployment section reports no quantitative outcomes such as number of tasks attempted, success rates, or failure examples, and the statement that the authors 'follow [11] to cache correct manipulation actions in every step' indicates that the experiment does not evaluate end-to-end manual-based manipulation; either provide full quantitative results without caching or explicitly reword the claim to state that only a qualitative demonstration was performed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the benchmark construction and baseline evaluation are not equivalent by construction; remaining concerns are validity risks, not circular reductions.

full rationale

The paper's contributions are dataset construction, benchmark task design, and a baseline model. The derivation chain—manuals generated from CAD models via an LLM-assisted, human-verified pipeline (§3.1), tasks defined from the resulting annotations (§3.1.3), and ManualPlan evaluated on those tasks (§4–5)—does not reduce to its own inputs by construction. No parameter is fitted to the evaluation labels; ManualPlan is a zero-shot planner, and its reported results are low (20.70% planning success on Track 1, 2.39% task success on Track 2), so there is no fitted prediction being relabeled as a result. GPT-4o is used both to generate the synthetic manuals and to assist the baseline planner (§5.1), which could create style self-consistency, but the paper asserts no equation or mechanism by which evaluation outcomes are forced by the generation process, and the human-verification stages break any direct equivalence. The concern that synthetic manuals may not faithfully represent real manufacturer manuals is a construct-validity and generalization risk, not a circularity pattern under the rubric. Self-citations appear in related-work surveys and component reuse (e.g., VoxPoser, SAPIEN), but none is load-bearing for the benchmark's central validity claim. Therefore no circular step is exhibited, and the appropriate score is 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claims rest on the correctness of the PartNet-Mobility CAD kinematics, the reliability of human-verified synthetic data, the sufficiency of the manuals relative to the tasks, and the fidelity of the SAPIEN simulator. No new physical entities are introduced. The main hand-set parameters are the evaluation success thresholds and the sampling interval used when generating part states.

free parameters (5)
  • Revolute joint success tolerance = ±30°
    Evaluation metric threshold in §3.3: a revolute part (knob, lever, door) manipulation is judged successful when within L ± 30°. This hand-chosen bound directly changes reported success rates for Track 2/3.
  • Prismatic joint success threshold = >25% L
    Section 3.3: buttons, containers, and sliders count as moved when movement exceeds 25% of joint range. This arbitrary threshold affects the reported task success rates.
  • Motion ratio threshold = MD_part / MD_appliance > 50%
    Section 3.3: for parts that move together with the appliance, success requires the part motion to exceed half the appliance motion. This hand-set criterion affects Track 2/3 results.
  • Knob rotation sampling interval = 60°
    Section 3.1.2: part function states are sampled at 60-degree intervals for knobs and levers, constraining the diversity of manual content and the downstream task space.
  • Button push times sampling range = unspecified pre-defined range
    Section 3.1.2: push times are randomly sampled from a pre-defined range that is not specified, limiting the reproducibility of the part-state generation.
assumptions (5)
  • domain assumption PartNet-Mobility CAD models carry correct kinematic annotations (movable part IDs, joint types, axes) that faithfully represent real appliance parts.
    The entire benchmark is built from these CAD models (§3.1.1). If the CAD kinematics are wrong or unrepresentative, the manuals and simulator evaluation inherit the errors.
  • domain assumption Human verification ensures the generated manuals, part annotations, and tasks are correct enough to serve as evaluation ground truth.
    Claimed in §3.1.2-3.1.4 with counts (2200+ annotations, 1400+ tasks, 1800+ figures). No inter-annotator agreement or residual error rate is reported, so correctness is assumed rather than demonstrated.
  • domain assumption A model reading only the manual (plus RGB-D observation) can in principle determine the correct manipulation plan for each task; the manual contains all necessary information and is internally consistent with the task ground truth.
    This is the premise of the three challenge tracks, especially Track 3 (§3.3). If manuals omit needed steps or contain ambiguities, low model success may reflect underspecified manuals rather than model failure.
  • domain assumption GPT-4o output is a reliable proxy for general multimodal competence in manual comprehension and planning.
    Used as the backbone for manual resolution, planning, and alignment in ManualPlan (§4.1, §5.1). The benchmark results depend on this model's behavior; other models may score differently.
  • domain assumption SAPIEN simulator dynamics and the CAD primitive action policies accurately model real-world appliance manipulation.
    Track 2/3 execution results are obtained in simulation (§5.2); the paper's real-robot section is qualitative, so simulator-to-real transfer is assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CheckManual: A New Challenge and Benchmark for Manual-based Appliance Manipulation." pith.science (2026). https://pith.science/paper/ILUGBM7J

@misc{pith2026250609343,
  author       = {Pith},
  title        = {Pith review of: CheckManual: A New Challenge and Benchmark for Manual-based Appliance Manipulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ILUGBM7J}},
  note         = {Machine review of arXiv:2506.09343}
}
read the original abstract

Correct use of electrical appliances has significantly improved human life quality. Unlike simple tools that can be manipulated with common sense, different parts of electrical appliances have specific functions defined by manufacturers. If we want the robot to heat bread by microwave, we should enable them to review the microwave manual first. From the manual, it can learn about component functions, interaction methods, and representative task steps about appliances. However, previous manual-related works remain limited to question-answering tasks while existing manipulation researchers ignore the manual's important role and fail to comprehend multi-page manuals. In this paper, we propose the first manual-based appliance manipulation benchmark CheckManual. Specifically, we design a large model-assisted human-revised data generation pipeline to create manuals based on CAD appliance models. With these manuals, we establish novel manual-based manipulation challenges, metrics, and simulator environments for model performance evaluation. Furthermore, we propose the first manual-based manipulation planning model ManualPlan to set up a group of baselines for the CheckManual benchmark.

Figures

Figures reproduced from arXiv: 2506.09343 by the authors.

Figure 1
Figure 1. Comparison between appliance manipulation with and without manual. If the manual is absent, the concrete func￾tions of the appliance part are unknown. As shown in the top image, the robot feels confused about tasks such as opening the microwave door. To correctly use the appliance, the robot must comprehend the manual content and follow its guidance to work. common sense. One appliance may have multiple mov￾able com… view at source ↗
Figure 2
Figure 2. Generation workflow of CheckManual dataset. In the leftmost part (Section 3.1.1), we analyze real manuals to learn about their formats and collect different categories of appliance CAD models. The middle part (Section 3.1.2 - 3.1.4) demonstrates the creation of manual materials, including appliance creation, task generation, and figure design. Human verifies every step to guarantee correctness. Based on this informa… view at source ↗
Figure 3
Figure 3. One example of appliance manual from our CheckManual dataset. Please zoom in for details. parts’ visual features, we drive the annotation process by the multimodal large language model (MLLM). Before an￾notation, manipulable part IDs and types are automatically labeled on the appliance overview figures through point-line annotation. Taking these figures as visual prompts, we in￾struct the MLLM to design the function… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: CheckManual dataset statistics. The above tables and figures from left to right are (a) Numerical statistics. (b) Proportion of different appliances. (c) Distribution of manipulable components on the appliances. (d) Distribution of manipulation task step lengths. we fu…
Figure 5
Figure 5. Figure 5: The framework of our ManualPlan model. It is composed of Manual Resolution, Manipulation Planing and Part Alignment modules. The ManualPlan can make high level planning to control the CAD-assisted primitive actions or open-vocabulary manipulation large model (e.g., Vox…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

34 extracted references · 19 canonical work pages

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,

    arXiv

  2. [2]

    Do as i can, not as i say: Grounding language in robotic affordances

    Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Cheb- otar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, et al. Do as i can, not as i say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.01691, 2022. 2

    arXiv 2022

  3. [3]

    Rt-2: Vision-language-action mod- els transfer web knowledge to robotic control, 2023

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, Pete Florence, Chuyuan Fu, Montse Gonzalez Arenas, Keerthana Gopalakr- ishnan, Kehang Han, Karol Hausman, Alexander Herzog, Jasmine Hsu, Brian Ichter, Alex Irpan, Nikhil Joshi, Ryan Julian, Dmitry Kalashnik...

    work page 2023

  4. [4]

    Rt-1: Robotics transformer for real- world control at scale, 2023

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakr- ishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, Ju- lian Ibarz, Brian Ichter, Alex Irpan, Tomas Jackson, Sally Jesmonth, Nikhil J Joshi, Ryan Julian, Dmitry Kalash- nikov, Yuheng Kuang, Isabel Leal, Kuang-Huei Lee, Sergey Levine, Yao Lu, Utsav M...

    work page 2023

  5. [5]

    Bridging zero- shot object navigation and foundation models through pixel- guided navigation skill

    Wenzhe Cai, Siyuan Huang, Guangran Cheng, Yuxing Long, Peng Gao, Changyin Sun, and Hao Dong. Bridging zero- shot object navigation and foundation models through pixel- guided navigation skill. In 2024 IEEE International Confer- ence on Robotics and Automation (ICRA), pages 5228–5234. IEEE, 2024. 2

    work page 2024

  6. [6]

    A density-based algorithm for discovering clusters in large spatial databases with noise

    Martin Ester, Hans-Peter Kriegel, J ¨org Sander, Xiaowei Xu, et al. A density-based algorithm for discovering clusters in large spatial databases with noise. In kdd, pages 226–231,

  7. [7]

    Anygrasp: Robust and efficient grasp perception in spa- tial and temporal domains

    Hao-Shu Fang, Chenxi Wang, Hongjie Fang, Minghao Gou, Jirong Liu, Hengxu Yan, Wenhai Liu, Yichen Xie, and Cewu Lu. Anygrasp: Robust and efficient grasp perception in spa- tial and temporal domains. IEEE Transactions on Robotics (T-RO), 2023. 1, 7

    work page 2023

  8. [8]

    Rt-trajectory: Robotic task general- ization via hindsight trajectory sketches, 2023

    Jiayuan Gu, Sean Kirmani, Paul Wohlhart, Yao Lu, Montser- rat Gonzalez Arenas, Kanishka Rao, Wenhao Yu, Chuyuan Fu, Keerthana Gopalakrishnan, Zhuo Xu, Priya Sundare- san, Peng Xu, Hao Su, Karol Hausman, Chelsea Finn, Quan Vuong, and Ted Xiao. Rt-trajectory: Robotic task general- ization via hindsight trajectory sketches, 2023. 2

    work page 2023

Show all 34 references
  1. [9]

    Copa: General robotic manipulation through spatial constraints of parts with foundation models

    Haoxu Huang, Fanqi Lin, Yingdong Hu, Shengjie Wang, and Yang Gao. Copa: General robotic manipulation through spatial constraints of parts with foundation models. arXiv preprint arXiv:2403.08248, 2024. 2

    2024 arXiv

  2. [10]

    V oxposer: Composable 3d value maps for robotic manipulation with language models

    Wenlong Huang, Chen Wang, Ruohan Zhang, Yunzhu Li, Jiajun Wu, and Li Fei-Fei. V oxposer: Composable 3d value maps for robotic manipulation with language models. arXiv preprint arXiv:2307.05973, 2023. 1, 2, 7, 8

    2023 arXiv

  3. [11]

    Rekep: Spatio-temporal reasoning of rela- tional keypoint constraints for robotic manipulation

    Wenlong Huang, Chen Wang, Yunzhu Li, Ruohan Zhang, and Li Fei-Fei. Rekep: Spatio-temporal reasoning of rela- tional keypoint constraints for robotic manipulation. arXiv preprint arXiv:2409.01652, 2024. 1, 2, 8

    2024 arXiv

  4. [12]

    Openvla: An open-source vision-language-action model

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Fos- ter, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kol- lar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. Ope...

    2024 arXiv

  5. [13]

    Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C. Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross Girshick. Segment anything. arXiv:2304.02643, 2023. 7, 8

    2023 arXiv

  6. [14]

    Sara-rt: Scaling up robotics transformers with self-adaptive robust attention

    Isabel Leal, Krzysztof Choromanski, Deepali Jain, Avinava Dubey, Jake Varley, Michael Ryoo, Yao Lu, Frederick Liu, Vikas Sindhwani, Quan Vuong, et al. Sara-rt: Scaling up robotics transformers with self-adaptive robust attention. In 2024 IEEE International Conference on Roboti...

    2024

  7. [15]

    Vision-language foundation models as effective robot imitators

    Xinghang Li, Minghuan Liu, Hanbo Zhang, Cunjun Yu, Jie Xu, Hongtao Wu, Chilam Cheang, Ya Jing, Weinan Zhang, Huaping Liu, et al. Vision-language foundation models as effective robot imitators. arXiv preprint arXiv:2311.01378,

    arXiv

  8. [16]

    Manipllm: Embodied multimodal large language model for object-centric robotic manipulation

    Xiaoqi Li, Mingxu Zhang, Yiran Geng, Haoran Geng, Yux- ing Long, Yan Shen, Renrui Zhang, Jiaming Liu, and Hao Dong. Manipllm: Embodied multimodal large language model for object-centric robotic manipulation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pa...

    2024

  9. [17]

    Sdxl- lightning: Progressive adversarial diffusion distillation,

    Shanchuan Lin, Anran Wang, and Xiao Yang. Sdxl- lightning: Progressive adversarial diffusion distillation,

  10. [18]

    Grounding dino: Marry- ing dino with grounded pre-training for open-set object de- tection, 2024

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, and Lei Zhang. Grounding dino: Marry- ing dino with grounded pre-training for open-set object de- tection, 2024. 7, 8

    2024

  11. [19]

    Instructnav: Zero-shot system for generic instruction navigation in unexplored environment

    Yuxing Long, Wenzhe Cai, Hongcheng Wang, Guanqi Zhan, and Hao Dong. Instructnav: Zero-shot system for generic instruction navigation in unexplored environment. arXiv preprint arXiv:2406.04882, 2024. 1

    2024 arXiv

  12. [20]

    Dis- cuss before moving: Visual language navigation via multi- expert discussions

    Yuxing Long, Xiaoqi Li, Wenzhe Cai, and Hao Dong. Dis- cuss before moving: Visual language navigation via multi- expert discussions. In 2024 IEEE International Conference on Robotics and Automation (ICRA) , pages 17380–17387. IEEE, 2024. 2

    2024

  13. [21]

    Hil-serl: Precise and dexterous robotic manipulation via human-in-the-loop reinforcement learning, 2024

    Jianlan Luo, Charles Xu, Jeffrey Wu, and Sergey Levine. Hil-serl: Precise and dexterous robotic manipulation via human-in-the-loop reinforcement learning, 2024. 1

    2024

  14. [22]

    Chang, Li Yi, Subarna Tripathi, Leonidas J

    Kaichun Mo, Shilin Zhu, Angel X. Chang, Li Yi, Subarna Tripathi, Leonidas J. Guibas, and Hao Su. PartNet: A large- scale benchmark for fine-grained and hierarchical part-level 3D object understanding. In The IEEE Conference on Com- puter Vision and Pattern Recognition (CVPR), 2019. 3

    2019

  15. [24]

    Question answering over electronic devices: A new benchmark dataset and a multi-task learning based QA framework

    Abhilash Nandy, Soumya Sharma, Shubham Maddhashiya, Kapil Sachdeva, Pawan Goyal, and NIloy Ganguly. Question answering over electronic devices: A new benchmark dataset and a multi-task learning based QA framework. In Findings of the Association for Computational Linguistics: E...

    2021

  16. [25]

    Omnimanip: Towards general robotic manipulation via object-centric interaction primitives as spatial constraints

    Mingjie Pan, Jiyao Zhang, Tianshu Wu, Yinghao Zhao, Wen- long Gao, and Hao Dong. Omnimanip: Towards general robotic manipulation via object-centric interaction primitives as spatial constraints. arXiv preprint arXiv:2501.03841 ,

    arXiv

  17. [26]

    Association for Computational Linguistics. 2

  18. [27]

    V AT-mart: Learning visual action trajectory proposals for manipulating 3d ARTiculated ob- jects

    Ruihai Wu, Yan Zhao, Kaichun Mo, Zizheng Guo, Yian Wang, Tianhao Wu, Qingnan Fan, Xuelin Chen, Leonidas Guibas, and Hao Dong. V AT-mart: Learning visual action trajectory proposals for manipulating 3d ARTiculated ob- jects. In International Conference on Learning Represen- tat...

    2022

  19. [28]

    FoundationPose: Unified 6d pose estimation and tracking of novel objects

    Bowen Wen, Wei Yang, Jan Kautz, and Stan Birchfield. FoundationPose: Unified 6d pose estimation and tracking of novel objects. In CVPR, 2024. 7, 8

    2024

  20. [29]

    Set-of-mark prompting unleashes extraordinary visual grounding in gpt-4v

    Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao. Set-of-mark prompting unleashes extraordinary visual grounding in gpt-4v. arXiv preprint arXiv:2310.11441, 2023. 7

    2023 arXiv

  21. [30]

    Chang, Leonidas J

    Fanbo Xiang, Yuzhe Qin, Kaichun Mo, Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang, Yifu Yuan, He Wang, Li Yi, Angel X. Chang, Leonidas J. Guibas, and Hao Su. SAPIEN: A simulated part-based interactive envi- ronment. In The IEEE Conference on Computer Vision and...

    2020

  22. [31]

    Generative category-level object pose estimation via diffusion models

    Jiyao Zhang, Mingdong Wu, and Hao Dong. Generative category-level object pose estimation via diffusion models. Advances in Neural Information Processing Systems , 36: 54627–54644, 2023. 7

    2023

  23. [32]

    Lidar-llm: Exploring the potential of large language models for 3d lidar understanding

    Senqiao Yang, Jiaming Liu, Ray Zhang, Mingjie Pan, Zoey Guo, Xiaoqi Li, Zehui Chen, Peng Gao, Yandong Guo, and Shanghang Zhang. Lidar-llm: Exploring the potential of large language models for 3d lidar understanding. arXiv preprint arXiv:2312.14074, 2023. 2

    2023 arXiv

  24. [33]

    Mpmqa: multimodal question answering on product man- uals

    Liang Zhang, Anwen Hu, Jing Zhang, Shuo Hu, and Qin Jin. Mpmqa: multimodal question answering on product man- uals. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 13958–13966, 2023. 1, 2

    2023

  25. [34]

    Omni6dpose: A benchmark and model for universal 6d object pose esti- mation and tracking

    Jiyao Zhang, Weiyao Huang, Bo Peng, Mingdong Wu, Fei Hu, Zijian Chen, Bo Zhao, and Hao Dong. Omni6dpose: A benchmark and model for universal 6d object pose esti- mation and tracking. In European Conference on Computer Vision, pages 199–216. Springer, 2024. 7

    2024

  26. [2021]

    Association for Computational Linguistics. 1, 2

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.