Pith. sign in

REVIEW 5 major objections 6 minor 37 references

The paper reports that WCM, a robot built on the SLAK architecture with asynchronous execution and chain-of-thought distillation from rollouts and teaching, achieves a 73.8% average success rate across nine real-world human-robot interactio

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 03:52 UTC pith:4RFUQBBI

load-bearing objection A solid architectural idea with a thin evaluation; the 73.8% headline is not yet credible, but the teaching-to-distillation loop is worth a serious look. the 5 major comments →

arxiv 2607.22999 v1 pith:4RFUQBBI submitted 2026-07-25 cs.RO cs.AIcs.HCcs.LG

WCM: World-Cognition Model for Generalizable Human-Robot Interaction

Yuzhen Chen , KC Zhou This is my paper
classification cs.RO cs.AIcs.HCcs.LG
keywords human-robot interactionmobile manipulationchain-of-thought distillationSLAK architectureasynchronous runtimeteaching modeworld modelgeneralization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that the bottleneck in human-robot interaction is not merely executing instructions but keeping a user in the loop: the robot should explain why it acts, accept mid-execution corrections, and learn from being taught. WCM addresses this by splitting the system into Sensing, Logic, Action, and Knowledge layers and running them asynchronously, so reasoning and dialogue do not block motion. It records every decision—scene, instruction, memory, reasoning, action, expected outcome, and realized outcome—from both autonomous runs and human teaching episodes, refines those records into cleaner chain-of-thought traces, and distills them back into the reasoning model. The reported result is a 73.8% average success rate across nine real-world tasks, including held-out tasks and a long-horizon task that started below 20% success and rose to 82% after teaching plus distillation. If true, this would be evidence that interaction itself—explanations, corrections, teaching—can be converted into training signal for a generalizable robot.

Core claim

On the paper's own terms, the central discovery is that a robot does not need a monolithic end-to-end policy to interact well: by making perception, reasoning, action, and memory separate but tightly coupled, and by using the resulting explicit decision records as reasoning supervision, a low-cost mobile manipulator can handle a suite of dialogue-driven tasks, transfer to unseen instructions, and improve from interactive teaching. The authors identify the mechanism as a weighted distillation of refined chain-of-thought traces from both autonomous rollouts and human-guided teaching episodes, with teaching examples weighted more heavily in the training objective. The long-horizon screwdriver-t

What carries the argument

The core object is the SLAK architecture, which separates Sensing, Logic, Action, and Knowledge into explicit layers connected by a shared state interface. Each decision is represented as a tuple: scene state, user instruction, retrieved memory, reasoning trace, selected action, expected outcome, and realized outcome. The asynchronous runtime decouples reasoning from execution so the robot can act, explain, and listen concurrently, while a validation check discards stale actions when the context changes. The training loop refines raw recorded traces—trimming detours from successes and correcting failures—and optimizes a chain-of-thought negative log-likelihood loss with teaching examples wei

Load-bearing premise

The load-bearing premise is that the nine hand-selected tasks and the authors' undefined success criteria give a fair, consistent measure of human-robot interaction capability; if scoring was lenient or the tasks were chosen to fit the system, the 73.8% figure would not establish that the architecture is responsible for the performance.

What would settle it

Have an independent evaluator re-run the nine tasks with pre-written, video-scored success criteria and a third-party rater; if the measured average success rate falls substantially below 73.8%, or if the reported ablation drop on part-level tasks does not reproduce, the central empirical claim is undermined.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A robot on hardware costing under $2,000 can sustain dialogue, explanation, mid-execution correction, and teaching, suggesting that interaction capability need not be reserved for high-end platforms.
  • Held-out transfer across four tasks not seen in CoT fine-tuning suggests that reasoning-supervision from decision records generalizes to new spoken instructions better than motor-level imitation.
  • The teaching mode's improvement—from below 20% to 82% after teaching plus distillation—indicates that natural-language guidance can substitute for teleoperation in acquiring long-horizon tasks.
  • Removing the asynchronous runtime inflates end-to-end run time by 1.4–1.7×, and removing part-level sensing cues drops the screwdriver-to-drawer success rate from 69% to 0%, identifying both as load-bearing components.
  • The system records failures as reusable experience and converts them into corrected chain-of-thought supervision, so unsuccessful runs contribute to future competence rather than being discarded.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the approach scales, the explicit separation of reasoning from execution could let a single reasoning model serve many robot bodies, since actions are bound to a reusable skill library rather than a hardware-specific policy.
  • The teaching-to-distillation loop suggests a path where end-users generate training data simply by teaching, potentially reducing the need for expert teleoperation datasets and enabling continual on-site improvement.
  • A testable extension is to check whether the recorded expected outcomes match what the scene state actually supports: if the reasoning model hallucinates outcomes, distillation would amplify the errors rather than correct them.
  • The case-study comparison, if extended to a controlled benchmark with identical success criteria and independent scoring, would clarify how much of the reported performance comes from the architecture versus the choice of tasks.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper presents WCM, a human-centered embodied agent built on the SLAK architecture (Sensing, Logic, Action, Knowledge) with an asynchronous runtime. WCM records decision traces from autonomous rollouts and human teaching episodes, refines them offline into chain-of-thought supervision, and distills this into its reasoning model. The authors report a 73.8% average success rate across nine real-world human-robot interaction tasks, including tasks held out from CoT fine-tuning and a long-horizon task acquired through teaching. They also provide a case-study comparison against a commercial system and ablations of the asynchronous runtime and the sensing layer.

Significance. The paper's main strengths are the clear articulation of the SLAK architecture, the idea of decoupling sensing, reasoning, action, and memory while providing an asynchronous runtime, and the human-in-the-loop teaching-to-distillation loop. The empirical effort on nine physical tasks, with per-task trial counts and video demonstrations, is valuable as an exploratory case study. If the reported 73.8% success rate were backed by a pre-registered success rubric, larger trial counts with confidence intervals, and a fair baseline, the result would be a meaningful step toward interactive, explainable robot control. However, as presented, the evaluation does not yet support the paper's strong claims about interaction quality, correction handling, or learning from teaching.

major comments (5)
  1. [Table I / §III-B] The headline success rate is not interpretable because 'success' is never operationally defined. For each of the nine tasks, Table I reports only a binary success ratio, with no rubric stating what constitutes a successful handover, placement, disposal, or 'hold the BBQ tongs.' For example, does handover success require the object to be placed in the user's hand, or merely released within reach? Does 'put the bottle in the trash can' require the bottle to be fully inside, or merely dropped toward the opening? Without a published, itemized success criterion, the reported 73.8% cannot be independently verified or compared across tasks. The Conclusion's acknowledgment that this is 'a case-study evaluation' (Sec. IV) does not resolve this, because the Abstract and Introduction present the number as the main evidence for the system's capabilities.
  2. [Table I / §III-B] The statistical basis for the headline is very thin. Each task is evaluated on only 9–15 attempts, and no confidence intervals, significance tests, or inter-rater reliability are reported. Concretely, 'Hand me the jacket' (6/9 success) and 'Put the bottle in the trash can' (9/14 success) are both reported as 67% and 64% respectively, but with these sample sizes the differences between tasks are well within binomial noise. The claim that WCM 'achieves' 73.8% therefore rests on a small, author-selected sample with no uncertainty quantification. I would expect at minimum binomial confidence intervals, a fixed number of trials per task decided in advance, and an explicit statement of how trials were generated (e.g., object placements, user utterances, environment variations).
  3. [§II-B1, Eq. (3)] The CoT distillation loop is partially self-referential and the refinement procedure R is not specified. The paper states that 'a stronger model ensemble reviews the recorded reasoning and outcomes... and produces cleaner reasoning traces through voting or consensus' but does not identify this ensemble, its size, its prompting, or how failures are converted into 'corrected CoT traces.' Because the same model family may be involved in both inference and refinement, the reported improvements from distillation could reflect the refinement model's bias rather than genuine task competence. The paper should specify the refinement model, provide examples of raw versus refined traces, and ablate the refinement step (e.g., training directly on raw rollouts versus refined rollouts). Without this, the claim that WCM 'turns interaction into training signal' is not falsifiable.
  4. [Appendix B / Table II] The case-study comparison with X-Square's WALL-OSS does not support the efficiency claims in the paper. Success rates are reported for WCM only; the baseline's per-task success rates are omitted. Moreover, the baseline's GPU busy time is estimated as the full movement time under the assumption that a VLA 'runs on the GPU at every control step,' while WCM's GPU time is measured on the actual system. This asymmetric methodology makes the comparison illustrative at best. If the paper wants to claim that WCM frees the GPU or is faster, it must measure the baseline on the same hardware/software stack or, at minimum, obtain the baseline's own measured GPU occupancy and success rates. As written, Table II's averaged '88.0%' success for WCM against an absent baseline is misleading.
  5. [Abstract / §III] There is a construct-validity gap between the paper's claims and what is measured. The Abstract says WCM 'explains, accepts mid-execution correction, and improves from interaction,' but Table I only reports binary task completion. No experiment reports explanation accuracy, the rate at which user corrections were correctly interpreted and executed, or the amount of improvement attributable to teaching versus to additional attempts. The ablation in Table III measures only success and run time, not whether the robot's reasoning traces are faithful. To support the paper's thesis, the evaluation needs interaction-specific metrics (e.g., correction success rate, explanation correctness judged against a rubric, learning gain from teaching episodes with a control condition that repeats untaught rollouts). Without such measures, the reported success rate is compatible with a conventional instruc
minor comments (6)
  1. [§II-A2, Eq. (1)] The validation threshold ε in Eq. (1) is never given a value or a sensitivity analysis. Since the runtime's behavior depends on this threshold, please state how it was chosen and whether results are robust to its value.
  2. [§II-B, Eq. (3)] The teaching loss weight λ is described as ≥1 but no value is reported. A brief hyperparameter table or a sentence on how λ was selected would improve reproducibility.
  3. [Table I footnotes] The footnotes state that four tasks are 'held out from CoT training set,' but the paper does not list which tasks were in the training set versus held out. Please make the training/held-out split explicit for each task.
  4. [Appendix B / Table II] The statement 'X-Square did not provide per-task success rates' is unusual for a comparison; please clarify whether these numbers come from public videos or from a direct request, and whether the movement times are measured from the same video start conditions.
  5. [General] Figure 2 and the demo figures are referenced, but the paper does not include the actual video frames or links to videos. Since the paper's contribution includes 'live demos,' please include supplementary material or URLs.
  6. [§I] The phrase 'physical AI, robot imagination, and manipulation [16, 32, 36, 7, 1, 21, 34]' is broad; a sentence distinguishing world models for prediction from the proposed interaction-oriented world-cognition model would help position the contribution.

Circularity Check

0 steps flagged

No significant circularity: the headline 73.8% success rate is an externally measured task-completion result, not a consequence of the training equations.

full rationale

WCM's central claim is an empirical 73.8% average success rate on nine physical HRI tasks, with four tasks explicitly marked as held out from CoT fine-tuning and one task evaluated before and after teaching/distillation. The success rates in Table I are external task-completion measurements, not consequences of the training objective in Eq. (3). The CoT flywheel (Eqs. 2–3) is self-referential as a training loop—WCM distills its own rollouts refined by a stronger model ensemble—but the paper does not use that loop to derive the evaluation numbers; the held-out tasks and the pre/post-teaching contrast provide an external check. Self-citations ([9], [14], [33]–[35]) are background citations for sensing and 3D/world-model baselines and are not load-bearing for the headline. The case-study comparison is explicitly disclaimed as illustrative rather than controlled, so it does not function as a forced derivation. The main weaknesses—undefined success criteria and absence of HRI-specific metrics for explanation or correction quality—are construct-validity concerns, not circular reductions. No equation or self-citation chain makes the claimed result equal to its own input.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 0 invented entities

The headline result rests on the pretrained knowledge of an unspecified base model, a pre-existing action-skill library, an unvalidated refinement ensemble, and the authors' success criteria. The paper's own contribution adds only two unreported free parameters, lambda and epsilon.

free parameters (2)
  • lambda (teaching loss weight) = not reported
    Eq. 3 weights teaching episodes by lambda >= 1; the value is never given, yet it controls the balance between autonomous and taught supervision.
  • epsilon (validation threshold) = not reported
    Eq. 1 uses epsilon to decide whether an action outcome matches expectation; the threshold is never quantified, so the replanning behavior is under-specified.
axioms (4)
  • domain assumption The pretrained LLM/VLM already has enough semantic and physical commonsense to ground instructions and solve held-out tasks.
    Table I marks four tasks as 'held out from the CoT training set; solved zero-shot by transfer', but the paper does not measure how much of this comes from the base model rather than from WCM.
  • domain assumption The low-level Action skill library reliably executes navigation, reaching, grasping, placing, handover, and articulated-object interactions.
    Sec. II-A-1d treats these skills as available; if they fail, task failures are attributed to the overall system even though the reasoning layer is not at fault.
  • ad hoc to paper A 'stronger model ensemble' can refine noisy autonomous rollouts into correct CoT traces.
    Sec. II-B1 invokes this ensemble as the source of training labels, but it is never identified or evaluated, making the self-improvement loop unverifiable.
  • domain assumption The authors' unstated success criteria are consistent and representative.
    Table I reports SR = successful/total attempts without defining what counts as success per task, leaving the headline open to lenient scoring.

pith-pipeline@v1.3.0-alltime-deepseek · 9264 in / 9409 out tokens · 95984 ms · 2026-08-01T03:52:21.951006+00:00 · methodology

0 comments
read the original abstract

Language agents can now interact fluently with users in software, but robots still struggle to bring comparable interaction to physical tasks. Current robot-control paradigms, including vision-language-action policies and world-model-based planners, are mainly optimized for instruction execution, leaving users with little visibility into why an action is chosen and few mechanisms to redirect, correct, or teach the robot through interaction. To solve this problem, we present the World-Cognition Model (WCM), a human-centered embodied agent built on the SLAK architecture (Sensing, Logic, Action, and Knowledge) and an asynchronous runtime. SLAK separates perception, reasoning, control, and memory, while the runtime allows reasoning, dialogue, and execution to proceed concurrently. WCM further introduces a human-in-the-loop teaching mode that enables users to interactively teach the robot difficult or long-horizon tasks. Teaching episodes and autonomous task rollouts are refined into chain-of-thought supervision to continually improve the model. WCM achieves a 73.8% average success rate across nine real-world human-robot interaction tasks, including tasks held out from CoT fine-tuning and a long-horizon task learned through teaching.

Figures

Figures reproduced from arXiv: 2607.22999 by KC Zhou, Yuzhen Chen.

Figure 1
Figure 1. Figure 1: The SLAK architecture. WCM organizes sensing, logic, action, and knowledge as explicit layers connected to a CoT distillation pipeline. Autonomous executions and human teaching episodes are refined into CoT training data for continual improvement. design lets WCM update the scene, goal, memory, and ac￾tion outcomes during execution, rather than hiding the full perception-to-action process inside a single e… view at source ↗
Figure 2
Figure 2. Figure 2: Human-in-the-loop teaching and sensing output. Left: natural-language teaching of a long-horizon task. Right: raw RGB-D observations and the corresponding interaction-aware scene state. The episode is refined and distilled with autonomous rollouts. slow or brittle. Teaching episodes enter the same refinement pipeline and shared distillation objective as autonomous execution records, but are weighted by λ i… view at source ↗
Figure 3
Figure 3. Figure 3: “Grab me the popcorn.” Object retrieval and handover [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: “Hand me the honey.” The robot locates the honey, grasps it, and hands it to the person [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: “Hand me the jacket.” Grasping a deformable object and delivering it to the person [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: “Hand me the green ratchet.” Retrieving a tool in a garage setting [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: “Hold the BBQ tongs.” Grasping and presenting the tongs to the person [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: “Grab me the water bottle.” Object retrieval and handover [PITH_FULL_IMAGE:figures/full_fig_p009_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: “Take the screwdriver to the drawer.” A two-stage manipulation: open the drawer, then place the screwdriver inside [PITH_FULL_IMAGE:figures/full_fig_p010_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: “Put the cup in the trash can.” (Outdoor) A multi-step task: pick up the cup, navigate outside, and dispose of it [PITH_FULL_IMAGE:figures/full_fig_p010_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: “Put the bottle in the trash can.” A multi-step task: pick up the bottle, locate the trash can, and dispose of it. APPENDIX B PERFORMANCE COMPARISON RESULTS TABLE II CASE-STUDY COMPARISON. PER-TASK COMPARISON OF WCM (OURS) AGAINST X-SQ. (X-SQUARE ROBOT; MODEL: WALL-OSS) ON SIX TASKS ACROSS THREE METRICS. ARROWS GIVE THE BETTER DIRECTION; BOLD MARKS THE BETTER METHOD PER COMPARED CELL. Movement time (s) ↓ … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

37 extracted references · 24 linked inside Pith

  1. [1]

    Cosmos world foun- dation model platform for physical ai.arXiv preprint arXiv:2501.03575, 2025

    Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, et al. Cosmos world foun- dation model platform for physical ai.arXiv preprint arXiv:2501.03575, 2025

    Pith/arXiv arXiv 2025

  2. [2]

    GR00T N1: An open foundation model for generalist humanoid robots

    Johan Bjorck, Fernando Castaneda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, et al. GR00T N1: An open foundation model for generalist humanoid robots. arXiv preprint arXiv:2503.14734, 2025

    Pith/arXiv arXiv 2025

  3. [3]

    InProceedings of The 9th Conference on Robot Learning, volume 305 ofPro- ceedings of Machine Learning Research, pages 17–40

    Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Robert Equi, et al.π 0.5: A vision-language-action model with open-world generalization. InProceedings of The 9th Conference on Robot Learning, volume 305 ofPro- ceedings of Machine Learning Research, pages 17–40. PMLR, 2025

    2025

  4. [4]

    InRobotics: Science and Systems (RSS), 2025

    Kevin Black, Noah Brown, Danny Driess, et al.π 0: A vision-language-action flow model for general robot control. InRobotics: Science and Systems (RSS), 2025. arXiv:2410.24164

    Pith/arXiv arXiv 2025

  5. [5]

    Lee, Maria Bauza, Todor Davchev, Yuxiang Zhou, et al

    Konstantinos Bousmalis, Giulia Vezzani, Dushyant Rao, Coline Devin, Alex X. Lee, Maria Bauza, Todor Davchev, Yuxiang Zhou, et al. Robocat: A self-improving gen- eralist agent for robotic manipulation.Transactions on Machine Learning Research, 2023. arXiv:2306.11706

    Pith/arXiv arXiv 2023

  6. [6]

    RT-1: Robotics transformer for real-world control at scale

    Anthony Brohan, Noah Brown, Justice Carbajal, et al. RT-1: Robotics transformer for real-world control at scale. InRobotics: Science and Systems (RSS), 2023. arXiv:2212.06817

    Pith/arXiv arXiv 2023

  7. [7]

    Genie: Generative interactive environments

    Jake Bruce, Michael D Dennis, Ashley Edwards, et al. Genie: Generative interactive environments. InPro- ceedings of the 41st International Conference on Ma- chine Learning, volume 235 ofProceedings of Machine Learning Research, pages 4603–4623. PMLR, 2024. arXiv:2402.15391

    Pith/arXiv arXiv 2024

  8. [8]

    Sam 3: Segment anything with concepts

    Nicolas Carion, Laura Gustafson, Yuan-Ting Hu, Shoub- hik Debnath, Ronghang Hu, Didac Suris, Chaitanya Ryali, Kalyan Vasudev Alwala, Haitham Khedr, An- drew Huang, Jie Lei, Tengyu Ma, Baishan Guo, Arpit Kalla, Markus Marks, Joseph Greer, Meng Wang, Peize Sun, Roman R ¨adle, Triantafyllos Afouras, Effrosyni Mavroudi, Katherine Xu, Tsung-Han Wu, Yu Zhou, Li...

    Pith/arXiv arXiv 2026

  9. [9]

    Mat- predict: A dataset and benchmark for learning material properties of diverse indoor objects.arXiv preprint arXiv:2505.13201, 2025

    Yuzhen Chen, Hojun Son, and Arpan Kusari. Mat- predict: A dataset and benchmark for learning material properties of diverse indoor objects.arXiv preprint arXiv:2505.13201, 2025

    Pith/arXiv arXiv 2025

  10. [10]

    Clark and Susan E

    Herbert H. Clark and Susan E. Brennan. Grounding in communication. In Lauren B. Resnick, John M. Levine, and Stephanie D. Teasley, editors,Perspectives on Socially Shared Cognition, pages 127–149. American Psychological Association, 1991

    1991

  11. [11]

    Implicit chain of thought reasoning via knowledge dis- tillation.arXiv preprint arXiv:2311.01460, 2023

    Yuntian Deng, Kiran Prasad, Roland Fernandez, et al. Implicit chain of thought reasoning via knowledge dis- tillation.arXiv preprint arXiv:2311.01460, 2023

    Pith/arXiv arXiv 2023

  12. [12]

    From explicit cot to implicit cot: Learning to internalize cot step by step.arXiv preprint arXiv:2405.14838, 2024

    Yuntian Deng, Yejin Choi, and Stuart Shieber. From explicit cot to implicit cot: Learning to internalize cot step by step.arXiv preprint arXiv:2405.14838, 2024

    Pith/arXiv arXiv 2024

  13. [13]

    Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, et al. PaLM-E: An embodied multimodal language model. InProceedings of the 40th International Conference on Machine Learning, volume 202 ofProceedings of Machine Learning Research, pages 8469–8488. PMLR,

  14. [14]

    You’ve got to feel it to believe it: Multi-modal bayesian inference for semantic and property prediction

    Parker Ewen, Hao Chen, Yuzhen Chen, Anran Li, Anup Bagali, Gitesh Gunjal, and Ram Vasudevan. You’ve got to feel it to believe it: Multi-modal bayesian inference for semantic and property prediction. InRobotics: Science and Systems (RSS), 2024. arXiv:2402.05872

    Pith/arXiv arXiv 2024

  15. [15]

    Octo: An open- source generalist robot policy

    Dibya Ghosh, Homer Rich Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, et al. Octo: An open- source generalist robot policy. InRobotics: Science and Systems (RSS), 2024. arXiv:2405.12213

    Pith/arXiv arXiv 2024

  16. [16]

    World models.arXiv preprint arXiv:1803.10122, 2018

    David Ha and J ¨urgen Schmidhuber. World models.arXiv preprint arXiv:1803.10122, 2018

    Pith/arXiv arXiv 2018

  17. [17]

    Principles of mixed-initiative user inter- faces

    Eric Horvitz. Principles of mixed-initiative user inter- faces. InProceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), pages 159– 166, 1999

    1999

  18. [18]

    Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes

    Cheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, et al. Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes. In Findings of the Association for Computational Linguis- tics (ACL), pages 8003–8017, 2023. arXiv:2305.02301

    Pith/arXiv arXiv 2023

  19. [19]

    Inner monologue: Embodied reasoning through planning with language models

    Wenlong Huang, Fei Xia, Ted Xiao, et al. Inner monologue: Embodied reasoning through planning with language models. InProceedings of The 6th Confer- ence on Robot Learning, volume 205 ofProceedings of Machine Learning Research, pages 1769–1782. PMLR,

  20. [20]

    Do as i can, not as i say: Grounding language in robotic affordances

    Brian Ichter, Anthony Brohan, Yevgen Chebotar, Chelsea Finn, Karol Hausman, Alexander Herzog, Daniel Ho, et al. Do as i can, not as i say: Grounding language in robotic affordances. InProceedings of The 6th Con- ference on Robot Learning, volume 205 ofProceedings of Machine Learning Research, pages 287–318. PMLR,

  21. [21]

    Dreamgen: Unlocking generalization in robot learning through video world models

    Joel Jang, Seonghyeon Ye, Zongyu Lin, Jiannan Xiang, Johan Bjorck, Yu Fang, Fengyuan Hu, et al. Dreamgen: Unlocking generalization in robot learning through video world models. InProceedings of The 9th Conference on Robot Learning, volume 305 ofProceedings of Machine Learning Research, pages 5170–5194. PMLR, 2025

    2025

  22. [22]

    OpenVLA: An open-source vision-language- action model

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, et al. OpenVLA: An open-source vision-language- action model. InProceedings of The 8th Conference on Robot Learning, volume 270 ofProceedings of Machine Learning Research, pages 2679–2713. PMLR, 2025. arXiv:2406.09246

    Pith/arXiv arXiv 2025

  23. [23]

    Code as policies: Language model programs for embod- ied control

    Jacky Liang, Wenlong Huang, Fei Xia, et al. Code as policies: Language model programs for embod- ied control. InIEEE International Conference on Robotics and Automation (ICRA), pages 9493–9500,

  24. [24]

    Let’s verify step by step

    Hunter Lightman, Vineet Kosaraju, Yura Burda, et al. Let’s verify step by step. InInternational Con- ference on Learning Representations (ICLR), 2024. arXiv:2305.20050

    Pith/arXiv arXiv 2024

  25. [25]

    Ground- ing dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, and Lei Zhang. Ground- ing dino: Marrying dino with grounded pre-training for open-set object detection. InEuropean Confer- ence on Computer Vision (ECCV), pages 38–55, 2024. arXiv:2303.05499

    Pith/arXiv arXiv 2024

  26. [26]

    Show your work: Scratchpads for intermediate com- putation with language models.arXiv preprint arXiv:2112.00114, 2021

    Maxwell Nye, Anders Andreassen, Guy Gur-Ari, et al. Show your work: Scratchpads for intermediate com- putation with language models.arXiv preprint arXiv:2112.00114, 2021

    Pith/arXiv arXiv 2021

  27. [27]

    Grounded sam: Assembling open-world models for di- verse visual tasks.arXiv preprint arXiv:2401.14159, 2024

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kun- chang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, Zhaoyang Zeng, Hao Zhang, Feng Li, Jie Yang, Hongyang Li, Qing Jiang, and Lei Zhang. Grounded sam: Assembling open-world models for di- verse visual tasks.arXiv preprint arXiv:2401.14159, 2024

    Pith/arXiv arXiv 2024

  28. [28]

    Sutton, Doina Precup, and Satinder Singh

    Richard S. Sutton, Doina Precup, and Satinder Singh. Between MDPs and semi-MDPs: A framework for tem- poral abstraction in reinforcement learning.Artificial Intelligence, 112(1–2):181–211, 1999

    1999

  29. [29]

    Solving math word problems with process- and outcome- based feedback.arXiv preprint arXiv:2211.14275, 2022

    Jonathan Uesato, Nate Kushman, Ramana Kumar, et al. Solving math word problems with process- and outcome- based feedback.arXiv preprint arXiv:2211.14275, 2022

    Pith/arXiv arXiv 2022

  30. [30]

    RoboGen: Towards unleashing infinite data for automated robot learning via generative simulation

    Yufei Wang, Zhou Xian, Feng Chen, Tsun-Hsuan Wang, Yian Wang, Katerina Fragkiadaki, Zackory Erickson, David Held, and Chuang Gan. RoboGen: Towards unleashing infinite data for automated robot learning via generative simulation. InProceedings of the 41st International Conference on Machine Learning, volume 235 ofProceedings of Machine Learning Research, pa...

    2024

  31. [31]

    Chain- of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, et al. Chain- of-thought prompting elicits reasoning in large language models. InAdvances in Neural Information Process- ing Systems (NeurIPS), volume 35, pages 24824–24837,

  32. [32]

    Daydreamer: World models for physical robot learning

    Philipp Wu, Alejandro Escontrela, Danijar Hafner, Pieter Abbeel, and Ken Goldberg. Daydreamer: World models for physical robot learning. InProceedings of The 6th Conference on Robot Learning, volume 205 ofProceed- ings of Machine Learning Research, pages 2226–2240. PMLR, 2023

    2023

  33. [33]

    Stream3d: Sequential multi-view 3d generation via evidential mem- ory.arXiv preprint arXiv:2605.21472, 2026

    Kaichen Zhou, Zeyang Bai, Xinhai Chang, Mengyu Wang, Paul Pu Liang, and Fangneng Zhan. Stream3d: Sequential multi-view 3d generation via evidential mem- ory.arXiv preprint arXiv:2605.21472, 2026

    Pith/arXiv arXiv 2026

  34. [34]

    Gem-4d: Geometry- enhanced video world models for robot manipulation

    Kaichen Zhou, Yuzhen Chen, Fangneng Zhan, Hang Hua, Grace Chen, Xinhai Chang, Ao Qu, Yilun Du, Zhuang Liu, Paul Pu Liang, et al. Gem-4d: Geometry- enhanced video world models for robot manipulation. arXiv preprint arXiv:2605.22882, 2026

    Pith/arXiv arXiv 2026

  35. [35]

    Page-4d: Disentangled pose and geometry estimation for vggt-4d perception

    Kaichen Zhou, Yuhan Wang, Grace Chen, Gaspard Beau- douin, Fangneng Zhan, Paul Pu Liang, and Mengyu Wang. Page-4d: Disentangled pose and geometry estimation for vggt-4d perception. InInternational Conference on Learning Representations (ICLR), 2026. arXiv:2510.17568

    Pith/arXiv arXiv 2026

  36. [36]

    Robodreamer: Learning compositional world models for robot imagination

    Siyuan Zhou, Yilun Du, Jiaben Chen, Yandong Li, Dit- Yan Yeung, and Chuang Gan. Robodreamer: Learning compositional world models for robot imagination. In Proceedings of the 41st International Conference on Ma- chine Learning, volume 235 ofProceedings of Machine Learning Research, pages 61885–61896. PMLR, 2024

    2024

  37. [37]

    Grab me the popcorn

    Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, et al. RT-2: Vision-language-action models transfer web knowledge to robotic control. InProceed- ings of The 7th Conference on Robot Learning, volume 229 ofProceedings of Machine Learning Research, pages 2165–2183. PMLR, 2023. arXiv:2307.15818. APPENDIXA HUMAN–ROBOTINTERACTIONDEMOS TABLE...

    Pith/arXiv arXiv 2023