Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

A new in-context imitation learning framework, ICLR, augments demonstration prompts with visual reasoning traces—polyline sketches of the gripper's future path in image space—and trains a single autoregressive transformer to generate both t

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 18:34 UTC pith:HGFNGBGY

load-bearing objection Solid, well-specified method with a real evaluation, but its own LIBERO results undercut the central claim that test-time visual reasoning helps. the 4 major comments →

arxiv 2603.07530 v2 pith:HGFNGBGY submitted 2026-03-08 cs.RO

ICLR: In-Context Imitation Learning with Visual Reasoning

Toan Nguyen , Weiduo Yuan , Songlin Wei , Hui Li , Daniel Seita , Yue Wang This is my paper
classification cs.RO
keywords in-context imitation learningvisual reasoning tracesrobotic manipulationautoregressive transformertask intentgeneralizationkeypoint trajectoriesreasoning dropout
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that in-context imitation learning—teaching a robot to copy a new task from a few demonstrations at test time—works better when the demonstrations include a visual reasoning trace: a short polyline in the camera image showing where the gripper will go next. The authors show that a single transformer trained to produce both the trace and the low-level action outperforms a strong action-only baseline across simulated and real manipulation tasks, and that the benefit holds for unseen tasks and novel objects. Their key evidence: overall success rates in real-world tasks rise from roughly 22 percent for the baseline to 60 percent for their full model, and from 16 to 54 percent in simulation for the best variant. Why it matters: robots that can infer intent from a few examples, rather than being retrained, could adapt to new tasks quickly with minimal human data.

Core claim

The central claim is that explicitly representing task intent as a future gripper trajectory in pixel space—rather than leaving it implicit in state–action pairs—makes in-context imitation more robust. Concretely, the paper augments each prompt demonstration with a five-point polyline of the gripper's positions in the third-view image, sampled evenly from the current time step to the end of the trajectory, and trains a causal transformer to autoregressively predict the next reasoning trace and then the next action chunk. At inference, the model first generates a trace, then uses it to condition action prediction. The paper reports consistent gains over an action-only baseline in both simulat

What carries the argument

The central object is the visual reasoning trace: a polyline of five keypoints in the third-view image, each giving the gripper's pixel location at uniformly spaced future time steps, with five points chosen to align with the phases of pick-and-place tasks. The carrying mechanism is a unified autoregressive transformer that interleaves three token streams—state tokens from camera images and proprioception, reasoning tokens embedding the trace, and action tokens—and is trained with a next-token objective that sums an action loss and a weighted reasoning loss. A dedicated training trick, reasoning dropout, randomly masks reasoning tokens in the target trajectory while keeping them in the promp

Load-bearing premise

The method assumes a five-point polyline of the gripper's future pixel positions—generated from the known trajectory at data-collection time and approximated by a vision-language model at test time—captures enough task intent to guide action prediction, and that the predicted traces are accurate enough to help rather than mislead, an assumption the paper's own simulation results (where dropping traces beats using them) show is fragile in low-diversity settings.

What would settle it

A direct test: on a suite of unseen tasks, compare the full model's success rate when it is fed (a) its own predicted traces, (b) ground-truth traces from the oracle, and (c) no traces. If ground-truth traces barely outperform predicted ones, the bottleneck is execution, not reasoning; if predicted traces perform as well as or worse than no traces, the claim that generated traces guide action prediction would need to be abandoned. A second test: take a task whose future trajectory cannot be captured by five evenly sampled points (e.g., a long multi-stage task with many reversals) and see wheth

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the claim holds, robotic in-context imitation policies can be prompted with demonstrations that include a simple visual sketch of intent, and the same policy transformer can generate both the reasoning and the action without any architectural change beyond adding a reasoning token stream.
  • The reasoning-dropout training trick suggests a spectrum of inference-time reasoning frequencies, letting a practitioner trade trace-generation cost against success rate; the paper shows reasoning every 8 steps performs nearly as well as every step while being roughly 8× faster.
  • The failure analysis implies that further gains would come mostly from improving low-level control (grasping and placement) and from better gripper localization, not from changing the reasoning representation itself.
  • The interpretable traces provide a debugging signal: when a rollout fails, the trace reveals whether the model misread the task intent or mis-executed the action.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's own simulation results, where the dropout variant beats the full model, suggest the value of explicit reasoning traces is environment-dependent: in low-diversity settings the model may internalize reasoning and generated traces become noise, while the real-world gain is where the method shines; a curriculum over trace noise or a confidence gate for when to trust a generated trace could
  • A natural extension is to replace the five-point polyline with a denser or multi-view representation for long-horizon and bimanual tasks, since five points are tuned to pick-and-place; the implied test is whether gains scale with trace granularity.
  • If the key mechanism is that traces force the model to attend to the future, a comparable benefit might be achieved with a learned latent 'intent' token optimized end-to-end rather than supervised pixel traces; comparing these would isolate whether the explicit visual format or the forward-looking objective matters.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes ICLR, a transformer-based in-context imitation learning method that augments prompt demonstrations with visual reasoning traces—five-point polylines of future gripper positions in the third-view camera image—and jointly learns to generate these traces and low-level actions in a unified autoregressive model. The method is evaluated on LIBERO in simulation and on a Franka Research 3 robot with a UMI gripper across 12 unseen real-world tasks. The paper reports that the full ICLR model outperforms ICRT and target-only ablations in real-world settings, while in simulation the reasoning-dropout variant outperforms the complete model. The central claimed contribution is that explicitly generating visual reasoning traces at inference improves in-context imitation and generalization to unseen tasks.

Significance. If the central claim were established, the paper would make a useful contribution: it provides a concrete recipe for creating visual reasoning traces, a unified architecture for trace-and-action generation, and a real-robot evaluation protocol that goes beyond common simulation-only studies. The real-world experiments, the prompt-type ablation, the reasoning-interval study, and the failure analysis are strengths. However, the current evidence is mixed. The full model underperforms its own reasoning-dropout variant on the main simulation benchmark, and the explanation offered for this discrepancy is not directly tested. As a result, the paper's headline claim of 'consistent improvements' from test-time visual reasoning is not yet supported.

major comments (4)
  1. [§V-B, Table I] The central claim—that generating visual reasoning traces at inference improves imitation—is contradicted by the paper's own simulation results. The complete model reaches 33.34% overall on LIBERO, while Ours Dropout reaches 54.05%; the gap is especially large in Kitchen (20.00 vs 60.22) and Living Room (11.20 vs 38.93). The explanation in §V-C (limited trace diversity, error propagation) is post hoc: no experiment measures trace prediction error, correlates it with action success, or varies trace diversity. The abstract's 'consistent improvements' are therefore not supported by Table I. Please add a direct test, e.g., report predicted-vs-ground-truth trace pixel error, correlate this error with rollout success, and evaluate an 'oracle trace' condition in which ground-truth future traces are provided at inference.
  2. [§V-C, Table II] The real-world comparison of Ours vs Ours Dropout is based on 10 rollouts per task with binary outcomes; 60.00% vs 46.67% over 60 rollouts may well be within sampling noise, and no confidence intervals or significance tests are provided. The claim that real-world scene diversity makes explicit reasoning more valuable is plausible but is not quantified. The paper should report per-task counts, standard errors, and the accuracy of Molmo2's gripper pointing on the actual test images. Without this, the real-world result does not offset the simulation failure.
  3. [§IV-B] The visual reasoning trace is defined as the future gripper trajectory sampled from the current time step to the end of the episode. This is essentially a re-encoding of future states/actions into image space. The paper does not compare against a baseline that predicts the same future trajectory in proprioceptive/end-effector space or as raw waypoints. Without such a control, it is unclear whether improvements over ICRT come from the visual/image-space representation, from the auxiliary future-prediction task, or from prompt-trace conditioning. The TO ablation removes prompt traces but does not remove the auxiliary reasoning objective, so it does not isolate the effect of visual traces as the causal mechanism.
  4. [§V-B] The LIBERO train/test split is described as 'randomly select' with no seed or fixed split, and the paper does not report standard deviations across training runs or test splits. Since the results include large differences between models, reproducibility would be improved by fixing and releasing the split, and by reporting variance. The current presentation makes it difficult to assess whether the reported rankings are stable.
minor comments (6)
  1. [§V-C] Typo: 'LIERBO' should be 'LIBERO'.
  2. [Eq. (1)] The reasoning loss weight is set to 0.3 empirically, but no sensitivity analysis is provided. Please justify or ablate this choice.
  3. [§IV-C] The masking strategy for reasoning tokens is described only as 'randomly mask a subset' with a ratio sampled from 0% to 100%. Please specify the sampling distribution, whether the ratio is per sequence or per token, and whether prompt traces are ever masked.
  4. [Figure 2] The Molmo2 prompt 'Point to the robot gripper' is mentioned in the text but not shown in the figure; also clarify the asymmetry between simulation (known camera parameters) and real world (VLM detection) in the trace-generation process.
  5. [Table III] The abbreviation 'Distr.' is used in the caption but not defined. Also, for the '2 Demos' configuration, specify whether the two demos are sampled without replacement and whether the same prompt set is used across all 10 rollouts.
  6. [§V-A] The paper states that shared components are identical, but the TO models omit prompt traces while our models include them. Please clarify whether the input embedding and attention masking differ between these variants, beyond the mere presence or absence of trace tokens.

Circularity Check

0 steps flagged

No significant circularity: the reasoning traces are a preprocessing of future gripper positions, and the main comparisons are against external baselines.

full rationale

The paper's derivation chain is self-contained rather than circular. Visual reasoning traces are generated as a fixed preprocessing step: in simulation from the gripper's 3D position and known camera parameters, and in the real world from a VLM pointing at the gripper (Section IV-B). These traces are not fitted parameters and are not functions of the model's own outputs. The model is trained with teacher-forced ground-truth traces and actions (Section IV-C), and action success is measured against external baselines (ICRT [5]) on unseen LIBERO tasks and real-world tasks. The fact that ICRT shares a coauthor (H. Li) is not load-bearing: it is a published, external, state-of-the-art baseline, not an unverified uniqueness theorem. The strongest candidate concern is that the reasoning polyline is sampled from the future horizon of the episode, so it is informationally redundant with the future actions being predicted; the paper's own simulation result (Ours Dropout 54.05% vs Ours 33.34% in Table I) shows that this oracle-future conditioning can hurt at test time when traces must be predicted rather than teacher-forced. That is a substantive empirical weakness and a train/test mismatch, but it is not a circular reduction: the reported improvements over ICRT are not forced by construction, no fitted quantity is renamed as a prediction, and no load-bearing claim rests on self-citation. Therefore no circularity step is identified and the score is 0.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

ICLR introduces no new physical entity, loss family, or representation class: it combines existing next-token in-context learning with existing MolmoAct-style keypoint traces. The load-bearing choices are all empirical hyperparameters or domain assumptions, so the ledger is short. The most important item is the assumption that Molmo2 gripper detections are accurate and that the five-point trace actually carries task intent; the paper provides no quantitative error analysis of the trace generator.

free parameters (4)
  • reasoning loss weight = 0.3
    Selected empirically to balance action and reasoning learning (Section IV-C, Eq. 1); no sensitivity analysis is given.
  • number of keypoints per trace = 5
    Chosen as a practical balance for pick-and-place (Section IV-B).
  • action chunk length = 16
    Inherited from ICRT/ACT-style chunking, fixed, not swept in this paper.
  • reasoning masking ratio = sampled from 0% to 100%
    Uniform random masking ratio chosen as regularization; no ablation on the distribution is provided.
axioms (4)
  • domain assumption Molmo2's 'Point to the robot gripper' outputs are accurate enough to serve as training traces in the real world
    Invoked in Section IV-B for real-world trace generation; accuracy is described as 'high precision' but not quantified as a detection rate or pixel error.
  • domain assumption A five-point evenly spaced polyline of future gripper pixel positions is a sufficient representation of task intent for pick-and-place/poking
    Introduced in Section IV-B; the paper itself notes limited trace diversity may restrict trace learning (Section V-C).
  • domain assumption Teacher-forced training with ground-truth traces transfers to autoregressively generated traces at test time
    Used in Section IV-C; the simulation results show the full model underperforms the dropout variant, indicating this transfer can fail in lower-diversity settings.
  • domain assumption LIBERO-Object / LIBERO-90 train-test splits and the custom real-world 12-task protocol are representative of in-context generalization
    Section V-B/V-C; the paper states 'there is no standard setting for in-context imitation learning on LIBERO'.

pith-pipeline@v1.3.0-alltime-deepseek · 11908 in / 8281 out tokens · 61886 ms · 2026-08-02T18:34:12.131111+00:00 · methodology

0 comments
read the original abstract

In-context imitation learning enables robots to adapt to new tasks from a small number of demonstrations without additional training. However, existing approaches typically condition only on state-action trajectories and lack explicit representations of task intent. This limitation hinders performance in complex and ambiguous task settings where the same actions may be consistent with different objectives. To address this, we present In-Context Imitation Learning with Visual Reasoning (ICLR), a novel framework that augments demonstration prompts with structured visual reasoning traces representing anticipated future robot trajectories in image space. ICLR also jointly learns to generate reasoning traces and low-level actions within a unified autoregressive transformer, enabling the model to mimic not only action prediction but also the reasoning process that leads to those actions. We extensively evaluate ICLR in both simulation and real-world manipulation tasks and demonstrate consistent improvements in success rates and generalization to unseen tasks and novel object configurations compared to other in-context imitation learning methods. These results suggest that incorporating embodied visual reasoning represents a promising direction for enhancing the robustness and generalization of robotic in-context learning systems.

Figures

Figures reproduced from arXiv: 2603.07530 by Daniel Seita, Hui Li, Songlin Wei, Toan Nguyen, Weiduo Yuan, Yue Wang.

Figure 1
Figure 1. Figure 1: General framework overview. Our method augments prompt demos with keypoint-based visual reasoning traces in the image space, shown above with the overlaid polyline in the middle column. During inference, the model also performs visual reasoning before predicting the subsequent low-level robot action. The task’s language description is included for clarity. ambiguous task settings, such as environments with… view at source ↗
Figure 2
Figure 2. Figure 2: Method overview. (A) To generate the visual reasoning trace at a given time step, we uniformly sample five third-view images from that time step to the end of the trajectory and use Molmo2 to predict the gripper’s pixel location in each image. (B) Multi-view camera observations and proprioceptive states are encoded by a state encoder to produce state tokens fs . Visual reasoning traces are embedded by a re… view at source ↗
Figure 3
Figure 3. Figure 3: Real robot setting. We use a Franka Research 3 robot arm equipped with a UMI gripper. Visual observations are captured by two RealSense cameras. Teleoperation for data collection and test-time prompt demonstration recording is performed using a GELLO system. Testing objects appearing in training episodes are shown in the bottom-left box, while completely unseen testing objects are shown in the bottom-right… view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative results. Rollout examples of our complete ICLR model in simulation (first two rows) and real-world settings (two bottom rows). All presented visual traces are predicted by our model. simulation, the differences between training and testing scene configurations, particularly object positions, are relatively small. This reduces the need for explicitly generating visual traces and allows the dropo… view at source ↗
Figure 5
Figure 5. Figure 5: Three types of prompt demonstrations. The task of picking up the tomato and putting it in the grey bowl is selected. 1 Demo Method 0 Distr. 1 Distr. Distr. Receptacle 2 Demos 3 Demos ICRT [5] 35 30 40 35 45 TO Dropout 45 25 20 25 45 TO 40 40 50 50 30 Ours Dropout 35 35 40 55 40 Ours 55 65 55 70 65 TABLE III: Results of different prompt types (%). “Distr.” stands for distractor. Success rates are calculated… view at source ↗
Figure 7
Figure 7. Figure 7: Results of different reasoning intervals. The experiment is conducted on the task of putting the hedgehog into the red box, with 10 rollouts for each model variant. performance is more often limited by downstream execution challenges rather than incorrect reasoning. Moreover, visual trace errors could potentially be mitigated by improved grip￾per localization models (e.g., future Molmo versions), which wou… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. SynthICL: Scalable In-context Imitation Learning with Synthetic Data

    cs.RO 2026-06 unverdicted novelty 6.0

    SynthICL trains flow-matching transformer policies for in-context imitation learning entirely from synthetic RGB data and reports 79% average success on 16 unseen real manipulation tasks with one test-time demonstration.

  2. World Action Models: A Survey

    cs.RO 2026-06 unverdicted novelty 3.0

    A survey that clarifies boundaries and organizes World Action Models by generation requirements and predictive substrates, identifying a trend toward generating less of the future.

Reference graph

Works this paper leans on

45 extracted references · 8 linked inside Pith · cited by 2 Pith papers

  1. [1]

    Good old-fashioned engineering can close the 100,000- year “data gap

    K. Goldberg, “Good old-fashioned engineering can close the 100,000- year “data gap” in robotics,” 2025

    2025

  2. [2]

    Droid: A large-scale in-the-wild robot manipulation dataset,

    A. Khazatsky, K. Pertsch, S. Nair, A. Balakrishna, S. Dasari, S. Karam- cheti, S. Nasiriany, M. K. Srirama, L. Y . Chen, K. Elliset al., “Droid: A large-scale in-the-wild robot manipulation dataset,” inRSS, 2024

    2024

  3. [3]

    Bc-z: Zero-shot task generalization with robotic imitation learning,

    E. Jang, A. Irpan, M. Khansari, D. Kappler, F. Ebert, C. Lynch, S. Levine, and C. Finn, “Bc-z: Zero-shot task generalization with robotic imitation learning,” inCoRL, 2022

    2022

  4. [4]

    Rovi-aug: Robot and viewpoint augmentation for cross-embodiment robot learning,

    L. Y . Chen, C. Xu, K. Dharmarajan, R. Cheng, K. Keutzer, M. Tomizuka, Q. Vuong, and K. Goldberg, “Rovi-aug: Robot and viewpoint augmentation for cross-embodiment robot learning,” in CoRL, 2024

    2024

  5. [5]

    Icrt: In-context imitation learning via next-token prediction,

    M. Fu, H. Huang, G. Datta, L. Y . Chen, W. Panitch, F. Liu, H. Li, and K. Goldberg, “Icrt: In-context imitation learning via next-token prediction,” inICRA, 2025

    2025

  6. [6]

    Novel demonstration generation with gaussian splatting enables ro- bust one-shot manipulation,

    S. Yang, W. Yu, J. Zeng, J. Lv, K. Ren, C. Lu, D. Lin, and J. Pang, “Novel demonstration generation with gaussian splatting enables ro- bust one-shot manipulation,” inRSS, 2025

    2025

  7. [7]

    World action models are zero- shot policies,

    S. Ye, Y . Ge, K. Zheng, S. Gao, S. Yu, G. Kurian, S. Indupuru, Y . L. Tan, C. Zhu, J. Xianget al., “World action models are zero- shot policies,”arXiv preprint arXiv:2602.15922, 2026

    Pith/arXiv arXiv 2026

  8. [8]

    A systematic study of data modalities and strategies for co-training large behavior models for robot manipulation,

    F. Lin, K. Arora, J. Mercat, H. Nishimura, P. Shah, C. Xu, M. Zhang, M. Zolotas, M. Angeles, O. Pfannenstiehlet al., “A systematic study of data modalities and strategies for co-training large behavior models for robot manipulation,”arXiv preprint arXiv:2602.01067, 2026

    arXiv 2026

  9. [9]

    Keypoint action tokens enable in-context imitation learning in robotics,

    N. Di Palo and E. Johns, “Keypoint action tokens enable in-context imitation learning in robotics,” inRSS, 2024

    2024

  10. [10]

    Instant policy: In-context imitation learning via graph diffusion,

    V . V osylius and E. Johns, “Instant policy: In-context imitation learning via graph diffusion,” inICLR, 2025

    2025

  11. [11]

    Mimicdroid: In-context learning for humanoid manipulation from human play videos,

    R. Shah, S. Liu, Q. Wang, Z. Jiang, S. Kumar, M. Seo, R. Mart ´ın- Mart´ın, and Y . Zhu, “Mimicdroid: In-context learning for humanoid manipulation from human play videos,” inICRA, 2026

    2026

  12. [12]

    Chain-of-thought prompting elicits reasoning in large language models,

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V . Le, D. Zhouet al., “Chain-of-thought prompting elicits reasoning in large language models,”NeurIPS, 2022

    2022

  13. [13]

    Ddcot: Duty-distinct chain-of-thought prompting for multimodal reasoning in language models,

    G. Zheng, B. Yang, J. Tang, H.-Y . Zhou, and S. Yang, “Ddcot: Duty-distinct chain-of-thought prompting for multimodal reasoning in language models,”NeurIPS, 2023

    2023

  14. [14]

    Multimodal chain-of-thought reasoning in language models,

    Z. Zhang, A. Zhang, M. Li, H. Zhao, G. Karypis, and A. Smola, “Multimodal chain-of-thought reasoning in language models,”TMLR, 2024

    2024

  15. [15]

    Few-shot in-context imitation learning via implicit graph alignment,

    V . V osylius and E. Johns, “Few-shot in-context imitation learning via implicit graph alignment,” inCoRL, 2023

    2023

  16. [16]

    Vid2robot: End-to- end video-conditioned policy learning with cross-attention transform- ers,

    V . Jain, M. Attarian, N. J. Joshi, A. Wahid, D. Driess, Q. Vuong, P. R. Sanketi, P. Sermanet, S. Welker, C. Chanet al., “Vid2robot: End-to- end video-conditioned policy learning with cross-attention transform- ers,” inRSS, 2024

    2024

  17. [17]

    Ricl: Adding in- context adaptability to pre-trained vision-language-action models,

    K. Sridhar, S. Dutta, D. Jayaraman, and I. Lee, “Ricl: Adding in- context adaptability to pre-trained vision-language-action models,” in CoRL, 2025

    2025

  18. [18]

    Thinkact: Vision-language-action reasoning via reinforced visual la- tent planning,

    C.-P. Huang, Y .-H. Wu, M.-H. Chen, Y .-C. F. Wang, and F.-E. Yang, “Thinkact: Vision-language-action reasoning via reinforced visual la- tent planning,” inNeurIPS, 2025

    2025

  19. [19]

    Robotic control via embodied chain-of-thought reasoning,

    M. Zawalski, W. Chen, K. Pertsch, O. Mees, C. Finn, and S. Levine, “Robotic control via embodied chain-of-thought reasoning,” inCoRL, 2024

    2024

  20. [20]

    Cot-vla: Visual chain-of-thought reasoning for vision-language-action models,

    Q. Zhao, Y . Lu, M. J. Kim, Z. Fu, Z. Zhang, Y . Wu, Z. Li, Q. Ma, S. Han, C. Finnet al., “Cot-vla: Visual chain-of-thought reasoning for vision-language-action models,” inCVPR, 2025

    2025

  21. [21]

    Molmoact: Action reasoning models that can reason in space,

    J. Lee, J. Duan, H. Fang, Y . Deng, S. Liu, B. Li, B. Fang, J. Zhang, Y . R. Wang, S. Leeet al., “Molmoact: Action reasoning models that can reason in space,” inICRA, 2026

    2026

  22. [22]

    Gemini robotics: Bringing ai into the physical world,

    Gemini Robotics Team, “Gemini robotics: Bringing ai into the physical world,”arXiv preprint arXiv:2503.20020, 2025

    Pith/arXiv arXiv 2025

  23. [23]

    pi0.5: a vision-language-action model with open-world generalization,

    Physical Intelligence, “pi0.5: a vision-language-action model with open-world generalization,” inCoRL, 2025

    2025

  24. [24]

    Hamster: Hierarchical action models for open-world robot manipulation,

    Y . Li, Y . Deng, J. Zhang, J. Jang, M. Memmel, C. R. Garrett, F. Ramos, D. Fox, A. Li, A. Guptaet al., “Hamster: Hierarchical action models for open-world robot manipulation,” inICLR, 2025

    2025

  25. [25]

    Coa-vla: Improving vision-language-action models via visual-text chain-of-affordance,

    J. Li, Y . Zhu, Z. Tang, J. Wen, M. Zhu, X. Liu, C. Li, R. Cheng, Y . Peng, Y . Penget al., “Coa-vla: Improving vision-language-action models via visual-text chain-of-affordance,” inICCV, 2025

    2025

  26. [26]

    Peek: Guiding and minimal image representations for zero-shot generalization of robot manipulation policies,

    J. Zhang, M. Memmel, K. Kim, D. Fox, J. Thomason, F. Ramos, E. Bıyık, A. Gupta, and A. Li, “Peek: Guiding and minimal image representations for zero-shot generalization of robot manipulation policies,” inICRA, 2026

    2026

  27. [27]

    Molmo2: Open weights and data for vision-language models with video understanding and grounding,

    C. Clark, J. Zhang, Z. Ma, J. S. Park, M. Salehi, R. Tripathi, S. Lee, Z. Ren, C. D. Kim, Y . Yanget al., “Molmo2: Open weights and data for vision-language models with video understanding and grounding,” arXiv preprint arXiv:2601.10611, 2026

    Pith/arXiv arXiv 2026

  28. [28]

    Gemini robotics 1.5: Pushing the frontier of generalist robots with advanced embodied reasoning, thinking, and motion transfer,

    Gemini Robotics Team, “Gemini robotics 1.5: Pushing the frontier of generalist robots with advanced embodied reasoning, thinking, and motion transfer,”arXiv preprint arXiv:2510.03342, 2025

    Pith/arXiv arXiv 2025

  29. [29]

    Qwen3-vl technical report,

    Qwen Team, “Qwen3-vl technical report,”arXiv preprint arXiv:2511.21631, 2025

    Pith/arXiv arXiv 2025

  30. [30]

    Sam 2: Segment anything in images and videos,

    N. Ravi, V . Gabeur, Y .-T. Hu, R. Hu, C. Ryali, T. Ma, H. Khedr, R. R ¨adle, C. Rolland, L. Gustafsonet al., “Sam 2: Segment anything in images and videos,” inICLR, 2025

    2025

  31. [31]

    Sim ´eoni, H

    O. Sim ´eoni, H. V . V o, M. Seitzer, F. Baldassarre, M. Oquab, C. Jose, V . Khalidov, M. Szafraniec, S. Yi, M. Ramamonjisoaet al., “Dinov3,” arXiv preprint arXiv:2508.10104, 2025

    Pith/arXiv arXiv 2025

  32. [32]

    End-to-end object detection with transformers,

    N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” in ECCV, 2020

    2020

  33. [33]

    Scaling open-vocabulary object detection,

    M. Minderer, A. Gritsenko, and N. Houlsby, “Scaling open-vocabulary object detection,”NeurIPS, 2023

    2023

  34. [34]

    Hand me the data: Fast robot adaptation via hand path retrieval,

    M. Hong, A. Liang, K. Kim, H. Rajaprakash, J. Thomason, E. Bıyık, and J. Zhang, “Hand me the data: Fast robot adaptation via hand path retrieval,” inICRA, 2026

    2026

  35. [35]

    Steerable vision-language- action policies for embodied reasoning and hierarchical control,

    W. Chen, J. Bhatia, C. Glossop, N. Mathihalli, R. Doshi, A. Tang, D. Driess, K. Pertsch, and S. Levine, “Steerable vision-language- action policies for embodied reasoning and hierarchical control,”arXiv preprint arXiv:2602.13193, 2026

    Pith/arXiv arXiv 2026

  36. [36]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosaleet al., “Llama 2: Open foundation and fine-tuned chat models,”arXiv preprint arXiv:2307.09288, 2023

    Pith/arXiv arXiv 2023

  37. [37]

    Set transformer: A framework for attention-based permutation-invariant neural networks,

    J. Lee, Y . Lee, J. Kim, A. Kosiorek, S. Choi, and Y . W. Teh, “Set transformer: A framework for attention-based permutation-invariant neural networks,” inICML, 2019

    2019

  38. [38]

    Learning fine-grained bimanual manipulation with low-cost hardware,

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn, “Learning fine-grained bimanual manipulation with low-cost hardware,” inRSS, 2023

    2023

  39. [39]

    Training strategies for efficient embodied reasoning,

    W. Chen, S. Belkhale, S. Mirchandani, O. Mees, D. Driess, K. Pertsch, and S. Levine, “Training strategies for efficient embodied reasoning,” inCoRL, 2025

    2025

  40. [40]

    Libero: Benchmarking knowledge transfer for lifelong robot learn- ing,

    B. Liu, Y . Zhu, C. Gao, Y . Feng, Q. Liu, Y . Zhu, and P. Stone, “Libero: Benchmarking knowledge transfer for lifelong robot learn- ing,”NeurIPS, 2023

    2023

  41. [41]

    Openvla: An open-source vision-language-action model,

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. P. Foster, P. R. Sanketi, Q. Vuonget al., “Openvla: An open-source vision-language-action model,” inCoRL, 2024

    2024

  42. [42]

    Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots,

    C. Chi, Z. Xu, C. Pan, E. Cousineau, B. Burchfiel, S. Feng, R. Tedrake, and S. Song, “Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots,” inRSS, 2024

    2024

  43. [43]

    Gello: A general, low- cost, and intuitive teleoperation framework for robot manipulators,

    P. Wu, Y . Shentu, Z. Yi, X. Lin, and P. Abbeel, “Gello: A general, low- cost, and intuitive teleoperation framework for robot manipulators,” in IROS, 2024

    2024

  44. [44]

    Lmact: A benchmark for in-context imitation learning with long multimodal demonstrations,

    A. Ruoss, F. Pardo, H. Chan, B. Li, V . Mnih, and T. Genewein, “Lmact: A benchmark for in-context imitation learning with long multimodal demonstrations,” inICML, 2025

    2025

  45. [45]

    Language models are few-shot learners,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askellet al., “Language models are few-shot learners,”NeurIPS, 2020

    2020