Pith. sign in

REVIEW 4 major objections 4 minor 55 references

Training-free Generation of Temporally Consistent Rewards from VLMs

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a training-free particle-filter tracker can turn a single VLM query per episode — subgoals, initial completion estimates, and affordance functions — into temporally consistent rewards that match hand-coded…

desk verdict The paper's particle filter as written cannot track subgoal changes, making the central claim unsupported, though the integration is promising. read the letter →

arxiv 2507.04789 v1 pith:BBKCIXRD submitted 2025-07-07 cs.RO

Yinuo Zhao , Jiale Yuan , Zhiyuan Xu , Xiaoshuai Hao , Xinyi Zhang , Kun Wu , Zhengping Che , Chi Harold Liu
show 1 more author
Jian Tang
This is my paper · ORCID
classification cs.RO
keywords vision-languagemodelsrewardgenerationreinforcementlearningrobotmanipulationparticlefilterBayesiantrackingtemporalconsistencysubgoaldecomposition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes T2-VLM, a way to generate rewards for robot reinforcement learning from a vision-language model without fine-tuning and without querying the VLM at every time step. The VLM is used once per episode to decompose a task into subgoals, estimate which subgoals are already complete, and write small functions that say where objects would appear if each subgoal were satisfied or unsatisfied. A particle filter — a Bayesian tracker that keeps a set of weighted hypotheses — then tracks the evolving subgoal completion status from object bounding-box trajectories extracted by SAM 2, and the reward is the change in that estimated status between decision intervals. The paper reports that this reward supports RL training about as well as hand-coded environment rewards on six manipulation tasks across three domains, while cutting VLM query counts by roughly 77–88 percent. The core bet is that temporal tracking corrects the VLM's noisy one-shot spatial judgments, making accurate rewards cheap.

What carries the argument

The key machinery is a particle filter over an N-dimensional subgoal hidden state, combined with what the paper calls VLM-coding affordance. For each subgoal i, the VLM writes code for functions that return point sets of where the target object would be if that subgoal were satisfied or unsatisfied; Eq. (5) takes the minimum distance from the observed bounding-box position to each point set, and Eq. (6) folds those distances into a particle weight. The particle filter propagates particles with a constant-velocity motion model, reweights by this observation model, and resamples using systematic resampling. The result is a continuous estimate of goal completion that changes smoothly over time, so rewards computed from differences in that estimate are temporally consistent and require only one VLM call per episode.

What would settle it

Take a set of annotated frames with known subgoal completion, run only the observation model (Eqs. (5)-(6)), and check whether the true subgoal-completion vector receives the highest particle weight more often than chance. If the distance-based weights do not track ground-truth completion — or if normalizing a product of distances that are not probabilities introduces bias — the central claim fails.

Watch

Extended reading notes

Core claim

The central claim is that a training-free, temporally consistent reward can be produced from a single VLM initialization by treating subgoal completion as a latent state and tracking it with a Bayesian particle filter. After one VLM query yields subgoals, an initial binary completion vector, and VLM-written affordance functions, SAM 2 supplies per-frame bounding boxes. Each particle is a soft hypothesis of the completion vector; it is propagated by a constant-velocity motion model, reweighted according to how close the observed boxes are to the VLM-generated satisfied or unsatisfied affordance point sets, and resampled. The estimated state is the weighted average of particles, and the reward at decision step t is a function of the change in that estimated state over the decision interval. The experiments demonstrate that such rewards drive RL to goal-completion ratios comparable to environmental rewards and outperform per-step VLM scoring and learned reward baselines.

Load-bearing premise

The whole reward signal rests on the assumption that the distance from a tracked object's bounding box to VLM-generated satisfied and unsatisfied position sets is a valid likelihood for whether a subgoal is complete; if that observation model is wrong, the particle-filter updates and the rewards built on them are not statistically grounded.

Editorial extensions

If this is right

  • Policies trained with T2-VLM rewards reach goal completion ratios comparable to policies trained on hand-coded environment rewards on the CLIPort and CALVIN tasks tested, while outperforming VLM-CaR and RL-VLM-F.
  • Across Claude-3.5-Sonnet, Qwen2-VL-72B, and Qwen2-VL-7B, T2-VLM raises reward accuracy and cuts VLM query counts and inference time relative to per-step VLM scoring.
  • The temporal consistency of the reward signal transfers to failure recovery: RL policies trained with T2-VLM recover from occupied-space, closed-drawer, dropped-object, and placement failures more often and in fewer meta-steps than SayCan and REFLECT.
  • The method's accuracy depends on the VLM's initialization quality; tasks with spatial relations the VLM misreads, such as 'next to,' still improve with tracking but leave room for better initial subgoal estimation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the paper leaves implicit: the same one-query-plus-tracking recipe should generalize to any source of affordance sets — a detector, a keypoint function, or human annotations — so the reward-generation mechanism is not tied to VLMs specifically.
  • A testable extension is to feed the tracker's uncertainty back into the RL agent, using high particle spread as a signal to trigger recovery skills before the reward drops.
  • Because the reward is a smoothed difference of an estimated latent state, the same machinery could serve as a dense success detector for real-world long-horizon manipulation, where per-step ground-truth labels are unavailable.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes T2-VLM, a training-free framework that generates rewards for robot manipulation by combining VLM-based subgoal decomposition with a particle filter that tracks subgoal completion status over time. At the start of an episode, a VLM outputs subgoals and an initial binary completion vector; thereafter, SAM2 extracts object bounding-box trajectories and VLM-coded affordance functions are used to define an observation likelihood. A particle filter estimates the hidden completion state, and rewards are computed from changes in this state between decision intervals. The authors report RL training results on CLIPort and CALVIN, reward-accuracy comparisons against per-step VLM scoring, ablation studies, and real-world reward-accuracy experiments.

Significance. If the method worked as described, it would be a useful contribution: avoiding per-step VLM queries through a lightweight temporal filter is a sensible direction, and the paper includes a broad evaluation across two simulators and a real robot, an open-source implementation, and explicit efficiency comparisons. The central claim that T2-VLM produces rewards accurate enough for RL training, however, is not supported by the written algorithm. The particle filter as presented cannot update the completion state from its initial value, so the reported reward signals and the resulting RL performance cannot follow from the described method. The paper also contains an inverted observation-likelihood formula and several reporting gaps (missing variance, a contradicted 'single query' claim, and no real-world RL training). Because the core mechanism needs to be redesigned and re-validated, the manuscript requires major revision.

major comments (4)
  1. [Sec. 3.2, Eq. (6)] The particle filter as written cannot detect subgoal completion changes. All K particles are initialized with the VLM vector h0, and Algorithm 1 contains no propagation step: it only computes weights (Eq. 6), normalizes, estimates h-hat_t, and resamples. Even if Eq. (4) were applied, the motion model with alpha=0.7 and beta=0.04 has mean h_{t-1}+alpha(h_{t-1}-h_{t-2}), which is zero for a subgoal that starts at 0 and has zero initial velocity, and the per-step noise std of 0.04 cannot move particles to 1 within an episode. Consequently h-hat_t remains approximately h0 for all t, and r_t = sigma(h-hat_t - h-hat_{t-tau}) is approximately zero. This invalidates the central claim in Sec. 4.1 that 'rewards generated by T2-VLM effectively support RL', and it also undermines the reward-accuracy numbers in Fig. 6 and Tables 2-3. Please provide a corrected algorithm with a transition model that allows the hidden state to explore both completion states, and show a concrete trajectory of h-hat_t (e.g., the drawer-open subgoal in Cleanup-desk) demonstrating that a subgoal initialized at 0 moves to near 1 when it is completed.
  2. [Sec. 3.2, Eq. (6)] The observation-likelihood formula is inverted relative to its prose description. For a particle with h_i=1 (subgoal i complete), Eq. (6) multiplies the weight by d0_i, the distance to the unsatisfied affordance, so the particle receives high weight when the observation is far from the satisfied affordance; for h_i=0 it uses d1_i, the distance to the satisfied affordance. This is the opposite of the stated goal of assigning 'higher weights to particles whose subgoal states match the observed positions.' Additionally, the raw distances in Eq. (5) are not normalized or otherwise converted into a probability p(I_t | h_t), so the particle weights lack a statistical interpretation. Please re-derive the likelihood so that high weight is assigned to the correct hypothesis and the weights are valid probabilities.
  3. [Sec. 4.4 and Appendix B] The claim that T2-VLM requires only a 'single query per episode' (Sec. 3, Fig. 1) is contradicted by the appendix's N-step re-check procedure, which issues a second Set-of-Mark prompt during random exploration, and by Table 2, which reports 200 VLM queries for T2-VLM. If the query count is per episode or per evaluation run, please state the exact protocol and reconcile it with the 'single query' wording. This matters because the efficiency advantage is one of the paper's two primary claims.
  4. [Sec. 4.2 and Sec. 4.4] The real-world experiments in Table 4 report reward accuracy and inference time only; they do not include RL training or policy evaluation in the real world. The abstract and Sec. 1 present the method as generating rewards that 'support RL', but that evidence is limited to the simulated CLIPort and CALVIN tasks. Please either add real-world RL training results or restrict the claims to reward accuracy for real-world settings. Relatedly, Fig. 5 shows only mean curves over three seeds with no error bars or individual runs, and Tables 1-4 report no variance; given the RL-comparison claim, at least standard deviations or seed-wise results should be reported.
minor comments (4)
  1. [Sec. 3.2, Eq. (4)] The expression for the mean in Eq. (4) appears to contain a typo: mu = h_{t-1} + alpha(h_{t-1}, h_{t-2}) is not a well-formed vector expression; it should presumably be h_{t-1} + alpha(h_{t-1} - h_{t-2}). Please correct the notation.
  2. [Algorithm 1] Algorithm 1 has two lines numbered '12' and no explicit propagation line. Please renumber the lines and add the missing propagation step, or state explicitly that no propagation is performed and justify that choice.
  3. [Sec. 3.2] The notation in Eq. (5) is under-specified: the norm is not defined, and the symbol I is used both for the bounding-box observation and for the identity matrix in Eq. (4). Please define all symbols and norms.
  4. [Sec. 3.3] The text says sigma depends on the decomposed subgoals, but Algorithm 1 defines sigma as a weighted sum of absolute differences between subgoal hidden states. Please make the definition of sigma consistent between the text, the algorithm, and the implementation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: T2-VLM's reward signal is benchmarked against ground-truth simulator states, and the VLM-derived components are inputs rather than fitted parameters.

full rationale

The paper's derivation chain is not circular. The VLM supplies h0 (the initial subgoal completion estimate) and the affordance functions f^1_VLM and f^0_VLM, but the claimed prediction is the time-evolving completion status h_t, which is updated from SAM2 bounding-box trajectories through Eqs. (4)-(6). The evaluation of reward accuracy and goal completion ratio is performed against ground-truth environment states from CLIPort/CALVIN and real-world tasks, not against the VLM's own outputs. The method's hyperparameters (K=100, alpha=0.7, beta=0.04) are taken from the external particle-filter reference [43], and no parameter is fitted to the benchmark. The per-step VLM-score baseline is a controlled comparison showing that T2-VLM's temporal tracking adds signal beyond a single VLM query. Self-citations appear only in contextual related-work sentences (e.g., [40], [32], [34]) and are not load-bearing for the central reward-generation claim. The internal-consistency concern that Algorithm 1 omits the propagation step of Eq. (4) is a correctness issue, not a circular reduction: even if the pseudocode is flawed, the described claim does not define the output as equal to its input. No circular step can be exhibited from the text.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The method's central machinery is a particle filter whose motion and observation models rest on hand-set hyperparameters and on the assumption that VLM-generated affordance code is a reliable spatial likelihood. No new physical entities are introduced; the subgoal hidden state is a latent representation, not an entity with independent falsifiable handles.

free parameters (5)
  • alpha = 0.7
    Soft update hyperparameter in the constant-velocity motion model (Eq. 4), adopted from [43] rather than fitted to new data; controls how fast hidden states evolve and affects robustness to SAM2 mis-detections.
  • beta = 0.04
    Covariance scale in the motion model, set to beta=0.04 following [43]; influences filter smoothness.
  • particle_count = 100
    Number of particles in the particle filter, set in Sec. 3.3; a standard hyperparameter.
  • PF_hidden_length = 25
    Number of temporal frames used for reward generation; Fig. 7 shows 25 gives the best accuracy/latency trade-off, but it is chosen by the authors on a validation task and the paper does not state whether it was fixed before the other tasks.
  • decision_interval_tau = not specified
    Interval between reward computations; no concrete value is given in the main text, which limits reproducibility.
assumptions (5)
  • domain assumption Subgoal completion hidden states evolve smoothly over time, modeled by a constant-velocity Gaussian motion model (Eq. 4).
    The text states 'Assuming that the latent hidden state changes smoothly with consistent robot manipulation' (Sec. 3.2). Binary completion states do not naturally obey constant-velocity dynamics, so this is a modeling assumption.
  • ad hoc to paper The minimum distance between observed bounding boxes and VLM-generated affordance point sets is a valid observation likelihood for subgoal completion (Eqs. 5-6).
    This is the core observation model, but it is introduced without a derivation and is not a normalized probability; the sign structure in Eq. (6) is also inconsistent with the prose.
  • domain assumption SAM 2 bounding box trajectories are accurate enough to support reward estimation.
    The particle filter's observations are entirely SAM2 trajectories; occlusions are mentioned as a motivation, but no failure analysis is provided.
  • domain assumption The initial image captures all task-relevant objects.
    Sec. 3.1 states 'Assuming the initial image captures all task-relevant objects'; the method adds a re-check later, but the assumption underlies the initial subgoal set.
  • domain assumption VLMs can decompose manipulation tasks into correct spatial subgoals zero-shot.
    Subgoal generation is delegated to the VLM with hand-written prompts; the paper notes that VLMs can produce false or incomplete subgoals (Fig. 8), so correctness is not guaranteed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Training-free Generation of Temporally Consistent Rewards from VLMs." pith.science (2026). https://pith.science/paper/BBKCIXRD

@misc{pith2026250704789,
  author       = {Pith},
  title        = {Pith review of: Training-free Generation of Temporally Consistent Rewards from VLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BBKCIXRD}},
  note         = {Machine review of arXiv:2507.04789}
}
abstract

Recent advances in vision-language models (VLMs) have significantly improved performance in embodied tasks such as goal decomposition and visual comprehension. However, providing accurate rewards for robotic manipulation without fine-tuning VLMs remains challenging due to the absence of domain-specific robotic knowledge in pre-trained datasets and high computational costs that hinder real-time applicability. To address this, we propose $\mathrm{T}^2$-VLM, a novel training-free, temporally consistent framework that generates accurate rewards through tracking the status changes in VLM-derived subgoals. Specifically, our method first queries the VLM to establish spatially aware subgoals and an initial completion estimate before each round of interaction. We then employ a Bayesian tracking algorithm to update the goal completion status dynamically, using subgoal hidden states to generate structured rewards for reinforcement learning (RL) agents. This approach enhances long-horizon decision-making and improves failure recovery capabilities with RL. Extensive experiments indicate that $\mathrm{T}^2$-VLM achieves state-of-the-art performance in two robot manipulation benchmarks, demonstrating superior reward accuracy with reduced computation consumption. We believe our approach not only advances reward generation techniques but also contributes to the broader field of embodied AI. Project website: https://t2-vlm.github.io/.

Figures

Figures reproduced from arXiv: 2507.04789 by the authors.

Figure 1
Figure 1. Illustrative example of current VLM-based re￾ward generation methods and T 2 -VLM. o represents high￾dimensional observations, such as images. τ denotes the robot’s decision intervals. h and hˆ indicate the goal completion status given by VLMs and Bayesian filters, respectively. r is the final reward calculated based on changes in h. T 2 -VLM only requires querying the VLMs once at the beginning of an episode and ef… view at source ↗
Figure 2
Figure 2. An overview of T 2 -VLM . First, the VLM decomposes the subgoals and provides an initial estimate of the goal completion status based on an image and language task description. Then, we introduce a Bayesian tracking algorithm to update the goal completion status using the temporal data processed by SAM 2. Finally, the model ensures temporal consistency while accurately generating rewards by tracking the completion s… view at source ↗
Figure 3
Figure 3. VLM-coding affordance in observation models. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Experiments across three domains with six distinct tasks. In CLIPort [29], tasks are “Place-same-color” where cubes are sorted into matching color bowls, and “Stack-tower” where cubes are stacked in a predefined color order. In CALVIN [21], tasks are “Make-line” where …
Figure 5
Figure 5. Figure 5: Goal completion ratio during RL training under different VLM-based rewards. All methods are running with three seeds. stable training performance, contributing to its precise re￾wards. In contrast, RL-VLM-F shows poor performance on CLIPort and higher variance on CALVI…
Figure 6
Figure 6. Figure 6: Ablation studies on VLM initialization and particle filters (PF). We compare the reward accuracy of T 2 -VLM under different initialization methods and without particle. of T 2 -VLM also depends on task difficulty. In Place-same￾color, where VLMs easily identify subgoa…
Figure 8
Figure 8. Figure 8: Re-check VLM initialization with Set-of-Mark (SoM) prompting in a long-horizon real-world task. CNN + + + + + + + + + + Self-A'en)on Critic ! Recover Actor " // ac0on Visual embeddings Gripper states Bounding boxes // Stop gradients Failure Actor "# OR ac0on // Target …
Figure 9
Figure 9. Figure 9: The network architecture for RL agent. GREEN], [PLACE2LEFT, GREEN], [PLACE2RIGHT, GREEN]] Primitive Skill Parameterization. The primitive skill considered in simulators is parameterized as SKILL[p], where p=[x, y, z] specifies the 3D coordinates of the tar￾get position…
Figure 10
Figure 10. Figure 10: Prompts for chain-of-thought object detection [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Prompts for sub-goals generation in the setup stage [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 12
Figure 12. Figure 12: Prompts for goal completion status queried VLMs [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]
Figure 13
Figure 13. Figure 13: An example of goal completion status identification [PITH_FULL_IMAGE:figures/full_fig_p017_13.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

55 extracted references · 39 canonical work pages

  1. [1]

    Do as i can, not as i say: Grounding language in robotic affordances

    Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Cheb- otar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, et al. Do as i can, not as i say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.01691, 2022. 6, 8, 12

    arXiv 2022

  2. [2]

    Vision-language models as a source of rewards

    Kate Baumli, Satinder Singh, Feryal Behbahani, Harris Chan, Gheorghe Comanici, Sebastian Flennerhag, Maxime Gazeau, Kristian Holsheimer, Dan Horgan, Michael Laskin, et al. Vision-language models as a source of rewards. InSec- ond Agent Learning in Open-Endedness Workshop, 2023. 8

    work page 2023

  3. [3]

    Spatialvlm: Endow- ing vision-language models with spatial reasoning capabili- ties

    Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. Spatialvlm: Endow- ing vision-language models with spatial reasoning capabili- ties. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 14455–14465,

  4. [4]

    Towards a unified agent with foundation models

    Norman Di Palo, Arunkumar Byravan, Leonard Hasenclever, Markus Wulfmeier, Nicolas Heess, and Martin Riedmiller. Towards a unified agent with foundation models. In Work- shop on Reincarnating Reinforcement Learning at ICLR 2023, 2023. 8

    work page 2023

  5. [5]

    Video language planning

    Yilun Du, Sherry Yang, Pete Florence, Fei Xia, Ayzaan Wahid, Pierre Sermanet, Tianhe Yu, Pieter Abbeel, Joshua B Tenenbaum, Leslie Pack Kaelbling, et al. Video language planning. In International Conference on Learning Repre- sentations, 2024. 1

    work page 2024

  6. [6]

    Manipulate- anything: Automating real-world robots using vision- language models

    Jiafei Duan, Wentao Yuan, Wilbert Pumacay, Yi Ru Wang, Kiana Ehsani, Dieter Fox, and Ranjay Krishna. Manipulate- anything: Automating real-world robots using vision- language models. In 8th Annual Conference on Robot Learn- ing, 2024. 8

    work page 2024

  7. [7]

    Phys- ically grounded vision-language models for robotic manip- ulation

    Jensen Gao, Bidipta Sarkar, Fei Xia, Ted Xiao, Jiajun Wu, Brian Ichter, Anirudha Majumdar, and Dorsa Sadigh. Phys- ically grounded vision-language models for robotic manip- ulation. In IEEE International Conference on Robotics and Automation, pages 12462–12469, 2024. 1

    work page 2024

  8. [8]

    Doremi: Grounding language model by detecting and recov- ering from plan-execution misalignment

    Yanjiang Guo, Yen-Jen Wang, Lihan Zha, and Jianyu Chen. Doremi: Grounding language model by detecting and recov- ering from plan-execution misalignment. In IEEE/RSJ In- ternational Conference on Intelligent Robots and Systems , pages 12124–12131, 2024. 1, 8

    work page 2024

Show all 55 references
  1. [9]

    Mixgen: A new multi- modal data augmentation

    Xiaoshuai Hao, Yi Zhu, Srikar Appalaraju, Aston Zhang, Wanqian Zhang, Bo Li, and Mu Li. Mixgen: A new multi- modal data augmentation. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 379–389, 2023. 1

    2023

  2. [10]

    V oxposer: Composable 3d value maps for robotic manipulation with language models

    Wenlong Huang, Chen Wang, Ruohan Zhang, Yunzhu Li, Ji- ajun Wu, and Li Fei-Fei. V oxposer: Composable 3d value maps for robotic manipulation with language models. In Conference on Robot Learning, pages 540–562, 2023. 8

    2023

  3. [11]

    Robobrain: A unified brain model for robotic manipulation from abstract to concrete

    Yuheng Ji, Huajie Tan, Jiayu Shi, Xiaoshuai Hao, Yuan Zhang, Hengyuan Zhang, Pengwei Wang, Mengdi Zhao, Yao Mu, Pengju An, et al. Robobrain: A unified brain model for robotic manipulation from abstract to concrete. In Proceed- ings of the Computer Vision and Pattern Recognitio...

    2025

  4. [12]

    Scaling up visual and vision-language representa- tion learning with noisy text supervision

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representa- tion learning with noisy text supervision. In International Conference on Machine Learning, pages 4904–491...

    2021

  5. [13]

    De- composed prompting: A modular approach for solving com- plex tasks

    Tushar Khot, Harsh Trivedi, Matthew Finlayson, Yao Fu, Kyle Richardson, Peter Clark, and Ashish Sabharwal. De- composed prompting: A modular approach for solving com- plex tasks. In The Eleventh International Conference on Learning Representations, 2023. 2

    2023

  6. [14]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4015–4026, 2023. 1

    2023

  7. [15]

    Multimodal sensor fusion with differentiable filters

    Michelle A Lee, Brent Yi, Roberto Mart ´ın-Mart´ın, Silvio Savarese, and Jeannette Bohg. Multimodal sensor fusion with differentiable filters. InIEEE/RSJ International Confer- ence on Intelligent Robots and Systems, pages 10444–10451,

  8. [16]

    What foundation models can bring for robot learning in manipulation: A survey

    Dingzhe Li, Yixiang Jin, Yuhao Sun, Hongze Yu, Jun Shi, Xiaoshuai Hao, Peng Hao, Huaping Liu, Fuchun Sun, Jian- wei Zhang, et al. What foundation models can bring for robot learning in manipulation: A survey. arXiv preprint arXiv:2404.18201, 2024. 1

    2024

  9. [17]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In In- ternational Conference on Machine Learning, pages 19730– 19742, 2023. 1

    2023

  10. [18]

    Code as policies: Language model programs for embodied con- trol

    Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code as policies: Language model programs for embodied con- trol. In International Conference on Robotics and Automa- tion, pages 9493–9500, 2023. 8

    2023

  11. [19]

    Reflect: Summa- rizing robot experiences for failure explanation and correc- tion

    Zeyi Liu, Arpit Bahety, and Shuran Song. Reflect: Summa- rizing robot experiences for failure explanation and correc- tion. In Conference on Robot Learning , pages 3468–3484,

  12. [20]

    Explorllm: Guiding exploration in reinforce- ment learning with large language models

    Runyu Ma, Jelle Luijkx, Zlatan Ajanovic, and Jens Kober. Explorllm: Guiding exploration in reinforce- ment learning with large language models. arXiv preprint arXiv:2403.09583, 2024. 5, 12

    2024 arXiv

  13. [21]

    Calvin: A benchmark for language- conditioned policy learning for long-horizon robot manip- ulation tasks

    Oier Mees, Lukas Hermann, Erick Rosete-Beas, and Wol- fram Burgard. Calvin: A benchmark for language- conditioned policy learning for long-horizon robot manip- ulation tasks. IEEE Robotics and Automation Letters , 7(3): 7327–7334, 2022. 5

    2022

  14. [22]

    The deep latent space particle filter for real-time data assimilation with uncertainty quantification

    Nikolaj T M ¨ucke, Sander M Boht ´e, and Cornelis W Oost- erlee. The deep latent space particle filter for real-time data assimilation with uncertainty quantification. Scientific Re- ports, 14(1):19447, 2024. 2

    2024

  15. [23]

    Pruthvi Patel, Swaroop Mishra, Mihir Parmar, and Chitta Baral. Is a question decomposition unit all we need? In Proceedings of the 2022 Conference on Empirical Meth- ods in Natural Language Processing, pages 4553–4569, Abu Dhabi, United Arab Emirates, 2022. Association for Com...

    2022

  16. [24]

    A real-to-sim-to-real approach to robotic manip- ulation with VLM-generated iterative keypoint rewards

    Shivansh Patel, Xinchen Yin, Wenlong Huang, Shubham Garg, Hooshang Nayyeri, Li Fei-Fei, Svetlana Lazebnik, and Yunzhu Li. A real-to-sim-to-real approach to robotic manip- ulation with VLM-generated iterative keypoint rewards. In 2nd CoRL Workshop on Learning Effective Abstract...

    2024

  17. [25]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In International Conference on Machine Learning...

    2021

  18. [26]

    Sam 2: Segment anything in images and videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman R¨adle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714, 2024. 1, 2, 3

    2024 arXiv

  19. [27]

    Vision-language models are zero- shot reward models for reinforcement learning

    Juan Rocamonde, Victoriano Montesinos, Elvis Nava, Ethan Perez, and David Lindner. Vision-language models are zero- shot reward models for reinforcement learning. In Interna- tional Conference on Learning Representations, 2024. 8

    2024

  20. [28]

    Chinchali

    S P Sharan, Ruihan Zhao, ufuk topcu, Zhangyang Wang, and Sandeep P. Chinchali. Plan diffuser: Grounding LLM planners with diffusion models for robotic manipulation. In Bridging the Gap between Cognitive Science and Robot Learning in the Real World: Progresses and New Directions,

  21. [29]

    Cliport: What and where pathways for robotic manipulation

    Mohit Shridhar, Lucas Manuelli, and Dieter Fox. Cliport: What and where pathways for robotic manipulation. In Con- ference on robot learning, pages 894–906, 2022. 5

    2022

  22. [30]

    Perceiver- actor: A multi-task transformer for robotic manipulation

    Mohit Shridhar, Lucas Manuelli, and Dieter Fox. Perceiver- actor: A multi-task transformer for robotic manipulation. In Conference on Robot Learning, pages 785–799, 2023. 8

    2023

  23. [31]

    Progprompt: Generating situ- ated robot task plans using large language models

    Ishika Singh, Valts Blukis, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, Jesse Thoma- son, and Animesh Garg. Progprompt: Generating situ- ated robot task plans using large language models. In In- ternational Conference on Robotics and Automation , ...

    2023

  24. [32]

    Reason-rft: Reinforcement fine-tuning for visual reasoning

    Huajie Tan, Yuheng Ji, Xiaoshuai Hao, Minglan Lin, Pengwei Wang, Zhongyuan Wang, and Shanghang Zhang. Reason-rft: Reinforcement fine-tuning for visual reasoning. arXiv preprint arXiv:2503.20752, 2025. 1

    2025

  25. [33]

    Cotdet: Affordance knowledge prompting for task driven object de- tection

    Jiajin Tang, Ge Zheng, Jingyi Yu, and Sibei Yang. Cotdet: Affordance knowledge prompting for task driven object de- tection. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 3068–3078, 2023. 12

    2023

  26. [34]

    Affordgrasp: In-context affordance reasoning for open-vocabulary task-oriented grasping in clutter

    Yingbo Tang, Shuaike Zhang, Xiaoshuai Hao, Pengwei Wang, Jianlong Wu, Zhongyuan Wang, and Shanghang Zhang. Affordgrasp: In-context affordance reasoning for open-vocabulary task-oriented grasping in clutter. arXiv preprint arXiv:2503.00778, 2025. 8

    2025 arXiv

  27. [35]

    Real-world offline reinforce- ment learning from vision language model feedback

    Sreyas Venkataraman, Yufei Wang, Ziyu Wang, Zackory Erickson, and David Held. Real-world offline reinforce- ment learning from vision language model feedback. arXiv preprint arXiv:2411.05273, 2024. 8

    2024 arXiv

  28. [36]

    Code as reward: Empowering reinforcement learning with vlms

    David Venuto, Mohammad Sami Nur Islam, Martin Klis- sarov, Doina Precup, Sherry Yang, and Ankit Anand. Code as reward: Empowering reinforcement learning with vlms. In International Conference on Machine Learning, 2024. 1, 5, 8

    2024

  29. [37]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191, 2024. 1, 4

    2024 arXiv

  30. [38]

    Rl-vlm-f: Rein- forcement learning from vision language foundation model feedback

    Yufei Wang, Zhanyi Sun, Jesse Zhang, Zhou Xian, Erdem Biyik, David Held, and Zackory Erickson. Rl-vlm-f: Rein- forcement learning from vision language foundation model feedback. In International Conference on Machine Learn- ing, 2024. 1, 5, 8

    2024

  31. [39]

    Any-point trajectory modeling for policy learning

    Chuan Wen, Xingyu Lin, John So, Kai Chen, Qi Dou, Yang Gao, and Pieter Abbeel. Any-point trajectory modeling for policy learning. arXiv preprint arXiv:2401.00025, 2023. 1

    2023 arXiv

  32. [40]

    Robomind: Benchmark on multi- embodiment intelligence normative data for robot manipula- tion

    Kun Wu, Chengkai Hou, Jiaming Liu, Zhengping Che, Xi- aozhu Ju, Zhuqin Yang, Meng Li, Yinuo Zhao, Zhiyuan Xu, Guang Yang, et al. Robomind: Benchmark on multi- embodiment intelligence normative data for robot manipula- tion. arXiv preprint arXiv:2412.13877, 2024. 1

    2024 arXiv

  33. [41]

    Set-of-mark prompting unleashes extraordinary visual grounding in gpt-4v

    Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao. Set-of-mark prompting unleashes extraordinary visual grounding in gpt-4v. arXiv preprint arXiv:2310.11441, 2023. 12

    2023 arXiv

  34. [42]

    Robot fine- tuning made easy: Pre-training rewards and policies for au- tonomous real-world reinforcement learning

    Jingyun Yang, Max Sobol Mark, Brandon Vu, Archit Sharma, Jeannette Bohg, and Chelsea Finn. Robot fine- tuning made easy: Pre-training rewards and policies for au- tonomous real-world reinforcement learning. In Interna- tional Conference on Robotics and Automation, pages 4804– ...

    2024

  35. [43]

    Par- ticle filters in latent space for robust deformable linear ob- ject tracking

    Yuxuan Yang, Johannes A Stork, and Todor Stoyanov. Par- ticle filters in latent space for robust deformable linear ob- ject tracking. IEEE Robotics and Automation Letters, pages 12577–12584, 2022. 3, 4

    2022

  36. [44]

    Sornet: Spatial object-centric representations for se- quential manipulation

    Wentao Yuan, Chris Paxton, Karthik Desingh, and Dieter Fox. Sornet: Spatial object-centric representations for se- quential manipulation. In Conference on Robot Learning , pages 148–157, 2022. 8

    2022

  37. [45]

    Robopoint: A vision-language model for spatial affordance prediction in robotics

    Wentao Yuan, Jiafei Duan, Valts Blukis, Wilbert Pumacay, Ranjay Krishna, Adithyavairavan Murali, Arsalan Mousa- vian, and Dieter Fox. Robopoint: A vision-language model for spatial affordance prediction in robotics. In 8th Annual Conference on Robot Learning, 2024. 8

    2024

  38. [46]

    Sam-e: Leveraging visual foundation model with sequence imitation for embodied ma- nipulation

    Junjie Zhang, Chenjia Bai, Haoran He, Zhigang Wang, Bin Zhao, Xiu Li, and Xuelong Li. Sam-e: Leveraging visual foundation model with sequence imitation for embodied ma- nipulation. In International Conference on Machine Learn- ing, 2024. 1

    2024

  39. [47]

    Mapnav: A novel memory representation via annotated semantic maps for vlm-based vision-and-language navigation

    Lingfeng Zhang, Xiaoshuai Hao, Qinwen Xu, Qiang Zhang, Xinyao Zhang, Pengwei Wang, Jing Zhang, Zhongyuan Wang, Shanghang Zhang, and Renjing Xu. Mapnav: A novel memory representation via annotated semantic maps for vlm-based vision-and-language navigation. arXiv preprint arXiv:...

    2025 arXiv

  40. [48]

    // ac0on Visual embeddingsGripper statesBounding boxes//Stop gradientsFailure Actor #

    Ruijie Zheng, Yongyuan Liang, Shuaiyi Huang, Jianfeng Gao, Hal Daum ´e III, Andrey Kolobov, Furong Huang, and Jianwei Yang. Tracevla: Visual trace prompting enhances spatial-temporal awareness for generalist robotic policies. arXiv preprint arXiv:2412.10345, 2024. 1 This suppl...

    2024 arXiv

  41. [49]

    The red cube is on top of the black cube. Now, decompose the following task using the provided visual information: Task: Place the cube from the gray tray into the matching colored bowl; cubes without a matching bowl should stay in place. Relevant Objects: a red cube, a green ...

  42. [52]

    The blue cube is on top of the gray tray

  43. [53]

    Figure 11

    The yellow cube is on top of the gray tray. Figure 11. Prompts for sub-goals generation in the setup stage. Prompts for identifying the completion status of each subgoal System Prompt: You assist in controlling a high-intelligence robotic arm that determine the spatial relatio...

  44. [54]

    The blue cube remains on the gray tray

  45. [55]

    The red cube is inside the red bowl

  46. [56]

    The green cube is inside the green bowl

  47. [57]

    Completion Status Identification of Sub-goals: System prompt: Detailed in Fig

    The yellow cube remains on the gray tray. Completion Status Identification of Sub-goals: System prompt: Detailed in Fig. 12. Task description: Place the cube from the gray tray into the matching colored bowl; cubes without a matching bowl should stay in place. Case 1. Sub-goal...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.