Pith. sign in

REVIEW 4 major objections 7 minor 1 cited by

GenFlowRL: Shaping Rewards with Generative Object-Centric Flow in Visual Reinforcement Learning

T0 review · 4 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read GenFlowRL turns generated object flow into dense rewards for robot learning, outperforming flow imitation and video-based RL on 10 manipulation tasks.

desk verdict The core idea is plausible and the simulation results are coherent, but the real-world 'cross-embodiment evaluations' in the abstract are not supported by what Section 4.5 actually does. read the letter →

arxiv 2508.11049 v1 pith:2IUKG4WS submitted 2025-08-14 cs.RO cs.CV

Kelin Yu , Sheng Zhang , Harshit Soora , Furong Huang , Heng Huang , Pratap Tokekar , Ruohan Gao This is my paper · ORCID
classification cs.ROcs.CV
keywords object-centricflowrewardshapingvisualreinforcementlearningrobotmanipulationcross-embodimenttransferdiffusionmodelsdelta-flowdeformableobjects
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a robot can learn dexterous manipulation policies by using generated object-centric flow, the tracked 2D motion of keypoints on an object, as a dense reward signal for reinforcement learning. The authors argue that this low-dimensional representation avoids the artifacts and cost of video-based reward models and bridges different robot embodiments. On ten simulation tasks spanning deformable, articulated, and contact-rich manipulation, GenFlowRL is reported to outperform flow-based imitation learning and video-guided RL baselines, and a real-world case study reports that rewards derived from human-hand demonstrations match robot rollouts. The practical stake is that easy-to-collect cross-embodiment demonstrations could replace large task-specific robot datasets for training reliable closed-loop policies.

What carries the argument

The load-bearing object is delta-flow, a per-timestep summary of object keypoint motion as centroid position, average translation, and average rotation relative to the first frame. It carries the argument by converting noisy generated 2D keypoint trajectories into a low-dimensional, embodiment-agnostic motion prior that can be matched online against the robot's observed flow. The hybrid reward model combines this dense matching reward with sparse state-aware subgoal rewards, and the policy consumes k-step lookahead generated delta-flow plus initial 3D centroids as conditions.

What would settle it

Run GenFlowRL on a task whose success requires a large out-of-plane rotation of the object, such as twisting a lid off a bottle, where 2D keypoint flow cannot represent the motion. If success rates fall to the level of the pure sparse reward baseline while a 3D-flow policy succeeds, the central claim that generated 2D flow is a sufficient reward prior fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that object-centric flow generated by a fine-tuned diffusion model can be condensed into a delta-flow representation, per-step centroid, average translation, and average rotation of tracked keypoints, and used as a dense reward by matching the robot's observed delta-flow to the generated delta-flow. Under a Gaussian tied-variance assumption, the distributional alignment reduces to squared-error matching, and the hybrid reward adds sparse state-aware subgoal rewards. The paper reports that this reward model yields faster convergence and higher success rates than flow-based imitation learning and video-based reward baselines, particularly on contact-rich and deformable-object tasks, and that the same flow-derived reward transfers from human hand videos to a real robot arm.

Load-bearing premise

The load-bearing premise is that generated object-centric delta-flow is a reliable and sufficient motion prior: if the diffusion model's flow is noisy, embodiment-biased, or misses out-of-plane rotations, the dense reward will guide the policy away from good behavior.

Editorial extensions

If this is right

  • On the five simulation tasks used for the main comparison, language-conditioned success rates rise substantially over the flow-based imitation baseline, for example from 35 to 80 on the folding task and from 45 to 85 on the pivoting task.
  • The dense delta-flow reward accelerates training and improves final performance over sparse reward, novelty-seeking exploration, and video-prediction-based reward methods, especially on assembly-style tasks.
  • Delta-flow enables reward shaping for deformable and articulated objects where 6D pose and 3D keypoint representations are unsuitable.
  • The real-world case study indicates that human hand demonstrations can provide matching flow rewards for a robot arm, suggesting cross-embodiment transfer without robot data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Extending delta-flow to full 3D flow could handle out-of-plane rotations, a limitation the paper itself names, and would likely broaden the method to tasks such as twisting or unscrewing.
  • Because the reward model is task-agnostic and the flow generator is trained once on 12K trajectories, the approach may amortize across new tasks and embodiments with only sparse goal definitions, though the paper evaluates a fixed task set.
  • The dense reward relies on online keypoint tracking; deployment robustness may depend on tracking quality under occlusion and lighting, which the paper does not stress-test beyond simulated noise.
  • The policy consumes future generated delta-flow as input, so one testable extension is to remove that future-flow condition at inference and see whether the reward alone can reconstruct the motion, isolating the contribution of the prior.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes GenFlowRL, a reinforcement learning framework that replaces video-based reward shaping with dense rewards derived from a generative object-centric 'delta-flow' prior. A diffusion-based flow generator is fine-tuned on cross-embodiment trajectories, the generated 2D keypoint flows are condensed into a delta-flow representation of centroid translation and average rotation, and a hybrid reward combines this flow-matching term with sparse environment feedback. The policy is trained with DrQv2 and conditioned on the current robot state, observed flow, future generated flow, and the initial 3D keypoint centroid. Experiments cover five Im2Flow2Act-style simulation tasks, five MetaWorld tasks, representation and noise ablations, and a real-world case study with human hand videos and a robot arm.

Significance. The central idea is plausible and potentially valuable: using generated object-centric flow as a cross-embodiment motion prior for reward shaping is lower-dimensional and more robust than video-based reward shaping, and the paper includes a useful noise-robustness analysis in Table 4 that directly probes the reliability of the flow prior. The delta-flow condensation is a sensible denoising step, and the comparison against VIPER and Diffusion Reward in Fig. 4 addresses the most relevant video-based reward-shaping line of work. However, the real-world study does not evaluate a learned policy, and the closest flow-based reward-shaping baseline, HuDor, is omitted; these gaps limit the strength of the central claims. The Gaussian tied-variance derivation in Sec. 3.2 is internally sound and does not constitute circular reasoning.

major comments (4)
  1. [§4.5 / Abstract] The headline real-world claim is not supported by the reported experiment. Section 4.5 states that the authors 'roll out open-loop robot trajectories aligned with each expert motion' and then compute the flow-matching reward between human and robot trajectories; no learned policy is executed, no task success rate is reported, and no baseline comparison is provided. Figure 7 therefore demonstrates only that the proposed reward signal increases along hand-aligned open-loop trajectories, and it does so using flow extracted from human videos rather than generated flow. The abstract's phrase 'real-world cross-embodiment evaluations' and the claim of 'consistently achieving superior performance' overstate this evidence. Either a closed-loop real-robot policy evaluation with success rates and baselines should be added, or the real-world claims should be explicitly limited to a reward-matchness feasibility probe.
  2. [§4.1, Table 2] Table 2 reports success rates as single point values with no statement of the number of seeds or variance, unlike Figs. 4–6, which show shaded standard deviations over three seeds. On a benchmark where several entries are separated by only 5–10 percentage points (e.g., Pouring under demonstration-conditioning, 100 vs. 95), it is impossible to judge whether the claimed 'consistent superiority' is statistically meaningful. Please report multiple seeds and error bars or confidence intervals for all main results, and define the success criteria used for each task.
  3. [§2 / §4.1] The closest reward-shaping prior, HuDor [11], is discussed in Related Work but is not included in any comparison. Because HuDor also derives dense shaped rewards from object-centric flow in a cross-embodiment setting, omitting it leaves unresolved whether the gains of GenFlowRL come from the proposed delta-flow and generated prior or are already obtainable with a single expert flow. Please add a HuDor-style baseline, or explain quantitatively why a direct comparison is not feasible.
  4. [§3.2 / §D.2] Equation (5) is advertised as a 'task-agnostic design' of the reward signal, but Appendix D.2 shows that the sparse component requires per-task subgoal and goal definitions: grasping versus contact rewards, task-specific final states, and task-specific success criteria (e.g., the pouring orientation range sampled from 5π/16 to 7π/16). This contradicts the task-agnostic claim. Either the sparse reward should be made genuinely task-agnostic, or the claim should be reworded to describe task-specified environment feedback rather than a task-agnostic design.
minor comments (7)
  1. [§4.1, Table 2] The Language-Conditioned column header appears to be mislabeled: 'PickNP. Pour Fold Fold Pivot' contains a duplicate Fold and omits Opening; please correct the column labels to match the five evaluated tasks.
  2. [§3.2, Eq. (2)] Equation (2) uses a cross product between 2D vectors; please state explicitly that this denotes the scalar 2D cross product.
  3. [§3.2, Eqs. (3)–(4)] The sentence that 'the robot flow mean TR serves as the target for the generated flow mean TG' is confusing, since under the tied-variance Gaussian assumption the KL divergence is symmetric in the two means and neither distribution is a target.
  4. [Appendix C.1] The text refers to 'CIIP [38]' for text embeddings; this appears to be a typo for CLIP.
  5. [§4.5 / Appendix F] The alignment between human demonstrations tracked at 5 Hz and robot rollouts tracked at 2.5 Hz is not described; please specify how the trajectories were temporally aligned before computing the reward.
  6. [§5] The conclusion's caveat that 2D flow may struggle with out-of-plane rotations is important and should be stated earlier in the limitations context, with a discussion of which of the ten tasks are affected.
  7. [Figure 7] The caption contains a typo: 'The shade area' should be 'The shaded area'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: reward shaping is externally validated; the real-world section is a support gap, not circularity.

full rationale

The derivation chain is self-contained. The dense reward in Eq. (4) is defined as a normalized L2 match between the robot's observational delta-flow T_R and the generated delta-flow T_G; this is an imitation-style shaping objective, not a hidden reuse of the evaluation metric. Task success in simulation is measured externally through environment-defined success criteria (Sec. 4.1, MetaWorld and Im2Flow2Act benchmarks), so the main claim is not equivalent to the reward definition by construction. The Gaussian tied-variance reduction from Eq. (3) to Eq. (4) is an explicit simplifying assumption acknowledged in footnote 1 and does not smuggle the conclusion into the premise. The flow generator is trained on 12K cross-embodiment trajectories and then used to produce priors, but the policy's performance is evaluated against external task goals, so the learned prior is an input, not an output masquerading as validation. Self-citations (Refs. [29, 55, 56, 59]) appear only as related work and are not load-bearing; no uniqueness theorem is imported from the authors' prior work. The real-world study (Sec. 4.5) is a 'reward matchness probing' experiment that computes the same flow-matching reward on manually aligned open-loop trajectories; observing a rising reward there is partly self-referential and does not by itself demonstrate closed-loop policy transfer, but this is an evidentiary limitation of the abstract's 'real-world cross-embodiment evaluations' claim, not a circularity in the method's derivation. The admitted 2D-flow limitation for out-of-plane rotations (Conclusion, Sec. 5) is a stated scope constraint, not circular. Overall, no step in the derivation reduces to its own inputs.

Assumptions & free parameters 8 free parameters · 5 assumptions · 1 invented entities

The central claim relies on several hand-tuned reward parameters and domain assumptions: the delta-flow Gaussian approximation, the sufficiency of 2D object flow as a task prior, and the reliability of the tracking and generation pipeline. The paper does not release code or data, and it does not provide sensitivity analysis for the reward constants. These are the main unverified inputs to the method.

free parameters (8)
  • C (reward scaling in delta-flow matching) = not specified
    Controls the clipping and variance in Eq. (4); no value or tuning procedure is reported.
  • alpha (state-aware reward weight) = 0.25
    Set in Sec. 3.2 with no sensitivity analysis.
  • beta (flow reward weight) = 0.75
    Set in Sec. 3.2 with no sensitivity analysis.
  • tau (reaching reward temperature) = 10
    Set in Sec. 3.2 and App. D.2.
  • Subgoal reward magnitude = 0.25
    Grasping and contact rewards are set to 0.25 per task (App. D.2).
  • Moving filter threshold = 50 pixels
    Chosen to remove static keypoints across all tasks (App. C.3).
  • SAM filter area threshold = 10,000
    Chosen to remove keypoints on the robot across all tasks (App. C.3).
  • Number of keypoints = 128
    Sampled from tracked keypoints; ablation shows insensitivity (App. E.2).
assumptions (5)
  • domain assumption Gaussian distributions with tied variance for generated and robot delta-flow
    The derivation of the dense reward as a weighted L2 distance in Eq. (4) relies on this assumption (Sec. 3.2, footnote 1). If the actual trajectory noise is not Gaussian or variances differ, the reward is still a heuristic Euclidean matching.
  • domain assumption Object-centric flow is a sufficient motion prior for the evaluated manipulation tasks
    The framework assumes that 2D keypoint trajectories capture task-relevant dynamics. The paper acknowledges in Sec. 5 that 2D flow may fail for out-of-plane rotations.
  • domain assumption The cross-embodiment dataset (12K trajectories from sphere robot, UR5, and Sawyer) is diverse enough for generalization
    No analysis of dataset coverage or embodiment gap is provided (Sec. 3.1, App. A).
  • domain assumption CoTracker provides reliable online keypoint tracking for reward computation
    Real-world reward matching depends on tracker accuracy; only qualitative evidence is given (App. D.1, E.3).
  • domain assumption Task success can be judged from state-based goal conditions
    The sparse reward requires task-specific goal definitions (App. D.2), undermining the paper's claim that the reward design is task-agnostic.
invented entities (1)
  • delta-flow (condensed object-centric flow representation)
    purpose: Condenses noisy 128-keypoint 2D flows into per-step centroid translation and rotation statistics for reward shaping and policy input.
    The representation is introduced in Sec. 3.2 and its benefits are evidenced only through the paper's own experiments. It is a methodological construct, not a physical entity with external falsifiable predictions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GenFlowRL: Shaping Rewards with Generative Object-Centric Flow in Visual Reinforcement Learning." pith.science (2026). https://pith.science/paper/2IUKG4WS

@misc{pith2026250811049,
  author       = {Pith},
  title        = {Pith review of: GenFlowRL: Shaping Rewards with Generative Object-Centric Flow in Visual Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2IUKG4WS}},
  note         = {Machine review of arXiv:2508.11049}
}
read the original abstract

Recent advances have shown that video generation models can enhance robot learning by deriving effective robot actions through inverse dynamics. However, these methods heavily depend on the quality of generated data and struggle with fine-grained manipulation due to the lack of environment feedback. While video-based reinforcement learning improves policy robustness, it remains constrained by the uncertainty of video generation and the challenges of collecting large-scale robot datasets for training diffusion models. To address these limitations, we propose GenFlowRL, which derives shaped rewards from generated flow trained from diverse cross-embodiment datasets. This enables learning generalizable and robust policies from diverse demonstrations using low-dimensional, object-centric features. Experiments on 10 manipulation tasks, both in simulation and real-world cross-embodiment evaluations, demonstrate that GenFlowRL effectively leverages manipulation features extracted from generated object-centric flow, consistently achieving superior performance across diverse and challenging scenarios. Our Project Page: https://colinyu1.github.io/genflowrl

Figures

Figures reproduced from arXiv: 2508.11049 by the authors.

Figure 1
Figure 1. Illustration of our GENFLOWRL framework, which guides visuomotor RL policy by taking the generative object-centric flow as task motion prior (Right). In our proposed hybrid reward model, dense flow matching between online trajectories and flow prior, synergizing with sparse state-aware reward, facilitates efficient, robust, and generalizable policy learning. Our extensive evaluation includes 10 challenging simulatio… view at source ↗
Figure 2
Figure 2. Architectural overview of our proposed GENFLOWRL framework, which encompasses the flow generation process (left), flow-derived policy learning (middle), and inference stage (right). In the object-centric flow generation process (Sec. 3.1), we: (a) adapt a pre-trained generative model decoder via flow-to-flow reconstruction; (b) fine-tune the latent motion module on flow generation conditioned on task descriptions an… view at source ↗
Figure 3
Figure 3. Overview of all task settings and robot execution demonstrations. Left: tasks in Im2Flow2Act [51] benchmark. Right: tasks from MetaWorld [57] benchmark. Columns denote textual task prompts, generated flows, and robot execution flows. 4. Experiments We are motivated to answer the following key questions: (Q1) Compared to flow-based Imitation Learning (IL), how effective and robust is our method, i.e., RL with our flo… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Comparing with other video-based reward models. The shaded area represents the standard deviation for three random seeds. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Representation Evaluations. The comparison of RL training results with different object-centric representations in three tasks. The shaded area represents the standard deviation for three random seeds [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 8
Figure 8. Figure 8: Visualization of cross-embodiment data. C. Flow Generation Implementation Details In this section, we aim to share more details about imple￾mentation, training, and processing of our flow generation model, which is similar to Im2Flow2Act [51]. C.1. Implementations The …
Figure 10
Figure 10. Figure 10: Visualization of the simulated noised 2D trajectory. [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 9
Figure 9. Figure 9: Visualization of the 2d delta-flow extractions. [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 11
Figure 11. Figure 11: Results of RL performance with different number of [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 12
Figure 12. Figure 12: Visualization of the comparison of trackers. [PITH_FULL_IMAGE:figures/full_fig_p015_12.png]
Figure 13
Figure 13. Figure 13: The qualitative result of the policy rollout in simulatior. [PITH_FULL_IMAGE:figures/full_fig_p016_13.png]
Figure 14
Figure 14. Figure 14: The qualitative result of the Flow Matching Reward Case Study in the real world. [PITH_FULL_IMAGE:figures/full_fig_p017_14.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. OmniTacTune: Policy-Agnostic Real-World RL for Tactile Residual Adaptation of Visual Policies

    cs.RO 2026-07 conditional novelty 6.0 of 10

    A policy-agnostic two-stage real-world RL method learns tactile residual corrections on frozen visual policies, lifting contact-rich task success from 5–40% to 85–100% in under 80 minutes.

Reference graph

Works this paper leans on

60 extracted references · 45 canonical work pages · cited by 1 Pith paper

  1. [11]

    Bridging the human to robot dexterity gap through object-oriented rewards, 2024

    Irmak Guzey, Yinlong Dai, Georgy Savva, Raunaq Bhirangi, and Lerrel Pinto. Bridging the human to robot dexterity gap through object-oriented rewards, 2024. 2, 3, 5, 8

    work page 2024

  2. [1]

    Exploration by random network distillation, 2018

    Yuri Burda, Harrison Edwards, Amos Storkey, and Oleg Klimov. Exploration by random network distillation, 2018. 7

    work page 2018

  3. [2]

    Karen Liu

    Yuanpei Chen, Chen Wang, Yaodong Yang, and C. Karen Liu. Object-centric dexterous manipulation from human mo- tion data, 2024. 3, 7

    work page 2024

  4. [3]

    Diffusion policy: Visuomotor policy learning via action dif- fusion, 2024

    Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action dif- fusion, 2024. 2

    work page 2024

  5. [4]

    Tapir: Tracking any point with per-frame initialization and temporal refinement, 2023

    Carl Doersch, Yi Yang, Mel Vecerik, Dilara Gokay, Ankush Gupta, Yusuf Aytar, Joao Carreira, and Andrew Zisserman. Tapir: Tracking any point with per-frame initialization and temporal refinement, 2023. 12, 15

    work page 2023

  6. [5]

    Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duck- worth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Marc Toussaint, Klaus Greff, Andy Zeng, Igor Mordatch, and Pete Florence. Palm-e: An embodie...

    arXiv 2023

  7. [6]

    Video prediction models as rewards for reinforcement learning

    Alejandro Escontrela, Ademi Adeniji, Wilson Yan, Ajay Jain, Xue Bin Peng, Ken Goldberg, Youngwoon Lee, Dani- jar Hafner, and Pieter Abbeel. Video prediction models as rewards for reinforcement learning. Advances in Neural In- formation Processing Systems, 36, 2024. 2, 3, 5, 7

    work page 2024

  8. [7]

    Tam- ing transformers for high-resolution image synthesis

    Patrick Esser, Robin Rombach, and Bj ¨orn Ommer. Tam- ing transformers for high-resolution image synthesis. CoRR, abs/2012.09841, 2020. 12, 13

    arXiv 2012

Show all 60 references
  1. [8]

    Learning robust re- wards with adversarial inverse reinforcement learning, 2018

    Justin Fu, Katie Luo, and Sergey Levine. Learning robust re- wards with adversarial inverse reinforcement learning, 2018. 2

    2018

  2. [9]

    FLIP: Flow-centric generative planning for general-purpose manipulation tasks

    Chongkai Gao, Haozhuo Zhang, Zhixuan Xu, Cai Zhehao, and Lin Shao. FLIP: Flow-centric generative planning for general-purpose manipulation tasks. In The Thirteenth Inter- national Conference on Learning Representations, 2025. 1, 2, 3

    2025

  3. [10]

    Animatediff: Animate your personalized text-to- image diffusion models without specific tuning, 2024

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to- image diffusion models without specific tuning, 2024. 2, 4, 13

    2024

  4. [12]

    Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor, 2018

    Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor, 2018. 2

    2018

  5. [13]

    Learning prehensile dexterity by imitating and emulating state-only observations, 2024

    Yunhai Han, Zhenyang Chen, Kyle A Williams, and Harish Ravichandar. Learning prehensile dexterity by imitating and emulating state-only observations, 2024. 3, 7

    2024

  6. [14]

    Generative adversarial im- itation learning, 2016

    Jonathan Ho and Stefano Ermon. Generative adversarial im- itation learning, 2016. 2

    2016

  7. [15]

    Video dif- fusion models

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video dif- fusion models. Advances in Neural Information Processing Systems, 35:8633–8646, 2022. 2

    2022

  8. [16]

    Spot: Se(3) pose trajectory diffusion for object-centric manipulation, 2024

    Cheng-Chun Hsu, Bowen Wen, Jie Xu, Yashraj Narang, Xi- aolong Wang, Yuke Zhu, Joydeep Biswas, and Stan Birch- field. Spot: Se(3) pose trajectory diffusion for object-centric manipulation, 2024. 3, 7

    2024

  9. [17]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, 2021. 4, 13

    2021

  10. [18]

    Ardup: Active region video diffusion for uni- versal policies

    Shuaiyi Huang, Mara Levy, Zhenyu Jiang, Anima Anand- kumar, Yuke Zhu, Linxi Fan, De-An Huang, and Abhinav Shrivastava. Ardup: Active region video diffusion for uni- versal policies. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 8465–8472,

    2024

  11. [19]

    Dif- fusion reward: Learning rewards via conditional video dif- fusion

    Tao Huang, Guangqi Jiang, Yanjie Ze, and Huazhe Xu. Dif- fusion reward: Learning rewards via conditional video dif- fusion. European Conference on Computer Vision (ECCV),

  12. [20]

    Rekep: Spatio-temporal reasoning of rela- tional keypoint constraints for robotic manipulation

    Wenlong Huang, Chen Wang, Yunzhu Li, Ruohan Zhang, and Li Fei-Fei. Rekep: Spatio-temporal reasoning of rela- tional keypoint constraints for robotic manipulation. In 8th Annual Conference on Robot Learning, 2024. 3, 7

    2024

  13. [21]

    Robots pre-train robots: Manipulation-centric robotic representation from large-scale robot datasets

    Guangqi Jiang, Yifei Sun, Tao Huang, Huanyu Li, Yongyuan Liang, and Huazhe Xu. Robots pre-train robots: Manipulation-centric robotic representation from large-scale robot datasets. arXiv preprint arXiv:2410.22325, 2024. 3

    2024 arXiv

  14. [22]

    Co- Tracker: It is better to track together

    Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Co- Tracker: It is better to track together. 2023. 4, 12, 13, 15

    2023

  15. [23]

    Egomimic: Scaling imitation learning via egocentric video,

    Simar Kareer, Dhruv Patel, Ryan Punamiya, Pranay Mathur, Shuo Cheng, Chen Wang, Judy Hoffman, and Danfei Xu. Egomimic: Scaling imitation learning via egocentric video,

  16. [24]

    Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Paul Rolland, Laura Gustafon, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross Girshick. Segment anything. arXiv preprint arXiv:2304.02643, 2023. 13

    2023 arXiv

  17. [25]

    Learning to Act from Actionless Videos through Dense Correspondences

    Po-Chen Ko, Jiayuan Mao, Yilun Du, Shao-Hua Sun, and Joshua B Tenenbaum. Learning to Act from Actionless Videos through Dense Correspondences. arXiv:2310.08576,

    arXiv

  18. [26]

    Learning hand-eye coordination for robotic grasping with large-scale data col- lection

    S Levine, P Pastor, A Krizhevsky, et al. Learning hand-eye coordination for robotic grasping with large-scale data col- lection. int symp on experimental robotics, 2016. 2

    2016

  19. [27]

    Offline reinforcement learning: Tutorial, review, and perspectives on open problems

    Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement learning: Tutorial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643, 2020. 2

    2005 arXiv

  20. [28]

    P3-po: Prescriptive point priors for visuo- spatial generalization of robot policies, 2024

    Mara Levy, Siddhant Haldar, Lerrel Pinto, and Abhinav Shirivastava. P3-po: Prescriptive point priors for visuo- spatial generalization of robot policies, 2024. 3

    2024

  21. [29]

    LEAGUE++: EMPOWERING CONTINUAL ROBOT LEARNING THROUGH GUIDED SKILL ACQUISITION WITH LARGE LANGUAGE MODELS

    Zhaoyi Li, Kelin Yu, Shuo Cheng, and Danfei Xu. LEAGUE++: EMPOWERING CONTINUAL ROBOT LEARNING THROUGH GUIDED SKILL ACQUISITION WITH LARGE LANGUAGE MODELS. In ICLR 2024 Workshop on Large Language Model (LLM) Agents , 2024. 2

    2024

  22. [30]

    Code as policies: Language model programs for embodied control

    Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code as policies: Language model programs for embodied control. In arXiv preprint arXiv:2209.07753, 2022. 2

    2022 arXiv

  23. [31]

    Dreamitate: Real-world visuomotor policy learn- ing via video generation

    Junbang Liang, Ruoshi Liu, Ege Ozguroglu, Sruthi Sud- hakar, Achal Dave, Pavel Tokmakov, Shuran Song, and Carl V ondrick. Dreamitate: Real-world visuomotor policy learn- ing via video generation. arXiv preprint arXiv:2406.16862,

    arXiv

  24. [32]

    Flowretrieval: Flow-guided data retrieval for few-shot imitation learning

    Li-Heng Lin, Yuchen Cui, Amber Xie, Tianyu Hua, and Dorsa Sadigh. Flowretrieval: Flow-guided data retrieval for few-shot imitation learning. In 8th Annual Conference on Robot Learning, 2024. 3

    2024

  25. [33]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European Conference on Computer Vision , pages 38–55. Springe...

    2025

  26. [34]

    Decoupled weight decay regularization, 2019

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, 2019. 13

    2019

  27. [35]

    Liv: Language-image representations and rewards for robotic control, 2023

    Yecheng Jason Ma, William Liang, Vaidehi Som, Vikash Kumar, Amy Zhang, Osbert Bastani, and Dinesh Jayara- man. Liv: Language-image representations and rewards for robotic control, 2023. 2

    2023

  28. [36]

    Eureka: Human-level reward design via coding large language models

    Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Eureka: Human-level reward design via coding large language models. arXiv preprint arXiv: Arxiv-2310.12931, 2023. 2

    arXiv

  29. [37]

    A real-to-sim-to-real approach to robotic manip- ulation with vlm-generated iterative keypoint rewards, 2025

    Shivansh Patel, Xinchen Yin, Wenlong Huang, Shubham Garg, Hooshang Nayyeri, Li Fei-Fei, Svetlana Lazebnik, and Yunzhu Li. A real-to-sim-to-real approach to robotic manip- ulation with vlm-generated iterative keypoint rewards, 2025. 3, 7

    2025

  30. [38]

    Learning transferable visual models from natural language supervision, 2021

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021. 2, 12

    2021

  31. [39]

    High-resolution image syn- thesis with latent diffusion models, 2022

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models, 2022. 2, 6, 12, 13

    2022

  32. [40]

    Proximal policy optimization algo- rithms, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Rad- ford, and Oleg Klimov. Proximal policy optimization algo- rithms, 2017. 2, 5

    2017

  33. [41]

    Motion before action: Diffusing object mo- tion as manipulation condition, 2025

    Yue Su, Xinyu Zhan, Hongjie Fang, Yong-Lu Li, Cewu Lu, and Lixin Yang. Motion before action: Diffusing object mo- tion as manipulation condition, 2025. 5

    2025

  34. [42]

    Embodiment-agnostic action plan- ning via object-part scene flow, 2024

    Weiliang Tang, Jia-Hui Pan, Wei Zhan, Jianshu Zhou, Huaxiu Yao, Yun-Hui Liu, Masayoshi Tomizuka, Mingyu Ding, and Chi-Wing Fu. Embodiment-agnostic action plan- ning via object-part scene flow, 2024. 3

    2024

  35. [43]

    Gpt-4 technical report, 2024

    OpenAI GPT-4 Team. Gpt-4 technical report, 2024. 2

    2024

  36. [44]

    Mujoco: A physics engine for model-based control

    Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 5026–5033, 2012. 6

    2012

  37. [45]

    Llama: Open and efficient foundation lan- guage models, 2023

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, Aure- lien Rodriguez, Armand Joulin, Edouard Grave, and Guil- laume Lample. Llama: Open and efficient foundation la...

    2023

  38. [46]

    This&that: Language-gesture controlled video generation for robot planning

    Boyang Wang, Nikhil Sridhar, Chao Feng, Mark Van der Merwe, et al. This&that: Language-gesture controlled video generation for robot planning. arXiv preprint arXiv:2407.05530, 2024. 1, 2

    2024 arXiv

  39. [47]

    Rl-vlm-f: Rein- forcement learning from vision language foundation model feedback, 2024

    Yufei Wang, Zhanyi Sun, Jesse Zhang, Zhou Xian, Erdem Biyik, David Held, and Zackory Erickson. Rl-vlm-f: Rein- forcement learning from vision language foundation model feedback, 2024. 2

    2024

  40. [48]

    ivideogpt: Interactive videogpts are scalable world models

    Jialong Wu, Shaofeng Yin, Ningya Feng, Xu He, Dong Li, Jianye Hao, and Mingsheng Long. ivideogpt: Interactive videogpts are scalable world models. Advances in Neural Information Processing Systems, 37:68082–68119, 2024. 2, 3

    2024

  41. [49]

    Text2reward: Reward shaping with language models for re- inforcement learning, 2024

    Tianbao Xie, Siheng Zhao, Chen Henry Wu, Yitao Liu, Qian Luo, Victor Zhong, Yanchao Yang, and Tao Yu. Text2reward: Reward shaping with language models for re- inforcement learning, 2024. 2

    2024

  42. [50]

    Learning by watching: Physical imitation of manipulation skills from human videos

    Haoyu Xiong, Quanzhou Li, Yun-Chun Chen, Homanga Bharadhwaj, Samarth Sinha, and Animesh Garg. Learning by watching: Physical imitation of manipulation skills from human videos. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 7827–7834....

    2021

  43. [51]

    Flow as the cross-domain manipulation interface, 2024

    Mengda Xu, Zhenjia Xu, Yinghao Xu, Cheng Chi, Gordon Wetzstein, Manuela Veloso, and Shuran Song. Flow as the cross-domain manipulation interface, 2024. 2, 3, 6, 7, 12, 13, 15

    2024

  44. [52]

    Videogpt: Video generation using vq-vae and trans- formers, 2021

    Wilson Yan, Yunzhi Zhang, Pieter Abbeel, and Aravind Srinivas. Videogpt: Video generation using vq-vae and trans- formers, 2021. 7

    2021

  45. [53]

    Rank2reward: Learning shaped reward functions from passive video

    Daniel Yang, Davin Tjia, Jacob Berg, Dima Damen, Pulkit Agrawal, and Abhishek Gupta. Rank2reward: Learning shaped reward functions from passive video. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 2806–2813. IEEE, 2024. 2

    2024

  46. [54]

    Mastering visual continuous control: Improved data- augmented reinforcement learning

    Denis Yarats, Rob Fergus, Alessandro Lazaric, and Lerrel Pinto. Mastering visual continuous control: Improved data- augmented reinforcement learning. In International Confer- ence on Learning Representations, 2022. 2, 5

    2022

  47. [55]

    Mimictouch: Leveraging multi-modal human tactile demonstrations for contact-rich manipulation

    Kelin Yu, Yunhai Han, Qixian Wang, Vaibhav Saxena, Dan- fei Xu, and Ye Zhao. Mimictouch: Leveraging multi-modal human tactile demonstrations for contact-rich manipulation. In 8th Annual Conference on Robot Learning, 2024. 3

    2024

  48. [56]

    Sketch-to-skill: Bootstrap- ping robot learning with human drawn trajectory sketches,

    Peihong Yu, Amisha Bhaskar, Anukriti Singh, Zahiruddin Mahammad, and Pratap Tokekar. Sketch-to-skill: Bootstrap- ping robot learning with human drawn trajectory sketches,

  49. [57]

    Meta-world: A benchmark and evaluation for multi-task and meta reinforce- ment learning, 2021

    Tianhe Yu, Deirdre Quillen, Zhanpeng He, Ryan Julian, Avnish Narayan, Hayden Shively, Adithya Bellathur, Karol Hausman, Chelsea Finn, and Sergey Levine. Meta-world: A benchmark and evaluation for multi-task and meta reinforce- ment learning, 2021. 6, 7, 12, 13

    2021

  50. [58]

    General flow as foundation affordance for scalable robot learning

    Chengbo Yuan, Chuan Wen, Tong Zhang, and Yang Gao. General flow as foundation affordance for scalable robot learning. arXiv preprint arXiv:2401.11439 , 2024. 2, 3, 6, 7

    2024 arXiv

  51. [59]

    Tracevla: Visual trace prompting enhances spatial-temporal awareness for generalist robotic policies

    Ruijie Zheng, Yongyuan Liang, Shuaiyi Huang, Jianfeng Gao, Hal Daum ´e III, Andrey Kolobov, Furong Huang, and Jianwei Yang. Tracevla: Visual trace prompting enhances spatial-temporal awareness for generalist robotic policies. arXiv preprint arXiv:2412.10345, 2024. 3 A. Cross-E...

    2024 arXiv

  52. [60]

    Same as the Im2FLow2Act [51], we encode the object flow into a latent space and train the generative model based on it

    for flow generation. Same as the Im2FLow2Act [51], we encode the object flow into a latent space and train the generative model based on it. Similar to the StableDiffu- sion [39], we use the auto encoder V A-GAN [7] to encode the flow into low dimentional embeddings. Then, to ...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.