Pith. sign in

REVIEW 3 major objections 5 minor 117 references

H2R-Bench: Benchmarking Human-to-Robot Manipulation Video Generation in World Models

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A new benchmark shows that video world models which produce polished videos still fail to transfer human manipulation demonstrations to specified robot embodiments.

desk verdict A well-built benchmark for the source-relative H2R transfer setting with an honest, carefully analyzed empirical finding; the MLLM-judged contact and embodiment metrics are the main soft spot but not fatal. read the letter →

arxiv 2608.13049 v1 pith:CO22TYLA submitted 2026-08-13 cs.RO cs.CV

Dingyi Rong , Yue Shi , Chaofan Ma , Jiezhang Cao , Zongrui Wang , Zeyu Zhang , Yao Mu , Guangtao Zhai
show 1 more author
Ning Liu
This is my paper · ORCID
classification cs.ROcs.CV
keywords human-to-robotvideotransferworldmodelsmanipulationbenchmarkcross-embodimentgenerationfunctionalcontactembodimentcorrectnessevaluationrobotlearningfromhumanvideos
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Video world models are generators that simulate how a scene evolves; the hope is that they can turn abundant egocentric human manipulation videos into robot demonstrations. This paper introduces H2R-Bench, a benchmark that tests whether eleven such models can actually do the transfer: given a human demo and a target robot embodiment, the model must produce a video that preserves the task goal, the required action events, the functional contact between end-effector and object, and the requested robot morphology. The benchmark scores 240 cases built from 120 egocentric clips across six manipulation families and two embodiments, and reports that current models are far from reliable: the best H2RCore is 84.6 out of 100, several models sit near 30, and the task-agnostic video-quality score is almost uncorrelated with transfer quality (Spearman ρ = 0.14). If the benchmark is right, visual polish is not evidence that a generated robot video would be usable as robot training data.

What carries the argument

The load-bearing mechanism is the source-relative scoring protocol wrapped around the H2RCore aggregate. The benchmark defines each case as a triple of human source video, target embodiment, and generation prompt, and keeps evaluation annotations separate from what the generator sees. Three AI vision-language judges independently score M1–M4 on 25 uniformly sampled frames against the source-derived annotation, with M3 comparing source and generated frames for functional contact and M4 isolating robot presence, absence of human hands, embodiment category, end-effector correctness, and structural consistency. The aggregate $H2RCore = 100(0.15 S_{goal} + 0.15 S_{action} + 0.30 S_{contact} + 0.30 S_{emb} + 0.10 S_{video})$ gives contact and embodiment 60% of the weight, which is what separates visually plausible videos from transfer-valid ones. M5 measures imaging quality, aesthetics, temporal stability, and motion smoothness without reference to the task.

What would settle it

Have a physical robot or a contact-physics simulator attempt the manipulations in the videos that the benchmark scores highest on contact and embodiment, and check whether the object state actually follows from the visible contact; if high-scoring videos consistently fail physical execution, the benchmark's transfer scores are measuring appearance rather than functional transfer.

Watch

Extended reading notes

Core claim

The central claim is that source-relative evaluation is required to judge human-to-robot transfer, and under that evaluation current video world models largely fail. Each test case specifies a human source video, a target embodiment (parallel-jaw gripper or dexterous hand), and a text prompt; the source video is treated as evidence of what happened, not a style reference. The benchmark annotates each source with weighted final-state predicates, required action events, functional-contact specifications, and embodiment constraints, then scores generated videos with five metrics: goal-state completion, action-event completion, functional contact transfer, embodiment correctness, and task-agnostic video quality. On the 240-case main evaluation, the three full-video-conditioned models lead with H2RCore between 74.5 and 84.6, frame-conditioned models cluster near 30–61, and actions or final states are often preserved even when contact and embodiment are wrong. The paper reads this as showing that task recognition and visual polish are not enough: embodiment consistency, functional interaction, and task execution remain unsolved for current video world models.

Load-bearing premise

The benchmark's conclusions depend on three AI vision-language judges being able to tell, from only 25 sampled frames of each video, whether a robot really made functional contact and whether the embodiment is correct.

Editorial extensions

If this is right

  • Source-conditioning interface matters: the three full-video-conditioned models occupy the top of the leaderboard, while frame-conditioned models often leave the human as the active manipulator or replace the end effector incorrectly.
  • Visual quality and transfer quality are decoupled: quality scores range only 0.73–0.81 while H2RCore spans 30.0–84.6, with rank correlation ρ = 0.14, so a model can look polished and still fail transfer.
  • The requested embodiment changes difficulty: all eleven models score higher on contact transfer for the dexterous hand, and nine of eleven score higher overall on H2RCore for the hand than for the parallel-jaw gripper.
  • A target-robot reference image is not a reliable fix: it substantially helps one model with the gripper while hurting two others, so embodiment grounding is model-dependent.
  • Current video world models are not yet dependable generators of robot-centric training data from human demonstrations, because leading models still fail on at least one of contact, embodiment, or task execution across the six manipulation families.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the near-zero correlation between visual quality and H2RCore implies that pipelines filtering generated robot videos by aesthetics or text-video alignment are selecting on the wrong signal; a cheap transfer-aware check such as the M3/M4 rubric would retain more usable training data.
  • Editorial inference: because M3 judges visible evidence and does not claim to verify physical causation, the benchmark likely measures apparent contact, not verified contact; a natural extension is to feed high-scoring generations into a contact-physics simulator or real gripper and test whether the object state follows from the visible interaction.
  • Editorial inference: the consistent dexterous-hand advantage suggests current generators transfer more readily to embodiments that resemble the human hand; this predicts that adding a humanoid hand or a soft gripper embodiment would systematically narrow the gap, which a larger embodiment set could test.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. H2R-Bench introduces a benchmark for evaluating human-to-robot manipulation video generation: given an egocentric human demonstration, a target embodiment (parallel-jaw gripper or dexterous hand), and a text prompt, a video generator must produce a robot manipulation video that preserves the source task goal, required action events, functional contact, and object responses while realizing the execution with the requested robot morphology. The benchmark is built from 120 EgoDex test clips, paired with two embodiments to yield 240 transfer cases across six manipulation families. Eleven video generators are evaluated with their native source-conditioning interfaces. Five metrics are defined: M1 goal-state completion, M2 action-event completion, M3 functional contact transfer, M4 embodiment correctness, and M5 task-agnostic video quality. M1-M4 are scored by three MLLM judges (Gemini 3.5 Flash, Qwen3.7-Plus, GPT-5.4) from 25 uniformly sampled frames; M5 combines MUSIQ, a CLIP-based aesthetic predictor, adjacent-frame stability, and AMT-S interpolation consistency. The aggregate H2RCore weights contact and embodiment at 0.30 each, goal and action at 0.15 each, and quality at 0.10. The main findings are that video-conditioned models (Seedance 2.0, Wan2.7, Kling-V3) rank highest, that M5 scores lie in a narrow range while H2RCore spans 30.0-84.6, and that the rank correlation between M5 and H2RCore is weak (Spearman rho = 0.14).

Significance. If the benchmark measurements are valid, H2R-Bench fills a real gap: existing video-generation benchmarks do not evaluate source-relative, cross-embodiment transfer from human demonstrations to robot embodiments. The paper's diagnostic decomposition into goal, action, contact, embodiment, and quality is well motivated, and the empirical claim that generic video quality is nearly decoupled from transfer quality (rho = 0.14) is important and falsifiable. The authors are methodologically careful in several ways: all annotations are manually verified against source videos; the evidence budget for M1-M4 is fixed across models; paired bootstrap confidence intervals are reported for the leading model comparisons; weight sensitivity is checked; and human validation is conducted on a substantial sample of 660 videos. The limitations section is unusually candid, acknowledging that the native-interface protocol confounds model capability with conditioning interface and that MLLM judgments from sampled frames can be uncertain.

major comments (3)
  1. [§B.6 (M3/M4) and §D]
  2. [§A.2 and §B.6]
  3. [Experiments, Native-Interface Protocol, and §D]
minor comments (5)
  1. [Figure 4]
  2. [§B.5 and Figure 4]
  3. [§B.2]
  4. [Table S7]
  5. [Appendix C]

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: H2R-Bench's scores are empirical metric aggregates, and its MLLM judge/annotation overlap is neither a formal reduction nor load-bearing.

full rationale

The paper makes no prediction from fitted parameters and derives no result from its inputs by construction. Section B.6 defines M1-M4 as MLLM-judged rubric scores over fixed evidence frames and M5 as standard task-agnostic quality metrics; H2RCore is an explicit weighted aggregate (Eq. S12). The headline findings (Table 2 and the weak quality-transfer association, Spearman rho = 0.14, in Figure 4) are empirical observations about those aggregated scores, not quantities forced by the definitions. The source annotations are produced by Qwen3.7-Plus and then manually verified against each source video (A.2), and Qwen3.7-Plus also serves as one of three MLLM judges (B.6). This is a possible same-family alignment, but it is not a circular reduction: none of the scoring equations (S1)-(S5), (S11) equals an annotation by construction, the three judge scores are averaged, and the human validation in B.7 independently reports r = 0.930 for the transfer subscore. The Limitations section explicitly concedes that sampled visual evidence and MLLM judgments can remain uncertain, which is a validity caveat rather than evidence of circularity. Self-citations in the paper (e.g., Shi et al. 2025, 2026) appear only in the Limitations discussion and are not load-bearing. No step exhibits Eq. X = Eq. Y by construction, no fitted parameter is renamed as a prediction, and no load-bearing claim rests on a self-citation chain.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim is an empirical evaluation, so the main burdens are the annotation and judging assumptions just listed. The hand-chosen weights affect the aggregate score but not the qualitative ranking. No new physical entities are introduced.

free parameters (2)
  • H2RCore weight vector = 0.15, 0.15, 0.30, 0.30, 0.10
    Hand-chosen weights in Eq. (3) prioritize contact and embodiment over goal, action, and quality. Sensitivity analysis (B.5) shows ranking stability under alternatives, but the aggregate and the VBench-vs-H2RCore correlation depend on this choice.
  • Failure-rate threshold = 0.5
    B.1 defines a diagnostic failure when a judge-averaged component score is below 0.5. This threshold is arbitrary but only used for descriptive failure-rate reporting, not for the main ranking.
assumptions (4)
  • domain assumption MLLM judges can reliably assess functional contact, object response, and embodiment consistency from 25 uniformly sampled frames of each video.
    M1-M4 rely on three proprietary MLLMs scoring visual evidence at a fixed frame budget (B.6). The validity of the benchmark hinges on this assumption; human validation on 660 videos partially supports it (B.7).
  • domain assumption The source annotations generated by Qwen3.7-Plus and then human-verified accurately capture the task goal, action events, and contact requirements of each EgoDex clip.
    A.2: Qwen3.7-Plus generates structured annotations from the source video and all are manually verified. If annotations misstate the task, the metrics score the wrong thing.
  • domain assumption The six task families (F1-F6) cover the relevant variation in manipulation state changes for the H2R setting.
    The taxonomy is designed for the benchmark (Table S1) and bounds generalizability; the paper itself notes coverage is limited to two embodiments and 120 sources (Section D).
  • standard math The video quality sub-metrics (MUSIQ, LAION aesthetic predictor, AMT-S interpolation) are valid task-agnostic quality measures.
    M5 uses established models with prior validation. These are standard tools and a reasonable external input.

how reviews work

0 comments
Cite this review

Pith. "Pith review of H2R-Bench: Benchmarking Human-to-Robot Manipulation Video Generation in World Models." pith.science (2026). https://pith.science/paper/CO22TYLA

@misc{pith2026260813049,
  author       = {Pith},
  title        = {Pith review of: H2R-Bench: Benchmarking Human-to-Robot Manipulation Video Generation in World Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CO22TYLA}},
  note         = {Machine review of arXiv:2608.13049}
}
read the original abstract

Large-scale manipulation data is essential for robot learning, yet collecting robot demonstrations remains expensive and difficult to scale. Meanwhile, abundant egocentric human manipulation videos provide rich behavioral experiences, but transferring them across embodiments remains challenging due to differences between human hands and robotic end-effectors. Recent advances in video world models offer a promising pathway to synthesize robot-centric manipulation videos from human observations, while their cross-embodiment transfer capability remains largely unexplored. Therefore, we introduce H2R-Bench, a benchmark for evaluating cross-embodiment human-to-robot manipulation video generation, where models transform egocentric human demonstrations into robot manipulation videos under specified embodiments. Each benchmark instance contains a human demonstration video, target embodiment constraints, and source-grounded annotations covering task goals, action events, functional contacts, and object responses. H2R-Bench evaluates generated videos through five dimensions, including goal-state completion, action-event completion, functional contact transfer, embodiment correctness, and general video quality. We benchmark eleven state-of-the-art video generation models across six manipulation families and two robot embodiments. Our evaluation reveals that current video world models remain limited in human-to-robot manipulation transfer: even leading models often fail in embodiment consistency, functional interaction, and task execution. H2R-Bench provides a systematic diagnostic framework for evaluating whether video world models can bridge the human-to-robot embodiment gap and convert human manipulation observations into robot-centric training resources.

Figures

Figures reproduced from arXiv: 2608.13049 by the authors.

Figure 1
Figure 1. Comparison between existing evaluation and H2R [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of H2R-Bench. The benchmark curates egocentric human manipulation demonstrations, conditions video [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Agreement between human and MLLM evaluators. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: VBench Video Quality versus H2RCore. Unlike [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Qualitative H2R transfer results with the shared source video and prompt. The unscored top row is the human source; [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

117 extracted references · 38 canonical work pages

  1. [1]

    OpenAI Blog , volume=

    Video generation models as world simulators , author=. OpenAI Blog , volume=

  2. [2]

    Forty-first International Conference on Machine Learning , year=

    Genie: Generative interactive environments , author=. Forty-first International Conference on Machine Learning , year=

  3. [3]

    Advances in Neural Information Processing Systems , volume=

    Worldmodelbench: Judging video generation models as world models , author=. Advances in Neural Information Processing Systems , volume=

  4. [6]

    Conference on Robot Learning , pages=

    Rt-2: Vision-language-action models transfer web knowledge to robotic control , author=. Conference on Robot Learning , pages=. 2023 , organization=

    2023

  5. [7]

    2024 IEEE International Conference on Robotics and Automation (ICRA) , pages=

    Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0 , author=. 2024 IEEE International Conference on Robotics and Automation (ICRA) , pages=. 2024 , organization=

    2024

  6. [11]

    International Journal of Computer Vision , volume=

    Rescaling egocentric vision: Collection, pipeline and challenges for epic-kitchens-100 , author=. International Journal of Computer Vision , volume=. 2022 , publisher=

    2022

  7. [12]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Ego4d: Around the world in 3,000 hours of egocentric video , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  8. [19]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    RoboWM-Bench: A Benchmark for Evaluating World Models in Robotic Manipulation , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

Show all 117 references
  1. [21]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Vbench: Comprehensive benchmark suite for video generative models , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  2. [23]

    IEEE Transactions on human-machine systems , volume=

    The grasp taxonomy of human grasp types , author=. IEEE Transactions on human-machine systems , volume=. 2015 , publisher=

    2015

  3. [24]

    Frontiers in Neurorobotics , volume=

    A survey of multifingered robotic manipulation: Biological results, structural evolvements, and learning methods , author=. Frontiers in Neurorobotics , volume=. 2022 , publisher=

    2022

  4. [25]

    arXiv preprint arXiv:2504.12609 , year=

    Crossing the human-robot embodiment gap with sim-to-real rl using one human demonstration , author=. arXiv preprint arXiv:2504.12609 , year=

    arXiv

  5. [26]

    arXiv preprint arXiv:2501.04169 , year=

    Learning to transfer human hand skills for robot manipulations , author=. arXiv preprint arXiv:2501.04169 , year=

    arXiv

  6. [27]

    arXiv preprint arXiv:2209.14792 , year=

    Make-a-video: Text-to-video generation without text-video data , author=. arXiv preprint arXiv:2209.14792 , year=

    arXiv

  7. [28]

    arXiv preprint arXiv:2210.02303 , year=

    Imagen video: High definition video generation with diffusion models , author=. arXiv preprint arXiv:2210.02303 , year=

    arXiv

  8. [29]

    arXiv preprint arXiv:2210.02399 , year=

    Phenaki: Variable length video generation from open domain textual description , author=. arXiv preprint arXiv:2210.02399 , year=

    arXiv

  9. [30]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Align your latents: High-resolution video synthesis with latent diffusion models , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  10. [31]

    arXiv preprint arXiv:2307.04725 , year=

    Animatediff: Animate your personalized text-to-image diffusion models without specific tuning , author=. arXiv preprint arXiv:2307.04725 , year=

    arXiv

  11. [32]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    Stablevideo: Text-driven consistency-aware diffusion video editing , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  12. [33]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Videocrafter2: Overcoming data limitations for high-quality video diffusion models , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  13. [34]

    SIGGRAPH Asia 2024 Conference Papers , pages=

    Lumiere: A space-time diffusion model for video generation , author=. SIGGRAPH Asia 2024 Conference Papers , pages=

    2024

  14. [35]

    International Conference on Learning Representations , volume=

    Cogvideox: Text-to-video diffusion models with an expert transformer , author=. International Conference on Learning Representations , volume=

  15. [40]

    International Conference on Machine Learning , pages=

    Liv: Language-image representations and rewards for robotic control , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  16. [41]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Affordances from human videos as a versatile representation for robotics , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  17. [42]

    European Conference on Computer Vision , pages=

    Track2act: Predicting point tracks from internet videos enables generalizable robot manipulation , author=. European Conference on Computer Vision , pages=. 2024 , organization=

    2024

  18. [43]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Vidbot: Learning generalizable 3d actions from in-the-wild 2d human videos for zero-shot robotic manipulation , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  19. [44]

    International Conference on Learning Representations , volume=

    Latent action pretraining from videos , author=. International Conference on Learning Representations , volume=

  20. [45]

    Proceedings of the IEEE/CVF international conference on computer vision , year=

    Moto: Latent motion token as the bridging language for robot manipulation , author=. Proceedings of the IEEE/CVF international conference on computer vision , year=

  21. [48]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Evalcrafter: Benchmarking and evaluating large video generation models , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  22. [49]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    T2vbench: Benchmarking temporal dynamics for text-to-video generation , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  23. [50]

    International Conference on Learning Representations , volume=

    Videophy: Evaluating physical commonsense for video generation , author=. International Conference on Learning Representations , volume=

  24. [51]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    T2v-compbench: A comprehensive benchmark for compositional text-to-video generation , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  25. [52]

    Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    OSCBench: Benchmarking Object State Change in Text-to-Video Generation , author=. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  26. [53]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Human2robot: Learning robot actions from paired human-robot videos , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  27. [57]

    2025 , month = oct, howpublished =

    Introducing Veo 3.1 and Advanced Capabilities in Flow , author =. 2025 , month = oct, howpublished =

    2025

  28. [58]

    2026 , month = jan, note =

    Grok Imagine API , author =. 2026 , month = jan, note =

    2026

  29. [63]

    Proceedings of the IEEE/CVF international conference on computer vision , pages=

    Musiq: Multi-scale image quality transformer , author=. Proceedings of the IEEE/CVF international conference on computer vision , pages=

  30. [64]

    International conference on machine learning , pages=

    Learning transferable visual models from natural language supervision , author=. International conference on machine learning , pages=. 2021 , organization=

    2021

  31. [65]

    2022 , howpublished=

    2022

  32. [66]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Amt: All-pairs multi-field transforms for efficient frame interpolation , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  33. [70]

    Advances in Neural Information Processing Systems (NeurIPS) , year=

    AttrSeg: Open-Vocabulary Semantic Segmentation via Attribute Decomposition-Aggregation , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=

  34. [71]

    Proceedings of the International Conference on Machine Learning (ICML) , year=

    MoMa: Modulating Mamba for Adapting Image Foundation Models to Video Recognition , author=. Proceedings of the International Conference on Machine Learning (ICML) , year=

  35. [72]

    European Conference on Computer Vision (ECCV) , year=

    ReMamber: Referring Image Segmentation with Mamba Twister , author=. European Conference on Computer Vision (ECCV) , year=

  36. [75]

    Advances in Neural Information Processing Systems (NeurIPS) , year=

    SaFiRe: Saccade-Fixation Reiteration with Mamba for Referring Image Segmentation , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=

  37. [76]

    British Machine Vision Conference (BMVC) , year=

    Open-vocabulary Semantic Segmentation with Frozen Vision-Language Models , author=. British Machine Vision Conference (BMVC) , year=

  38. [77]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year=

    GenMask: Adapting DiT for Segmentation via Direct Mask Generation , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year=

  39. [78]

    International Journal of Computer Vision (IJCV) , year=

    Multi-Modal Prototypes for Open-World Semantic Segmentation , author=. International Journal of Computer Vision (IJCV) , year=

  40. [80]

    Displays , volume=

    DARF: Depth-Aware Generalizable Neural Radiance Field , author=. Displays , volume=. 2025 , doi=

    2025

  41. [81]

    Bahl, S.; Mendonca, R.; Chen, L.; Jain, U.; and Pathak, D. 2023. Affordances from human videos as a versatile representation for robotics. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 13778--13790

    2023

  42. [82]

    Bansal, H.; Lin, Z.; Xie, T.; Zong, Z.; Yarom, M.; Bitton, Y.; Jiang, C.; Sun, Y.; Chang, K.-W.; and Grover, A. 2025. Videophy: Evaluating physical commonsense for video generation. In International Conference on Learning Representations, volume 2025, 102075--102121

    2025

  43. [83]

    Bar-Tal, O.; Chefer, H.; Tov, O.; Herrmann, C.; Paiss, R.; Zada, S.; Ephrat, A.; Hur, J.; Liu, G.; Raj, A.; et al. 2024. Lumiere: A space-time diffusion model for video generation. In SIGGRAPH Asia 2024 Conference Papers, 1--11

    2024

  44. [84]

    Bharadhwaj, H.; Mottaghi, R.; Gupta, A.; and Tulsiani, S. 2024. Track2act: Predicting point tracks from internet videos enables generalizable robot manipulation. In European Conference on Computer Vision, 306--324. Springer

    2024

  45. [85]

    Black, K.; Brown, N.; Driess, D.; Esmail, A.; Equi, M.; Finn, C.; Fusai, N.; Groom, L.; Hausman, K.; Ichter, B.; et al. 2024. pi\_0: A Vision-Language-Action Flow Model for General Robot Control. arXiv preprint arXiv:2410.24164

    2024 arXiv

  46. [86]

    W.; Fidler, S.; and Kreis, K

    Blattmann, A.; Rombach, R.; Ling, H.; Dockhorn, T.; Kim, S. W.; Fidler, S.; and Kreis, K. 2023. Align your latents: High-resolution video synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 22563--22575

    2023

  47. [87]

    Brohan, A.; Brown, N.; Carbajal, J.; Chebotar, Y.; Dabis, J.; Finn, C.; Gopalakrishnan, K.; Hausman, K.; Herzog, A.; Hsu, J.; et al. 2022. Rt-1: Robotics transformer for real-world control at scale. arXiv preprint arXiv:2212.06817

    2022 arXiv

  48. [88]

    Brooks, T.; Peebles, B.; Holmes, C.; DePue, W.; Guo, Y.; Jing, L.; Schnurr, D.; Taylor, J.; Luhman, T.; Luhman, E.; et al. 2024. Video generation models as world simulators. OpenAI Blog, 1(8): 1

    2024

  49. [89]

    D.; Edwards, A.; Parker-Holder, J.; Shi, Y.; Hughes, E.; Lai, M.; Mavalankar, A.; Steigerwald, R.; Apps, C.; et al

    Bruce, J.; Dennis, M. D.; Edwards, A.; Parker-Holder, J.; Shi, Y.; Hughes, E.; Lai, M.; Mavalankar, A.; Steigerwald, R.; Apps, C.; et al. 2024. Genie: Generative interactive environments. In Forty-first International Conference on Machine Learning

    2024

  50. [90]

    Chen, H.; Sun, B.; Zhang, A.; Pollefeys, M.; and Leutenegger, S. 2025 a . Vidbot: Learning generalizable 3d actions from in-the-wild 2d human videos for zero-shot robotic manipulation. In Proceedings of the Computer Vision and Pattern Recognition Conference, 27661--27672

    2025

  51. [91]

    Chen, Y.; Ge, Y.; Li, Y.; Ge, Y.; Ding, M.; Shan, Y.; and Liu, X. 2025 b . Moto: Latent motion token as the bridging language for robot manipulation. In Proceedings of the IEEE/CVF international conference on computer vision

    2025

  52. [92]

    Ci, H.; Liu, X.; Yang, P.; Song, Y.; and Shou, M. Z. 2025. H2R-Grounder: A Paired-Data-Free Paradigm for Translating Human Interaction Videos into Physically Grounded Robot Videos. arXiv preprint arXiv:2512.09406

    2025

  53. [93]

    M.; Furnari, A.; Kazakos, E.; Ma, J.; Moltisanti, D.; Munro, J.; Perrett, T.; Price, W.; et al

    Damen, D.; Doughty, H.; Farinella, G. M.; Furnari, A.; Kazakos, E.; Ma, J.; Moltisanti, D.; Munro, J.; Perrett, T.; Price, W.; et al. 2022. Rescaling egocentric vision: Collection, pipeline and challenges for epic-kitchens-100. International Journal of Computer Vision, 130(1): 33--55

    2022

  54. [94]

    Deng, Y.; Pan, Z.; Zhang, H.; Li, X.; Hu, R.; Ding, Y.; Zou, Y.; Zeng, Y.; and Zhou, D. 2026. Rethinking Video Generation Model for the Embodied World. arXiv preprint arXiv:2601.15282

    2026

  55. [95]

    Google . 2025. Introducing Veo 3.1 and Advanced Capabilities in Flow. Google Official Blog. Released October 15, 2025

    2025

  56. [96]

    Grauman, K.; Westbury, A.; Byrne, E.; Chavis, Z.; Furnari, A.; Girdhar, R.; Hamburger, J.; Jiang, H.; Liu, M.; Liu, X.; et al. 2022. Ego4d: Around the world in 3,000 hours of egocentric video. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition...

    2022

  57. [97]

    HaCohen, Y.; Brazowski, B.; Chiprut, N.; Bitterman, Y.; Kvochko, A.; Berkowitz, A.; Shalem, D.; Lifschitz, D.; Moshe, D.; Porat, E.; et al. 2026. LTX-2: Efficient Joint Audio-Visual Foundation Model. arXiv preprint arXiv:2601.03233

    2026 arXiv

  58. [98]

    M.; Carrington, P.; Zimmermann, R.; and Chen, J

    Han, X.; Zhu, B.; Hu, S.; Li, F. M.; Carrington, P.; Zimmermann, R.; and Chen, J. 2026. OSCBench: Benchmarking Object State Change in Text-to-Video Generation. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 3...

    2026

  59. [99]

    J.; Sivapurapu, M.; and Zhang, J

    Hoque, R.; Huang, P.; Yoon, D. J.; Sivapurapu, M.; and Zhang, J. 2025. Egodex: Learning dexterous manipulation from large-scale egocentric video. arXiv preprint arXiv:2505.11709

    2025 arXiv

  60. [100]

    Huang, Z.; He, Y.; Yu, J.; Zhang, F.; Si, C.; Jiang, Y.; Zhang, Y.; Wu, T.; Jin, Q.; Chanpaisit, N.; et al. 2024. Vbench: Comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 21807--21818

    2024

  61. [101]

    Ji, P.; Xiao, C.; Tai, H.; and Huo, M. 2024. T2vbench: Benchmarking temporal dynamics for text-to-video generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5325--5335

    2024

  62. [102]

    Jiang, F.; Chen, Y.; Xu, K.; Liu, Y.; Wang, H.; Shen, Z.; Lu, J.; Huang, S.; Wang, Y.; Xie, C.; et al. 2026. RoboWM-Bench: A Benchmark for Evaluating World Models in Robotic Manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4455--4460

    2026

  63. [103]

    Ke, J.; Wang, Q.; Wang, Y.; Milanfar, P.; and Yang, F. 2021. Musiq: Multi-scale image quality transformer. In Proceedings of the IEEE/CVF international conference on computer vision, 5148--5157

    2021

  64. [104]

    Kong, W.; Tian, Q.; Zhang, Z.; Min, R.; Dai, Z.; Zhou, J.; Xiong, J.; Li, X.; Wu, B.; Zhang, J.; et al. 2024. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603

    2024 arXiv

  65. [105]

    LAION-AI . 2022. LAION-Aesthetics Predictor V1 . https://github.com/LAION-AI/aesthetic-predictor

    2022

  66. [106]

    Lepert, M.; Fang, J.; and Bohg, J. 2025. Masquerade: Learning from in-the-wild human videos using data-editing. arXiv preprint arXiv:2508.09976

    2025 arXiv

  67. [107]

    Li, D.; Fang, Y.; Chen, Y.; Yang, S.; Cao, S.; Wong, J.; Luo, M.; Wang, X.; Yin, H.; Gonzalez, J.; et al. 2026 a . Worldmodelbench: Judging video generation models as world models. Advances in Neural Information Processing Systems, 38

    2026

  68. [108]

    Li, D.; Fei, Z.; Li, T.; Dou, Y.; Chen, Z.; Yang, J.; Fan, M.; Xu, J.; Wang, J.; Gu, B.; et al. 2026 b . Skyreels-v3 technique report. arXiv preprint arXiv:2601.17323

    2026

  69. [109]

    Li, G.; Lyu, Y.; Liu, Z.; Hou, C.; Zhang, J.; and Zhang, S. 2025 a . H2r: A human-to-robot data augmentation for robot pre-training from videos. arXiv preprint arXiv:2505.11920

    2025

  70. [110]

    Li, H.; Wang, J.; Mei, Z.; Majumdar, A.; Chen, J.; and Zhu, B. 2026 c . RoboTrustBench: Benchmarking the Trustworthiness of Video World Models for Robotic Manipulation. arXiv preprint arXiv:2606.01600

    2026 arXiv

  71. [111]

    Li, H.; Zhang, I.; Ouyang, R.; Wang, X.; Zhu, Z.; Yang, Z.; Zhang, Z.; Wang, B.; Ni, C.; Qin, W.; et al. 2025 b . Mimicdreamer: Aligning human and robot demonstrations for scalable vla training. arXiv preprint arXiv:2509.22199

    2025

  72. [112]

    Li, Q.; Deng, Y.; Liang, Y.; Luo, L.; Zhou, L.; Yao, C.; Zeng, L.; Feng, Z.; Liang, H.; Xu, S.; et al. 2025 c . Scalable vision-language-action model pretraining for robotic manipulation with real-life human activity videos. arXiv preprint arXiv:2510.21571

    2025

  73. [113]

    Li, Z.; Zhu, Z.-L.; Han, L.-H.; Hou, Q.; Guo, C.-L.; and Cheng, M.-M. 2023. Amt: All-pairs multi-field transforms for efficient frame interpolation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9801--9810

    2023

  74. [114]

    Liu, Y.; Cun, X.; Liu, X.; Wang, X.; Zhang, Y.; Chen, H.; Liu, Y.; Zeng, T.; Chan, R.; and Shan, Y. 2024. Evalcrafter: Benchmarking and evaluating large video generation models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 22139--22149

    2024

  75. [115]

    Ma, C.; Mao, Z.; Yang, Y.; Zeng, F.; Shi, Y.; Zhou, Y.; Cao, X.; and Yao, J. 2026 a . Reason, Then Re-reason: Cross-view Revisiting Improves Spatial Reasoning. arXiv preprint arXiv:2606.11683

    2026 arXiv

  76. [116]

    Ma, C.; Yang, Y.; Ju, C.; Zhang, F.; Zhang, Y.; and Wang, Y. 2023 a . AttrSeg: Open-Vocabulary Semantic Segmentation via Attribute Decomposition-Aggregation. In Advances in Neural Information Processing Systems (NeurIPS)

    2023

  77. [117]

    Ma, C.; Yang, Y.; Wang, Y.; Zhang, Y.; and Xie, W. 2022 a . Open-vocabulary Semantic Segmentation with Frozen Vision-Language Models. In British Machine Vision Conference (BMVC)

    2022

  78. [118]

    Ma, J.; Zhang, E.; Yang, H.; Li, D.; Xu, C.; Wang, G.; and Wang, H. 2026 b . Robot Learning from Human Videos: A Survey. arXiv preprint arXiv:2604.27621

    2026 arXiv

  79. [119]

    J.; Kumar, V.; Zhang, A.; Bastani, O.; and Jayaraman, D

    Ma, Y. J.; Kumar, V.; Zhang, A.; Bastani, O.; and Jayaraman, D. 2023 b . Liv: Language-image representations and rewards for robotic control. In International Conference on Machine Learning, 23301--23320. PMLR

    2023

  80. [120]

    J.; Sodhani, S.; Jayaraman, D.; Bastani, O.; Kumar, V.; and Zhang, A

    Ma, Y. J.; Sodhani, S.; Jayaraman, D.; Bastani, O.; Kumar, V.; and Zhang, A. 2022 b . Vip: Towards universal visual reward and representation via value-implicit pre-training. arXiv preprint arXiv:2210.00030

    2022 arXiv

  81. [121]

    Mao, Z.; Yang, Y.; Ma, C.; Jiang, D.; Yao, J.; Zhang, Y.; and Wang, Y. 2025. SaFiRe: Saccade-Fixation Reiteration with Mamba for Referring Image Segmentation. In Advances in Neural Information Processing Systems (NeurIPS)

    2025

  82. [122]

    Nair, S.; Rajeswaran, A.; Kumar, V.; Finn, C.; and Gupta, A. 2022. R3m: A universal visual representation for robot manipulation. arXiv preprint arXiv:2203.12601

    2022 arXiv

  83. [123]

    O’Neill, A.; Rehman, A.; Maddukuri, A.; Gupta, A.; Padalkar, A.; Lee, A.; Pooley, A.; Gupta, A.; Mandlekar, A.; Jain, A.; et al. 2024. Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0. In 2024 IEEE International Conference on Robo...

    2024

  84. [124]

    W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

    Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, 8748--8763. PmLR

    2021

  85. [125]

    Seedance, T.; Chen, D.; Chen, L.; Chen, X.; Chen, Y.; Chen, Z.; Chen, Z.; Cheng, F.; Cheng, T.; Cheng, Y.; et al. 2026. Seedance 2.0: Advancing video generation for world complexity. arXiv preprint arXiv:2604.14148

    2026 arXiv

  86. [126]

    Shi, Y.; Rong, D.; Chen, C.; Ma, C.; Ni, B.; and Zhang, W. 2025. DARF: Depth-Aware Generalizable Neural Radiance Field. Displays, 88: 102996

    2025

  87. [127]

    Shi, Y.; Shi, R.; Xiong, Y.; Ni, B.; and Zhang, W. 2026. CEI-3D: Collaborative Explicit-Implicit 3D Reconstruction for Realistic and Fine-Grained Object Editing. arXiv preprint arXiv:2603.11810

    2026

  88. [128]

    Smith, L.; Dhawan, N.; Zhang, M.; Abbeel, P.; and Levine, S. 2019. Avid: Learning multi-stage tasks via pixel-level translation of human videos. arXiv preprint arXiv:1912.04443

    2019 arXiv

  89. [129]

    Song, Y.; Liu, C.; Mao, W.; and Shou, M. Z. 2025. Mitty: Diffusion-based Human-to-Robot Video Generation. arXiv preprint arXiv:2512.17253

    2025

  90. [130]

    Sun, K.; Huang, K.; Liu, X.; Wu, Y.; Xu, Z.; Li, Z.; and Liu, X. 2025. T2v-compbench: A comprehensive benchmark for compositional text-to-video generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, 8406--8416

    2025

  91. [131]

    Team, K.; Chen, J.; Ci, Y.; Du, X.; Feng, Z.; Gai, K.; Guo, S.; Han, F.; He, J.; He, K.; et al. 2025 a . Kling-Omni Technical Report. arXiv preprint arXiv:2512.16776

    2025 arXiv

  92. [132]

    L.; Cai, X.; Huang, Q.; Kang, Z.; Li, H.; Liang, S.; Ma, L.; Ren, S.; Wei, X.; Xie, R.; et al

    Team, M. L.; Cai, X.; Huang, Q.; Kang, Z.; Li, H.; Liang, S.; Ma, L.; Ren, S.; Wei, X.; Xie, R.; et al. 2025 b . Longcat-video technical report. arXiv preprint arXiv:2510.22200

    2025

  93. [133]

    Wan, T.; Wang, A.; Ai, B.; Wen, B.; Mao, C.; Xie, C.-W.; Chen, D.; Yu, F.; Zhao, H.; Yang, J.; et al. 2025. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314

    2025 arXiv

  94. [134]

    Wang, D.; Li, R.; Han, F.; Ma, C.; Song, W.; Wang, S.; Wang, Y.; Xin, Y.; Liu, H.; Zhang, Z.; Ding, S.; Wang, T.; Cheng, Z.; Lin, T.; Jin, C.; Yu, K.; Chen, J.; Wang, W.; Wei, Z.; and Wang, J. 2026 a . DeepGen 1.0: A Lightweight Unified Multimodal Model for Advancing Image Gen...

    2026

  95. [135]

    Wang, D.; Ma, C.; Han, F.; Wu, S.; Song, W.; Wang, Y.; Zhang, Z.; Wang, T.; Wang, S.; Wei, Z.; and Wang, J. 2026 b . UniReason 1.0: A Unified Reasoning Framework for World Knowledge Aligned Image Generation and Editing. arXiv preprint arXiv:2602.02437

    2026

  96. [136]

    Wang, D.; Wei, R.; Shi, Y.; Xu, C.; Chen, S.; Luo, D.; Yang, T.; Yang, X.; Sui, W.; Qin, Y.; Tang, R.; and Mu, Y. 2026 c . Rein3D: Reinforced 3D Indoor Scene Generation with Panoramic Video Diffusion Models. arXiv preprint arXiv:2604.10578

    2026 arXiv

  97. [137]

    Wu, B.; Zou, C.; Li, C.; Huang, D.; Yang, F.; Tan, H.; Peng, J.; Wu, J.; Xiong, J.; Jiang, J.; et al. 2025. Hunyuanvideo 1.5 technical report. arXiv preprint arXiv:2511.18870

    2025 arXiv

  98. [138]

    xAI . 2026. Grok Imagine API. Official model release and API documentation

    2026

  99. [139]

    Xia, D.; Shi, Y.; Mu, Y.; Ji, H.; Ma, C.; Zhou, Y.; Chen, H.; Liu, Y.; Cao, J.; and Zhai, G. 2026. RoboProcessBench: Benchmarking Process-Aware Understanding in Vision-Language Robotic Manipulation. arXiv preprint arXiv:2606.13040

    2026 arXiv

  100. [140]

    Xie, S.; Cao, H.; Weng, Z.; Xing, Z.; Chen, H.; Shen, S.; Leng, J.; Wu, Z.; and Jiang, Y.-G. 2026. Human2robot: Learning robot actions from paired human-robot videos. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, 11078--11086

    2026

  101. [141]

    Yang, Y.; Ma, C.; Ju, C.; Zhang, F.; Yao, J.; Zhang, Y.; and Wang, Y. 2024 a . Multi-Modal Prototypes for Open-World Semantic Segmentation. International Journal of Computer Vision (IJCV)

    2024

  102. [142]

    Yang, Y.; Ma, C.; Mao, Z.; Yao, J.; Zhang, Y.; and Wang, Y. 2025. MoMa: Modulating Mamba for Adapting Image Foundation Models to Video Recognition. In Proceedings of the International Conference on Machine Learning (ICML)

    2025

  103. [143]

    Yang, Y.; Ma, C.; Yao, J.; Zhong, Z.; Zhang, Y.; and Wang, Y. 2024 b . ReMamber: Referring Image Segmentation with Mamba Twister. In European Conference on Computer Vision (ECCV)

    2024

  104. [144]

    Yang, Y.; Zhuang, X.; Cai, Y.; Ma, C.; Bai, S.; Yao, J.; Zhang, Y.; Lin, J.; and Wang, Y. 2026. GenMask: Adapting DiT for Segmentation via Direct Mask Generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2026

  105. [145]

    J.; Yang, J.; Peng, B.; Mandlekar, A.; Tan, R.; Chao, Y.-W.; Lin, B

    Ye, S.; Jang, J.; Jeon, B.; Joo, S. J.; Yang, J.; Peng, B.; Mandlekar, A.; Tan, R.; Chao, Y.-W.; Lin, B. Y.; et al. 2025. Latent action pretraining from videos. In International Conference on Learning Representations, volume 2025, 28213--28239

    2025

  106. [146]

    Zhang, J.; Chen, X.; Chen, A.; Lv, C.; Li, D.; Zhou, G.; Yin, H.; Yuan, H.; Li, H.; Li, J.; et al. 2026 a . Qwen-RobotWorld Technical Report: Unifying Embodied World Modeling through Language-Conditioned Video Generation. arXiv preprint arXiv:2606.17030

    2026 arXiv

  107. [147]

    Zhang, Y.; Dong, W.; Shi, Y.; Liang, Y.; Gao, J.; Yang, Q.; Lyu, Y.; Liang, Z.; Liu, Y.; Xu, C.; Guo, X.; Sui, W.; Jin, Y.; Yang, X.; Xu, Y.; and Mu, Y. 2026 b . R3DP: Real-Time 3D-Aware Policy for Embodied Manipulation. arXiv preprint arXiv:2603.14498

    2026

  108. [148]

    L.; Fu, L.; et al

    Zheng, R.; Niu, D.; Xie, Y.; Wang, J.; Xu, M.; Jiang, Y.; Casta \ n eda, F.; Hu, F.; Tan, Y. L.; Fu, L.; et al. 2026. Egoscale: Scaling dexterous manipulation with diverse egocentric human data. arXiv preprint arXiv:2602.16710

    2026

  109. [149]

    Zitkovich, B.; Yu, T.; Xu, S.; Xu, P.; Xiao, T.; Xia, F.; Wu, J.; Wohlhart, P.; Welker, S.; Wahid, A.; et al. 2023. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, 2165--2183. PMLR

    2023

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.