Pith. sign in

REVIEW 3 major objections 6 minor 112 references

AgentHOI claims that text alone can drive human-object interaction video generation: four agents write a second-by-second action plan, and an implicit text-motion alignment distills motion priors into the video model, removing the need for

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 05:22 UTC pith:YPRCK3U7

load-bearing objection AgentHOI is a real step toward text-only HOI video generation, but the evaluation overstates consistency and never measures the planner reliability it depends on. the 3 major comments →

arxiv 2607.22241 v2 pith:YPRCK3U7 submitted 2026-07-24 cs.CV

AgentHOI: Multi-Agent Reasoning for Human-Object-Interaction Video Generation via Implicit Representation Alignment

Ziyao Huang , Shunkai Li , Juan Cao , Chenyu Li , Youliang Zhang , Zixiang Zhou , Cong Wang , Yuan Zhou
show 2 more authors
Qinglin Lu Fan Tang
This is my paper
classification cs.CV
keywords human-object interaction video generationtext-driven controlmulti-agent reasoningimplicit motion alignmentaction timelinevideo diffusionsynthetic data training
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that human-object interaction (HOI) video can be generated from scratch with nothing but a text prompt and two reference images — one of a person, one of an object — with no pose, depth, 3D model, or motion capture needed at inference. The central bet is that a multi-agent reasoning layer can translate vague user intent into a physically grounded, second-by-second action plan, and that distilling a text-to-motion model's internal motion knowledge into the video generator, through relational feature alignment, teaches the generator to follow that plan. If true, it removes the main scalability bottleneck of earlier HOI methods, which required expensive explicit motion signals for every new object, and opens the door to arbitrary user-supplied objects and typed instructions. The paper reports consistent gains over previous state-of-the-art methods on object appearance preservation, motion smoothness, and adherence to multi-step instructions, backed by ablations, a new cross-motion test set, and user studies.

Core claim

AgentHOI claims that human-object interaction videos can be generated from scratch under purely textual control: given a reference photo of a person, a photo of an object, and a text prompt, the system synthesizes a short video of the person interacting with the object — wearing it, riding it, showing it, handing it between hands — without any pose, depth, 3D, or motion-capture input at inference. The text-only control is achieved in two stages. First, four specialized agents decompose the prompt into a second-by-second action timeline, grounding it in physical attributes extracted independently from the two images. Second, the video generation model is trained with an implicit text-motion a

What carries the argument

The load-bearing mechanism is the 'thinking-before-generation' pipeline. A multi-agent reasoning module — a visual-perception agent, an interaction-analysis agent, a motion planner, and a clipping refiner — converts the text prompt plus two reference images into a structured, second-by-second action timeline (one-second intervals, at most five seconds total). The visual-perception agent extracts physical attributes of the human and object separately, without assuming shared context; the interaction-analysis agent infers feasible interactions while treating the prompt as the highest-priority action anchor; the motion planner produces the temporal decomposition; and the clipping refiner rewrit

Load-bearing premise

The entire text-only control claim rests on the off-the-shelf vision-language model reliably extracting physical attributes from two unrelated images and writing a correct, physically grounded, second-by-second action timeline — and the paper itself concedes that VLM hallucinations can produce misidentified objects or missing actions that propagate into the generated video.

What would settle it

Take a set of object images with heavy occlusion, unusual textures, or ambiguous shapes that the vision-language model tends to misclassify, pair them with prompts that depend on the true object identity, and check whether AgentHOI's output video depicts the wrong object or drops the required action. A high failure rate on such inputs, or a strong correlation between manually rated plan errors and low interaction-quality scores, would show that the text-only control claim rides on the VLM's accuracy rather than on the video model's learned motion priors.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If correct, HOI video generation no longer depends on motion capture, pose sequences, depth, or 3D modeling at inference, so any photographable object can be animated from a typed instruction.
  • The second-by-second action timeline is an interpretable intermediate artifact: users could inspect, edit, or re-order the plan before generation, enabling human-in-the-loop control over interaction details.
  • Because the text-to-motion model is used only during training, inference cost and complexity remain those of a plain text-to-video model — the motion understanding is baked into the weights.
  • The hybrid real-synthetic recipe with high-noise-only synthetic sampling suggests a transferable way to use synthetic data to broaden object and interaction diversity without degrading fine texture quality.
  • Attaching the same agent pipeline to two existing text-driven baselines improved their object consistency and text adherence, indicating the planning module's benefit is not specific to the chosen video backbone.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the planning layer is the true bottleneck, swapping in a stronger or fine-tuned vision-language model should lift video quality without retraining the video generator; a systematic study relating plan correctness, measured by manual rating, to final video adherence would localize where errors originate.
  • The clipping-refiner idea — rewriting a plan into staged sub-actions (align, touch, guide, fit) to avoid penetration — could generalize to other generation tasks with contact, such as hand-object manipulation or furniture assembly, where semantic staging substitutes for geometric constraints.
  • Since the TRD alignment matches relational similarities rather than raw feature values, it may transfer across video backbones with different token layouts or parameter counts; testing whether a smaller backbone benefits from the same distillation would clarify the generality of the mechanism.
  • The fixed 5-second duration and 1-second granularity are current design constraints; extending the planner to longer horizons or variable granularity is a natural next step that the paper does not address.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes AgentHOI, a text-driven human-object-interaction (HOI) video generation framework. It follows a 'thinking-before-generation' schema: a four-agent reasoning module (visual perception, interaction analysis, motion planning, clipping refinement) converts a text prompt plus a human image and an object image into a second-by-second action timeline, which is then fed to a Wan2.1-I2V-14B-based video generator. To improve text-to-motion controllability without explicit motion inputs at inference, the method trains the video model with a token-relation-distillation (TRD) loss that aligns its intermediate spatial/temporal features with those of a text-to-motion model. The training corpus combines 71K filtered real video clips and 37K synthetic clips, with synthetic data used only at high-noise timesteps. Experiments are reported on a self-built 72-sample cross-motion benchmark, on the HOMA and AnchorCrafter test sets, and through a user study, claiming consistent improvements in object appearance preservation, motion smoothness, and multi-step instruction adherence.

Significance. If the claims hold, this is a practically significant systems contribution: it removes explicit pose/depth/motion-capture inputs at inference for HOI video generation, replacing them with an interpretable agent-generated action timeline and an implicit text-motion feature alignment. The paper has several genuine strengths: the four-agent decomposition is clearly specified with full system prompts in the appendix; the TRD alignment is ablated; the mixed real/synthetic data pipeline is described in unusual detail; the code is released; and the evaluation spans multiple benchmarks plus a user study. The ablations isolate the contributions of the agent module, the clipping refiner, the implicit alignment, and the synthetic-data strategy. However, the central 'purely textual control' claim depends on the reliability of VLM planning, and the quantitative evidence has important gaps that currently prevent full verification.

major comments (3)
  1. [Sec. 6 / Sec. 3.1 / Table 3] The central 'purely textual control' claim is not supported by any plan-quality measurement. At inference, the four-agent module is the only channel through which motion is specified; Sec. 6 concedes that VLM hallucinations can produce erroneous action timelines (e.g., misidentified objects or missing actions) and that the method does not enforce strict timing control. Table 3 only reports downstream video metrics for different planner VLMs; it never measures plan correctness, hallucination rate, or temporal faithfulness. Please add a plan-level evaluation on the 72-sample cross-motion set: object misidentification rates, missing/extra action rates, human-judged plan faithfulness, and per-segment timing correctness, together with a failure-mode analysis linking plan errors to video quality. Without this, the 'purely textual control' claim is not established beyond anecdotal examples.
  2. [Sec. 4.2 / Tables 1, 2, 5] No uncertainty quantification is provided. All quantitative metrics are point estimates; the cross-motion test set has only 72 samples, and several differences are plausibly within noise (e.g., Obj-DINO 0.510 vs 0.490 for w/o agent; MS 0.9957 vs 0.9977 for w/o clipping agent). The user study uses 30 participants and 40 trials per test set but reports only raw preference percentages without significance tests. On the HOMA self-driven set (Table 2a), HOMA outperforms the method on Obj-DINO (0.655 vs 0.612) and AES (0.586 vs 0.525), and the user-study preference is only 60-63%. Please provide per-metric confidence intervals or bootstrap estimates, and significance tests for both automatic metrics and user-study preferences, before claiming 'consistent' SOTA superiority.
  3. [Sec. 4.1.2 / Sec. 4.1.4 / Appendix B] The evaluation is partly self-referential, which weakens the external-validity claims. The cross-motion benchmark is self-built with only 72 samples and manually written text prompts; the HOMA and AnchorCrafter benchmarks have overlapping author sets; and the automatic VLM evaluator is from the same InternVL-3.5 family as the planner used in the proposed method. The protocol for manually annotating text prompts on the self-driven benchmarks is not described, and no inter-annotator agreement is reported. Because the planner and the evaluator share a model family, the reported InternVL scores may reward the planner's own output style. Please add an independent VLM evaluation or calibration against human judgments, provide the full annotation protocol and agreement statistics, and report results separately from the self-built benchmark so that the reader can assess generalization independen
minor comments (6)
  1. [Sec. 3.1.3 / Sec. 4.5] The strict one-second granularity and five-second maximum duration are design choices that are not ablated or discussed as limitations. Several multi-action prompts in Fig. 14 involve more than five natural seconds of activity; please discuss how the temporal cap affects these cases or provide an ablation.
  2. [Table 1, 'w/o clipping agent' row] Removing the clipping refiner gives a higher motion-smoothness score (MS 0.9977) than the full model (0.9957). The paper should explain why the refiner improves interaction quality but slightly lowers this metric, or whether this difference is within noise.
  3. [Table 4, 'VACE-14B+agent' row] VACE-14B+agent exceeds the full method on DD (0.990 vs 0.986) and InternVL-O (0.973 vs 0.970). Please clarify how this interacts with the claim that implicit text-motion alignment provides complementary gains beyond improved prompting.
  4. [Sec. 4.1.4] TVA is not computed for pose-driven methods, which makes cross-method comparison incomplete. Consider reporting a common text-alignment metric for all methods, or explicitly state this limitation in the metric description.
  5. [Appendix B] The VLM-based evaluation uses ten binary questions per dimension. Please report the correlation of these scores with human judgments, and include the exact full question set and prompting template in the supplementary material for reproducibility.
  6. [Abstract / Sec. 1] The word 'significantly' is used repeatedly without statistical support. Please temper the wording or add significance tests; otherwise the reader may over-read the quantitative margins.

Circularity Check

0 steps flagged

No significant circularity: the derivation is a training/distillation pipeline with independent benchmarks; self-citations are comparative, not load-bearing.

full rationale

AgentHOI's chain is: (1) a multi-agent VLM planner converts images+text into a second-by-second textual action timeline; (2) a video diffusion model is trained with a TRD auxiliary loss that aligns its intermediate features with those of a text-to-motion model M_motion trained on ground-truth skeleton/object-mask sequences; (3) M_motion is removed at inference. No equation in the paper defines the predicted quantity in terms of a fitted parameter: T_M is produced zero-shot by InternVL, and the video model's objective is L = L_FM + mu*L_TRD, with L_TRD comparing relative similarities between M and M_motion features—not between the output video and the input plan. The evaluation does use the authors' HOMA/AnchorCrafter benchmarks and InternVL-based metrics, and the same InternVL family appears in the data/planning pipeline; these are legitimate evaluation-design concerns but do not constitute a reduction of the claimed result to its inputs. The paper explicitly concedes VLM hallucination risk (Sec. 6), which is a reliability limitation, not circularity. Accordingly no self-definitional, fitted-prediction, or self-citation-load-bearing step can be exhibited from the text.

Axiom & Free-Parameter Ledger

5 free parameters · 8 axioms · 0 invented entities

The central claim rests on a stack of engineering choices and transfer assumptions rather than on a small set of mathematical axioms. The most load-bearing items are the reliability of the VLM planning layer and the transferability of skeleton/mask text-to-motion features to RGB-latent video features; both are supported only by the paper's own ablations. No new physical entities are introduced.

free parameters (5)
  • TRD loss weight μ = 0.45
    Chosen by the authors for the final objective L = L_FM + μ L_TRD (Eq. 5); no selection criterion or sensitivity analysis is given.
  • TRD alignment layers = {0,2,4,6,8,10,12,14}
    Selected for feature alignment in Sec. 3.2.2; ablations show high layers hurt, but the exact low-layer set is a design choice.
  • Synthetic-data timestep fraction = top 30% (t≳900)
    Synthetic data applied only at high-noise timesteps (Sec. 3.3.2); threshold is empirical.
  • Agent temporal granularity and max duration = 1 sec; ≤5 sec
    Motion planner is constrained to one-second intervals and five-second videos (Sec. 3.1.3); the paper does not test other granularities.
  • Training scale and data ratio = 71K real + 37K synthetic; 9,500 steps; 480p; lr 1e-6
    Dataset size and training budget are engineering choices; no scaling analysis is provided.
axioms (8)
  • domain assumption The Wan2.1-I2V-14B video diffusion backbone provides a sufficient base for HOI generation and can absorb text-to-motion priors.
    Used throughout Sec. 3.2; no proof that the backbone's text and image conditioning is adequate for the new task.
  • domain assumption Intermediate features of a skeleton/object-mask text-to-motion model encode transferable human-object motion priors that can be aligned to RGB-latent video diffusion features via TRD.
    Sec. 3.2.2 states "the hidden tokens of Y^M encode informative priors"; the paper relies on this transfer across representations without a formal argument.
  • standard math Spatial and temporal relative-token similarity (TRD loss from VideoRepa [67]) is an appropriate objective for aligning motion structure and does not distort appearance.
    Eqs. (2)-(4) adopt VideoRepa's relational alignment; the paper provides no correctness proof for this transfer setting.
  • domain assumption InternVL-3.5-38B reliably performs visual attribute extraction, affordance reasoning, and temporal planning.
    The whole agent pipeline (Sec. 3.1) is built on this; the paper acknowledges hallucination risk in Sec. 6.
  • domain assumption DWPose and SAM2 provide sufficiently accurate skeletons and object masks to serve as ground-truth motion videos Y_M.
    Sec. 4.1.1; errors in these upstream detectors propagate into M_motion and the TRD signal.
  • domain assumption Synthetic videos generated by Wan2.2-I2V are valid training targets for learning real HOI dynamics.
    Sec. 3.3.1 labels them "virtual GT videos"; the authors acknowledge degraded visual quality and restrict them to high-noise timesteps.
  • domain assumption The test sets are representative and do not overlap the 71K training clips.
    Sec. 4.1.2; no deduplication or overlap analysis is reported between the curated online training data and HOMA/AnchorCrafter test clips.
  • domain assumption Automated metrics (Obj-DINO, VBench, TVA, InternVL) and the user study measure interaction naturalness and text adherence.
    Sec. 4.1.4; no correlation analysis between VLM metrics and human preferences is given.

pith-pipeline@v1.3.0-alltime-deepseek · 31471 in / 18872 out tokens · 169732 ms · 2026-08-01T05:22:05.706900+00:00 · methodology

0 comments
read the original abstract

Recent advances in video diffusion models have spurred interest in human-object interaction (HOI) video generation, which demands fine-grained control over interaction logic beyond single-subject animation. However, existing HOI methods rely heavily on explicit motion control, limiting scalability and generalization across diverse objects and interactions. In this study, we propose AgentHOI, a text-driven HOI video generation following a thinking-before-generation framework that bridges the gap between high-level textual intent and physical execution through multi-agent reasoning over perception, interaction, and motion planning. Building upon the generated interaction plans, we further strengthen text-driven motion understanding. We introduce an implicit text-motion alignment strategy that distills text-to-motion priors into the video diffusion model, enabling robust HOI synthesis without explicit motion inputs at inference. Experiments show that AgentHOI significantly improves interaction naturalness, object appearance preservation, and adherence to complex textual instructions across challenging object-centric scenarios such as wearing and riding. The code is available at https://github.com/bone-11/agenthoi.

Figures

Figures reproduced from arXiv: 2607.22241 by Chenyu Li, Cong Wang, Fan Tang, Juan Cao, Qinglin Lu, Shunkai Li, Youliang Zhang, Yuan Zhou, Zixiang Zhou, Ziyao Huang.

Figure 1
Figure 1. Figure 1: Following a “thinking-before-generation” schema, the proposed AgentHOI first generates detailed textual motion controls and [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The pipeline for the proposed AgentHOI. Our framework consists of two core components: agent-based HOI reasoning and [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Visualization of real data pipeline and synthetic data pipeline. We build an HOI dataset via a hybrid real–synthetic pipeline. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Comparison with SOTAs. Video results can be found in the supplementary materials. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Comparison with SOTAs on HOMA test set. Video [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Ablation on the agent-based HOI reasoning and planning [PITH_FULL_IMAGE:figures/full_fig_p010_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Ablation on implicit text–motion representation align [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗
Figure 10
Figure 10. Figure 10: Ablation on mixed-source data and the high-noise syn [PITH_FULL_IMAGE:figures/full_fig_p011_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Results of training with only synthetic data, compared [PITH_FULL_IMAGE:figures/full_fig_p012_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Qualitative comparison of different VLMs (InternVL [PITH_FULL_IMAGE:figures/full_fig_p012_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Further comparisons by attaching our agent pipeline to [PITH_FULL_IMAGE:figures/full_fig_p013_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: More results on challenging HOI scenarios, including [PITH_FULL_IMAGE:figures/full_fig_p013_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Limitation. Due to resolution constraints, text rendered [PITH_FULL_IMAGE:figures/full_fig_p014_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Interface of the A/B user study system. In each trial, participants are shown the input human image, input object image, and [PITH_FULL_IMAGE:figures/full_fig_p019_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Overview of the automated real-data processing [PITH_FULL_IMAGE:figures/full_fig_p021_17.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

112 extracted references · 35 linked inside Pith

  1. [1]

    Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023. 2

    Pith/arXiv arXiv 2023

  2. [2]

    Z-image: An efficient image generation foundation model with single-stream diffusion transformer

    Huanqia Cai, Sihan Cao, Ruoyi Du, Peng Gao, Steven Hoi, Zhaohui Hou, Shijie Huang, Dengyang Jiang, Xin Jin, Liangchen Li, et al. Z-image: An efficient image generation foundation model with single-stream diffusion transformer. arXiv preprint arXiv:2511.22699, 2025. 13

    Pith/arXiv arXiv 2025

  3. [3]

    Humo: Human-centric video generation via collaborative multi-modal conditioning.arXiv preprint arXiv:2509.08519, 2025

    Liyang Chen, Tianxiang Ma, Jiawei Liu, Bingchuan Li, Zhuowei Chen, Lijie Liu, Xu He, Gen Li, Qian He, and Zhiyong Wu. Humo: Human-centric video generation via collaborative multi-modal conditioning.arXiv preprint arXiv:2509.08519, 2025. 3, 8, 12 14

    Pith/arXiv arXiv 2025

  4. [4]

    Multi-subject open-set personalization in video generation

    Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Yuwei Fang, Kwot Sin Lee, Ivan Skorokhodov, Kfir Aber- man, Jun-Yan Zhu, Ming-Hsuan Yang, and Sergey Tulyakov. Multi-subject open-set personalization in video generation. InProceedings of the Computer Vision and Pattern Recogni- tion Conference (CVPR), pages 6099–6110, 2025. 3

    2025

  5. [5]

    Hunyuanvideo-avatar: High-fidelity audio-driven hu- man animation for multiple characters.arXiv preprint arXiv:2505.20156, 2025

    Yi Chen, Sen Liang, Zixiang Zhou, Ziyao Huang, Yifeng Ma, Junshu Tang, Qin Lin, Yuan Zhou, and Qinglin Lu. Hunyuanvideo-avatar: High-fidelity audio-driven hu- man animation for multiple characters.arXiv preprint arXiv:2505.20156, 2025. 3

    Pith/arXiv arXiv 2025

  6. [6]

    Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks. InPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pages 24185–24198, 2024. 8

    2024

  7. [7]

    Echomimic: Lifelike audio-driven portrait animations through editable landmark conditions

    Zhiyuan Chen, Jiajiong Cao, Zhiquan Chen, Yuming Li, and Chenguang Ma. Echomimic: Lifelike audio-driven portrait animations through editable landmark conditions. InPro- ceedings of the AAAI Conference on Artificial Intelligence (AAAI), pages 2403–2410, 2025. 3

    2025

  8. [8]

    Wan-animate: Unified character animation and replacement with holistic replication.arXiv preprint arXiv:2509.14055, 2025

    Gang Cheng, Xin Gao, Li Hu, Siqi Hu, Mingyang Huang, Chaonan Ji, Ju Li, Dechao Meng, Jinwei Qi, Penchong Qiao, et al. Wan-animate: Unified character animation and replacement with holistic replication.arXiv preprint arXiv:2509.14055, 2025. 3

    arXiv 2025

  9. [9]

    Vlogger: Multimodal diffusion for embodied avatar synthe- sis.arXiv preprint arXiv:2403.08764, 2024

    Enric Corona, Andrei Zanfir, Eduard Gabriel Bazavan, Nikos Kolotouros, Thiemo Alldieck, and Cristian Sminchisescu. Vlogger: Multimodal diffusion for embodied avatar synthe- sis.arXiv preprint arXiv:2403.08764, 2024. 3

    Pith/arXiv arXiv 2024

  10. [10]

    Svimo: Synchronized diffu- sion for video and motion generation in hand-object interac- tion scenarios.arXiv preprint arXiv:2506.02444, 2025

    Lingwei Dang, Ruizhi Shao, Hongwen Zhang, Wei Min, Yebin Liu, and Qingyao Wu. Svimo: Synchronized diffu- sion for video and motion generation in hand-object interac- tion scenarios.arXiv preprint arXiv:2506.02444, 2025. 3

    Pith/arXiv arXiv 2025

  11. [11]

    Re-hold: Video hand object interaction reen- actment via adaptive layout-instructed diffusion model

    Yingying Fan, Quanwei Yang, Kaisiyuan Wang, Hang Zhou, Yingying Li, Haocheng Feng, Errui Ding, Yu Wu, and Jing- dong Wang. Re-hold: Video hand object interaction reen- actment via adaptive layout-instructed diffusion model. In Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR), pages 17550–17560, 2025. 2, 3

    2025

  12. [12]

    Imos: Intent-driven full-body motion synthesis for human-object interactions

    Anindita Ghosh, Rishabh Dabral, Vladislav Golyanik, Chris- tian Theobalt, and Philipp Slusallek. Imos: Intent-driven full-body motion synthesis for human-object interactions. In Computer Graphics Forum (CGF), pages 1–12. Wiley On- line Library, 2023. 3

    2023

  13. [13]

    Active intelligence in video avatars via closed-loop world modeling.arXiv preprint arXiv:2512.20615, 2025

    Xuanhua He, Tianyu Yang, Ke Cao, Ruiqi Wu, Cheng Meng, Yong Zhang, Zhuoliang Kang, Xiaoming Wei, and Qifeng Chen. Active intelligence in video avatars via closed-loop world modeling.arXiv preprint arXiv:2512.20615, 2025. 3

    arXiv 2025

  14. [14]

    Animate anyone: Consistent and controllable image- to-video synthesis for character animation

    Li Hu. Animate anyone: Consistent and controllable image- to-video synthesis for character animation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8153–8163, 2024. 2, 3

    2024

  15. [15]

    Animate anyone 2: High-fidelity character image animation with environment affordance.arXiv preprint arXiv:2502.06145, 2025

    Li Hu, Guangyuan Wang, Zhen Shen, Xin Gao, Dechao Meng, Lian Zhuo, Peng Zhang, Bang Zhang, and Liefeng Bo. Animate anyone 2: High-fidelity character image animation with environment affordance.arXiv preprint arXiv:2502.06145, 2025. 3

    Pith/arXiv arXiv 2025

  16. [16]

    Storyagent: Cus- tomized storytelling video generation via multi-agent collab- oration.arXiv preprint arXiv:2411.04925, 2024

    Panwen Hu, Jin Jiang, Jianqi Chen, Mingfei Han, Shengcai Liao, Xiaojun Chang, and Xiaodan Liang. Storyagent: Cus- tomized storytelling video generation via multi-agent collab- oration.arXiv preprint arXiv:2411.04925, 2024. 3

    Pith/arXiv arXiv 2024

  17. [17]

    Genmac: compositional text-to-video generation with multi-agent collaboration.arXiv preprint arXiv:2412.04440, 2024

    Kaiyi Huang, Yukun Huang, Xuefei Ning, Zinan Lin, Yu Wang, and Xihui Liu. Genmac: compositional text-to-video generation with multi-agent collaboration.arXiv preprint arXiv:2412.04440, 2024. 3

    Pith/arXiv arXiv 2024

  18. [18]

    Vbench: Comprehensive bench- mark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive bench- mark suite for video generative models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 21807–21818, 2024. 8

    2024

  19. [19]

    Make-your-anchor: A diffusion-based 2d avatar generation framework

    Ziyao Huang, Fan Tang, Yong Zhang, Xiaodong Cun, Juan Cao, Jintao Li, and Tong-Yee Lee. Make-your-anchor: A diffusion-based 2d avatar generation framework. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6997–7006, 2024. 3

    2024

  20. [20]

    Homa: Towards generic human-object interac- tion in multimodal driven human animation with weak con- ditions

    Ziyao Huang, Zixiang Zhou, Juan Cao, Yifeng Ma, Yi Chen, Zejing Rao, Zhiyong Xu, Hongmei Wang, Qin Lin, Yuan Zhou, et al. Homa: Towards generic human-object interac- tion in multimodal driven human animation with weak con- ditions. InProceedings of the SIGGRAPH Asia 2025 Con- ference Papers, pages 1–12, 2025. 2, 3, 7, 8

    2025

  21. [21]

    Sonic: Shifting focus to global audio perception in portrait animation.arXiv preprint arXiv:2411.16331, 2024

    Xiaozhong Ji, Xiaobin Hu, Zhihong Xu, Junwei Zhu, Chum- ing Lin, Qingdong He, Jiangning Zhang, Donghao Luo, Yi Chen, Qin Lin, et al. Sonic: Shifting focus to global audio perception in portrait animation.arXiv preprint arXiv:2411.16331, 2024. 3

    Pith/arXiv arXiv 2024

  22. [22]

    Omnihuman-1.5: Instilling an active mind in avatars via cognitive simulation.arXiv preprint arXiv:2508.19209, 2025

    Jianwen Jiang, Weihong Zeng, Zerong Zheng, Jiaqi Yang, Chao Liang, Wang Liao, Han Liang, Yuan Zhang, and Mingyuan Gao. Omnihuman-1.5: Instilling an active mind in avatars via cognitive simulation.arXiv preprint arXiv:2508.19209, 2025. 3

    Pith/arXiv arXiv 2025

  23. [23]

    Autonomous character-scene interaction synthesis from text instruction

    Nan Jiang, Zimo He, Zi Wang, Hongjie Li, Yixin Chen, Siyuan Huang, and Yixin Zhu. Autonomous character-scene interaction synthesis from text instruction. InSIGGRAPH Asia 2024 Conference Papers, pages 1–11, 2024. 3

    2024

  24. [24]

    Scaling up dynamic human-scene interaction mod- eling

    Nan Jiang, Zhiyuan Zhang, Hongjie Li, Xiaoxuan Ma, Zan Wang, Yixin Chen, Tengyu Liu, Yixin Zhu, and Siyuan Huang. Scaling up dynamic human-scene interaction mod- eling. InProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 1737– 1747, 2024. 3

    2024

  25. [25]

    Vace: All-in-one video creation and editing.arXiv preprint arXiv:2503.07598, 2025

    Zeyinzi Jiang, Zhen Han, Chaojie Mao, Jingfeng Zhang, Yulin Pan, and Yu Liu. Vace: All-in-one video creation and editing.arXiv preprint arXiv:2503.07598, 2025. 8, 12

    Pith/arXiv arXiv 2025

  26. [26]

    Auto-encoding variational bayes.arXiv preprint arXiv:1312.6114, 2013

    Diederik P Kingma. Auto-encoding variational bayes.arXiv preprint arXiv:1312.6114, 2013. 5

    Pith/arXiv arXiv 2013

  27. [27]

    Segment anything

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. InProceedings of the IEEE/CVF International 15 Conference on Computer Vision (ICCV), pages 4015–4026,

  28. [28]

    Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603, 2024

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603, 2024. 2

    Pith/arXiv arXiv 2024

  29. [29]

    Black Forest Labs, Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dock- horn, Jack English, Zion English, Patrick Esser, et al. Flux. 1 kontext: Flow matching for in-context image generation and editing in latent space.arXiv preprint arXiv:2506.15742,

    Pith/arXiv arXiv

  30. [30]

    Object motion guided human motion synthesis.ACM Transactions on Graphics (TOG), 42(6):1–11, 2023

    Jiaman Li, Jiajun Wu, and C Karen Liu. Object motion guided human motion synthesis.ACM Transactions on Graphics (TOG), 42(6):1–11, 2023. 3

    2023

  31. [31]

    Ho- comp: Interaction-aware human-object composition.Ad- vances in Neural Information Processing Systems (NeurIPS), 38:97617–97653, 2026

    Dong Liang, Jinyuan Jia, Yuhao Liu, and Rynson Lau. Ho- comp: Interaction-aware human-object composition.Ad- vances in Neural Information Processing Systems (NeurIPS), 38:97617–97653, 2026. 3

    2026

  32. [32]

    Cyberhost: Taming audio- driven avatar diffusion model with region codebook atten- tion.arXiv preprint arXiv:2409.01876, 2024

    Gaojie Lin, Jianwen Jiang, Chao Liang, Tianyun Zhong, Ji- aqi Yang, and Yanbo Zheng. Cyberhost: Taming audio- driven avatar diffusion model with region codebook atten- tion.arXiv preprint arXiv:2409.01876, 2024. 3

    Pith/arXiv arXiv 2024

  33. [33]

    Omnihuman-1: Rethinking the scaling-up of one-stage conditioned human animation models.arXiv preprint arXiv:2502.01061, 2025

    Gaojie Lin, Jianwen Jiang, Jiaqi Yang, Zerong Zheng, and Chao Liang. Omnihuman-1: Rethinking the scaling-up of one-stage conditioned human animation models.arXiv preprint arXiv:2502.01061, 2025. 3

    Pith/arXiv arXiv 2025

  34. [34]

    Flow matching for generative mod- eling.arXiv preprint arXiv:2210.02747, 2022

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximil- ian Nickel, and Matt Le. Flow matching for generative mod- eling.arXiv preprint arXiv:2210.02747, 2022. 6

    Pith/arXiv arXiv 2022

  35. [35]

    Improving video generation with human feedback.arXiv preprint arXiv:2501.13918, 2025

    Jie Liu, Gongye Liu, Jiajun Liang, Ziyang Yuan, Xiaokun Liu, Mingwu Zheng, Xiele Wu, Qiulin Wang, Wenyu Qin, Menghan Xia, et al. Improving video generation with human feedback.arXiv preprint arXiv:2501.13918, 2025. 8

    Pith/arXiv arXiv 2025

  36. [36]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. InEuro- pean Conference on Computer Vision (ECCV), pages 38–55. Springer, 2024. 6

    2024

  37. [37]

    Vista: A test- time self-improving video generation agent.arXiv preprint arXiv:2510.15831, 2025

    Do Xuan Long, Xingchen Wan, Hootan Nakhost, Chen-Yu Lee, Tomas Pfister, and Sercan ¨O Arık. Vista: A test- time self-improving video generation agent.arXiv preprint arXiv:2510.15831, 2025. 3

    arXiv 2025

  38. [38]

    Mimo: Controllable character video synthesis with spatial decomposed modeling

    Yifang Men, Yuan Yao, Miaomiao Cui, and Liefeng Bo. Mimo: Controllable character video synthesis with spatial decomposed modeling. InProceedings of the Computer Vision and Pattern Recognition Conference (CVPR), pages 21181–21191, 2025. 3

    2025

  39. [39]

    Echomimicv2: Towards striking, simplified, and semi-body human animation.arXiv preprint arXiv:2411.10061, 2024

    Rang Meng, Xingyu Zhang, Yuming Li, and Chenguang Ma. Echomimicv2: Towards striking, simplified, and semi-body human animation.arXiv preprint arXiv:2411.10061, 2024. 3

    arXiv 2024

  40. [40]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 8

    Pith/arXiv arXiv 2023

  41. [41]

    Manivideo: Generating hand-object manipulation video with dexterous and generalizable grasping

    Youxin Pang, Ruizhi Shao, Jiajun Zhang, Hanzhang Tu, Yun Liu, Boyao Zhou, Hongwen Zhang, and Yebin Liu. Manivideo: Generating hand-object manipulation video with dexterous and generalizable grasping. InProceedings of the Computer Vision and Pattern Recognition Conference (CVPR), pages 12209–12219, 2025. 2, 3

    2025

  42. [42]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF interna- tional conference on computer vision (ICCV), pages 4195– 4205, 2023. 5

    2023

  43. [43]

    Hoi-diff: Text-driven synthe- sis of 3d human-object interactions using diffusion models

    Xiaogang Peng, Yiming Xie, Zizhao Wu, Varun Jampani, Deqing Sun, and Huaizu Jiang. Hoi-diff: Text-driven synthe- sis of 3d human-object interactions using diffusion models. arXiv preprint arXiv:2312.06553, 2023. 3

    Pith/arXiv arXiv 2023

  44. [44]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. InInternational conference on machine learning (ICML), pages 8748–8763. PMLR, 2021. 5

    2021

  45. [45]

    Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of machine learning research, 21(140):1–67, 2020

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of machine learning research, 21(140):1–67, 2020. 5

    2020

  46. [46]

    Sam 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman R¨adle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024. 6, 7

    Pith/arXiv arXiv 2024

  47. [47]

    Human4dit: 360-degree human video gen- eration with 4d diffusion transformer.ACM Transactions on Graphics (TOG), 43(6), 2024

    Ruizhi Shao, Youxin Pang, Zerong Zheng, Jingxiang Sun, and Yebin Liu. Human4dit: 360-degree human video gen- eration with 4d diffusion transformer.ACM Transactions on Graphics (TOG), 43(6), 2024. 3

    2024

  48. [48]

    idit-hoi: Inpainting-based hand object interaction reenactment via video diffusion trans- former.arXiv preprint arXiv:2506.12847, 2025

    Zhelun Shen, Chenming Wu, Junsheng Zhou, Chen Zhao, Kaisiyuan Wang, Hang Zhou, Yingying Li, Haocheng Feng, Wei He, and Jingdong Wang. idit-hoi: Inpainting-based hand object interaction reenactment via video diffusion trans- former.arXiv preprint arXiv:2506.12847, 2025. 3

    Pith/arXiv arXiv 2025

  49. [49]

    Roformer: Enhanced transformer with rotary position embedding.Neurocomputing, 568:127063,

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding.Neurocomputing, 568:127063,

  50. [50]

    Animate-x: Universal character image an- imation with enhanced motion representation.ICLR 2025,

    Shuai Tan, Biao Gong, Xiang Wang, Shiwei Zhang, Dandan Zheng, Ruobing Zheng, Kecheng Zheng, Jingdong Chen, and Ming Yang. Animate-x: Universal character image an- imation with enhanced motion representation.ICLR 2025,

    2025

  51. [51]

    Emo: Emote portrait alive generating expressive portrait videos with audio2video diffusion model under weak conditions

    Linrui Tian, Qi Wang, Bang Zhang, and Liefeng Bo. Emo: Emote portrait alive generating expressive portrait videos with audio2video diffusion model under weak conditions. InEuropean Conference on Computer Vision (ECCV), pages 244–260. Springer, 2024. 3

    2024

  52. [52]

    Stableanimator: High- 16 quality identity-preserving human image animation.arXiv preprint arXiv:2411.17697, 2024

    Shuyuan Tu, Zhen Xing, Xintong Han, Zhi-Qi Cheng, Qi Dai, Chong Luo, and Zuxuan Wu. Stableanimator: High- 16 quality identity-preserving human image animation.arXiv preprint arXiv:2411.17697, 2024. 3

    Pith/arXiv arXiv 2024

  53. [53]

    Wan: Open and advanced large-scale video gen- erative models.arXiv preprint arXiv:2503.20314, 2025

    Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, et al. Wan: Open and advanced large-scale video gen- erative models.arXiv preprint arXiv:2503.20314, 2025. 2, 5, 8

    Pith/arXiv arXiv 2025

  54. [54]

    Dreamactor-h1: High-fidelity human- product demonstration video generation via motion-designed diffusion transformers.arXiv preprint arXiv:2506.10568,

    Lizhen Wang, Zhurong Xia, Tianshu Hu, Pengrui Wang, Pengfei Wei, Zerong Zheng, Ming Zhou, Yuan Zhang, and Mingyuan Gao. Dreamactor-h1: High-fidelity human- product demonstration video generation via motion-designed diffusion transformers.arXiv preprint arXiv:2506.10568,

    Pith/arXiv arXiv

  55. [55]

    Unianimate: Taming unified video diffusion mod- els for consistent human image animation.arXiv preprint arXiv:2406.01188, 2024

    Xiang Wang, Shiwei Zhang, Changxin Gao, Jiayu Wang, Xiaoqiang Zhou, Yingya Zhang, Luxin Yan, and Nong Sang. Unianimate: Taming unified video diffusion mod- els for consistent human image animation.arXiv preprint arXiv:2406.01188, 2024. 3

    Pith/arXiv arXiv 2024

  56. [56]

    Unianimate-dit: Human image animation with large-scale video diffusion transformer.arXiv preprint arXiv:2504.11289, 2025

    Xiang Wang, Shiwei Zhang, Longxiang Tang, Yingya Zhang, Changxin Gao, Yuehuan Wang, and Nong Sang. Unianimate-dit: Human image animation with large-scale video diffusion transformer.arXiv preprint arXiv:2504.11289, 2025. 3, 8

    arXiv 2025

  57. [57]

    Humanvid: Demystifying training data for camera-controllable human image animation

    Zhenzhi Wang, Yixuan Li, Yanhong Zeng, Youqing Fang, Yuwei Guo, Wenran Liu, Jing Tan, Kai Chen, Tianfan Xue, Bo Dai, et al. Humanvid: Demystifying training data for camera-controllable human image animation. InThe Thirty- eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024. 3

    2024

  58. [58]

    Dreamfactory: Pio- neering multi-scene long video generation with a multi-agent framework.arXiv preprint arXiv:2408.11788, 2024

    Zhifei Xie, Daniel Tang, Dingwei Tan, Jacques Klein, Tegawend F Bissyand, and Saad Ezzini. Dreamfactory: Pio- neering multi-scene long video generation with a multi-agent framework.arXiv preprint arXiv:2408.11788, 2024. 3

    Pith/arXiv arXiv 2024

  59. [59]

    Interdiff: Generating 3d human-object interactions with physics-informed diffusion

    Sirui Xu, Zhengyuan Li, Yu-Xiong Wang, and Liang-Yan Gui. Interdiff: Generating 3d human-object interactions with physics-informed diffusion. InProceedings of the IEEE/CVF International Conference on Computer Vision (CVPR), pages 14928–14940, 2023. 3

    2023

  60. [60]

    Anchorcrafter: Animate cyberanchors saling your products via human-object interacting video generation

    Ziyi Xu, Ziyao Huang, Juan Cao, Yong Zhang, Xiaodong Cun, Qing Shuai, Yuchen Wang, Linchao Bao, Jintao Li, and Fan Tang. Anchorcrafter: Animate cyberanchors saling your products via human-object interacting video generation. arXiv preprint arXiv:2411.17383, 2024. 2, 3, 7, 8

    Pith/arXiv arXiv 2024

  61. [61]

    Hoi-swap: Swapping objects in videos with hand- object interaction awareness.Advances in Neural Informa- tion Processing Systems (NeurIPS), 37:77132–77164, 2024

    Zihui Sherry Xue, Romy Luo, Changan Chen, and Kristen Grauman. Hoi-swap: Swapping objects in videos with hand- object interaction awareness.Advances in Neural Informa- tion Processing Systems (NeurIPS), 37:77132–77164, 2024. 2, 3

    2024

  62. [62]

    Effec- tive whole-body pose estimation with two-stages distillation

    Zhendong Yang, Ailing Zeng, Chun Yuan, and Yu Li. Effec- tive whole-body pose estimation with two-stages distillation. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCVW), pages 4210–4220, 2023. 7

    2023

  63. [63]

    Representation alignment for generation: Training diffu- sion transformers is easier than you think.arXiv preprint arXiv:2410.06940, 2024

    Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, and Saining Xie. Representation alignment for generation: Training diffu- sion transformers is easier than you think.arXiv preprint arXiv:2410.06940, 2024. 5

    Pith/arXiv arXiv 2024

  64. [64]

    Mora: Enabling generalist video generation via a multi-agent framework.arXiv preprint arXiv:2403.13248,

    Zhengqing Yuan, Yixin Liu, Yihan Cao, Weixiang Sun, Hao- long Jia, Ruoxi Chen, Zhaoxu Li, Bin Lin, Li Yuan, Lifang He, et al. Mora: Enabling generalist video generation via a multi-agent framework.arXiv preprint arXiv:2403.13248,

    Pith/arXiv arXiv

  65. [65]

    Extracting motion and ap- pearance via inter-frame attention for efficient video frame interpolation

    Guozhen Zhang, Yuhan Zhu, Haonan Wang, Youxin Chen, Gangshan Wu, and Limin Wang. Extracting motion and ap- pearance via inter-frame attention for efficient video frame interpolation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5682– 5692, 2023. 7

    2023

  66. [66]

    Sadtalker: Learning realistic 3d motion coefficients for stylized audio- driven single image talking face animation

    Wenxuan Zhang, Xiaodong Cun, Xuan Wang, Yong Zhang, Xi Shen, Yu Guo, Ying Shan, and Fei Wang. Sadtalker: Learning realistic 3d motion coefficients for stylized audio- driven single image talking face animation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 8652–8661, 2023. 3

    2023

  67. [67]

    Vide- orepa: Learning physics for video generation through re- lational alignment with foundation models.arXiv preprint arXiv:2505.23656, 2025

    Xiangdong Zhang, Jiaqi Liao, Shaofeng Zhang, Fanqing Meng, Xiangpeng Wan, Junchi Yan, and Yu Cheng. Vide- orepa: Learning physics for video generation through re- lational alignment with foundation models.arXiv preprint arXiv:2505.23656, 2025. 5, 6

    Pith/arXiv arXiv 2025

  68. [68]

    Mim- icmotion: High-quality human motion video generation with confidence-aware pose guidance.arXiv preprint arXiv:2406.19680, 2024

    Yuang Zhang, Jiaxi Gu, Li-Wen Wang, Han Wang, Junqi Cheng, Yuefeng Zhu, and Fangyuan Zou. Mim- icmotion: High-quality human motion video generation with confidence-aware pose guidance.arXiv preprint arXiv:2406.19680, 2024. 2, 3

    Pith/arXiv arXiv 2024

  69. [69]

    Social agent: Mastering dyadic nonverbal behavior generation via conversational llm agents

    Zeyi Zhang, Yanju Zhou, Heyuan Yao, Tenglong Ao, Xiao- hang Zhan, and Libin Liu. Social agent: Mastering dyadic nonverbal behavior generation via conversational llm agents. InProceedings of the SIGGRAPH Asia 2025 Conference Pa- pers, pages 1–12, 2025. 3

    2025

  70. [70]

    Videogen-of-thought: Step-by-step generating multi-shot video with minimal manual interven- tion.arXiv preprint arXiv:2412.02259, 2024

    Mingzhe Zheng, Yongqi Xu, Haojian Huang, Xuran Ma, Yexin Liu, Wenjie Shu, Yatian Pang, Feilong Tang, Qifeng Chen, Harry Yang, et al. Videogen-of-thought: Step-by-step generating multi-shot video with minimal manual interven- tion.arXiv preprint arXiv:2412.02259, 2024. 3

    arXiv 2024

  71. [71]

    Al- legro: Open the black box of commercial-level video gener- ation model.arXiv preprint arXiv:2410.15458, 2024

    Yuan Zhou, Qiuyue Wang, Yuxuan Cai, and Huan Yang. Al- legro: Open the black box of commercial-level video gener- ation model.arXiv preprint arXiv:2410.15458, 2024. 2

    Pith/arXiv arXiv 2024

  72. [72]

    Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models.arXiv preprint arXiv:2504.10479, 2025

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shen- glong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models.arXiv preprint arXiv:2504.10479, 2025. 8

    Pith/arXiv arXiv 2025

  73. [73]

    aligning backpack opening with shoulder area

    Shenhao Zhu, Junming Leo Chen, Zuozhuo Dai, Yinghui Xu, Xun Cao, Yao Yao, Hao Zhu, and Siyu Zhu. Champ: Controllable and consistent human image animation with 3d parametric guidance. InEuropean Conference on Computer Vision (ECCV), 2024. 3 17 A. Details of User Study Interface We implement the user study as a web-based two- alternative forced-choice (2AFC...

    2024

  74. [74]

    Image-1: Human image (standalone)

  75. [75]

    human_physical_attributes

    Image-2: Object image (standalone) -------------------------------- Extraction Scope -------------------------------- Extract ONLY information that is: - directly observable in the given image - physically grounded - independent of the other image - free of interaction or intention assumptions -------------------------------- Human Physical Attributes (St...

  76. [76]

    Human-Object Compatibility Reasoning Based ONLY on provided attributes, reason about: - Whether the human pose and hand state support interaction - Whether object scale and affordances allow engagement - Whether spatial relations make interaction feasible Focus on consistency, not perception

  77. [77]

    Plausible Interaction Hypotheses (Non- exclusive) Infer interaction(s) that could reasonably happen next: Examples: - grasping / holding - operating / opening / pressing - wearing / applying - lifting / carrying - presenting / showing - preparing for interaction (pre-grasp, pre-use) These are hypothetical affordance-driven interactions, not labels of obse...

  78. [78]

    Intention-Level Inference Using human role, object function, and scene context, infer the most likely human goal IF interaction occurs, such as: - using the object - inspecting or examining - presenting or demonstrating - adjusting or positioning - preparing for a subsequent action This intention is speculative but grounded in constraints

  79. [79]

    Your task is NOT to invent new actions

    Downstream HOI Generation Support Explicitly reason about: - how interaction could start - what motion primitives would be required - what constraints must be respected (hand occupancy, stability, orientation) -------------------------------- Action Alignment Constraint (Highest Priority) -------------------------------- You must treat the init_prompt as ...

  80. [80]

    Extract explicit action verbs from init_prompt

Showing first 80 references.