{"id":"cad9eb48-91e4-4d74-aeb3-6bebdec4ce8a","arxiv_id":"2607.22241","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"AgentHOI generates human-object interaction videos from text plus one human image and one object image, using multi-agent action planning and implicit text-to-motion feature alignment inside a video diffusion model.","lead":"AgentHOI is a video-generation system that turns a person photo, an object photo, and a text description into a short video of the person using that object. It first runs several AI agents that write a second-by-second action plan, then distills text-to-motion knowledge into a video diffusion model so no pose or motion-capture data is needed at run time.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Text-only control claim hinges on unquantified VLM planning reliability; the paper concedes hallucinations can corrupt action timelines, yet provides no plan-quality or failure-rate analysis.","rationale":"The reader's weakest_assumption correctly identifies VLM planning reliability as the critical dependency of the central text-only control claim. I agree: the agent-based reasoning module is the only motion-specifying component at inference, so if its output is frequently wrong, the pipeline cannot fulfill the user's intent regardless of the video generator's quality. The paper itself flags this in §6, making it an explicit, self-admitted limitation. The strongest alternative concern — that 'consistently outperforms SOTAs' is contradicted by HOMA results — is real but slightly less central: it affects the strength of the empirical claim, whereas VLM reliability affects whether the mechanism works at all. The proposed gold-timeline comparison directly isolates the planning layer's contribution and would settle whether hallucinations are merely occasional noise or a systematic bottleneck. This concern is addressable through additional analysis, so it does not change the reader's conditional verdict; it strengthens the conditions under which the paper should be accepted.","tokens_in":31765,"tokens_out":7226,"duration_ms":76571,"concrete_test":"Take the 72-sample cross-motion-driven test set (or a random subset of the self-driven sets). For each prompt, generate two versions of AgentHOI output: (A) using the full agent-produced timeline, and (B) using a human-verified gold timeline that a trained annotator writes from the same text prompt and input images, keeping the video generator and all other settings identical. Compare downstream metrics (Obj-DINO, TVA, InternVL-I, and a small user study) between A and B. If B significantly outperforms A, the VLM planning step is a bottleneck and the text-only claim is fragile; if A ≈ B, hallucination errors are not load-bearing and the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing link in AgentHOI is the agent-based reasoning module (§3.1): at inference it is the only channel through which motion is specified. The paper's central claim — 'generates HOI videos from scratch under purely textual control' (§2.2) — holds only if the VLM-produced second-by-second action timeline is physically grounded and faithful to the prompt. Section 6 concedes the opposite: 'VLM hallucinations in the multi-agent pipeline can occasionally lead to erroneous action timelines (e.g., misidentified objects or missing actions) that propagate into the generated video.' But no measurement is given of how often this occurs or how much it degrades downstream video quality. Table 3 only compares downstream metrics across different planner VLMs; it does not evaluate plan correctness. The HOMA self-driven results (Table 2a) already hint at fragility: AgentHOI scores below HOMA on Obj-DINO (0.612 vs 0.655) and AES (0.525 vs 0.586), and the user-study preference against HOMA is only 60–63%. If timeline hallucination rates are non-negligible, the 'text-only control' claim is unreliable and the system's average performance may be carried by the video generator rather than by the reasoning layer.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AgentHOI, a text-driven human-object-interaction (HOI) video generation framework. It follows a 'thinking-before-generation' schema: a four-agent reasoning module (visual perception, interaction analysis, motion planning, clipping refinement) converts a text prompt plus a human image and an object image into a second-by-second action timeline, which is then fed to a Wan2.1-I2V-14B-based video generator. To improve text-to-motion controllability without explicit motion inputs at inference, the method trains the video model with a token-relation-distillation (TRD) loss that aligns its intermediate spatial/temporal features with those of a text-to-motion model. The training corpus combines 71K filtered real video clips and 37K synthetic clips, with synthetic data used only at high-noise timesteps. Experiments are reported on a self-built 72-sample cross-motion benchmark, on the HOMA and AnchorCrafter test sets, and through a user study, claiming consistent improvements in object appearance preservation, motion smoothness, and multi-step instruction adherence.","tokens_in":32088,"tokens_out":7197,"duration_ms":70701,"significance":"If the claims hold, this is a practically significant systems contribution: it removes explicit pose/depth/motion-capture inputs at inference for HOI video generation, replacing them with an interpretable agent-generated action timeline and an implicit text-motion feature alignment. The paper has several genuine strengths: the four-agent decomposition is clearly specified with full system prompts in the appendix; the TRD alignment is ablated; the mixed real/synthetic data pipeline is described in unusual detail; the code is released; and the evaluation spans multiple benchmarks plus a user study. The ablations isolate the contributions of the agent module, the clipping refiner, the implicit alignment, and the synthetic-data strategy. However, the central 'purely textual control' claim depends on the reliability of VLM planning, and the quantitative evidence has important gaps that currently prevent full verification.","major_comments":[{"comment":"The central 'purely textual control' claim is not supported by any plan-quality measurement. At inference, the four-agent module is the only channel through which motion is specified; Sec. 6 concedes that VLM hallucinations can produce erroneous action timelines (e.g., misidentified objects or missing actions) and that the method does not enforce strict timing control. Table 3 only reports downstream video metrics for different planner VLMs; it never measures plan correctness, hallucination rate, or temporal faithfulness. Please add a plan-level evaluation on the 72-sample cross-motion set: object misidentification rates, missing/extra action rates, human-judged plan faithfulness, and per-segment timing correctness, together with a failure-mode analysis linking plan errors to video quality. Without this, the 'purely textual control' claim is not established beyond anecdotal examples.","section":"Sec. 6 / Sec. 3.1 / Table 3"},{"comment":"No uncertainty quantification is provided. All quantitative metrics are point estimates; the cross-motion test set has only 72 samples, and several differences are plausibly within noise (e.g., Obj-DINO 0.510 vs 0.490 for w/o agent; MS 0.9957 vs 0.9977 for w/o clipping agent). The user study uses 30 participants and 40 trials per test set but reports only raw preference percentages without significance tests. On the HOMA self-driven set (Table 2a), HOMA outperforms the method on Obj-DINO (0.655 vs 0.612) and AES (0.586 vs 0.525), and the user-study preference is only 60-63%. Please provide per-metric confidence intervals or bootstrap estimates, and significance tests for both automatic metrics and user-study preferences, before claiming 'consistent' SOTA superiority.","section":"Sec. 4.2 / Tables 1, 2, 5"},{"comment":"The evaluation is partly self-referential, which weakens the external-validity claims. The cross-motion benchmark is self-built with only 72 samples and manually written text prompts; the HOMA and AnchorCrafter benchmarks have overlapping author sets; and the automatic VLM evaluator is from the same InternVL-3.5 family as the planner used in the proposed method. The protocol for manually annotating text prompts on the self-driven benchmarks is not described, and no inter-annotator agreement is reported. Because the planner and the evaluator share a model family, the reported InternVL scores may reward the planner's own output style. Please add an independent VLM evaluation or calibration against human judgments, provide the full annotation protocol and agreement statistics, and report results separately from the self-built benchmark so that the reader can assess generalization independen","section":"Sec. 4.1.2 / Sec. 4.1.4 / Appendix B"}],"minor_comments":[{"comment":"The strict one-second granularity and five-second maximum duration are design choices that are not ablated or discussed as limitations. Several multi-action prompts in Fig. 14 involve more than five natural seconds of activity; please discuss how the temporal cap affects these cases or provide an ablation.","section":"Sec. 3.1.3 / Sec. 4.5"},{"comment":"Removing the clipping refiner gives a higher motion-smoothness score (MS 0.9977) than the full model (0.9957). The paper should explain why the refiner improves interaction quality but slightly lowers this metric, or whether this difference is within noise.","section":"Table 1, 'w/o clipping agent' row"},{"comment":"VACE-14B+agent exceeds the full method on DD (0.990 vs 0.986) and InternVL-O (0.973 vs 0.970). Please clarify how this interacts with the claim that implicit text-motion alignment provides complementary gains beyond improved prompting.","section":"Table 4, 'VACE-14B+agent' row"},{"comment":"TVA is not computed for pose-driven methods, which makes cross-method comparison incomplete. Consider reporting a common text-alignment metric for all methods, or explicitly state this limitation in the metric description.","section":"Sec. 4.1.4"},{"comment":"The VLM-based evaluation uses ten binary questions per dimension. Please report the correlation of these scores with human judgments, and include the exact full question set and prompting template in the supplementary material for reproducibility.","section":"Appendix B"},{"comment":"The word 'significantly' is used repeatedly without statistical support. Please temper the wording or add significance tests; otherwise the reader may over-read the quantitative margins.","section":"Abstract / Sec. 1"}],"recommendation":"major_revision","confidential_remarks":"This is a solid engineering contribution with clear components and useful ablations, but the evaluation currently has three load-bearing problems: no plan-quality measurement for the central text-only claim, no uncertainty quantification anywhere, and a self-referential evaluation loop (self-built benchmark, overlapping author benchmarks, and same-family VLM as planner and evaluator). I would ask the authors to add the plan-level evaluation, confidence intervals/significance tests, and an independent evaluator before publication. If these are provided, the contribution is likely acceptable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a genuine systems contribution, but its headline claim is a bit ahead of the numbers. The new combination is multi-agent HOI planning (perception, relation, motion, clipping refiner) plus TRD-based distillation of text-to-motion features into a Wan2.1 I2V backbone, which removes explicit motion input at inference. That is worth having, and the ablations are mostly well done: the clipping refiner is a clever, cheap fix for penetration artifacts; showing the agent pipeline also lifts VACE-14B and HUMO is a nice control; the mixed real/synthetic training with high-noise-only synthetic data is sensible.\n\nThe soft spots are concentrated in evaluation and in the load-bearing assumption about the planner. The paper concedes in Section 6 that VLM hallucinations can produce erroneous action timelines, but it never quantifies how often this happens or how much it degrades output. Since the planner is the only motion channel at inference, that is not a minor detail — it is the central reliability question. The HOMA self-driven results are mixed: lower Obj-DINO and AES than the motion-conditioned HOMA baseline, and user preference of only 60–63%. That undercuts “consistently outperforms.” Also, no error bars or significance tests appear anywhere, and the VLM-based metrics use the same model family (InternVL) that generates the plans, so there is a real risk of rewarding the pipeline's own style. The self-built 72-sample cross-motion test set and manual prompt annotation are okay for a capability demo, but not enough for sweeping claims.\n\nNone of this is fatal. The method works, the design is coherent, and the limitations section is honest. But the paper needs plan-quality metrics, significance tests, and ideally an external human evaluation of the VLM metrics before acceptance. I'd send it to peer review rather than desk reject.\n\nWho should read it: people working on HOI video generation, text-conditioned human animation, and agent-based video pipelines. I'd probably cite it, and it's a reasonable reading-group paper.","headline":"AgentHOI is a real step toward text-only HOI video generation, but the evaluation overstates consistency and never measures the planner reliability it depends on.","tokens_in":32580,"tokens_out":3618,"would_cite":true,"duration_ms":35814,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AgentHOI claims that text alone can drive human-object interaction video generation: four agents write a second-by-second action plan, and an implicit text-motion alignment distills motion priors into the video model, removing the need for","keywords":["human-object interaction video generation","text-driven control","multi-agent reasoning","implicit motion alignment","action timeline","video diffusion","synthetic data training"],"falsifier":"Take a set of object images with heavy occlusion, unusual textures, or ambiguous shapes that the vision-language model tends to misclassify, pair them with prompts that depend on the true object identity, and check whether AgentHOI's output video depicts the wrong object or drops the required action. A high failure rate on such inputs, or a strong correlation between manually rated plan errors and low interaction-quality scores, would show that the text-only control claim rides on the VLM's accuracy rather than on the video model's learned motion priors.","tokens_in":31669,"feed_emoji":"🎬","tokens_out":4708,"duration_ms":47895,"temperature":0.7,"pith_summary":"This paper tries to establish that human-object interaction (HOI) video can be generated from scratch with nothing but a text prompt and two reference images — one of a person, one of an object — with no pose, depth, 3D model, or motion capture needed at inference. The central bet is that a multi-agent reasoning layer can translate vague user intent into a physically grounded, second-by-second action plan, and that distilling a text-to-motion model's internal motion knowledge into the video generator, through relational feature alignment, teaches the generator to follow that plan. If true, it removes the main scalability bottleneck of earlier HOI methods, which required expensive explicit motion signals for every new object, and opens the door to arbitrary user-supplied objects and typed instructions. The paper reports consistent gains over previous state-of-the-art methods on object appearance preservation, motion smoothness, and adherence to multi-step instructions, backed by ablations, a new cross-motion test set, and user studies.","feed_headline":"Text alone can now drive human-object interaction video","feed_subtitle":"A multi-agent planner writes a second-by-second script; the video model learns motion without pose or depth input.","key_machinery":"The load-bearing mechanism is the 'thinking-before-generation' pipeline. A multi-agent reasoning module — a visual-perception agent, an interaction-analysis agent, a motion planner, and a clipping refiner — converts the text prompt plus two reference images into a structured, second-by-second action timeline (one-second intervals, at most five seconds total). The visual-perception agent extracts physical attributes of the human and object separately, without assuming shared context; the interaction-analysis agent infers feasible interactions while treating the prompt as the highest-priority action anchor; the motion planner produces the temporal decomposition; and the clipping refiner rewrit","core_discovery":"AgentHOI claims that human-object interaction videos can be generated from scratch under purely textual control: given a reference photo of a person, a photo of an object, and a text prompt, the system synthesizes a short video of the person interacting with the object — wearing it, riding it, showing it, handing it between hands — without any pose, depth, 3D, or motion-capture input at inference. The text-only control is achieved in two stages. First, four specialized agents decompose the prompt into a second-by-second action timeline, grounding it in physical attributes extracted independently from the two images. Second, the video generation model is trained with an implicit text-motion a","pith_inferences":["If the planning layer is the true bottleneck, swapping in a stronger or fine-tuned vision-language model should lift video quality without retraining the video generator; a systematic study relating plan correctness, measured by manual rating, to final video adherence would localize where errors originate.","The clipping-refiner idea — rewriting a plan into staged sub-actions (align, touch, guide, fit) to avoid penetration — could generalize to other generation tasks with contact, such as hand-object manipulation or furniture assembly, where semantic staging substitutes for geometric constraints.","Since the TRD alignment matches relational similarities rather than raw feature values, it may transfer across video backbones with different token layouts or parameter counts; testing whether a smaller backbone benefits from the same distillation would clarify the generality of the mechanism.","The fixed 5-second duration and 1-second granularity are current design constraints; extending the planner to longer horizons or variable granularity is a natural next step that the paper does not address."],"forward_implications":["If correct, HOI video generation no longer depends on motion capture, pose sequences, depth, or 3D modeling at inference, so any photographable object can be animated from a typed instruction.","The second-by-second action timeline is an interpretable intermediate artifact: users could inspect, edit, or re-order the plan before generation, enabling human-in-the-loop control over interaction details.","Because the text-to-motion model is used only during training, inference cost and complexity remain those of a plain text-to-video model — the motion understanding is baked into the weights.","The hybrid real-synthetic recipe with high-noise-only synthetic sampling suggests a transferable way to use synthetic data to broaden object and interaction diversity without degrading fine texture quality.","Attaching the same agent pipeline to two existing text-driven baselines improved their object consistency and text adherence, indicating the planning module's benefit is not specific to the chosen video backbone."],"fun_headline_variants":["Text-only prompts now generate human-object interaction videos","Multi-agent reasoning brings text-driven HOI video generation","No pose or depth needed: AgentHOI synthesizes HOI from text alone","Implicit text-motion alignment powers new HOI video model","AgentHOI: second-by-second scripts replace explicit motion inputs"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The entire text-only control claim rests on the off-the-shelf vision-language model reliably extracting physical attributes from two unrelated images and writing a correct, physically grounded, second-by-second action timeline — and the paper itself concedes that VLM hallucinations can produce misidentified objects or missing actions that propagate into the generated video.","fun_headline_variants_meta":{"raw":{"variants":["Text-only prompts now generate human-object interaction videos","Multi-agent reasoning brings text-driven HOI video generation","No pose or depth needed: AgentHOI synthesizes HOI from text alone","Implicit text-motion alignment powers new HOI video model","AgentHOI: second-by-second scripts replace explicit motion inputs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000185,"raw_usage":{"total_tokens":1139,"prompt_tokens":709,"completion_tokens":430,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":453,"completion_tokens_details":{"reasoning_tokens":345}},"tokens_in":453,"tokens_out":430,"duration_ms":4509,"temperature":1.0,"reasoning_tokens":345,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T05:22:05.706900+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of object images with heavy occlusion, unusual textures, or ambiguous shapes that the vision-language model tends to misclassify, pair them with prompts that depend on the true object identity, and check whether AgentHOI's output video depicts the wrong object or drops the required action. A high failure rate on such inputs, or a strong correlation between manually rated plan errors and low interaction-quality scores, would show that the text-only control claim rides on the VLM's accuracy rather than on the video model's learned motion priors.","supporting_citations":[],"review_version":1}