{"id":"74f6239c-fa26-42db-83f5-16ee092831ab","arxiv_id":"2505.21146","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A motion diffusion model with decoupled trajectory and keyframe-pose control, wrapped in an MLLM agent system, produces more controllable 3D human motion from images and text.","lead":"Researchers built a system, IKMo, that creates 3D human motions from a text prompt, a few keyframe images, and a requested walking path. It handles the path and the poses in separate channels and reports better path and pose fidelity than earlier methods, with a front end that uses AI agents to turn images into motion controls.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Decoupling claim is untested: the reported SOTA margins are confounded by the test-time Motion Optimization module; no coupled-variant ablation isolates the effect of parallel encoders.","rationale":"Attribution of the improvement is crucial because the abstract and Section 2.2 motivate decoupling as the mechanism; the reader's strongest_claim repeats that causal reading. The evidence, however, does not isolate it. Table 4's large effect of Motion Optimization, combined with the absence of a coupled ablation, means the SOTA numbers could be driven by extra inference-time optimization or by the rotation augmentation rather than by decoupled control. This does not imply the method is invalid—it may still be a strong system—but it does mean the central claim as stated is not yet supported. The proposed test directly settles this. I do not see a need to move away from the reader's CONDITIONAL verdict; the missing ablation adds a specific condition for acceptance, so verdict_should_be is UNCHANGED (still conditional). My concern differs from the reader's weakest_assumption about image-pose compatibility, which is about the end-to-end image pipeline; the core decoupled-control claim is more load-bearing for the paper's contribution.","tokens_in":16974,"tokens_out":9949,"duration_ms":109607,"concrete_test":"Implement and evaluate a coupled-control variant on HumanML3D: keep the full two-stage Motion Optimization (Eqs. 9–10), random-rotation augmentation, and all training hyperparameters identical to \"Ours\", but replace the parallel Trajectory Encoder + Pose Encoder + fused ControlNet with one ControlNet that takes a concatenation of trajectory and keyframe-pose features. Compare Traj.err, Loc.err, Avg.err, Pose Dist, FID, and R-precision. If the coupled variant is statistically indistinguishable from or better than the decoupled model, the paper's core decoupling claim fails; if the decoupled model is clearly better, the concern is resolved. For completeness, also report the decoupled model without Motion Optimization to bound the contribution of test-time optimization.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in Section 5.2 (\"best performance across all metrics under the pelvis+keyframes setting\") is about the full IKMo system, but the paper's causal interpretation—that decoupling trajectory and pose processing is what drives the wins—is not isolated by any experiment. The method differs from OmniControl and CondMDI in multiple simultaneous respects: parallel Trajectory/Pose encoders and fused ControlNet (§4.3), a test-time Motion Optimization that gradient-updates the noised motion with a dynamic weighting of exactly the L_traj and L_pose losses used in the reported control metrics (Eqs. 5–10), and random-rotation pose augmentation. Table 4 shows removing Motion Optimization alone raises Traj.err from 0.0246 to 0.3006, Loc.err from 0.0076 to 0.1982, Avg.err from 0.0250 to 0.3491, and Pose Dist from 0.0210 to 0.0618. Since no baseline is reported as having this same optimization module, the large margins over SOTA cannot be attributed specifically to decoupling. In particular, there is no ablation that replaces the parallel encoders with a coupled variant while keeping the optimization module fixed, so the abstract's load-bearing statement (that decoupled trajectory and pose processing is what improves control) remains an interpretation rather than a demonstrated result.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes IKMo, a motion diffusion framework for simultaneous trajectory and keyframe-pose control. The core idea is to decouple trajectory and pose conditioning: a test-time Motion Optimization stage gradient-perturbs the noised motion using a dynamic combination of L_traj and L_pose, and a Motion ControlNet injects features from separate Trajectory and Pose encoders. On top of this, an MLLM-based multi-agent system converts user images and texts into a motion description, keyframe poses, and a trajectory. Experiments on HumanML3D and KIT-ML report state-of-the-art control metrics, ablations show that both stages contribute, and a 10-participant user study is used to support the image-keyframed pipeline.","tokens_in":17214,"tokens_out":10289,"duration_ms":108787,"significance":"If the central claim were firmly established, the paper would be a useful step toward flexible motion control from heterogeneous constraints, and the image-keyframed interface is practically appealing. The paper is transparent about its implementation details and ablates both proposed modules, and the supplementary description of the MLLM agents is helpful. However, the main conceptual claim—that decoupling trajectory and pose processing is what drives the reported improvements—is not isolated by any experiment, and the headline control metrics largely coincide with the inference-time optimization objective. The user study is underpowered and compares against a strawman baseline rather than the actual SOTA keyframe method. The paper is best read as a system paper with a promising pipeline; the scientific claim about decoupling needs additional evidence.","major_comments":[{"comment":"The central claim that decoupling trajectory and pose processing is responsible for the performance gains is not isolated by any experiment. Relative to OmniControl and CondMDI, IKMo changes several factors at once: parallel Trajectory/Pose encoders with a fused ControlNet (§4.3), a test-time gradient-perturbation stage with dynamic weighting (Eqs. 9–10), and random-rotation pose augmentation. Table 4 ablates Motion Optimization and Motion ControlNet, but no variant replaces the parallel encoders with a coupled or fused encoder while keeping the optimization module fixed. The large margins in Tables 1–2 therefore cannot be attributed specifically to decoupling; please add such an ablation (for example, a single encoder over concatenated trajectory and pose features, with the same optimization and augmentation) and report all metrics for that variant.","section":"§4.3, §5.2, Table 4"},{"comment":"The control metrics Traj.err, Loc.err, Avg.err, and the new Pose Dist are L2 distances between generated positions and the constraint values, which are essentially the same quantities that Eqs. 5, 6, and 9 minimize at inference through the Motion Optimization module. Consequently, the improvement from enabling Motion Optimization in Table 4 (e.g., Traj.err from 0.3006 to 0.0246) is partly by construction, and it is unclear how much of the reported margin over baselines reflects genuine control quality rather than the choice of evaluation metric matching the optimized objective. Please report, at least for the ablation variants, the metrics that are not direct targets of the inference-time loss (FID, R-precision, Foot skating, Diversity), and consider reporting the constraint error before versus after the gradient perturbation.","section":"§5.1, §B.3, Eqs. 5–10"},{"comment":"The claim in Section 5.2 that OmniControl 'experiences a significant drop in performance when keyframe conditions are added' is based on comparing the pelvis-only OmniControl result taken from the original paper with the pelvis+keyframes result re-evaluated under the authors' unified setting; Appendix C.1 further states that the better of the two re-evaluations was reported. This mixes experimental settings and selects the favorable baseline number, so it does not support the conclusion that a single control mechanism is inherently suboptimal. All baselines and conditions should be evaluated under one identical protocol, or the causal interpretation of this cross-setting comparison should be removed.","section":"Appendix C.1, Table 1"},{"comment":"The user study does not compare IKMo against the actual keyframe-based SOTA method CondMDI; the 'CondMDI-style baseline' is the authors' own motion diffusion model with randomly sampled keyframe poses from HumanML3D. With only 10 participants and two videos, this design cannot support the abstract's claim that the MLLM-based agent pre-processing 'makes generated motion more in line with users' expectation.' A comparison against the actual CondMDI interface, or at least a content-matched baseline using keyframes derived from the same images, is needed, together with a larger sample and appropriate significance testing.","section":"§5.5, Table 6"},{"comment":"The image-to-motion front end assumes that TokenHMR poses mapped into the HumanML3D convention (Eq. 11) are compatible with training-time poses up to small random rotations. The Limitations paragraph appropriately acknowledges that intermediate pose extraction may introduce errors, but no quantitative test of the full image-to-motion pipeline on real user photos (including scale and skeleton mismatches) is provided. The rotation augmentation addresses only global rotation, not the other failure modes; either add such an evaluation or narrow the claims about robustness of the IKMo pipeline.","section":"§4.3, §A.2, Limitations"}],"minor_comments":[{"comment":"The abstract contains grammatical errors: 'We conducts' should be 'We conduct', 'users' exception' should be 'users' expectation', and the phrasing 'meets more in line' is ungrammatical.","section":"Abstract"},{"comment":"The labels 'Ours (on rotations)' and 'Ours (w/ rotations)' are used inconsistently; please adopt a single naming convention throughout.","section":"Tables 1 and 5, Section 5.2"},{"comment":"The definition of Pose Dist should specify which joints are included in the average (all 22 joints or only controlled joints) and whether the pelvis-centered transform is applied identically to generated and reference poses; the current wording is ambiguous.","section":"Appendix B.3"},{"comment":"CondMDI is not evaluated under the new Pose Dist metric, so the claim of 'best performance across all metrics' is not fully supported for the pose-similarity metric; if CondMDI's representation is incompatible, this should be stated explicitly in the main text.","section":"Table 3"},{"comment":"The statement that CondMDI 'suffers from larger trajectory control errors and is similarly difficult to manually configure for effective guidance' is asserted without a citation; it should be supported by the quantitative results or softened.","section":"§2.2"},{"comment":"The no-rotation model has better FID and lower control errors on the standard evaluation (0.177 FID, Traj.err 0.0176) than the rotation-augmented model (0.239 FID, Traj.err 0.0246), yet the rotation-augmented version is used in all subsequent comparisons; the paper should explicitly discuss this trade-off.","section":"Table 5"}],"recommendation":"major_revision","confidential_remarks":"The paper is clearly written and the system is practically useful, but the headline claim about decoupling requires the missing coupled-encoder ablation, and the circularity between the control metrics and the optimized loss needs to be addressed. The baseline-number selection described in Appendix C.1 is also concerning and should be corrected. The novelty claim of being 'the first framework to use human images as keyframe cues for motion control' may need careful wording, since keyframe images are already used in video-generation works such as MotionBridge; for 3D motion generation the claim appears plausible but should be checked against closely related pose-conditioned motion work."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this is a competent, well-motivated paper with a real contribution—parallel trajectory/pose encoders plus inference-time gradient optimization, and an MLLM-agent front end that turns images and text into motion specifications. The ablations show both stages matter, and the full system beats the baselines they compare against on the reported metrics.\n\nBut the paper's central causal claim—that decoupling is what drives the gains—is not tested. The reported SOTA margins come from a system that differs from OmniControl and CondMDI in several ways, most notably the Motion Optimization module that directly minimizes the same L2 losses used as evaluation metrics. There is no ablation that swaps the parallel encoders for a coupled variant while keeping the optimization fixed. So the abstract's 'decoupled ... improves control' statement is an interpretation, not a demonstrated result. The stress-test note gets this right.\n\nThat said, the paper is honest in places. It reports ablations with and without each module, and the Limitations paragraph openly concedes the image-to-motion pipeline depends on intermediate pose extraction. The authors also disclose that the pelvis-only baseline numbers are taken partly from the original papers, and they re-ran OmniControl on their hardware and report the better of the two results—a bit generous, but not misleading.\n\nThe circularity concern is real but proportionate. The control metrics (Traj.err., Loc.err., Avg.err., Pose Dist) are essentially the same L2 distances as the losses being minimized during inference. Large improvements on those metrics are partly by construction. That is common in this literature, but the paper would be stronger if it reported a metric that is not directly optimized, such as foot skating or FID under control, and if it gave error bars or significance tests. The user study with 10 participants and a simulated CondMDI baseline is weak but not central.\n\nWho is this for? Researchers building controllable motion generation or multimodal conditioning pipelines. It is a useful reference to cite for the architecture and the agent pipeline, but I would not take the SOTA numbers at face value without a cleaner ablation. It deserves a serious referee—it is a solid, readable paper that would benefit from a revision where the decoupling claim is isolated and the evaluation is sharpened.\n\nRecommendation: send to peer review, but the referee should push for the missing ablation and more careful metric reporting.","headline":"Useful control method for motion generation, but the paper's central claim that decoupling drives the gains is not actually isolated in the experiments.","tokens_in":17745,"tokens_out":1995,"would_cite":false,"duration_ms":23220,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that decoupling trajectory and keyframe-pose control, rather than fusing them, improves every controlled-motion metric and enables image-keyframed generation.","keywords":["human motion generation","motion diffusion model","keyframe pose control","trajectory control","image-keyframed motion","multimodal large language models","controllable generation","motion ControlNet"],"falsifier":"Run a head-to-head ablation that replaces the parallel Trajectory and Pose encoders with a single fused encoder while keeping Motion Optimization, data, and training budget identical: if the fused version matches IKMo's Trajectory Error and Pose Dist on HumanML3D, the central claim that decoupling causes the gains would be refuted.","tokens_in":16740,"feed_emoji":"🕺","tokens_out":12181,"duration_ms":114290,"temperature":0.7,"pith_summary":"This paper tries to establish that when a motion diffusion model is given both a trajectory and keyframe poses, the two constraints should be handled through separate pathways rather than merged into one condition. It claims that this decoupling, implemented with a gradient-based Motion Optimization stage and a Motion ControlNet with parallel trajectory and pose encoders, yields state-of-the-art results under trajectory-plus-keyframe control on both HumanML3D and KIT-ML, improving trajectory following, keyframe pose matching, and motion realism together. The same model is wrapped in a user-facing pipeline, IKMo, in which multimodal language-model agents turn user images and text into 3D keyframe poses, a motion description, and a trajectory. A ten-participant user study supports the claim that image keyframes make generated motion closer to what users expect. If the claims hold, the payoff is practical: animators and general users can specify motion style and spatial path from images and simple text, without manually posing a skeleton or drawing 3D curves.","feed_headline":"Decoupled pose and trajectory control beats joint control","feed_subtitle":"On HumanML3D and KIT-ML, trajectory error drops to ~5% of prior joint-conditioned models.","key_machinery":"The mechanism that carries the argument is decoupled trajectory-pose conditioning with two parallel control pathways. In Motion Optimization, the noised motion $x_t$ is updated by gradient descent on $\\alpha L_{\\mathrm{traj}}(x_t, c_{\\mathrm{traj}}) + (1-\\alpha) L_{\\mathrm{pose}}(x_t, c_{\\mathrm{pose}})$, with $\\alpha = L_{\\mathrm{traj}}/(L_{\\mathrm{traj}} + L_{\\mathrm{pose}})$ balancing the two error scales; before the pose loss is computed, the constraint pose is translated so its root aligns with the trajectory constraint or with the current motion's root projection. In Motion Control, a Trajectory Encoder and a Pose Encoder process the two conditions in parallel, and their fused features are injected through a Motion ControlNet, defined as a trainable copy of the transformer encoder of the diffusion backbone, into the denoising network. The image front end is a multi-agent MLLM system whose Motion Design Agent recovers SMPL poses via TokenHMR, maps them to a canonical 22-joint HumanML3D-compatible format, and whose Trajectory Planning Agent generates parameterized curves; training with small random global rotations on sampled poses is what makes poses extracted from arbitrary photos compatible with HumanML3D motion data.","core_discovery":"IKMo's central claim is that trajectory constraints and keyframe-pose constraints have different semantics—trajectory is the absolute spatial path of the root joint, while a keyframe pose is a relative full-body configuration—so a diffusion model that fuses them into one representation before conditioning sacrifices both control accuracy and pose fidelity. The paper therefore keeps the two signals decoupled through two stages: Motion Optimization, which perturbs the noised motion by gradient descent on a weighted sum of a trajectory loss and an alignment-aware pose loss, and Motion Control, in which separate Trajectory and Pose encoders feed features into a Motion ControlNet attached to the denoising backbone. On HumanML3D and KIT-ML, the method reports the best numbers on every metric under the pelvis+keyframes setting, cutting Trajectory Error to about 5% of OmniControl's and CondMDI's values, Location Error to 4–11% of theirs, and Average Error to 15–18% of theirs, with a new Pose Dist metric also favoring IKMo. For image input, the paper claims the first framework to use human images as keyframe cues: a multi-agent MLLM system extracts 3D poses with TokenHMR, maps them to a HumanML3D-compatible 22-joint format, plans parameterized trajectories, and generates a text prompt, and the user study indicates this image-keyframed pipeline produces motions closer to user expectations than randomly sampled keyframe poses.","pith_inferences":["The decoupling principle should transfer to other mixed control signals in generative animation, such as absolute scene or contact positions combined with relative body poses, where joint fusion would again entangle coordinate frames; a direct test would swap trajectory and keyframe inputs for scene and contact inputs while keeping the two-stage parallel design.","The scale of the reported gains suggests that prior joint-conditioning models lose capacity to representational conflict rather than to insufficient data, a prediction that could be checked by scaling up the baselines' encoders and seeing whether the gap persists.","Because the multi-agent front end already outputs structured poses, trajectories, and text, a natural extension is coordinating several characters or a moving camera from the same structured specification.","Evaluating the image front end on real photographs with known mocap ground truth would isolate pose-extraction error from generation error and test whether the small-rotation compatibility assumption holds outside HumanML3D."],"forward_implications":["Under the pelvis+keyframes setting, the paper reports state-of-the-art values on every metric on both HumanML3D and KIT-ML, with Trajectory Error at roughly 5% of OmniControl's and CondMDI's, Location Error at 4–11% of theirs, and Average Error at 15–18% of theirs.","Image keyframes extracted from user photos or video frames can specify motion style and timing, so animation control no longer requires manually designing 3D trajectories or posing a full skeleton.","Training with small random pose rotations makes the model robust to the arbitrary global orientation of image-derived poses, preserving FID and R-Precision when rotations are applied at evaluation.","The ablations show the Motion ControlNet is the main driver of keyframe pose fidelity, with Pose Dist degrading 7.72 times without it, while Motion Optimization cuts trajectory, location, and average errors by over 90%.","The MLLM agent pre-processing yields motions rated more similar to the reference video than a pose-sampling baseline in the ten-participant user study."],"supporting_citations":[{"why":"OmniControl supplies the base motion diffusion framework, the two-stage trajectory guidance design, and the main joint-conditioning baseline under pelvis+keyframes.","marker":"[58]"},{"why":"CondMDI is the SOTA keyframe-guided baseline that IKMo must beat in the pelvis+keyframes comparison and in the motivation for decoupling.","marker":"[8]"},{"why":"HumanML3D provides the primary training and evaluation dataset and the metric protocol (FID, R-Precision, Diversity) used in all quantitative comparisons.","marker":"[15]"},{"why":"KIT-ML provides the second dataset for cross-dataset evaluation of trajectory and keyframe control.","marker":"[41]"},{"why":"MDM is the underlying human motion diffusion model whose transformer backbone IKMo copies for its encoders and ControlNet.","marker":"[50]"},{"why":"TokenHMR recovers SMPL 3D poses from user keyframe images in the Motion Design Agent.","marker":"[12]"},{"why":"SMPL defines the body model and joint structure that gets mapped to the canonical 22-joint HumanML3D format.","marker":"[32]"},{"why":"ControlNet is the architectural pattern behind the motion ControlNet that injects fused trajectory-pose features into the denoising backbone.","marker":"[64]"},{"why":"GMD provides the control-accuracy metrics (foot skating, trajectory error, location error, average error) and is a pelvis-only baseline.","marker":"[24]"},{"why":"MotionLLM contributes the MoVid video clips used for video-style keyframe extraction and the user study.","marker":"[5]"}],"fun_headline_variants":["Decoupled pose and trajectory control beats joint conditioning","Separating pose and trajectory boosts motion quality","Diffusion splits pose and trajectory for better motion","Decouple pose and trajectory for improved control","Two-stage conditioning decouples trajectory and pose"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The image-keyframed pipeline assumes that 3D poses recovered from user photographs by TokenHMR and mapped to the HumanML3D coordinate system are compatible with training-time motion poses up to small random rotations; if real photos bring scale, skeleton, or camera-projection mismatches beyond those rotations, the visual-input front end degrades.","fun_headline_variants_meta":{"raw":{"variants":["Decoupled pose and trajectory control beats joint conditioning","Separating pose and trajectory boosts motion quality","Diffusion splits pose and trajectory for better motion","Decouple pose and trajectory for improved control","Two-stage conditioning decouples trajectory and pose"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00147,"raw_usage":{"total_tokens":5984,"prompt_tokens":1093,"completion_tokens":4891,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":709,"completion_tokens_details":{"reasoning_tokens":4822}},"tokens_in":709,"tokens_out":4891,"duration_ms":35273,"temperature":1.0,"reasoning_tokens":4822,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:34:14.219951+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a head-to-head ablation that replaces the parallel Trajectory and Pose encoders with a single fused encoder while keeping Motion Optimization, data, and training budget identical: if the fused version matches IKMo's Trajectory Error and Pose Dist on HumanML3D, the central claim that decoupling causes the gains would be refuted.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CondMDI is the SOTA keyframe-guided baseline that IKMo must beat in the pelvis+keyframes comparison and in the motivation for decoupling."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"MDM is the underlying human motion diffusion model whose transformer backbone IKMo copies for its encoders and ControlNet."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"TokenHMR recovers SMPL 3D poses from user keyframe images in the Motion Design Agent."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"GMD provides the control-accuracy metrics (foot skating, trajectory error, location error, average error) and is a pelvis-only baseline."}],"review_version":1}