{"id":"e27c71f9-a561-42b1-b41b-46d733d4a20f","arxiv_id":"2411.12831","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"MotionDistill optimizes SMPL-X body poses with SDS gradients from video diffusion models, producing plausible animation for common actions but failing on rare ones.","lead":"This paper tests whether open-source text-to-video diffusion models can animate a 3D human body by optimizing its pose with score distillation. It finds that common actions such as running and walking animate plausibly, while rare actions such as cartwheels fail.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Section 5 target-motion prior, not the video diffusion model, may explain the reported common-action success; the paper's central attribution is therefore not yet established.","rationale":"The reader's weakest assumption identifies exactly the concern I find most load-bearing: the implementation initializes and constrains PoseField using the mean and standard deviation of the target motion's pose parameters. This assumption is stated plainly in Section 5, and it directly undermines the paper's central attribution--that common actions succeed and rare actions fail because of the video diffusion model. If the target-motion prior is doing the work, the method is not a text-to-motion generator but a motion-refinement scheme that already knows the action class. I agree with the reader that this makes the result a conditional proof of concept. I recommend keeping the verdict unchanged because the concern could plausibly be resolved by a clarifying experiment or a clarified description of the prior, and the paper's qualitative evidence still suggests the video models have some motion knowledge. A neutral-initialization rerun and a clear statement of the prior's source would settle whether the concern lands. I do not see an additional independent objection beyond what the reader already identified, and I do not regard the lack of quantitative evaluation or code as a separate load-bearing flaw relative to the prior confound.","tokens_in":10117,"tokens_out":4425,"duration_ms":48371,"concrete_test":"Re-run the 'running' and 'cartwheel' conditions with the same VideoCrafter2 SDS pipeline but with PoseField initialized to a neutral T-pose and with no target-action clamping, keeping all other hyperparameters fixed. If 'running' still produces plausible motion, the target prior is not load-bearing; if it collapses to a near-static pose, the prior is responsible for the reported success. Additionally, the authors should specify what dataset the 'target pose parameters' are computed from; if they are action-specific MoCap statistics, the method requires labeled target-motion data at test time and the text-to-motion claim is not met.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 5 states: 'We initialized the PoseField to output the mean pose of the target motion and constrained its output within a range defined by three times the standard deviation of the target pose parameters.' This is the load-bearing step for the paper's central conclusion. The method is framed as text-to-motion ('Given a text prompt y, we aim to determine the optimal joint rotations'), but this initialization and clamping injects the statistical distribution of the exact motion class to be generated. If 'target motion' means action-specific MoCap statistics, then for 'running' the optimizer is initialized inside a known-good pose region and clamped to it; the SDS signal only needs to refine local details, so success does not demonstrate that the video diffusion model can synthesize running from text. Conversely, for 'cartwheel' the target statistics may be scarce or poorly matched, so the failure could reflect the prior or the SMPL-X parameterization rather than the video model. The paper's attribution of rare-action failures to the video diffusion model (Section 5, Figure 4) is thus confounded. The latent-ablation experiment still starts from a stage-1 video rendered with the same target-prior-initialized PoseField, so it does not remove the confound. This is an internal-consistency problem, not merely a disagreement with consensus: the stated objective is text-only generation, while the implementation uses target-motion statistics. If the authors clarify that the statistics come from a generic human pose prior, the concern diminishes; as written, it directly weakens the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes MotionDistill, a pipeline that generates SMPL-X human animation by optimizing a small MLP (PoseField) that maps frame indices to body-pose parameters, using score distillation sampling (SDS) gradients from open-source text-to-video diffusion models (ModelScope, ZeroScope, VideoCrafter). The method is evaluated visually on prompts such as 'running', 'walking', 'punching', and 'doing a cartwheel'. The authors report plausible results for common actions, partial failures for 'punching', and extreme failures for 'cartwheel'. A latent-space ablation optimizes video latents directly with the same SDS objective, and the paper concludes that the observed failures point to limitations of the video diffusion models rather than the body representation. The paper positions itself as an initial proof-of-concept study of whether open-source text-to-video diffusion models can guide human motion synthesis.","tokens_in":10381,"tokens_out":2664,"duration_ms":29132,"significance":"If the central attribution were established, the paper would be a useful early empirical probe of a fast-moving question: whether video diffusion priors can drive human motion generation without MoCap supervision. The choice of three open-source models, the SDS-based formulation, and the latent-space ablation are sensible exploratory steps. However, the key claim that failures stem from the video diffusion model is currently confounded by the target-motion prior used in initialization and clamping, and the evidence is entirely qualitative. The work is therefore a promising proof-of-concept whose main empirical conclusion needs substantially stronger support before it can bear the weight the manuscript places on it.","major_comments":[{"comment":"The implementation as written contradicts the text-to-motion framing of Section 4. The paper states: 'We initialized the PoseField to output the mean pose of the target motion and constrained its output within a range defined by three times the standard deviation of the target pose parameters.' This injects the statistical distribution of the exact motion to be generated. For 'running', the optimizer starts inside a known-good pose region and is clamped there, so success does not demonstrate that the video diffusion model can synthesize running from text alone; it may only show that SDS can refine local details around a provided prior. For 'cartwheel' or other rare actions, the target-motion statistics may be scarce, noisy, or mismatched, so the failure could reflect the prior or the SMPL-X parameterization rather than the video model. The latent-space ablation in Section 5 starts from a Stage-1 video rendered with the same prior-initialized PoseField, so it does not remove the confound. I see two possible remedies: (a) remove or substantially weaken the target-motion initialization/clamping and show the common/rare distinction persists; or (b) explicitly reframe the method as motion refinement conditioned on action-specific pose statistics, which would materially change the paper's claims. Without one of these, the paper's central attribution is not established.","section":"Section 5"},{"comment":"The evaluation is entirely visual, with no quantitative metrics, no error bars, no multiple seeds, and no comparison to any baseline. Statements such as 'the models generated more natural motion sequences for running' and 'VideoCrafter demonstrates a higher degree of realism' in Figure 4 are not backed by measurements. The common/rare distinction is the paper's central empirical finding, but it rests on the authors' subjective inspection of a handful of sequences. At minimum, the paper should report quantitative motion-quality metrics (e.g., per-frame pose plausibility, motion smoothness, text-motion alignment scores, or user-study ratings) and should include multiple random seeds per prompt and per model so that variability (or lack thereof) is visible. Without this, the conclusions in Sections 5 and 6 are not adequately supported.","section":"Section 5"}],"minor_comments":[{"comment":"The regularization term is written as a sum of differences between consecutive pose vectors without a norm or squared norm, so as written it is not a scalar loss. It should be something like ||θ_i+1 − θ_i||^2 or an absolute/norm-based form.","section":"Equation 5"},{"comment":"There is a missing parenthesis: the expectation is written as Eσ,ϵ[w(σ)ϵ̂(Zα,σ,ϵ|y, σ) − ϵ) ∂Zα/∂α], which does not balance; the closing parenthesis appears twice, and the intended (ϵ̂ − ϵ) factor should be written correctly.","section":"Equation 4"},{"comment":"The supplementary results are referenced via a GitHub URL that the reader is told to visit; for a journal version, the repository should either contain the code and full videos with clear instructions or the supplementary material should be included in the submission package.","section":"Footnote 3"},{"comment":"The paper alternates between 'SMPLx' and 'SMPL-X'; please use the official 'SMPL-X' consistently.","section":"Notation"}],"recommendation":"major_revision","confidential_remarks":"The target-prior confound identified in Section 5 is a genuine internal-consistency problem, not merely a disagreement with consensus; the paper's main conclusion depends on removing or reinterpreting it. The manuscript is also quite short for a journal submission and the evaluation is thin. If the authors can supply an ablation without the target-motion prior and add quantitative or user-study evaluation, the paper may become a solid exploratory contribution, but in its current form I cannot recommend acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"At the heart of it: the paper shows that SDS from open-source text-to-video diffusion models can animate an SMPL-X body for common actions like running/walking, and that it fails for rare ones like cartwheels. That is a useful proof of concept. But the central attribution—that the video model is the bottleneck—is confounded by the implementation detail in Section 5: PoseField is initialized to the mean pose of the target motion and clamped within three standard deviations of the target pose parameters. That means the optimizer knows the action class before it sees any text. For common actions the prior is well-sampled, so the success is unsurprising; for rare actions the prior is likely poor, so the failure cannot cleanly be blamed on the video model. The latent ablation in Figure 4 starts from a video rendered with the same prior-initialized PoseField, so it does not remove the confound.\n\nWhat the paper does well: the method is clearly described, the choice of three open-source models and the common/rare comparison is a nice empirical contribution, and the authors are appropriately modest about scope. The qualitative examples do show a real difference between models and actions.\n\nThe soft spots in proportion: the evaluation is entirely visual—no quantitative metrics, no baselines, no error bars—and no code is released, which limits reproducibility. The regularization and SDS coefficients are not discussed beyond their values. The confound is the main issue; the rest is typical for a workshop-level proof of concept.\n\nWho should read it: someone working on human motion from video priors or character animation. It is a starting point, not a finished method. My recommendation for review: send it to an editor for peer review, but expect major revision—clarify or remove the target-prior initialization, rerun the ablation without that prior, and add at least basic quantitative motion metrics. With those changes the claim about video models could actually land.","headline":"A clear proof of concept for SDS-driven SMPL-X animation, but the target-prior initialization in Section 5 confounds the central claim that video models are the bottleneck.","tokens_in":10930,"tokens_out":2878,"would_cite":false,"duration_ms":28069,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper shows that open-source text-to-video diffusion models, guided by score distillation sampling, can synthesize plausible SMPL-X human animations for common actions like running and walking, but fail for rare actions like…","keywords":["human motion generation","score distillation sampling","text-to-video diffusion","SMPL-X","MotionDistill","diffusion models","character animation"],"falsifier":"Run MotionDistill on a 'running' prompt while initializing PoseField from the mean pose and covariance of a different action, such as sitting, and hold all other settings fixed; if the generated motion no longer looks like running or converges to the sitting prior, then the reported success for common actions depends on the target-motion-specific initialization rather than on the video diffusion model. Conversely, if running still emerges from a neutral initialization, the paper's attribution of failures to the video model is directly supported.","tokens_in":9912,"feed_emoji":"🏃","tokens_out":3180,"duration_ms":32578,"temperature":0.7,"pith_summary":"This paper asks whether open-source text-to-video diffusion models can serve as a guide for generating human body animation from a text prompt, without relying on motion capture data. It proposes MotionDistill, a pipeline that optimizes SMPL-X body poses frame by frame using score distillation sampling gradients from a video diffusion model. The central finding is that this works well for common activities such as running and walking, but degrades for uncommon actions such as punching and fails outright for cartwheels. By ablating the optimization directly in the video model's latent space, the authors argue that the limitation comes from the video diffusion model's bias toward common motions, not from the SMPL-X pose representation. A sympathetic reader would care because it charts a path toward extracting animation directly from video models rather than from scarce motion capture data.","feed_headline":"Diffusion video models animate common actions, fail at rare ones","feed_subtitle":"A score-distillation pipeline extracts human motion from text, showing the video model — not the body model — sets the ceiling.","key_machinery":"The central object is MotionDistill, a two-stage pipeline. Stage one defines a PoseField, a two-hidden-layer MLP $P_\\alpha : \\tau \\to \\theta_b$ that maps a frame index to SMPL-X body pose parameters; these parameters are fed through the SMPL-X layer $M(\\beta,\\theta,\\psi)$ to produce a mesh, which is rasterized from a random camera on a circular trajectory to form an $F$-frame video. Stage two encodes that video into the latent space of a video diffusion model, adds noise, and computes score distillation sampling gradients $\\nabla_\\alpha L_{\\mathrm{SDS-T}} = \\lambda_{\\mathrm{SDS}}^t \\mathbb{E}_{\\sigma,\\epsilon}[w(\\sigma)(\\hat{\\epsilon}_\\phi(Z_{\\alpha,\\sigma,\\epsilon}|y,\\sigma)-\\epsilon)\\frac{\\partial Z_\\alpha}{\\partial \\alpha}]$, with an additional image-diffusion SDS term and a temporal smoothness regularization on consecutive poses. This machinery lets gradient information from a pretrained video diffusion model flow back into the pose parameters, so the text prompt alone is supposed to drive the motion.","core_discovery":"The paper claims that a differentiable animation pipeline built from an MLP pose field, an SMPL-X body model, and a differentiable renderer can be steered by score distillation sampling from a text-to-video diffusion model to produce plausible human motion for common actions. Running and walking come out nearly correct, punching is only partially realized, and cartwheels fail even at the level of a plausible pose. The authors further claim, based on a latent-space ablation in which the rendered video is detached and the video diffusion latent is optimized directly, that these failures are attributable to the video diffusion models themselves and not to the body representation or the SDS formulation. They observe that newer video models such as VideoCrafter2 give markedly more realistic motion than older ones such as ModelScope and ZeroScope, which supports the conclusion that progress in the underlying text-to-video model is the key lever for better motion generation.","pith_inferences":["A direct way to test the paper's attribution would be to initialize PoseField from a neutral or mismatched pose distribution and run the same prompts; if common actions still emerge while rare ones fail, the video model's learned bias is confirmed, whereas if performance collapses the mean-pose initialization was doing the work.","The paper implicitly assumes that the video diffusion model's gradient signal is informative about human motion, but the same pipeline could be applied to other articulated objects, such as animals or robots, to probe whether the model has an action-specific rather than human-specific motion prior.","The authors' latent-space ablation could be extended into a quantitative measure of action bias: for a fixed set of prompts, the variance of the optimized latents across frames could serve as a proxy for how confidently the video model represents each action.","Because the method optimizes only the 21 major body joints and keeps hands and face fixed, the punching failure may partly stem from under-represented hand motion; a version that also optimizes finger poses would separate a representation gap from a model gap in the semi-failure cases."],"forward_implications":["If the central claim is right, motion generation can bypass motion capture data and instead tap the much larger corpus of video data already embedded in text-to-video diffusion models.","The failure pattern across actions indicates that open-source video diffusion models have an internal bias toward frequently occurring human activities, so evaluating any text-to-motion method needs to separate common from rare actions.","Because the latent-space ablation reproduces the same failure modes, the pose representation and SDS formulation are probably not the bottleneck; improving the video foundation model should directly improve the generated animation.","Newer video diffusion models such as VideoCrafter2 already yield more realistic motion than older ones, so the method's quality is expected to scale upward as open-source text-to-video models continue to improve.","MotionDistill can serve as a proof of concept and a diagnostic tool: success and failure cases expose which actions a given video diffusion model actually understands."],"supporting_citations":[{"why":"Introduces score distillation sampling, the core gradient mechanism used to drive the pose optimization.","marker":"[37]"},{"why":"Provides the temporal variation of SDS for video latents that the paper adapts in its stage-two gradient estimation.","marker":"[49]"},{"why":"Defines the SMPL-X body model, the articulated human representation that the PoseField optimizes.","marker":"[32]"},{"why":"VideoCrafter2 is the newer video diffusion model that produces the most realistic motion in the experiments.","marker":"[8]"},{"why":"ModelScope is one of the older open-source video diffusion models compared in the study, showing weaker frame variation.","marker":"[59]"},{"why":"ZeroScope is the second older video model used as a baseline in the comparison.","marker":"[6]"},{"why":"Prior work that applies SDS to human motion generation in rotation space, establishing the baseline the paper distinguishes itself from by operating in pixel space.","marker":"[46]"},{"why":"The human motion diffusion model trained on MoCap data, representing the motion-capture-dependent approach the paper aims to step beyond.","marker":"[55]"}],"fun_headline_variants":["Diffusion video models handle common actions, not rare ones","Text-to-video models cap motion quality, not the body model","Score distillation reveals video model as motion bottleneck","Common moves animate well; rare ones fail with SDS pipeline","Video model, not SMPL-X, limits human motion synthesis"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The pipeline initializes PoseField at the mean pose of the target motion and bounds outputs to three times the standard deviation of that motion's pose parameters, so it assumes access to the statistical distribution of the exact motion it is supposed to generate from text alone.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion video models handle common actions, not rare ones","Text-to-video models cap motion quality, not the body model","Score distillation reveals video model as motion bottleneck","Common moves animate well; rare ones fail with SDS pipeline","Video model, not SMPL-X, limits human motion synthesis"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00019,"raw_usage":{"total_tokens":1304,"prompt_tokens":873,"completion_tokens":431,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":489,"completion_tokens_details":{"reasoning_tokens":350}},"tokens_in":489,"tokens_out":431,"duration_ms":5204,"temperature":1.0,"reasoning_tokens":350,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T17:09:06.245378+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run MotionDistill on a 'running' prompt while initializing PoseField from the mean pose and covariance of a different action, such as sitting, and hold all other settings fixed; if the generated motion no longer looks like running or converges to the sitting prior, then the reported success for common actions depends on the target-motion-specific initialization rather than on the video diffusion model. Conversely, if running still emerges from a neutral initialization, the paper's attribution of failures to the video model is directly supported.","supporting_citations":[{"cited_title":"In: The Eleventh International Conference on Learning Representations (2023) 2, 3, 5","cited_arxiv_id":null,"evidence_quote":"Introduces score distillation sampling, the core gradient mechanism used to drive the pose optimization."},{"cited_title":"In: Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., Scar- lett, J","cited_arxiv_id":null,"evidence_quote":"Provides the temporal variation of SDS for video latents that the paper adapts in its stage-two gradient estimation."},{"cited_title":"In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition","cited_arxiv_id":null,"evidence_quote":"Defines the SMPL-X body model, the articulated human representation that the PoseField optimizes."},{"cited_title":"In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","cited_arxiv_id":null,"evidence_quote":"VideoCrafter2 is the newer video diffusion model that produces the most realistic motion in the experiments."},{"cited_title":"co / cerspense / zeroscope_v2_576w 2, 7","cited_arxiv_id":null,"evidence_quote":"ZeroScope is the second older video model used as a baseline in the comparison."},{"cited_title":"In: The Twelfth International Conference on Learning Represen- tations (2024) 1, 3","cited_arxiv_id":null,"evidence_quote":"Prior work that applies SDS to human motion generation in rotation space, establishing the baseline the paper distinguishes itself from by operating in pixel space."},{"cited_title":"In: The Eleventh International Conference on Learning Representations (2023), https://openreview.net/forum?id=SJ1kSyO2jwu 1, 3","cited_arxiv_id":null,"evidence_quote":"The human motion diffusion model trained on MoCap data, representing the motion-capture-dependent approach the paper aims to step beyond."}],"review_version":1}