REVIEW 5 major objections 4 minor 3 cited by
The paper claims that motion in generated video can be attributed to specific training clips, and that fine-tuning a text-to-video model on the top ten percent of those clips improves temporal consistency and physical plausibility beyond fu
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-03 10:44 UTC pith:66SB556I
load-bearing objection Worth reading: a practical, clearly-described motion-specific data-selection method for video diffusion, but the attribution claim itself is only validated end-to-end, not causally. the 5 major comments →
Motion Attribution for Video Generation
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that motion can be attributed in video generative models by computing gradients of a motion-weighted denoising loss rather than the whole-video loss: when each pixel's loss contribution is weighted by normalized flow magnitude, the resulting gradient inner products rank training clips by their influence on temporal dynamics rather than on appearance. The paper demonstrates that the ranking is useful: fine-tuning a pretrained text-to-video model on the top 10% of clips selected by this ranking raises dynamic degree from about 40-41 to 47.6, matches or exceeds full-dataset fine-tuning, and achieves a 74.1% human preference win rate over the base model. It also reports that
What carries the argument
The central object is the motion-weighted loss mask: optical-flow displacement magnitudes between consecutive frames are min-max normalized to [0,1], bilinearly downsampled to the VAE latent grid, and multiplied into the per-location denoising error before backpropagation; an all-ones mask recovers the ordinary loss. This mask is combined with three efficiency choices: a single fixed denoising timestep with a shared noise draw for all training-query pairs, division by frame count to remove a duration bias, and a random Hadamard projection to 512-dimensional gradient vectors whose cosine similarity is the influence score. The mask is what makes the attribution motion-specific; the single-samp
Load-bearing premise
The whole method rests on the idea that gradients of the original, not-yet-fine-tuned model at one moment of denoising, with one fixed noise pattern, correctly predict which clips will teach motion after fine-tuning; the paper checks the final fine-tuned quality but never verifies that link directly.
What would settle it
Build a synthetic training corpus in which one clip's motion is deterministically reused in the query, then compute Motive scores and check whether that clip ranks first; if it does not, the score is not tracking the source of motion. Alternatively, fine-tune after removing the top 10% of Motive-ranked clips and measure motion smoothness and dynamic degree: no degradation would show the ranking is not causal.
If this is right
- Fine-tuning on the top 10% of attributed clips beats full-dataset fine-tuning on dynamic degree (47.6 vs 42.0) and subject consistency, while preserving motion smoothness.
- Motion attribution can filter negative-influence clips: the lowest-ranked clips are mostly static footage, camera-only motion, or cartoon content that dilutes temporal learning.
- Because per-clip gradients are computed once and stored as small projected vectors, adding a new query costs one backward pass plus a dot product, making iterative data curation practical after a one-time corpus pass.
- Cross-motion overlap patterns are similar across two different large video datasets, suggesting the attribution reflects shared structure in how motion is learned rather than dataset-specific noise.
- The same qualitative gains transfer to a larger 5B-parameter model, with dynamic degree rising from 42.0 to 48.3 under Motive-selected fine-tuning.
Where Pith is reading between the lines
- If the ranking is causal, an unstated but direct use is diagnosis: generate a video with a failure, run Motive with that video as query, and the top-ranked training clips are likely sources of the learned bad dynamics; one could remove or downweight them instead of fine-tuning.
- The motion weight comes from instantaneous flow magnitude, so the attribution emphasizes current speed over longer-range trajectory coherence; a natural extension is segment-level or event-level attribution that assigns influence to one phase of a motion rather than the whole clip, a limitation the paper itself notes.
- Because the method only needs per-example gradients, the same motion-masked influence procedure could be applied to other temporal generative settings, such as world models or audio-video generation, by replacing the flow-based mask with the appropriate motion or energy representation.
- The single-timestep choice suggests a testable hypothesis: different denoising timesteps carry different motion information, so rankings computed at very early and very late timesteps may select disjoint clips and could improve fine-tuning in complementary ways.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces Motive, a gradient-based data attribution framework for video diffusion models that aims to attribute generated motion to specific training clips. The method combines motion-weighted loss masks (using AllTracker optical flow), a fixed-timestep/shared-noise single-sample gradient estimator, a Fastfood random projection to D'=512, frame-length normalization, and majority-vote aggregation across query videos to select influential fine-tuning subsets. The authors evaluate Motive by fine-tuning Wan2.1-T2V-1.3B on the top 10% of selected clips from VIDGEN-1M and 4DNeX-10M and report improved VBench dynamic degree (47.6 vs. 41.3 for random selection) and a 74.1% human preference win rate over the base model. Additional results on Wan2.2-TI2V-5B and ablations on projection dimension and frame-length normalization are provided.
Significance. If the central attribution claim holds, this would be the first motion-specific data attribution framework for video generative models, with practical value for data curation and artifact diagnosis. The paper has notable strengths: it is the first to adapt influence-style attribution to temporal dynamics via motion masking; it provides detailed computational efficiency analysis and reproducible hyperparameters; it includes cross-dataset consistency analysis (Fig. 7) and ablations of projection dimension and frame-length normalization; and it demonstrates that the proposed selection pipeline improves both automated and human-judged motion quality. The empirical gains are substantial but, as detailed below, the evidence does not yet establish that the per-clip influence scores themselves are causally predictive of fine-tuning outcomes, rather than merely being a useful selection heuristic.
major comments (5)
- [§3.1, §4.2, Eq. (17), Algorithm 1] The paper's central claim is that Motive scores predict which training clips will change motion after fine-tuning. However, the only evidence is the end-to-end selection experiment (Table 1, Table 2): fine-tuning on the top-10% of clips ranked by Motive improves VBench dynamic degree and human preference. This validates a selection pipeline, not the per-clip attribution itself. No leave-one-out retraining, counterfactual upweighting, or known-influence benchmark is provided. Since the selected subset may correlate with caption quality, motion saliency, or text-category overlap with the query set, the causal link between the gradient-alignment score in Eq. (17) and downstream motion is not demonstrated. The 'predictivity' desideratum in §3.1 is asserted but not directly tested.
- [Tables 1 and 5; §4.3] Tables 1 and 5 report VBench scores without error bars, confidence intervals, or repeated seeds. The headline dynamic degree gap (47.6 vs. 41.3) comes from a single fine-tuning run per method; given the known sensitivity of video fine-tuning to initialization and data order, this gap could be within seed noise. Table 2 reports human preference percentages but no inter-annotator agreement (e.g., Cohen's kappa) or confidence intervals. The authors should provide variance estimates or multiple seeds to support the quantitative claims.
- [§4.4, App. F.1] The ablations validate internal consistency rather than ground-truth agreement. The single-timestep ablation reports ρ=66% agreement with a 10-timestep reference, but that reference is itself a gradient-similarity estimator, not a causal ground truth. The projection-dimension ablation measures agreement with unprojected scores (ρ=74.7% at D'=512), which again is internal. Furthermore, key hyperparameters (tfix=751, D'=512, top-10% budget, majority-vote threshold) are fixed without sensitivity analysis. A reader cannot tell whether the reported fine-tuning gains are robust to these choices.
- [App. G.2] The appendix explicitly states that the framework does not account for classifier-free guidance (CFG), which is used at inference time in all standard pipelines, and that temporal segments are diluted by whole-clip averaging. Both limitations directly affect the alignment between the training-time attribution objective and the inference-time behavior the paper claims to explain. The paper should either incorporate CFG into the attribution (e.g., by using the CFG-modified gradient) or temper the claim that Motive identifies clips that 'strongly affect motion' in generated videos, since generation uses CFG.
- [App. F and §4.1] The query set and the test prompts are drawn from the same ten motion categories (compress, bounce, roll, etc.). This shared taxonomy creates a possible positive bias: the selection is optimized for categories that are then used to evaluate the fine-tuned model. The paper would be stronger with held-out motion categories or with test prompts that are unseen in the query taxonomy, to demonstrate that the improvement generalizes beyond the curated motion vocabulary.
minor comments (4)
- [Table 1] The aesthetic quality column for the base model reads '45.365.7' (missing separator), which should be '45.3 65.7'. Please proofread all tables for formatting.
- [App. F.1] The paper states t=751 is 'the midpoint of the 1000-step denoising trajectory' but also says it corresponds to 'scheduler index 500'. Clarify the relationship between the timestep index and the scheduler step, since 'midpoint' and 'index 500' are not obviously the same in flow matching.
- [Fig. 6] The x-axis label 'All' appears to duplicate; likely the left figure should be 'Number of Samples' with a legend, and the right figure 'Motion Bin'. Please make the figure caption self-contained.
- [§4.4] The frame-length normalization ablation reports ρ=78.0% correlation with video length before standardization and a 54.0% reduction after. It would be helpful to state the post-standardization correlation value explicitly for completeness.
Circularity Check
No significant circularity: Motive's scores are defined by gradient cosine similarity, and the fine-tuning outcomes are measured independently; the only self-citation (ICONS for majority voting) is not load-bearing.
full rationale
The core attribution score (Eq. 17) is defined as a cosine similarity between projected, normalized gradients of a motion-weighted loss (Eqs. 15–16). The downstream improvements from fine-tuning on top-scoring clips are measured independently on VBench (Table 1) and via human preference (Table 2); these outcomes do not enter the score definition. Hyperparameters such as tfix=751 and D'=512 are selected by internal consistency with multi-timestep or full-gradient references (§4.4, Fig. 4), not by fitting the VBench numbers, so this is not a fitted-input-called-prediction case. The only self-citation that appears in the method is the majority-voting aggregation from ICONS [Wu et al., 2024] (Eq. 18); it is a simple, independently justifiable aggregation choice and is not load-bearing for the central attribution claim. Appendix G.2 explicitly acknowledges that CFG is not accounted for and that segment-level motion information can be diluted; these weaken the causal interpretation of the scores but do not make the derivation circular. There is no step in which the prediction is equivalent to the input by construction, no uniqueness theorem imported from the authors, and no ansatz smuggled in via self-citation. Score 2 reflects only the minor, non-load-bearing self-citation; the central derivation is self-contained and empirically tested.
Axiom & Free-Parameter Ledger
free parameters (6)
- tfix (single timestep) =
751 (scheduler index 500)
- D' (projection dimension) =
512
- Top-K percentile / voting threshold =
10% of dataset
- Frame standardization =
81 frames at 16 fps
- Fine-tuning learning rate and epochs =
1e-5, 1 epoch (dataset repeated 50x)
- zeta (numerical stabilizer) =
1e-6
axioms (6)
- domain assumption Gradient cosine similarity approximates true data influence (identity-preconditioned influence function)
- domain assumption A single shared (t, epsilon) preserves relative influence rankings
- domain assumption Optical-flow magnitude masks isolate temporal dynamics from static appearance
- domain assumption Initialization-time gradients predict which clips improve motion after fine-tuning
- domain assumption VBench dynamic degree/motion smoothness and human preference measure motion quality
- domain assumption Frame-length normalization 1/F removes duration bias
read the original abstract
Despite the rapid progress of video generation models, the role of data in influencing motion is poorly understood. We present Motive (MOTIon attribution for Video gEneration), a motion-centric, gradient-based data attribution framework that scales to modern, large, high-quality video datasets and models. We use this to study which fine-tuning clips improve or degrade temporal dynamics. Motive isolates temporal dynamics from static appearance via motion-weighted loss masks, yielding efficient and scalable motion-specific influence computation. On text-to-video models, Motive identifies clips that strongly affect motion and guides data curation that improves temporal consistency and physical plausibility. With Motive-selected high-influence data, our method improves both motion smoothness and dynamic degree on VBench, achieving a 74.1% human preference win rate compared with the pretrained base model. To our knowledge, this is the first framework to attribute motion rather than visual appearance in video generative models and to use it to curate fine-tuning data.
Figures
Forward citations
Cited by 3 Pith papers
-
LaMo: Self-Supervised Latent Motion Priors for Physical Realism in Video Generation
LaMo adds self-supervised latent motion priors via a motion drift loss during training and motion prior guidance during sampling to boost physical fidelity in video diffusion models like CogVideoX.
-
Variance Reduction for Expectations with Diffusion Teachers
CARV amortizes upstream diffusion teacher costs over noise resamples with timestep importance sampling and stratified-inverse-CDF sampling, delivering 2-3x effective compute gains in text-to-3D experiments and order-o...
-
Variance Reduction for Expectations with Diffusion Teachers
CARV introduces a hierarchical Monte Carlo estimator with amortized reuse, importance sampling, and stratification that yields 2-3x effective compute gains on diffusion-teacher pipelines while cutting gradient varianc...
Reference graph
Works this paper leans on
-
[1]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
-
[2]
@esa (Ref
\@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...
-
[3]
\@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...
-
[4]
@open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...
arXiv 2022
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.