{"id":"c2dc795b-2d7e-4dde-b9aa-7c240054dfd0","arxiv_id":"2501.16714","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A temporal attention purification and skip-connection rerouting method that separates motion learning from appearance learning in text-to-video diffusion model customization.","lead":"This paper presents a method for customizing the motion in text-to-video diffusion models while preventing the appearance of the reference videos from leaking into the output. It does this by fine-tuning only the Key part of temporal attention and by rerouting skip connections during generation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper never directly measures whether generated motion matches the reference videos; human motion-fidelity votes are near-ties with MotionDirector, so the 'motion more consistent' claim (and the TAP/AH design rationale) lacks direct support.","rationale":"I read the paper in good faith. The method is simple, the ablation structure is clear, and the appearance-side gains (Text Align. 28.52 vs 27.55 for MotionDirector in one-shot; ViCLIP 26.52 vs 25.54) are consistent across tables. Those support the appearance-alignment part of the claim, albeit without variance estimates. The motion half of the claim is where the argument is least secure. No reported automatic metric directly compares generated motion to the reference motion. Temporal consistency measures self-coherence, not fidelity to the reference; VBench average contains a motion-smoothness component but is reported as an aggregate across many dimensions; and the motion classifier in the probing study is trained on real human actions and tested on a single concept, so it is not a general motion-fidelity measure. The human evaluation does include a motion fidelity question, but the few-shot result against MotionDirector is 51.5/48.5, essentially a tie, and no confidence intervals or significance tests are reported. The hidden-state classifier for AH (Fig. 4) defines the positive class as 'TAP-like' and the negative class as 'vanilla T2V-DM', so showing AH hidden states classify as TAP-like only demonstrates that AH resembles the already-adapted model; it does not independently establish that the resulting videos retain the reference motion. The reader's identified assumption (Value sufficiency) is a meaningful mechanistic risk, and I partially agree with it, but I see the more load-bearing issue as the absence of any direct measurement of the central claim's motion component. If such a measurement were added and showed parity or superiority, the Value-sufficiency assumption would be adequate as a heuristic for the tested regimes. If it showed inferiority, the paper would need to be reframed as an appearance-leakage-reduction method rather than a motion-preserving one. Therefore I recommend keeping the verdict CONDITIONAL, with the added condition that a reference-based motion fidelity evaluation (with error bars) be provided. This does not change the reader's verdict, so the verdict_should_be field is UNCHANGED.","tokens_in":13570,"tokens_out":8721,"duration_ms":85300,"concrete_test":"Re-run the one-shot (LOVEU-TGVE) and few-shot (UCF Sports) evaluations with a reference-based motion fidelity metric, e.g., average cosine similarity between optical-flow embeddings or pose-sequence embeddings of generated videos and the corresponding reference videos, computed with 5 independent seeds per prompt. If the mean difference between Ours and MotionDirector falls within one standard error on this metric, the claim of superior motion consistency is not established and the verdict should remain conditional on additional evidence.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim has two parts: appearance alignment and motion consistency. Appearance is supported by Text Align and ViCLIP gains, though without error bars. Motion consistency is the weaker half. The automatic metrics reported in Tables 2 and 3 (Text Align, Temporal Consist., Aesthetic, ViCLIP, VBench Average) do not compare generated motion to the reference motion; Temporal Consist. only measures frame-to-frame coherence, and VBench Average dilutes motion submetrics. The probing experiment (Table 1) uses a motion classifier trained on human actions, reports only a single concept, and may be confounded by appearance. The human 'motion fidelity' votes are near-ties with MotionDirector (51.5/48.5 in the few-shot comparison) with no significance test. The hidden-state analysis for AH (Fig. 4) trains a classifier on TAP versus T2V-DM hidden states and then shows AH resembles TAP; this is circular because 'motion' is defined by TAP, not by an independent reference-based measurement. Accordingly, the paper's claim of 'motion more consistent with the reference videos' is not substantiated. The reader's Value-sufficiency assumption (Sec. 3.2) is a plausible mechanistic risk, but it is subordinate: even if Value embeddings were sufficient, the evidence that the full pipeline preserves reference motion is missing.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses motion customization in text-to-video diffusion models, building on MotionDirector's dual-path LoRA adaptation. It introduces Temporal Attention Purification (TAP), which restricts temporal LoRA updates to the Key projection so that pretrained Value embeddings are recombined rather than changed, and Appearance Highway (AH), which reroutes U-Net skip connections from the outputs of temporal transformers to the outputs of spatial transformers. A phased LoRA integration (PLI) uses the adapted model for early denoising steps and the vanilla model later. Experiments on TGVE and UCF Sports compare against prior customization and controllable generation methods on automatic and human metrics.","tokens_in":13840,"tokens_out":5009,"duration_ms":45080,"significance":"If the central claim holds, the paper offers a simple, training-light recipe that materially reduces appearance leakage in motion customization, and it is among the few works to explicitly tie architectural choices (attention projections, skip connections) to the motion/appearance separation. The method is clearly specified, the code is released, and the ablations isolate each contribution. However, the motion side of the claim is currently under-evidenced: no automatic metric compares generated motion to reference motion, and the human motion-fidelity vote against MotionDirector is a near tie. Because the method's rationale is precisely that it preserves motion while improving appearance, this gap is consequential.","major_comments":[{"comment":"The abstract and Sec. 5 claim that the method generates 'motion more consistent with the reference videos,' but no reported automatic metric measures agreement with the reference motion. Temporal Consist. is frame-to-frame CLIP coherence, and VBench Average dilutes motion submetrics; Tables 2 and 3 contain no reference-based motion comparison (e.g., optical-flow or trajectory similarity between generated and reference videos). In human evaluation, Motion Fidelity against MotionDirector is 51.5/48.5, a near tie with no confidence intervals or significance test. The motion-consistency half of the central claim is therefore not substantiated.","section":"§4.3, Table 3"},{"comment":"The probing experiment that motivates TAP, and specifically the choice to adapt only K rather than Q or V, is a single-concept study using a motion classifier with no error bars or significance testing. It also uses a proxy 'App. Leak' score computed as CLIP similarity to 'a person,' which is confounded when the generated subject is, e.g., a monkey or a robot. Given that the sufficiency of pretrained Value embeddings is assumed rather than derived, this is thin evidence for an architectural design choice that is central to the method.","section":"§3.2, Table 1"},{"comment":"The hidden-state analysis meant to show that AH 'does not disturb motion generation' is circular. The motion classifier is trained on TAP hidden states as positives and vanilla T2V-DM hidden states as negatives; showing that AH hidden states are classified as TAP-like only demonstrates that AH resembles TAP, not that either matches the reference motion. An independent, reference-based measure of generated motion is needed to support the claim that AH preserves motion fidelity.","section":"§3.3, Fig. 4"},{"comment":"The appearance-highway scale β and the phased-LoRA threshold τ are tuned on the same benchmark that is later reported as the final result (β is selected via Table 6, and PLI is ablated only on/off in Table 4 without a sweep over τ). This introduces test-set tuning into the headline numbers and should be addressed with held-out validation or at least a sensitivity analysis for τ.","section":"§4.4, Tables 4 and 6"}],"minor_comments":[{"comment":"Table 2's caption reads 'few-shot' while Sec. 4.2 and the surrounding text describe it as one-shot; Table 3 is the few-shot evaluation. Please correct the caption.","section":"Table 2 caption"},{"comment":"In Eq. (2), α_t and the overbarred α are used without definition, and the value of τ is never stated. The text says 'early steps' use the adapted model while the displayed condition t>τ uses the adapted model; this is consistent if the denoising index t decreases from T to 1, but that convention should be made explicit.","section":"§3.4, Eq. (2)"},{"comment":"The acronym is introduced as 'PIL' in Sec. 1 but 'PLI' is used in Sec. 4 and Table 4; please use a single acronym consistently.","section":"§1 and §4.4"},{"comment":"The bars in Fig. 4 have no error bars and no sample-size information; please specify how the similarities are computed and over how many videos.","section":"Fig. 4"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of the venue and the method is reasonable, but the central 'motion consistency' claim needs more direct evidence. Please require the authors to add reference-based motion metrics (e.g., optical-flow or trajectory comparisons) and error bars/significance tests for the human evaluation, and to disclose or control for the test-set tuning of β and τ."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: TAP and AH are simple, well-motivated tweaks that do seem to buy improved text alignment and reduced appearance leakage over MotionDirector. But the paper's stronger claim—that motion stays consistent with the reference videos—is not supported by any direct measurement; human motion-fidelity votes tie with MotionDirector, and the indirect evidence is partly circular.\n\nThe solid part: Table 1's probing study is a genuinely useful systematic comparison of which temporal-attention components should carry a LoRA. Finding that Key-only adaptation preserves motion quality while improving appearance metrics is a concrete, actionable result. AH is a clever cheap intervention: rerouting skip connections from temporal to spatial transformer outputs, with the FreeU-style intuition that skip connections carry appearance. The ablations show each component helps, and the gains in Text Align and ViCLIP are consistent across one-shot and few-shot settings. Method description is clear enough to reproduce.\n\nSoft spots, in order of importance. First, motion consistency to the reference is never directly measured. The automatic metrics (Text Align, Temporal Consist., Aesthetic, ViCLIP, VBench Average) do not compare generated motion to reference motion; Temporal Consist. is only frame-to-frame coherence. Human motion fidelity is 51.5/48.5 vs MotionDirector—a tie without significance testing. Figure 4's hidden-state analysis trains a classifier to distinguish TAP from T2V-DM and then shows AH resembles TAP; that defines 'motion' by TAP's own output, so it can't validate the full method's motion preservation. Second, the Value-sufficiency assumption is a heuristic: the probing supports it for the single tested concept, but there's no argument for why pretrained V projections should be a sufficient basis for arbitrary new motions. That's a moderate risk, not a fatal one. Third, no error bars or significance tests anywhere, and beta (and likely tau) are chosen on the benchmark itself; the beta ablation shows a plateau, so the concern is minor but real.\n\nBottom line: this is a competent incremental advance over MotionDirector, not a breakthrough. It deserves serious peer review—the problem is real, the tweaks are clean, and the empirical pattern is consistent. But a referee should demand a direct motion-fidelity metric (optical flow or trajectory matching to the reference) and significance testing before accepting the central claim.","headline":"TAP and AH are clean, well-motivated tweaks that consistently reduce appearance leakage over MotionDirector, but the paper never directly measures whether generated motion matches the reference video, so the headline motion claim is under-supported.","tokens_in":14353,"tokens_out":4750,"would_cite":true,"duration_ms":42367,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adapting only the Key embeddings of temporal attention and rerouting U-Net skip connections transfers a reference video's motion without copying its appearance.","keywords":["text-to-video generation","motion customization","diffusion models","LoRA adaptation","temporal attention","appearance leakage","U-Net skip connections","motion-appearance separation"],"falsifier":"Find one motion concept—for example, a deformation or articulation never present in the pretraining distribution—that cannot be reproduced when only Query/Key are adapted while Value and feed-forward are frozen, yet is faithfully reproduced when Value is also adapted; that single counterexample would show TAP's restriction is not universally sufficient. A more direct check is to train the exact TAP/AH pipeline on reference clips whose appearance and motion are deliberately matched, then swap the appearance in the prompt and measure whether the reference video's background object still appears in more than a trivial fraction of generated videos.","tokens_in":13390,"feed_emoji":"🎬","tokens_out":8109,"duration_ms":72966,"temperature":0.7,"pith_summary":"Motion customization usually means showing a text-to-video model a handful of clips with the same motion and expecting it to reproduce that motion under new text prompts. The problem is 'appearance leakage': the adapted model also copies backgrounds, objects, or identity from the reference clips, so prompts about a sea dragon or a sandwich still come out looking like the shark or pizza in the reference videos. This paper claims the leakage can be removed by two surgical changes: adapt only the Key (or Query/Key) projections of the temporal attention with LoRA, and reroute each U-Net skip connection to start from the non-adapted spatial transformer instead of the adapted temporal transformer. A third trick—using the adapted model only for early denoising steps—keeps late-step appearance generation in the vanilla model. If these claims hold, motion concept transfer no longer has to trade away appearance fidelity.","feed_headline":"Two tweaks transfer video motion without copying the reference scene","feed_subtitle":"Adapting only key embeddings and rerouting skip connections keeps the motion while dropping leaked backgrounds.","key_machinery":"The load-bearing machinery is a pair of surgical changes inside a spatial-temporal U-Net, plus an inference-time schedule. TAP (Temporal Attention Purification) restricts the motion LoRA to the Query/Key subspace of temporal attention, so weight updates only re-rank how existing Value embeddings are combined; the paper justifies this with a probing table comparing adaptation of Query, Key, Value, and feed-forward modules. AH (Appearance Highway) moves the origin of each encoder-to-decoder skip connection from the adapted temporal transformer's output to the non-adapted spatial transformer's output, leveraging the known role of skip connections as carriers of high-frequency appearance information; scaling that branch by a factor $\\beta$ adjusts how strongly the vanilla appearance dominates. PLI (Phased LoRA Integration) toggles between the adapted and vanilla model depending on the denoising step, on the empirical observation that later steps specialize in fine appearance. Together these components ensure the adapted branch is responsible for motion and the unadapted branch for appearance.","core_discovery":"The central claim is that a text-to-video diffusion model can learn a new motion from reference clips without encoding the clips' appearance, provided the adaptation is shaped by where motion and appearance actually live in the network. In Temporal Attention Purification (TAP), the temporal LoRA is applied only to the attention projections that determine which Value embeddings get combined—the Query and Key—leaving the pretrained Value embeddings and feed-forward layers untouched, on the assumption that those Values already contain all the visual ingredients a new motion needs. In Appearance Highway (AH), each U-Net skip connection is re-pointed so that the decoder receives its high-frequency appearance shortcut from the non-adapted spatial transformer rather than from the LoRA-adapted temporal transformer, cutting off the path by which leaked appearance would re-enter. A phased LoRA integration schedule then runs the adapted model only during early denoising, when motion is decided, and the vanilla model later, when fine appearance is decided. The paper demonstrates through quantitative metrics and human evaluation that this combination yields videos whose appearance matches the text prompt while motion matches the reference videos, on both one-shot and few-shot setups.","pith_inferences":["Beyond the paper, the same 'reroute the appearance shortcut' recipe might apply to any LoRA-adapted diffusion module that contaminates output appearance, such as personalized image generation or subject-driven video editing; the general principle is to locate which architectural pathway carries appearance and keep it unadapted.","The authors' assumption that pretrained Value embeddings are a complete motion dictionary is directly testable: probe with motion concepts far outside the training distribution (fluid dynamics, cloth, articulated machinery) and see whether only Query/Key adaptation still suffices.","A cleaner test of appearance leakage than CLIP text similarity would use pairs of reference videos with identical motion but maximally different appearance, then measure how much object/background identity transfers under TAP/AH; this would isolate the separation mechanism from language-alignment confounds.","The phased integration idea could combine with other conditioning mechanisms, such as structure or pose control, to reduce their appearance side effects, since it only requires a schedule over denoising steps."],"forward_implications":["A user with only one reference clip can transfer its motion to an arbitrary new subject and scene described by text, without unwanted background objects from the reference video reappearing.","Few-shot motion customization (three to five clips per motion category) improves in appearance diversity and motion fidelity, so the approach is useful when a motion concept is defined by several examples.","Because only the Key projection of temporal attention is adapted, the learned motion representation is smaller and carries less appearance information, making it easier to share and reuse across prompts.","The separation strategy transfers across different text-to-video backbones built on spatial-temporal U-Nets, so it is not tied to one base model.","The phased schedule suggests that diffusion-based customization generally should confine adapted modules to early denoising steps, since late steps are where appearance overfitting occurs."],"supporting_citations":[{"why":"Supplies the dual-path LoRA adaptation baseline that the paper modifies and compares against for motion customization.","marker":"[47]"},{"why":"Provides the evidence that U-Net skip connections mainly convey high-frequency appearance, motivating the rerouting strategy.","marker":"[28]"},{"why":"Supplies the motion classifier used in probing experiments to measure whether adapted modules preserve motion quality.","marker":"[41]"},{"why":"Serves as the primary pre-trained text-to-video base model on which the motion LoRA is trained.","marker":"[31]"},{"why":"Serves as a second base model to test whether the separation strategies transfer across backbones.","marker":"[34]"},{"why":"Is an alternative motion-customization method based on noise residuals, used as a comparison in one-shot evaluation.","marker":"[18]"},{"why":"Provides the CLIP similarity used for text-alignment and appearance-leakage metrics.","marker":"[23]"},{"why":"Provides the multi-dimension benchmark used for automatic evaluation.","marker":"[17]"},{"why":"Provides the video-language model used as a supplementary alignment metric.","marker":"[36]"},{"why":"Provides the one-shot reference-video dataset and prompts used for evaluation.","marker":"[39]"}],"fun_headline_variants":["Motion transfer that leaves the scenery behind","Separating video motion from appearance in one go","Tweaks keep video motion while dropping reference background","Motion without appearance leakage in text-to-video models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method rests on the assumption that the pretrained Value embeddings in the temporal attention already contain all the building blocks needed to compose any new motion, so only the Query and Key projections that combine them need to be adapted; the paper supports this with a small probing study rather than a general proof.","fun_headline_variants_meta":{"raw":{"variants":["Motion transfer that leaves the scenery behind","Separating video motion from appearance in one go","Tweaks keep video motion while dropping reference background","Motion without appearance leakage in text-to-video models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000231,"raw_usage":{"total_tokens":1548,"prompt_tokens":1073,"completion_tokens":475,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":689,"completion_tokens_details":{"reasoning_tokens":416}},"tokens_in":689,"tokens_out":475,"duration_ms":5284,"temperature":1.0,"reasoning_tokens":416,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T11:11:23.437809+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Find one motion concept—for example, a deformation or articulation never present in the pretraining distribution—that cannot be reproduced when only Query/Key are adapted while Value and feed-forward are frozen, yet is faithfully reproduced when Value is also adapted; that single counterexample would show TAP's restriction is not universally sufficient. A more direct check is to train the exact TAP/AH pipeline on reference clips whose appearance and motion are deliberately matched, then swap the appearance in the prompt and measure whether the reference video's background object still appears in more than a trivial fraction of generated videos.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the evidence that U-Net skip connections mainly convey high-frequency appearance, motivating the rerouting strategy."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the motion classifier used in probing experiments to measure whether adapted modules preserve motion quality."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Is an alternative motion-customization method based on noise residuals, used as a comparison in one-shot evaluation."}],"review_version":1}