{"id":"b5d38bcf-959d-4409-8a54-9ae281558ba7","arxiv_id":"2506.09229","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"CREPA fine-tunes video diffusion models with a loss that pulls each frame's hidden representation toward pretrained features of adjacent frames, improving semantic consistency in generated videos.","lead":"The paper introduces CREPA, a training trick that fine-tunes video diffusion models by aligning each frame's internal features with pretrained features of neighboring frames. It reports better visual quality and fewer objects changing identity across frames than plain fine-tuning or a per-frame version of the existing REPA method.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Neighbor DINOv2 features are likely near-duplicates of the current frame, so CREPA's Eq. 6 may reduce to REPA* with ~1.74x larger alignment weight; the paper never controls for this.","rationale":"The reader's conditional verdict is reasonable: the evidence would be stronger with error bars, code, and a non-circular diagnostic. Stress-testing the central claim further, the most load-bearing assumption is that the cross-frame terms in Eq. 6 provide a distinct supervisory signal beyond per-frame REPA. Given the reported hyperparameters (d=1, tau=1) and the 4x temporal compression, adjacent-frame DINOv2 features are likely highly correlated with the current-frame feature, making the cross-frame loss nearly a scaled version of the per-frame loss. The paper does not report the actual feature similarity or run a matched-regularization-weight control, so the causal role of cross-frame alignment is underdetermined. Fig. 2's CKNNA increase is expected because the loss directly maximizes that similarity. This concern does not refute the empirical gains, but it means the method's novelty is not isolated from a simple loss-weight increase. The proposed control experiment would separate the two explanations cleanly. Since this is a missing control rather than a demonstrated error, the conditional verdict remains appropriate.","tokens_in":14874,"tokens_out":9069,"duration_ms":87852,"concrete_test":"Run a control on one dataset (e.g., Disney or Crush) with identical training budget, LoRA rank, and lambda, but replace the two neighbor targets y_{f+-1} in Eq. 6 with the current-frame target y_f, keeping the same number of terms and weights (call it REPA++). If REPA++ matches CREPA on VBench semantic-consistency metrics, FVD/IS, and NVS, the gains are a regularization-strength artifact and the cross-frame mechanism is not supported. If CREPA clearly wins, neighbor features carry real temporal signal. Also report the median cosine similarity between DINOv2 features y_f and y_{f+-1} on training videos; values above about 0.95 make the control essential.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that aligning hidden states to neighboring clean-frame DINOv2 features (Eq. 6) is what improves semantic consistency. The load-bearing premise is that the cross-frame terms carry information beyond the current-frame term. This is untested. With d=1 and tau=1 as reported, the two added neighbor terms make the alignment loss approximately (1+2e^{-1/tau}) = 1.74 times the REPA* loss (Eq. 4), assuming the neighbor features are similar to the current-frame feature. For small temporal offsets (effective adjacency <= 4 after 4x latent compression), adjacent-frame DINOv2 features in video are typically near-duplicates, so this assumption is plausible. CREPA is then REPA* with a larger effective lambda, and the VBench, FVD/IS, NVS, and user-study gains could stem from stronger alignment regularization, not from any temporal cross-frame signal. Fig. 2's CKNNA diagnostic does not resolve this: it measures the exact similarity the loss maximizes, so an increase is expected by construction. The paper does not report the similarity between the current-frame and adjacent-frame DINOv2 features on its training data, nor a REPA* baseline with matched total regularization weight, nor a normalized CREPA loss.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Cross-frame Representation Alignment (CREPA), a regularization objective for fine-tuning video diffusion models (VDMs) with parameter-efficient methods such as LoRA. Starting from REPA*, a direct adaptation of the image-domain REPA objective that aligns each frame's hidden state with its own DINOv2 feature, the authors add terms that align the hidden state of a frame with the pretrained features of adjacent frames, weighted by an exponential decay in temporal distance. The loss is combined with the standard score-matching objective. The method is evaluated on CogVideoX-5B and Hunyuan Video over seven datasets (cartoon style, physical interactions, 3D scenes, movie scenes) using VBench, FVD/IS, novel view synthesis, and a user study, with comparisons against vanilla fine-tuning and REPA*. The paper reports consistent improvements, especially on semantic-consistency metrics.","tokens_in":15190,"tokens_out":3111,"duration_ms":33782,"significance":"If the central claim holds, CREPA is a simple and practical regularizer that improves both visual fidelity and cross-frame semantic coherence in VDM fine-tuning, with a single-GPU training budget and no architectural changes. The paper's strengths include a clearly specified objective (Eq. 6), experiments across two large models and multiple datasets, and the inclusion of both automatic metrics and human evaluation. The proposed method is easy to implement on top of existing REPA-style losses, which could make it immediately useful. However, the paper's mechanistic claim—that the cross-frame terms specifically, rather than simply a stronger overall alignment signal, are responsible for the gains—is not yet supported by the evidence. The main empirical results lack error bars and significance tests, and the diagnostic in Fig. 2 measures exactly the quantity the loss maximizes, so it cannot independently validate the mechanism. These issues make the contribution promising but not yet fully established.","major_comments":[{"comment":"The cross-frame terms are not compared against a matched-strength REPA* baseline. With d=1 and tau=1, the total weight on the two neighbor terms is 2e^{-1} ≈ 0.736 times the current-frame term, so the overall alignment weight is about 1.736 times that of Eq. (4). Since the effective temporal adjacency after 4× latent compression is only a few frames, adjacent-frame DINOv2 features may be near-duplicates of the current-frame feature; if so, CREPA reduces to REPA* with a larger lambda. The paper does not report the similarity between y_f and y_k for k=f±d on its training data, nor does it include a REPA* run with lambda scaled by 1.736 (or a normalized version of L_align). Without this control, the improvements in Tables 1–3 and Fig. 25 do not establish that cross-frame information, rather than stronger alignment regularization, drives the gains. This is load-bearing for the paper's central claim.","section":"Sec. 4.2, Eq. (6) and Sec. 5.1 (implementation details)"},{"comment":"The empirical observation in Fig. 2 measures CKNNA between hidden states and pretrained features of current and adjacent frames, which is precisely the quantity the CREPA loss (Eq. 6) maximizes. An increase in this metric is expected by construction and therefore cannot serve as independent evidence that the method improves temporal semantic structure. The paper should provide a diagnostic that is not directly optimized, such as the redundancy of adjacent-frame features (e.g., cosine similarity between y_f and y_{f±d}), or an ablation in which the neighbor terms are replaced by current-frame features with a total weight matched to CREPA.","section":"Sec. 4.2, Fig. 2, and Appendix B"},{"comment":"All quantitative claims are reported without error bars, confidence intervals, or significance tests. Many VBench differences are small (e.g., 0.5145 vs 0.5207 for Aesthetic Quality on Hunyuan Video, Table 1), and the FVD/IS evaluation is limited to a single dataset (DL3DV) with one I2V model. The user study uses 20 samples per criterion with no reported inter-annotator agreement or statistical test. Since the paper's central claim is empirical, the absence of variance estimates makes it difficult to assess whether the observed improvements are reliable beyond the specific checkpoint and seed used.","section":"Sec. 5.2, Tables 1–3, and Appendix E"}],"minor_comments":[{"comment":"The text says 'as shown in Table 4' but the VBench results for the main comparison are in Table 1; Table 4 in the appendix reports layer-search ablations. The reference should be corrected.","section":"Sec. 5.2, Quantitative Results"},{"comment":"There is a typo: 'beter' should be 'better'.","section":"Fig. 1 caption"},{"comment":"The descriptions of Text–Video Alignment and Visual Quality are duplicated; criterion (1) says it reflects frame-level clarity and artifacts, which is the same wording as criterion (2). This should be clarified.","section":"Appendix E"},{"comment":"The linear probing dataset is constructed from all fine-tuning datasets used in the paper, which may leak information about the target datasets into the layer-selection procedure. The authors should clarify whether the linear probes are trained on held-out videos or only on the training splits, and whether this selection affects the reported results.","section":"Sec. A"}],"recommendation":"major_revision","confidential_remarks":"The central mechanism—that cross-frame features provide non-redundant temporal information—is plausible but not yet tested against a matched-strength REPA* baseline. Given the small metric differences, the paper would be considerably strengthened by an explicit control. I would not recommend rejection because the method is simple and the empirical trend across many settings suggests some benefit, but the current evidence does not yet isolate the claimed cause."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take on arXiv:2506.09229. The one-line read: this is a useful, cheap trick for fine-tuning large video diffusion models—align hidden states to DINOv2 features of adjacent frames—and the empirical work spans two big models and seven datasets. The main claim, however, is not yet proven: the paper doesn't rule out that CREPA is just REPA* with a larger effective alignment weight.\n\nWhat is actually new: Eq. (6) is a genuine extension of REPA. The motivation is reasonable: per-frame alignment ignores temporal context, and the authors show qualitatively that REPA* can yield fragmented objects. The implementation is simple and the cost is modest: 3000 iterations on a single A100 for a 5B model. The evaluation is broad: VBench, FVD/IS, NVS with 3DGS, and a user study, across seven datasets. That is more than most fine-tuning papers do, and the qualitative results look plausible.\n\nSoft spots, in order of seriousness. First, the load-bearing control is missing. With d=1 and tau=1, the two neighbor terms contribute 2e^{-1} ≈ 0.74 of the current-frame term. If DINOv2 features of adjacent frames—effective offset at most 4 frames after the VAE's 4x temporal compression—are near-duplicates, CREPA reduces to REPA* with 1.74x the alignment weight. The paper never reports the similarity between y_f and y_{f±1} on its training data, nor a REPA* baseline with matched total regularization weight, nor a normalized CREPA loss. Without that, the VBench/FVD/NVS gains could come from stronger regularization, not from any temporal signal. Second, Fig. 2's CKNNA diagnostic is circular: it measures the exact similarity the loss maximizes, so an increase is expected by construction; it cannot serve as evidence for the mechanism. Third, the quantitative tables have no error bars or significance tests, and several VBench differences are small or even reversed (e.g., Imaging Quality on CogVideoX-5B favors REPA*).\n\nThese issues are fixable, not fatal. The paper is honest and well-written, the method is clearly specified, and the authors promise code. If they add the matched-regularization control and measure adjacent-feature similarity, the central claim will either hold or be cleanly refuted.\n\nWho this is for: practitioners fine-tuning video diffusion models at user scale, and researchers working on representation alignment for generative models. It deserves a serious referee, but the referee should push for the control experiment and error bars.\n\nRecommendation: send to peer review. Worth engaging.","headline":"A cheap and useful fine-tuning trick, but the paper's causal story is undercut by a missing matched-regularization control.","tokens_in":15689,"tokens_out":4581,"would_cite":true,"duration_ms":40271,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that adding a cross-frame representation alignment loss to LoRA fine-tuning of video diffusion models improves both visual fidelity and cross-frame semantic coherence over vanilla fine-tuning and per-frame REPA.","keywords":["cross-frame representation alignment","video diffusion models","fine-tuning","LoRA","DINOv2","semantic consistency","distillation regularization","novel view synthesis"],"falsifier":"Train the same video diffusion model under CREPA with adjacency $d=0$, $d=1$, and $d=3$ while holding all else fixed. If $d=3$ does not increase CKNNA similarity to adjacent frames over $d=1$, or if subject consistency on VBench decreases with larger $d$, then the exponential neighbor weighting is not tracking a semantic manifold. A second direct test: replace the adjacent-frame term with a single temporally averaged DINOv2 feature; if that matches or exceeds CREPA, the benefit is smoothing rather than semantic alignment.","tokens_in":14706,"feed_emoji":"🎬","tokens_out":8947,"duration_ms":76849,"temperature":0.7,"pith_summary":"Fine-tuning large video diffusion models on small attribute-specific datasets is unstable: models trained with only a score-matching loss converge slowly and often break object identity across frames. The paper claims that a regularization term pulling each frame's internal representation toward the pretrained visual features of its neighboring frames fixes much of this. Its cross-frame alignment loss adds an exponentially weighted similarity between a frame's projected hidden state and the clean-frame features of frames at distance $d$, on top of the usual per-frame alignment. Trained for 3,000 iterations on a single A100, CREPA improves VBench consistency metrics, FVD/IS, novel-view-synthesis quality, and user preference over vanilla fine-tuning and over the per-frame REPA baseline across seven datasets.","feed_headline":"Neighbor-frame alignment beats per-frame REPA in video fine-tuning","feed_subtitle":"A new regularizer ties each frame's hidden state to DINOv2 features of adjacent frames, improving semantic consistency and visual fidelity.","key_machinery":"The load-bearing mechanism is a distillation regularizer applied to a diffusion-transformer encoder layer located by linear probing: a small MLP $h_\\phi$ projects the noisy hidden state $h^f_t$ into the feature space of DINOv2, and the loss pulls that projection toward the current frame's clean feature and, with weight $e^{-|k-f|/\\tau}$, toward features of frames at distances $\\pm d$ ($d=1$, $\\tau=1$). The exponential decay ensures nearby frames dominate, so the target is the local direction of the video's feature trajectory rather than a global average. This turns the fine-tuning objective into $L = L_{\\text{score}} + \\lambda L_{\\text{align}}$ with $\\lambda=0.5$ for CogVideoX and $\\lambda=1$ for Hunyuan.","core_discovery":"The central discovery is that per-frame representation alignment (REPA*) under-constrains video diffusion fine-tuning: a noisy hidden state can match its own frame's clean feature while sitting at an arbitrary distance from adjacent frames' features, and this freedom produces temporally inconsistent generations. CREPA resolves the under-constraint by maximizing $\\text{sim}(\\bar{y}_f, h_\\phi(h^f_t)) + \\sum_{k\\in\\{f\\pm d\\}} e^{-|k-f|/\\tau}\\, \\text{sim}(\\bar{y}_k, h_\\phi(h^f_t))$ for each frame $f$, with $\\bar{y}_k = E(x^k_0)$ extracted by a pretrained DINOv2 encoder from clean neighboring frames. In the authors' experiments this raises CKNNA similarity to adjacent frames while maintaining current-frame alignment, and translates into better subject/background consistency and lower FVD (281.2 vs 291.4 for REPA* and 305.5 for vanilla on their DL3DV setup) on CogVideoX-5B and Hunyuan Video.","pith_inferences":["If the DINOv2 feature trajectory is the right temporal manifold, CREPA should also help during pre-training of video foundation models, not just fine-tuning; the authors note this as future work.","The exponential weighting suggests a clean experiment the paper does not run: as $\\tau \\to 0$ the objective becomes local frame matching, and as $\\tau \\to \\infty$ it becomes a uniform temporal average; performance should peak at intermediate $\\tau$ if the manifold story holds, and would reveal whether the gains are from smoothing or semantic tracking.","Since layer search is done once per model and shared, CREPA can be composed with other fine-tuning regularizers, such as long-context tuning or attention modifications, as an orthogonal consistency term."],"forward_implications":["Applied to any DiT-based video diffusion model, CREPA needs only the chosen encoder layer and a small MLP, so it adds no inference-time cost and works with existing LoRA pipelines.","With 3,000 iterations (9-13 hours on one A100) it enables attribute-specific fine-tuning within a user-level budget, which the paper demonstrates on cartoon style, physical interactions, and static 3D scenes.","Because it improves frame-to-frame semantic coherence, downstream tasks that consume generated video—camera-pose estimation and 3D reconstruction with COLMAP and 3DGS—also improve, as shown by PSNR/SSIM/LPIPS on novel view synthesis.","The regularization is attribute-agnostic: the same $d=1$, $\\tau=1$ setting works across seven datasets, suggesting it can be bundled into fine-tuning toolkits as a default."],"supporting_citations":[{"why":"Defines the per-frame representation alignment (REPA) baseline that CREPA extends and compares against.","marker":"[50]"},{"why":"Provides the DINOv2 features used as the clean-frame targets for alignment.","marker":"[33]"},{"why":"Supplies the denoising-autoencoder interpretation of diffusion transformers that motivates aligning noisy encoder hidden states to clean features.","marker":"[47]"},{"why":"CogVideoX-5B is one of the two large video diffusion models on which CREPA is fine-tuned and evaluated.","marker":"[48]"},{"why":"Hunyuan Video is the other large model used to test CREPA across diverse datasets.","marker":"[23]"},{"why":"VBench's consistency metrics (subject/background consistency, motion smoothness) provide the quantitative evidence for improved semantic coherence.","marker":"[20]"},{"why":"CKNNA is the kernel-alignment metric used to measure cross-frame representation alignment in the empirical analysis.","marker":"[21]"},{"why":"LoRA is the parameter-efficient fine-tuning method used in all experiments, the setting CREPA is designed for.","marker":"[18]"},{"why":"DL3DV is the dataset used to evaluate CREPA's effect on 3D spatial consistency through novel view synthesis.","marker":"[26]"}],"fun_headline_variants":["Cross-frame alignment improves video diffusion fine-tuning","Neighbor-frame features beat per-frame REPA in video fine-tuning","Aligning frames to neighbors improves video consistency","Cross-frame regularization for stable video fine-tuning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the DINOv2 features of nearby clean frames form a temporally coherent manifold, so pulling a noisy frame's hidden state toward them improves semantic tracking; if those features are not coherent across adjacent frames, the extra term would just average frames together and smooth away distinct content.","fun_headline_variants_meta":{"raw":{"variants":["Cross-frame alignment improves video diffusion fine-tuning","Neighbor-frame features beat per-frame REPA in video fine-tuning","Aligning frames to neighbors improves video consistency","Cross-frame regularization for stable video fine-tuning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000781,"raw_usage":{"total_tokens":3463,"prompt_tokens":971,"completion_tokens":2492,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":587,"completion_tokens_details":{"reasoning_tokens":2432}},"tokens_in":587,"tokens_out":2492,"duration_ms":18989,"temperature":1.0,"reasoning_tokens":2432,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T04:53:46.049383+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same video diffusion model under CREPA with adjacency $d=0$, $d=1$, and $d=3$ while holding all else fixed. If $d=3$ does not increase CKNNA similarity to adjacent frames over $d=1$, or if subject consistency on VBench decreases with larger $d$, then the exponential neighbor weighting is not tracking a semantic manifold. A second direct test: replace the adjacent-frame term with a single temporally averaged DINOv2 feature; if that matches or exceeds CREPA, the benefit is smoothing rather than semantic alignment.","supporting_citations":[{"cited_title":"Oquab, T","cited_arxiv_id":null,"evidence_quote":"Provides the DINOv2 features used as the clean-frame targets for alignment."},{"cited_title":"Xiang, H","cited_arxiv_id":null,"evidence_quote":"Supplies the denoising-autoencoder interpretation of diffusion transformers that motivates aligning noisy encoder hidden states to clean features."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CogVideoX-5B is one of the two large video diffusion models on which CREPA is fine-tuned and evaluated."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CKNNA is the kernel-alignment metric used to measure cross-frame representation alignment in the empirical analysis."}],"review_version":1}