{"id":"0c62801a-19ad-4dc9-a242-5386bd5e70d9","arxiv_id":"2506.05343","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"An 8B text-to-video model built by adapting Stable Diffusion 3.5 with a 3D video autoencoder reaches near-leading VBench scores after four weeks of NPU training.","lead":"ContentV is an 8-billion-parameter text-to-video model trained on 256 NPUs for four weeks, reporting a VBench score of 85.14. The report shows how reusing a pre-trained image model and a video autoencoder can produce competitive video generation with far less compute than typical GPU clusters.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'SOTA' claim selects the higher of two ContentV VBench scores (Long 85.14 vs Short 84.11) while competitor rows are single unlabeled official leaderboard values; Table 3 itself places Vidu-Q1 and Wan2.1 above ContentV.","rationale":"The reader's weakest_assumption correctly identifies the evaluation-protocol ambiguity as the load-bearing issue: the paper does not say which VBench mode its 85.14 belongs to relative to the single official leaderboard numbers used for competitors, and the two quoted ContentV scores differ by about one point. I agree that this is the decisive concern because the headline claim is comparative: 'state-of-the-art performance' and 'matches leading open-source models' depend entirely on the rank ordering, and adjacent VBench rows are separated by less than the observed Long/Short gap. The paper's own Table 3 also contradicts the abstract's SOTA wording by showing Vidu-Q1 and Wan2.1 above ContentV even in the favorable Long row. The rest of the paper, including the VAE-adaptation FID trajectory, the staged training recipe, and the infrastructure description, is coherent and enough to make the engineering result plausible, but it does not rescue the unverified comparison. The user study is too small (five annotators, no confidence intervals) and uses an unusual GSB formula, so it cannot independently establish superiority over Wan2.1. A concrete mode-matched VBench rerun on the released checkpoint would settle whether the claimed rank ordering is real or an artifact of prompt-mode selection; without that, the conditional verdict is appropriate.","tokens_in":17149,"tokens_out":8434,"duration_ms":104253,"concrete_test":"Run the official VBench evaluation code on the released ContentV checkpoint in both available prompt modes (short and long), using the same per-dimension prompt lists, number of generated videos per prompt, resolution, sampler, and CFG settings as the official leaderboard. Report per-dimension and aggregate scores separately for each mode, then compare mode-matched rows for Vidu-Q1, Wan2.1-14B, Sora, HunyuanVideo, and CogVideoX-5B. If ContentV's mode-matched score remains below Wan2.1 and Vidu-Q1, the abstract's 'state-of-the-art (85.14)' should be revised to a mode-matched and rank-accurate claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central comparative claim rests on one VBench number whose evaluation protocol is not matched to the leaderboard numbers it is compared with. Section 8.2 reports two ContentV scores, ContentV(Long)=85.14 and ContentV(Short)=84.11, in a table where every competitor appears once with values attributed to the 'official VBench leaderboard.' The paper never states whether those official rows are short-prompt, long-prompt, or an average, nor how many prompts per dimension ContentV used, nor the sampling/CFG settings. The abstract then quotes only the higher Long score. This is not cosmetic: switching ContentV from Long to Short changes its overall score by about one point, which is comparable to or larger than the gaps separating adjacent rows in Table 3 (e.g., Sora 84.28, HunyuanVideo 83.24, CogVideoX-5B 81.61). Even the favorable Long row places ContentV below Vidu-Q1 (87.41) and Wan2.1-14B (86.22), so the abstract's 'state-of-the-art (85.14)' is internally contradicted by the same table. The claim of a 'slight edge over Wan2.1' comes only from the user study, not from VBench, and that user study uses five annotators with no reported confidence intervals. The headline assertion therefore depends on an unstated evaluation-mode convention; without that convention, 'SOTA' is not established and the practical significance of the training recipe is not accurately summarized.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript presents ContentV, an 8B-parameter text-to-video model initialized from Stable Diffusion 3.5 Large and adapted to video generation by replacing the 2D VAE with the Wan2.1 3D VAE and adding 3D position embeddings. The training recipe consists of multi-stage flow-matching pre-training at increasing resolutions and durations, followed by SFT on a high-quality subset and an RLHF stage that uses VideoAlign and MPS as reward models. The training is performed on 256 NPUs over about four weeks. The authors report a VBench score of 85.14 and claim state-of-the-art performance, plus a user study showing preference over CogVideoX-5B, HunyuanVideo-13B, and Wan2.1-14B. The paper also describes a large-scale data curation pipeline, asynchronous encoding infrastructure, and 3D parallelism strategies.","tokens_in":17487,"tokens_out":4875,"duration_ms":52093,"significance":"If the central claims are substantiated, the paper would make a meaningful contribution: it demonstrates that a minimal adaptation of an open image-diffusion model, combined with a swapped 3D VAE, staged flow-matching training, and RLHF, can produce a competitive video model with limited compute on NPUs. The paper's strengths include detailed training configurations (Tables 1 and 2), an explicit description of the data filtering pipeline, quantitative VAE-adaptation curves (FID progression in Section 5.2), and the public release of code and models. However, the headline comparative claims are currently undermined by an internally inconsistent use of VBench scores and by evaluation-protocol ambiguity, so the significance cannot be fully assessed until those issues are resolved.","major_comments":[{"comment":"The abstract's claim of \"state-of-the-art performance (85.14 on VBench)\" is contradicted by Table 3. ContentV(Long) = 85.14 is below Vidu-Q1 (87.41) and Wan2.1-14B (86.22), and ContentV(Short) = 84.11 is below several additional entries. The paper never states whether the official leaderboard rows are short-prompt, long-prompt, or a weighted average, nor does it report the number of prompts per dimension, sampling steps, CFG, resolution, or the aggregation rule used for ContentV. Because the Long/Short difference is about one point, comparable to or larger than the gaps between adjacent rows, selecting the higher Long score for the abstract is not a defensible SOTA claim. The authors should either match the official VBench evaluation protocol exactly and report all settings, or restrict the claim to \"competitive with leading open-source models\" and clearly report both Long and Short scores.","section":"Abstract and §8.2, Table 3"},{"comment":"The RLHF stage is trained with VideoAlign and MPS as reward models, and Table 4 then uses VideoAlign win rates as evidence of improvement from RLHF. This is partially circular: the reported gains may reflect optimization of the evaluation metric itself rather than generalizable quality improvements. Please provide an independent evaluation (for example, a larger human study or evaluation with a different VLM not used in training) and clarify whether the VideoAlign scores in Table 4 are computed on held-out prompts that were not seen during RLHF training.","section":"§6.2 and Table 4"},{"comment":"The user study reports GSB ratios of 1.57, 1.68, and 1.30 against CogVideoX, HunyuanVideo, and Wan2.1, respectively, but the manuscript states only that five human annotators were invited. It does not report the number of prompts or videos evaluated, the total number of ratings, confidence intervals, or significance tests. With five annotators, the claimed \"slight edge over Wan2.1\" is not statistically established. Please report the full protocol and uncertainty estimates.","section":"§8.3"},{"comment":"The stated RLHF objective in Eq. (2) includes a KL-regularization term βD_KL, but the implementation \"ignore[s] the KL constraint for the sake of computational efficiency.\" This means the algorithm optimizes a different objective and removes the standard protection against reward overoptimization. Please state the modified objective explicitly and provide evidence that reward overoptimization is not occurring, for example by plotting reward scores against an independent quality metric over the course of RLHF training.","section":"§6.2, Eq. (2)"}],"minor_comments":[{"comment":"The sentence about watermark detection ends with \"black borders,.\" — the punctuation should be corrected to \"black borders.\".","section":"§3.1"},{"comment":"The sentence \"As shown in the Figure 1, the number of attention layers and dimensions of ContentV are consistent with those of SD3.5L\" appears to refer to Table 1 rather than Figure 1; please fix the cross-reference.","section":"§4.1"},{"comment":"The caption describing flow shift sampling is garbled: \"using small-step updates with more steps in the high-noise stage and small-step updates with fewer large-step updates in the low-noise stage\" should be clarified.","section":"§5.1, Figure 5"},{"comment":"The hardware description says \"The NPU feature 64 GB of High Bandwidth Memory\" — \"feature\" should be \"features\".","section":"§7.1"},{"comment":"The bullet list for the three progressive training stages would be easier to read if each stage name ended with a colon consistently (e.g., \"Stage 3: high-resolution, long-duration training.\").","section":"§5.3"},{"comment":"For VBench leaderboard comparisons, please specify the leaderboard version and the date the official scores were retrieved, since leaderboard values can change over time.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a technical report whose main value is the efficient training recipe on NPUs. The load-bearing problem is the unsupported SOTA claim: Table 3 itself contradicts the abstract, and the evaluation protocol for ContentV's VBench numbers is not specified. I would not reject the paper outright because the underlying recipe may be sound and the claims are fixable with a more careful evaluation section, but the current version needs substantial revision before archival publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the useful part: this is a concrete, reproducible-in-principle recipe for adapting an existing image DiT (SD3.5L) to video by swapping in a 3D VAE (Wan2.1) and adding temporal position info. The staged training schedule (VAE adaptation, low-res short, low-res long, high-res long, SFT, RLHF) is described with enough detail — batch sizes, learning rates, steps — that another lab could try it on alternative hardware. The FID curve for VAE adaptation (294.3 → 13.05 in 1600 steps) is a real, quantitative result. The NPU-only training claim, if the artifacts ship, matters for labs without H100s.\n\nThe soft spots are all in the headline. The abstract says 'state-of-the-art (85.14 on VBench)' but Table 3 shows ContentV(Long) at 85.14, below Vidu-Q1 (87.41) and Wan2.1 (86.22). The body even admits 'only inferior to Wan2.1.' So the SOTA claim is not supported by the paper's own data. Also, 85.14 is the long-prompt score; the short-prompt score is 84.11, and there's no statement about which number matches the leaderboard protocol or how the official rows were aggregated. That's a one-point swing that matters when adjacent competitors are separated by ~0.1–1.0. The user study that produces the 'edge over Wan2.1' uses five annotators, no confidence intervals, so it is suggestive, not conclusive. And the RLHF section trains toward VideoAlign/MPS rewards and then uses VideoAlign as the evaluation metric in Table 4, which partially trains the metric into the result.\n\nNone of this kills the engineering contribution. The recipe is plausible and the staged training results are consistent. But the abstract needs to be rewritten to say 'competitive with leading open-source models at half the parameters' rather than 'state-of-the-art,' and the evaluation protocol needs to be pinned down (prompt split, aggregation, sampling settings, uncertainty). Code and model release are promised; they need to actually appear.\n\nWho's this for: engineering teams training video models on limited or non-NVIDIA hardware. A methods reviewer would find the infrastructure and training details useful. The scientific claims about VBench ranking need fixing, not reinvention. I'd send it to review with the expectation of major revision on the evaluation section, because the substance is there.","headline":"Useful NPU training recipe, but the abstract's SOTA claim is contradicted by the paper's own Table 3.","tokens_in":18123,"tokens_out":2458,"would_cite":false,"duration_ms":28365,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ContentV is an 8B-parameter text-to-video model that the authors report reaching 85.14 on VBench after four weeks of training on 256 NPUs, by minimally adapting Stable Diffusion 3.5 Large.","keywords":["text-to-video generation","diffusion transformer","flow matching","3D variational autoencoder","reinforcement learning from human feedback","efficient training","NPU training","VBench"],"falsifier":"Run ContentV through VBench under a documented prompt split, sampling configuration, and aggregation rule; if the resulting overall score does not reproduce 85.14 and does not stay above HunyuanVideo's reported 83.24, the paper's central ranking and efficiency claim are not established.","tokens_in":16901,"feed_emoji":"🎬","tokens_out":7470,"duration_ms":74747,"temperature":0.7,"pith_summary":"ContentV is an 8-billion-parameter text-to-video model trained on 256 NPUs for about a month, with the authors reporting a 85.14 VBench overall score. The paper's central claim is that a nearly untouched image-generation model, Stable Diffusion 3.5 Large, can be turned into a competitive video generator by swapping its two-dimensional autoencoder for a causal three-dimensional autoencoder, adding temporal position information, and training with staged flow matching plus a light RLHF stage. The efficiency comes from reusing the open-source image model's learned knowledge instead of training video capabilities from scratch. If the claim holds, it lowers the compute barrier for video generation research and shows that NPUs can plausibly substitute for high-end GPUs on this workload.","feed_headline":"An 8B video model trains on 256 NPUs in four weeks","feed_subtitle":"It swaps a 3D autoencoder into Stable Diffusion 3.5 to hit 85.14 on VBench at half the parameters.","key_machinery":"The load-bearing change is the 3D VAE swap: a causal 3D autoencoder with temporal downsample 4 and spatial downsample 8 compresses images and videos into a shared latent space, and the existing SD3.5L image DiT is retrained on those latents for only a few hundred steps. Around that sits a staged flow-matching recipe, moving from short low-resolution clips to long high-resolution clips, with dynamic bucketing by aspect ratio and duration. The third mechanism is a compute-light RLHF loop that makes the flow-matching sampler differentiable by enabling gradients only on a random subset of denoising steps, then rewards the decoded first frame using a CLIP-based reward model, keeping memory within 64 GB per NPU.","core_discovery":"The central discovery is that minimal adaptation of a pretrained image diffusion transformer is sufficient for strong text-to-video generation. ContentV keeps the SD3.5L MMDiT backbone and its text encoder, replaces the 2D VAE with the causal 3D VAE from Wan2.1, and adds temporal position embeddings; the model adapts to the new latent space within roughly 500 steps, and FID at 512 resolution recovers from 294.3 with the unadapted VAE to 13.05 after 1600 steps, close to the original 12.8. Training then moves through stages of increasing duration and resolution under a flow-matching objective, with flow shift 1 during training and 17 during sampling, followed by SFT on a filtered one-million-clip subset and RLHF that backpropagates through randomly selected denoising steps. The authors report an 85.14 VBench overall score for long-prompt evaluation and human-preference win rates against CogVideoX-5B, HunyuanVideo-13B, and Wan2.1-14B.","pith_inferences":["A direct test would be to apply the same VAE-swap recipe to a different pretrained image diffusion backbone; the reported FID recovery curve predicts that a few hundred adaptation steps should restore image quality before video training begins.","Because the RLHF reward is computed on the first frame, the reported motion-quality gains are likely indirect, mediated by the shared denoiser rather than by direct temporal supervision; comparing first-frame rewards against full-video rewards would isolate that effect.","The human-preference edge over Wan2.1 despite a lower VBench score suggests VBench's aggregate weights dimensions differently than human raters do, which could be checked by publishing per-dimension VBench scores for the same videos used in the user study.","Comparing against official leaderboard numbers rather than running every model under one harness is the main source of uncertainty, so a public re-evaluation with fixed prompts, sampling steps, and CFG scale would settle how much of the efficiency claim depends on the evaluation protocol."],"forward_implications":["An existing open-source image diffusion transformer can be reused for video with a VAE swap and a few hundred adaptation steps, so video training inherits image knowledge rather than starting from random weights.","A 256-NPU, four-week budget is claimed to be enough to reach the reported VBench range, a much smaller compute envelope than the hundreds or thousands of GPUs reported for comparable video models.","The staged schedule of image adaptation, short video, long video, and high-resolution training, followed by SFT and RLHF, should transfer as a recipe to other base image models or other 3D VAEs.","Backpropagating through only some denoising steps lets RLHF optimize final video quality without full differentiability, and decoding only the first frame keeps the post-training step within a 64 GB memory limit.","If the evaluation protocol is held fixed, an 8B model built this way can stay competitive with 13B and 14B open models, suggesting raw parameter count is not the dominant factor in VBench scores in this range."],"supporting_citations":[{"why":"Supplies the pretrained SD3.5L MMDiT backbone and text encoder that ContentV adapts for video.","marker":"[12]"},{"why":"Supplies the causal 3D VAE used in the open-source ContentV.","marker":"[46]"},{"why":"Provides the flow-matching objective and sampling procedure used throughout training.","marker":"[27]"},{"why":"Is the benchmark whose 85.14 overall score anchors the main efficiency claim.","marker":"[19]"},{"why":"Inspires the differentiable-sampling RLHF approach with gradients on selected denoising steps.","marker":"[50]"},{"why":"Serves as the initial VLM-based reward model and as a stage-wise evaluation metric.","marker":"[28]"},{"why":"Is the CLIP-based reward model actually used to improve visual quality in RLHF.","marker":"[55]"}],"fun_headline_variants":["Swap a VAE to turn an image model into video SOTA","4 weeks, 256 NPUs: video model beats larger rivals","ContentV: minimal tweaks, maximum video generation efficiency","From SD3.5 to video: one swap, 85.14 VBench"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The main load-bearing premise is that ContentV's reported VBench score was computed under the same evaluation setup, meaning the same prompts, sampling settings, and averaging rule, as the comparison models' official scores; the paper does not report those settings.","fun_headline_variants_meta":{"raw":{"variants":["Swap a VAE to turn an image model into video SOTA","4 weeks, 256 NPUs: video model beats larger rivals","ContentV: minimal tweaks, maximum video generation efficiency","From SD3.5 to video: one swap, 85.14 VBench"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000336,"raw_usage":{"total_tokens":1846,"prompt_tokens":919,"completion_tokens":927,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":535,"completion_tokens_details":{"reasoning_tokens":850}},"tokens_in":535,"tokens_out":927,"duration_ms":10497,"temperature":1.0,"reasoning_tokens":850,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T10:22:23.299361+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run ContentV through VBench under a documented prompt split, sampling configuration, and aggregation rule; if the resulting overall score does not reproduce 85.14 and does not stay above HunyuanVideo's reported 83.24, the paper's central ranking and efficiency claim are not established.","supporting_citations":[{"cited_title":"Deep reward supervisions for tuning text-to-image diffusion models","cited_arxiv_id":null,"evidence_quote":"Inspires the differentiable-sampling RLHF approach with gradients on selected denoising steps."},{"cited_title":"Learning multi-dimensional human preference for text-to-image generation","cited_arxiv_id":null,"evidence_quote":"Is the CLIP-based reward model actually used to improve visual quality in RLHF."}],"review_version":1}