{"id":"272e809f-9511-42f4-90ec-17810ad15400","arxiv_id":"2505.10584","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A technical report describing 2B and 13.4B video diffusion models, a distributed data processing pipeline, large-scale training infrastructure, inference acceleration, and marketing applications, without releasing models or weights.","lead":"ByteDance reports a family of video generation models and the distributed infrastructure used to train them for advertising use cases. The paper is a technical system report, not an experimental study, and its main public commitment is that the data processing pipeline will be open-sourced.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 36% MFU claim lacks a FLOPs definition; as written the headline training-efficiency number cannot be verified, reproduced, or compared.","rationale":"The reader's verdict is REJECT because the paper lacks systematic evaluation and reads as a technical disclosure. That is a fair and well-supported objection: the only quantitative evidence is a single inpainting benchmark, and the text-to-video and image-to-video quality claims rest on qualitative figures. However, as a stress-test of the central claim, the most load-bearing defect is narrower and more specific: the paper's headline quantitative results, 36% MFU and 2.35x inference speedup, are stated without the measurement definitions that make them checkable. The paper contains enough architectural detail to show the MFU number is sensitive to exactly what is counted, but it never provides the FLOPs formula. In the absence of released weights or code for the main models, these numbers are the only quantitative support for the 'industry-level' claim, so their unverifiability is the weakest point in the argument. This warrants CONDITIONAL rather than flat REJECT: the manuscript should be accepted only if the authors add a precise FLOPs definition, hardware assumptions, and measurement methodology for both MFU and the speedup. I disagree with the reader's weakest_assumption only in emphasis: the lack of human evaluation is real, but the missing definition of the headline numerical claims is the more decisive and more precisely fixable flaw.","tokens_in":20139,"tokens_out":2214,"duration_ms":23641,"concrete_test":"Recompute the MFU for Stage 3 (61x640x640, batch 1024, ~O(100K) steps) using the paper's 13.4B parameter count, with and without the >3B adaLN MLPs from Section 5.1, with 115k-token 125-frame sequences, on the stated TP8+CP+ZeRO-DP configuration. Back out the implied per-step wall-clock time from MFU = FLOPs_done / (num_devices x peak_FLOPs_per_device x step_time). Then state explicitly which FLOPs definition reproduces the claimed 36% and whether the implied step time is consistent with the reported cluster scale. If the number only holds under one unstated assumption, the claim needs that definition in the paper.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's strongest quantitative claims are the 36% MFU for the 13.4B model and the 2.35x inference speedup, but neither is accompanied by the definitions needed to check them. MFU is only meaningful if the numerator (model FLOPs) and denominator (hardware peak FLOPs times step time) are precisely specified. The paper does not state which operations are counted: the text encoders and 3D VAE are decoupled from DiT training (Section 5), so their cost may be excluded; CFG is used during inference, so training FLOPs presumably exclude the twin forward pass; attention is full 3D attention with K=1 at the final stages, but sparse attention with K=4 was used earlier (Section 3.2.3); and the paper itself says the dedicated adaLN conditioning MLPs add over 3B parameters (Section 5.1), so whether those FLOPs are included materially changes the number. Without the exact FLOPs formula and the hardware peak-FLOPs assumption, 36% is an unverifiable headline. The 2.35x speedup is similarly under-specified: no hardware, batch size, resolution, sampling steps, caching step interval, or latency breakdown is given for the measurement. This is a precision gap in the argument, not an accusation of misreporting, but it makes the central engineering evidence unfalsifiable as written.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper describes Aquarius, a family of video generation models (2B Single-DiT and 13.4B Multimodal-DiT) aimed at marketing applications. It presents a distributed data curation pipeline, architectural choices, multi-stage flow-matching training, hybrid-parallel infrastructure with a reported 36% MFU, training-free inference acceleration claimed at 2.35x, and downstream applications including digital humans, video inpainting, and video personalization. The only quantitative comparison against an external method is an inpainting evaluation against ProPainter on a 50-video test set (Table 3); the main text-to-video and image-to-video claims rest on qualitative examples and internally developed scoring models.","tokens_in":20434,"tokens_out":4227,"duration_ms":43253,"significance":"If the engineering and performance claims were substantiated, Aquarius would be a useful industrial case study, particularly the explicit tensor-parallel gradient synchronization, fine-grained recomputation selection, and decoupled encoder/DiT training. The data-pipeline and inpainting design descriptions contain reusable recipes. However, as submitted, the key performance claims are not backed by systematic, independently checkable evidence; the paper reads more as an engineering blog post than a verifiable research report.","major_comments":[{"comment":"The claim that the 13.4B model achieves 36% MFU is not verifiable as stated because the FLOPs definition is missing. The paper does not specify which operations are counted in the numerator (e.g., whether the decoupled VAE and text encoders are included, whether the adaLN conditioning MLPs that add over 3B parameters are counted, or which attention variant is used), nor does it state the hardware peak-FLOPs assumption in the denominator. Without this information the number cannot be reproduced or compared with other systems.","section":"Section 5"},{"comment":"The 2.35x inference speedup is reported without a measurement protocol: no hardware, batch size, resolution, number of sampling steps, caching step interval, or latency breakdown is given. The claim that quality is preserved is supported only by an unspecified validation-set check. As written, the acceleration result is not reproducible.","section":"Section 6"},{"comment":"The inpainting comparison against ProPainter is based on 50 short 720p videos resized to 432x240, with a single baseline, no error bars, no statistical significance test, and VFID is not defined. This evidence is too thin to support the sentence that the method 'surpasses ProPainter, the state-of-the-art video inpainting algorithm,' especially because the differences in PSNR/SSIM are small.","section":"Table 3 / Section 7.2"},{"comment":"The central claims of 'exceptional performance', strong instruction following, and realistic commercial-grade digital humans are supported only by qualitative figures and by internally trained scoring models used in the data-filtering pipeline (Section 2.2.2). There is no independent human evaluation protocol, preference study, external benchmark, or comparison with commercial video-generation models for the main text-to-video and image-to-video capabilities. Internal filters cannot validate the headline performance claims.","section":"Sections 1 and 7.1"}],"minor_comments":[{"comment":"The section heading 'Training Efficieny' contains a typo and should read 'Training Efficiency'.","section":"Section 5 heading"},{"comment":"The loss is described as mean squared error, but the displayed expression lacks a square on the norm; it should read \\|u(X_t,y,t;\\theta)-V_t\\|^2.","section":"Equation (2)"},{"comment":"The step counts are given as order-of-magnitude values such as 'O(100K)'; exact or more precise step counts would improve reproducibility.","section":"Table 1"},{"comment":"The open-source framework is called 'Aquarius-Datapipe' in the Abstract and Section 2 but 'Aquarius-Raydata' in Section 8; the naming should be made consistent.","section":"Abstract and Section 8"},{"comment":"It is unclear whether the 10B MoE model is part of the Aquarius family or a separate exploratory experiment; its status and relation to the 2B and 13.4B models should be clarified.","section":"Section 3.2.3"},{"comment":"The paper repeatedly states that the framework is designed for models with hundreds of billions of parameters, but all reported results are at 2B and 13.4B scale; this claim should be tempered or supported with concrete design/experiment evidence.","section":"Abstract and Section 8"}],"recommendation":"reject","confidential_remarks":"The manuscript's main performance claims lack independent evaluation, and the only external comparison is a small inpainting benchmark. The engineering sections are informative, but the evidentiary bar for the paper's central claims is not met; adding the required human evaluation, external benchmarks, and precise measurement definitions would constitute a substantially new submission rather than a routine revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper is best read as an engineering disclosure from ByteDance's marketing-video team, not as a research contribution. The genuinely useful parts are the training-infrastructure details: the explicit gradient-synchronization fix for non-partitioned patchify layers under tensor parallelism, the memory-latency ratio heuristic for selective recomputation, the decoupling of frozen-encoder inference from DiT training to avoid DP mismatch, and the token-balanced bucket design for multi-resolution training. Those are plausible, actionable observations that a practitioner would not easily find elsewhere.\n\nThe soft spots are real but mostly where you'd expect them. The headline numbers are under-specified: the 36% MFU claim does not state which FLOPs are counted (DiT only? includes the 3B adaLN MLPs? excludes text encoders and VAE? which sparse-attention stage?) nor the hardware peak-FLOPs assumption, so it cannot be verified or reproduced. The 2.35x inference speedup has no hardware, resolution, step count, or caching schedule. The main text-to-video and image-to-video claims rest on qualitative figures and internal captioners/quality models; there is no systematic evaluation, no human preference study, and no comparison to commercial systems. The one external comparison, video inpainting against ProPainter, is narrow (50 short videos, no error bars) and uses a separate small DiT, not the main Aquarius model. The abstract itself concedes that proper metrics are planned.\n\nNone of this is fatal for what the paper is trying to do. It reads as a technical report intended to share operational experience, and on those terms it mostly succeeds. I'd send it to a serious referee, but only if the venue is explicit that it accepts this kind of disclosure and the authors are required to add measurement definitions and at least one non-internal evaluation. As it stands, it does not demonstrate a scientific advance.\n\nWho gets value: engineers building large-scale video-gen training stacks, especially on non-NVIDIA accelerators (xPUs). I probably wouldn't cite it in my own work, but I'd bring it to a reading group focused on systems for generative models.\n\nRecommendation: desk review, not desk reject — but the review should ask for precise measurement definitions and a stronger evaluation section before acceptance.","headline":"An honest engineering disclosure with useful training-infra details, whose headline efficiency numbers are too under-specified to check.","tokens_in":20977,"tokens_out":2129,"would_cite":false,"duration_ms":21790,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that a five-part engineering recipe yields industrial-scale marketing-video generation at 2B and 13.4B scales, with 36% training efficiency and a 2.35x inference speedup.","keywords":["video generation","diffusion transformer","flow matching","distributed training","inference acceleration","video inpainting","video personalization","marketing scenarios"],"falsifier":"An independent evaluation would settle it: run the released Aquarius checkpoints on a fixed, diverse prompt set with automatic video-quality metrics and side-by-side human preference against the open models it positions itself alongside, and recompute the 36% MFU from a standard training stage's logs. If the quality gap is large or the MFU and caching numbers do not reproduce, the central claims fail.","tokens_in":19953,"feed_emoji":"🎥","tokens_out":16298,"duration_ms":143749,"temperature":0.7,"pith_summary":"This paper is a system disclosure of Aquarius, a family of diffusion-transformer (DiT) video generation models aimed at marketing and advertising production. Its central claim is that commercial-grade video generation is achievable with a specific, disclosed recipe: a distributed data pipeline, a 2B single-stream DiT for smaller budgets and a 13.4B multimodal-DiT for larger ones, flow-matching training through staged buckets that mix resolutions, aspect ratios, and frame counts, and a parallel training and inference stack. The report quantifies the engineering payoff as 36% model-FLOPs utilization (MFU) for the 13.4B model and a 2.35x inference speedup from training-free diffusion and attention caching. The work matters because it turns what are usually closed industrial choices—data filtering, annotation, parallelism, memory management, caching—into a reproducible blueprint for other teams. The evidence offered is mostly qualitative samples plus one video-inpainting benchmark, so the quality claims rest on those samples and on the reported engineering numbers.","feed_headline":"13.4B video model hits 36% training efficiency, 2.35x speedup","feed_subtitle":"Aquarius discloses the full stack—data, staged training, parallelism, caching—behind commercial marketing video.","key_machinery":"The load-bearing machinery is the integration of five named mechanisms. The 3D VAE with CausalConv3D and spatial-temporal tiling compresses video $4\\times8\\times8$ and supports extrapolation to arbitrary lengths while enabling tile-wise parallel encoding and decoding. The DiT pair—Single-DiT at 2B with cross-attention text conditioning, Multimodal-DiT at 13.4B with dual-stream self-attention and a multimodal large language model text encoder—is the generative backbone, and 3D RoPE (rotary position embedding applied separately to time, height, and width) lets a single model handle mixed aspect ratios and frame counts. Flow matching defines the training objective, with the model predicting the velocity $V_t = X_1 - X_0$ of the linear interpolation between noise and video. The training scaffold is bucket-based multi-stage curriculum plus hybrid parallelism—tensor and sequence parallelism inside a node, optional context parallelism for ultra-long sequences, and ZeRO-style data parallelism outside—with recomputation and offloading chosen by measured memory-latency ratios. Inference acceleration uses training-free caching of DiT-layer offsets and attention outputs, withholding cache for the first ten denoising steps, combined with parallel VAE decoding.","core_discovery":"On its own terms, Aquarius claims that the hard part of industrial video generation is not a single algorithmic breakthrough but the integration of five components, each with a concrete design choice. The 3D variational autoencoder (3D VAE) compresses video by $4\\times8\\times8$ using causal 3D convolutions and spatial-temporal tiling, which allows arbitrary-length videos to be encoded and decoded in parallel. The architecture split is resolved by scale: a 2B Single-DiT (text conditioning through cross-attention, using a multilingual text encoder) for instruction-following use, and a 13.4B Multimodal-DiT (separate visual and text streams that concatenate and self-attend in later layers, with an in-house multimodal large language model text encoder and CLIP-based timestep guidance) for larger reasoning budgets. Training is flow matching on token-equivalent buckets, staged from 320px text-to-image through 61-frame 320px video to joint image-video pretraining at 960px and a 500k-sample supervised fine-tuning stage for human-centric marketing content. On inference, the paper reports DiT caching (reusing output-feature offsets of rear layers across sampling steps) and attention caching, with no caching in the first ten denoising steps, for a total 2.35x speedup. The report further claims the same backbone serves video inpainting through a from-scratch small DiT with temporal MultiDiffusion and video personalization by concatenating reference-image tokens along the sequence axis.","pith_inferences":["This is an inference: the paper compares 2B single-stream against 13.4B multimodal, so the architecture choice is not isolated; a same-parameter-count ablation would be the natural way to see which scaling decision actually drives quality.","This is an inference: the proprietary data layer—aesthetic scorers, human-quality scoring, structured captioning with 7B and 34B captioners, and human-reviewed SFT sets—is likely the hardest part to reproduce, even with the architecture and training code disclosed.","This is an inference: applying the same caching and tile-parallel VAE ideas to open DiT video codebases would test whether the 2.35x speedup is framework-general or specific to Aquarius's architecture and training.","This is an inference: if Aquarius-Datapipe is truly open-sourced, a community benchmark could determine how much of downstream video quality comes from the filtering and annotation cascade itself rather than from model size."],"forward_implications":["A 2B single-stream DiT with a strong multilingual text encoder can carry instruction-following marketing-video tasks, so teams without 13B-scale budgets have a viable smaller entry point.","If reproduced, the 36% MFU figure and the communication-computation overlap, explicit patchify gradient synchronization, and recomputation/offloading balance give other industrial trainers concrete efficiency targets.","The paper's caching recipe—reuse rear-DiT layer offsets and attention outputs, skip caching for the first ten denoising steps—is claimed to deliver a 2.35x inference speedup with stable generation quality.","The bucket strategy makes multi-resolution, multi-aspect-ratio, and multi-duration training a matter of editing a configuration file, because configurations like $\\{1,29,640,640\\}$ and $\\{1,125,320,320\\}$ are token-equivalent.","For video completion, the paper claims a from-scratch flow-matching DiT with temporal MultiDiffusion surpasses ProPainter on PSNR, SSIM, and VFID in as few as four inference steps."],"supporting_citations":[{"why":"Establishes the DiT (Transformer-based diffusion) backbone that Aquarius builds on and scales.","marker":"(Peebles and Xie, 2023)"},{"why":"Supplies the flow-matching objective and velocity parametrization used for all Aquarius training stages.","marker":"(Lipman et al., 2023)"},{"why":"Provides the activation-memory analysis, selective recomputation, and sequence parallelism used to reach 36% MFU.","marker":"(Korthikanti et al., 2023)"},{"why":"Supplies ZeRO optimizer-state partitioning used as the outermost data-parallel memory strategy.","marker":"(Rajbhandari et al., 2020)"},{"why":"Supplies the tensor-parallelism pattern adapted by Aquarius's TP-SP inner parallel group.","marker":"(Shoeybi et al., 2019)"},{"why":"Supplies CausalConv3D, the convolution design behind the 3D VAE's unified spatial-temporal compression.","marker":"(Yu et al., 2024)"},{"why":"Supplies rotary position embedding, extended to 3D RoPE for multi-resolution, multi-aspect, and multi-duration generation.","marker":"(Su et al., 2023)"},{"why":"Supplies MultiDiffusion, used in the inpainting model for temporal consistency on long videos.","marker":"(Bar-Tal et al., 2023)"},{"why":"Provides ProPainter, the baseline the video-completion comparison must beat, and the mask generation pattern used in training.","marker":"(Zhou et al., 2023)"},{"why":"Supplies the training-free diffusion and attention caching precedents that the 2.35x inference speedup is based on.","marker":"(Chen et al., 2024; Zhao et al., 2025)"}],"fun_headline_variants":["Aquarius: full-stack recipe for industrial marketing video","13.4B video gen: 36% MFU, 2.35x speedup, full stack","Marketing video at scale: Aquarius discloses five-component design","2B and 13.4B DiTs: Aquarius shows industrial video path","Aquarius unveils industrial video gen: data, training, inference"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the paper's qualitative video samples and a single video-inpainting benchmark suffice to establish the claimed commercial-grade quality of the 2B and 13.4B models, since the report offers no human evaluation, preference study, or systematic metric for its main text-to-video and image-to-video claims.","fun_headline_variants_meta":{"raw":{"variants":["Aquarius: full-stack recipe for industrial marketing video","13.4B video gen: 36% MFU, 2.35x speedup, full stack","Marketing video at scale: Aquarius discloses five-component design","2B and 13.4B DiTs: Aquarius shows industrial video path","Aquarius unveils industrial video gen: data, training, inference"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000246,"raw_usage":{"total_tokens":1647,"prompt_tokens":1159,"completion_tokens":488,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":775,"completion_tokens_details":{"reasoning_tokens":387}},"tokens_in":775,"tokens_out":488,"duration_ms":3833,"temperature":1.0,"reasoning_tokens":387,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:31:30.721787+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"An independent evaluation would settle it: run the released Aquarius checkpoints on a fixed, diverse prompt set with automatic video-quality metrics and side-by-side human preference against the open models it positions itself alongside, and recompute the 36% MFU from a standard training stage's logs. If the quality gap is large or the MFU and caching numbers do not reproduce, the central claims fail.","supporting_citations":[],"review_version":1}