{"id":"52031b02-d5cf-4333-a369-840fc746fe43","arxiv_id":"2511.12035","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Timeripple cuts vDiT self-attention compute by up to 85% by reusing partial attention scores of spatially and temporally correlated tokens across channels, with VBench quality essentially unchanged.","lead":"Video-generating transformer models spend most of their compute comparing every part of every frame with every other part; this paper shows those comparisons are mostly redundant because neighboring frames and pixels look alike in the model's latent space. It reuses attention scores of similar tokens, cutting attention compute by up to 85% with roughly unchanged video quality — though the headline end-to-end speedup is estimated, not yet measured in hardware.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (3) defines Δ as the intra-window standard deviation of a single token's channel values, not a pairwise similarity metric; the reuse condition does not bound the error in copied partial attention scores.","rationale":"I read the paper as claiming that channel-level spatio-temporal correlations allow safe reuse of partial attention scores. For that claim to hold, the similarity metric used to decide reuse must guarantee that the reused scores are close to the exact ones. The reader's weakest assumption focused on threshold transfer/calibration; mine is more basic: Eq. (3) as written does not appear to measure token-to-token similarity, so the reuse rule may be selecting tokens whose partial scores are not actually similar. This is an internal-consistency concern rather than a generalization concern. The paper's strong empirical results (e.g., PSNR > 30 dB) suggest the authors likely implemented a different metric (perhaps a true RMS difference between two tokens), but the text is ambiguous or erroneous. Because the method is otherwise clearly described and could be reimplemented with the intended metric, the CONDITIONAL verdict remains appropriate: the paper should not be fully accepted until the similarity metric is clarified and the approximation error is either bounded or empirically validated. I partially disagree with the reader's weakest assumption because I identify a different condition that is more directly load-bearing for the method's correctness. The concrete test above would settle whether the metric as written produces valid reuse; if it fails, the central mechanism is unsound. If it passes, the remaining concerns about threshold tuning and speedup estimation are secondary but still justify a conditional acceptance.","tokens_in":14024,"tokens_out":15254,"duration_ms":144369,"concrete_test":"Implement the reuse scheme exactly as described in §3.3 using Eq. (3) on HunyuanVideo with the reported θ schedule. For each layer and timestep, compute the Frobenius norm error between the approximated attention map (with reuse) and the exact attention map. Then, across all token pairs, measure the correlation between Δ and the actual dot-product difference |Q_i·K_j − Q_{i'}·K_j| for reused pairs. If pairs with Δ below threshold show large dot-product errors (e.g., more than 10× the threshold-scaled value) or if the attention-map error is not monotonically controlled by θ, the reuse condition is invalid. This directly tests whether Eq. (3) is a valid similarity measure for partial-score reuse.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central mechanism in §3.3 is that tokens whose channel-window standard error Δ is below a threshold can reuse a neighbor's partial attention scores. But Eq. (3) computes Δ(a) = sqrt((1/K)Σ(a_i − mean)^2) for a window a of token channels; this measures the spread of values within one token, not the similarity between two tokens. The text says 'compute the standard error ∆ between every two adjacent frames,' which implies a pairwise difference, but the formula does not compare two tokens. If Δ is applied to a single token, two tokens can both have small intra-window variance yet very different means (e.g., one all 0.1, another all 10.0), so their dot products with a given key differ enormously. Even if a is interpreted as the difference vector between two tokens, Eq. (3) is the standard deviation of the differences, which is zero for any constant offset (e.g., [100,100,100] has Δ=0). Thus the threshold condition does not guarantee that the reused partial scores are close to the computed ones. The paper does not provide any analysis bounding the approximation error in the attention map as a function of Δ. This is more fundamental than threshold calibration: if the similarity metric is misspecified, the computed savings are based on invalid reuse. The empirical results in Table 2 and Fig. 7 suggest a working implementation, but the manuscript as written does not define the metric precisely enough to re-implement or to verify the approximation guarantee.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Timeripple, a training-free technique to accelerate self-attention in video diffusion transformers (vDiTs) by reusing partial attention scores for tokens that are spatially or temporally correlated along the channel dimension. The authors first analyze attention patterns and attribute them to channel-level spatio-temporal correlations, then design a reuse scheme based on a thresholded standard-error metric, an adaptive per-step threshold schedule (Eq. 4), and evaluate on four vDiTs against several baselines. They report up to 85% self-attention savings, near-identical VBench scores, and up to 2.7x end-to-end speedup.","tokens_in":14259,"tokens_out":5582,"duration_ms":51886,"significance":"The core idea — exploiting channel-level spatio-temporal correlations in latent space rather than imposing attention-map sparsity patterns from LLMs — is timely and potentially influential. The paper includes a broad evaluation on four public vDiTs with 950 VBench prompts, multiple quality metrics, and per-model hyperparameter tables, which is a strength. If the reuse predicate is made precise and the speedup is validated on real hardware, this could be a meaningful step toward efficient video diffusion inference. However, as written, the load-bearing claims rest on an under-specified reuse metric and an estimated, not measured, end-to-end speedup.","major_comments":[{"comment":"The reuse condition is not defined as a pairwise similarity. Eq. (3) computes the standard deviation of a single K-channel window a, but the text says \"compute the standard error ∆ between every two adjacent frames\" and \"Token pairs with ∆ below a predefined threshold.\" If a is a single token's channel window, two tokens can both have small Δ yet very different means, producing very different partial attention scores. If a is instead the elementwise difference of two tokens, then any constant-offset pair (e.g., [1,1,1] vs. [101,101,101]) has Δ=0 even though the partial-score error is large. The manuscript provides no bound on the approximation error in the attention map as a function of Δ. This is load-bearing: the 85% savings claim presupposes that the threshold condition identifies pairs whose partial scores are actually close. The authors must specify exactly which vector enters Eq. (","section":"§3.3, Eq. (3)"},{"comment":"The reported speedups are not measured end-to-end. The text states: \"we estimate speedup by proportionally reducing the self-attention latency based on the amount of computation reduced,\" because no existing kernel supports the resulting unstructured sparsity. Thus the Latency and Speedup columns for Timeripple rows (e.g., 260.85 s and 2.66x for HunyuanVideo) are synthetic, not wall-clock results from an implementation. Since self-attention is only ~78% of runtime, the headline \"up to 2.7× end-to-end speedup\" (Abstract, Introduction) is contingent on a kernel that does not exist. Please relabel these as estimated/theoretical speedups, give the exact formula used, report the overhead of the reuse checks, and either temper the abstract/conclusion claims or implement a prototype to measure actual latency.","section":"§4.2, Table 2 (Performance)"},{"comment":"The adaptive framework is described as an \"analytical model,\" but Eq. (4) is a linear interpolation with four per-model hyperparameters (Table 1) selected so that the measured per-step MSE is flat: the text says \"we select threshold values such that they induce the same MSE at each step.\" This is a fitted calibration, not a predictive model derived from attention error statistics. The manuscript should explicitly call Eq. (4) a calibrated schedule, report sensitivity of results to the Table 1 hyperparameters, and demonstrate that the schedule generalizes beyond the ~10 prompts used in Figs. 8–9 to the full 950-prompt benchmark.","section":"§3.3, Fig. 9, Eq. (4)"}],"minor_comments":[{"comment":"The baselines Mask-1 and Mask-2 are not defined in the text or caption, and the MSE metric is not specified (over tokens? frames? prompts?). Please clarify.","section":"Fig. 7"},{"comment":"Citations are inconsistent: HunyuanVideo is cited as [17] in Table 1 and as [37] in §3.2; CogVideo appears as [14] while CogVideoX is [45]. Please unify the citation style.","section":"References"},{"comment":"The formula computes a standard deviation, but the text calls it \"standard error.\" Also, the notation does not define K or explain how a window is constructed along the temporal, x, or y axes. The threshold values θ_x and θ_y mentioned in the text are not listed in Table 1.","section":"Eq. (3)"},{"comment":"The abstract states \"<0.06% loss on VBench\" without specifying the aggregation; Table 2 shows a 0.14-point loss on Wan2.1 (81.17 vs. 81.03). Report the average or the per-model values explicitly.","section":"Abstract and Table 2"},{"comment":"For Timeripple rows, the \"Theoretical Speedup\" and \"Speedup\" columns are identical, which is confusing because the text says the speedup is estimated. Please add a footnote or legend making clear that these are estimates, not wall-clock measurements.","section":"Table 2"}],"recommendation":"major_revision","confidential_remarks":"The empirical evaluation is broad and the direction is promising, but the two load-bearing issues — the under-specified reuse predicate and the synthetic speedup — need to be resolved before I can recommend acceptance. The quality results are strong enough that a careful revision, including a precise definition of the reuse metric and a tempered presentation of the performance numbers, could make the paper publishable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know about this paper is that the core idea is worth taking seriously: it identifies that RoPE channel groups in vDiTs map to spatial versus temporal attention patterns, and reuses partial attention scores along those channel axes. That is a real departure from prior work that works directly on attention maps, and the evaluation across four models suggests the method is not pure fantasy. The VBench numbers are close to baseline, and the PSNR/SSIM/LPIPS gains over the baselines are consistent. If a kernel actually implemented this and delivered close to the estimated speedup, it would matter for serving video diffusion models.\n\nBut the manuscript has soft spots that need to be addressed before I would trust the numbers. Most importantly, the definition of Δ in Eq. (3) as the standard deviation of a window of token channels does not, as written, measure similarity between two tokens. The text says it is a standard error computed between adjacent frames, but the formula looks like within-token variance. Two tokens can have identical Δ yet very different mean channel values, so the dotted products with a key can differ a lot. The paper provides no bound on the resulting attention-map error from the reuse condition. This could be a fixable clarity issue if the implementation actually uses a pairwise difference, but the current text is not reproducible.\n\nThe speedup also deserves a caveat: it is estimated by proportionally reducing measured self-attention latency, not from a real kernel that skips the unstructured sparsity. The abstract presents 2.7× without that caveat. And the abstract's \"<0.06% VBench loss\" is not matched by any single row in Table 2: the 85% config on HunyuanVideo actually improves VBench, while Wan2.1 loses 0.17%. That kind of cherry-picked summary needs fixing. There is also no held-out tuning protocol for the per-model thresholds, so I would not cite the exact quality-preserving claims without further validation, and no code is released.\n\nThat said, the empirical work is substantial, the method is described well enough to identify the main mechanism, and the limitations about kernel integration are stated honestly. This is not a desk-reject-quality paper. It deserves peer review, but a referee should push hard on the Δ definition, the estimation methodology, and the abstract's precision.","headline":"Timeripple is a well-motivated attention-reuse scheme for vDiTs with a genuinely new channel-level analysis, but the similarity metric in §3.3 is underspecified and the headline speedup is an estimate, so the paper needs careful revision before the claims can be trusted.","tokens_in":14924,"tokens_out":2671,"would_cite":false,"duration_ms":26548,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that the attention patterns in video diffusion transformers are artifacts of channel-level spatial and temporal correlations, making 85% of self-attention computation reusable with near-zero quality loss.","keywords":["video diffusion transformer","self-attention acceleration","spatio-temporal correlation","partial attention score reuse","latent space redundancy","training-free acceleration","adaptive threshold","VBench"],"falsifier":"A concrete check: run a vDiT with the reuse method on a fixed prompt set, and for every reused token compute the true partial attention score it replaced; if more than a few percent of reused scores differ from the true values by an amount larger than the delta test implies, the reuse criterion is failing. Alternatively, a real kernel executing the exact sparse pattern should reproduce the estimated speedup on an H100; an end-to-end wall-clock speedup well below the claimed 2.7x would show the performance gain is not realized.","tokens_in":13750,"feed_emoji":"⚡","tokens_out":7967,"duration_ms":63490,"temperature":0.7,"pith_summary":"This paper tries to establish that the self-attention computation dominating video diffusion transformers (vDiTs) is largely redundant. The authors show that the varied patterns in vDiT attention maps are not intrinsic to attention, but are produced by spatial and temporal correlations among tokens along the channel dimension. They then propose a training-free reuse strategy: when a token's channel-window values are close to a neighbor's along the time, x, or y axis, the neighbor's partial attention score is reused instead of recomputed. On four open vDiT models they report up to 85% fewer self-attention operations with under 0.06% VBench quality loss, and an estimated up to 2.7x end-to-end speedup. Because self-attention accounts for about 78% of vDiT inference time, this targets the dominant bottleneck without retraining.","feed_headline":"85% of video-attention compute is skippable via channel reuse","feed_subtitle":"Reusing partial scores of similar tokens along time, x, and y axes keeps quality loss below 0.06%.","key_machinery":"The mechanism is channel-windowed partial-score reuse. For each query and key token, the method computes the standard error across a small channel window with its immediate neighbor along the temporal, x, and y axes (Eq. 3); passing any axis's threshold marks the token reusable. Reusable tokens borrow the neighbor's already-computed partial dot product channel-by-channel, while only non-reusable tokens are sparsely computed, using an OR-aggregated mask to decide what to skip. A second component, the adaptive threshold schedule (Eq. 4), scales the reuse threshold linearly between i_min and i_max denoising steps, matching the observed decay of reconstruction error so the induced error stays ro","core_discovery":"On the paper's own terms, the central claim is that the spatial and temporal structure of video latents is so strong that per-channel dot products forming the attention map are locally constant along the time, x, and y axes, in both query and key. Because rotary position embedding splits channels into those three semantic groups, tokens adjacent along one axis have near-identical values in the corresponding channels, so their partial attention scores are nearly identical. The paper argues that a cheap standard-error test (window size 2) can mark such tokens as reusable, allowing up to 85% of dot-product work to be skipped while the final video stays essentially unchanged (VBench drop under 0","pith_inferences":["We infer that the reported speedup, estimated by proportionally reducing measured self-attention latency, will only become a wall-clock win once a kernel or IO-aware algorithm implements the unstructured sparse pattern; until then the 2.7x figure is prospective.","We infer that the same channel-correlation reuse principle could transfer to other RoPE-based generative models with axis-wise channel groups, such as image diffusion transformers, a direction the paper does not pursue.","We infer that the threshold schedule's prompt-independence, observed on about ten prompts, deserves a wider calibration test before relying on it across the full 950-prompt benchmark suite.","We infer that the single-axis, window-size-2 similarity test is conservative; a joint spatial-temporal tolerance could yield higher savings, though the paper does not evaluate this."],"forward_implications":["The reuse is training-free and only assumes a RoPE layout with time/x/y channel groups, so it transfers across vDiT architectures; the paper demonstrates this on four open models.","At equal compute savings, reuse keeps an order of magnitude lower output MSE than token masking, implying that redundant attention work should be reused, not zeroed.","Because reuse happens inside a single self-attention pass with no cross-step caching, its memory overhead is small compared with methods that store intermediate features.","The threshold schedule depends on the denoising step, not the prompt, so one table of thresholds applies across inputs, simplifying deployment.","Reuse is compatible with existing sparse masking: combining them yields a further speedup with only minor quality loss, showing the two strategies are complementary."],"fun_headline_variants":["Timeripple reuses attention to cut video diffusion compute by 85%","Spatio-temporal reuse: 85% less attention work in video DiTs","Video attention speedup: 85% compute saved via channel reuse","Timeripple saves 85% attention compute with <0.06% quality loss","Reusing attention scores along time and space speeds vDiTs 85%"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that tokens whose channel-window values are within a small standard error of a neighbor produce partial attention scores close enough that reusing them — under a threshold schedule calibrated on about ten prompts — keeps output error negligible across the full 950-prompt benchmark suite.","fun_headline_variants_meta":{"raw":{"variants":["Timeripple reuses attention to cut video diffusion compute by 85%","Spatio-temporal reuse: 85% less attention work in video DiTs","Video attention speedup: 85% compute saved via channel reuse","Timeripple saves 85% attention compute with <0.06% quality loss","Reusing attention scores along time and space speeds vDiTs 85%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000264,"raw_usage":{"total_tokens":1437,"prompt_tokens":738,"completion_tokens":699,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":482,"completion_tokens_details":{"reasoning_tokens":599}},"tokens_in":482,"tokens_out":699,"duration_ms":6453,"temperature":1.0,"reasoning_tokens":599,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T22:06:41.444754+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete check: run a vDiT with the reuse method on a fixed prompt set, and for every reused token compute the true partial attention score it replaced; if more than a few percent of reused scores differ from the true values by an amount larger than the delta test implies, the reuse criterion is failing. Alternatively, a real kernel executing the exact sparse pattern should reproduce the estimated speedup on an H100; an end-to-end wall-clock speedup well below the claimed 2.7x would show the performance gain is not realized.","supporting_citations":[],"review_version":1}