{"id":"57c30cca-556f-41b3-bcd1-dd8fff6cde39","arxiv_id":"2509.16518","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"FG-Attn accelerates video DiT inference by computing attention over only the relevant Mx1 key slices per query group, using an asynchronous gather-load kernel, and reports up to 1.65x end-to-end speedup with negligible quality degradation.","lead":"This paper introduces FG-Attn, a sparse attention method for video diffusion models that skips individual key slices instead of whole blocks, reporting around 1.5x average speedups with minimal quality loss. It matters because attention dominates video diffusion inference, so a low-overhead way to exploit fine-grained sparsity could cut the cost of long, high-resolution video generation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Average-query mask (§4.4) can skip keys essential for individual queries; without recall measurement, the 'negligible degradation' claim is unsupported.","rationale":"The reader's weakest assumption correctly identifies the average-query masking heuristic as the main unvalidated load-bearing component. The paper's central claim is a speedup with negligible quality loss; the speedup side has a plausible kernel design and code release, but the quality side depends on the mask selecting the right keys. The qavg heuristic is a strong approximation that can fail precisely when query groups cross semantic boundaries. Since the paper does not measure mask recall or per-prompt quality variance, the claim of negligible degradation is not yet established. The malformed threshold equation in §4.4 adds to the concern by preventing reproduction. The reader's CONDITIONAL verdict is appropriate: the method is promising, but the pooling strategy needs direct validation. I agree with the reader's focus and see no need to change the verdict, though the concrete recall test would substantially strengthen or refute the quality claim.","tokens_in":15143,"tokens_out":4869,"duration_ms":48086,"concrete_test":"Run a mask-recall audit on Wan 2.1 1.3B 480p: for 10 prompts across sampled denoising iterations, layers, and heads, compute ground-truth per-query masks by marking key k as essential for query q_i when its attention weight (or pre-softmax score) exceeds threshold τ. Then build the qavg mask by thresholding k·qavg with the same τ. Report recall (fraction of essential keys captured) and missed attention mass (total attention weight of essential keys not captured). If recall < 95% or missed attention mass exceeds ~1% for any prompt, the pooling heuristic is not safe for that prompt. Also re-run VBench per-prompt with 3 seeds and check whether any prompt drops by more than 0.05 in any metric.","verdict_should_be":"UNCHANGED","load_bearing_attack":"FG-Attn's quality-preservation claim rests on the average-query masking heuristic (§4.4): a key k is loaded for a group of M queries only if k·qavg passes a threshold, where qavg is the mean of the queries. This is sound only if all M queries in a slice have similar attention patterns. The paper provides no evidence for that. Adjacent latent tokens in video can straddle object boundaries, motion discontinuities, or attend to different text tokens; in such cases a key that scores low against qavg can still have a high attention weight for one individual q_i. If that key is skipped, the softmax is computed over a subset that excludes an essential score, and the output changes materially. The paper validates quality only with aggregate VBench numbers and three visual prompts; aggregate scores can hide per-prompt failures, and no error bars are reported. Additionally, the pooling threshold is not actually specified: §4.4 gives 'exp(k·qavg /√D)/D < τ1' with τ1 undefined, making the mask-generation rule irreproducible. Until mask recall is measured—the fraction of per-query significant keys retained by the qavg mask—there is no basis to conclude that quality is preserved for the pooling variant.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FG-Attn, a fine-grained sparse attention mechanism for video diffusion transformers. Instead of skipping entire M×M blocks of attention scores as in block-sparse attention, FG-Attn skips M×1 slices (M≥16, N=1 in the tile), where a slice corresponds to M queries and a single key. To make this efficient on H100 GPUs, the authors introduce an asynchronous gather-load primitive that packs sparse key/value vectors into tensor-core-compatible tiles in shared memory. Two training-free mask-generation strategies are proposed: a caching strategy that reuses the significant-slice mask across denoising iterations, and a pooling strategy that uses the average query of a group to score keys. The paper reports end-to-end speedups on Wan 2.1 and HunyuanVideo, with VBench quality metrics showing negligible degradation, and claims that FG-Attn can supersede existing block-sparse attention methods.","tokens_in":15469,"tokens_out":5035,"duration_ms":43789,"significance":"If the reported results hold, FG-Attn addresses a real bottleneck: attention computation dominates video DiT inference, and existing 64×64 block-sparse methods leave much of the fine-grained sparsity unexploited. The M×1 slice granularity, combined with the asynchronous gather-load primitive, is a plausible and interesting systems contribution, and the open-source code link is a strength. The paper also provides a useful quantitative analysis of attention-map sparsity at different block sizes (Table 2). However, the central quantitative claim is undermined by inconsistent speedup numbers across the abstract, the body, and the conclusions, as well as by the absence of the claimed FlashInfer and FlashAttention-3 comparisons in the experimental section. The quality-preservation claim also rests on the average-query heuristic whose threshold is undefined and whose recall is never measured. These issues are fixable within the scope of the paper, so the work warrants revision rather than rejection.","major_comments":[{"comment":"The speedup numbers are internally inconsistent: the metadata abstract reports up to 1.40×/1.18× average over FlashAttention 3; the full-text abstract reports average 1.55×/1.41× for 480p/720p; the body (§5.2) reports an average 1.48× and up to 1.65×; and Fig. 12 reports attention-kernel speedups of 1.93× average and 2.38× max. The experiments in Section 5 compare against a PyTorch baseline, SparseVideoGen, and RadialAttention, but not against FlashAttention 3 or FlashInfer, both of which appear in the abstract/related work. This makes the central speedup claim irreproducible. Please reconcile all reported numbers and provide a table of raw end-to-end times with clearly stated baselines.","section":"Abstract (metadata), §5.2, §8"},{"comment":"The average-query masking strategy is not sufficiently validated. The threshold in Eq. (the formula with exp(k·qavg/√D)/D < τ1) is undefined: τ1 is never specified in the text or in Section 5.1 (the cached threshold is given as 0.5/N, but the pooling threshold is not). More importantly, the quality claim 'negligible degradation' is supported only by aggregate VBench scores in Table 3 and three visual examples. No error bars are reported, and per-prompt failures can be hidden by aggregate scores. Since the pooling mask is derived from qavg rather than from each individual query, a key that is important for one query in the group may be skipped if it scores low against qavg. Please report mask recall (fraction of significant per-query keys retained by the qavg mask) and per-prompt quality metrics (e.g., PSNR/LPIPS against the full-attention output, or a dispersion measure of VBench scores)","section":"§4.4 and Table 3"},{"comment":"The memory overhead of the sparse-index mask is stated as O(BHN^2/M) and dismissed as 'negligible runtime overhead,' but the absolute memory footprint is never quantified. For N≈74,000 (720p, 5s), M=128, and a typical number of heads, the cached mask [B,H,N/M,N] of 32-bit integers can be several GB per batch. This is not negligible for the caching strategy and may affect its feasibility on memory-constrained GPUs. Please report the measured HBM usage for the evaluated configurations and separate memory overhead from runtime overhead.","section":"§4.2 and §4.5"},{"comment":"The speedup plots and ablation do not report the number of runs, variance, or confidence intervals. The normalized end-to-end times in Fig. 11 appear to be single measurements; the ablation in Fig. 13 has no error bars. Given that diffusion sampling is stochastic, run-to-run variance in generation time (due to dynamic masking and asynchronous loads) should be characterized. Please include at least 3–5 repeated runs for each configuration and report mean ± std, or justify why variance is negligible.","section":"§5.2, Fig. 11-13"}],"minor_comments":[{"comment":"Typo: 'altenative' should be 'alternative' in the description of the second mask-generation strategy.","section":"§4.1"},{"comment":"Typo: 'spatio templaral tokens' should be 'spatio-temporal tokens'.","section":"§5.1"},{"comment":"Legend labels 'FGSpAttn-pooled' and 'FGSpAttn-cached' are inconsistent with the text's 'FG-Attn-pooling' and 'FG-Attn-cached'. Please use one consistent naming throughout.","section":"§5.2, Fig. 11-12"},{"comment":"The notation for thresholds is inconsistent: τcached is mentioned in the text, the pooling formula uses τ1, and Section 5.1 introduces a threshold of 0.5/N for the cached strategy. Please define every threshold symbol at first use.","section":"§4.4"},{"comment":"The FLOP-reduction estimates for 16×16 blocks (70%) and 128×1 slices (55%) are not tied to the measured speedups. Consider adding a small table that maps sparsity, FLOP reduction, and measured kernel speedup for each configuration, so the reader can distinguish theoretical savings from achieved speedups.","section":"§1 and §3.2"},{"comment":"Some references are incomplete or inconsistently formatted (e.g., 'Spargeattention' vs. 'SpargeAttention'; several arXiv entries lack version numbers). Please proofread the reference list.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The core idea is promising and the systems contribution is interesting, but the manuscript currently contains multiple conflicting speedup figures (metadata abstract vs. full-text abstract vs. body) and the pooling threshold is unspecified. These are serious reproducibility issues, though they appear fixable. I would ask the authors to provide a single consistent set of measured numbers, a full description of the pooling threshold, and a recall/quality analysis of the average-query mask before a final decision. The mismatch between the abstract's FlashInfer/FlashAttention-3 claims and the experiments actually reported should also be resolved explicitly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"FG-Attn is a real systems contribution: the Mx1 slice-level skipping and the asynchronous gather-load primitive are genuinely new relative to the block-sparse work that dominates this area, and the end-to-end speedups on Wan 2.1 and HunyuanVideo are the kind of numbers that make you pay attention. But the paper reports three different speedup figures across the abstract and body, and the quality-preservation claim rests heavily on a heuristic that isn't actually stress-tested. Read it as a promising kernel paper with a couple of loose ends, not as a settled result.\n\nWhat is actually new: the observation in Figure 6 and Table 2 that attention maps in video DiTs have fine-grained sparsity that 64×64 blocking throws away. That's a real finding. The gather-load kernel that packs sparse keys into tensor-core-friendly tiles is a credible engineering contribution, and the caching mask strategy, which reuses a mask across denoising iterations, is a sensible training-free approach. The paper is honest that the pooling variant is a lightweight alternative, and it does compare against SparseVideoGen and Radial Attention rather than only against baselines.\n\nSoft spots: the numbers don't line up. The metadata abstract says 1.40×/1.18× over FlashAttention 3; the full-text abstract says 1.55×/1.41× for 480p/720p; the body reports 1.48× average and 1.65× max. That's not a fatal flaw—these may be different configurations—but it makes an independent check harder. The 64×64 sparsity number in Section 1 (15%) doesn't match Table 2 (22.8%). The pooling threshold τ1 in §4.4 is literally never defined; the line `exp(k·qavg/√D)/D < τ1` is a typo or an incomplete thought. And the quality evaluation is aggregate VBench plus three visual prompts per model, with no error bars and no measurement of mask recall. The pooling heuristic—using the mean query to decide which keys matter for all M queries—could drop a key that is essential for one individual query, and the paper doesn't test that. These are all fixable, but they should be fixed.\n\nWho should read it: anyone working on efficient inference for long-context diffusion models or on GPU kernels for sparse attention. It deserves a serious referee. Send it to review, but ask for consolidated speedup numbers, a defined threshold, and ideally a recall analysis or per-prompt quality spread. The central speedup is plausible, and the gather-load primitive is worth a look regardless.","headline":"Genuinely new fine-grained sparse attention kernel with plausible video-DiT speedups, but the quality claim rests on an under-tested heuristic and the numbers need reconciling.","tokens_in":15985,"tokens_out":2876,"would_cite":true,"duration_ms":24473,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that skipping attention scores as Mx1 slices rather than MxM blocks makes video diffusion generation over 1.5x faster with negligible quality loss.","keywords":["sparse attention","video diffusion transformer","inference acceleration","GPU kernel","attention mask","fine-grained sparsity","tensor cores","diffusion model inference"],"falsifier":"Take a set of prompts, compute full attention maps, mark every Mx1 slice containing at least one score above the threshold used by FG-Attn, and compare that ground-truth slice set with the slices selected by FG-Attn-cached and FG-Attn-pooling; if the predicted mask's recall is well below 100% on typical prompts, the claimed quality preservation is not supported. Additionally, run FG-Attn-pooling over a prompt set and check per-prompt quality-metric variance against the baseline—large variance would show the mean-query heuristic is prompt-dependent.","tokens_in":15017,"feed_emoji":"🎬","tokens_out":5906,"duration_ms":49874,"temperature":0.7,"pith_summary":"The paper's central claim is that attention maps in video diffusion transformers are sparse at a much finer grain than current coarse block-sparse methods assume, and that this sparsity can be harvested for real speedups. It proposes FG-Attn, which skips attention-score computation per Mx1 slice—a block of M contiguous queries evaluated against a single key—instead of per MxM block. To keep the kernel efficient, it introduces an asynchronous gather-load primitive that fetches only the relevant sparse key/value vectors into shared memory and packs them into tensor-core-compatible tiles, hiding address-generation latency behind attention computation. Two training-free mask strategies decide which slices to keep: caching the mask from an earlier denoising iteration, and thresholding dot products of a group's mean query. On state-of-the-art video models, FG-Attn reports an average 1.48x end-to-end speedup, up to 1.65x, with essentially unchanged aggregate quality.","feed_headline":"Fine-grained attention slicing speeds video diffusion up to 1.65x","feed_subtitle":"Skipping Mx1 score slices instead of whole blocks cuts generation time with near-unchanged video quality.","key_machinery":"The central object is the Mx1 slice of the attention-score matrix: one column of scores produced by M contiguous queries against a single key. FG-Attn decides at this granularity which slices to compute, skipping 64x64 or 128x128 blocks entirely and instead keeping only the slices that the mask marks as significant. The enabling mechanism is the asynchronous gather-load primitive, which takes the sparse list of key/value indices, distributes address generation across threads and warps, and loads exactly those vectors into shared memory in a swizzled, tensor-core-ready tile while the compute side of the kernel proceeds in parallel. The mask itself comes from two training-free procedures: a ca","core_discovery":"FG-Attn establishes that the attention-score matrices of video diffusion transformers contain abundant fine-grained sparsity: dropping to 16x16 blocks reveals roughly 70% of scores below threshold, versus only about 22% at 64x64 blocks, yet generating with only 20% of scores still yields valid videos. The method exploits this by treating each Mx1 slice (M=128 in the H100 implementation) as the unit of skipping, and by loading only the keys and values corresponding to the kept slices. The load is implemented with a new asynchronous gather-load primitive that computes addresses for sparse vector indices in parallel threads and issues asynchronous memory loads, so the overhead does not sit on t","pith_inferences":["If the reported slice-level sparsity and cross-iteration stability generalize, the same Mx1 skipping plus gather-load recipe should transfer to other long-context diffusion workloads (3D, audio, image) and to LLM prefill, though the paper itself flags that memory-bound decoding may not hide the gather latency.","The average-query heuristic makes a testable locality assumption: adjacent spatial-temporal embeddings have similar attention patterns. A direct measurement of the divergence between top-k keys per mean query and top-k keys per individual query would give a sharper bound on when FG-Attn-pooling degrades than aggregate quality scores do.","Mask recall—how many truly significant slices the cached or pooled mask retains—would be a stronger quality gate than VBench aggregates; per-prompt quality variance would reveal whether threshold choices work uniformly across prompts or only on average."],"forward_implications":["Video diffusion attention is far sparser than block-sparse methods exploit; switching from 64x64 blocks to 16x16 or Mx1 granularity exposes roughly 3x more skippable computation on typical attention maps.","FG-Attn delivers average 1.48x (up to 1.65x) end-to-end video generation speedup on single-H100 runs of Wan 2.1 1.3B/14B and HunyuanVideo, and the attention kernel itself is about 2x faster than the PyTorch baseline.","Because the mask can be cached across denoising iterations, the one-time cost of computing a full attention map is amortized over the remaining iterations, making the sparse kernel nearly free of masking overhead.","FG-Attn's slice-based skipping is a drop-in replacement for block-sparse kernels: it matches or exceeds the speed of prior coarse-grained sparse attention methods on the same models, and learned or static masks from those methods can be fed directly into FG-Attn.","The method is training-free, so it can be applied to existing video diffusion models without fine-tuning or retraining."],"fun_headline_variants":["Skip fine-grained tiles to speed video diffusion 1.4x","FG-Attn: 70% sparsity, 1.4x faster video generation","Fine-grained attention sparsity accelerates video diffusion","Slicing attention finely cuts video diffusion latency 1.4x","Exploiting fine-grained sparsity speeds video generation 1.4x"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the set of significant attention slices for a group of queries can be reliably predicted from the group's mean query (or from the previous denoising iteration); if queries within a group attend to genuinely different keys, or patterns shift between iterations, the mask drops essential scores and quality suffers.","fun_headline_variants_meta":{"raw":{"variants":["Skip fine-grained tiles to speed video diffusion 1.4x","FG-Attn: 70% sparsity, 1.4x faster video generation","Fine-grained attention sparsity accelerates video diffusion","Slicing attention finely cuts video diffusion latency 1.4x","Exploiting fine-grained sparsity speeds video generation 1.4x"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000282,"raw_usage":{"total_tokens":1536,"prompt_tokens":806,"completion_tokens":730,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":550,"completion_tokens_details":{"reasoning_tokens":635}},"tokens_in":550,"tokens_out":730,"duration_ms":6705,"temperature":1.0,"reasoning_tokens":635,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T16:03:44.594628+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of prompts, compute full attention maps, mark every Mx1 slice containing at least one score above the threshold used by FG-Attn, and compare that ground-truth slice set with the slices selected by FG-Attn-cached and FG-Attn-pooling; if the predicted mask's recall is well below 100% on typical prompts, the claimed quality preservation is not supported. Additionally, run FG-Attn-pooling over a prompt set and check per-prompt quality-metric variance against the baseline—large variance would show the mean-query heuristic is prompt-dependent.","supporting_citations":[],"review_version":1}