{"id":"43ace1b7-ed16-4363-93d0-603308283315","arxiv_id":"2509.10312","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"ClusCa accelerates diffusion transformers by clustering tokens and computing only one token per cluster, achieving 4.96x FLOPs speedup on FLUX with ImageReward preserved.","lead":"This paper introduces ClusCa, a method that speeds up diffusion transformers by clustering image tokens and computing only one representative token per group at each denoising step, then sharing its features with the rest. It reports up to 4.96x FLOPs reduction on FLUX and 6.21x on HunyuanVideo while roughly maintaining or improving quality metrics.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Cluster-stability/co-variation assumption behind one-token-per-cluster propagation is load-bearing but only qualitatively supported; no ARI at cycle length or update-coherence metric is reported.","rationale":"I considered the more obvious reporting issues—FLOPs-vs-latency labeling and the undefined O parameter—but neither attacks the validity of the mechanism; they can be fixed by rewording. The propagation rule is the method, and its success hinges on the cluster-stability and co-variation assumptions. The paper supplies ARI for adjacent steps and a 9-token visualization, which is suggestive but not the needed evidence for N-1 reuse. A single oracle experiment can settle the question. The reader's conditional verdict remains appropriate; no adjustment is needed.","tokens_in":17796,"tokens_out":11143,"duration_ms":103085,"concrete_test":"On DiT-XL/2 with the Table 3 setting N=6, K=16, γ=0.005, run the unmodified model at each skipped timestep to obtain ground-truth features, and compare against ClusCa's propagated cache. Report (a) ARI between the clusterings at the cycle-start step t and the last step of that cycle t-N+1, and (b) the median cosine similarity between the selected token's true feature change and each co-clustered token's true feature change. If ARI at Δt=N-1 is below about 0.6, or the median update-cosine is below about 0.9, the one-token-per-cluster propagation is not faithful and the quality results are not explained by cluster stability.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.3 reuses K-Means cluster assignments from a full-calculation step for N-1 skipped timesteps, and Section 3.4 propagates the single computed token of each cluster to all other members via C(x_i)=γ·μ(i)+(1-γ)·C(x_i). The central claim that this preserves quality depends on two unquantified assumptions: (i) cluster memberships remain valid over the full cache cycle, and (ii) one representative's feature change is representative of all members' feature changes. Observation 2 (§3.2) reports ARI above 0.8 for adjacent timesteps, but Tables 1-3 use N=5-7, so ARI at Δt=N-1 is the relevant quantity and is not given; Figure 5's caption mentions Δt but no long-gap numbers appear. Observation 1's dynamic part ('high consistency in motion patterns') is supported only by the 9-token PCA sketch in Figure 2b; Figure 2a quantifies static intra-cluster distance, not co-variation of updates. Thus the method's error-accumulation claim is not yet backed by a measurement of the exact property the algorithm relies on.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ClusCa, a training-free, plug-and-play acceleration method for diffusion transformers. At the first timestep of each cache cycle, all tokens are computed and clustered with K-Means; at the following N-1 timesteps only one token per cluster (K tokens total) is computed, and its updated feature is propagated to the other members of the cluster by a weighted combination with the previously cached feature. Experiments on DiT-XL/2, FLUX.1-dev, and HunyuanVideo report large FLOPs reductions (up to 4.96x on FLUX and 6.21x on HunyuanVideo) with roughly preserved or even improved quality metrics, plus ablations over the propagation ratio, cluster count, and cache interval.","tokens_in":17969,"tokens_out":5798,"duration_ms":48645,"significance":"If the reported results hold, ClusCa is a meaningful contribution to the feature-caching literature: it is training-free, architecture-agnostic in principle, and evaluated on external benchmarks (ImageNet, DrawBench, VBench) against recent baselines (FORA, ToCa, DuCa, TaylorSeer). The use of ARI to quantify cluster stability over timesteps is a step in the right direction, and the code release supports reproducibility. The main caveats are that the headline acceleration figure is FLOPs-based rather than wall-clock-based, and that the central cluster-stability assumption is validated only at adjacent timesteps, not at the cache-cycle lengths actually used in the experiments.","major_comments":[{"comment":"The abstract's claim of \"4.96x acceleration on FLUX\" is a FLOPs-based speedup (748.48 TFLOPs vs 3719.50 TFLOPs), while the same configuration (ClusCa, N=6, O=1) shows a wall-clock latency speedup of only 3.63x (7.10s vs 25.82s) in Table 1. Since the paper is aimed at inference acceleration, the user-facing metric is latency, and the abstract should state both numbers and clearly label which one is being reported; otherwise the headline overstates the practical speedup by roughly 37%.","section":"Abstract and Table 1"},{"comment":"Observation 2 reports ARI above 0.8 only for adjacent timesteps, but ClusCa reuses cluster assignments for N-1 skipped timesteps with N=5, 6, and 7 in Tables 1-3. The load-bearing assumption is that clusters remain valid across the full cache cycle, so the paper should report ARI at Δt = N-1 (at least for N=5,6,7) and, ideally, the correlation of per-token feature updates within a cluster versus across clusters. The current PCA visualization in Figure 2(b) covers only nine tokens and is qualitative, so the co-variation claim is not yet backed by a measurement of the exact property the propagation rule relies on.","section":"Section 3.2 and 3.3"},{"comment":"The cache-update equation C(x_i) = γ·μ(i) + (1-γ)·C(x_i) uses C(x_i) on both sides without distinguishing the old and new cache values, and the behavior of the update when O=1 is essentially a moving average between a randomly selected representative token's feature and the previous cached feature. Because a different token is selected randomly at each partial-calculation step, the propagation injects stochastic variation into non-computed tokens; the paper should either provide a variance or error analysis of this mechanism or include an ablation comparing random selection against a deterministic choice such as the token nearest to the cluster centroid.","section":"Section 3.4 and Section 4.5"}],"minor_comments":[{"comment":"The text states that at N=5 the ImageReward values of ToCa and DuCa decrease to 0.9731 and 0.9896, respectively, but Table 1 lists no such values; the corresponding rows show ToCa(N=6) at 0.9802 and DuCa(N=5) at 0.9955. Please correct the text to match the table or clarify which configurations are being cited.","section":"Section 4.2 vs Table 1"},{"comment":"The caption says Δt is the timestep span between compared clusters and also describes the plot as \"ARI between adjacent timesteps\"; please make clear whether the reported ARI is for adjacent steps only or for a range of Δt values, and if so, which Δt values are used.","section":"Figure 5 caption"},{"comment":"References [19] and [20] appear to be the same paper (TaylorSeer, arXiv:2503.06923) and are cited in Sections 2.2 and 3.1 with different numbers; please merge them or renumber consistently.","section":"References"},{"comment":"The \"Efficient Attention\" column is not defined in the table or the text; please state what the checkmark means and whether all methods, including ClusCa, use the same attention implementation when latency is measured.","section":"Table 1"}],"recommendation":"major_revision","confidential_remarks":"I concur with the conditional assessment. The core idea is simple and the experimental coverage is strong, but the FLOPs-versus-latency framing of the headline result and the missing full-cycle cluster-stability measurement are exactly the points that need to be fixed before publication. I would not reject the paper, but the authors should be required to report wall-clock speedups prominently and to directly measure ARI/co-variation at the cache intervals actually used."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid, incremental engineering paper that should go to review, but the authors need to fix a few things before it's citable at face value.\n\nWhat's new: the combination of K-Means spatial clustering with temporal feature caching, using a propagation ratio to mix cluster representatives into cached features. ToCa/DuCa do token-wise caching, TaylorSeer does forecasting, none exploit spatial grouping. The mechanism is simple and training-free, and the experiments span DiT, FLUX, and HunyuanVideo, with competitive FID/ImageReward/VBench numbers. The code is on GitHub, and the ablation on gamma is actually informative.\n\nWhere it's soft: (1) The abstract's \"4.96x acceleration\" is FLOPs; wall-clock is 3.63x in Table 1. That's a misleading headline. (2) The paper says they fix O=1 token per cluster \"throughout our implementation\" but Table 1 shows O=2 achieving the best ImageReward (0.9961 vs 0.9825 for O=1). That contradicts the text. (3) The load-bearing assumption that cluster assignments remain valid over the full cache cycle (N-1 steps) is only supported by ARI for adjacent timesteps. The figure caption mentions Δt but the paper never reports ARI at Δt=4-6. That's a direct gap in the evidence. (4) No error bars or significance testing on ImageReward/VBench, which for 200 prompts is a minor but real issue.\n\nThe cluster-stability concern is real but not fatal. The propagation ratio gamma is small (~0.005), so the update is mostly the old cached value with a small correction. The ablation shows that tiny correction helps a lot. So even if clusters drift, the damage is limited. I'd like to see the long-gap ARI and an update-coherence metric, but I don't think the method collapses without them.\n\nThe citations are fine; a few baselines are from the same group, but they're also compared against FORA and TeaCache. The \"16 tokens\" claim is model-specific (K=16 for FLUX), not a universal constant.\n\nBottom line: This deserves a serious referee. It's a useful incremental contribution, not a breakthrough. If the authors fix the FLOPs/wall-clock language, reconcile the O=2 results with the O=1 claim, and add long-gap ARI, I'd cite it.","headline":"Solid, incremental training-free acceleration for diffusion transformers: the spatial-clustering plus temporal-caching combination is new and works in experiments, but the headline FLOPs number, the O=1/O=2 inconsistency, and missing long-gap cluster-stability analysis need fixing before the results are taken at face value.","tokens_in":18575,"tokens_out":5224,"would_cite":true,"duration_ms":42658,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By computing only one token per spatial cluster during skipped denoising steps, ClusCa claims up to 4.96x faster FLUX image generation and 6.21x faster HunyuanVideo generation while preserving or slightly improving quality.","keywords":["diffusion transformer","feature caching","spatial clustering","K-Means","text-to-image generation","text-to-video generation","inference acceleration","token reuse"],"falsifier":"Measure intra-cluster feature distance at each skipped timestep inside a cache cycle on prompts with small fast-moving objects or rapid lighting changes; if the distance grows past the point where the weighted cache update diverges from a full recomputation, the affected regions should show ghosting or detail loss and the speedup-quality trade-off should collapse.","tokens_in":17517,"feed_emoji":"⚡","tokens_out":6653,"duration_ms":56097,"temperature":0.7,"pith_summary":"ClusCa aims to prove that diffusion transformers hide a second, largely untapped redundancy: tokens within a single denoising step are similar enough that a few representatives can carry a whole step. The paper's strategy is to cluster tokens at every full-calculation timestep, compute one token per cluster during the skipped steps, and blend the fresh representative values into the cached values of all other tokens. This adds a spatial reuse channel on top of existing temporal caching, and the empirical claim is that it both speeds up inference and slows the error accumulation that limits long caching cycles. Concretely, the paper reports 4.96x speedup on FLUX with an ImageReward of 99.49%, slightly above the uncached model, and up to 6.21x on HunyuanVideo with a VBench score near the baseline. If this holds, training-free inference acceleration for image and video diffusion transformers can come from exploiting spatial structure, not only from skipping steps or pruning tokens.","feed_headline":"Spatial clustering makes diffusion transformers 4-6x faster","feed_subtitle":"Cluster-driven caching keeps ImageReward above the uncached FLUX model while cutting compute.","key_machinery":"The machinery is a cluster-conditional cache update. K-Means groups image or video tokens at the last layer of a full-calculation step; at partial steps one token per cluster is recomputed; and the cluster's freshly computed mean $\\mu(i)$ is interpolated with each non-computed token's cached value using propagation ratio $\\gamma$. The load-bearing object is the cluster assignment vector, whose temporal stability (ARI above 0.8) lets a single representative stand for many tokens, and whose spatial tightness (intra-cluster distance roughly 100x smaller than the global token distance) makes the interpolation accurate.","core_discovery":"The central claim is that, during the denoising trace, tokens fall into clusters that keep their shape across adjacent timesteps (adjusted Rand index above 0.8), and tokens within a cluster not only lie close in feature space but move along similar trajectories. ClusCa exploits this by dividing timesteps into cache cycles: at the first step of a cycle every token is computed and clustered with K-Means; over the next N-1 steps only K tokens, one per cluster, are recomputed. The refreshed representative features are spread to co-clustered tokens through a weighted update, $C(x_i)=\\gamma \\mu(i)+(1-\\gamma)C(x_i)$, where $\\mu(i)$ is the mean of freshly computed representatives in the cluster and $\\gamma$ is a propagation ratio. The paper argues that this spatial reuse, combined with temporal reuse from the cache, keeps all tokens updated without computing all of them, and that it is this propagation rather than token importance that prevents quality collapse at high speedup.","pith_inferences":["If cluster coherence is what matters, then the random choice of one token per cluster implies that token importance scores are unnecessary; cheaper alternatives to K-Means, such as spatial tiling or hashing, might deliver similar gains.","The reported ImageReward gain over the uncached FLUX model may be metric-specific; a direct human-preference study would separate genuine aesthetic improvement from reward-model alignment.","The method's premise is most vulnerable in video scenes with fast motion, occlusion, or rapid lighting change, so a natural extension is to make reclustering frequency or propagation ratio depend on estimated motion or intra-cluster variance rather than a fixed cycle length.","Because ClusCa refreshes representatives from the network while forecasting methods predict the drift of cached values, the two ideas are complementary and could be combined to extend cache-cycle length further."],"forward_implications":["Feature caching methods that previously treated spatial tokens as independent can now be layered with cluster-driven propagation, so the gains should stack with temporal-cache, step-reduction, and attention-compression approaches.","At high speedup ratios, ClusCa reports stable FID on ImageNet with DiT where FORA, ToCa, and DuCa degrade, suggesting that spatial propagation specifically suppresses the error accumulation that limits long cache intervals.","On video, the method reports better VBench fidelity sub-metrics (PSNR, SSIM, LPIPS) than the compared cache baselines at 5.54x, indicating that cluster reuse can preserve temporal consistency rather than destroy it.","Because the method is training-free and inference-only, it applies to already-deployed diffusion transformers without retraining or weight modification."],"supporting_citations":[{"why":"Defines the Diffusion Transformer architecture that ClusCa accelerates.","marker":"[25]"},{"why":"Introduces temporal feature caching for diffusion models, the baseline reuse idea ClusCa extends with spatial reuse.","marker":"[24]"},{"why":"Provides the token-wise feature caching baseline and the important-token update scheme that ClusCa contrasts with cluster-wise updates.","marker":"[41]"},{"why":"Baseline dual feature caching method compared across DiT, FLUX, and HunyuanVideo.","marker":"[42]"},{"why":"FORA fast-forward caching baseline that ClusCa must beat at equal acceleration ratios.","marker":"[29]"},{"why":"TaylorSeer forecast-based caching method whose forecast mechanism ClusCa builds on and compares against.","marker":"[19]"},{"why":"FLUX.1-dev model used for the main text-to-image speedup and quality comparison.","marker":"[15]"},{"why":"HunyuanVideo model used for text-to-video evaluation.","marker":"[33]"}],"fun_headline_variants":["Cluster caching cuts diffusion tokens by 90% for 5x speed","One token per cluster: 4.96x faster diffusion with no quality drop","ClusCa: 90% fewer tokens, 4.96x speedup in diffusion transformers","Compute one token per cluster: diffusion transformers 5x faster"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes that clusters formed at a full-calculation step stay valid for the next several skipped steps, so one randomly chosen token from each cluster can stand for all tokens in that cluster, without any stated bound on when motion or fine detail breaks that coherence.","fun_headline_variants_meta":{"raw":{"variants":["Cluster caching cuts diffusion tokens by 90% for 5x speed","One token per cluster: 4.96x faster diffusion with no quality drop","ClusCa: 90% fewer tokens, 4.96x speedup in diffusion transformers","Compute one token per cluster: diffusion transformers 5x faster"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00022,"raw_usage":{"total_tokens":1469,"prompt_tokens":987,"completion_tokens":482,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":603,"completion_tokens_details":{"reasoning_tokens":397}},"tokens_in":603,"tokens_out":482,"duration_ms":4042,"temperature":1.0,"reasoning_tokens":397,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:54:52.593657+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure intra-cluster feature distance at each skipped timestep inside a cache cycle on prompts with small fast-moving objects or rapid lighting changes; if the distance grows past the point where the weighted cache update diverges from a full recomputation, the affected regions should show ghosting or detail loss and the speedup-quality trade-off should collapse.","supporting_citations":[],"review_version":1}