Pith. sign in

REVIEW 3 major objections 7 minor 65 references

Massive activations in Diffusion Transformers are the detail knobs: suppress them for guidance, reweight them for better dense features—no retraining.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-12 05:45 UTC pith:J6XC7PV2

load-bearing objection Solid empirical DiT systems paper: MA structure is real and dual-use EMA works across models, but novelty is partly incremental on the authors’ own DiTF/DG line and the “clean detail residual” claim is only partially secured. the 3 major comments →

arxiv 2607.02968 v1 pith:J6XC7PV2 submitted 2026-07-03 cs.CV

Awakening Diffusion Transformers: Eliciting Stronger Generation and Understanding via Massive Activation Modulation

Chaofan Gan , Zicheng Zhao , Yuanpeng Tu , Xi Chen , Ziran Qin , Tieyuan Chen , Supavadee Aramvith , Mehrtash Harandi
show 1 more author
Weiyao Lin
This is my paper
classification cs.CV
keywords Diffusion Transformersmassive activationsAdaLNdetail guidanceclassifier-free guidancedense visual representationtraining-free modulationtext-to-image generation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Diffusion Transformers hide a structured internal signal called massive activations: a few fixed feature channels that fire very strongly across every spatial token and grow mainly with the denoising timestep, not with the text prompt. The paper shows these channels are tightly tied to AdaLN residual scales and do two different jobs. In generation they carry fine local detail (textures, small parts) while leaving global semantics largely intact; in representation they force many tokens to share the same high-magnitude direction and blur dense discrimination. EMA is a training-free way to use that signal on purpose: Detail Guidance builds a detail-starved counterfactual by suppressing those channels and steers sampling toward richer detail (including local regions and efficient partial forwards), while MREP reuses AdaLN modulation and normalized MA maps to make DiT features more discriminative for correspondence, segmentation, and depth. If the dual role holds, the same internal pattern becomes a unified handle for both better images/videos and better dense understanding.

Core claim

In representative DiTs, massive activations are spatially distributed across image tokens but concentrated in a small set of fixed dimensions that align with AdaLN residual scaling factors and are primarily modulated by denoising timestep rather than text. That structure makes them critical for fine-grained detail synthesis with limited effect on global semantics, while the same shared high-magnitude directions make raw DiT features overly similar across tokens and weaken dense discrimination. Task-aware modulation of these activations—suppression for detail guidance in sampling, AdaLN reweighting plus spatial MA maps for representation—elicits stronger generation and understanding without t

What carries the argument

Eliciting Massive Activation (EMA): MA-driven Detail Guidance (DG) suppresses calibrated MA dimensions at an intermediate block to form a detail-deficient counterfactual residual that guides sampling (with partial-forward, CFG, and token-level Local DG variants); MA-modulated Representation Extraction (MREP) applies pretrained AdaLN channel-wise modulation to reduce MA directional dominance and concatenates spatially normalized MA maps to keep useful spatial structure.

Load-bearing premise

That a one-time median-ratio pick of a few channels on a small calibration set, suppressed with a fixed sigmoid schedule at a grid-searched intermediate block, cleanly isolates detail rather than mixing in semantics or model-specific artifacts.

What would settle it

On a fixed DiT and prompt set, intervene only the calibrated MA channels versus the same count of non-MA channels at the same block and schedule: if detail metrics (aesthetic/HPS) and human preference for local texture collapse for both interventions equally, or if prompt-alignment metrics collapse when MAs are suppressed, the claimed detail-specific residual is false.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Suppressing MA dimensions at intermediate blocks yields a detail-specific guidance direction that improves fine detail and can be combined with classifier-free guidance.
  • Sharing the prefix forward before the intervention block makes detail guidance cheaper than a full second denoising pass.
  • Token-local MA suppression enables selective refinement of chosen regions without rewriting the rest of the image.
  • AdaLN-modulated features plus normalized MA spatial maps raise dense-task scores (correspondence, segmentation, depth) over raw DiT hidden states.
  • The same MA structure appears across image and video DiTs, so the modulation recipe transfers to spatiotemporal generation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If MAs are mainly timestep-driven residual scales, other AdaLN-conditioned denoisers (beyond pure DiTs) may admit similar detail and representation knobs with only calibration, not architecture changes.
  • A natural next test is whether the same MA residual can be inverted—amplifying rather than suppressing selected dimensions—to trade detail for global coherence or style.
  • Because MA channels are few and fixed, they are natural candidates for low-bit quantization or caching schedules that preserve detail while cutting memory.
  • Local DG suggests interactive editors could treat MA suppression masks as soft “detail brushes” driven by user regions rather than full re-sampling.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper analyzes Massive Activations (MAs) in Diffusion Transformers and reports a DiT-specific structure: MAs are spatially distributed across tokens but concentrated in a few fixed channels that align with AdaLN residual scales and are primarily timestep-modulated rather than text-modulated. From this, the authors argue a dual role—MAs support fine-grained detail synthesis in generation, while their shared high-magnitude directions reduce spatial discriminability of raw DiT features for dense understanding. They propose EMA, a training-free framework with two modules: Detail Guidance (DG), which suppresses MA channels to form a detail-deficient counterfactual and uses the residual as a guidance signal (with partial-forward, CFG combination, and Local DG variants); and MREP, which reuses AdaLN modulation to reduce MA directional dominance and concatenates spatially normalized MA maps. Experiments on multiple image DiTs, video DiTs, ImageNet class-conditional generation, and dense tasks (correspondence, segmentation, depth) report consistent gains.

Significance. If the analysis and methods hold, the work provides a useful internal-activation account of DiTs that unifies generation-side detail control and representation-side feature cleanup under one training-free modulation principle. Strengths include multi-architecture generation results (SD3/SD3.5/Flux, PixArt-α, SANA, DiT-XL/2, Wan), explicit MA vs non-MA intervention evidence, CFG complementarity, partial-forward cost accounting, Local DG controllability, and MREP ablations on dense tasks. These make MA modulation a practical, reusable lever rather than only a diagnostic observation, with clear value for sampling guidance and DiT feature extraction without retraining.

major comments (3)
  1. §V-A, Eqs. (7)–(11) and §IV-C: The central generative claim treats the residual D_θ(z_t,c)−D_θ(ẑ_t,c) as a largely detail-specific direction. Figs. 5–6 and the near-0.5 CLIP/BLIP win rates after MA disruption support limited semantic damage, and Table XI shows MA channels beat random channels. Still missing is a direct residual-disentanglement check (e.g., residual cosine similarity to CFG residuals; layout/pose/identity probes; or controlled semantic edits under fixed seeds). Because MAs are AdaLN-aligned and timestep-modulated (Fig. 4), a single intermediate block k with a fixed sigmoid schedule can couple the residual to stage/scale artifacts. Please add at least one quantitative disentanglement analysis, or temper the ‘detail-specific’ language to match the current aggregate evidence.
  2. §V-A (MA selection and schedule) and free parameters τ_MA, k, ρ_t: MA dimensions are fixed from a small calibration set via a median-ratio threshold, then suppressed with defaults τ_ρ=0.5, λ_ρ=12 and grid-searched k (6/20/22). Table XI ablations help, but the manuscript does not report sensitivity of selected channels across calibration sets/prompts, stability of I^k_MA across models/seeds, or performance under modest τ_MA / schedule perturbations. Given that EMA’s claim is that calibrated MAs are stable shared structure, a short sensitivity table (channel overlap; metric change under τ_MA and k±) is needed to secure the ‘once-calibrated, reused for all prompts’ assumption.
  3. §VI-D / Tables VI–VIII: MREP gains are clear versus ‘w/o MREP’, but several dense-task comparisons mix unsupervised DiT features with supervised or differently trained baselines (e.g., SCorrSAN, CATs++, VPD variants). Please clarify protocol parity (frozen features vs fine-tuning, probe heads, timesteps/blocks, resolution) and, where possible, report the same probe setup for DiTF, w/o MREP, and MREP so the contribution of MA modulation is isolated from evaluation-protocol differences.
minor comments (7)
  1. Abstract: typo ‘Detail G}uidance’ (stray brace).
  2. Fig. 2 / §IV-A: state explicitly how many MA dimensions are selected per model and whether the same absolute channel indices recur across checkpoints or only relative magnitude ranks.
  3. Eq. (10) and Algorithm 1: define the normalized timestep convention (t∈[0,1] vs discrete sampler indices) so the ρ_t schedule is reproducible across SD3/Flux/Wan samplers.
  4. Table II vs Table III: HPSv2.1 numbers appear under different evaluation settings; add a short note on prompt sets and sampling steps to avoid cross-table over-interpretation.
  5. §II-D / related work: DiTF and the prior DG arXiv are closely related; a clearer novelty paragraph separating analysis contributions from method reuse would help readers.
  6. Fig. 13: axis labels and which model/setting each panel uses could be made more self-contained in the caption.
  7. Local DG (§V-C, Fig. 12): quantitative region-level metrics (or user preference on local edits) would strengthen the qualitative demos.

Circularity Check

1 steps flagged

No significant circularity: MA structure is measured, DG/MREP are designed methods evaluated on external metrics; mild self-citation of authors' DiTF/DG priors is not load-bearing.

specific steps
  1. self citation load bearing [Sec. II-D Related Work; citations [36] DiTF and [45] prior DG arXiv]
    "Notably, DiTF [36] observes that DiTs exhibit fixed-dimensional massive activations across spatial tokens when used as feature extractors, while DG [45] further suggests that such activations are closely related to local detail synthesis in DiT-based generation."

    The dual-role framing (MAs for detail synthesis and for representation) is partly prefigured in the authors' own DiTF and DG preprints with overlapping author lists. This is mild narrative self-citation, not a load-bearing uniqueness or sole justification: Section IV re-measures MA structure and intervention effects in this manuscript, and EMA is evaluated on external generation/understanding benchmarks. Does not force the reported residual or MREP gains by construction.

full rationale

The paper's chain is observational then constructive, not definitional. Section IV measures MA spatial/channel structure, AdaLN alignment, and timestep vs text modulation on SD3/Flux/DiT-XL (Figs. 2–4); generative impact is shown by one-layer intervention with external CLIP/BLIP and detail metrics (Figs. 5–6), not by defining detail as the residual. DG (Eqs. 7–11) builds a counterfactual by suppressing calibrated MA channels and uses the residual as guidance—standard self-guidance form—then reports gains on Pick-a-Pic, HPSv2.1, Aesthetic, VBench, ImageNet FID/IS against CFG and other guidance baselines. MREP reuses pretrained AdaLN and normalized MA maps and is scored on SPair/AP-10K/PF-Pascal, ADE20K, NYUv2. Hyperparameter choices (τ_MA, ρ schedule, k, w) are ablated (Tables XI–XII, Fig. 13) rather than fitted then re-predicted as the same quantity. Self-citations to DiTF [36] and the authors' prior DG note [45] frame related work and the dual-use story, but the present manuscript re-presents analyses and external-benchmark evaluations; the central claims do not reduce to those citations alone, nor to a uniqueness theorem or renamed known law. Score 1 reflects only that mild, non-load-bearing self-citation backdrop.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 3 invented entities

The central claim rests on empirical activation statistics in pretrained DiTs, the self-guidance construction pattern, and several hand-chosen intervention hyperparameters. No new physical entities are postulated; the free parameters are calibration and guidance knobs that materially affect reported gains.

free parameters (4)
  • MA ratio threshold τ_MA
    Defines which channels count as massive via a_k,i > τ_MA * median; fixed after calibration and reused for all prompts.
  • Intervention block index k
    Chosen by grid search per model (defaults k=6 SD3, k=20 SD3.5, k=22 Flux); controls where MA suppression is applied.
  • Retention schedule (τ_ρ=0.5, λ_ρ=12)
    Sigmoid decay of MA retention along normalized timestep; hand-set defaults that shape the detail-deficient branch.
  • Detail guidance scale w (and CFG λ when combined)
    Linear extrapolation strength for DG residual; selected for reported aesthetic/detail metrics.
axioms (4)
  • domain assumption Self-guidance by contrasting the model with a deliberately degraded counterpart yields a useful sampling direction.
    Imported from Autoguidance-style methods (Sec. V-A, Table I) and used to justify DG residual construction.
  • domain assumption AdaLN residual scales α_k are the primary channel-wise controllers of MA locations and magnitudes.
    Supported by correlation plots (Fig. 4) and then reused as the MREP modulator (Eqs. 15–17).
  • standard math Standard latent diffusion sampling and CFG formulation hold for the evaluated DiTs.
    Preliminaries Sec. III; EMA is applied on top of existing samplers without changing training.
  • ad hoc to paper A small calibration set’s mean absolute activations suffice to identify stable MA dimensions for all later prompts.
    Sec. V-A MA dimension selection; not proven to be unique or optimal across domains.
invented entities (3)
  • EMA (Eliciting Massive Activation) framework no independent evidence
    purpose: Unify MA modulation for both generation (DG) and representation (MREP).
    Training-free wrapper around MA interventions; independent evidence is empirical tables only.
  • Detail Guidance (DG) / Local DG no independent evidence
    purpose: Construct detail-deficient counterfactual by MA suppression and residual-guide sampling, optionally token-local.
    Method construct; overlaps authors’ prior DG arXiv [45].
  • MREP (MA-modulated REPresentation extraction) no independent evidence
    purpose: AdaLN-modulate hidden states and concatenate normalized MA maps for dense features.
    Method construct closely related to DiTF [36] MA modulation for correspondence.

pith-pipeline@v1.1.0-grok45 · 28522 in / 3198 out tokens · 31947 ms · 2026-07-12T05:45:30.028923+00:00 · methodology

0 comments
read the original abstract

Massive Activations (MAs) have been widely observed in Transformer-based models, yet their structure and functional roles in Diffusion Transformers (DiTs) remain insufficiently understood. In this work, we systematically analyze MAs in representative DiTs and find that they are spatially distributed across image tokens while concentrated in a small set of fixed feature dimensions. We further show that these dimensions are closely aligned with AdaLN residual scaling factors and are primarily modulated by the denoising timestep rather than text conditions. This structure leads to two task-dependent effects: for generation, MAs are critical for fine-grained detail synthesis while having limited influence on global semantics; for understanding, their shared high-magnitude directions make raw DiT features overly similar across spatial tokens and weaken dense feature discrimination. Based on these findings, we introduce Eliciting Massive Activation (EMA), a training-free framework that leverages Massive Activations (MAs) as a unified modulation signal to improve both generative and representational capabilities of DiTs. For generation, EMA proposes MA-driven Detail G}uidance (DG), which suppresses MA dimensions to construct a detail-deficient counterfactual prediction and guides sampling toward finer visual details. DG further supports efficient partial-forward inference, integration with classifier-free guidance, and token-level Local DG for refining selected image regions. For understanding, EMA introduces MA-modulated REPresentation extraction (MREP), which uses pretrained AdaLN channel-wise modulation to reduce MA directional dominance and concatenates spatially normalized MA maps to preserve useful spatial structure. Extensive experiments demonstrate that EMA consistently improves both the generation quality and representation capability of DiTs.

Figures

Figures reproduced from arXiv: 2607.02968 by Chaofan Gan, Mehrtash Harandi, Supavadee Aramvith, Tieyuan Chen, Weiyao Lin, Xi Chen, Yuanpeng Tu, Zicheng Zhao, Ziran Qin.

Figure 1
Figure 1. Figure 1: EMA improves DiTs across visual generation and understanding tasks. For generation, EMA improves fine-grained detail synthesis and enables controllable local refinement. For understanding, EMA produces more discriminative DiT representations, benefiting dense perception tasks such as correspondence and segmentation. vention shows that disrupting MA dimensions substantially weakens fine-grained local detail… view at source ↗
Figure 2
Figure 2. Figure 2: Massive Activations in DiTs. The activation magnitudes of internal hidden states. We present the average magnitudes over 1,000 text prompts. Massive activations are consistently concentrated in a few fixed dimensions across all patch tokens. that they are spatially distributed, channel-concentrated, AdaLN-related, and primarily timestep-modulated activa￾tion components. • We reveal the dual role of MAs: th… view at source ↗
Figure 3
Figure 3. Figure 3: Properties of massive activations in DiT-XL. (a) Activation distribution of the hidden states along DiT layers (b) Activation distribution of the hidden states along training iterations (c) Activation distribution of the hidden states across different model sizes. Massive activations occur throughout all layers and persist across different model sizes. dimension Activation value Scaling value Activation va… view at source ↗
Figure 4
Figure 4. Figure 4: Impact of the input timestep and text on Massive Activations (MAs). (a) Comparison of the distributions of hidden-state z k t activations and their corresponding residual scaling factor αk. (b) Respective impact of input timestep and text embeddings on the magnitude distribution of MAs, where we compare the MAs of 1000 different text inputs. The massive activations are governed by the residual scaling fact… view at source ↗
Figure 5
Figure 5. Figure 5: Visual comparison of MA intervention. Disrupting massive activations (MAs) leads to a pronounced degradation in fine-grained image details, whereas perturbing non-MA dimensions has negligible impact on the generated outputs. This contrast indicates that MAs play a critical role in the synthesis of visual details during the generative process. Win probability comparison SD3 Original MAs Disrupted SD3-5 [PI… view at source ↗
Figure 6
Figure 6. Figure 6: Win probability comparison of MA intervention. Win probability comparison for different models where we evaluate the model from two perspectives: Prompt Alignment (Blipscore and Clipscore) and Local Detail Quality (HPSv2.1 and Laion-Aesthetics). Disrupting massive activations markedly degrades the fidelity of fine-grained details in the generated images while exerting minimal impact on semantic content. on… view at source ↗
Figure 7
Figure 7. Figure 7: Visual comparison with and without MREP. Raw DiT features w/o MREP exhibit poor semantic coherence and structural quality. Our MREP module effectively suppresses MA-dominated channels, yielding more semantically coherent and spatially discriminative representations. TABLE I UNIFIED FORMULATIONS OF DIFFERENT GUIDANCE METHODS. cˆ DENOTES THE UNCONDITIONAL PROMPT, θ ∗ DENOTES UNDER-CAPABILITY MODEL, AND zˆt D… view at source ↗
Figure 8
Figure 8. Figure 8: Overall framework of EMA. EMA leverages Massive Activations (MAs) as task-aware internal modulation signals in pretrained Diffusion Transformers (DiTs). For visual generation, EMA introduces Detail Guidance (DG), which constructs a detail-deficient counterfactual branch by suppressing MA dimensions and exploits the resulting residual signal to enhance fine-grained visual details. For visual understanding, … view at source ↗
Figure 9
Figure 9. Figure 9: Visualization of MA spatial maps. MAs appear in fixed feature dimensions but produce structured responses across spatial tokens. This suggests that MA dimensions still encode useful spatial cues. pretrained AdaLN channel-wise weights from the same block. Following Equations (2) and (3), the AdaLN parameters are γk, βk, αk = MLPk(t, c), (15) where γk, βk, αk ∈ R C are channel-wise parameters. Since the MA d… view at source ↗
Figure 10
Figure 10. Figure 10: Visual results of Detail Guidance (DG) on SD3. Baseline denotes visual outputs without CFG. DG produces high-quality images with improved fine-grained details compared to Baseline. The CFG output is included as a reference for better comparison of detail quality. and report mean intersection over union (mIoU). For depth estimation, we evaluate on NYUv2 [61] and report standard metrics, including absolute … view at source ↗
Figure 11
Figure 11. Figure 11: Integration with CFG. Rows 1-2: SD3; Rows 3-4: SD3.5. Incorporating DG into Classifier-Free Guidance (CFG) yields outputs with markedly richer and more refined local details. C. Main Results on Video Generation We further evaluate DG on Wan video models [9]. Com￾pared with image generation, video generation poses a more challenging setting, as enhancing local details must also preserve temporal consistenc… view at source ↗
Figure 12
Figure 12. Figure 12: Visual results of Local DG. Selectively applying DG to MAs in target spatial tokens enhances the details of the selected region while keeping other regions largely unchanged. TABLE IX COMPUTATIONAL OVERHEAD OF EFFICIENT PARTIAL-FORWARD DG. LATENCY IS MEASURED FOR 1024X1024 GENERATION ON ONE NVIDIA L40S GPU. Model Type Memory (GB) Latency (s) Aesthetic SD3 Cond 17 2.3 5.58 CFG 20 4.3 5.80 DG 20 3.5 6.01 SD… view at source ↗
Figure 13
Figure 13. Figure 13: Investigations of intervention depth m, scales λ and w for SD3. We analyze how the DG intervention block, CFG scale λ, and DG scale w affect generation quality. The results show that DG benefits from intervening at appropriate intermediate blocks and provides a flexible trade-off between semantic alignment and fine-grained detail enhancement. TABLE X EVALUATION OF DG ON NON-STANDARD DITS. OUR DG STRATEGY … view at source ↗
Figure 14
Figure 14. Figure 14: User study on SD3, SD3.5, and Flux1. We report win rates comparing CFG with CFG+DG from prompt alignment, detail preservation, and color consistency. layer-wise MA distributions. 4) Guidance scales.: We present the quantitative results across different scales in [PITH_FULL_IMAGE:figures/full_fig_p015_14.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

65 extracted references · 22 linked inside Pith

  1. [1]

    High- resolution image synthesis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High- resolution image synthesis with latent diffusion models,” inCVPR, 2022

    2022

  2. [2]

    Photorealistic text-to-image diffusion models with deep language understanding,

    C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K. Ghasemipour, R. Gontijo Lopes, B. Karagol Ayan, T. Salimans et al., “Photorealistic text-to-image diffusion models with deep language understanding,”NeurIPS, 2022

    2022

  3. [3]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,”Advances in neural information processing systems, vol. 30, 2017

    2017

  4. [4]

    Scalable diffusion models with transformers,

    W. Peebles and S. Xie, “Scalable diffusion models with transformers,” inICCV, 2023

    2023

  5. [5]

    Scaling rectified flow transformers for high-resolution image synthesis,

    P. Esser, S. Kulal, A. Blattmann, R. Entezari, J. M ¨uller, H. Saini, Y . Levi, D. Lorenz, A. Sauer, F. Boeselet al., “Scaling rectified flow transformers for high-resolution image synthesis,” inICML, 2024

    2024

  6. [6]

    black-forest labs, “Flux,” https://blackforestlabs.ai/ announcing-black-forest-labs/, 2024

    2024

  7. [7]

    Cogvideox: Text-to-video diffusion models with an expert transformer,

    Z. Yang, J. Teng, W. Zheng, M. Ding, S. Huang, J. Xu, Y . Yang, W. Hong, X. Zhang, G. Fenget al., “Cogvideox: Text-to-video diffusion models with an expert transformer,”arXiv preprint arXiv:2408.06072, 2024

    Pith/arXiv arXiv 2024

  8. [8]

    Cogvideo: Large- scale pretraining for text-to-video generation via transformers,

    W. Hong, M. Ding, W. Zheng, X. Liu, and J. Tang, “Cogvideo: Large- scale pretraining for text-to-video generation via transformers,”arXiv preprint arXiv:2205.15868, 2022

    Pith/arXiv arXiv 2022

  9. [10]

    Unsupervised semantic correspondence using stable diffusion,

    E. Hedlin, G. Sharma, S. Mahajan, H. Isack, A. Kar, A. Tagliasacchi, and K. M. Yi, “Unsupervised semantic correspondence using stable diffusion,”NeurIPS, 2024

    2024

  10. [11]

    Emergent correspondence from image diffusion,

    L. Tang, M. Jia, Q. Wang, C. P. Phoo, and B. Hariharan, “Emergent correspondence from image diffusion,”Advances in Neural Information Processing Systems, vol. 36, pp. 1363–1389, 2023

    2023

  11. [12]

    A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence,

    J. Zhang, C. Herrmann, J. Hur, L. Polania Cabrera, V . Jampani, D. Sun, and M.-H. Yang, “A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence,”NeurIPS, 2024

    2024

  12. [13]

    Sd4match: Learning to prompt stable diffusion model for semantic matching,

    X. Li, J. Lu, K. Han, and V . A. Prisacariu, “Sd4match: Learning to prompt stable diffusion model for semantic matching,” inCVPR, 2024

    2024

  13. [14]

    Massive activations in large language models,

    M. Sun, X. Chen, J. Z. Kolter, and Z. Liu, “Massive activations in large language models,”arXiv preprint arXiv:2402.17762, 2024

    Pith/arXiv arXiv 2024

  14. [15]

    Efficient streaming language models with attention sinks,

    G. Xiao, Y . Tian, B. Chen, S. Han, and M. Lewis, “Efficient streaming language models with attention sinks,” inThe Twelfth International Conference on Learning Representations, 2024

    2024

  15. [16]

    Vision transformers need registers,

    T. Darcet, M. Oquab, J. Mairal, and P. Bojanowski, “Vision transformers need registers,” inThe Twelfth International Conference on Learning Representations, 2024

    2024

  16. [17]

    Classifier-free diffusion guidance,

    J. Ho and T. Salimans, “Classifier-free diffusion guidance,” arXiv:2207.12598, 2022

    Pith/arXiv arXiv 2022

  17. [18]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” NeurIPS, 2020

    2020

  18. [19]

    Diffusion models beat gans on image synthesis,

    P. Dhariwal and A. Nichol, “Diffusion models beat gans on image synthesis,”NeurIPS, 2021

    2021

  19. [20]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, “An image is worth 16x16 words: Transformers for image recognition at scale,”arXiv:2010.11929, 2020

    Pith/arXiv arXiv 2010

  20. [21]

    Pixart-alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis,

    J. Chen, J. Yu, C. Ge, L. Yao, E. Xie, Y . Wu, Z. Wang, J. Kwok, P. Luo, H. Luet al., “Pixart-alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis,”arXiv:2310.00426, 2023

    Pith/arXiv arXiv 2023

  21. [22]

    Eliminating oversaturation and artifacts of high guidance scales in diffusion models,

    S. Sadat, O. Hilliges, and R. M. Weber, “Eliminating oversaturation and artifacts of high guidance scales in diffusion models,” inThe Thirteenth International Conference on Learning Representations, 2024

    2024

  22. [23]

    Cfg++: Manifold- constrained classifier free guidance for diffusion models,

    H. Chung, J. Kim, G. Y . Park, H. Nam, and J. C. Ye, “Cfg++: Manifold- constrained classifier free guidance for diffusion models,”arXiv preprint arXiv:2406.08070, 2024

    Pith/arXiv arXiv 2024

  23. [24]

    Guiding a diffusion model with a bad version of itself,

    T. Karras, M. Aittala, T. Kynk ¨a¨anniemi, J. Lehtinen, T. Aila, and S. Laine, “Guiding a diffusion model with a bad version of itself,”Ad- vances in Neural Information Processing Systems, vol. 37, pp. 52 996– 53 021, 2024

    2024

  24. [25]

    Improving sample quality of diffusion models using self-attention guidance,

    S. Hong, G. Lee, W. Jang, and S. Kim, “Improving sample quality of diffusion models using self-attention guidance,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 7462–7471

    2023

  25. [26]

    Self-rectifying diffusion sampling with perturbed-attention guidance,

    D. Ahn, H. Cho, J. Min, W. Jang, J. Kim, S. Kim, H. H. Park, K. H. Jin, and S. Kim, “Self-rectifying diffusion sampling with perturbed-attention guidance,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 1–17

    2024

  26. [27]

    Spatiotemporal skip guidance for enhanced video diffusion sampling,

    J. Hyung, K. Kim, S. Hong, M.-J. Kim, and J. Choo, “Spatiotemporal skip guidance for enhanced video diffusion sampling,” inProceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 11 006–11 015

    2025

  27. [28]

    Segdiff: Image segmentation with diffusion probabilistic models,

    T. Amit, T. Shaharbany, E. Nachmani, and L. Wolf, “Segdiff: Image segmentation with diffusion probabilistic models,”arXiv:2112.00390, 2021

    Pith/arXiv arXiv 2021

  28. [29]

    Label-efficient semantic segmentation with diffusion models,

    D. Baranchuk, I. Rubachev, A. V oynov, V . Khrulkov, and A. Babenko, “Label-efficient semantic segmentation with diffusion models,” arXiv:2112.03126, 2021

    Pith/arXiv arXiv 2021

  29. [30]

    Open- vocabulary panoptic segmentation with text-to-image diffusion models,

    J. Xu, S. Liu, A. Vahdat, W. Byeon, X. Wang, and S. De Mello, “Open- vocabulary panoptic segmentation with text-to-image diffusion models,” inCVPR, 2023

    2023

  30. [31]

    Unleashing text- to-image diffusion models for visual perception,

    W. Zhao, Y . Rao, Z. Liu, B. Liu, J. Zhou, and J. Lu, “Unleashing text- to-image diffusion models for visual perception,” inICCV, 2023

    2023

  31. [32]

    Diffusion hyperfeatures: Searching through time and space for semantic corre- spondence,

    G. Luo, L. Dunlap, D. H. Park, A. Holynski, and T. Darrell, “Diffusion hyperfeatures: Searching through time and space for semantic corre- spondence,”NeurIPS, 2024

    2024

  32. [33]

    Deep vit features as dense visual descriptors,

    S. Amir, Y . Gandelsman, S. Bagon, and T. Dekel, “Deep vit features as dense visual descriptors,”arXiv:2112.05814, 2021

    Pith/arXiv arXiv 2021

  33. [34]

    Dinov2: Learning robust visual features without supervision,

    M. Oquab, T. Darcet, T. Moutakanni, H. V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Noubyet al., “Dinov2: Learning robust visual features without supervision,”arXiv:2304.07193, 2023

    Pith/arXiv arXiv 2023

  34. [35]

    Telling left from right: Identifying geometry-aware semantic correspondence,

    J. Zhang, C. Herrmann, J. Hur, E. Chen, V . Jampani, D. Sun, and M.- H. Yang, “Telling left from right: Identifying geometry-aware semantic correspondence,” inCVPR, 2024

    2024

  35. [36]

    Un- leashing diffusion transformers for visual correspondence by modulating massive activations,

    C. Gan, Y . Tu, X. Chen, T. Chen, Y . Li, M. Harandi, and W. Lin, “Un- leashing diffusion transformers for visual correspondence by modulating massive activations,” inThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025

    2025

  36. [37]

    Repurposing video diffusion transformers for robust point tracking,

    S. Son, H. An, C. Kim, H. Ko, J. Nam, D. Chung, S. Jin, J. Yi, J. Min, J. Huret al., “Repurposing video diffusion transformers for robust point tracking,”arXiv preprint arXiv:2512.20606, 2025

    Pith/arXiv arXiv 2025

  37. [38]

    Unveiling a core linguistic region in large language models,

    J. Zhao, Z. Zhang, Y . Ma, Q. Zhang, T. Gui, L. Gao, and X. Huang, “Unveiling a core linguistic region in large language models,”arXiv preprint arXiv:2310.14928, 2023

    Pith/arXiv arXiv 2023

  38. [39]

    SLMRec: Distilling large language models into small for sequential recommendation,

    W. Xu, Q. Wu, Z. Liang, J. Han, X. Ning, Y . Shi, W. Lin, and Y . Zhang, “SLMRec: Distilling large language models into small for sequential recommendation,” inThe Thirteenth International Conference on Learning Representations, 2025. [Online]. Available: https://openreview.net/forum?id=G4wARwjF8M

    2025

  39. [40]

    Massive values in self-attention modules are the key to contextual knowledge understanding,

    M. Jin, K. Mei, W. Xu, M. Sun, R. Tang, M. Du, Z. Liu, and Y . Zhang, “Massive values in self-attention modules are the key to contextual knowledge understanding,”arXiv preprint arXiv:2502.01563, 2025

    Pith/arXiv arXiv 2025

  40. [41]

    Denoising vision transformers,

    J. Yang, K. Z. Luo, J. Li, C. Deng, L. Guibas, D. Krishnan, K. Q. Weinberger, Y . Tian, and Y . Wang, “Denoising vision transformers,” in European Conference on Computer Vision. Springer, 2024, pp. 453– 469

    2024

  41. [42]

    Hq-dit: Efficient diffusion transformer with fp4 hybrid quantization,

    W. Liu and S. Q. Zhang, “Hq-dit: Efficient diffusion transformer with fp4 hybrid quantization,”arXiv preprint arXiv:2405.19751, 2024

    Pith/arXiv arXiv 2024

  42. [43]

    Vidit-q: Efficient and accurate quantization of diffusion transformers for image and video generation,

    T. Zhao, T. Fang, H. Huang, E. Liu, R. Wan, W. Soedarmadji, S. Li, Z. Lin, G. Dai, S. Yanet al., “Vidit-q: Efficient and accurate quantization of diffusion transformers for image and video generation,”arXiv preprint arXiv:2406.02540, 2024

    Pith/arXiv arXiv 2024

  43. [44]

    Tinyfusion: Diffusion transform- ers learned shallow,

    G. Fang, K. Li, X. Ma, and X. Wang, “Tinyfusion: Diffusion transform- ers learned shallow,” inProceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 18 144–18 154

    2025

  44. [45]

    Massive activations are the key to local detail synthesis in diffusion transformers,

    C. Gan, Z. Zhao, Y . Tu, X. Chen, Z. Qin, T. Chen, M. Harandi, and W. Lin, “Massive activations are the key to local detail synthesis in diffusion transformers,”arXiv preprint arXiv:2510.11538, 2025. 17

    arXiv 2025

  45. [46]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778

    2016

  46. [47]

    Visual reasoning with a general conditioning layer,

    E. Perez, F. Strub, H. De Vries, V . Dumoulin, and A. C. Film, “Visual reasoning with a general conditioning layer,”AAAI, 2018

    2018

  47. [48]

    Elucidating the design space of diffusion-based generative models,

    T. Karras, M. Aittala, T. Aila, and S. Laine, “Elucidating the design space of diffusion-based generative models,”Advances in neural infor- mation processing systems, vol. 35, pp. 26 565–26 577, 2022

    2022

  48. [49]

    Sana: Efficient high-resolution image synthesis with linear diffusion transformers,

    E. Xie, J. Chen, J. Chen, H. Cai, H. Tang, Y . Lin, Z. Zhang, M. Li, L. Zhu, Y . Luet al., “Sana: Efficient high-resolution image synthesis with linear diffusion transformers,”arXiv preprint arXiv:2410.10629, 2024

    Pith/arXiv arXiv 2024

  49. [50]

    Wan: Open and advanced large-scale video generative models,

    T. Wan, A. Wang, B. Ai, B. Wen, C. Mao, C.-W. Xie, D. Chen, F. Yu, H. Zhao, J. Yanget al., “Wan: Open and advanced large-scale video generative models,”arXiv preprint arXiv:2503.20314, 2025

    Pith/arXiv arXiv 2025

  50. [51]

    Pick-a-pic: An open dataset of user preferences for text-to-image generation,

    Y . Kirstain, A. Polyak, U. Singer, S. Matiana, J. Penna, and O. Levy, “Pick-a-pic: An open dataset of user preferences for text-to-image generation,”Advances in neural information processing systems, vol. 36, pp. 36 652–36 663, 2023

    2023

  51. [52]

    Human preference score v2: A solid benchmark for evaluating human pref- erences of text-to-image synthesis,

    X. Wu, Y . Hao, K. Sun, Y . Chen, F. Zhu, R. Zhao, and H. Li, “Human preference score v2: A solid benchmark for evaluating human pref- erences of text-to-image synthesis,”arXiv preprint arXiv:2306.09341, 2023

    Pith/arXiv arXiv 2023

  52. [53]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clarket al., “Learning transferable visual models from natural language supervision,” inInternational conference on machine learning. PmLR, 2021, pp. 8748–8763

    2021

  53. [54]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation,

    J. Li, D. Li, C. Xiong, and S. Hoi, “Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation,” inInternational conference on machine learning. PMLR, 2022, pp. 12 888–12 900

    2022

  54. [55]

    Laion-aesthetics,

    C. Schuhmann, “Laion-aesthetics,” https://laion.ai/blog/laion-aesthetics/, 2022, accessed: 2023-11-10

    2022

  55. [56]

    Vbench: Comprehensive benchmark suite for video generative models,

    Z. Huang, Y . He, J. Yu, F. Zhang, C. Si, Y . Jiang, Y . Zhang, T. Wu, Q. Jin, N. Chanpaisitet al., “Vbench: Comprehensive benchmark suite for video generative models,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 21 807–21 818

    2024

  56. [57]

    Spair-71k: A large-scale benchmark for semantic correspondence,

    J. Min, J. Lee, J. Ponce, and M. Cho, “Spair-71k: A large-scale benchmark for semantic correspondence,”arXiv:1908.10543, 2019

    Pith/arXiv arXiv 1908

  57. [58]

    Proposal flow,

    B. Ham, M. Cho, C. Schmid, and J. Ponce, “Proposal flow,” inCVPR, 2016

    2016

  58. [59]

    Ap-10k: A benchmark for animal pose estimation in the wild,

    H. Yu, Y . Xu, J. Zhang, W. Zhao, Z. Guan, and D. Tao, “Ap-10k: A benchmark for animal pose estimation in the wild,”arXiv:2108.12617, 2021

    Pith/arXiv arXiv 2021

  59. [60]

    Scene parsing through ade20k dataset,

    B. Zhou, H. Zhao, X. Puig, S. Fidler, A. Barriuso, and A. Torralba, “Scene parsing through ade20k dataset,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 633– 641

    2017

  60. [61]

    Indoor segmentation and support inference from rgbd images,

    N. Silberman, D. Hoiem, P. Kohli, and R. Fergus, “Indoor segmentation and support inference from rgbd images,” inEuropean conference on computer vision. Springer, 2012, pp. 746–760

    2012

  61. [62]

    Cfg-zero*: Improved classifier-free guidance for flow matching models,

    W. Fan, A. Y . Zheng, R. A. Yeh, and Z. Liu, “Cfg-zero*: Improved classifier-free guidance for flow matching models,”arXiv preprint arXiv:2503.18886, 2025

    Pith/arXiv arXiv 2025

  62. [63]

    S 2-guidance: Stochastic self guidance for training-free enhancement of diffusion models,

    C. Chen, J. Zhu, X. Feng, N. Huang, M. Wu, F. Mao, J. Wu, X. Chu, and X. Li, “S 2-guidance: Stochastic self guidance for training-free enhancement of diffusion models,”arXiv preprint arXiv:2508.12880, 2025

    arXiv 2025

  63. [64]

    Learning semantic correspondence with sparse annotations,

    S. Huang, L. Yang, B. He, S. Zhang, X. He, and A. Shrivastava, “Learning semantic correspondence with sparse annotations,” inECCV, 2022

    2022

  64. [65]

    Cats++: Boosting cost aggregation with convolutions and transformers,

    S. Cho, S. Hong, and S. Kim, “Cats++: Boosting cost aggregation with convolutions and transformers,”TPAMI, 2022. Chaofan Ganreceived the Bachelor’s degree in Software Engineering from Huazhong University of Science and Technology, Wuhan, China in 2022. He is currently pursuing the Ph.D. degree in Electronic Information and Electrical Engineering at Shangh...

    2022

  65. [66]

    degree with the School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University

    He is currently pursuing the Ph.D. degree with the School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University. His research interests include efficient generative modeling, visual autoregressive generation, and model compression. Tieyuan Chenreceived the Bachelor’s degree in the school of electronic information and electric...

    2023