Pith. sign in

REVIEW 4 major objections 5 minor 19 references

Representation autoencoder latent spaces, though anisotropic and curved, can support Drifting-based one-step distillation, reaching FID 1.77 on ImageNet 256 in 10k steps without an auxiliary MAE feature extractor.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 04:41 UTC pith:7RERPK2H

load-bearing objection Plausible one-step RAE distillation method, but the paper's own theorem contradicts its proof and the headline FID rests on a single unseeded run with test-set hyperparameter tuning. the 4 major comments →

arxiv 2606.15553 v2 pith:7RERPK2H submitted 2026-06-14 cs.LG cs.AI

Distilling Drifting Transformers with Representation Autoencoders

Jiawei Zhang , Mengfei Xia , Gen Li , Yuantao Gu This is my paper
classification cs.LG cs.AI
keywords representation autoencodersDrifting modelsone-step distillationflow matchinglatent space geometryImageNet 256FID evaluationgenerative modeling
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Drift-RAE claims that the geometry of representation autoencoder (RAE) latent spaces, which makes trajectory-based one-step distillation unstable, is exactly what Drifting-based distillation needs. The paper shows that RAE latents are semantically concentrated within each class and token, so the attraction term in the Drifting field survives in high dimension; reconstruction-based VAE latents are too dispersed, and the field degenerates. By applying the Drifting objective per token, with a single softmax, perturbed negative samples, and partially detached negatives, the method distills a pretrained DiTDH-XL flow model into a one-step generator in 10k training steps (about 16 epochs), reaching FID 1.77 on ImageNet 256 and beating prior RAE distillation without requiring the auxiliary MAE used by the original Drifting Model. A sympathetic reader would care because it suggests that one-step generation in representation spaces is a matter of choosing the right distribution-level objective rather than architectural headroom.

Core claim

On the paper's own terms, the central discovery is that Drifting—a distribution-level attraction–repulsion field between real and generated latents—can replace ODE-trajectory matching for distilling flow models in RAE latent spaces, and that the RAE latent geometry is concentrated enough to make the positive term of that field effective without an extra feature extractor. The paper proves (under a simplified high-dimensional model) that when positive samples are overly dispersed on a sphere, the positive component of the Drifting field vanishes while the negative component remains, collapsing optimization; it then gives per-class, per-token statistics showing RAE latents avoid this failure,

What carries the argument

The load-bearing object is the normalized Drifting field, computed on each patch token: for each generated token, a softmax-weighted pull toward nearby real latents plus a softmax-weighted push away from other generated tokens (Eq. 11). The paper shows this field is the gradient of the optimal discriminator logit between kernel-smoothed empirical distributions, which motivates three modifications: keeping only the sample-index softmax, adding Laplace-distributed noise to generated tokens before computing the field, and computing extra negative samples with detached gradients. The pretrained flow model serves as the one-step generator initialization, which the paper argues is what makes the D

Load-bearing premise

The load-bearing premise is that RAE latents are sufficiently concentrated within each class and token (NN-d 1.0664, S-MMD 0.07913) for the positive attraction term of the Drifting field to survive in high dimension; if that compactness fails, the objective in Eq. (10)–(11) degenerates as in Theorem 1 and an auxiliary feature extractor would be needed.

What would settle it

Train Drift-RAE on a reconstruction-based VAE latent space (e.g., SD-VAE) under the same 10k-step protocol: Theorem 1 predicts collapse or severe degradation, because the positive term vanishes for dispersed latents. If the method still works, the high-dimensional compactness story is wrong; if it collapses as predicted, the concentration statistics become the decisive evidence. Alternatively, measure per-token, per-class NN-d and S-MMD on a fresh RAE checkpoint; if the values move close to the SD-VAE row, Drift-RAE should fail there.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • One-step distillation in RAE latent spaces no longer requires an auxiliary MAE feature extractor, simplifying the distillation pipeline and reducing compute.
  • The same per-token Drifting objective can be applied to other RAE-based flow and DiT models, including larger or text-conditioned backbones, without retraining the encoder.
  • Drifting can succeed where trajectory-based distillation fails: the requirement is a sufficiently concentrated latent space, not straight teacher trajectories.
  • The 10k-step budget (roughly 16 epochs) means one-step RAE distillation is practical in a class-balanced setup, rather than requiring hundreds of epochs.
  • From-scratch Drifting in RAE spaces remains unsolved; the paper's decode-encode attempt reaches only FID 7.04, so the claim is specifically about distillation from a pretrained teacher.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because the Drifting field is a Monte Carlo estimate of a score difference, the temperature τ likely controls a bias–variance trade-off; annealing τ over training, an experiment the paper does not run, could further stabilize the regime with only 64 gradient-carrying negatives.
  • Editorial inference: the paper's concentration claim is protocol-sensitive—computed per class and per token; applying Drift-RAE to a different RAE checkpoint would likely require re-running those NN-d and S-MMD statistics before dropping the auxiliary encoder.
  • Editorial inference: the decode-encode result for from-scratch training suggests the RAE decoder acts as an off-manifold projection; a testable extension is to inject a similar off-manifold projection into the distillation objective when latents are less compact.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces Drift-RAE, a method for one-step distillation of flow models trained in Representation Autoencoder (RAE) latent spaces. It argues that RAE latents are semantically concentrated but geometrically anisotropic, making trajectory-based distillation unstable; it proposes using Drifting Models, which operate on distribution discrepancy rather than trajectories. The authors provide statistical and theoretical analyses, claim a theorem showing the drifting field degenerates in overly dispersed spaces, and introduce modifications (single softmax, input perturbation, detached negative samples) motivated by a Diffusion-GAN connection. Experiments on ImageNet 256 report FID 1.77 with DiTDH-XL after 10k steps, outperforming prior RAE distillation and matching the original Drifting Model without an auxiliary MAE. The paper also reports geometry statistics and ablations.

Significance. If the reported results are reproducible, the contribution is substantial: it would establish that one-step distillation in RAE spaces can reach quality comparable to the original Drifting Model while removing the auxiliary MAE, and it provides a geometric explanation for why Drifting works in RAE spaces. The connection between Drifting and Diffusion-GAN (Proposition 1) is a useful theoretical bridge. The specific contributions — the anisotropy/concentration analysis, the modifications to the Drifting objective, and the training-efficiency claim (10k steps vs. hundreds of epochs) — are of interest to the generative modeling community. However, the current paper's central quantitative claim (1.77 FID) rests on a single run and on a theorem that is internally inconsistent as stated; the evidence is therefore not yet conclusive.

major comments (4)
  1. [§3.2 / Appendix A.1, Theorem 1] Theorem 1 claims (2) V_j → 0. The proof in Appendix A.1 instead derives Eq. (30): ||V_j|| → sqrt(1/3) r as d→∞. The positive component V_j^+ does vanish, but the total field does not; it converges to a non-zero repulsive vector. The subsequent 'collapse to the origin' inference from comparing sqrt(1/3)r with the cube diagonal r√d is not equivalent to V_j → 0. This contradicts the theorem statement. Since the theorem is used in §3.2 to justify removing the MAE feature extractor, the theoretical claim needs to be corrected (either restate the theorem as 'V_j^+ → 0 and V_j is dominated by repulsion' or revise the proof). As written, the theorem is false.
  2. [Abstract vs. §4.2 Table 4] The abstract reports 1.48 FID with 16-epoch distillation, but the body reports 1.77 FID for DiTDH-XL (Table 4) and 2.12 for DiTDH-L. No configuration or table in the paper attains 1.48. This discrepancy, unexplained, suggests either a different run, a different evaluation protocol, or instability. The reader cannot determine which number is the actual reported result. Please reconcile the abstract and the main text, and state the evaluation protocol precisely.
  3. [§4.3 Tables 3, 6, 7] The central SOTA claim relies on FID comparisons among configurations selected by ablating Npos/Nneg/Nextra_neg on the target benchmark itself. Table 6 shows that varying Nextra_neg from 192 to 0 changes FID from 1.77 to 1.79, a difference of 0.02, while the margin over the best prior RAE distillation (MF-RAE, 2.03) is 0.26. With single unseeded runs and no confidence intervals, it is impossible to assess whether the reported margins are due to configuration choice or run-to-run noise. Please provide at least 3 seeds for the main configurations, report mean±std, or use a held-out validation split for hyperparameter selection.
  4. [§3.2 Table 2 and Appendix B.1] The removal of the MAE feature extractor is justified by the claim that RAE latents are 'sufficiently concentrated,' based on NN-d and S-MMD computed with a per-class, per-token protocol. The manuscript itself notes that a concurrent work (Zhang et al., 2026) reports different geometry under a different aggregation protocol. Since the entire method (Eq. (10)–(11)) depends on this compactness premise, the protocol's appropriateness is load-bearing. Please provide additional evidence directly tied to the Drifting objective, e.g., measure the norm of the positive attraction term on real RAE latents versus SD-VAE latents under the same sampling scheme, or run an ablation that includes an MAE feature extractor to verify that the performance gap is negligible.
minor comments (5)
  1. [§4.1] The batch size is not stated; 'Nclass=32' and 10k steps correspond to 'roughly 16 epochs' only if the per-step batch size is known. Please specify.
  2. [Eq. (10)] z_j ∼ N(0,I) is written with a space; Nneg is defined but the total number of generated samples is not clearly linked to the batch structure.
  3. [§3.3] 'thereotically' is a typo; should be 'theoretically'.
  4. [Appendix A.1, after Eq. (30)] The argument about the cube diagonal is confusing; clarify that the field vector is small relative to the scale of the space, not that the field vanishes.
  5. [Table 4] FDDINOv2 values for Drift-RAE are reported (46.11 for XL), but the main text says 'improved FDDINOv2' compared to Drifting Model-L/2 (146.88); this is a large improvement and would benefit from a qualitative explanation or a sanity check.

Circularity Check

0 steps flagged

No circular derivation found; the Drift-RAE claim rests on an externally defined Drifting objective, frozen pretrained checkpoints, and empirical FID evaluation, not on a fitted constant or a self-citation chain.

full rationale

The paper's central derivation chain is not circular. The one-step generator is defined as G_theta(z) = z - v_theta(z,1) (Eq. 8) and initialized from a pretrained flow model; the Drift-RAE loss (Eqs. 10-11) is a stop-gradient drifting objective computed from real RAE latents and generated samples. No reported FID is the result of solving for a parameter that was itself defined from that FID, and no quantity labeled a prediction reduces by construction to its fitted input. The Drifting field is adopted from an external reference (Deng et al. 2026), not from prior work by the present authors, and the RAE encoder and teacher flow model are frozen pretrained checkpoints. The Diffusion-GAN connection in Proposition 1 is a derived identity and is explicitly related to Lai et al. (2026), so it is not a renamed known result presented as new. The decision to remove the MAE is motivated by Theorem 1 and the statistics in Table 2; whether that motivation is sound is a correctness and evidence question, not a circularity one. In particular, the proof of Theorem 1 appears internally inconsistent: the theorem claims V_j -> 0, while Eq. (30) in the proof derives ||V_j|| -> sqrt(1/3) r. The paper's own limitations (Sec. 4.4) admit the theory is a simplified model with a gap to real RAE latents, and Appendix B.1 notes a concurrent work (Zhang et al. 2026) reports different geometry. These are substantive validity concerns, but they do not reduce the paper's derivation to its own inputs. Likewise, hyperparameter selection on the target FID benchmark and the single unseeded run are statistical-evidence concerns, not definitional circularity. Under the required standard of quoting a specific reduction, no circular step is present.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 0 invented entities

No new physical entities are introduced. The free parameters are mostly hyperparameters inherited from the Drifting baseline or selected by FID ablation. The key unvalidated input is the geometric assumption about RAE compactness, plus the simplified high-dimensional model in Theorem 1, which is internally inconsistent as written.

free parameters (6)
  • Drifting temperature τ = {0.02, 0.05, 0.2}
    Kernel bandwidth in the drifting field, inherited from the original Drifting baseline; chosen by hand, no sensitivity analysis or automatic selection.
  • Sample counts = Npos=256, Nneg=64, Nextra_neg=192, Ntotal_neg=256
    Selected by FID ablations on ImageNet 256 (Tables 6-7); the balanced 256/256 configuration gives the best FID and is the final hyperparameter setting.
  • Learning rate schedule = 3e-5 → 3e-7 over 10,000 steps
    Hand-chosen linear decay; no sensitivity analysis reported.
  • Input perturbation scale = τ times a Laplace-distributed norm
    Noise added to generated latents (Eq. 12); magnitude tied to the same temperature τ, chosen by hand.
  • Training budget = 10,000 steps (≈16 epochs)
    Fixed budget; no study of whether more or fewer steps would change the result.
  • EMA and optimizer settings = EMA=0.9995, warmup 1000 steps, AdamW β=(0.9,0.95), wd=0, grad clip=1.0
    Standard choices inherited from prior work; no ablation.
axioms (6)
  • domain assumption Drifting fields annihilated ⇒ generated distribution equals real distribution (Deng et al. 2026)
    Basis of the Drift objective used in §3.1; relied on without re-derivation in this paper.
  • domain assumption RAE latent space is sufficiently compact/concentrated that the positive attraction term in Eq. (11) does not vanish
    Load-bearing for removing the auxiliary MAE in §3.3; based on NN-d and S-MMD in Table 2, while a concurrent paper (Zhang et al. 2026) reports different geometry and the comparison protocol is ad hoc.
  • ad hoc to paper The high-dimensional sphere/cube model (positive samples on S^{d-1}, negatives in [-r,r]^d) approximates real VAE/RAE latent geometry
    Used in Theorem 1; the authors admit in §4.4 that this simplified model leaves a gap from the actual distribution of RAE latents.
  • domain assumption The pretrained flow model v_θ(z,1) is a sufficiently good initialization for one-step distillation
    Stated in §3.2: poor initialization is detrimental to Drifting, but distillation starts from the teacher. No formal condition is given for when this holds.
  • domain assumption Token-wise independent drifting objectives on each spatial token produce a good joint generator
    Eq. (10) sums the drifting loss over C patch tokens; no theoretical guarantee that per-token distribution matching yields a correct joint distribution over tokens.
  • standard math The optimal-discriminator interpretation (Proposition 1) extends from l=2 kernels to the l=1 normalized directions used in practice
    Appendix A.2 derives the connection for exponential kernels; the practical implementation uses l=1 normalized displacement directions, which is only approximately the score difference.

pith-pipeline@v1.3.0-alltime-deepseek · 16750 in / 18916 out tokens · 177532 ms · 2026-08-04T04:41:52.582786+00:00 · methodology

0 comments
read the original abstract

Despite the significant training acceleration and promising performance, Representation Autoencoders (RAEs) are mainly criticized for poor distillation effectiveness. In this work, we argue that RAE is competent at high-quality one-step generation. We achieve 1.48 FID with only 16-epoch distillation on ImageNet 256 dataset, surpassing various state-of-the-art methods. To achieve this, we quantitatively study the geometrical behavior of different underlying data spaces. We conclude that conventional distillation methods heavily rely on priors of plain teacher denoising trajectories, while RAE incurs much more complex trajectories with poor properties due to ill anisotropical latent space. We introduce the recently proposed drifting field as the distillation methodology, which makes use of semantically rich RAE latents and provides direct supervision involving no dependency. Bridging our Drift-RAE with previous generative paradigms, we propose several insightful modifications, including the first extrapolation-based guided sampling pipeline for one-step generation with barely no cost. The code will be made publicly available.

Figures

Figures reproduced from arXiv: 2606.15553 by Gen Li, Jiawei Zhang, Mengfei Xia, Yuantao Gu.

Figure 1
Figure 1. Figure 1: Curvatures of ODE trajectories in RAE and SD-VAE latent spaces [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Visualizations of generated samples from distilled DiTDH-XL (FID = 1.77). carrying samples to 64 and vary only the number of detached auxiliary samples. Increasing only one side, either positive or negative samples, leads to substantial performance degradation. In contrast, the best performance is achieved when the positive and negative samples used in the Drifting computation are balanced, with both set t… view at source ↗
Figure 3
Figure 3. Figure 3: Additional visualizations of generated samples from distilled DiT [PITH_FULL_IMAGE:figures/full_fig_p019_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Additional visualizations of generated samples from distilled DiT [PITH_FULL_IMAGE:figures/full_fig_p020_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Additional visualizations of generated samples from distilled DiT [PITH_FULL_IMAGE:figures/full_fig_p021_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Additional visualizations of generated samples from distilled DiT [PITH_FULL_IMAGE:figures/full_fig_p022_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

19 extracted references · 12 linked inside Pith

  1. [2]

    That is to say, the positive part directly vanishes

    = 2(d−1) d4 .(29) Therefore y→0 for almost any x∈R d as the dimension d goes to infinity. That is to say, the positive part directly vanishes. Recall that V− j → q 1 3 rasdgoes to infinity, we can deduce that ∥Vj∥ → r 1 3 rasd→+∞.(30) Note that the length of the diagonal of [−r, r]d is r √ d, and 1 3 d →0 when d goes to infinity. Therefore we can deduce t...

    2023

  2. [5]

    Distribution Matching Distillation without Fake Score Network.arXiv preprint arXiv:2605.19256,

    7 Youngjoong Kim, Deokyeong Lee, and Jaesik Park. Distribution Matching Distillation without Fake Score Network.arXiv preprint arXiv:2605.19256,

    Pith/arXiv arXiv

  3. [10]

    Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference.arXiv preprint arXiv:2310.04378,

    7 Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference.arXiv preprint arXiv:2310.04378,

    Pith/arXiv arXiv

  4. [12]

    Flow Map Distillation Without Data.arXiv preprint arXiv:2511.19428,

    6 Shangyuan Tong, Nanye Ma, Saining Xie, and Tommi Jaakkola. Flow Map Distillation Without Data.arXiv preprint arXiv:2511.19428,

    arXiv

  5. [13]

    Scaling Text-to-Image Diffusion Transformers with Representation Autoencoders.arXiv preprint arXiv:2601.16208,

    7 Shengbang Tong, Boyang Zheng, Ziteng Wang, Bingda Tang, Nanye Ma, Ellis Brown, Jihan Yang, Rob Fergus, Yann LeCun, and Saining Xie. Scaling Text-to-Image Diffusion Transformers with Representation Autoencoders.arXiv preprint arXiv:2601.16208,

    arXiv

  6. [14]

    Improved Distribution Matching Distillation for Fast Image Synthesis

    7 Tianwei Yin, Michaël Gharbi, Taesung Park, Richard Zhang, Eli Shechtman, Fredo Durand, and William T Freeman. Improved Distribution Matching Distillation for Fast Image Synthesis. InAdvances in Neural Information Processing System, 2024a. 1 Tianwei Yin, Michaël Gharbi, Taesung Park, Richard Zhang, Eli Shechtman, Fredo Durand, and William T Freeman. Impr...

    Pith/arXiv arXiv

  7. [15]

    RiT: Vanilla Diffusion Transformers Suffice in Representation Space.arXiv preprint arXiv:2605.21981,

    1 Le Zhang, Ning Mang, and Aishwarya Agrawal. RiT: Vanilla Diffusion Transformers Suffice in Representation Space.arXiv preprint arXiv:2605.21981,

    Pith/arXiv arXiv

  8. [16]

    Diffusion transformers with representation autoencoders.arXiv preprint arXiv:2510.11690,

    17 Boyang Zheng, Nanye Ma, Shengbang Tong, and Saining Xie. Diffusion transformers with representation autoencoders.arXiv preprint arXiv:2510.11690,

    Pith/arXiv arXiv

  9. [18]

    This connection provides the motivation for the modifications introduced in Section 3.3

    framework. This connection provides the motivation for the modifications introduced in Section 3.3. First, the softmax weights in Drifting arise from differentiating the log-density of an exponential- kernel mixture. Therefore, the additional y-softmax used in the original implementation is not directly induced by this derivation and may alter the gradien...

    2023

  10. [19]

    We suspect that the discrepancy mainly comes from the aggregation protocol

    also studies representation-space geometry and reports conclusions that appear different from ours. We suspect that the discrepancy mainly comes from the aggregation protocol. Zhang et al. (2026) measures global statistics after mixing samples from all classes and aggregating token positions, while our analysis is performed per class and per token. For ou...

    2026

  11. [2009]

    Generative Modeling via Drifting.arXiv preprint arXiv:2602.04770,

    6 Mingyang Deng, He Li, Tianhong Li, Yilun Du, and Kaiming He. Generative Modeling via Drifting.arXiv preprint arXiv:2602.04770,

    Pith/arXiv arXiv

  12. [2017]

    Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets.arXiv preprint arXiv:2311.15127,

    5 Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, Varun Jampani, and Robin Rombach. Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets.arXiv preprint arXiv:2311.15127,

    Pith/arXiv arXiv

  13. [2019]

    FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space.arXiv preprint arXiv:2506.15742,

    6 Black Forest Labs, Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dockhorn, Jack English, Zion English, Patrick Esser, Sumith Kulal, Kyle Lacey, Yam Levi, Cheng Li, Dominik Lorenz, Jonas Müller, Dustin Podell, Robin Rombach, Harry Saini, Axel Sauer, and Luke Smith. FLUX.1 Kontext: Flow Matching for In-Context Imag...

    Pith/arXiv arXiv

  14. [2020]

    Meanflow transformers with representation autoencoders.arXiv preprint arXiv:2511.13019,

    1, 2 Zheyuan Hu, Chieh-Hsin Lai, Ge Wu, Yuki Mitsufuji, and Stefano Ermon. Meanflow transformers with representation autoencoders.arXiv preprint arXiv:2511.13019,

    arXiv

  15. [2022]

    Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation

    7 Axel Sauer, Frederic Boesel, Tim Dockhorn, Andreas Blattmann, Patrick Esser, and Robin Rombach. Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation. InSIGGRAPH Asia, 2024a. 1 Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial Diffusion Distillation. In European Conference on Computer Vision, 202...

    Pith/arXiv arXiv

  16. [2023]

    One-step Latent-free Image Generation with Pixel Mean Flows.arXiv preprint arXiv:2601.22158,

    1, 2, 3 Yiyang Lu, Susie Lu, Qiao Sun, Hanhong Zhao, Zhicheng Jiang, Xianbang Wang, Tianhong Li, Zhengyang Geng, and Kaiming He. One-step Latent-free Image Generation with Pixel Mean Flows.arXiv preprint arXiv:2601.22158,

    Pith/arXiv arXiv

  17. [2024]

    pi-Flow: Policy-Based Few-Step Generation via Imitation Distillation.arXiv preprint arXiv:2510.14974,

    3, 16 Hansheng Chen, Kai Zhang, Hao Tan, Leonidas Guibas, Gordon Wetzstein, and Sai Bi. pi-Flow: Policy-Based Few-Step Generation via Imitation Distillation.arXiv preprint arXiv:2510.14974,

    arXiv

  18. [2025]

    A unified view of drifting and score-based models.arXiv preprint arXiv:2603.07514,

    1 Chieh-Hsin Lai, Bac Nguyen, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yuki Mitsufuji, Stefano Ermon, and Molei Tao. A unified view of drifting and score-based models.arXiv preprint arXiv:2603.07514,

    Pith/arXiv arXiv

  19. [2026]

    SDXL-Lightning: Progressive Adversarial Diffusion Distillation

    15 Shanchuan Lin, Anran Wang, and Xiao Yang. SDXL-Lightning: Progressive Adversarial Diffusion Distillation. arXiv preprint arXiv:2402.13929,

    Pith/arXiv arXiv