Pith. sign in

REVIEW 4 major objections 5 minor 61 references

Generative Lines Matching Models

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Straight probability-flow lines learned from a deterministic ODE pairing produce one- and two-step image generation with FID scores below the multi-step teacher.

desk verdict Strong one-step/two-step distillation results, but the headline FIDs are largely carried by the adversarial loss; the straight-line matching alone underperforms the teacher. read the letter →

arxiv 2412.06403 v1 pith:CNNBMUY5 submitted 2024-12-09 cs.CV

Ori Matityahu , Raanan Fattal This is my paper · ORCID
classification cs.CV
keywords generativemodelingdiffusionmodelsflowmatchingscoreprobabilityODEstraight-linetrajectoriessamplingefficiencyoptimaltransport
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that denoising diffusion, score-matching, and flow-matching models share a hidden degeneracy: when the regression loss pairs every noise vector with every target image, the low signal-to-noise limit forces the denoiser to predict a constant mean, creating false basins of attraction that curve the sampling trajectories and force many steps. To remove this degeneracy, the Lines Matching Model (LMM) uses a pretrained deterministic ODE sampler to assign each noise vector one concrete target image, then trains a network to map the interpolated noisy image back to that exact endpoint. Because each training pair knows its destination, the learned flow lines are globally straight and constant-speed, so one or two function evaluations are enough. On CIFAR-10, ImageNet 64x64, and AFHQ 64x64, the LMM reports FID scores lower than prior distillation and flow-matching baselines at NFE 1 and 2. The paper also proves that replacing random pairing with mini-batch optimal transport cannot fix the problem efficiently, because the required batch size grows exponentially with dimension.

What carries the argument

The load-bearing object is the deterministic pairing $\psi^*(x_0) = N^*_{\mathrm{Sampler}}(x_0)$ inherited from the EDM probability-flow ODE. The EDM sampler's multi-step integration defines a well-posed change of variable between Gaussian source noise and the data distribution, so each training example consists of a noise vector and one concrete target image; Eq. 3 then regresses the network $N_\theta(x_1 + \sigma x_0, \sigma)$ to the endpoint $x_1$. Because every point along the interpolating segment is trained toward the same endpoint, the flow lines are globally straight and the velocity field is constant-speed, with $v/\sigma = (N_\theta(x_\sigma, \sigma) - x_\sigma)/\sigma$. Since $N_\theta$ outputs clean signals rather than velocity vectors, the objective can be swapped for a perceptual loss, augmented by an adversarial loss, and restricted to the noise levels used at sampling time.

What would settle it

Concretely, train an LMM whose teacher is an EDM trained on CIFAR-10 with one class deleted, then report per-class FID or mode recall for the deleted class at NFE=1. If the deleted class is absent, the teacher's marginal quality is the binding assumption; if it appears with good fidelity, the adversarial term overrides pairing bias.

Watch

Extended reading notes

Core claim

The paper's central claim is that the averaging in standard denoising, score, and flow-matching losses creates a singularity at low signal-to-noise ratio: the optimal predictor collapses to the mean of the source or target distribution, producing false basins of attraction that bend sampling trajectories and inflate the required number of steps. LMM avoids this by taking the deterministic correspondence $x_1 = N^*_{\mathrm{Sampler}}(x_0)$ from a pretrained EDM ODE sampler, which is an implicit change of variable between $p_0$ and $q$, and training $N_\theta$ on the loss in Eq. 3 so that the network maps every point on the segment $x_1 + \sigma x_0$ back to $x_1$. The iso-lines of the learned field are then globally straight, constant-speed lines, and the remaining synthesis error is concentrated at the endpoints rather than in integration. On the three benchmarks, one or two LMM steps produce lower FID than the multi-step teacher and the compared one-step distillation baselines.

Load-bearing premise

The training endpoints $x_1 = N^*_{\mathrm{Sampler}}(x_0)$ produced by the pretrained EDM are assumed to be faithful draws from the target distribution $q$ and to come from a well-defined change of variable; if the teacher's numerical integration drifts or its marginal is biased, the LMM inherits that bias because only the adversarial term touches real data.

Editorial extensions

If this is right

  • On conditional CIFAR-10, LMM reaches FID 1.57 at one function evaluation and 1.39 at two, below the 35-step EDM teacher's 1.79.
  • On ImageNet 64x64, LMM reports FID 1.47 at NFE=1 and 1.17 at NFE=2, improving on the compared one-step distillation baselines.
  • On AFHQ 64x64, LMM at NFE=2 reaches FID 1.54, beating the 79-step EDM's 1.96.
  • Additional sampling steps beyond two give only marginal FID gains because LMM flow lines are nearly straight; the dominant error is endpoint accuracy, not numerical integration.
  • Mini-batch optimal transport cannot remove the low-SNR attraction in high dimensions, because the batch size needed to find meaningful pairings grows exponentially with dimension.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same deterministic-pairing trick should transfer to any ODE-based generative teacher; if a DDIM or VP-ODE sampler provides the change of variable, the straightening argument would apply unchanged, though the noise parameterization would differ.
  • Because the adversarial loss is the only training term that sees real data, a strong discriminator could in principle compensate for a biased teacher; a direct test is to train LMM from a teacher with one class removed and ask whether the deleted mode reappears.
  • The exponential batch-size result suggests that scaling mini-batch OT is the wrong direction for high-dimensional pairing; learned transport maps of the kind LMM inherits from an ODE are a more promising route.
  • Extending the pairing construction to latent diffusion models would give a one-step generator in latent space; the same line-matching loss and discriminator setup should port directly once a deterministic latent ODE sampler is available.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Lines Matching Models (LMM), a generative model that trains a network to predict clean signal endpoints x1 from perturbed inputs x1 + σx0 along globally straight lines, where the pairing (x0, x1) is produced by a deterministic ODE-based sampler of a pre-trained Elucidated Diffusion Model (EDM). The training loss (Eq. 3) uses a perceptual VGG-based reconstruction norm and is optionally augmented with an adversarial loss (Eq. 5) and a sampling-optimized training (SOT) strategy. The authors argue that this approach avoids a low-SNR regression singularity that distorts the trajectories of denoising diffusion, score-matching, and flow-matching models, and they report state-of-the-art FID scores at 1–2 NFEs on CIFAR-10 (1.57/1.39 conditional), ImageNet 64×64 (1.47/1.17), and AFHQ 64×64 (2.68/1.54). The paper also contains a theoretical analysis (Appendix A.2) claiming that minibatch optimal transport pairing suffers from a curse of dimensionality, requiring batch sizes exponential in the signal dimension.

Significance. If the empirical results are valid, the LMM is a competitive low-NFE generation method that matches or beats recent distillation/GAN hybrids (e.g., CTM, SiD) on standard benchmarks. The paper's strengths include a clear diagnosis of the low-SNR regression degeneracy, a simple and effective training formulation, thorough ablations with reported means and standard deviations, and a detailed experimental protocol in Appendix A.4. The theoretical result on OT minibatch coupling is interesting but, as discussed below, is currently under-supported. The main conceptual novelty is the use of pre-trained deterministic ODE endpoints as an unambiguous pairing for straight-line flow training; this is a practical contribution even though its causal role in the final FID scores is not cleanly isolated from the adversarial loss.

major comments (4)
  1. [Section 3 and Appendix A.3, Table 7] The load-bearing attribution of the state-of-the-art results to the line-matching objective is not supported by the ablations. Table 7 shows the best no-adversarial CIFAR-10 FID is 3.124 ± 0.024 (NFE=1, VGG) and 2.796 ± 0.020 (NFE=2), while the EDM teacher with NFE=35 achieves 1.79; the advertised 1.57/1.39 numbers are obtained only after adding the discriminator loss of Eq. (5). The abstract and Section 3 present straight-line matching as the source of the efficiency and quality gains. To make this credible, the paper should either (i) report the LMM's FID relative to the teacher's endpoint distribution at matched NFEs, (ii) estimate the approximation error between p_{N*_Sampler} and q (e.g., by comparing the teacher's single-step output distribution to real data), or (iii) explicitly reframe the contribution as a hybrid distillation-plus-GAN method whose gains stem substantially from the adversarial component. Without such a clarification, the link between 'well-defined change-of-variable' and the headline FID scores is not established.
  2. [Appendix A.2, Eqs. (15)–(17)] The proof of the exponential batch-size requirement is dimensionally inconsistent. Eq. (15) defines ⟨x,y⟩ as the normalized inner product (1/d) Σ x_i y_i, but Eq. (17) expands the squared Euclidean norm ||E[x*_1] − x0||^2 using this same normalized inner product without restoring the factor d. The identity ||a − b||^2 = ||a||^2 + ||b||^2 − 2⟨a,b⟩ is valid only for the standard Euclidean inner product. As written, the lower bound in Eq. (17) is incorrect, and the subsequent bound in Eq. (20) does not follow. The qualitative conclusion (exponential n in d) may survive a corrected derivation, but the present analysis is not a valid proof and should be repaired or explicitly downgraded to a heuristic remark.
  3. [Section 3, Eq. (3) and Appendix A.4] The assumption p_{N*_Sampler} ≈ q is asserted rather than validated. The EDM sampler is a numerical ODE integration with N=18 or 40 steps (NFE=35 or 79) and a learned score function; both introduce approximation and discretization bias. The no-adversarial LMM results (FID 3.12 vs. teacher 1.79 on CIFAR-10) indicate that the learned endpoint mapping does not even reproduce the teacher's output distribution, let alone q. The paper should explicitly list this as a limitation and, ideally, report an empirical discrepancy measure between the no-adversarial LMM output distribution and the teacher's output distribution, as well as between the teacher's output distribution and q. Without this, the 'well-defined change-of-variable' framing overstates what Eq. (3) actually achieves.
  4. [Section 4, Tables 5 and 9] The AFHQ headline numbers are internally inconsistent and underspecified. The abstract reports 2.68/1.54 (NFE=1/2), the introduction reports 2.8/1.61, and Table 5 lists 2.68/1.54. Table 9 shows that the VGG+ADL configuration gives 2.687/1.545 while VGG+ADL+SOT gives 2.767/1.776, meaning the SOT strategy actually degrades AFHQ FID. The text claims SOT improves results and does not clarify which configuration is used for the claimed state-of-the-art numbers. This ambiguity must be resolved for the results to be reproducible.
minor comments (5)
  1. [Section 1] The phrase 'fundamental course-of-dimensionality' should be 'fundamental curse-of-dimensionality'.
  2. [Section 4] The sentence 'the discriminator architecture and hyper-parameters we used were not we were not tailored to this dataset' contains a duplicated phrase 'we were not'.
  3. [Appendix A.2, Eq. (16)] The convergence statement '⟨x,y⟩/d → N(0,d^{-1})' conflicts with the definition of ⟨x,y⟩ in Eq. (15). For large d, the correct limit is ⟨x,y⟩ → N(0,1/d) (or, if ⟨x,y⟩ denotes the unnormalized sum, the limit is N(0,d)). The notation should be harmonized.
  4. [References] The references Albergo & Vanden-Eijnden 2023a and 2023b are identical entries (same title, venue, and URL) and should be merged.
  5. [Appendix A.4] The source distribution is written as p0 = N(0, σmax), but the surrounding text and the sampling update imply that the intended covariance is σ_max^2 I (or the notation should be explained, e.g., 'N(0, σmax)' meaning a normal with standard deviation σmax). This is ambiguous.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: LMM's teacher pairing is external (EDM/Karras), the adversarial loss anchors training to real data, FID is measured against real data, and the OT analysis is a self-contained asymptotic bound.

full rationale

The paper's derivation chain is not circular in any load-bearing way. The training pairs (x0, N*_Sampler(x0)) in Eq. 3 are generated by the external EDM teacher of Karras et al., not by the present authors, and the paper contains no load-bearing self-citations. The central SOTA FID claims are evaluated against real benchmark data, and the improvement from 3.12/2.80 (VGG, no ADL) to 1.57/1.39 (VGG+ADL+SOT) is driven by a discriminator loss (Eq. 5) trained on authentic samples from q, so the headline numbers are not forced by the line-matching loss alone. The no-ADL ablation (FID 3.12 vs teacher 1.79) shows the line-matching step alone does not even reproduce the teacher's endpoints, which is a correctness/performance concern about the unvalidated assumption p_N*_Sampler ≈ q, but it is a stated domain assumption rather than a circular reduction. The straightness of LMM trajectories is indeed imposed by design: Eq. 3 regresses Nθ to the endpoint x1 along the line x1+σx0, and Eq. 4 then gives a constant-speed field if the fit is perfect; the paper explicitly says 'we obtain constancy in speed by design.' The empirical straightness measurements therefore check the degree of fit to the training objective rather than discover a new property, but the paper does not disguise this as an independent prediction. The OT curse-of-dimensionality bound in Appendix A.2 is a self-contained asymptotic calculation for Gaussian-to-Gaussian transport and does not reuse the paper's fitted values. No self-citation chain, imported uniqueness theorem, or ansatz-smuggling citation is present. Residual issues (teacher-bias propagation, AFHQ number inconsistency between abstract and introduction, discriminator sensitivity) are correctness or reporting concerns, not circularity.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central empirical method rests on the teacher-student pairing, where EDM sampler outputs serve as ground truth, and on the assumption that straight-line interpolation between those endpoints is conflict-free. The theoretical OT analysis rests on standard high-dimensional concentration but contains a norm inconsistency. No new physical or conceptual entities are introduced beyond the LMM model itself.

free parameters (3)
  • lambda_lines = 0.5
    Hand-chosen weight balancing the line-matching loss against the adversarial loss; ablations fix it rather than fit it, so it is a design choice rather than a fitted constant.
  • EDM teacher NFE = 35 for CIFAR-10, 79 for ImageNet and AFHQ
    The deterministic sampler N*_Sampler is run with these many function evaluations to produce training pairs; the pairing and therefore the learned lines depend on this choice.
  • Noise schedule rho, sigma_min = rho=7, sigma_min=0.002
    Noise schedule hyper-parameters taken from EDM; they define the sigma grid used in training and sampling.
assumptions (5)
  • domain assumption The EDM deterministic sampler N*_Sampler induces a well-defined change-of-variable and its output distribution approximates q.
    Section 3, Eq. 3: the entire training set of clean target samples comes from the teacher model, not from q directly; if the teacher drifts or is biased, LMM inherits the bias.
  • domain assumption Straight line segments between paired endpoints do not intersect at the same time sigma, so Eq. 3 has no conflicting regression targets.
    Section 3, paragraph 'An exception to this claim'; the paper argues this from non-intersecting ODE trajectories, but straight chords can intersect even when the ODE trajectories do not.
  • standard math High-dimensional Gaussian concentration: normalized dot products of independent d-dimensional Gaussians concentrate around 0 with scale d^{-1/2}, and nearest-neighbor improvement is governed by the extremal Gaussian maximum bound.
    Appendix A.2, Eqs. 15 to 20; used to derive the exponential batch-size requirement for minibatch OT.
  • domain assumption The VGG perceptual loss and the adversarial loss improve sample fidelity without distorting the marginal distribution learned by the line-matching objective.
    Section 3, 'Domain-Specific Loss' and 'Adversarial Loss'; these losses are added to the endpoint-matching objective and their effect is validated only empirically on FID.
  • ad hoc to paper A sufficiently expressive network that minimizes Eq. 3 will have the straight lines x1 + sigma x0 as its iso-contours.
    Section 3, 'Globally Straight Trajectories'; this is the design rationale for LMM, and it is approximate because the network is not a perfect fit and lines may conflict.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Generative Lines Matching Models." pith.science (2026). https://pith.science/paper/CNNBMUY5

@misc{pith2026241206403,
  author       = {Pith},
  title        = {Pith review of: Generative Lines Matching Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CNNBMUY5}},
  note         = {Machine review of arXiv:2412.06403}
}
read the original abstract

In this paper we identify the source of a singularity in the training loss of key denoising models, that causes the denoiser's predictions to collapse towards the mean of the source or target distributions. This degeneracy creates false basins of attraction, distorting the denoising trajectories and ultimately increasing the number of steps required to sample these models. We circumvent this artifact by leveraging the deterministic ODE-based samplers, offered by certain denoising diffusion and score-matching models, which establish a well-defined change-of-variables between the source and target distributions. Given this correspondence, we propose a new probability flow model, the Lines Matching Model (LMM), which matches globally straight lines interpolating the two distributions. We demonstrate that the flow fields produced by the LMM exhibit notable temporal consistency, resulting in trajectories with excellent straightness scores. Beyond its sampling efficiency, the LMM formulation allows us to enhance the fidelity of the generated samples by integrating domain-specific reconstruction and adversarial losses, and by optimizing its training for the sampling procedure used. Overall, the LMM achieves state-of-the-art FID scores with minimal NFEs on established benchmark datasets: 1.57/1.39 (NFE=1/2) on CIFAR-10, 1.47/1.17 on ImageNet 64x64, and 2.68/1.54 on AFHQ 64x64. Finally, we provide a theoretical analysis showing that the use of optimal transport to relate the two distributions suffers from a curse of dimensionality, where the pairing set size (mini-batch) must scale exponentially with the signal dimension.

Figures

Figures reproduced from arXiv: 2412.06403 by the authors.

Figure 1
Figure 1. LMM Generated CIFAR-10 Samples. Class unconditional on the left, and conditional on [PITH_FULL_IMAGE:figures/full_fig_p019_1.png] view at source ↗
Figure 2
Figure 2. LMM Generated Conditional ImageNet 64×64 Samples. Rows correspond to different classes [PITH_FULL_IMAGE:figures/full_fig_p019_2.png] view at source ↗
Figure 3
Figure 3. LMM Generated AFHQ 64×64 Samples. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 38 canonical work pages

  1. [1]

    Albergo and Eric Vanden - Eijnden

    Michael S. Albergo and Eric Vanden - Eijnden. Building normalizing flows with stochastic interpolants. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 . OpenReview.net, 2023 a . URL https://openreview.net/pdf?id=li7qeBbCR1t

    work page 2023

  2. [2]

    Albergo and Eric Vanden - Eijnden

    Michael S. Albergo and Eric Vanden - Eijnden. Building normalizing flows with stochastic interpolants. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 . OpenReview.net, 2023 b . URL https://openreview.net/pdf?id=li7qeBbCR1t

    work page 2023

  3. [3]

    Near-linear time approximation algorithms for optimal transport via sinkhorn iteration

    Jason Altschuler, Jonathan Weed, and Philippe Rigollet. Near-linear time approximation algorithms for optimal transport via sinkhorn iteration. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS'17, pp.\ 1961–1971, Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN 9781510860964

    work page 1961

  4. [4]

    Beyer, Jonathan Goldstein, Raghu Ramakrishnan, and Uri Shaft

    Kevin S. Beyer, Jonathan Goldstein, Raghu Ramakrishnan, and Uri Shaft. When is ''nearest neighbor'' meaningful? In Catriel Beeri and Peter Buneman (eds.), ICDT, volume 1540 of Lecture Notes in Computer Science, pp.\ 217--235. Springer, 1999. ISBN 3-540-65452-6

    work page 1999

  5. [5]

    Large scale GAN training for high fidelity natural image synthesis

    Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale GAN training for high fidelity natural image synthesis. In International Conference on Learning Representations, 2019. URL https://openreview.net/forum?id=B1xsqj09Fm

    2019

  6. [6]

    Weiss, Mohammad Norouzi, and William Chan

    Nanxin Chen, Yu Zhang, Heiga Zen, Ron J. Weiss, Mohammad Norouzi, and William Chan. Wavegrad: Estimating gradients for waveform generation, 2020

    work page 2020

  7. [7]

    Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordinary differential equations. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS'18, pp.\ 6572–6583, Red Hook, NY, USA, 2018. Curran Associates Inc

    work page 2018

  8. [8]

    Clarke and Stephen Van Gorder

    Allan J. Clarke and Stephen Van Gorder. On fitting a straight line to data when the noise in both variables is unknown. Journal of Atmospheric and Oceanic Technology, 30 0 (1): 0 151 -- 158, 2013. doi:10.1175/JTECH-D-12-00067.1. URL https://journals.ametsoc.org/view/journals/atot/30/1/jtech-d-12-00067_1.xml

    work page doi:10.1175/jtech-d-12-00067.1 2013

Show all 61 references
  1. [9]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp.\ 8780--8794. Curran Associates, Inc., ...

    2021

  2. [10]

    Taming transformers for high-resolution image synthesis

    Patrick Esser, Robin Rombach, and Björn Ommer. Taming transformers for high-resolution image synthesis. In CVPR, pp.\ 12873--12883. Computer Vision Foundation / IEEE, 2021. URL http://dblp.uni-trier.de/db/conf/cvpr/cvpr2021.html#EsserRO21

    2021

  3. [11]

    Alaya, Aur\' e lie Boisbunon, Stanislas Chambon, Laetitia Chapel, Adrien Corenflos, Kilian Fatras, Nemo Fournier, L\' e o Gautheron, Nathalie T.H

    R\' e mi Flamary, Nicolas Courty, Alexandre Gramfort, Mokhtar Z. Alaya, Aur\' e lie Boisbunon, Stanislas Chambon, Laetitia Chapel, Adrien Corenflos, Kilian Fatras, Nemo Fournier, L\' e o Gautheron, Nathalie T.H. Gayraud, Hicham Janati, Alain Rakotomamonjy, Ievgen Redko, Antoin...

    2021

  4. [12]

    Generative adversarial nets

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger (eds.), Advances in Neural Information Proces...

    2014

  5. [13]

    Susskind

    Jiatao Gu, Shuangfei Zhai, Yizhe Zhang, Lingjie Liu, and Joshua M. Susskind. BOOT : Data-free distillation of denoising diffusion models with bootstrapping. In ICML 2023 Workshop on Structured Probabilistic Inference & Generative Modeling , 2023. URL https://openreview.net/for...

    2023

  6. [14]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp.\ 6840--6851. Curran Associates, Inc., 2020. URL http...

    2020

  7. [15]

    Video diffusion models

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. arXiv:2204.03458, 2022

    2022 arXiv

  8. [16]

    Fleet, and Ting Chen

    Allan Jabri, David J. Fleet, and Ting Chen. Scalable adaptive computation for iterative generation. In Proceedings of the 40th International Conference on Machine Learning, ICML'23. JMLR.org, 2023

    2023

  9. [17]

    Perceptual losses for real-time style transfer and super-resolution

    Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling (eds.), Computer Vision -- ECCV 2016, pp.\ 694--711, Cham, 2016. Springer International Publishing. IS...

    2016

  10. [18]

    Rebooting acgan: auxiliary classifier gans with stable training

    Minguk Kang, Woohyeon Shim, Minsu Cho, and Jaesik Park. Rebooting acgan: auxiliary classifier gans with stable training. In Proceedings of the 35th International Conference on Neural Information Processing Systems, NIPS '21, Red Hook, NY, USA, 2024. Curran Associates Inc. ISBN...

    2024

  11. [19]

    Training generative adversarial networks with limited data

    Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Training generative adversarial networks with limited data. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS '20, Red Hook, NY, USA, 2020...

    2020

  12. [20]

    Elucidating the design space of diffusion-based generative models

    Tero Karras, Miika Aittala, Samuli Laine, and Timo Aila. Elucidating the design space of diffusion-based generative models. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS '22, Red Hook, NY, USA, 2022. Curran Associates Inc. I...

    2022

  13. [21]

    Kendall and A

    M.G. Kendall and A. Stuart. The Advanced Theory of Statistics. Vol. 2: Inference and: Relationsship. Griffin, 1973. URL https://books.google.co.il/books?id=elabQwAACAAJ

    1973

  14. [22]

    Consistency trajectory models: Learning probability flow ODE trajectory of diffusion

    Dongjun Kim, Chieh-Hsin Lai, Wei-Hsiang Liao, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yutong He, Yuki Mitsufuji, and Stefano Ermon. Consistency trajectory models: Learning probability flow ODE trajectory of diffusion. In The Twelfth International Conference on Learning ...

    2024

  15. [23]

    Guided-tts: A diffusion model for text-to-speech via classifier guidance

    Heeseung Kim, Sungwon Kim, and Sungroh Yoon. Guided-tts: A diffusion model for text-to-speech via classifier guidance. In International Conference on Machine Learning, 2021. URL https://api.semanticscholar.org/CorpusID:246430592

    2021

  16. [24]

    Kingma and Max Welling

    Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In Yoshua Bengio and Yann LeCun (eds.), 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings , 2014. URL http://arxiv.org/abs...

    2014 arXiv

  17. [25]

    Diffwave: A versatile diffusion model for audio synthesis, 2021

    Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis, 2021

    2021

  18. [26]

    Minimizing trajectory curvature of ODE -based generative models

    Sangyun Lee, Beomsu Kim, and Jong Chul Ye. Minimizing trajectory curvature of ODE -based generative models. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Mac...

    2023

  19. [27]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=PqvMRDCJT9t

    2023

  20. [28]

    Flow straight and fast: Learning to generate and transfer data with rectified flow

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 . OpenReview.net, 2023. URL https://op...

    2023

  21. [29]

    Instaflow: One step is enough for high-quality diffusion-based text-to-image generation

    Xingchao Liu, Xiwen Zhang, Jianzhu Ma, Jian Peng, and qiang liu. Instaflow: One step is enough for high-quality diffusion-based text-to-image generation. In The Twelfth International Conference on Learning Representations, 2024 a . URL https://openreview.net/forum?id=1k4yZbbDqX

    2024

  22. [30]

    Sora: A review on background, technology, limitations, and opportunities of large vision models, 2024 b

    Yixin Liu, Kai Zhang, Yuan Li, Zhiling Yan, Chujie Gao, Ruoxi Chen, Zhengqing Yuan, Yue Huang, Hanchi Sun, Jianfeng Gao, Lifang He, and Lichao Sun. Sora: A review on background, technology, limitations, and opportunities of large vision models, 2024 b

    2024

  23. [31]

    CM - GAN : Stabilizing GAN training with consistency models

    Haoye Lu, Yiwei Lu, Dihong Jiang, Spencer Ryan Szabados, Sun Sun, and Yaoliang Yu. CM - GAN : Stabilizing GAN training with consistency models. In ICML 2023 Workshop on Structured Probabilistic Inference & Generative Modeling , 2023. URL https://openreview.net/forum?id=Uh2WwUyiAv

    2023

  24. [32]

    Knowledge distillation in iterative generative models for improved sampling speed

    Eric Luhman and Troy Luhman. Knowledge distillation in iterative generative models for improved sampling speed. CoRR, abs/2101.02388, 2021. URL https://arxiv.org/abs/2101.02388

    2021 arXiv

  25. [33]

    Improved denoising diffusion probabilistic models

    Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp.\ 8162--8171...

    2021

  26. [34]

    GLIDE: towards photorealistic image generation and editing with text-guided diffusion models

    Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. GLIDE: towards photorealistic image generation and editing with text-guided diffusion models. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Cs...

    2022

  27. [35]

    Aram - Alexandre Pooladian, Heli Ben - Hamu, Carles Domingo - Enrich, Brandon Amos, Yaron Lipman, and Ricky T. Q. Chen. Multisample flow matching: Straightening flows with minibatch couplings. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, ...

    2023

  28. [36]

    Grad-tts: A diffusion probabilistic model for text-to-speech, 2021

    Vadim Popov, Ivan Vovk, Vladimir Gogoryan, Tasnima Sadekova, and Mikhail Kudinov. Grad-tts: A diffusion probabilistic model for text-to-speech, 2021

    2021

  29. [37]

    Hierarchical text-conditional image generation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. ArXiv, abs/2204.06125, 2022. URL https://api.semanticscholar.org/CorpusID:248097655

    2022 arXiv

  30. [38]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj \"o rn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 10684--10695, 2022

    2022

  31. [39]

    Sara Mahdavi, Raphael Gontijo-Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Lit, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Raphael Gontijo-Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion ...

    2024

  32. [40]

    Progressive distillation for fast sampling of diffusion models

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022 . OpenReview.net, 2022. URL https://openreview.net/forum?id=TIdIXIpzhoI

    2022

  33. [41]

    Stylegan-xl: Scaling stylegan to large diverse datasets

    Axel Sauer, Katja Schwarz, and Andreas Geiger. Stylegan-xl: Scaling stylegan to large diverse datasets. In ACM SIGGRAPH 2022 Conference Proceedings, SIGGRAPH '22, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 9781450393379. doi:10.1145/3528233.3530738

    2022

  34. [42]

    Make-a-video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, Devi Parikh, Sonal Gupta, and Yaniv Taigman. Make-a-video: Text-to-video generation without text-video data. In The Eleventh International Conference on Lea...

    2023

  35. [43]

    Deep unsupervised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In Francis Bach and David Blei (eds.), Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of ...

    2015

  36. [44]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 . OpenReview.net, 2021 a . URL https://openreview.net/forum?id=St1giarCHLP

    2021

  37. [45]

    Improved techniques for training consistency models

    Yang Song and Prafulla Dhariwal. Improved techniques for training consistency models. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=WNzy9bRDvG

    2024

  38. [46]

    Generative modeling by estimating gradients of the data distribution

    Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d Alch\' e -Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, ...

    2019

  39. [47]

    Improved techniques for training score-based generative models

    Yang Song and Stefano Ermon. Improved techniques for training score-based generative models. In Hugo Larochelle, Marc'Aurelio Ranzato, Raia Hadsell, Maria - Florina Balcan, and Hsuan - Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on ...

    2020

  40. [48]

    Sliced score matching: A scalable approach to density and score estimation

    Yang Song, Sahaj Garg, Jiaxin Shi, and Stefano Ermon. Sliced score matching: A scalable approach to density and score estimation. In Proceedings of the Thirty-Fifth Conference on Uncertainty in Artificial Intelligence, UAI 2019, Tel Aviv, Israel, July 22-25, 2019 , pp.\ 204, 2...

    2019

  41. [49]

    Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole

    Yang Song, Jascha Sohl - Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May ...

    2021

  42. [50]

    Consistency models

    Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In Proceedings of the 40th International Conference on Machine Learning, ICML'23. JMLR.org, 2023

    2023

  43. [51]

    Catastrophic forgetting and mode collapse in gans

    Hoang Thanh-Tung and Truyen Tran. Catastrophic forgetting and mode collapse in gans. In 2020 International Joint Conference on Neural Networks (IJCNN), pp.\ 1--10, 2020. doi:10.1109/IJCNN48605.2020.9207181

    2020

  44. [52]

    Improving and generalizing flow-based generative models with minibatch optimal transport

    Alexander Tong, Kilian FATRAS, Nikolay Malkin, Guillaume Huguet, Yanlei Zhang, Jarrid Rector-Brooks, Guy Wolf, and Yoshua Bengio. Improving and generalizing flow-based generative models with minibatch optimal transport. Transactions on Machine Learning Research, 2024. ISSN 2835-8856

    2024

  45. [53]

    A connection between score matching and denoising autoencoders

    Pascal Vincent. A connection between score matching and denoising autoencoders. Neural Comput., 23 0 (7): 0 1661–1674, jul 2011. ISSN 0899-7667. doi:10.1162/NECO_a_00142

    2011 doi

  46. [54]

    One-step diffusion with distribution matching distillation

    Tianwei Yin, Micha \"e l Gharbi, Richard Zhang, Eli Shechtman, Fr \'e do Durand, William T Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. In CVPR, 2024

    2024

  47. [55]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018

    2018

  48. [56]

    Differentiable augmentation for data-efficient gan training

    Shengyu Zhao, Zhijian Liu, Ji Lin, Jun-Yan Zhu, and Song Han. Differentiable augmentation for data-efficient gan training. In Conference on Neural Information Processing Systems (NeurIPS), 2020

    2020

  49. [57]

    Score identity distillation: Exponentially fast distillation of pretrained diffusion models for one-step generation

    Mingyuan Zhou, Huangjie Zheng, Zhendong Wang, Mingzhang Yin, and Hai Huang. Score identity distillation: Exponentially fast distillation of pretrained diffusion models for one-step generation. In International Conference on Machine Learning, 2024. URL https://arxiv.org/abs/2404.04057

    2024 arXiv

  50. [58]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  51. [59]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  52. [60]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  53. [61]

    We used the same network architecture and hyper-parameters as existing models, with all the implementation details provided in Appendix append:impl

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.