Pith. sign in

REVIEW 3 major objections 3 minor 2 cited by

Physics Informed Distillation for Diffusion Models

T0 review · 3 major / 3 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A diffusion model can be distilled into a single-step generator by training a student trajectory function to solve the teacher's probability-flow ODE with a physics-informed residual loss.

desk verdict A clean, honest distillation method with a weak theoretical lemma that does not cover the LPIPS implementation. read the letter →

arxiv 2411.08378 v1 pith:P62JFCQZ submitted 2024-11-13 cs.LG cs.AI

Joshua Tian Jin Tee , Kang Zhang , Hee Suk Yoon , Dhananjaya Nagaraja Gowda , Chanwoo Kim , Chang D. Yoo This is my paper · ORCID
classification cs.LGcs.AI
keywords physicsinformeddistillationdiffusionmodelssingle-stepsamplingprobabilityflowODEknowledgenumericaldifferentiationtrajectoryfunctionLPIPS
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a diffusion model can be distilled into a single-step generator without any real or synthetic training images by treating the teacher as the deterministic probability-flow ODE that maps noise to clean images, and training a student trajectory function $x_\theta(z,t)$ with a physics-informed residual loss. The loss asks the student to satisfy the teacher's ODE at sampled collocation times, with the teacher evaluated at the student's own predicted point and detached from the gradient. The paper proves that if this residual is exactly zero, the student reproduces the Euler discretization of the teacher's ODE, bounding trajectory error by $\mathcal{O}(\Delta t)$. On CIFAR-10 and ImageNet 64x64, the method reaches FID 3.92 and 9.49 with a single function evaluation, matching recent distillation methods while needing no synthetic dataset and showing a predictable trend in its main hyperparameter.

What carries the argument

The load-bearing object is the student trajectory function $x_\theta(z,t)$ with the hard-boundary parametrization $x_\theta(z,t)=c_{\mathrm{skip}}(t)z+c_{\mathrm{out}}(t)X_\theta(c_{\mathrm{in}}(T)z,c_{\mathrm{noise}}(t))$, using $c_{\mathrm{skip}}(t)=t/T$ and $c_{\mathrm{out}}(t)=(T-t)/T$ so the boundary condition $x_\theta(z,T)=z$ holds by construction. The objective that carries the argument is the PID residual of Eq. 11, which replaces the exact time derivative in the PINN loss with a numerical difference and evaluates the teacher at the student's own predicted point with stop-gradient. At zero loss this residual forces the student to trace the Euler discretization of the teacher's ODE, which is the mechanism behind Lemma 1's $\mathcal{O}(\Delta t)$ bound and the method's single-step sampling.

What would settle it

After training PID on CIFAR-10, evaluate the residual of Eq. 11 on a fixed set of noise samples: if it plateaus well above zero while FID stays near 3.92, the zero-loss precondition of Lemma 1 is not met. Then compare the student's single-step output $x_\theta(z,\epsilon)$ with a high-accuracy numerical solution of the teacher's ODE for the same $z$; if the discrepancy does not shrink as the discretization number $N$ grows, the claimed $\mathcal{O}(\Delta t)$ trajectory approximation fails.

Watch

Extended reading notes

Core claim

The central claim is that the teacher's probability-flow ODE is not just an analysis tool but a training objective: a student network $x_\theta(z,t)$, initialized from the teacher, can be trained with only noise samples and teacher evaluations to approximate the full trajectory from noise at time $T$ to the clean image at time $\epsilon$. The training objective is the PID residual, $$d\!\left(x_\$\theta$(z,t_i)-t_i\frac{x_\$\theta$(z,t_i)-x_\$\theta$(z,t_{i+1})}{t_i-t_{i+1}},\ \mathrm{sg}(D_\$\varphi$(x_\$\theta$(z,t_i),t_i))\right),$$ where $D_\phi$ is the teacher denoiser and $\mathrm{sg}$ stops gradients. A hard boundary parametrization $c_{\mathrm{skip}}(t)=t/T$, $c_{\mathrm{out}}(t)=(T-t)/T$ enforces $x_\theta(z,T)=z$ identically, numerical differentiation replaces automatic differentiation, and LPIPS replaces $L2$ because pixel-wise losses waste capacity. Lemma 1 states that at zero loss the student's trajectories match the Euler discretization of the teacher ODE up to $\mathcal{O}(\Delta t)$; empirically, training this objective yields single-step FIDs of 3.92 on CIFAR-10 and 9.49 on ImageNet 64x64.

Load-bearing premise

The load-bearing premise is that gradient descent on the bootstrapped PID residual reaches a fixed point close to zero loss; Lemma 1 only bounds the error when the loss is exactly zero, and the LPIPS metric used in the main experiments is not a proper metric as the proof requires.

Editorial extensions

If this is right

  • Sampling becomes a single forward pass: draw $z\sim\mathcal{N}(0,T^2I)$ and output $x_\theta(z,\epsilon)$, with no iterative solver.
  • Distillation requires no synthetic dataset: training uses only noise samples and teacher evaluations, removing the data-generation cost of methods like DSNO.
  • The main hyperparameter, discretization number $N$, behaves predictably: larger $N$ monotonically improves FID without increasing per-iteration training time, so it needs no dataset-specific tuning.
  • Using second-order central-difference numerical differentiation improves CIFAR-10 FID from 3.92 to 3.68 at the same two-evaluation cost.
  • The same objective compresses models: a student with a quarter of the teacher's parameters reaches FID 8.29 on CIFAR-10.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the teacher is used only as an ODE oracle, the same recipe should transfer to latent or text-conditioned diffusion models where synthetic-data distillation is even more expensive; this extension is not tested in the paper.
  • The zero-loss equivalence to an Euler solver suggests PID is learning to imitate one particular numerical integrator, so pairing higher-order numerical differentiation with higher-order teacher solvers is a natural way to push the error below $\mathcal{O}(\Delta t)$.
  • The ablations show sensitivity to teacher initialization and stop-gradient, which implies the bootstrapped residual is a local refinement of the teacher's own trajectory rather than a standalone solver; a testable consequence is that architecturally different students may need a different initialization or a curriculum.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper proposes Physics Informed Distillation (PID), a data-free distillation method that trains a student network x_theta(z,t) to approximate the probability-flow ODE trajectories of a pretrained diffusion teacher. The student is trained by minimizing a residual loss that compares the teacher's denoising output at the student's predicted point with a numerical-differentiation-based Euler update of the student trajectory. Single-step generation is obtained by evaluating the student at the ODE endpoint. Experiments on CIFAR-10 and ImageNet 64x64 report FID/IS scores, ablations over discretization, metric choice, initialization, stop-gradient, and numerical differentiation, and training-time comparisons. The authors claim competitive performance with recent distillation methods while avoiding synthetic data and method-specific hyperparameter tuning, and they provide a theoretical bound (Lemma 1) on the discretization error when the loss is zero.

Significance. If the central claim held, PID would be a practically attractive data-free distillation scheme: it requires no synthetic dataset, has a predictable dependence on the discretization number, and the ablations in Sections 6.2-6.6 provide useful design guidance. The public code and pretrained checkpoints support reproducibility. The CIFAR-10 result (FID 3.92) is competitive with recent single-step methods, and the convergence trend with increasing N in Figure 6 is a genuinely useful empirical property. However, the ImageNet 64x64 result (FID 9.49) is substantially behind the strongest baselines, and the theoretical support for the method is weaker than the paper suggests. The core idea is interesting and the empirical study is careful, but the headline theoretical guarantee does not apply to the metric used in the main experiments, and no evidence is provided that the training dynamics reach the zero-residual regime required by Lemma 1.

major comments (3)
  1. [Section 4.3 and Appendix A.2, Lemma 1] Lemma 1 is proved under the assumption that d is a proper metric with d(x,y)=0 iff x=y, as stated in Eq. (12). The paper explicitly says at the end of Section 4.3 that LPIPS is not a proper distance metric, yet LPIPS is the default metric in Algorithm 1 and in Tables 1 and 2. Therefore the only theoretical guarantee does not apply to the method as evaluated. The authors should either extend the lemma to LPIPS-like non-metric losses, or restrict the theoretical claim to the L2 variant and present the LPIPS experiments as empirical. In addition, the proof shows that zero loss makes the student exactly reproduce an Euler discretization of the teacher ODE; the O(Delta t) bound is then the standard Euler error, so the lemma is essentially a restatement of the objective plus a textbook solver bound, not an independent validation of the training dynamics.
  2. [Section 4.3, Eq. (11) and Algorithm 1] Lemma 1 is conditional on LPID = 0, but the paper never reports the achieved LPID value or the algebraic residual r_i = x_theta(z,t_i) - t_i * (x_theta(z,t_i)-x_theta(z,t_{i+1}))/(t_i-t_{i+1}) - sg(D_phi(x_theta(z,t_i),t_i)). Because the teacher target is bootstrapped from the student's own trajectory with a stop-gradient, there is no guarantee that gradient descent reaches the zero-residual fixed point; the ablations in Sections 6.2-6.4 show strong sensitivity to automatic differentiation, initialization, and stop-gradient, indicating that the fixed point is not reached trivially. Without reporting the residual trajectory and final residual, the claimed O(Delta t) accuracy of the trained student relative to the teacher's Euler trajectories is not established.
  3. [Abstract and Table 2] The claim of "performance comparable to recent distillation methods" is not supported on ImageNet 64x64. PID reaches FID 9.49, whereas Diff-Instruct, Consistency Model, and DSNO reach 4.24, 6.20, and 7.83 respectively. This is a substantial gap, not a small margin. The authors should either soften the claim to state explicitly that PID is competitive on CIFAR-10 but lags on ImageNet, or provide additional evidence (e.g., longer training or tuned hyperparameters) before claiming comparable performance on the larger benchmark.
minor comments (3)
  1. [Algorithm 1] Line 3 samples i uniformly from {0,...,N}, but line 5 uses x_theta(z,t_{i+1}); for i=N the index t_{N+1} is undefined. The sampling range should be {0,...,N-1} unless t_{N+1} is otherwise defined.
  2. [Eq. (7) and Appendix A.4] The parametrization notation is inconsistent: Eq. (7) defines c_in(T) = 1/sqrt(0.5^2+T^2), but Appendix A.4 states that c_in(T) is set to T. Please clarify which one is used.
  3. [Section 6.6 and Figure 6] The sentence "a higher discretization number has no effect on training time as seen in Figure 6" is not supported by the figure, which plots FID versus training iterations rather than wall-clock time. The claim may be true because each iteration uses a fixed two-point stencil, but it should be stated as such or supported by timing data.

Circularity Check

1 steps flagged · score 4.0 of 10

The theoretical Lemma 1 reduces to the PID loss definition plus standard Euler error; empirical FID results remain independent external evidence.

  1. self definitional [Section 4.3, paragraph following Lemma 1; supported by Appendix A.2, Eq. (13), and Eq. (11).]
    "An intriguing aspect of this theorem lies in its connection to Euler solvers and first-order numerical gradient approximations. Specifically, when our model achieves a loss of 0, the approximate trajectories traced by the PID-trained model, denoted as xθ(z,t ), effectively mirrors those obtained using a simple Euler solver employing an equivalent number of steps, denoted as N."

    The lemma's conclusion is obtained by rearranging the PID loss (Eq. 11): LPID = 0 asserts that the student's finite-difference expression equals the teacher denoiser at the student's own state, which is exactly the Euler update for the teacher ODE (Appendix A.2, Eq. 13). The O(Δt) bound is then the standard global error of the Euler solver, not an independent property derived for PID. Thus the claimed theoretical guarantee is a consistency statement of the loss definition; it provides no evidence that gradient descent reaches a zero-residual fixed point, and it does not apply to the LPIPS metric used in the main experiments because the proof's property x = y iff d(x,y) = 0 fails for LPIPS.

full rationale

PID's central empirical validation is self-contained: single-step FID/IS values on CIFAR-10 and ImageNet 64x64 are measured against held-out datasets and compared with published baselines, so those claims are not circular. The circularity burden is confined to the theoretical analysis. Lemma 1 assumes LPID = 0, and the proof in Appendix A.2 immediately rearranges Eq. (11) into the Euler update Eq. (13); the O(Δt) bound is the standard Euler method global error imported from numerical analysis. Hence the 'theoretical bound' is a restatement of the objective: it says that if the student exactly satisfies the loss, the student's discrete trajectory is the teacher's Euler trajectory. That is a soundness or consistency statement, but it is not an independent validation that training reaches the zero-loss fixed point, and the paper's own ablations (random initialization, stop-gradient removal, and automatic differentiation) show the fixed point is not trivially reached. Moreover, the lemma requires a proper metric (x = y iff d(x,y) = 0), while the main experiments use LPIPS, which the paper admits 'is not a proper distance metric'; therefore the stated guarantee does not even apply to the evaluated configuration. These are correctness gaps, and the theoretical claim is partially circular because the conclusion is built into the loss by construction. Score 4 reflects partial circularity in the theoretical derivation with an independent empirical core.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim relies on the teacher defining a well-posed probability flow ODE, a proper metric for the theoretical bound, and the validity of numerical differentiation. The main empirical results do not depend on fitted constants, but the discretization number N is a hand-selected default. No new entities are postulated.

free parameters (1)
  • discretization number N = 250
    Default grid size chosen after Figure 6 shows FID plateau near N=250. The paper presents this as a predictable trend, but 250 is still a hand-selected default and the O(Delta t) bound depends on it.
assumptions (4)
  • domain assumption The teacher diffusion model D_phi is Lipschitz continuous in x on the time interval [epsilon, T].
    Invoked in Section 4.1 to guarantee a unique solution to the probability flow ODE via Picard-Lindelof (Grant 1999). The authors note Lipschitz constants explode near the origin (Yang et al. 2023), so this is a nontrivial assumption on [epsilon, T].
  • domain assumption The distance metric d satisfies x=y iff d(x,y)=0.
    Used in Appendix A.2 to turn LPID=0 into pointwise equality. The paper then replaces d with LPIPS in all main experiments despite noting LPIPS is not a proper metric, so the lemma does not directly apply to the reported results.
  • domain assumption First-order backward difference (x(t)-x(t-Delta t))/Delta t approximates the time derivative well enough across the sampled grid.
    Section 4.2 adopts numerical differentiation to avoid automatic differentiation; the error control is delegated to Lemma 1's O(Delta t) bound.
  • domain assumption The EDM pretrained teacher correctly models the score of the data distribution, so the probability flow ODE in Eq. 6 is the right system to solve.
    Section 3.1 and 4.1 rely on Song et al. (2021b) score matching and Karras et al. (2022) EDM configuration.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Physics Informed Distillation for Diffusion Models." pith.science (2026). https://pith.science/paper/P62JFCQZ

@misc{pith2026241108378,
  author       = {Pith},
  title        = {Pith review of: Physics Informed Distillation for Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P62JFCQZ}},
  note         = {Machine review of arXiv:2411.08378}
}
read the original abstract

Diffusion models have recently emerged as a potent tool in generative modeling. However, their inherent iterative nature often results in sluggish image generation due to the requirement for multiple model evaluations. Recent progress has unveiled the intrinsic link between diffusion models and Probability Flow Ordinary Differential Equations (ODEs), thus enabling us to conceptualize diffusion models as ODE systems. Simultaneously, Physics Informed Neural Networks (PINNs) have substantiated their effectiveness in solving intricate differential equations through implicit modeling of their solutions. Building upon these foundational insights, we introduce Physics Informed Distillation (PID), which employs a student model to represent the solution of the ODE system corresponding to the teacher diffusion model, akin to the principles employed in PINNs. Through experiments on CIFAR 10 and ImageNet 64x64, we observe that PID achieves performance comparable to recent distillation methods. Notably, it demonstrates predictable trends concerning method-specific hyperparameters and eliminates the need for synthetic dataset generation during the distillation process. Both of which contribute to its easy-to-use nature as a distillation approach for Diffusion Models. Our code and pre-trained checkpoint are publicly available at: https://github.com/pantheon5100/pid_diffusion.git.

Figures

Figures reproduced from arXiv: 2411.08378 by the authors.

Figure 1
Figure 1. An overview of the proposed method, which involves training a model [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Conditional image generation comparison on ImageNet 64 [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Comparison between automatic differentiation and numerical differentiation on CIFAR-10 dataset. [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Comparison between student model weights random initialized and initialized with teacher model [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: The impact of stop gradient in the PID training on CIFAR-10 dataset. [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: Training curve with different discretizations [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 8
Figure 8. Figure 8: Trajectory comparisons on ImageNet with EDM teacher (top) and PID student (bottom). [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]
Figure 9
Figure 9. Figure 9: Unconditional image generation comparison on CIFAR-10 for the same seed. Left panel: random [PITH_FULL_IMAGE:figures/full_fig_p020_9.png]
Figure 10
Figure 10. Figure 10: Unconditional image generation comparison on CIFAR-10 for the same seed. [PITH_FULL_IMAGE:figures/full_fig_p022_10.png]
Figure 11
Figure 11. Figure 11: Conditional image generation comparison on ImageNet 64 [PITH_FULL_IMAGE:figures/full_fig_p023_11.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Amortized Moment Matching for Visual Generation

    cs.LG 2026-07 accept novelty 6.0 of 10

    Amortized Fréchet Distance uses neural nets to match conditional means and covariances, yielding stronger one-step visual generators than explicit FD-loss or multi-step teachers.

  2. Parallel Decoding Distillation for Fast Image and Video Generation

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A trajectory-based distillation method trains a student to predict multiple mean velocities per network evaluation, enabling 4-8 step generation with competitive quality and improved diversity.

Reference graph

Works this paper leans on

53 extracted references · 24 canonical work pages · cited by 2 Pith papers

  1. [1]

    Label-efficient semantic segmentation with diffusion models

    Dmitry Baranchuk, Andrey Voynov, Ivan Rubachev, Valentin Khrulkov, and Artem Babenko. Label-efficient semantic segmentation with diffusion models. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=SlxSY2UZQT

    work page 2022

  2. [2]

    Large scale GAN training for high fidelity natural image synthesis

    Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale GAN training for high fidelity natural image synthesis. In International Conference on Learning Representations, 2019. URL https://openreview.net/forum?id=B1xsqj09Fm

    2019

  3. [3]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv \'e J \'e gou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp.\ 9650--9660, 2021

    2021

  4. [4]

    Improved baselines with momentum contrastive learning

    Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297, 2020

    arXiv 2003

  5. [5]

    Can-pinn: A fast physics-informed neural network based on coupled-automatic--numerical differentiation method

    Pao-Hsiung Chiu, Jian Cheng Wong, Chinchun Ooi, My Ha Dao, and Yew-Soon Ong. Can-pinn: A fast physics-informed neural network based on coupled-automatic--numerical differentiation method. Computer Methods in Applied Mechanics and Engineering, 2022

    work page 2022

  6. [6]

    Scientific machine learning through physics--informed neural networks: where we are and what’s next

    Salvatore Cuomo, Vincenzo Schiano Di Cola, Fabio Giampaolo, Gianluigi Rozza, Maziar Raissi, and Francesco Piccialli. Scientific machine learning through physics--informed neural networks: where we are and what’s next. Journal of Scientific Computing, 2022

    work page 2022

  7. [7]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, 2009

    work page 2009

  8. [8]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems, 2021

    work page 2021

Show all 53 references
  1. [9]

    One-step diffusion distillation via deep equilibrium models

    Zhengyang Geng, Ashwini Pokle, and J Zico Kolter. One-step diffusion distillation via deep equilibrium models. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=b6XvK2de99

    2023

  2. [10]

    Generative adversarial networks

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 2020

    2020

  3. [11]

    Lecture 4: Picard-lindelöf theorem

    Christopher Grant. Lecture 4: Picard-lindelöf theorem. http://www.math.byu.edu/ grant/courses/m634/f99/lec4.pdf, 1999

    1999

  4. [12]

    Bootstrap your own latent-a new approach to self-supervised learning

    Jean-Bastien Grill, Florian Strub, Florent Altch \'e , Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning. Advances in neur...

    2020

  5. [13]

    Boot: Data-free distillation of denoising diffusion models with bootstrapping

    Jiatao Gu, Shuangfei Zhai, Yizhe Zhang, Lingjie Liu, and Joshua M Susskind. Boot: Data-free distillation of denoising diffusion models with bootstrapping. In ICML 2023 Workshop on Structured Probabilistic Inference \ & \ Generative Modeling , 2023

    2023

  6. [14]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), ...

    2017

  7. [15]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 2017 b

    2017

  8. [16]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33: 0 6840--6851, 2020

    2020

  9. [17]

    Multilayer feedforward networks are universal approximators

    Kurt Hornik, Maxwell Stinchcombe, and Halbert White. Multilayer feedforward networks are universal approximators. Neural networks, 1989

    1989

  10. [18]

    Gotta go fast when generating data with score-based models

    Alexia Jolicoeur-Martineau, Ke Li, R \'e mi Pich \'e -Taillefer, Tal Kachman, and Ioannis Mitliagkas. Gotta go fast when generating data with score-based models. arXiv preprint arXiv:2105.14080, 2021

    2021 arXiv

  11. [19]

    Elucidating the design space of diffusion-based generative models

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL https://openrevi...

    2022

  12. [20]

    Consistency trajectory models: Learning probability flow ODE trajectory of diffusion

    Dongjun Kim, Chieh-Hsin Lai, Wei-Hsiang Liao, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yutong He, Yuki Mitsufuji, and Stefano Ermon. Consistency trajectory models: Learning probability flow ODE trajectory of diffusion. In The Twelfth International Conference on Learning ...

    2024

  13. [21]

    Auto-encoding variational bayes

    Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013

    2013 arXiv

  14. [22]

    Glow: Generative flow with invertible 1x1 convolutions

    Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1x1 convolutions. Advances in neural information processing systems, 2018

    2018

  15. [23]

    Berg, Wan-Yen Lo, Piotr Dollar, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollar, and Ross Girshick. Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision ...

    2023

  16. [24]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical Report 0, University of Toronto, Toronto, Ontario, 2009. URL https://www.cs.toronto.edu/ kriz/learning-features-2009-TR.pdf

    2009

  17. [25]

    Artificial neural networks for solving ordinary and partial differential equations

    Isaac E Lagaris, Aristidis Likas, and Dimitrios I Fotiadis. Artificial neural networks for solving ordinary and partial differential equations. IEEE transactions on neural networks, 1998

    1998

  18. [26]

    Srdiff: Single image super-resolution with diffusion probabilistic models

    Haoying Li, Yifan Yang, Meng Chang, Shiqi Chen, Huajun Feng, Zhihai Xu, Qi Li, and Yueting Chen. Srdiff: Single image super-resolution with diffusion probabilistic models. Neurocomputing, 2022

    2022

  19. [27]

    On the variance of the adaptive learning rate and beyond

    Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, and Jiawei Han. On the variance of the adaptive learning rate and beyond. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=rkgz2aEKDr

    2020

  20. [28]

    Flow straight and fast: Learning to generate and transfer data with rectified flow

    Xingchao Liu, Chengyue Gong, and qiang liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=XVjTT1nw5z

    2023

  21. [29]

    DPM -solver: A fast ODE solver for diffusion probabilistic model sampling in around 10 steps

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. DPM -solver: A fast ODE solver for diffusion probabilistic model sampling in around 10 steps. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Proc...

    2022

  22. [30]

    Knowledge distillation in iterative generative models for improved sampling speed

    Eric Luhman and Troy Luhman. Knowledge distillation in iterative generative models for improved sampling speed. arXiv preprint arXiv:2101.02388, 2021

    2021 arXiv

  23. [31]

    Diff-instruct: A universal approach for transferring knowledge from pre-trained diffusion models

    Weijian Luo, Tianyang Hu, Shifeng Zhang, Jiacheng Sun, Zhenguo Li, and Zhihua Zhang. Diff-instruct: A universal approach for transferring knowledge from pre-trained diffusion models. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openr...

    2023

  24. [32]

    GLIDE : Towards photorealistic image generation and editing with text-guided diffusion models

    Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Mcgrew, Ilya Sutskever, and Mark Chen. GLIDE : Towards photorealistic image generation and editing with text-guided diffusion models. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, C...

    2022

  25. [33]

    Cdpmsr: Conditional diffusion probabilistic models for single image super-resolution

    Axi Niu, Kang Zhang, Trung X Pham, Jinqiu Sun, Yu Zhu, In So Kweon, and Yanning Zhang. Cdpmsr: Conditional diffusion probabilistic models for single image super-resolution. arXiv preprint arXiv:2302.12831, 2023

    2023 arXiv

  26. [34]

    Pytorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019

    2019

  27. [35]

    Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations

    Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 2019

    2019

  28. [36]

    Hierarchical text-conditional image generation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 2022

    2022 arXiv

  29. [37]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Information...

    2022

  30. [38]

    Image super-resolution via iterative refinement

    Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J Fleet, and Mohammad Norouzi. Image super-resolution via iterative refinement. TPAMI, 2022 b

    2022

  31. [39]

    Progressive distillation for fast sampling of diffusion models

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=TIdIXIpzhoI

    2022

  32. [40]

    Improved techniques for training gans

    Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. Advances in neural information processing systems, 2016

    2016

  33. [41]

    Very deep convolutional networks for large-scale image recognition

    Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014

    2014 arXiv

  34. [42]

    Deep unsupervised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning, 2015

    2015

  35. [43]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. 2021 a . URL https://openreview.net/forum?id=St1giarCHLP

    2021

  36. [44]

    Score-based generative modeling through stochastic differential equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. 2021 b . URL https://openreview.net/forum?id=PxTIG12RRHS

    2021

  37. [45]

    Consistency models

    Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In Proceedings of the 40th International Conference on Machine Learning, ICML'23. JMLR.org, 2023

    2023

  38. [46]

    On the theory of the brownian motion

    George E Uhlenbeck and Leonard S Ornstein. On the theory of the brownian motion. Physical review, 1930

    1930

  39. [47]

    On the theory of the brownian motion ii

    Ming Chen Wang and George Eugene Uhlenbeck. On the theory of the brownian motion ii. Reviews of modern physics, 1945

    1945

  40. [48]

    Deblurring via stochastic refinement

    Jay Whang, Mauricio Delbracio, Hossein Talebi, Chitwan Saharia, Alexandros G Dimakis, and Peyman Milanfar. Deblurring via stochastic refinement. In CVPR, 2022

    2022

  41. [49]

    Diffusion models for implicit image segmentation ensembles

    Julia Wolleb, Robin Sandk \"u hler, Florentin Bieder, Philippe Valmaggia, and Philippe C Cattin. Diffusion models for implicit image segmentation ensembles. In International Conference on Medical Imaging with Deep Learning, 2022

    2022

  42. [50]

    Eliminating lipschitz singularities in diffusion models

    Zhantao Yang, Ruili Feng, Han Zhang, Yujun Shen, Kai Zhu, Lianghua Huang, Yifei Zhang, Yu Liu, Deli Zhao, Jingren Zhou, et al. Eliminating lipschitz singularities in diffusion models. arXiv preprint arXiv:2306.11251, 2023

    2023 arXiv

  43. [51]

    Fast sampling of diffusion models with exponential integrator

    Qinsheng Zhang and Yongxin Chen. Fast sampling of diffusion models with exponential integrator. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=Loek7hfb46P

    2023

  44. [52]

    Fast sampling of diffusion models via operator learning

    Hongkai Zheng, Weili Nie, Arash Vahdat, Kamyar Azizzadenesheli, and Anima Anandkumar. Fast sampling of diffusion models via operator learning. In International Conference on Machine Learning, pp.\ 42390--42402. PMLR, 2023

    2023

  45. [53]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.