Pith. sign in

REVIEW 5 major objections 4 minor 39 references

Generative Models: Principles, Architectures, and Applications

T0 review · 5 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Pith's one-line read This book claims all of generative AI—VAEs, GANs, diffusion, flow matching, and sampling—can be taught as one coherent distribution-transformation subject, from ELBO math to Stable Diffusion 3.

desk verdict A useful, comprehensive textbook-style review of generative modeling whose organizational framing is genuinely helpful, but internal math errors in the foundations need fixing before it can be trusted as a self-contained resource. read the letter →

arxiv 2608.08101 v1 pith:WSIAVDQL submitted 2026-08-08 cs.AI

classification cs.AI
keywords generativemodelsvariationalautoencodersdiffusionflowmatchingscoreexpectation-maximizationdistributiontransformationarchitectures
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This book tries to establish that the sprawling field of generative AI can be organized as a single subject: every major model family is a distribution-transformation problem, and one shared mathematical toolkit suffices to understand them. The book argues that if a reader masters the evidence lower-bound (ELBO), the forward/reverse noising idea, score functions, and conditional vector fields, they can derive the objectives of VAEs, GANs, diffusion models, and flow-matching models from common principles. The pedagogical payoff it claims is that no separate course is needed for each family, and that the same variational bound and transport picture carry across all of them. The text aims to be self-contained mathematically, starting from SVD and Gaussian identities and building up to U-Nets, ControlNet, Diffusion Transformers, and DPM-Solver++.

What carries the argument

The load-bearing objects are the evidence lower-bound and the distribution-transformation framing. The book develops $\mathrm{ELBO} = \ln p_\theta(\mathbf{x}) - D_{\mathrm{KL}}[q \| p_\theta(\cdot|\mathbf{x})]$ as the shared objective for VAEs, EM, and diffusion models; forward and reverse noising processes $q(\mathbf{x}_t|\mathbf{x}_0)$ for diffusion and score-based models; conditional and marginal probability paths with vector fields for flow matching; and exponential-integrator solvers for accelerated sampling. The ELBO decomposition $\ln p_\theta(\mathbf{X}) = \mathrm{ELBO} + D_{\mathrm{KL}}[q \| p_\theta(\mathbf{z}|\mathbf{x})]$ is the recurring identity that connects variational inference, EM's E-step, and diffusion training objectives throughout the book.

What would settle it

Verify that Theorem 1.5's proof of spectral decomposition can be reconstructed using only the material printed in this book's Chapter 1; if the proof is not recoverable without consulting Lu (2021a), the self-contained mathematical promise is falsified. As a second concrete test, implement the GMM EM derivation of Example 2.2 on a two-component toy dataset: if the described Q-function maximization does not converge to the known maximum-likelihood solution, the derivation chain is broken.

Watch

Extended reading notes

Core claim

On the author's terms, the central claim is pedagogical and conceptual: generative modeling is not a zoo of unrelated tricks but a small set of connected ideas organized under the distribution-transformation framing. Concretely, the book asserts that GANs, VAEs, normalizing flows, diffusion models, score-based models, and flow matching all learn a deterministic or stochastic map from a simple tractable prior (typically the standard Gaussian) to the target data distribution, and that their training objectives are unified through the ELBO, KL divergence, and conditional/marginal probability paths. It further claims that this organization is accurate and complete enough to guide a reader from foundational probability and linear algebra to state-of-the-art systems, including Stable Diffusion 3, and to advanced sampling algorithms such as DEIS, DPM-Solver, and DPM-Solver++.

Load-bearing premise

The book's promise of self-contained mathematical underpinnings rests on the correctness and availability of the author's earlier texts, since foundational proofs (spectral decomposition, SVD existence, Woodbury identity) are deferred to those references rather than proved in this book.

Editorial extensions

If this is right

  • A reader who follows the ELBO derivations through Chapters 2 and 3 can derive the VAE loss, the EM Q-function, and the DDPM noise-prediction objective as special cases of the same lower-bound argument.
  • Diffusion models are presented as three equivalent views—denoising prediction, score matching, and SDE/ODE solving—so tools developed for one view transfer to the others.
  • Flow matching and normalizing flows are shown as complementary frameworks that offer exact likelihoods and stable training, unified with diffusion under the same transport picture.
  • Modern architectures (U-Net, ControlNet, DiT, multimodal DiT) and guidance mechanisms (classifier and classifier-free) are presented as modular components of the distribution-transformation framework, not as isolated innovations.
  • Advanced samplers such as DPM-Solver and DPM-Solver++ drastically reduce the number of function evaluations while preserving fidelity, addressing the main computational bottleneck of iterative diffusion sampling.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The book leaves implicit that its distribution-transformation view is model-family-agnostic, so a sampler derived for diffusion (e.g., DPM-Solver) should in principle apply to any learned velocity field, including flow-matching models; the derivations give the reader the tools to test that claim directly.
  • The book's ELBO-first progression suggests a testable pedagogical hypothesis: teaching diffusion models after ELBO-based VAEs makes them easier to learn than an adversarial-first curriculum. This could be evaluated through a learning-outcome study comparing the two orderings.
  • The book's promise of self-contained mathematics is contingent on the availability and correctness of the author's earlier texts, since foundational proofs—spectral decomposition, SVD existence, and Woodbury's identity—are deferred to those references rather than proved here.
  • A practical reading strategy implied by the book is to verify Exercise 1.7 (SVD existence) independently before relying on later chapters, since the SVD theorem underlies the PCA/linear-autoencoder equivalence developed in Chapter 2.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. This manuscript is a textbook-length exposition of generative modeling, spanning mathematical preliminaries, latent-variable models (VAEs and GANs), diffusion models, flow matching, score-based methods, advanced samplers, and modern generative architectures. Its stated aim, in the preface, is to guide readers through the foundational principles, mathematical underpinnings, and practical architectures of generative AI. The book is organized as a sequence of chapters with embedded definitions, theorems, examples, and problem sets. The core derivations for the ELBO, EM, VAE losses, and diffusion objectives follow the standard literature, but the manuscript also contains several incorrect mathematical statements in its foundational chapters and exercises.

Significance. The book has genuine pedagogical value if corrected: the topic selection is comprehensive, the progression from ELBO/EM to VAEs and diffusion models is sensible, and many of the derivations are standard and clearly motivated. I credit the manuscript for attempting to connect classical variational inference to modern deep generative models and for including exercise problems that ask readers to prove key results. However, the book's central claim of providing the mathematical underpinnings of generative models is directly undermined by internal errors in Chapter 1 and Section 2.1.4, which are prerequisites for the rest of the text. Since these are errors in the manuscript itself rather than merely deferred citations, they are load-bearing for the pedagogical promise. The manuscript does not include machine-checked proofs or reproducible code, so its value rests on the correctness of its mathematical exposition.

major comments (5)
  1. [§2.1.4, Eq. (2.18)] The mean-field ELBO expansion is incorrect as written. The displayed formula writes the integrand as ∏_q q_{znq}(z_{nq}) ln pθ(zn,xn) − Σ_q q_{znq}(z_{nq}) ln q_{znq}(z_{nq}), integrated over dzn. Because dzn is the full latent vector, the entropy term must involve the full product ∏_q q_{znq}(z_{nq}) multiplying each ln q_{znq}(z_{nq}); equivalently the correct integrand is (∏_q q_{znq}) [ln pθ(zn,xn) − Σ_q ln q_{znq}(z_{nq})]. The printed sum without the product cannot arise from the preceding line and would mislead any reader trying to derive the mean-field coordinate ascent updates.
  2. [§1.2.5, Figure 1.6] The example G(x,y)=√(x²+y²) claims that directional derivatives exist at the origin for directions d=[a,b]^T with a,b≠0, but the defining limit from Definition 1.16 is lim_{μ→0} (|μ|√(a²+b²))/μ, whose one-sided limits are +√(a²+b²) and −√(a²+b²). The two-sided limit therefore does not exist for any nonzero direction d. This example does not illustrate the intended phenomenon; it actually contradicts Definition 1.16 and needs to be corrected.
  3. [Chapter 1 Problem 1.4, Eq. (1.36)] The univariate joint Gaussian likelihood contains N·Sx in the exponent, where Sx = Σ_{n=1}^N (x_n − ar x)². The correct decomposition is exp(−(1/(2σ²))[N(ar x−μ)² + Sx]); the second term should be Sx, not N·Sx. This is inconsistent with the multivariate formula displayed immediately below, which correctly uses tr(Σ^{−1}Sx) without an extra factor of N in the second exponential term.
  4. [Chapter 1 Problem 1.8] Parts (a) and (b) state E[x|y] = x when x is independent of y, and E[c|x] = x for a constant c. Both statements are wrong: the first should be E[x|y] = E[x] (a constant random variable), and the second should be E[c|x] = c. As printed, the exercise contradicts the unbiasedness property E[E[x|y]] = E[x] derived in Eq. (1.25) and teaches the opposite of the defining property of conditional expectation.
  5. [§1.2.5, Theorem 1.21] The quadratic approximation theorem is stated under the hypothesis that f is twice continuously differentiable, but the claim includes an O(∥y−x∥³) remainder term. Under only C² regularity, the sharp Taylor remainder is o(∥y−x∥²); the O(∥d∥³) bound requires third-order differentiability and is not a consequence of the stated hypothesis. The statement either needs a stronger hypothesis or a corrected remainder order.
minor comments (4)
  1. [Notation, Functions table] The notation table lists both ∥x∥=∥x∥1 and ∥x∥=∥x∥∞ for the same symbol ∥x∥, while Section 1.2.1 explicitly sets the convention ∥x∥=∥x∥₂. This shorthand should be made consistent or the table entries should be corrected.
  2. [§1.2.5, Theorem 1.20] The result labeled 'Linear approximation theorem' is actually a second-order Taylor expansion with Lagrange remainder. Renaming it, or clearly separating it from the first-order expansion in the following lines, would avoid confusion for readers.
  3. [§2.1.1, Eq. (2.2)] Equation (2.2) writes the marginal likelihood as a product over n of pθ(xn,zn)/pθ(zn|xn), but zn is unobserved, so this is not a form that can be evaluated or maximized directly. The identity is true for any fixed zn only after conditioning; as printed it may mislead readers about what constitutes the marginal likelihood.
  4. [General citation practice] Several foundational results are deferred to the author's own prior texts, including the spectral theorem (Theorem 1.5), the proof of SVD (Exercise 1.7), Woodbury's matrix identity, and the Gaussian moment generating function. For a book that promises mathematical underpinnings, at least the statements of these results should appear in the text, or standard published references should be cited alongside the self-citations.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation: the book is an exposition of standard results, with some proofs deferred to the author's own prior textbooks and several internal mathematical errors that are correctness issues, not circularity.

full rationale

I walked the derivation chain of the book's main technical content: the ELBO/EM derivations (Sections 2.1.2-2.1.4), the VAE objectives (Sections 2.2.4-2.2.6), and the diffusion/flow material summarized in later chapters. These are standard textbook derivations; none of the book's conclusions is defined in terms of another conclusion, and there are no fitted parameters being relabeled as predictions. The only self-referential feature is the deferral of foundational proofs to the author's own earlier books. For example, Theorem 1.5 states the spectral decomposition and gives 'Proof See Lu (2021a).' Exercise 1.7 asks the reader to prove SVD using that spectral theory, and Problem 1.6 refers to Woodbury's identity via 'see, for example, Lu (2021a).' These citations are self-citations, and they are load-bearing only in the sense that the book is not self-contained; they are not circular because the cited theorems are classical, externally checkable mathematical facts with stated assumptions that do not include the book's target results. The internal mathematical errors identified by the reviewer (the mean-field ELBO expansion in Eq. (2.18), the directional-derivative claim in Section 1.2.5, the joint Gaussian likelihood in Problem 1.4, and the conditional-expectation statements in Problem 1.8) are substantive correctness risks, but they are not cases of a derivation reducing to its inputs by construction. I therefore find no significant circularity. The score of 1 reflects the presence of repeated self-citations rather than any circular dependence.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The book makes no empirical claims and fits no data. It relies on standard mathematical background and on domain assumptions about data representation. No invented entities are introduced.

assumptions (4)
  • domain assumption Training data consists of i.i.d. samples x_1,...,x_N from an intractable data distribution p_data(x).
    Stated in Section 1.1 as the standard generative modeling setup; the whole book's math depends on this surrogate assumption.
  • domain assumption All objects of interest (images, video, audio, molecules) can be represented as vectors x in R^D.
    Stated at the start of Section 1.1; this vector representation underlies every derivation in the book.
  • domain assumption Neural networks can parameterize arbitrary conditional distributions (encoder/decoder), i.e., universal approximation is assumed.
    Used throughout Chapters 2 and 3, e.g., q_λ(z|x) and p_θ(x|z) are treated as flexible neural nets (Section 2.2.7).
  • standard math Standard probability, linear algebra, and calculus background (e.g., spectral theorem, SVD existence, Jensen's inequality).
    Invoked in Chapter 1 and proofs; the book cites Lu (2021a) for proofs rather than deriving them.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Generative Models: Principles, Architectures, and Applications." pith.science (2026). https://pith.science/paper/WSIAVDQL

@misc{pith2026260808101,
  author       = {Pith},
  title        = {Pith review of: Generative Models: Principles, Architectures, and Applications},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WSIAVDQL}},
  note         = {Machine review of arXiv:2608.08101}
}
read the original abstract

Generative AI has emerged as one of the most transformative forces in modern artificial intelligence, reshaping how we create, imagine, and interact with digital content. From photorealistic images to coherent text, from immersive videos to novel molecular structures, generative models now power applications that were once confined to science fiction. This book is designed to guide readers through the foundational principles, mathematical underpinnings, and practical architectures that underpin this revolution.

Figures

Figures reproduced from arXiv: 2608.08101 by the authors.

Figure 1.1
Figure 1.1. The first paradigm of generative models: Sequential decoding and sched￾uled parallel decoding. In the corresponding visualization, rows 1 and 3 illustrate latent mask inputs at each decoding iteration, while rows 2 and 4 present the generated samples produced at each step. Traditional autoregressive models initialize generation with entirely unknown latent codes (light gray) and fill in latent representations sequen… view at source ↗
Figure 1.2
Figure 1.2. The second paradigm of generative models [PITH_FULL_IMAGE:figures/full_fig_p019_1_2.png] view at source ↗
Figure 1.3
Figure 1.3. Comparison of the full and reduced SVD. White entries correspond to zero values, while blue entries denote values that are not necessarily zero. Exercise 1.7 (Proof of SVD). Based on the above discussion and spectral decom￾position theory, prove the existence of the singular value decomposition for an arbitrary real matrix X ∈ R N×D. 1.2.3 Probability Theory A random variable is a quantity that takes different value… view at source ↗
Figures from the paper (53 more)
Figure 1.4
Figure 1.4. Figure 1.4: Joint PDF for a 2-D distribution px,y (shown in shades of blue) and its marginal distributions px and py (shown as blue lines) [PITH_FULL_IMAGE:figures/full_fig_p034_1_4.png]
Figure 1.5
Figure 1.5. Figure 1.5: Density and contour plots (blue=low, yellow=high) of the multivariate Gaus￾sian distribution over the R 2 space for various values of the covariance/scale matrix with a zero-mean vector [PITH_FULL_IMAGE:figures/full_fig_p037_1_5.png]
Figure 1.6
Figure 1.6. Figure 1.6: Plot for the function G(x, y) = p x 2 + y 2, in which case any directional derivative for the direction d = [a, b] ⊤ with a ̸= 0 and b ̸= 0 at point [0, 0]⊤ exists. However, the partial derivatives at this point do not exist. An important subtlety in multivariate dif…
Figure 2.1
Figure 2.1. Figure 2.1: Conceptual illustration of generative models as distribution transformation problems. 2.1. Background: ELBO, EM, and VI This section presents the foundational principles of variational inference (VI or variational Bayesian inference), the evidence lower-bound (ELBO),…
Figure 2.2
Figure 2.2. Figure 2.2: Orthographic pro￾jection of a cylinder onto two perpendicular planes (the walls of a corner), clearly showing how a single 3D object can ap￾pear as two distinct 2D shapes when viewed from orthogonal directions. computational efficiency, yielding scalable inference so…
Figure 2.3
Figure 2.3. Figure 2.3: Graphical model represen￾tation of latent variable models. Green circles denote global latent variables, or￾ange circles represent observed (shaded) and latent (unshaded) variables, and plates indicate repeated structures across data points. n = 1..N z n x n θ served…
Figure 2.4
Figure 2.4. Figure 2.4: Illustration of the ELBO decomposition given by (2.6a) or (2.6b), which holds for any choice of distribution q(z). Be￾cause DKL[q ∥ p] ≥ 0, the quan￾tity F(q, θ) is a lower bound on the log-marginal likelihood func￾tion L(θ) = ln pθ(X ). KL(q k p) F(q, θ) L(θ) = ln p…
Figure 2.5
Figure 2.5. Figure 2.5: Description of an autoencoder. 2.2.1 Autoencoder In machine learning, an autoencoder (AE) performs dimensionality reduction by reducing the number of features that characterize a dataset (denoted as x in [PITH_FULL_IMAGE:figures/full_fig_p066_2_5.png]
Figure 2.6
Figure 2.6. Figure 2.6: Illustration of a VQAE. The “flame” symbol indicates that the corresponding component is learned from data. ▶ Optimization of VQAE. To train the encoder and decoder, we first define the overall optimization objective for the VQAE. Since the VQAE is fundamentally an A…
Figure 2.7
Figure 2.7. Figure 2.7: Once an AE has been trained, new data can be generated by decoding points randomly sampled from the latent space. The quality and relevance of the resulting gener￾ated data rely on the regularity of the latent space. objective. The standard reconstruction loss define…
Figure 2.8
Figure 2.8. Figure 2.8: Conceptual illustration of an irregular latent space that inhibits the use of AEs for generating new content. Adapted from Anwar (2021). ▶ From AE to VAE. As discussed above, to use the decoder of an AE for generative purposes, we must ensure that the latent space is…
Figure 2.9
Figure 2.9. Figure 2.9: Comparison between a standard AE and a VAE. prompt. Consequently, the generator fails to learn meaningful generative patterns and tends to produce blurry, repetitive, low-diversity images for identical text prompts, severely limiting generalization and generation fle…
Figure 2.10
Figure 2.10. Figure 2.10: Generation using a continuous and smooth latent space. training samples should decode to content that is semantically intermediate between the two original inputs, as it could plausibly be sampled by the decoder in either context. The core objective of a VAE is ther…
Figure 2.11
Figure 2.11. Figure 2.11: Graphical representation for VAE. Use the variational distribution qλ(z | x) to approximate the intractable posterior pθ(z | x). 2.2.4 VAE via Generative Intuition To introduce the VAE framework (Kingma and Welling, 2013; Rezende et al., 2014; Kingma and Welling, 20…
Figure 2.12
Figure 2.12. Figure 2.12: Graphical model representation of latent variable models under variational inference. Green circles denote prior variables, orange circles represent observed and latent variables, and plates indicate replicated structures. In the VAE, rather than computing a separat…
Figure 2.13
Figure 2.13. Figure 2.13: Comparison of standard VAE and hierarchical VAE, illustrating the hier￾archical VAE with T hierarchical latent levels. The generative process is formulated as a Markov chain, where each latent z (t) is generated only from the preceding latent variable z (t+1). The s…
Figure 2.14
Figure 2.14. Figure 2.14: Schematic illustration of the GAN framework. In this architecture, the dis￾criminator neural network Dλ(x) is trained to differentiate genuine training samples from synthetic samples generated by the generator network Gθ(z). The generator is optimized to maximize th…
Figure 2.15
Figure 2.15. Figure 2.15: Schematic illustration of a CGAN, in which a discriminator neural network is trained to distinguish between real samples from the training set, and synthetic samples produced by the generator network. The generator takes latent z and guided information y as inputs, …
Figure 2.16
Figure 2.16. Figure 2.16: Schematic illustra￾tion of the generator in an AC￾GAN, in which the decoder is used to enforce that the encoder keeps raw information as much as pos￾sible. Adapted from Lu and Yi (2022); Lu and Ding (2022). Conv transpose layers Dense CNN/ MLP concat. Norm Dense Lat…
Figure 2.17
Figure 2.17. Figure 2.17: Conceptual diagram illustrating the challenges of training GANs. It depicts a one-dimensional data space x containing a fixed but unknown data distribution pdata(x) and an initial generator distribution pG(x). The optimal discriminator D∗ (x) exhibits near￾zero grad…
Figure 2.18
Figure 2.18. Figure 2.18: Plots of − ln(v) and ln(1 − v), illustrating the sharply contrasting gradient behavior near v = 0 and v = 1. 0.0 0.2 0.4 0.6 0.8 1.0 v (Discriminator output) −6 −4 −2 0 2 4 6 Loss value − ln(v) ln(1 − v) − 1 2 ln(v) + 1 2 ln(1 − v) min θ n J G LSGAN(θ) = Eplatent(z)…
Figure 3.1
Figure 3.1. Figure 3.1: Comparison of standard VAE and hierarchical VAE, depicting the hierarchical VAE framework with T hierarchical latent variables. The generative process is modeled as a Markov chain, where each latent variable zt is generated only from the preceding latent variable zt+…
Figure 3.2
Figure 3.2. Figure 3.2: Conceptual illustration of the comparison between VAE and diffusion models. (A1) Dimension consistency. Each step of the forward and backward processes adopts identical input and output dimensions, such that all latent variables zt share the same dimensionality as th…
Figure 3.3
Figure 3.3. Figure 3.3: Graphical representation of the diffusion model. The original data x = x0 is shown by the shaded node, since it is an observed variable, whereas the noise-corrupted data z1, . . . , zT are considered to be latent variables. The noise process is defined by the forward…
Figure 3.4
Figure 3.4. Figure 3.4: Diffusion step for a set of images with t = 0, 10, 30, 50, 500, 999 steps. In the process, we adopt a linear variance schedule for the forward diffusion process (i.e., βt = β1+ t−1 T −1 (βT −β1), for t = 1, 2, . . . , T and T = 1000), following the protocol establish…
Figure 3.5
Figure 3.5. Figure 3.5: Illustration of diffusion kernel. In practice, noise is not added incrementally but applied in a single step corresponding to timestep t. We observe that each intermediate distribution takes a simple closed-form Gaussian form that can be directly sampled. This proper…
Figure 3.6
Figure 3.6. Figure 3.6: Illustration of the reverse distribution q(zt−1 | zt) derived via Bayes’ theo￾rem (3.16) for scalar random variables. Upper. The blue curve on the right shows the marginal distribution q(zt−1) modeled as a mixture of three Gaussians, while the left panel displays the…
Figure 3.7
Figure 3.7. Figure 3.7: Illustration of the binomial diffusion model. Green = pixel on (1), light = pixel off (0), gray = corrupted / uncertain. Each forward step follows a Bernoulli mixture: zt = (1 − βt)zt−1 + βt · coin flip. The reverse chain is trained to reverse one binomial flip step …
Figure 3.8
Figure 3.8. Figure 3.8: Conceptual illustration of sampling from a DDPM. 3.2.3 Latent Diffusion Models (LDMs) We have observed that diffusion models can be computationally demanding, as they se￾quentially reverse a noise process that may involve hundreds or even thousands of steps. Song et …
Figure 3.9
Figure 3.9. Figure 3.9: Conceptual illustration of the difference between standard DDPMs and LDMs. LDMs represent a computationally efficient variant of DDPMs. Instead of performing diffu￾sion directly on high-resolution data, LDMs conduct diffusion on compressed latent tokens and use a dec…
Figure 3.10
Figure 3.10. Figure 3.10: Illustration of latent diffusion models. Encoding the original data x0 into a latent vector z0; and diffusion is applied to the latent vector in a LDM. Specifically, LDMs move the entire diffusion process from high-resolution pixel space into a compressed latent spa…
Figure 3.11
Figure 3.11. Figure 3.11: Improving the DDPM with learning the variance and alternative noise sched￾ules [PITH_FULL_IMAGE:figures/full_fig_p133_3_11.png]
Figure 3.12
Figure 3.12. Figure 3.12: Let p(y) be an arbitrary probability distribution. Then the score function of p is de￾fined as ∇ ln p(y), i.e., as the gra￾dient of the log-likelihood of p with respect to x. The score has an intu￾itive meaning: ∇ ln p(x) points in the direction of steepest ascent o…
Figure 4.1
Figure 4.1. Figure 4.1: The flow-based models blueprint [PITH_FULL_IMAGE:figures/full_fig_p156_4_1.png]
Figure 4.2
Figure 4.2. Figure 4.2: Conceptual illustration of a single layer of the RealNVP normaliz￾ing flow model. Network Γ1 evaluates the function exp(γθ (z1)), while network B1 computes βθ (z1). The resulting output vector is then specified by (4.9). Following the recovery of z1, we compute the n…
Figure 4.3
Figure 4.3. Figure 4.3: Conceptual illustration of the full RealNVP framework. By stacking two layers structured as in [PITH_FULL_IMAGE:figures/full_fig_p162_4_3.png]
Figure 4.4
Figure 4.4. Figure 4.4: Conceptual illustration of two alternative architectures for autoregressive nor￾malizing flows. While the masked autoregressive flow enables efficient likelihood evaluation, its counterpart—the inverse autoregressive flow—supports efficient sampling. ▶ Inverse autore…
Figure 4.5
Figure 4.5. Figure 4.5: A flow ψt : R D → R D is induced by a time-dependent velocity field vt : R D → R D, whose instantaneous spatial movements at every coordinate point are visualized via the arrow field in the illustration (with D = 2 in this setup). We demonstrate the flow evolution ac…
Figure 4.6
Figure 4.6. Figure 4.6: We gradually interpolate from random noise to target images along a Gaussian￾conditioned probability path. 4.2.2 Conditional and Marginal Probability Path In the previous subsection, we introduce generative flow models as generative frameworks parameterized by a neur…
Figure 4.7
Figure 4.7. Figure 4.7: Simulation of ODE dynamics with a conditional vector field for conditional probability path modeling. To derive the marginal vector field, we first define the conditional vector field. Given a data point x ∈ R D, let v θ ∗ t (· | x) denote a conditional vector field …
Figure 5.1
Figure 5.1. Figure 5.1: Particles evolving under the Langevin dynamics given by (5.4), with p(x) taken to be a Gaussian mixture with two modes. 5.1. Score Matching However, the loss function defined in (5.3) suffers from a critical practical limitation: neither the original loss nor its dir…
Figure 5.2
Figure 5.2. Figure 5.2: Presents four distinct sampling trajectories generated by Langevin dynamics for a Gaussian mixture model, all initialized from a starting point between two modes. The left subfigure visualizes these trajectories over a three-dimensional contour landscape of the targe…
Figure 5.3
Figure 5.3. Figure 5.3: Shown are one-dimensional OU processes (5.18) with a = 0.2, where σ in￾creases from left to right. The σ = 0 case yields a deterministic flow: smooth trajectories that decay monotonically to the origin as t → ∞. In contrast, σ > 0 introduces random￾ness, with sample …
Figure 6.1
Figure 6.1. Figure 6.1: Relative changes of ϵ θ t (xt) with respect to normalized timestep t are relatively small, especially when t < 0.7. 1.0 0.8 0.6 0.4 0.2 Normalized timestep t 0.0 0.5 1.0 1.5 2.0 2.5 3.0 k  θ t− ∆t(xt− ∆t) −  θ t (xt)k 2 Relative change of  θ t which completes the …
Figure 7.1
Figure 7.1. Figure 7.1: Conceptual illustration of CLIP. Whereas standard image models jointly train an image feature extractor and a linear classifier to predict predefined class labels, CLIP jointly trains an image encoder and a text encoder to predict the correct pairings within a batch …
Figure 7.2
Figure 7.2. Figure 7.2: The original U-Net architecture (example for 16 × 16 pixels in the lowest resolution). Each blue box corresponds to a multi-channel feature map. The number of channels is denoted on top of the box. The h-w-size is provided at the lower-left edge of the box. White box…
Figure 7.3
Figure 7.3. Figure 7.3: Three components in generative model architectures. The middle component can generate both a low-resolution image that can be recognized by human (e.g., in Imagen (Saharia et al., 2022a)) or a noise image that cannot be recognized (e.g., in SD v1.x series (Rombach et…
Figure 7.4
Figure 7.4. Figure 7.4: Conceptual illustration of a simplified U-Net architecture used in the SD v1.x series, adapted from the original U-Net framework in [PITH_FULL_IMAGE:figures/full_fig_p256_7_4.png]
Figure 7.5
Figure 7.5. Figure 7.5: Using human pose maps to control image generation, producing character images that match the pose diagrams. ControlNet (Zhang et al., 2023), introduced in this subsection, addresses this limitation by enabling visual conditional control. Built upon the pre-trained St…
Figure 7.6
Figure 7.6. Figure 7.6: Left (a): A standard Stable Diffusion architecture. Right (b): Additional architecture in ControlNet. The “snowflake” symbol indicates that the corresponding com￾ponent is frozen during training. Adapted from Zhang et al. (2023). Principle of ControlNet We further ex…
Figure 7.7
Figure 7.7. Figure 7.7: Detailed view of a ControlNet block. Left (a): a standard Stable Diffusion block. Right (b): its mirrored copy in ControlNet with an appended 1×1 convolution. The “snowflake” symbol indicates that the corresponding component is frozen during training. Adapted from Zh…
Figure 7.8
Figure 7.8. Figure 7.8: An overview of the diffusion transformer architecture. Elementwise addition is denoted by ⊕. The input latent representation is decomposed into patches and processed by several (here L) DiTBlocks. Adapted from Peebles and Xie (2023); Chen et al. (2024). The diffusion…
Figure 7.9
Figure 7.9. Figure 7.9: An overview of the multimodal diffusion transformer (MM-DiT) architecture. Concatenation is denoted by ⊗; split is indicated by ⊖; and elementwise addition is denoted by ⊕. The input latent is decomposed into patches and processed by several (here d) MM￾DiT Blocks. A…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 12 canonical work pages

  1. [8]

    An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929,

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929,

  2. [10]

    Query-key normalization for transformers

    Alex Henry, Prudhvi Raj Dachapally, Shubham Shantaram Pawar, and Yuxuan Chen. Query-key normalization for transformers. InFindings of the Association for Compu- tational Linguistics: EMNLP 2020, pages 4246–4253,

  3. [13]

    If you use this software, please cite it as below

    URL https://doi.org/10.5281/zenodo.5143773. If you use this software, please cite it as below. 264 BIBLIOGRAPHY Arieh Iserles.A first course in the numerical analysis of differential equations. Number

  4. [14]

    Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980,

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980,

  5. [15]

    Auto-encoding variational Bayes.arXiv preprint arXiv:1312.6114,

    Diederik P Kingma and Max Welling. Auto-encoding variational Bayes.arXiv preprint arXiv:1312.6114,

  6. [16]

    Flux.1 kontext: Flow matching for in-context image generation and editing in latent space.arXiv preprint arXiv:2506.15742,

    Black Forest Labs, Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dockhorn, Jack English, Zion English, Patrick Esser, et al. Flux.1 kontext: Flow matching for in-context image generation and editing in latent space.arXiv preprint arXiv:2506.15742,

  7. [17]

    Flow matching for generative modeling.arXiv preprint arXiv:2210.02747,

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling.arXiv preprint arXiv:2210.02747,

  8. [18]

    Flow matching guide and code.arXiv preprint arXiv:2412.06264,

    Yaron Lipman, Marton Havasi, Peter Holderrieth, Neta Shaul, Matt Le, Brian Karrer, Ricky TQ Chen, David Lopez-Paz, Heli Ben-Hamu, and Itai Gat. Flow matching guide and code.arXiv preprint arXiv:2412.06264,

Show all 39 references
  1. [19]

    Flow straight and fast: Learning to generate and transfer data with rectified flow.arXiv preprint arXiv:2209.03003,

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow.arXiv preprint arXiv:2209.03003,

  2. [21]

    Practical topics in optimization.arXiv preprint arXiv:2503.05882,

    Jun Lu. Practical topics in optimization.arXiv preprint arXiv:2503.05882,

  3. [22]

    A first course in sparse optimization.arXiv preprint arXiv:2601.06173,

    Jun Lu. A first course in sparse optimization.arXiv preprint arXiv:2601.06173,

  4. [24]

    Autoencoding conditional GAN for portfolio allocation diversification

    266 BIBLIOGRAPHY Jun Lu and Shao Yi. Autoencoding conditional GAN for portfolio allocation diversification. arXiv preprint arXiv:2207.05701,

  5. [25]

    Understanding diffusion models: A unified perspective.arXiv preprint arXiv:2208.11970,

    Calvin Luo. Understanding diffusion models: A unified perspective.arXiv preprint arXiv:2208.11970,

  6. [29]

    SDXL: Improving latent diffusion models for high- resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M¨ uller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion models for high- resolution image synthesis. InInternational Conference on Learning Representations, volume 2024, pages 1862–1874,

  7. [30]

    Movie Gen: A cast of media foundation models.arXiv preprint arXiv:2410.13720,

    Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih-Yao Ma, Ching-Yao Chuang, et al. Movie Gen: A cast of media foundation models.arXiv preprint arXiv:2410.13720,

  8. [31]

    Hierarchical text-conditional image generation with CLIP latents.arXiv preprint arXiv:2204.06125, 1(2):3,

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with CLIP latents.arXiv preprint arXiv:2204.06125, 1(2):3,

  9. [32]

    Kandinsky: an improved text-to-image synthesis with image prior and latent diffusion

    Anton Razzhigaev, Arseniy Shakhmatov, Anastasia Maltseva, Vladimir Arkhipkin, Igor Pavlov, Ilya Ryabov, Angelina Kuts, Alexander Panchenko, Andrey Kuznetsov, and De- nis Dimitrov. Kandinsky: an improved text-to-image synthesis with image prior and latent diffusion. InProceedin...

  10. [33]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020a. Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution.Advances in neural information processing systems, 32,

  11. [35]

    Nextstep-1: Toward autoregressive image generation with continuous tokens at scale.arXiv preprint arXiv:2508.10711,

    NextStep Team, Chunrui Han, Guopeng Li, Jingwei Wu, Quan Sun, Yan Cai, Yuang Peng, Zheng Ge, Deyu Zhou, Haomiao Tang, et al. Nextstep-1: Toward autoregressive image generation with continuous tokens at scale.arXiv preprint arXiv:2508.10711,

  12. [38]

    SANA: Efficient high-resolution image synthesis with linear diffusion transformers.arXiv preprint arXiv:2410.10629,

    Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Haotian Tang, Yujun Lin, Zhekai Zhang, Muyang Li, Ligeng Zhu, Yao Lu, et al. SANA: Efficient high-resolution image synthesis with linear diffusion transformers.arXiv preprint arXiv:2410.10629,

  13. [39]

    Fast sampling of diffusion models with exponential integrator.arXiv preprint arXiv:2204.13902,

    Qinsheng Zhang and Yongxin Chen. Fast sampling of diffusion models with exponential integrator.arXiv preprint arXiv:2204.13902,

  14. [1992]

    Automated variational inference in probabilistic programming.arXiv preprint arXiv:1301.1299,

    David Wingate and Theophane Weber. Automated variational inference in probabilistic programming.arXiv preprint arXiv:1301.1299,

  15. [1995]

    Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598,

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598,

  16. [1998]

    GLIDE: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741,

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Mc- Grew, Ilya Sutskever, and Mark Chen. GLIDE: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741,

  17. [2003]

    Estimating or propagating gradients through stochastic neurons for conditional computation.arXiv preprint arXiv:1308.3432,

    261 BIBLIOGRAPHY Yoshua Bengio, Nicholas L´ eonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation.arXiv preprint arXiv:1308.3432,

  18. [2008]

    Learning to encode text as human-readable summaries using generative adversarial networks

    Yaushian Wang and Hung-Yi Lee. Learning to encode text as human-readable summaries using generative adversarial networks. InProceedings of the 2018 conference on empirical methods in natural language processing, pages 4187–4195,

  19. [2009]

    ediff-i: Text-to-image diffusion models with an ensemble of expert denoisers.arXiv preprint arXiv:2211.01324,

    Yogesh Balaji, Seungjun Nah, Xun Huang, Arash Vahdat, Jiaming Song, Qinsheng Zhang, Karsten Kreis, Miika Aittala, Timo Aila, Samuli Laine, et al. ediff-i: Text-to-image diffusion models with an ensemble of expert denoisers.arXiv preprint arXiv:2211.01324,

  20. [2013]

    W¨ urstchen: An efficient architecture for large-scale text-to-image diffusion models

    Pablo Pernias, Dominic Rampas, Mats L Richter, Christopher Pal, and Marc Aubreville. W¨ urstchen: An efficient architecture for large-scale text-to-image diffusion models. In International Conference on Learning Representations, volume 2024, pages 25097–25109,

  21. [2014]

    Score-based generative modeling with critically-damped langevin diffusion.arXiv preprint arXiv:2112.07068,

    Tim Dockhorn, Arash Vahdat, and Karsten Kreis. Score-based generative modeling with critically-damped langevin diffusion.arXiv preprint arXiv:2112.07068,

  22. [2016]

    An introduction to flow matching and diffusion models

    Peter Holderrieth and Ezra Erives. An introduction to flow matching and diffusion models. arXiv preprint arXiv:2506.02070,

  23. [2018]

    Conditional generative adversarial nets.arXiv preprint arXiv:1411.1784,

    Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets.arXiv preprint arXiv:1411.1784,

  24. [2019]

    Sliced score matching: A scalable approach to density and score estimation

    Yang Song, Sahaj Garg, Jiaxin Shi, and Stefano Ermon. Sliced score matching: A scalable approach to density and score estimation. InUncertainty in artificial intelligence, pages 574–584. PMLR, 2020b. Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano ...

  25. [2020]

    Ffjord: Free-form continuous dynamics for scalable reversible generative models.arXiv preprint arXiv:1810.01367,

    Will Grathwohl, Ricky TQ Chen, Jesse Bettencourt, Ilya Sutskever, and David Duvenaud. Ffjord: Free-form continuous dynamics for scalable reversible generative models.arXiv preprint arXiv:1810.01367,

  26. [2021]

    Nice: Non-linear independent compo- nents estimation.arXiv preprint arXiv:1410.8516,

    262 BIBLIOGRAPHY Laurent Dinh, David Krueger, and Yoshua Bengio. Nice: Non-linear independent compo- nents estimation.arXiv preprint arXiv:1410.8516,

  27. [2022]

    PixArt-alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis

    Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li. PixArt-alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis. InInternational conference on learning representation...

  28. [2023]

    Unsupervised speech representation learning using wavenet autoencoders.IEEE/ACM transactions on audio, speech, and language processing, 27(12):2041–2053,

    Jan Chorowski, Ron J Weiss, Samy Bengio, and A¨ aron Van Den Oord. Unsupervised speech representation learning using wavenet autoencoders.IEEE/ACM transactions on audio, speech, and language processing, 27(12):2041–2053,

  29. [2024]

    Wavegrad: Estimating gradients for waveform generation.arXiv preprint arXiv:2009.00713,

    Nanxin Chen, Yu Zhang, Heiga Zen, Ron J Weiss, Mohammad Norouzi, and William Chan. Wavegrad: Estimating gradients for waveform generation.arXiv preprint arXiv:2009.00713,

  30. [2025]

    Numerical matrix decomposition.arXiv preprint arXiv:2107.02579, 2021a

    Jun Lu. Numerical matrix decomposition.arXiv preprint arXiv:2107.02579, 2021a. Jun Lu. A rigorous introduction to linear models.Elive Press, arXiv preprint arXiv:2105.04240, 2021b. Jun Lu. A survey on Bayesian inference for Gaussian mixture model.arXiv preprint arXiv:2108.1175...

  31. [2026]

    A hybrid approach on conditional GAN for portfolio analysis

    Jun Lu and Danny Ding. A hybrid approach on conditional GAN for portfolio analysis. In IoT Based Control Networks and Intelligent Systems: Proceedings of 3rd ICICNIS 2022, pages 849–868. Springer,

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.