Pith. sign in

REVIEW 4 major objections 6 minor 54 references

Guiding Time-Varying Generative Models with Natural Gradients on Exponential Family Manifold

T0 review · 4 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Time-varying generative models can be trained by projecting their evolution onto an exponential-family manifold and matching the projected change to the natural gradient of the KL divergence, avoiding MCMC and likelihood evaluation.

desk verdict A genuinely new idea with useful closed-form particle updates, but the central projection-gap assumption is unproven and the proof has fixable typos; it deserves peer review. read the letter →

arxiv 2502.07650 v2 pith:HTSPMDU7 submitted 2025-02-11 stat.ML cs.LG

Song Liu , Leyang Wang , Yakun Wang This is my paper · ORCID
classification stat.MLcs.LG MSC 62B1068T07
keywords naturalgradientdescentexponentialfamilytimescorematchingtime-varyinggenerativemodelsdrift-basedkernelmethodsneuraltangentparticle
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes a way to train time-varying generative models by borrowing the natural-gradient machinery of parametric statistics. The authors show that the instantaneous change of a generative model's output distribution can be projected onto an exponential-family manifold using time score matching, producing a parameter-space velocity. They then train the generative model so this projected velocity equals the natural gradient of the KL divergence between the target distribution and the exponential-family model, which drives the generative samples toward the target without likelihood evaluation or MCMC. For drift-based models the projection has a closed-form limit, and the resulting kernel and neural-tangent particle updates are explicit. The claim matters because it connects modern generative-model training to classical statistical optimisation and offers a training signal that does not require adversarial balancing or diffusion bridges.

What carries the argument

The load-bearing object is the time-score projection $\delta_{t_0}(w)$: the natural-parameter velocity of the exponential-family curve that best matches the instantaneous change of the generative distribution in the time-score-matching sense. For drift models the machine reduces to the identity $\lim_{\sigma\to 0}\delta_{t_0}(w)=F_{t_0}^{-1}E[\nabla T(X_{t_0})h(X_{t_0};w)]$, which turns natural-gradient matching into a least-squares problem over drift functions. The optimal RKHS drift (Theorem 4.3) and its neural-tangent analogue provide closed-form particle update rules. The exponential-family manifold $\mathcal{M}(T)$ is the guiding frame: its sufficient statistic $T$ determines what the generative model is steered toward.

What would settle it

Run the particle algorithm with $T(x)=[x,x^2]$ on a target that is a mixture of two well-separated Gaussians, and record the time-score projection residual alongside the maximum mean discrepancy (MMD) between generated particles and target samples. The paper's own Figure 3 shows particles converging to the best Gaussian fit rather than splitting between modes; a quantitative version would show that the residual stays large while the particles keep moving, which would mean the algorithm is matching the projection's natural gradient, not the target's.

Watch

Extended reading notes

Core claim

The central claim is that a time-varying generative model $X_t = g(Z,t;w)$ can be guided by the natural gradient of $\mathrm{KL}[p,q_\theta]$ on an exponential-family manifold $\mathcal{M}(T)$, without ever evaluating the likelihood of $q_\theta$. The paper proves that the time score of $g$ can be projected onto $\mathcal{M}(T)$ by minimising a time-score-matching objective, giving the projected velocity $\delta_{t_0}(w) = -\left(\int \lambda_{t_0}(t)\mathrm{Cov}[T(X_t)]\,dt\right)^{-1}\int \partial_t\lambda_{t_0}(t)\,E[T(X_t)]\,dt$ (Theorem 3.1). For drift models $g(X_{t_0},t;w)=X_{t_0}+(t-t_0)h(X_{t_0};w)$, the limit $\sigma\to 0$ yields the closed form $\delta_{t_0}(w)=F_{t_0}^{-1}E[\nabla T(X_{t_0})h(X_{t_0};w)]$, where $F_{t_0}=\mathrm{Cov}[T(X_{t_0})]$ is the Fisher information. Matching this to the Euclidean gradient $\nabla L(\theta_{t_0})$ under the $F^{-1}$ norm gives an objective whose RKHS optimum is $h_{w^*}(x)=E[\nabla T(X)\nabla\nabla k(X,x)]^{\top}\Gamma^{-1}\nabla L(\theta)$, with a neural-tangent analogue; these are closed-form particle updates. The paper calls the general algorithm implicit NGD (iNGD) and the particle versions KiNG and ntKiNG, and validates them on toy, graphical-model, domain-adaptation, and denoising tasks.

Load-bearing premise

The whole scheme rests on the assumption that the generative model's output distribution stays close to the chosen exponential-family distribution throughout training; if it does not, the 'natural gradient' being matched is the gradient of a different, projected model, and the guidance can push samples the wrong way.

Editorial extensions

If this is right

  • If the central claim is right, any time-varying generative model—not just drift models—can in principle be trained by matching its projected change to a natural gradient, giving an alternative to adversarial training and to manually constructed diffusion bridges.
  • For drift models the particle updates are closed form, so no generative-network fitting is needed at each step; KiNG and ntKiNG are simple to implement and stay stable at step size 1 in the reported experiments.
  • The guiding manifold is chosen by the user through $T$; informative sufficient statistics (RBF features, graphical-model edges, pretrained energy-based-model features) accelerate convergence and can recover structure such as a graphical model in far fewer iterations.
  • Because the algorithm transports particles to match $\mathrm{KL}[p,q_t]$, it can be used for covariate-shift correction by learning the inverse transport map, and for denoising with a pretrained energy-based model as $T$.
  • Algorithm 1 applies to non-particle generative models as well, so the construction is claimed to be more general than particle-based gradient flows.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the paper leaves implicit is that the projection residual $\mathbb{E}[(s_t(X_t)-\langle\delta,T(X_t)-\mathbb{E}[T]\rangle)^2]$ can serve as a diagnostic for whether the exponential-family guide is trustworthy; a large residual means the matched $\delta$ is not the natural gradient of the actual generative distribution, so one could adaptively enrich $T$ until the residual drops.
  • Because the projection comes from time score matching, the method is a score-based parametric flow; a testable extension is to replace the Gaussian smoothing kernel with other kernels and check whether the $\sigma\to 0$ limit for drift models is the unique unbiased projection or an artifact of the kernel.
  • The closed-form updates define a new class of kernel flows guided by a parametric model; comparing KiNG and ntKiNG with other kernelized gradient-flow samplers on the same targets would clarify whether the benefit comes from the natural-gradient metric or from the exponential-family projection itself.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a framework, implicit NGD (iNGD), for training time-varying generative models by aligning their time-score-matching projection onto an exponential-family manifold with the natural gradient descent update of KL[p, q_theta]. The authors derive a projection formula for general generative models, specialize it to drift-based models where a closed-form limit as the time kernel width tends to zero is obtained, and then derive kernel and neural-tangent-kernel particle-update rules that avoid MCMC. Experiments on toy distributions, graphical model recovery, covariate shift, and image denoising are reported. The central claim is that this projection-matching procedure lets a flexible implicit generative model be guided by a parametric probabilistic model, inheriting the efficiency of NGD while retaining the expressiveness of the generative model.

Significance. If the central assumption were rigorously justified, the paper would offer a practical and conceptually clean way to equip implicit generative models with a natural-gradient-based training signal, with closed-form particle updates for drift models and no need for MCMC or an explicit likelihood. The main strengths are the closed-form expressions in Theorems 4.2 and 4.3, the modular treatment of the sufficient statistic T, and the breadth of experiments, including a domain-adaptation application. The principal weakness is that the correctness of Algorithm 1 depends on an unproven assumption that the generator's output distribution remains exactly on the exponential-family manifold; without a residual bound or diagnostic, the matched quantity can be a self-referential functional of the generator rather than a natural gradient. The proof of Theorem 4.3 also contains an algebraic inconsistency that must be fixed, although the final formula is correct.

major comments (4)
  1. [Section 3.3, Eq. (6), Algorithm 1] The matching target \nabla_N L(\theta_{t_0}) is estimated using samples from g(Z, t; w_t), but \theta_{t_0} is the parameter of the exponential-family projection, not of the generator. The substitution is justified only by the assumption that q_{g_0} \in M(T) and that the trajectory q_{g_t} is "precisely traced" by the projected updates. When q_{g_t} leaves M(T), E_{q_{g_t}}[T] and Cov_{q_{g_t}}[T] are moments of the generator's distribution, not of any exponential-family q_\theta, so the quantity matched to \delta(w) is not the natural gradient of KL[p, q_\theta] for any well-defined parametric model. The manuscript gives no bound on the projection residual, no consistency argument, and no user-facing diagnostic. Because this assumption underlies the correctness of Algorithm 1, it must either be proved in a meaningful special case or the method must be reframed as a heuristic that guides particles by approximate projections, with the gap made explicit.
  2. [Appendix C, Eqs. (17) and (18)] The proof of Theorem 4.3 expands Eq. (8) with a linear term -2 \nabla^\top L(\theta_{t_0}) F^{-1}_{t_0} E[T(X_{t_0}) h_w(X_{t_0})], but Eq. (7) requires E[\nabla T(X_{t_0}) h_w(X_{t_0})], where \nabla T is the Jacobian of the sufficient statistic. The same missing Jacobian appears in Eq. (18). The final formula (9) is correct and does match the objective, but the proof as printed is inconsistent with the theorem statement and must be corrected.
  3. [Section 4.1, Theorem 4.3] The matrices F_{t_0} and \Gamma in Eq. (9) are defined through Cov[T(X_{t_0})] and expectations over X_{t_0} \sim q_{g_t}. If the generator samples are not from an exponential-family member, these objects are not Fisher information matrices or natural-gradient covariances of any q_\theta, and the Woodbury/least-squares derivation does not amount to a natural-gradient step of a parametric model. The failure mode in Figure 3, where particles stop at the best Gaussian fit for a bimodal target, is exactly what one expects when the projection gap, rather than NGD, controls the motion. Please state this limitation in the main text and provide a diagnostic that would tell a user when the projection gap is large, for example an estimate of the TSM projection residual or an MMD between q_{g_t} and its exponential-family projection.
  4. [Theorem 3.1, Eq. (5) and Appendix A] The invertibility of C = \int \lambda_{t_0}(t) Cov[T(X_t)] dt is asserted in Appendix A to be equivalent to the invertibility of the Fisher information. This equivalence holds only if X_t is distributed according to an exponential-family member with sufficient statistic T; for a general generative model q_{g_t}, no such identification exists. The invertibility condition should be stated as a separate assumption on the generator's distribution, not derived from the Fisher information of a model that the generator does not follow.
minor comments (6)
  1. [Appendix B, Lemma B.1] The lemma defines \lambda_\sigma with a normalizing constant 1/\sqrt{2\pi\sigma^2}, whereas the main text in Eq. (3) defines \lambda_{t_0}(t) = \exp(-(t-t_0)^2/\sigma^2) without normalization. The normalization cancels in \delta_{t_0}, but the notation should be made consistent to avoid confusion in the proof of Theorem 4.2.
  2. [Appendix D] The Stein exponential family subsection contains an incomplete citation, "More discussions on the Stein operator could be found in []", and Eq. (22) appears to drop the expectation over p when deriving \nabla_\theta L(\theta) = F^{-1} E_{q_\theta}[S_p f(x)]. Please complete the reference and the derivation.
  3. [Section 4.2, Remark] The diagonalized scalar-kernel approximation [K(X_{t_0}, X'_{t_0})]_{l,m} = k(X_{t_0}, X'_{t_0}) if l=m and 0 otherwise is introduced without justification; please add a comment on when this approximation is expected to be accurate and what it changes in the interpretation of Eq. (11).
  4. [Figure 3 caption] The caption says "the initial distribution q_0 is plotted on the left ... while the target distribution p is plotted on the right", but both panels have their own left and right sides; please clarify which panel and which side are meant.
  5. [Section 5.1] The comparison with reverse-KL Wasserstein gradient flow and MMD flow uses MMD as the evaluation metric, which is the same objective that MMD flow directly minimizes; this potential bias should be discussed when interpreting the ranking in Figure 5.
  6. [Section 2.4] The paper relies on Williams et al. (2025) for the TSM projection identity, but the provenance is stated only briefly; please make explicit in Section 2.4 which results are imported and which are new to this paper.

Circularity Check

1 steps flagged · score 4.0 of 10

The central projection identity is imported from the authors' own prior work without a self-contained proof, while the §3.3 'precisely traced' assumption is an unverified invariant rather than a constructional circularity.

  1. self citation load bearing [Appendix A; used in Section 3.2, Theorem 3.1 and Eq. (5)]
    "Theorem A.1. (Theorem 4.1 in [Williams et al., 2025]) Equation (4) can be rewritten as the following form L(δ) = ∫ λ_{t0}(t) E[⟨δ, T(Xt) − E[T (X′t)]⟩²] dt + 2 ∫ ∂tλ_{t0}(t) E [⟨δ, T(Xt)⟩] dt + const. (12) Notice that Equation (12) is a quadratic minimisation problem."

    The paper's central projection δ_{t0}(w), Eq. (5), is the quantity matched to NGD in Eq. (6), and it is the minimizer of Eq. (4). However, the equivalence (4) ↔ (12) that makes this minimizer computable is not derived in this paper: Appendix A simply restates the result as 'Theorem 4.1 in [Williams et al., 2025]' and then differentiates the already-rewritten quadratic (12), so the core TSM-to-projection step is not self-contained. Williams et al. shares two authors with the present paper (Song Liu and Leyang Wang), making this a load-bearing citation to the authors' own prior work rather than an independent derivation. The later drift limit (Theorem 4.2) and the RKHS/NTK updates inherit this unproved equivalence.

full rationale

The paper contains one load-bearing self-citation: the TSM projection identity (Eq. 4 rewritten as Eq. 12) is taken from Williams et al. (2025) with overlapping authorship, and the appendix proves only the minimizer of the already-rewritten quadratic, not the equivalence itself. This raises the circularity score to 4. The §3.3 assumption that q_g0 ∈ M(T) and that the trajectory is 'precisely traced' by projected updates is an acknowledged idealization; it is a correctness gap and an unverified invariant, but it is not by itself an equation-level reduction, so it does not push the score to 6 or above. The paper's closed-form particle updates (Theorems 4.2 and 4.3) are derived from the projection formula and would stand if the Williams result were independently verified, so the central claim retains independent content beyond the citation chain. The authors' own limitations section admits that the benefits of parametric-space guidance 'remain to be clarified,' which is consistent with treating the §3.3 assumption as a limitation rather than as a manufactured circularity.

Assumptions & free parameters 6 free parameters · 7 assumptions · 0 invented entities

The central derivation relies on two unproved structural assumptions (expressive T, trajectory traced by projection) and on unstated hyperparameters in experiments. The free parameters are mostly tuning choices; the largest fitted input is the pretrained EBM used as T in Section 5.4.

free parameters (6)
  • time kernel width σ = chosen by hand, e.g. 0.1
    In the TSM projection (Eq. 4, Section 3.2), σ controls the local-regression bias. The paper notes that nonzero σ introduces bias and says reasonable σ choices work well, but no selection rule is given.
  • RKHS ridge regularization λ = not specified in the main text
    Regularization term in Eq. (8) and (10) is required for the closed-form updates; experiments do not report the λ values used, so the particle behavior depends on an unreported tuning choice.
  • RBF kernel bandwidth = median pairwise distance of samples
    All RBF kernels and MMD metrics use the median heuristic (Appendix F), a data-dependent choice that affects Γ and the drift.
  • step size ϵ = 1 (Sections 5.1, 5.2) or 0.1 (Section 5.3)
    The Euler update X_{t+1}=X_t+ϵh(X_t) and Algorithm 1 step depend on a user-chosen step size; no adaptive scheme is given.
  • RBF basis count/landmarks for T = not specified
    The expressive sufficient statistic T in Sections 5.1-5.3 is an RBF feature vector; the number and choice of basis functions are not stated, and dim(T) controls the size of the inverted matrix Γ.
  • pretrained EBM feature extractor = weights trained by denoising score matching on target samples
    In Section 5.4, T is the penultimate layer of a deep EBM trained on Y, so the sufficient statistic itself is fitted to the target data; the method's success in denoising depends on this fitted representation.
assumptions (7)
  • standard math TSM projection identity (Theorem A.1, from Williams et al. 2025): J(δ)=∫λ E[⟨δ,B_t⟩²]dt + 2∫∂tλ E[⟨δ,T(X_t)⟩]dt + const.
    This integration-by-parts expansion underlies Theorem 3.1 and is imported from the authors' own prior work rather than derived here. It is a standard calculus identity, so the burden is mild.
  • standard math Fisher information of an exponential-family distribution equals Cov_qθ[T] and is invertible.
    Used throughout (Section 2.2, Definition 2.3, Eq. (2), and Appendix A) to define natural gradients and to invert C in Theorem 3.1.
  • ad hoc to paper q_g0 ∈ M(T) and the trajectory q_gt is precisely traced by the projected updates, so samples from g(Z,t;w) can be used to approximate E_qθ[T] and Cov_qθ[T].
    Stated in Section 3.3 just before Algorithm 1. This is the paper's central modeling assumption: if it fails, the natural-gradient target is evaluated at a distribution that does not match θ_t.
  • ad hoc to paper If T is expressive enough, different time scores do not collapse to the same projection δ_t0.
    Section 3.2 Remarks say the rigorous proof is left as future work. Without this, the projection may lose the information needed to guide the generative model.
  • standard math For drift models, X_t = X_t0+(t-t0)h(X_t0;w), so the Taylor expansion of E[T(X_t)] has no higher-order terms.
    Used in the proof of Theorem 4.2 (Appendix B) to evaluate ∫∂tλ E[T]dt exactly.
  • standard math The RKHS kernel k is twice differentiable and the reproducing property permits interchanging expectations with inner products.
    Used in the derivation of Theorem 4.3 and in converting w* to the pointwise drift formula Eq. (9).
  • standard math For the Stein exponential family, E_p[S_p f]=0, allowing natural gradient of KL[p,q] to be expressed with unnormalized p.
    Invoked in Appendix D (Eq. 22), but the reference for Stein operators is left empty ('[]'), so the reader cannot trace the proof.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Guiding Time-Varying Generative Models with Natural Gradients on Exponential Family Manifold." pith.science (2026). https://pith.science/paper/HTSPMDU7

@misc{pith2026250207650,
  author       = {Pith},
  title        = {Pith review of: Guiding Time-Varying Generative Models with Natural Gradients on Exponential Family Manifold},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HTSPMDU7}},
  note         = {Machine review of arXiv:2502.07650}
}
read the original abstract

Optimising probabilistic models is a well-studied field in statistics. However, its connection with the training of generative models remains largely under-explored. In this paper, we show that the evolution of time-varying generative models can be projected onto an exponential family manifold, naturally creating a link between the parameters of a generative model and those of a probabilistic model. We then train the generative model by moving its projection on the manifold according to the natural gradient descent scheme. This approach also allows us to efficiently approximate the natural gradient of the KL divergence without relying on MCMC for intractable models. Furthermore, we propose particle versions of the algorithm, which feature closed-form update rules for any parametric model within the exponential family. Through toy and real-world experiments, we validate the effectiveness of the proposed algorithms. The code of the proposed algorithms can be found at https://github.com/anewgithubname/iNGD.

Figures

Figures reproduced from arXiv: 2502.07650 by the authors.

Figure 1
Figure 1. Technical notions illustrated. Matching the pro [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The evolution of parametric distributions under [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. The evolution of particles Xt under KiNG on the manifold of Gaussian distributions. Each red line is a trajec￾tory of a particle in the space-time. The initial distribution q0 is plotted on the left as black dotted lines, while the target distribution p is plotted on the right as blue dotted lines [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Particle trajectories when using more expressive [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 6
Figure 6. Figure 6: Sparsity pattern of Θ recovered by graphical lasso, using samples trained by ntKiNG with different sufficient statistics and number of iterations (red boxes indicate miss￾ing edges, the less boxes the better) of the random variables. One can imagine that if the gener￾a…
Figure 7
Figure 7. Figure 7: The inverse mapping ψ −1 found by ntKiNG. The classification boundary of a source classifier trained on p is depicted in four distinct colors. Target domain samples are marked with •, and KiNG transforms these samples to new positions indicated by ×. Notably, many samp…
Figure 8
Figure 8. Figure 8: Denoised samples obtained by ntKiNG and WGF. [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Denoised MNIST images to sample from unknown distributions. These methods first draw samples from an initial distribution and then move them according to a time-dependent velocity field [Liu, 2017, Chewi et al., 2020, Maurais and Marzouk, 2024]. A typical family of met…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

54 extracted references · 41 canonical work pages

  1. [1]

    Natural gradient works efficiently in learning

    Shun-Ichi Amari. Natural gradient works efficiently in learning. Neural computation, 10 0 (2): 0 251--276, 1998

    1998

  2. [2]

    Methods of information geometry, volume 191

    Shun-ichi Amari and Hiroshi Nagaoka. Methods of information geometry, volume 191. American Mathematical Soc., 2000

    2000

  3. [3]

    Gradient flows: in metric spaces and in the space of probability measures

    Luigi Ambrosio, Nicola Gigli, and Giuseppe Savar \'e . Gradient flows: in metric spaces and in the space of probability measures. Springer Science & Business Media, 2008

    2008

  4. [4]

    Kernelized wasserstein natural gradient

    M Arbel, A Gretton, W Li, and G Montufar. Kernelized wasserstein natural gradient. In International Conference on Learning Representations, 2020

    work page 2020

  5. [5]

    Kernel conditional exponential family

    Michael Arbel and Arthur Gretton. Kernel conditional exponential family. In International Conference on Artificial Intelligence and Statistics, 2018

    work page 2018

  6. [6]

    Maximum mean discrepancy gradient flow

    Michael Arbel, Anna Korba, Adil Salim, and Arthur Gretton. Maximum mean discrepancy gradient flow. Advances in Neural Information Processing Systems, 2019

    work page 2019

  7. [7]

    Wasserstein generative adversarial networks

    Martin Arjovsky, Soumith Chintala, and L \'e on Bottou. Wasserstein generative adversarial networks. In International Conference on Machine Learning, 2017

    work page 2017

  8. [8]

    Exact natural gradient in deep linear networks and its application to the nonlinear case

    Alberto Bernacchia, M \'a t \'e Lengyel, and Guillaume Hennequin. Exact natural gradient in deep linear networks and its application to the nonlinear case. Advances in Neural Information Processing Systems, 2018

    work page 2018

Show all 54 references
  1. [9]

    Diffusion schr\"odinger bridge with applications to score-based generative modeling

    Valentin De Bortoli, James Thornton, Jeremy Heng, and Arnaud Doucet. Diffusion schr\"odinger bridge with applications to score-based generative modeling. In Advances in Neural Information Processing Systems, 2021

    2021

  2. [10]

    Statistical inference, 2nd Edition

    George Casella and Roger Berger. Statistical inference, 2nd Edition. CRC press, 2024

    2024

  3. [11]

    Neural ordinary differential equations

    Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. Advances in Neural Information Processing Systems, 2018

    2018

  4. [12]

    Rethinking the diffusion models for missing data imputation: A gradient flow perspective

    Zhichao Chen, Haoxuan Li, Fangyikang Wang, Odin Zhang, Hu Xu, Xiaoyu Jiang, Zhihuan Song, and Hao Wang. Rethinking the diffusion models for missing data imputation: A gradient flow perspective. In Advances in Neural Information Processing Systems, 2024 a

    2024

  5. [13]

    Teng: Time-evolving natural gradient for solving pdes with deep neural nets toward machine precision

    Zhuo Chen, Jacob McCarran, Esteban Vizcaino, Marin Soljacic, and Di Luo. Teng: Time-evolving natural gradient for solving pdes with deep neural nets toward machine precision. In International Conference on Machine Learning, 2024 b

    2024

  6. [14]

    Underdamped langevin mcmc: A non-asymptotic analysis

    Xiang Cheng, Niladri S Chatterji, Peter L Bartlett, and Michael I Jordan. Underdamped langevin mcmc: A non-asymptotic analysis. In Conference on learning theory, 2018

    2018

  7. [15]

    Log-Concave Sampling

    Sinho Chewi. Log-Concave Sampling. 2024

    2024

  8. [16]

    Svgd as a kernelized wasserstein gradient flow of the chi-squared divergence

    Sinho Chewi, Thibaut Le Gouic, Chen Lu, Tyler Maunu, and Philippe Rigollet. Svgd as a kernelized wasserstein gradient flow of the chi-squared divergence. In Advances in Neural Information Processing Systems, 2020

    2020

  9. [17]

    Scalable wasserstein gradient flow for generative modeling through unbalanced optimal transport

    Jaemoo Choi, Jaewoong Choi, and Myungjoo Kang. Scalable wasserstein gradient flow for generative modeling through unbalanced optimal transport. In International Conference on Machine Learning, 2024

    2024

  10. [18]

    Density ratio estimation via infinitesimal classification

    Kristy Choi, Chenlin Meng, Yang Song, and Stefano Ermon. Density ratio estimation via infinitesimal classification. In International Conference on Artificial Intelligence and Statistics, 2022

    2022

  11. [19]

    Optimal transport for domain adaptation

    Nicolas Courty, R \'e mi Flamary, Devis Tuia, and Alain Rakotomamonjy. Optimal transport for domain adaptation. IEEE transactions on pattern analysis and machine intelligence, 39 0 (9): 0 1853--1865, 2016

    2016

  12. [20]

    Sparse inverse covariance estimation with the graphical lasso

    Jerome Friedman, Trevor Hastie, and Robert Tibshirani. Sparse inverse covariance estimation with the graphical lasso. Biostatistics, 9 0 (3): 0 432--441, 2007

    2007

  13. [21]

    Deep generative learning via variational gradient flow

    Yuan Gao, Yuling Jiao, Yang Wang, Yao Wang, Can Yang, and Shunkang Zhang. Deep generative learning via variational gradient flow. In International Conference on Machine Learning, 2019

    2019

  14. [22]

    Fast approximate natural gradient descent in a kronecker factored eigenbasis

    Thomas George, C \'e sar Laurent, Xavier Bouthillier, Nicolas Ballas, and Pascal Vincent. Fast approximate natural gradient descent in a kronecker factored eigenbasis. Advances in Neural Information Processing Systems, 2018

    2018

  15. [23]

    Geodesic flow kernel for unsupervised domain adaptation

    Boqing Gong, Yuan Shi, Fei Sha, and Kristen Grauman. Geodesic flow kernel for unsupervised domain adaptation. In Conference on Computer Vision and Pattern Recognition, 2012

    2012

  16. [24]

    Generative adversarial nets

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in Neural Information Processing Systems, 2014

    2014

  17. [25]

    Borgwardt, Malte J

    Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Sch \"o lkopf, and Alexander Smola. A kernel two-sample test. Journal of Machine Learning Research, 13 0 (25): 0 723--773, 2012

    2012

  18. [26]

    A kronecker-factored approximate fisher matrix for convolution layers

    Roger Grosse and James Martens. A kronecker-factored approximate fisher matrix for convolution layers. In International Conference on Machine Learning, 2016

    2016

  19. [27]

    Vector quantized diffusion model for text-to-image synthesis

    Shuyang Gu, Dong Chen, Jianmin Bao, Fang Wen, Bo Zhang, Dongdong Chen, Lu Yuan, and Baining Guo. Vector quantized diffusion model for text-to-image synthesis. In Conference on Computer Vision and Pattern Recognition, 2022

    2022

  20. [28]

    Posterior sampling based on gradient flows of the MMD with negative distance kernel

    Paul Hagemann, Johannes Hertrich, Fabian Altekr \"u ger, Robert Beinert, Jannis Chemseddine, and Gabriele Steidl. Posterior sampling based on gradient flows of the MMD with negative distance kernel. In International Conference on Learning Representations, 2024

    2024

  21. [29]

    Training neural samplers with reverse diffusive kl divergence

    Jiajun He, Wenlin Chen, Mingtian Zhang, David Barber, and Jos \'e Miguel Hern \'a ndez-Lobato. Training neural samplers with reverse diffusive kl divergence. arXiv preprint arXiv:2410.12456, 2025

    2025 arXiv

  22. [30]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 2020

    2020

  23. [31]

    Neural tangent kernel: convergence and generalization in neural networks

    Arthur Jacot, Franck Gabriel, and Cl\' e ment Hongler. Neural tangent kernel: convergence and generalization in neural networks. In Advances in Neural Information Processing Systems, 2018

    2018

  24. [32]

    Kingma and Max Welling

    Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In International Conference on Learning Representations, 2014

    2014

  25. [33]

    Neural speech synthesis with transformer network

    Naihan Li, Shujie Liu, Yanqing Liu, Sheng Zhao, and Ming Liu. Neural speech synthesis with transformer network. In AAAI Conference on Artificial Intelligence, 2019 a

    2019

  26. [34]

    Affine natural proximal learning

    Wuchen Li, Alex Tong Lin, and Guido Mont \'u far. Affine natural proximal learning. In International Conference on Geometric Science of Information, 2019 b

    2019

  27. [35]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In International Conference on Learning Representations, 2023

    2023

  28. [36]

    Stein variational gradient descent as gradient flow

    Qiang Liu. Stein variational gradient descent as gradient flow. In Advances in Neural Information Processing Systems, 2017

    2017

  29. [37]

    Minimizing f -divergences by interpolating velocity fields

    Song Liu, Jiahao Yu, Jack Simons, Mingxuan Yi, and Mark Beaumont. Minimizing f -divergences by interpolating velocity fields. In International Conference on Machine Learning, 2024

    2024

  30. [38]

    Flow straight and fast: Learning to generate and transfer data with rectified flow

    Xingchao Liu, Chengyue Gong, and qiang liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In International Conference on Learning Representations, 2023

    2023

  31. [39]

    Optimizing neural networks with kronecker-factored approximate curvature

    James Martens and Roger Grosse. Optimizing neural networks with kronecker-factored approximate curvature. In International Conference on Machine Learning, 2015

    2015

  32. [40]

    Sampling in unit time with kernel fisher-rao flow

    Aimee Maurais and Youssef Marzouk. Sampling in unit time with kernel fisher-rao flow. In International Conference on Machine Learning, 2024

    2024

  33. [41]

    Alemi, Jascha Sohl-Dickstein, and Samuel S

    Roman Novak, Lechao Xiao, Jiri Hron, Jaehoon Lee, Alexander A. Alemi, Jascha Sohl-Dickstein, and Samuel S. Schoenholz. Neural tangents: Fast and easy infinite neural networks in python. In International Conference on Learning Representations, 2020

    2020

  34. [42]

    Dataset shift in machine learning

    Joaquin Qui \ n onero-Candela, Masashi Sugiyama, Anton Schwaighofer, and Neil D Lawrence. Dataset shift in machine learning. Mit Press, 2009

    2009

  35. [43]

    Variational inference with normalizing flows

    Danilo Jimenez Rezende and Shakir Mohamed. Variational inference with normalizing flows. In International Conference on Machine Learning, 2015

    2015

  36. [44]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj \"o rn Ommer. High-resolution image synthesis with latent diffusion models. In Conference on Computer Vision and Pattern Recognition, 2022

    2022

  37. [45]

    How to train your energy-based models

    Yang Song and Diederik P Kingma. How to train your energy-based models. arXiv preprint arXiv:2101.03288, 2021

    2021 arXiv

  38. [46]

    Score-based generative modeling through stochastic differential equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021

    2021

  39. [47]

    Density estimation in infinite dimensional exponential families

    Bharath Sriperumbudur, Kenji Fukumizu, Arthur Gretton, Aapo Hyv\" a rinen, and Revant Kumar. Density estimation in infinite dimensional exponential families. Journal of Machine Learning Research, 18 0 (57): 0 1--59, 2017

    2017

  40. [48]

    Direct importance estimation with model selection and its application to covariate shift adaptation

    Masashi Sugiyama, Shinichi Nakajima, Hisashi Kashima, Paul Buenau, and Motoaki Kawanabe. Direct importance estimation with model selection and its application to covariate shift adaptation. In Advances in Neural Information Processing Systems, 2008

    2008

  41. [49]

    Naturalspeech: End-to-end text-to-speech synthesis with human-level quality

    Xu Tan, Jiawei Chen, Haohe Liu, Jian Cong, Chen Zhang, Yanqing Liu, Xi Wang, Yichong Leng, Yuanhao Yi, Lei He, et al. Naturalspeech: End-to-end text-to-speech synthesis with human-level quality. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  42. [50]

    Graphical models, exponential families, and variational inference

    Martin J Wainwright, Michael I Jordan, et al. Graphical models, exponential families, and variational inference. Foundations and Trends in Machine Learning , 1 0 (1--2): 0 1--305, 2008

    2008

  43. [51]

    Diffusion-gan: Training gans with diffusion

    Zhendong Wang, Huangjie Zheng, Pengcheng He, Weizhu Chen, and Mingyuan Zhou. Diffusion-gan: Training gans with diffusion. In International Conference on Learning Representations, 2023

    2023

  44. [52]

    Kernel ridge regression

    Max Welling. Kernel ridge regression. https://web2.qatar.cmu.edu/ gdicaro/10315-Fall19/additional/welling-notes-on-kernel-ridge.pdf, 2019. Accessed: 2025-02-10

    2019

  45. [53]

    Sampling as optimization in the space of measures: The langevin dynamics as a composite optimization problem

    Andre Wibisono. Sampling as optimization in the space of measures: The langevin dynamics as a composite optimization problem. In Conference on Learning Theory, 2018

    2018

  46. [54]

    High-dimensional differential parameter inference in exponential family using time score matching

    Daniel J Williams, Leyang Wang, Qizhen Ying, Song Liu, and Mladen Kolar. High-dimensional differential parameter inference in exponential family using time score matching. International conference on artificial intelligence and statistics, 2025

    2025

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.