Pith. sign in

REVIEW 3 major objections 5 minor 30 references

A preconditioned update that tracks MMD particle flows can reach a global minimum of the non-convex minimum-MMD objective.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 23:21 UTC pith:DJKIDXT7

load-bearing objection Solid non-convex global-convergence theory for min-MMD via adaptive-lengthscale projection of the MMD flow; Assumption 3 is load-bearing and uncertified on the successful trajectories. the 3 major comments →

arxiv 2607.03871 v1 pith:DJKIDXT7 submitted 2026-07-04 cs.LG stat.ML

A Gradient Flow Perspective on Minimum MMD Estimation

Sophia Seulkee Kang , Louis Sharrock , Xiaoyuan Cheng , Fran\c{c}ois-Xavier Briol , Zonghao Chen This is my paper
classification cs.LG stat.ML
keywords minimum MMD estimationMMD gradient flowpreconditioned gradient descentadaptive kernel lengthscalegradient dominancelikelihood-free inferencecomposite goodness-of-fit
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Minimum MMD estimation is popular for robust, likelihood-free parameter fitting, but the optimization problem is typically non-convex, so ordinary gradient descent is only guaranteed to find stationary points. This paper designs a preconditioned gradient update whose one-step change in the objective stays close to a nonparametric MMD flow on the space of probability measures. Under an adaptive kernel-lengthscale schedule, a trajectory-wise gradient-dominance condition, and a uniform bound on how well the parametric tangent space can approximate the nonparametric velocity field, the iterates are shown to drive the MMD (or the excess MMD under misspecification) to its global minimum in expectation. Experiments on multi-modal mixtures, g-and-k, Lotka–Volterra with corruption, and composite goodness-of-fit tests show faster and more reliable recovery of the global minimizer than standard or natural-gradient baselines.

Core claim

Under explicit gradient-dominance and projection-residual conditions, a preconditioned gradient descent that projects the nonparametric MMD velocity field onto the parametric tangent space converges asymptotically to a global minimizer of the minimum-MMD objective, even when that objective is non-convex.

What carries the argument

Preconditioned parametric update (PGD): the closed-form ridge-regularized least-squares projection of the MMD witness-function gradient onto the span of the generator Jacobian, paired with a monotonically decreasing kernel lengthscale.

Load-bearing premise

At every iterate the parametric model must be able to approximate the nonparametric MMD descent direction with residual angle strictly less than ninety degrees and with uniformly bounded coefficients; if that residual is too large, the method makes no progress.

What would settle it

On a well-specified parametric family where the projection residual repeatedly stays near orthogonal (R close to 1) while ordinary multi-start gradient descent still recovers the global minimizer, the claimed global convergence of PGD would fail.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies optimization of minimum MMD estimators for parametric models of the form P_θ = (G_θ)#ρ. It first proves asymptotic global convergence of a nonparametric MMD particle scheme with adaptive Gaussian/Matérn lengthscales under a trajectory-wise gradient-dominance condition (Theorem 3.1). Motivated by that result, it proposes a preconditioned parametric update (PGD) obtained by projecting the nonparametric MMD vector field onto the tangent span of the generator Jacobian (Eqs. 6–7), and proves asymptotic global convergence of the resulting iterates to a global MMD minimizer (well-specified) or to the best attainable MMD value (misspecified) under gradient dominance, a uniform projection-residual bound R<1 (Assumption 3), diminishing steps, adaptive lengthscales, and a sample-size schedule (Theorem 4.6). Empirically, PGD is compared to GD and natural-gradient MMD on g-and-k, Lotka–Volterra under corruption, and a composite GoF toggle-switch task, with ablations and multi-seed reporting.

Significance. If the analysis holds under its stated conditions, the paper supplies one of the first non-convex global-convergence guarantees for minimum MMD estimation, bridging nonparametric MMD gradient-flow theory and parametric practice. Strengths include detailed one-step descent lemmas and Taylor remainders (Appendices A–B), an explicit finite-sample error bound, a clear geometric interpretation of the preconditioner versus natural gradient (Remark 4.5, Appendix C), public code, multi-seed medians/percentiles, and ablations of ℓ_t, λ, γ, and n. The composite GoF experiment is a concrete applied payoff. The main caveat is that the load-bearing projection residual (Assumption 3) is hard to check and is not certified on the experimental trajectories, so the empirics currently support practical usefulness more than theorem confirmation.

major comments (3)
  1. Assumption 3 and Theorem 4.6: the global-convergence claim for PGD rests on a uniform residual bound J_t(u⋆_t) ≤ R²‖∇f‖² with R<1 and ‖u⋆_t‖≤U independent of t. The paper itself calls this “difficult to verify in practice” and only supplies a radial toy example (Lemma E.4 / Example 4.1). In §5 (g-and-k, Lotka–Volterra, toggle-switch), residual angles / estimated R are never reported, so the trajectories that appear to reach the global minimizer are not certified to satisfy the hypothesis under which the theorem guarantees progress (cf. Proposition 4.3 and the (1−R) factor in the descent lemma Eq. 10 / Lemma E.3). Without that certificate the empirics show that PGD often works, not that it works for the reason claimed. Please either (i) report residual diagnostics along the experimental paths, or (ii) substantially weaken/relax Assumption 3 and restate the theorem accordingly.
  2. §5 / Assumption 2: for Lotka–Volterra the authors state they cannot verify the uniform first- and second-order derivative bounds required by Assumption 2 (square-root Euler–Maruyama coefficients near the positive orthant). Theorem 4.6 is therefore not known to apply to that experiment. Either restrict the formal claim to models where Assumption 2 is checked (g-and-k, toggle-switch), or add a local/regularized analysis that covers the simulator used in Figure 5.
  3. Theorem 4.6 is only asymptotic (T→∞) and pathwise gradient dominance is assumed rather than proved for the parametric family. The paper notes that non-asymptotic rates would need stronger conditions on γ_t, ℓ_t, and R. Given that the abstract advertises “asymptotic global convergence under explicit … conditions,” the abstract and introduction should more clearly flag that both dominance and residual conditions are trajectory-wise assumptions that are not a priori guaranteed for general generators, and that the main experiments do not currently verify the residual condition.
minor comments (5)
  1. Figure 1 / Figure 7: the illustrative two-mode mixture landscape is helpful; consider adding the residual angle or ‖sin α‖ along the PGD path in the same figure to connect the toy example to Assumption 3.
  2. Notation: the manuscript switches between MMD_ℓ, MMD²_ℓ, and MMD_ℓ∞ in figures; a short notation paragraph or consistent subscripting would help.
  3. §4 computational complexity: the O(ndp² + p³) cost is stated clearly; a brief note on when Jacobian-vector products make PGD competitive with NGD for larger p would be useful.
  4. Typos / polish: “agrowingbodyofwork”, “parametrized”, occasional missing spaces in the introduction; “its linear span” → “its linear span.”
  5. Table 1: report wall-clock under a fixed compute budget as well as iteration count, since multi-start GD (I=400) is the main baseline and GPU parallelization is mentioned only briefly.

Circularity Check

0 steps flagged

No significant circularity: global-convergence claims follow from explicit one-step descent inequalities, Taylor matching, and discrete Gronwall under openly stated trajectory assumptions; empirics and hyper-parameter grids are separate from the theorems.

full rationale

The derivation chain is self-contained. Nonparametric adaptive MMD descent (Eq. 3) is shown in Theorem 3.1 to drive MMD_ℓ(P_t,Q)→0 via a one-step descent lemma (Prop. A.1) that uses only kernel Lipschitz/continuity properties (Lemmas E.1, E.6–E.9) plus a trajectory-wise gradient-dominance hypothesis that is assumed, not fitted. Parametric PGD is obtained by matching the first-order γ_t terms of the nonparametric and parametric Taylor expansions of MMD² (display after Eq. 5), yielding the least-squares projection (Eq. 6–7) whose residual is controlled by Assumption 3; the one-step comparison is Prop. 4.3 / B.1 and the finite-sample error is Prop. B.4. These are combined into the descent inequality (Eq. 10) and closed by discrete Gronwall to obtain Theorem 4.6. None of the equalities reduce by construction to a fitted quantity or to a self-defined normalization. Self-citations (Chen et al. 2021/2025b/2026b) supply motivation and related flow analyses but are not used as uniqueness theorems that forbid alternatives; the proofs stand alone in Appendices A–B. Hyper-parameter schedules (ℓ_t, λ_t, γ_t) are free parameters required only to satisfy the stated summability and decay conditions; grid search appears only in the experimental section and does not enter the theorems. Assumption 3 is load-bearing and hard to verify, but that is a verification gap, not circularity. Score 1 reflects only the ordinary presence of overlapping-author citations that are non-load-bearing for the central claims.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

Central claims rest on standard kernel/measure-theory background plus three paper-specific regularity conditions (kernel class, generator smoothness, projection residual) and free schedules for step size, lengthscale and ridge that must satisfy explicit inequalities. No new physical entities are postulated.

free parameters (4)
  • lengthscale schedule (ℓ_t → ℓ)
    Exponential or polynomial decay rates (η or β) are chosen by grid search; the limiting ℓ is a statistical hyper-parameter; theorems require monotonic decrease to a positive limit and a trajectory-wise dominance inequality involving ℓ_t.
  • step-size schedule (γ_t)
    Polynomial decay γ_t = γ_0 (1+t)^{-β} with β∈(0,1); must satisfy γ_t ∈ (0,1/(2C(1-R))) and sum to infinity.
  • ridge λ_t
    Theory sets λ_t ∝ (γ_t p² (1+ℓ^{-1}) ℓ_t^{-2})^{2/5}; practice uses fixed small constants chosen by search.
  • dominance constant C and residual R
    C appears in the gradient-dominance inequality; R<1 bounds the projection residual; both are assumed independent of t and not estimated from data.
axioms (5)
  • domain assumption Kernel is Gaussian or Matérn with s>1 (Assumption 1)
    Used for Lipschitz constants of ∇f, Fourier conjugacy for lengthscale change, and bounded second derivatives in Taylor remainders.
  • domain assumption Generator G_θ twice C¹ with uniform second-moment bounds on G, ∂_θG, ∂²_θG (Assumption 2)
    Controls remainder terms in the parametric Taylor expansion and ensures P_θ ∈ P_2.
  • ad hoc to paper Projection residual uniformly bounded by R<1 and coefficient U (Assumption 3)
    Load-bearing for the one-step comparison (Prop. 4.3) and for stationarity transfer (Lemma E.3); paper admits it is hard to verify.
  • ad hoc to paper Trajectory-wise gradient dominance of MMD² along the realized path
    Replaces convexity; checked a posteriori in experiments; required for both Thm 3.1 and 4.6.
  • standard math Standard facts on RKHS embeddings, continuity equation, discrete Gronwall
    Background used throughout Appendices A–B.

pith-pipeline@v1.1.0-grok45 · 52215 in / 2920 out tokens · 25667 ms · 2026-07-11T23:21:19.780479+00:00 · methodology

0 comments
read the original abstract

Minimum maximum mean discrepancy (MMD) estimation has emerged as a robust and likelihood-free alternative to maximum likelihood estimation for parameter estimation. Yet, despite its practical success, the associated optimization problem remains poorly understood, with theoretical guarantees for existing algorithms hinging on convexity assumptions that rarely hold in practice. We address this gap by proposing a preconditioned gradient descent (PGD) scheme, establishing its asymptotic \emph{global} convergence under explicit gradient-dominance and projection-residual conditions. Our approach is inspired by recent progress on MMD gradient flows, a nonparametric descent scheme on the space of probability measures. We provide extensive empirical evidence that our PGD scheme outperforms standard gradient descent across a range of challenging parameter estimation and composite hypothesis testing problems.

Figures

Figures reproduced from arXiv: 2607.03871 by Fran\c{c}ois-Xavier Briol, Louis Sharrock, Sophia Seulkee Kang, Xiaoyuan Cheng, Zonghao Chen.

Figure 1
Figure 1. Figure 1: Illustrative example. Both the target distribution Q and the model distribution Pθ are two-mode Gaussian mixtures, where θ = (σ1, σ2) denotes the standard deviations of the two modes. We compare minimum MMD estimators trained by standard gradient descent (GD) and our preconditioned gradient descent (PGD). The left panel shows the optimization trajectories, with the yellow star marking the global minimizer,… view at source ↗
Figure 2
Figure 2. Figure 2: A geometric inter￾pretation of the projection. Assumption 3 admits a natural geometric interpretation. For the first inequality of Assumption 3, R can be interpreted as a uniform￾in-t upper bound on |sin(α)|, where α denotes the projection angle shown in [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Mixture of Gaussians. From left to right: first and final locations of 100 particles; MMD versus iterations; MMD versus particle number N; the left- and right-hand sides of the condition in Theorem 3.1. Results are over 10 seeds; curves show medians and bands the 25–75% percentiles. et al. [2019] and Theorem 3.2 of Chérief-Abdellatif and Näf [2025]), whereas our implementation approximates θ ∗ through a fi… view at source ↗
Figure 4
Figure 4. Figure 4: G-and-k distribution. From left to right: two optimization trajectories toward the global optimum (yellow star), MMD versus iteration, and left- and right-hand sides of the condition in Theorem 4.6. In the left two plots, two trajectories for each method are shown. Results are over 10 seeds; curves show medians and bands the 25–75% percentiles. ℓt = max{ℓ, 10.0 × 0.9999t}. The decay factor 0.9999 was selec… view at source ↗
Figure 5
Figure 5. Figure 5: Lotka-Volterra model. Left: Two optimization trajectories toward the global optimum (yellow star) under two initializations. Middle: Evolution of θ1, θ2 under different initializations. Right: Evolution of θ1, θ2 under different corruption levels. Results are over 10 seeds; curves show medians and bands the 25–75% percentiles. preconditioner due to the double expectation. More empirical studies are thus ne… view at source ↗
Figure 6
Figure 6. Figure 6: Ablation study: Standard GD with fixed/adaptive kernel lengthscales and our PGD with fixed/adaptive kernel lengthscales. Left: G-and-K distribution and Right: Lotka-Volterra model in well-specified setting with θ0 = (90, 90). g-and-k distribution. The relative importance of the two components may be problem-dependent; in practice, we therefore recommend using both. 6 Conclusions, Limitations and Future Wor… view at source ↗
Figure 7
Figure 7. Figure 7: Landscape of MMD2 ℓ (Pθ, Q) for the illustrative experiment with ℓ = {0.2, 1.0, 5.0, 10.0}. Appendix D Additional Experimental Details In this section, we provide additional details for reproducing our experiments in Section 5, as well as additional experiments to further support our claims made in the main text. All experiments were run on a workstation equipped with an AMD Ryzen 9 5950X 16-Core Processor… view at source ↗
Figure 8
Figure 8. Figure 8: Computational time. Left: G-and-k distribution. Middle & Right: Lotka-Volterra model. D.2 Mixture of Gaussians The target distribution Q is an equally weighted mixture of eight Gaussian distributions on R 2 whose centers lie on a circle of radius 2.0. Specifically, the i-th component has mean 2 cos( 2πi 8 ), 2 sin( 2πi 8 )  for i = {0, . . . , 7}, and a diagonal covariance matrix 0.04Id. The initial N par… view at source ↗
Figure 9
Figure 9. Figure 9: Ablation of all hyperparameters in PGD Top row: G-and-k distribution. Bottom row: Lotka-Volterra model in the well-specified case with initial parameter θ0 = (90, 90). size 600 randomly subsampled at each iteration. The same set of samples is used for standard GD and for the PGD scheme of Briol et al. [2019]. GD used a fixed lengthscale, ℓt = ℓ = 2.0, whereas PGD used an adaptive lengthscale, ℓt = max{2.0,… view at source ↗
Figure 10
Figure 10. Figure 10: The example of Q = Unif(S d−1 ) has stationary points that are not global minima. Proof. Since θ ⋆ is a stationary point, then, for any direction u ∈ R p , the first order condition gives 0 = Ez∼ρ h ∇fℓ,Pθ⋆ ,Q(Gθ ⋆ (z))⊤JθGθ ⋆ (z)u i . In other words, at such a stationary point, the function z 7→ ∇fℓ,Pθ⋆ ,Q(Gθ ⋆ (z)) is orthogonal in L2(ρ) to every tangent direction of the form z 7→ JθGθ ⋆ (z)u with u ∈ R… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

30 extracted references · 4 linked inside Pith

  1. [1]

    On the convergence of SGD with biased gradients.arXiv preprint arXiv:2008.00051,

    Ahmad Ajalloeian and Sebastian U Stich. On the convergence of SGD with biased gradients.arXiv preprint arXiv:2008.00051,

    Pith/arXiv arXiv 2008

  2. [2]

    16 Pierre Alquier, Badr-Eddine Chérief-Abdellatif, Alexis Derumigny, and Jean-David Fermanian

    ISSN 2824-7795. 16 Pierre Alquier, Badr-Eddine Chérief-Abdellatif, Alexis Derumigny, and Jean-David Fermanian. Estimation of copulas via maximum mean discrepancy.Journal of the American Statistical Association, 118(543):1997–2012,

    1997

  3. [3]

    Estimation of time series by maximum mean discrepancy.arXiv preprint arXiv:2601.11233,

    Pierre Alquier, Jean-David Fermanian, and Benjamin Poignard. Estimation of time series by maximum mean discrepancy.arXiv preprint arXiv:2601.11233,

    arXiv

  4. [4]

    Statistical inference for generative models with maximum mean discrepancy.arXiv:1906.05944,

    François-Xavier Briol, Alessandro Barp, Andrew B Duncan, and Mark Girolami. Statistical inference for generative models with maximum mean discrepancy.arXiv:1906.05944,

    Pith/arXiv arXiv 1906

  5. [5]

    Berrett, and Theodoros Damoulas

    Mengqi Chen, Charita Dellaporta, Thomas B. Berrett, and Theodoros Damoulas. Total robustness in Bayesian nonlinear regression for measurement error problems under model misspecification. arXiv preprint arXiv:2510.03131, 2025a. Yindong Chen, Yiwei Wang, Lulu Kang, and Chun Liu. A deterministic sampling method via maximum mean discrepancy flow with adaptive...

    arXiv

  6. [6]

    Parametric MMD estimation with missing values: robustness to missingness and data model misspecification.arXiv preprint arXiv:2503.00448,

    Badr-Eddine Chérief-Abdellatif and Jeffrey Näf. Parametric MMD estimation with missing values: robustness to missingness and data model misspecification.arXiv preprint arXiv:2503.00448,

    arXiv

  7. [7]

    On the Wasserstein gradient flow interpretation of drifting models.arXiv preprint arXiv:2605.05118,

    Arthur Gretton, Li Kevin Wenliang, Alexandre Galashov, James Thornton, Valentin De Bortoli, and Arnaud Doucet. On the Wasserstein gradient flow interpretation of drifting models.arXiv preprint arXiv:2605.05118,

    Pith/arXiv arXiv

  8. [8]

    Minimum distance summaries for robust neural posterior estimation.arXiv preprint arXiv:2602.09161v1,

    Sherman Khoo, Dennis Prangle, Song Liu, and Mark Beaumont. Minimum distance summaries for robust neural posterior estimation.arXiv preprint arXiv:2602.09161v1,

    arXiv

  9. [9]

    Minimum Wasserstein distance estimator under covariate shift: closed-form, super-efficiency and irregularity.arXiv:2601.07282,

    Junjun Lang, Qiong Zhang, and Yukun Liu. Minimum Wasserstein distance estimator under covariate shift: closed-form, super-efficiency and irregularity.arXiv:2601.07282,

    arXiv

  10. [10]

    Prediction-centricuncertainty quantification via MMD

    ZheyangShen, JeremiasKnoblauch, SamuelPower, andChrisJOates. Prediction-centricuncertainty quantification via MMD. InProceedings of the 28th International Conference on Artificial Intelligence and Statistics, pages 649–657. PMLR, 2025a. 21 Zikai Shen, Zonghao Chen, Dimitri Meunier, Ingo Steinwart, Arthur Gretton, and Zhu Li. Nonpara- metric instrumental v...

    arXiv

  11. [11]

    Sobolev regularized MMD gradient flow.arXiv preprint arXiv:2605.11884,

    Chenyang Tian, Bharath K Sriperumbudur, Arthur Gretton, and Zonghao Chen. Sobolev regularized MMD gradient flow.arXiv preprint arXiv:2605.11884,

    Pith/arXiv arXiv

  12. [12]

    Appendix C provides a detailed discussion on the connection of our projected gradient descent (PGD) scheme with natural gradient descent

    22 Appendices Appendix A provides proof for main convergence theorem of the nonparametric descent scheme; Appendix B provides proof for main convergence theorem of the parametric descent scheme. Appendix C provides a detailed discussion on the connection of our projected gradient descent (PGD) scheme with natural gradient descent. Appendix D give addition...

    2022

  13. [13]

    ∥∇fℓ∥2 Ld 2(Pt) − ∥∇fℓ∥Ld 2(Pt) Cd,sCk p ℓ2 t −ℓ 2 ℓ2 # + Ckγ2 ℓ2

    The last inequality holds by the first moment bound onU∼ν . Then, for the second term on the right hand side of Eq. (A.4), we use the triangular-inequality and Lemma E.6: ∥∇fℓt∥2 Ld 2(Pt) ≤2∥∇f ℓ∥2 Ld 2(Pt) + 2∥∇f ℓt − ∇fℓ∥2 Ld 2(Pt) ≤2∥∇f ℓ∥2 Ld 2(Pt) +C k ℓ2 t −ℓ 2 ℓ3 2 . Here, Ck is a constant depending only on the kernelk and is independent ofℓ. Combi...

    2022

  14. [14]

    approximation error

    The target of interest to be upper bounded corresponds to the “approximation error” in least-square regression; see Fischer and Steinwart [2020], Shen et al. [2025b]. The proof of Lemma B.2 is therefore a standard technique. Proposition B.4(Closeness of PPGD θ1 and PPGD θ1,m ).Let γ > 0, λ > 0, evaluation lengthscaleℓ > 0, and update lengthscaleℓ0 > 0be f...

    2020

  15. [15]

    By definition, recall that ∆θPGD = n Ez∼ρ h JθGθ0(z)⊤JθGθ0(z) i +λId o−1 Ez∼ρ h JθGθ0(z)⊤∇fℓ0,Pθ0 ,Q(Gθ0(z)) i ∆θPGD 0,m = n Ez∼ρ h JθGθ0(z)⊤JθGθ0(z) i +λId o−1 Ez∼ρ h JθGθ0(z)⊤∇fℓ0,Pθ0 ,bQm (Gθ0(z)) i . So we have ∆θPGD −∆θ PGD 0,m ≤λ −1 · Ez∼ρ h JθGθ0(z)⊤ ∇fℓ0,Pθ0 ,Q(Gθ0(z))− ∇f ℓ0,Pθ0 ,bQm (Gθ0(z)) i ≤λ −1√p·E z∼ρ h ∇fℓ0,Pθ0 ,Q(Gθ0(z))− ∇f ℓ0,Pθ0 ,bQm ...

    2017

  16. [16]

    Therefore, we have MMD2 ℓ(Pθ,P θ+h) =h ⊤ n Ez,z ′∼ρ h JθGθ(z)⊤∇1∇2kℓ(Gθ(z), G θ(z′))JθGθ(z′) io h+o(∥h∥ 2)

    And, d2 dh2 h=0 1 2 MMD2 ℓ(Pθ,P θ+h) = ZZ JθGθ(z)⊤∇1∇2k(Gθ(z), Gθ(z′))JθGθ(z′)dρ(z)dρ(z ′) =E z,z ′∼ρ h JθGθ(z)⊤∇1∇2kℓ(Gθ(z), G θ(z′))JθGθ(z′) i . Therefore, we have MMD2 ℓ(Pθ,P θ+h) =h ⊤ n Ez,z ′∼ρ h JθGθ(z)⊤∇1∇2kℓ(Gθ(z), G θ(z′))JθGθ(z′) io h+o(∥h∥ 2). The resulting minimizer of Eq. (C.2) thus admits a closed-form [Briol et al., 2019] h=γ n Ez,z ′∼ρ h J...

    2019

  17. [17]

    For standard gradient descent (GD), we use a fixed step sizeγ = 10 −4, initialization θ0 = (4.0, 3.5), and T = 650iterations

    The kernel for computing theMMD2 ℓ(Pθ,Q )is a Gaussian kernel with kernel lengthscaleℓ= 1.0. For standard gradient descent (GD), we use a fixed step sizeγ = 10 −4, initialization θ0 = (4.0, 3.5), and T = 650iterations. For our preconditioned gradient descent (PGD), we usen = 512 particles drawn fromPθ to estimate the preconditioning matrix, ridge regulari...

    2024

  18. [18]

    [2019], and the parameter domain isΘ = [−10, 10] × [0.1, 10] × [−10, 10] × [0, 2]for( a, b, c,exp(k))

    The true value is set toθ∗ = (3, 1, 1,exp (−log 2))following the setting of Briol et al. [2019], and the parameter domain isΘ = [−10, 10] × [0.1, 10] × [−10, 10] × [0, 2]for( a, b, c,exp(k)). We next check the integrability requirements in Assumption

    2019

  19. [19]

    size600randomly subsampled at each iteration

    We have access to a fixed set ofm = 1000IID samples from the targetQ, and approximate expectations with respect toQ using minibatches of 35 0.03 0.1 0.3 1 10 100 step size, γ 10−1 100 101 102 ∥θfinal − θtrue∥2 0.5 0.8 0.9 0.95 0.99 lengthscale exponential, η 10−1 100 0.01 0.1 0.2 0.3 0.5 1 lengthscale polynomial, β 10−1 100 2 10 50 100 300 600 1000 n 10−1 ...

    2019

  20. [20]

    The same set of samples is used for standard GD and the PGD scheme of Briol et al

    Since bothm and n are relatively small here, we use the full batch of samples without subsampling. The same set of samples is used for standard GD and the PGD scheme of Briol et al. [2019]. GD used a fixed lengthscale, ℓt =ℓ= 30, whereas PGD used an adaptive lengthscale,ℓt = max{30,3000×0.9995 t}. In the right two panels of Figure 8, we report the computa...

    2019

  21. [21]

    [2019], where convergence is achieved within only a few tens of iterations, since the code is not publicly available

    We were unable to reproduce the results in Figure 6 of Briol et al. [2019], where convergence is achieved within only a few tens of iterations, since the code is not publicly available. We suspect that they might use a substantially larger step size than we do. More experimental evidence is therefore required to better understand which preconditioner is p...

    2019

  22. [22]

    The upper bound γ≤ 0.4also controls the negative powers ofuT appearing in the observation map. SinceuT is positively truncated Gaussian conditional on the previous latent states, its negative moments are finite up to order strictly smaller than one; hence the factorsu−γ T appearing in the observation model have finite second moments. The inverse-CDF repre...

    1941

  23. [23]

    Since our PGD does not require extensive initialization search, it is computationally much cheaper than the GD scheme used in Key et al

    This indicates that our PGD scheme can reliably find low-MMD solutions without the extensive initialization search used by GD. Since our PGD does not require extensive initialization search, it is computationally much cheaper than the GD scheme used in Key et al. [2025]. Appendix E Auxiliary Results Lemma E.1.Let φ : [0,∞ ) →R be either the Gaussian profi...

    2025

  24. [24]

    Lemma E.2(Strong convexity implies gradient dominance).Suppose thatθ7→MMD 2 ℓ(Pθ,Q )is α strongly convex

    Applying the mean-value theorem again concludes the proof. Lemma E.2(Strong convexity implies gradient dominance).Suppose thatθ7→MMD 2 ℓ(Pθ,Q )is α strongly convex. Suppose Assumption 2 holds. Then,MMD2 ℓ(Pθ,Q ) −min θ∈Θ MMD2 ℓ(Pθ,Q ) ≤ 2p α Ez∼ρ h ∥∇fℓ,Pθ,Q (Gθ(z))∥2 i . Proof.Sinceαstrongly convex implies gradient dominance [Karimi et al., 2016], we hav...

    2016

  25. [25]

    Let P0 ∈ P 2(Rd)and let the next iterate beP1 = (Id−γG) #P0

    Let G : Rd →R d and assume G∈L d 2(P0). Let P0 ∈ P 2(Rd)and let the next iterate beP1 = (Id−γG) #P0. Then, we have MMD2 ℓ(P1,Q)≤MMD 2 ℓ(P0,Q)−2γ Z ∇fℓ,P0,Q(x)⊤G(x)dP0(x) + Ckγ2 ℓ2 ∥G∥2 Ld 2(P0). Here,C k depends only on the kernel profile and not on the lengthscale. Proof. Define, for t∈ [0, 1], Gt(x) := x−tγG (x)and Pt := (Gt)#P0, so thatP0 = (G0)#P0 and...

    1976

  26. [26]

    There exists a probability measureν on Rd such that, for anyx∈R d, EU∼ν ∇fℓ,P,Q(x+U) = ℓℓ′−1 d ∇fℓ′,P,Q(x),E U∼ν [∥U∥]≤C d,s p ℓ′2 −ℓ 2.(E.4) Here,C d,s is a constant depending only on the dimensiondand the smoothness parameters. Proof.Notice that EU∼ν ∇fℓ,P,Q(x+U) = Z Rd Z Rd ∇1kℓ(x+u, y) d(P−Q)(y) dν(u) = Z Rd ∇x Z Rd kℓ(x+u, y) dν(u) d(P−Q)(y), where t...

    1976

  27. [27]

    Moreover, EU∼ν [∥U∥] ≤C d,s p ℓ′2 −ℓ 2 for some constant Cd,s depending only on the dimensiond and the smoothness parameters

    Then, for anyℓ′ > ℓ > 0, there exists a probability measure ν on Rd such that EU∼ν [kℓ(x+ U, y)] = ℓℓ′−1 d kℓ′(x, y)holds for any x, y∈R d. Moreover, EU∼ν [∥U∥] ≤C d,s p ℓ′2 −ℓ 2 for some constant Cd,s depending only on the dimensiond and the smoothness parameters. Proof. Write kℓ(x, y) = ψℓ(x−y ), and denoteSℓ = F[ψℓ] : Rd →R as the Fourier transform ofψ...

    2004

  28. [28]

    Then, h(r) = s+ d 2 h log 1 + ℓ′2r 2s −log 1 + ℓ2r 2s i , and hence h′(r) = s+ d 2 (ℓ′2−ℓ2)/(2s) 1+ ℓ2r 2s 1+ ℓ′2r 2s

    To prove the claim, seth(r) := −logg (r). Then, h(r) = s+ d 2 h log 1 + ℓ′2r 2s −log 1 + ℓ2r 2s i , and hence h′(r) = s+ d 2 (ℓ′2−ℓ2)/(2s) 1+ ℓ2r 2s 1+ ℓ′2r 2s . The prefactor is positive becauseℓ′ > ℓ. Moreover, for everya > 0, the mapr7→ (1+ ar)−1 is completely monotone, since (−1)n dn drn (1 +ar) −1 =n!a n(1 +ar) −n−1 ≥0,∀n∈N. Therefore both factors r7...

    2012

  29. [29]

    This concludes the proof of the Matérn case. Remark E.10(Extension to more general kernels).Lemma E.8 shows that, for the Gaussian or Matérn kernel, MMD flow with noise injection is equivalent to MMD flow with an increased kernel lengthscale. This observation was first made in Glaser [2020]. The above equivalence can be generalized to other translation-in...

    2020

  30. [30]

    conjugacy

    provides a more general family of kernels and distributions for which such relation holds, which they refer to as “conjugacy”. In this paper, we focus explicitly on the Gaussian and Matérn kernels, which are widely used in minimum MMD estimation [Briol et al., 2019, Chérief-Abdellatif and Alquier, 2022], leaving other kernels as future work. 46

    2019