pith. sign in

arxiv: 2601.14147 · v2 · submitted 2026-01-20 · 🧮 math.OC · stat.CO

Gradient flow for finding E-optimal designs

Jieling Shi , Kim-Chuan Toh , Xin T. Tong , Weng Kee Wong This is my paper

Pith reviewed 2026-05-16 12:20 UTC · model grok-4.3

classification 🧮 math.OC stat.CO
keywords E-optimalityWasserstein gradient flowoptimal experimental designsemidefinite programmingnonsmooth optimizationinformation matrixregression modelssteepest ascent
0
0 comments X

The pith

A constrained steepest-ascent field in Wasserstein space produces a flow whose limit points are first-order stationary for the nonsmooth E-criterion.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a gradient-flow method for E-optimal design problems in which the criterion is nonsmooth when the smallest eigenvalue of the information matrix has multiplicity greater than one. Working in the space of probability measures equipped with the 2-Wasserstein metric, the authors replace the undefined gradient with a steepest-ascent direction obtained by solving a convex optimization problem over the tangent cone at each empirical measure. They prove that the resulting continuous-time flow satisfies an exact energy-dissipation identity and that every accumulation point is a first-order stationary point for the E-criterion. The discrete particle implementation reduces each ascent step to a semidefinite program whose size equals the multiplicity of the minimal eigenvalue. This approach is shown to be more reliable than particle-swarm methods on response-surface and logistic-regression examples.

Core claim

Working in the 2-Wasserstein space, the Wasserstein gradient for smooth criteria coincides with the Euclidean particle gradient up to a constant factor, with the approximation gap for equal-weight designs vanishing at an explicit rate. For the nonsmooth E-criterion we construct a constrained Wasserstein steepest-ascent field by maximizing feasible directional derivatives over the tangent cone of the design space. The resulting flow satisfies an exact energy identity and every limit point is first-order stationary. The particle ascent computation reduces to a convex semidefinite programme whose dimension equals the multiplicity of the smallest eigenvalue.

What carries the argument

The constrained Wasserstein steepest-ascent field obtained by maximizing feasible directional derivatives of the E-criterion over the tangent cone at the current empirical measure.

If this is right

  • The flow satisfies an exact energy identity.
  • Every limit point of the flow is first-order stationary for the E-criterion.
  • Each particle ascent step reduces to a convex semidefinite program whose dimension equals the multiplicity of the smallest eigenvalue.
  • The method extends directly to other nonsmooth minimax criteria in optimal design.
  • Numerical tests on second-order response surface models and seven-dimensional logistic regression attain near-optimal E-values and outperform particle swarm optimization in higher dimensions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same constrained-ascent construction may apply to other nonsmooth criteria that arise in statistical optimal design.
  • It supplies a concrete bridge between Wasserstein geometry and convex programming for non-differentiable objective functions.
  • One could test the method on larger-scale design problems where traditional algorithms become unreliable.
  • Mean-field analysis of the particle system might yield convergence rates beyond the stationarity result proved here.

Load-bearing premise

Directional derivatives of the E-criterion exist at the points of interest and the tangent cone of the design space admits a well-defined maximizer that supplies a feasible ascent direction.

What would settle it

A numerical simulation of the flow in which the claimed energy identity is violated, or a computed limit point that fails the first-order stationarity condition for the E-criterion.

Figures

Figures reproduced from arXiv: 2601.14147 by Jieling Shi, Kim-Chuan Toh, Weng Kee Wong, Xin T. Tong.

Figure 1
Figure 1. Figure 1: Convergence curves for the D-optimal design problem obtained by the particle Wasserstein gradient flow (WGF) and particle swarm optimiza￾tion (PSO) methods under different constraints. The PSO curve is averaged over 100 independent runs. 4.2 E-optimal design In this subsection, we evaluate the performance of the particle Wasserstein gradient flow (WGF) method introduced in section 3 for the E-optimal de￾si… view at source ↗
Figure 2
Figure 2. Figure 2: Convergence curves for the E-optimal design problem for the second-order response surface model under different dimensions and con￾straints, obtained by the particle Wasserstein gradient flow (WGF) and particle swarm optimization (PSO) methods. The PSO curve is averaged over 100 independent runs. and use this f in the information matrix Mρ as in (2). We study local E-optimal designs of the form max ρ∈P2 [… view at source ↗
Figure 3
Figure 3. Figure 3: Convergence curves for the E-optimal design problem in the lo￾gistic model, computed via particle Wasserstein gradient flow (WGF) and particle swarm optimization (PSO). The PSO curve is averaged over 100 independent runs, while the WGF curve is smoothed by averaging the ob￾jective values over every 10 iterations. In summary, our numerical study supports two conclusions. First, for the 29 [PITH_FULL_IMAGE:… view at source ↗
read the original abstract

The $E$-optimality criterion for a regression model maximizes the smallest eigenvalue of the information matrix and becomes non-differentiable when this eigenvalue has multiplicity greater than one. Working in the $2$-Wasserstein space, we show that the Wasserstein gradient at an empirical measure coincides, up to a constant factor, with the Euclidean particle gradient for smooth criteria such as $D$- and $L$-optimality, and that the approximation gap for equal-weight $N$-particle designs vanishes at an explicit rate. The main challenge is the nonsmooth $E$-criterion, for which the Wasserstein gradient does not exist. We replace it with a constrained Wasserstein steepest-ascent field obtained by maximizing feasible directional derivatives over the tangent cone of the design space, and prove that the resulting flow satisfies an exact energy identity and that every limit point is first-order stationary. The particle ascent computation reduces to a convex semidefinite programme whose dimension equals the multiplicity of the smallest eigenvalue. In numerical comparisons on second-order response surface models and a seven-dimensional logistic regression model, the constrained Wasserstein steepest-ascent method attains near-optimal $E$-criterion values and is markedly more reliable than particle swarm optimization in higher-dimensional settings. The framework applies more broadly to other nonsmooth minimax criteria in optimal design, and a numerical experiment on the minimax-single-parameter criterion confirms that the method attains the theoretical optimum.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 2 minor

Summary. The paper develops a gradient-flow method in the 2-Wasserstein space for E-optimal design of regression experiments. It shows that the Wasserstein gradient coincides with the Euclidean particle gradient for smooth criteria (D- and L-optimality) and that the gap for equal-weight N-particle designs vanishes at an explicit rate. For the nonsmooth E-criterion it replaces the gradient by a constrained steepest-ascent field obtained by maximizing directional derivatives over the tangent cone, proves that the resulting flow satisfies an exact energy identity, and shows that every limit point is first-order stationary. The particle update reduces to a convex SDP whose size equals the multiplicity of the smallest eigenvalue. Numerical comparisons on second-order response-surface and seven-dimensional logistic models demonstrate near-optimal performance and greater reliability than particle-swarm optimization.

Significance. If the central claims hold, the work supplies a theoretically justified, computationally tractable algorithm for a classically nonsmooth optimal-design problem, together with an energy identity and stationarity guarantee that are new for this setting. The reduction to a convex SDP of modest size and the extension to other minimax criteria are likely to be useful in both theory and practice, especially in moderate-to-high-dimensional models where existing heuristics are unreliable.

minor comments (2)
  1. [Abstract] Abstract: the explicit rate at which the approximation gap for equal-weight N-particle designs vanishes is stated to exist but not displayed; adding the rate (or its order) would improve immediate readability.
  2. [Section 3 (or wherever the steepest-ascent field is defined)] The manuscript would benefit from a short remark clarifying whether the tangent-cone maximization remains well-posed when the multiplicity of the smallest eigenvalue changes along the flow.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the positive summary of our work and the recommendation to accept the manuscript. The referee's description accurately reflects the paper's contributions on the constrained Wasserstein steepest-ascent flow for E-optimal designs, the energy identity, stationarity results, SDP reduction, and numerical comparisons.

Circularity Check

0 steps flagged

No significant circularity; derivation is self-contained

full rationale

The paper defines the constrained Wasserstein steepest-ascent field by maximizing directional derivatives of the E-criterion over the tangent cone of the design space, then proves an energy identity and first-order stationarity for the resulting flow. These steps rely on standard properties of the 2-Wasserstein space, the variational characterization of the directional derivative of lambda_min, and convex semidefinite programming; none of the load-bearing claims reduce by the paper's own equations to a fitted parameter, self-referential definition, or self-citation chain. The coincidence of Wasserstein and Euclidean gradients for smooth criteria is shown directly from the metric definition, and the nonsmooth E-case replacement is justified by the tangent-cone maximization without importing uniqueness theorems or ansatzes from prior author work. The framework is therefore independent of its inputs and externally falsifiable via the stated SDP reduction.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claim rests on standard results from optimal transport (Wasserstein geometry and tangent cones) and convex analysis; no free parameters are fitted to data and no new entities are postulated.

axioms (2)
  • domain assumption Directional derivatives of the E-criterion exist and the tangent cone of the design space is well-defined
    Invoked to construct the constrained steepest-ascent field and to prove the energy identity and stationarity.
  • standard math Standard properties of the 2-Wasserstein space on probability measures
    Used to equate the Wasserstein gradient with the Euclidean particle gradient for smooth criteria and to define the flow.

pith-pipeline@v0.9.0 · 5554 in / 1564 out tokens · 59844 ms · 2026-05-16T12:20:51.554134+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

65 extracted references · 65 canonical work pages

  1. [1]

    Ambrosio, N

    L. Ambrosio, N. Gigli, and G. Savar´ e.Gradient flows: in metric spaces and in the space of probability measures. Springer Science & Business Media, 2008

    work page 2008

  2. [2]

    Atkinson, A

    A. Atkinson, A. Donev, and R. Tobias.Optimum experimental designs, with SAS, volume 34. OUP Oxford, 2007

    work page 2007

  3. [3]

    M. P. Berger and W. K. Wong.An introduction to optimal designs for social and biomedical research. John Wiley & Sons, 2009

    work page 2009

  4. [4]

    Berkholz, D

    R. Berkholz, D. R¨ ohlig, and R. Guthke. Data and knowledge based experimental design for fermentation process optimization.Enzyme and microbial technology, 27(10):784–788, 2000

    work page 2000

  5. [5]

    D. P. Bertsekas.Nonlinear Programming. Athena Scientific, 2nd edi- tion, 1999

    work page 1999

  6. [6]

    Boyd and L

    S. Boyd and L. Vandenberghe.Convex Optimization. Cambridge Uni- versity Press, 2004

    work page 2004

  7. [7]

    Chaloner and K

    K. Chaloner and K. Larntz. Optimal bayesian design applied to logistic regression experiments.Journal of Statistical Planning and Inference, 21(2):191–208, 1989

    work page 1989

  8. [8]

    Chamon and A

    L. Chamon and A. Ribeiro. Approximate supermodularity bounds for experimental design. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wal- lach, R. Fergus, S. Vishwanathan, and R. Garnett, editors,Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017

    work page 2017

  9. [9]

    P. Y. Chen, R. B. Chen, and W. K. Wong. Particle swarm optimization for searching efficient experimental designs: A review.Wiley interdis- ciplinary reviews, 14, 2022. 38

    work page 2022

  10. [10]

    P. Y. Chen, X. Fang, R. B. Chen, S. Zhou, J. J. Lee, and W. K. Wong. Swarm-based search procedure for finding multi-stage designs for phase ii clinical trials.Journal of Computational and Graphical Statistics. in press

    work page

  11. [11]

    Dette and Y

    H. Dette and Y. Grigoriev. E-optimal designs for second-order response surface models.The Annals of Statistics, 42(4):1635–1656, Aug. 2014

    work page 2014

  12. [12]

    Dette and L

    H. Dette and L. M. Haines. E-optimal designs for linear and nonlinear models with two parameters.Biometrika, 81(4):739–754, 1994

    work page 1994

  13. [13]

    Dette, V

    H. Dette, V. Melas, and A. Pepelyshev. Localc- ande-optimal designs for exponential regression models.Ann Inst Stat Math, 58:407–426, 2006

    work page 2006

  14. [14]

    Dette, V

    H. Dette, V. B. Melas, and A. Pepelyshev. Standardized maximin E-optimal designs for the Michaelis-Menten model.Statistica Sinica, pages 1147–1163, 2003

    work page 2003

  15. [15]

    Dette, A

    H. Dette, A. Pepelyshev, and A. Zhigljavsky. Improving updating rules in multiplicative algorithms for computing D-optimal designs.Compu- tational Statistics & Data Analysis, 53(2):312–320, 2008

    work page 2008

  16. [16]

    Dette and W

    H. Dette and W. J. Studden. Geometry of E-optimality.The Annals of Statistics, 21(1):416–433, 1993

    work page 1993

  17. [17]

    Dette and W

    H. Dette and W. K. Wong. E-optimal designs for the michaelis–menten model.Statistics & Probability Letters, 44(4):405–408, 1999

    work page 1999

  18. [18]

    Dobos, Z

    L. Dobos, Z. Banko, and J. Abonyi. Optimal experiment design tech- niques integrated with time-series segmentation.8th IEEE Interna- tional Symposium on Applied Machine Intelligence and Informatics, pages 207–210, 2010

    work page 2010

  19. [19]

    J. Duan, W. Gao, and H. K. T. Ng. Efficient computational algorithm for optimal continuous experimental designs.Journal of Computational and Applied Mathematics, 350:98–113, 2019

    work page 2019

  20. [20]

    B. P. M. Duarte and A. C. Atkinson. Optimum designs for clinical trials in personalized medicine when response variance depends on treatment. Journal of Biopharmaceutical Statistics, page 1–18, 2024

    work page 2024

  21. [21]

    V. V. Fedorov.Theory of optimal experiments. Academic Press, 1972. English translation. 39

    work page 1972

  22. [22]

    Flaherty, A

    P. Flaherty, A. Arkin, and M. Jordan. Robust design of biological experiments. InAdvances in Neural Information Processing Systems, pages 363–370, 2005

    work page 2005

  23. [23]

    Fournier and A

    N. Fournier and A. Guillin. On the rate of convergence in wasser- stein distance of the empirical measure.Probability Theory and Related Fields, 162(3-4):707–738, 2015

    work page 2015

  24. [24]

    L. L. Gao, W.-K. Wong, and J. Zhou. Further insights into E-optimal re- gression designs with applications to biomedical problems. Manuscript, under review, 2025

    work page 2025

  25. [25]

    Heiligers

    B. Heiligers. E-optimal designs in weighted polynomial regression.An- nals of Statistics, 22:917–929, 1994

    work page 1994

  26. [26]

    R. Jin, M. Guerra, Q. Li, and S. Wright. Optimal design for linear models via gradient flow, June 2024

    work page 2024

  27. [27]

    Kennedy and R

    J. Kennedy and R. Eberhart. Particle swarm optimization. InProceed- ings of ICNN’95-international conference on neural networks, volume 4, pages 1942–1948. IEEE, 1995

    work page 1942

  28. [28]

    K. F. Kerr. Optimality criteria for the design of 2-color microarray stud- ies.Statistical applications in genetics and molecular biology, 11(1):Ar- ticle 10, 2012

    work page 2012

  29. [29]

    J. Kiefer. Optimum experimental designs.Journal of the Royal Statis- tical Society. Series B (Methodological), pages 272–319, 1959

    work page 1959

  30. [30]

    J. Kiefer. General equivalence theory for optimum designs (approximate theory).The annals of Statistics, pages 849–879, 1974

    work page 1974

  31. [31]

    L. A. Labadi. Some refinements on fedorov’s algorithms for construct- ing d-optimal designs.Brazilian Journal of Probability and Statistics, 29(1):53–70, 2015

    work page 2015

  32. [32]

    Lanzetti, S

    N. Lanzetti, S. Bolognani, and F. D”orfler. First-order conditions for optimization in the wasserstein space.SIAM Journal on Mathematics of Data Science, 7(1):274–300, 2025

    work page 2025

  33. [33]

    X. Liu, R. Yue, Z. Zhang, and W. K. Wong. G-optimal designs for hi- erarchical linear models: an equivalence theorem and a nature-inspired meta-heuristic algorithm.Soft Computing, 25:13549–13565, 2021. 40

    work page 2021

  34. [34]

    S. B. Love, S. Brown, C. J. Weir, C. Harbron, C. Yap, B. Gaschler- Markefski, J. Matcham, L. Caffrey, C. McKevitt, S. Clive, C. Craddock, J. Spicer, and V. Cornelius. Embracing model-based designs for dose- finding trials.British Journal of Cancer, 117(3):332–339, 2017

    work page 2017

  35. [35]

    Mandal, W

    A. Mandal, W. K. Wong, and Y. Yu. Algorithmic searches for optimal designs.Handbook of Design and Analysis of Experiments, pages 755– 783, 2015

    work page 2015

  36. [36]

    V. B. Melas. Analytical theory of E-optimal designs for polynomial regression. In N. Balakrishnan, V. B. Melas, and S. Ermakov, editors, Advances in Stochastic Simulation Methods, Statistics for Industry and Technology, pages 85–115. Birkh¨ auser, Boston, MA, 2000

    work page 2000

  37. [37]

    Mikaeili

    F. Mikaeili. D-optimum design for cubic without 3-way effect on the simplex.Journal of Statistical Planning and Inference, 21:107–115, 1989

    work page 1989

  38. [38]

    Mikaeili

    F. Mikaeili. D-optimum design for full cubic on q-simplex.Journal of Statistical Planning and Inference, 35(1):121–130, 1993

    work page 1993

  39. [39]

    Nocedal and S

    J. Nocedal and S. J. Wright.Numerical Optimization. Springer, 2nd edition, 2006

    work page 2006

  40. [40]

    Parvu and J

    V. Parvu and J. Morgan. E-optimal designs for three treatments.Jour- nal of Statistical Planning and Inference, 138(3):642–653, 2008

    work page 2008

  41. [41]

    A. J. Pazman.Foundations of Optimal Experimental Designs. Springer, New York, 1986

    work page 1986

  42. [42]

    M. F. Rempel and J. Zhou. On exact k-optimal designs minimizing the condition number.Communications in Statistics - Theory and Methods, 43(6):1114–1131, 2014

    work page 2014

  43. [43]

    Rosa and R

    S. Rosa and R. Harman. Optimal approximate designs for comparison with control in dose-escalation studies.Test, 26(3):638–660, 2017

    work page 2017

  44. [44]

    J. Royle. Exchange algorithms for constructing large spatial designs. Journal of Statistical Planning and Inference, 100(2):121–134, 2002

    work page 2002

  45. [45]

    Sahu and P

    N. Sahu and P. Babu. A new monotonic algorithm for theE-optimal ex- periment design problem.Statistics & Probability Letters, 174:109097, 2021. 41

    work page 2021

  46. [46]

    T. J. Santner, B. J. Williams, and W. I. Notz.The design and analysis of computer experiments. Springer, 2018

    work page 2018

  47. [47]

    R. Simon. Optimal two-stage designs for phase II clinical trials.Con- trolled clinical trials, 10(1):1–10, 1989

    work page 1989

  48. [48]

    L. Smaga. E-optimal and highly e-efficient designs with (mod 4) corre- lated observations.Linear and Multilinear Algebra, 65(11):2293–2305, 2017

    work page 2017

  49. [49]

    I. Y. Smets, K. J. Versyck, and J. F. Van Impe. Optimal control theory: A generic tool for identification and control of (bio-) chemical reactors. Annual Reviews in Control, 26(1):57–73, 2002

    work page 2002

  50. [50]

    K. Smith. On the standard deviations of adjusted and interpolated values of an observed polynomial function and its constants and the guidance they give towards a proper choice of the distribution of obser- vations.Biometrika, 12:1–85, 1918

    work page 1918

  51. [51]

    Smucker, M

    B. Smucker, M. Krzywinski, and N. Altman. Optimal experimental design.Nature Methods, 15(8):559–560, 2018

    work page 2018

  52. [52]

    Stokes, A

    Z. Stokes, A. Mandal, and W. K. Wong. Using differential evolution to design optimal experiments.Chemometrics and Intelligent Laboratory Systems, 199:103955, 2020

    work page 2020

  53. [53]

    Stokes, W

    Z. Stokes, W. K. Wong, and H. Xu. Metaheuristic solutions to order- of-addition design problems.Journal of Computational and Graphical Statistics, 33(3):1006–1016, 2024

    work page 2024

  54. [54]

    Telen, N

    D. Telen, N. Van Riet, F. Logist, and J. Van Impe. A differentiable reformulation for e-optimal design of experiments in nonlinear dynamic biosystems.Mathematical Biosciences, 264:1–7, 2015

    work page 2015

  55. [55]

    K. C. Toh, M. J. Todd, and R. H. T¨ ut¨ unc¨ u. SDPT3 — a Matlab software package for semidefinite programming.Optimization Methods and Software, 11(1-4):545–581, Jan. 1999

    work page 1999

  56. [56]

    R. H. Tutuncu, K. C. Toh, and M. J. Todd. Solving semidefinite- quadratic-linear programs using SDPT3.Mathematical Programming, 95(2):189–217, Feb. 2003

    work page 2003

  57. [57]

    Villani.Optimal Transport: Old and New

    C. Villani.Optimal Transport: Old and New. Grundlehren der mathe- matischen Wissenschaften. Springer, 2009. 42

    work page 2009

  58. [58]

    J. M. Whitacre. Recent trends indicate rapid growth of nature-inspired optimization in academia and industry.Computing, 93:121–133, 2011

    work page 2011

  59. [59]

    J. M. Whitacre. Survival of the flexible: explaining the recent popu- larity of nature-inspired optimization within a rapidly evolving world. Computing, 93(2):135–146, 2011

    work page 2011

  60. [60]

    W. Wong, Y. Ryeznik, O. Sverdlov, P. Y. Chen, X. Fang, R. Chen, S. Zhou, and J. Lee. Nature-inspired metaheuristics for optimizing dose-finding and computationally challenging clinical trial designs.Clin Trials, 22:422–429, 2025

    work page 2025

  61. [61]

    H. P. Wynn. The sequential generation ofd-optimum experimental designs.The Annals of Mathematical Statistics, 41(5):1655–1664, 1970

    work page 1970

  62. [62]

    H. P. Wynn. Results in the theory and construction of d-optimum experimental designs.Journal of the Royal Statistical Society Series B: Statistical Methodology, 34(2):133–147, 1972

    work page 1972

  63. [63]

    W. Xu, W. K. Wong, K. C. Tan, and J. X. Xu. Finding high- dimensional d-optimal designs for logistic models via differential evolu- tion.IEEE Access, 7(1):7133–7146, 2019

    work page 2019

  64. [64]

    J. J. Ye and J. Zhou. Minimizing the condition number to construct design points for polynomial regression models.SIAM Journal on Op- timization, 23(1):666–686, 2013

    work page 2013

  65. [65]

    Z. Yue, X. Zhang, P. V. D. Driessche, and J. Zhou. Constructing K- optimal designs for regression models.Statistical Papers, 64:205–226, 2023. 43

    work page 2023