pith. sign in

arxiv: 2605.04691 · v1 · submitted 2026-05-06 · 🧮 math.OC

Uncertainty Quantification Methods for Optimal Excitation Design in Parameter Identification

Kevin Schmidt , Nicola Henkelmann , Christoph Mark , Johannes von Keler This is my paper

Pith reviewed 2026-05-08 16:45 UTC · model grok-4.3

classification 🧮 math.OC
keywords parameter identificationoptimal excitation designuncertainty quantificationpolynomial chaos expansionWasserstein distancesensitivity analysisoptimal transportvehicle dynamics
0
0 comments X

The pith

Optimal excitations for parameter identification are designed by maximizing global sensitivity of target parameters using two new uncertainty quantification techniques.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a systematic way to choose input signals that give experiments enough information to identify specific model parameters reliably. It does so by maximizing the global sensitivity of those parameters over all possible excitations. Two complementary methods solve the high computational cost of repeated sensitivity checks during this optimization. An intrusive polynomial chaos expansion builds fast surrogate models when the system equations are known. A non-intrusive approach based on Wasserstein distances from optimal transport quantifies sensitivity for black-box models without needing internal dynamics. Both are shown to work on vehicle dynamics models, with experimental confirmation on a test vehicle that the resulting excitations outperform standard choices.

Core claim

Optimal excitations are obtained by maximizing the global sensitivity of target parameters across the space of possible excitation functions; this optimization is made tractable by an intrusive polynomial chaos expansion surrogate for systems with known structure and by a novel non-intrusive sensitivity measure that employs Wasserstein distances to compare output distributions without reference to internal equations, yielding excitations with higher information content for parameter estimation as validated on vehicle dynamics models.

What carries the argument

Global sensitivity maximization over excitation functions, evaluated either by intrusive polynomial chaos expansion surrogates or by non-intrusive Wasserstein-distance comparisons of output distributions.

If this is right

  • Sensitivity evaluation becomes fast enough to embed inside an outer optimization loop over excitation functions.
  • The non-intrusive Wasserstein method extends optimal excitation design to black-box simulation codes that cannot be modified.
  • Both approaches lower the experimental effort needed to reach reliable parameter values in engineering models.
  • Vehicle-dynamics identification improves measurably when the designed excitations replace standard test signals.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The Wasserstein-based sensitivity could be paired with existing gradient-free optimizers to handle even larger excitation spaces.
  • The same sensitivity-maximization idea may transfer directly to excitation design for parameter identification in robotics or chemical process models.
  • Real-time adaptation of the excitation during an ongoing experiment could be built on top of the non-intrusive distance measure.

Load-bearing premise

Maximizing the global sensitivity of the target parameters across possible excitations will produce signals that contain enough information for reliable and accurate parameter estimation.

What would settle it

A controlled simulation or vehicle experiment in which excitations selected by the proposed sensitivity-maximization procedure produce parameter estimates whose accuracy is no better than, or worse than, estimates obtained from conventional or random excitations.

Figures

Figures reproduced from arXiv: 2605.04691 by Christoph Mark, Johannes von Keler, Kevin Schmidt, Nicola Henkelmann.

Figure 1
Figure 1. Figure 1: Sensitivities of the spring damper system. view at source ↗
Figure 2
Figure 2. Figure 2: Overall sensitivity scores. The total height indicates the total sensitivity score of the spring damper system, the dark blue part the effective one. parameters are uniformly distributed with q = 2 and c ∼ U(1.8 N/m, 2.2 N/m), d ∼ U(0.9 Ns/m, 1.1 Ns/m). In addition, the displacement x(t) is considered as the scalar output of the system with m = 1. Using a PCE-order of d = 3, the resulting surrogate model (… view at source ↗
Figure 4
Figure 4. Figure 4: Sampling error for different sample sizes. Infinity norm error between the intrusive and non-intrusive method computed over M = 100 experiments. The box plots illustrate the median, the 25th and 75th percentiles, while the whiskers denote the outliers. Definition 1. Given an empirical distribution P Ns y , we define the sample mean and variance as µˆ y B N −1 s X Ns i=1 yi , Σˆ y B N −1 s X Ns i=1 (yi − µˆ… view at source ↗
Figure 3
Figure 3. Figure 3: Exemplary scatter plot. Scatter plot with Ns = 30 samples (θi , yi) divided into M = 6 bins. p1 and p2 denote the frequencies of xi being located in bins Θ1 and Θ2, respectively view at source ↗
Figure 5
Figure 5. Figure 5: Steps to solve the optimization problem (44). view at source ↗
Figure 6
Figure 6. Figure 6: Spring damper system with optimal inputs. view at source ↗
Figure 8
Figure 8. Figure 8: Illustration of the lateral vehicle dynamics. view at source ↗
Figure 7
Figure 7. Figure 7: Shape of the cost function for identification and resulting estimates. view at source ↗
Figure 10
Figure 10. Figure 10: Test vehicle ID.Buzz on the test track. notice that both models exhibit similar sensitivity characteristics, however, due to the neglected dynamics, the linear model does not consider the sensitivity in steady-state. In view at source ↗
Figure 11
Figure 11. Figure 11: Identification results for a reference input set and two optimized inputs. Each row of this figure corresponds to one maneuver; the row label on the left indicates whether it belongs to the reference set (Ref. #1–#5, orange labels) or to the optimized set (Opt. #1–#2, blue labels). The columns show, from left to right, the desired and actual steering angles u, δ, the yaw-rate residual ψ˙ − ψ˙meas, the lon… view at source ↗
Figure 12
Figure 12. Figure 12: Ranking of maneuver sensitivities. Comparison of the overall impact score ∥∆S ∥1 for the reference maneuvers (orange) and optimized maneuvers (blue) from view at source ↗
read the original abstract

Parameter identification is crucial in virtual engineering processes, yet determining appropriate system excitations for identifying specific parameters remains challenging. In practice, extensive experimental programs often fail to generate data with sufficient information content for reliable parameter estimation. This work presents a systematic approach for deriving optimal excitations by maximizing the global sensitivity of target parameters across the space of possible excitation functions. To address the computational challenge of sensitivity evaluation during optimization, we develop two complementary approaches based on uncertainty quantification (UQ) methods. For systems with known mathematical structure, we present an intrusive polynomial chaos expansion (PCE) method that constructs deterministic surrogate models, enabling rapid sensitivity computation. For black-box models where intrusive approaches are not feasible, we introduce a novel non-intrusive method based on optimal transport theory, specifically using Wasserstein distances to quantify sensitivity measures without requiring knowledge of internal system dynamics. Both methods significantly reduce computational costs, making optimal excitation design practical for complex engineering systems. We demonstrate the effectiveness of both approaches on vehicle dynamics models, showing substantial improvements in parameter identification capability. The benefit for parameter identification is further validated experimentally on a test vehicle and compared to the state of the art.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 3 minor

Summary. The paper develops two UQ-based methods to optimize system excitations for parameter identification by maximizing global sensitivity of target parameters. For models with known structure, an intrusive PCE surrogate enables fast deterministic sensitivity computation; for black-box models, a non-intrusive Wasserstein-distance approach from optimal transport quantifies sensitivities without internal dynamics. Both reduce computational cost relative to direct Monte Carlo. Effectiveness is demonstrated on vehicle-dynamics models, with experimental validation on a test vehicle claiming substantial gains over state-of-the-art excitations.

Significance. If the chosen sensitivity indices are shown to be monotonically related to reduced posterior variance or estimator covariance, the work would provide a practical, computationally tractable framework for excitation design in engineering systems where exhaustive testing is infeasible. The dual intrusive/non-intrusive formulation is a clear strength, offering flexibility across modeling regimes while leveraging established PCE and optimal-transport tools.

major comments (3)
  1. [§3 and §4] §3 (PCE method) and §4 (Wasserstein method): the central claim that maximizing the global sensitivity index (Sobol-type or Wasserstein distance between output measures) produces excitations with sufficient information content for reliable parameter estimation is not supported by a derivation or inequality relating the index to the reduction in posterior covariance or Fisher information. When parameters are correlated or the output map is nonlinear, this equivalence does not hold in general; no counter-example check or bounding argument is supplied.
  2. [§5.3] §5.3 (vehicle-dynamics demonstrations) and experimental section: only increases in the sensitivity index are reported for the optimized excitations. No direct metrics of identification quality—such as RMSE of recovered parameters, condition number of the estimator covariance, or posterior variance reduction—are provided for the optimized versus baseline excitations, leaving the claimed improvement in parameter identification unverified.
  3. [§4.2] §4.2 (non-intrusive Wasserstein formulation): the sensitivity measure is defined via Wasserstein distance between output distributions, yet the manuscript does not address how this distance behaves under non-Gaussian measurement noise or when the parameter-to-output map exhibits trade-offs; the optimization may therefore select excitations that are sensitive but do not resolve the target parameters.
minor comments (3)
  1. [§2] Notation for the excitation function space and the sensitivity functional is introduced without a compact summary table; a single table collecting definitions would improve readability.
  2. [experimental validation] The experimental validation section references 'state of the art' without citing the specific prior excitation-design methods used for comparison.
  3. [§5] Figure captions for the vehicle-dynamics results do not state the number of Monte Carlo samples or PCE order used, making reproducibility difficult.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive and detailed comments. These have highlighted important aspects regarding the theoretical justification and empirical validation of our methods. We address each major comment point by point below, indicating the revisions we will incorporate.

read point-by-point responses

  1. Referee: [§3 and §4] §3 (PCE method) and §4 (Wasserstein method): the central claim that maximizing the global sensitivity index (Sobol-type or Wasserstein distance between output measures) produces excitations with sufficient information content for reliable parameter estimation is not supported by a derivation or inequality relating the index to the reduction in posterior covariance or Fisher information. When parameters are correlated or the output map is nonlinear, this equivalence does not hold in general; no counter-example check or bounding argument is supplied.

    Authors: We acknowledge that the manuscript does not contain a rigorous derivation or inequality that directly equates maximization of the chosen global sensitivity indices to reductions in posterior covariance or the Fisher information matrix. The approach is motivated by the widespread use of sensitivity-based criteria in optimal experimental design literature as practical proxies for identifiability. We agree that this proxy relationship requires explicit caveats for correlated parameters and nonlinear maps. In the revision we will add a dedicated discussion subsection (in §3 and §4) that (i) cites relevant sensitivity-based design references, (ii) provides a simple analytical bounding argument under additive Gaussian noise and linear output maps, and (iii) includes a brief counter-example illustrating the breakdown when strong correlations or nonlinearities are present. revision: yes

  2. Referee: [§5.3] §5.3 (vehicle-dynamics demonstrations) and experimental section: only increases in the sensitivity index are reported for the optimized excitations. No direct metrics of identification quality—such as RMSE of recovered parameters, condition number of the estimator covariance, or posterior variance reduction—are provided for the optimized versus baseline excitations, leaving the claimed improvement in parameter identification unverified.

    Authors: The current demonstrations emphasize the increase in sensitivity indices, while the experimental section reports improved parameter recovery relative to state-of-the-art excitations. To directly verify the identification-quality claim, we will augment §5.3 and the experimental results with the requested metrics: RMSE of recovered parameters, condition numbers of the estimator covariance matrices, and estimated posterior variance reductions, all computed for the optimized excitations versus the baseline and state-of-the-art cases. revision: yes

  3. Referee: [§4.2] §4.2 (non-intrusive Wasserstein formulation): the sensitivity measure is defined via Wasserstein distance between output distributions, yet the manuscript does not address how this distance behaves under non-Gaussian measurement noise or when the parameter-to-output map exhibits trade-offs; the optimization may therefore select excitations that are sensitive but do not resolve the target parameters.

    Authors: The Wasserstein distance quantifies discrepancy between output distributions induced by parameter perturbations and is in principle compatible with any noise distribution for which the push-forward measures can be estimated. We agree that the manuscript does not explicitly examine robustness to non-Gaussian noise or the effect of parameter trade-offs. In the revision we will expand §4.2 with a short analysis of these issues and add a numerical illustration (using the same vehicle model) that compares the selected excitations under Gaussian versus heavy-tailed noise and under a deliberately introduced parameter trade-off. revision: yes

Circularity Check

0 steps flagged

No significant circularity; derivation uses independent UQ constructions

full rationale

The paper defines optimal excitations via maximization of global sensitivity indices computed either through intrusive PCE surrogate models or non-intrusive Wasserstein distances on output measures. These are presented as applications of established UQ and optimal transport tools to the excitation-design problem. No equation reduces the target information content or posterior variance to the chosen sensitivity index by definition, no fitted parameter is relabeled as a prediction, and no load-bearing premise rests solely on self-citation. The assumption that sensitivity maximization yields informative excitations is stated as a modeling choice rather than derived from the method itself.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review prevents identification of specific free parameters, axioms, or invented entities; none are explicitly named in the provided text.

pith-pipeline@v0.9.0 · 5498 in / 1198 out tokens · 53906 ms · 2026-05-08T16:45:32.008437+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 40 canonical work pages

  1. [1]

    Prentice Hall, Upper Saddle River, NJ, 2nd edition, 1999

    Lennart Ljung.System identification: theory for the user. Prentice Hall, Upper Saddle River, NJ, 2nd edition, 1999

    work page 1999

  2. [2]

    Optimal input signals for parameter estimation in dynamic systems–Survey and new results.IEEE Transactions on Automatic Control, 19(6):753–768, 2003

    Raman Mehra. Optimal input signals for parameter estimation in dynamic systems–Survey and new results.IEEE Transactions on Automatic Control, 19(6):753–768, 2003

    work page 2003

  3. [3]

    John Wiley & Sons, Hoboken, NJ, 2017

    Johan Schoukens, Rik Pintelon, and Yves Rolain.Mastering system identification in 100 exercises. John Wiley & Sons, Hoboken, NJ, 2017

    work page 2017

  4. [4]

    Academic Press, San Diego, CA, 2001

    Douglas F Elliott and K Ramamohan Rao.Signals and systems. Academic Press, San Diego, CA, 2001

    work page 2001

  5. [5]

    John Wiley & Sons, Hoboken, NJ, 2nd edition, 2012

    Rik Pintelon and Johan Schoukens.System identification: a frequency domain approach. John Wiley & Sons, Hoboken, NJ, 2nd edition, 2012

    work page 2012

  6. [6]

    Optimal experiment design with respect to the intended model application.Automatica, 22:543–554, 1986

    Michel Gevers and Lennart Ljung. Optimal experiment design with respect to the intended model application.Automatica, 22:543–554, 1986

    work page 1986

  7. [7]

    Academic Press, New York, NY , 1972

    Valerii Vasilievich Fedorov.Theory of optimal experiments. Academic Press, New York, NY , 1972

    work page 1972

  8. [8]

    Oxford University Press, Oxford, UK, 2007

    Anthony Atkinson, Alexander Donev, and Randall Tobias.Optimum Experimental Designs, With SAS. Oxford University Press, Oxford, UK, 2007

    work page 2007

  9. [9]

    Global sensitivity analysis: the primer

    Andrea Saltelli, Marco Ratto, Terry Andres, Francesca Campolongo, Jes- sica Cariboni, Debora Gatelli, Michaela Saisana, and Stefano Tarantola. Global sensitivity analysis: the primer. John Wiley & Sons, 2008

    work page 2008

  10. [10]

    Springer Science & Business Media, 2010

    Olivier Le Maître and Omar M Knio.Spectral methods for uncer- tainty quantification: with applications to computational fluid dynamics. Springer Science & Business Media, 2010

    work page 2010

  11. [11]

    Global sensitivity analysis using polynomial chaos expan- sions.Reliability engineering&system safety, 93(7):964–979, 2008

    Bruno Sudret. Global sensitivity analysis using polynomial chaos expan- sions.Reliability engineering&system safety, 93(7):964–979, 2008

    work page 2008

  12. [12]

    Optimal experimental design for probabilistic model discrimination using polynomial chaos

    Stefan Streif, Felix Petzke, Ali Mesbah, Rolf Findeisen, and Richard D Braatz. Optimal experimental design for probabilistic model discrimination using polynomial chaos. InProc. IFAC world congress, pages 4103–4109, 2014

    work page 2014

  13. [13]

    An efficient method for stochastic optimal control with joint chance constraints for nonlinear systems.International Journal of Robust and Nonlinear Control, 29(15):5017–5037, 2019

    Joel A Paulson and Ali Mesbah. An efficient method for stochastic optimal control with joint chance constraints for nonlinear systems.International Journal of Robust and Nonlinear Control, 29(15):5017–5037, 2019

    work page 2019

  14. [14]

    Stability for receding- horizon stochastic model predictive control

    Joel A Paulson, Stefan Streif, and Ali Mesbah. Stability for receding- horizon stochastic model predictive control. InProc. American Control Conference (ACC), pages 937–943. IEEE, 2015

    work page 2015

  15. [15]

    Texas A&M University, 2008

    James Robert Fisher.Stability analysis and control of stochastic dynamic systems using polynomial chaos. Texas A&M University, 2008

    work page 2008

  16. [16]

    Wiley, Hoboken, NJ, 2003

    Thomas P Ryan.Modern experimental design. Wiley, Hoboken, NJ, 2003

    work page 2003

  17. [17]

    Bayesian experimental design: a review.Statistical Science, 10(3):273–304, 1995

    Kathryn Chaloner and Isabella Verdinelli. Bayesian experimental design: a review.Statistical Science, 10(3):273–304, 1995

    work page 1995

  18. [18]

    Bayesian emulation of complex multi-output and dynamic computer models.Journal of Statistical Plan- ning and Inference, 140(3):640–651, 2014

    Stefano Conti and Anthony O’Hagan. Bayesian emulation of complex multi-output and dynamic computer models.Journal of Statistical Plan- ning and Inference, 140(3):640–651, 2014

    work page 2014

  19. [19]

    Sensitivity estimates for nonlinear mathematical models.Math

    Ilya Meyerovich Sobo´l. Sensitivity estimates for nonlinear mathematical models.Math. Model. Comput. Exp., 1(4):407–414, 1993

    work page 1993

  20. [20]

    On distributionally robust chance-constrained linear programs.Journal of Optimization Theory and Applications, 130(1):1–22, 2006

    Giuseppe Carlo Calafiore and L El Ghaoui. On distributionally robust chance-constrained linear programs.Journal of Optimization Theory and Applications, 130(1):1–22, 2006

    work page 2006

  21. [21]

    Multivariate chebyshev inequalities

    Albert W Marshall and Ingram Olkin. Multivariate chebyshev inequalities. The Annals of Mathematical Statistics, pages 1001–1014, 1960

    work page 1960

  22. [22]

    Polynomial chaos expansions and stochastic finite element methods.Risk and reliability in geotechnical engineering, pages 265–300, 2014

    Bruno Sudret. Polynomial chaos expansions and stochastic finite element methods.Risk and reliability in geotechnical engineering, pages 265–300, 2014

    work page 2014

  23. [23]

    The homogeneous chaos.American Journal of Mathe- matics, 60(4):897–936, 1938

    Norbert Wiener. The homogeneous chaos.American Journal of Mathe- matics, 60(4):897–936, 1938

    work page 1938

  24. [24]

    The Wiener–Askey polynomial chaos for stochastic differential equations.SIAM journal on scientific computing, 24(2):619–644, 2002

    Dongbin Xiu and George Em Karniadakis. The Wiener–Askey polynomial chaos for stochastic differential equations.SIAM journal on scientific computing, 24(2):619–644, 2002

    work page 2002

  25. [25]

    Comments on truncation errors for polynomial chaos ex- pansions.IEEE Control Systems Letters, 2(1):169–174, 2017

    Tillmann Mühlpfordt, Rolf Findeisen, Veit Hagenmeyer, and Timm Faulwasser. Comments on truncation errors for polynomial chaos ex- pansions.IEEE Control Systems Letters, 2(1):169–174, 2017

    work page 2017

  26. [26]

    Springer, 2008

    Cédric Villani.Optimal transport: old and new. Springer, 2008

    work page 2008

  27. [27]

    On a formula for the L2 Wasserstein metric between measures on Euclidean and Hilbert spaces.Mathematische Nachrichten, 147(1):185–203, 1990

    Matthias Gelbrich. On a formula for the L2 Wasserstein metric between measures on Euclidean and Hilbert spaces.Mathematische Nachrichten, 147(1):185–203, 1990

    work page 1990

  28. [28]

    An extension of kakutani’s theorem on infinite product measures to the tensor product of semifinite w*-algebras.Transactions of the American Mathematical Society, 135:199–212, 1969

    Donald Bures. An extension of kakutani’s theorem on infinite product measures to the tensor product of semifinite w*-algebras.Transactions of the American Mathematical Society, 135:199–212, 1969

    work page 1969

  29. [29]

    Global sensitivity analysis via optimal transport.Management Science, 2024

    Emanuele Borgonovo, Alessio Figalli, Elmar Plischke, and Giuseppe Savaré. Global sensitivity analysis via optimal transport.Management Science, 2024

    work page 2024

  30. [30]

    Survey of numerical methods for trajectory optimization

    John T Betts. Survey of numerical methods for trajectory optimization. Journal of guidance, control, and dynamics, 21:193–207, 1998

    work page 1998

  31. [31]

    Princeton university press, 2011

    Daniel Liberzon.Calculus of variations and optimal control theory: a concise introduction. Princeton university press, 2011

    work page 2011

  32. [32]

    Optimal experiment design with respect to the intended model application.Automatica, 41:1159–1167, 2005

    Michel Gevers, Xavier Bombois, Benoit Codrons, Gérard Scorletti, and Brian DO Anderson. Optimal experiment design with respect to the intended model application.Automatica, 41:1159–1167, 2005

    work page 2005

  33. [33]

    On the numerical condition of polynomials in bernstein form.Computer Aided Geometric Design, 4(3):191–216, 1987

    Rida T Farouki and VT Rajan. On the numerical condition of polynomials in bernstein form.Computer Aided Geometric Design, 4(3):191–216, 1987

    work page 1987

  34. [34]

    Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces.Journal of Global Optimization, 11(4):341–359, 1997

    Rainer Storn and Kenneth Price. Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces.Journal of Global Optimization, 11(4):341–359, 1997

    work page 1997

  35. [35]

    Practical bayesian op- timization of machine learning algorithms.Advances in neural information processing systems, 25, 2012

    Jasper Snoek, Hugo Larochelle, and Ryan P Adams. Practical bayesian op- timization of machine learning algorithms.Advances in neural information processing systems, 25, 2012

    work page 2012

  36. [36]

    SciPy 1.0: fundamental algorithms for scientific computing in Python.Nature Methods, 17(3):261–272, 2020

    Pauli Virtanen, Ralf Gommers, Travis E Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, et al. SciPy 1.0: fundamental algorithms for scientific computing in Python.Nature Methods, 17(3):261–272, 2020

    work page 2020

  37. [37]

    Eric Tseng, and Davor Hrovat

    Paolo Falcone, Francesco Borrelli, Jahan Asgari, H. Eric Tseng, and Davor Hrovat. Predictive Active Steering Control for Autonomous Vehicle 16 Systems.IEEE Transactions on Control Systems Technology, 15:566–580, 2007

    work page 2007

  38. [38]

    A. T. van Zanten. Bosch ESP Systems: 5 Years of Experience.SAE Transactions, 109:428–436, 2000

    work page 2000

  39. [39]

    Jarrod M. Snider. Automatic Steering Methods for Autonomous Auto- mobile Path Tracking.Robotics Institute, Carnegie Mellon University, 2009

    work page 2009

  40. [40]

    Elsevier, 2005

    Hans Pacejka.Tire and vehicle dynamics. Elsevier, 2005. 17

    work page 2005