pith. sign in

arxiv: 2411.02740 · v5 · submitted 2024-11-05 · 💻 cs.LG · cond-mat.mtrl-sci· physics.app-ph· physics.comp-ph· physics.data-an

An information-matching approach to optimal experimental design and active learning

Pith reviewed 2026-05-23 18:11 UTC · model grok-4.3

classification 💻 cs.LG cond-mat.mtrl-sciphysics.app-phphysics.comp-phphysics.data-an
keywords information matchingoptimal experimental designactive learningFisher Information Matrixquantities of interestsloppy parametersconvex optimization
0
0 comments X

The pith

An information-matching criterion selects training data that constrain only the parameters needed for quantities of interest.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces an information-matching criterion based on the Fisher Information Matrix to select the most informative training data from a candidate pool. Models often contain many unidentifiable parameters, yet quantities of interest typically depend on fewer parameter combinations. The criterion ensures selected data supply enough information to learn only those relevant combinations. Formulated as a convex optimization problem, the method scales to large models and is demonstrated in power systems, underwater acoustics, and active learning for material science, where small optimal datasets suffice for precise predictions.

Core claim

The central claim is that an information-matching criterion based on the Fisher Information Matrix selects the most informative training data from a candidate pool such that the selected data contain sufficient information to learn only those parameters that are needed to constrain downstream QoIs; the criterion is formulated as a convex optimization problem.

What carries the argument

The information-matching criterion based on the Fisher Information Matrix, which matches the information content of candidate data points to the information required to constrain the QoIs.

If this is right

  • A relatively small set of optimal training data suffices to achieve precise QoI predictions.
  • The convex formulation makes the selection scalable to large models and datasets.
  • The criterion serves as an effective query function inside active learning loops.
  • The approach applies across modeling problems in power systems, underwater acoustics, and material science.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Experimental costs could drop in domains where each measurement is expensive.
  • The same logic might apply to other domains that rely on sloppy models, such as chemical kinetics.
  • Integration into large-scale machine-learning pipelines could reduce the data volume needed for downstream tasks.

Load-bearing premise

Models often contain many unidentifiable sloppy parameters while quantities of interest depend on a relatively small number of parameter combinations.

What would settle it

A controlled test in which data chosen by the information-matching criterion produce worse QoI prediction accuracy than randomly chosen data of equal size, when both are used to train the same model.

Figures

Figures reproduced from arXiv: 2411.02740 by 2, (2) Achilles Heel Technologies, 3) ((1) Brigham Young University, (3) SLAC National Accelerator Laboratory, (4) University of Electronic Science, (5) University of Minnesota, (6) Lawrence Livermore National Laboratory), Alex M. Stankovic (3), Benjamin L. Francis (2), CA, Chengdu, China, Ellad B. Tadmor (5), Ilia Nikiforov (5), Mark K. Transtrum (1, Menlo Park, Mingjian Wen (4), Minneapolis, MN, Orem, Provo, Technology of China, Tracianne B. Neilsen (1), USA, UT, Vasily V. Bulatov (6), Vincenzo Lordi (6), Yonatan Kurniawan (1).

Figure 1
Figure 1. Figure 1: FIG. 1 [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: as buses are highlighted in different colors based on their respective areas. Notably, there are overlaps (double-highlighted buses) between optimal PMU place￾ments for full and (the union of) three subnetworks, while non-overlapping locations are a consequence of enforcing observability for each of the subnetworks separately. Next, we consider an optimal sensor placement prob￾lem in passive acoustic sourc… view at source ↗
Figure 3
Figure 3. Figure 3: FIG. 3 [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: FIG. 4 [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 1
Figure 1. Figure 1: FIG. 1 [PITH_FULL_IMAGE:figures/full_fig_p013_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: FIG. 2 [PITH_FULL_IMAGE:figures/full_fig_p016_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: FIG. 3 [PITH_FULL_IMAGE:figures/full_fig_p017_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: FIG. 4 [PITH_FULL_IMAGE:figures/full_fig_p018_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: FIG. 5 [PITH_FULL_IMAGE:figures/full_fig_p021_5.png] view at source ↗
read the original abstract

The efficacy of mathematical models heavily depends on the quality of the training data, yet collecting sufficient data is often expensive and challenging. Many modeling applications require inferring parameters only as a means to predict other quantities of interest (QoI). Because models often contain many unidentifiable (sloppy) parameters, QoIs often depend on a relatively small number of parameter combinations. Therefore, we introduce an information-matching criterion based on the Fisher Information Matrix to select the most informative training data from a candidate pool. This method ensures that the selected data contain sufficient information to learn only those parameters that are needed to constrain downstream QoIs. It is formulated as a convex optimization problem, making it scalable to large models and datasets. We demonstrate the effectiveness of this approach across various modeling problems in diverse scientific fields, including power systems and underwater acoustics. Finally, we use information-matching as a query function within an Active Learning loop for material science applications. In all these applications, we find that a relatively small set of optimal training data can provide the necessary information for achieving precise predictions. These results are encouraging for diverse future applications, particularly active learning in large machine learning models.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 4 minor

Summary. The manuscript proposes an information-matching criterion derived from the Fisher Information Matrix to select training data for optimal experimental design and active learning. The approach exploits model sloppiness by ensuring selected data constrain only the low-dimensional parameter combinations needed to predict downstream quantities of interest (QoIs), rather than all parameters. It is formulated as a convex optimization problem and demonstrated on applications in power systems, underwater acoustics, and active learning for materials science, where small optimal datasets suffice for precise QoI predictions.

Significance. If the central claims hold, the work provides a scalable, QoI-focused alternative to standard OED that can reduce data collection costs in sloppy models common across scientific domains. The convex formulation and cross-field demonstrations (power systems, acoustics, materials) are strengths that support potential adoption in active learning pipelines for large models. The emphasis on matching information content to QoI sensitivity rather than full identifiability is a clear conceptual advance over conventional D-optimal or A-optimal designs.

minor comments (4)
  1. Abstract: the statement that 'a relatively small set of optimal training data can provide the necessary information for achieving precise predictions' should be supported by explicit quantitative metrics (e.g., QoI error reduction factors or confidence interval widths) rather than qualitative description.
  2. The integration of the information-matching query function into the active learning loop (final application) would benefit from a pseudocode listing or explicit comparison against standard uncertainty-sampling baselines to clarify the incremental benefit.
  3. Notation: the distinction between the full Fisher Information Matrix and the projected QoI-relevant submatrix should be introduced with consistent symbols early in the methods section to avoid reader confusion when moving between the convex program and the application results.
  4. Figure captions for the application results should include the size of the candidate pool and the fraction of data selected, to allow direct assessment of data efficiency claims.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for their positive summary and significance assessment of our manuscript, as well as for recommending minor revision. We appreciate the recognition that our information-matching approach provides a scalable, QoI-focused alternative to standard OED for sloppy models.

Circularity Check

0 steps flagged

No significant circularity identified

full rationale

The paper proposes an information-matching criterion based on the Fisher Information Matrix, formulated as a convex optimization problem to select training data that constrains only the parameter combinations relevant to downstream QoIs. This builds directly on the standard observation of sloppy models and low-dimensional QoI dependence, which is an external premise from the literature rather than a self-derived input. No equations or steps in the provided abstract reduce by construction to fitted parameters or self-citations; the method is presented as a scalable formulation with demonstrations across independent applications (power systems, acoustics, materials) serving as external checks. The derivation chain remains self-contained against established optimal experimental design benchmarks without load-bearing self-referential reductions.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

Approach rests on standard statistical assumptions about the Fisher Information Matrix and the premise of sloppy models; no free parameters, invented entities, or ad-hoc axioms are identifiable from the abstract.

axioms (2)
  • domain assumption The Fisher Information Matrix captures the relevant information content of data for model parameters.
    Core premise of the proposed criterion, standard in statistics.
  • domain assumption Quantities of interest depend on a relatively small number of parameter combinations in models with many unidentifiable parameters.
    Explicitly stated in the abstract as the motivation for the method.

pith-pipeline@v0.9.0 · 5897 in / 936 out tokens · 39169 ms · 2026-05-23T18:11:05.624951+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

48 extracted references · 48 canonical work pages

  1. [1]

    Learn on the Fly

    Conference Name: IEEE Transactions on Power Systems. 8F Soudi and K Tomsovic. Optimal distribution protection des ign: quality of solution and computational analysis. International Journal of Electrical Power & Energy Systems , 21(5):327–335, June 1999. 9D. A. Wood and D. J. Allwright. Optimisation of hydrophone placement: a dynamical systems approach. Eu...

  2. [3]

    Precision

    David L. Streiner and Geoffrey R. Norman. “Precision” and “Accu racy”: Two Terms That Are Neither. Journal of Clinical Epidemiology, 59(4):327–330, April 2006

  3. [4]

    Transtrum, Benjamin B

    Mark K. Transtrum, Benjamin B. Machta, and James P. Sethna. Ge ometry of nonlinear least squares with applications to sloppy models and optimization. Physical Review E , 83(3):036701, March 2011. Publisher: American Physical Soci ety

  4. [5]

    CVXPY: A Python-embedde d modeling language for convex optimization

    Steven Diamond and Stephen Boyd. CVXPY: A Python-embedde d modeling language for convex optimization. Journal of Machine Learning Research , 17(83):1–5, 2016

  5. [6]

    A rewriting system for convex optimization problems

    Akshay Agrawal, Robin Verschueren, Steven Diamond, and Step hen Boyd. A rewriting system for convex optimization problems. Journal of Control and Decision , 5(1):42–60, 2018. 11 -1 0 1 2 3 a a 0 (Å) 0 1 2 3 4 5E E c (eV) T arget Optimal (a) G X U L G K 0.0 0.01 0.02 0.03Energy (eV) T arget Optimal (b) FIG. 4: Uncertainties of (a) the energy E as a functio...

  6. [7]

    Implementation and evaluation of sdpa 6.0 (semidefinite programming algorithm 6.0)

    Makoto Yamashita, Katsuki Fujisawa, and Masakazu Kojima. Implementation and evaluation of sdpa 6.0 (semidefinite programming algorithm 6.0). Optimization Methods and Software , 18(4):491–505, 2003

  7. [8]

    Latest Developments in the SDPA Family for Solving Large-Scale SDPs , pages 687–713

    Makoto Yamashita, Katsuki Fujisawa, Mituhiro Fukuda, Kaz uhiro Kobayashi, Kazuhide Nakata, and Maho Nakata. Latest Developments in the SDPA Family for Solving Large-Scale SDPs , pages 687–713. Springer US, Boston, MA, 2012

  8. [9]

    A numerical evaluation of highly accurate mul tiple-precision arithmetic version of semidefinite programming solver: Sdpa-gmp, -qd and -dd

    Maho Nakata. A numerical evaluation of highly accurate mul tiple-precision arithmetic version of semidefinite programming solver: Sdpa-gmp, -qd and -dd. In 2010 IEEE International Symposium on Computer-Aided Contr ol System Design , pages 29–34, 2010

  9. [10]

    Exploiting sparsity in linear and nonlinear matrix inequalities via positive semidefinite matrix complet ion

    Sunyoung Kim, Masakazu Kojima, Martin Mevissen, and Makot o Yamashita. Exploiting sparsity in linear and nonlinear matrix inequalities via positive semidefinite matrix complet ion. Mathematical Programming, 129(1):33–68, Sep 2011

  10. [11]

    Conic optimization via operator splitting and homoge- neous self-dual embedding

    Brendan O’Donoghue, Eric Chu, Neal Parikh, and Stephen Boyd. Conic optimization via operator splitting and homoge- neous self-dual embedding. Journal of Optimization Theory and Applications , 169(3):1042–1068, June 2016

  11. [12]

    Operator splitting for a homogeneous e mbedding of the linear complementarity problem

    Brendan O’Donoghue. Operator splitting for a homogeneous e mbedding of the linear complementarity problem. SIAM Journal on Optimization , 31:1999–2023, August 2021

  12. [13]

    Globa lly convergent type–I Anderson acceleration for non-smooth fixed-point iterations

    Junzi Zhang, Brendan O’Donoghue, and Stephen Boyd. Globa lly convergent type–I Anderson acceleration for non-smooth fixed-point iterations. SIAM Journal on Optimization , 30(4):3170–3197, 2020

  13. [14]

    Maher, Frederic Matter, Erik M¨ uhmer, Benjamin M¨ uller, Marc E

    Ksenia Bestuzheva, Mathieu Besan¸ con, Wei-Kun Chen, A ntonia Chmiela, Tim Donkiewicz, Jasper van Doornmalen, Leon Eifler, Oliver Gaul, Gerald Gamrath, Ambros Gleixner, Leona Gottw ald, Christoph Graczyk, Katrin Halbig, Alexander Hoen, Christopher Hojny, Rolf van der Hulst, Thorsten Koch, Ma rco L¨ ubbecke, Stephen J. Maher, Frederic Matter, Erik M¨ uhmer,...

  14. [15]

    Pfetsch Tristan Gally and Stefan Ulbrich

    Marc E. Pfetsch Tristan Gally and Stefan Ulbrich. A framework for s olving mixed-integer semidefinite programs. Opti- mization Methods and Software , 33(3):594–632, 2018

  15. [16]

    Horn and Charles R

    Roger A. Horn and Charles R. Johnson. Positive definite matrices . Cambridge University Press, New York, 2nd ed. edition, 2013

  16. [17]

    Positive Definite Matrices

    Bhatia Rajendra. Positive Definite Matrices. Princeton Series in Applied Mathematics. Princeton University Press, 2007

  17. [18]

    Edwards, S

    William Yuill, A. Edwards, S. Chowdhury, and S. P. Chowdhu ry. Optimal PMU placement: A comprehensive literature review. In 2011 IEEE Power and Energy Society General Meeting , pages 1–8, July 2011. ISSN: 1944-9925

  18. [19]

    IEEE 14-Bus Sys tem

    Illinois Center for a Smarter Electric Grid. IEEE 14-Bus Sys tem. https://icseg.iti.illinois.edu/ ieee-14-bus-system/ . Accessed: 2024–08–12

  19. [20]

    IEEE 39-Bus Sys tem

    Illinois Center for a Smarter Electric Grid. IEEE 39-Bus Sys tem. https://icseg.iti.illinois.edu/ ieee-39-bus-system/ . Accessed: 2024–08–06

  20. [21]

    Hajian, A

    M. Hajian, A. M. Ranjbar, T. Amraee, and A. R. Shirani. Optimal Placement of Phasor Measurement Units: Particle Swarm Optimization Approach. In 2007 International Conference on Intelligent Systems Appl ications to Power Systems , pages 1–6, November 2007

  21. [22]

    Baldwin, L

    T.L. Baldwin, L. Mili, M.B. Boisen, and R. Adapa. Power s ystem observability with minimal phasor measurement 12 placement. IEEE Transactions on Power Systems , 8(2):707–715, May 1993. Conference Name: IEEE Transactions o n Power Systems

  22. [23]

    Transtrum, Benjamin L

    Mark K. Transtrum, Benjamin L. Francis, Andrija T. Saric, and Ale ksandar M. Stankovic. Simultaneous Global Identifi- cation of Dynamic and Network Parameters in Transient Stability S tudies. In 2018 IEEE Power & Energy Society General Meeting (PESGM) , pages 1–5, August 2018. ISSN: 1944-9933

  23. [24]

    Westwood, C

    Evan K. Westwood, C. T. Tindle, and N. R. Chapman. A normal mode model for acousto-elastic ocean environments. The Journal of the Acoustical Society of America , 100(6):3631–3645, December 1996

  24. [25]

    D. A. Wood and D. J. Allwright. Optimisation of hydrophone pl acement: a dynamical systems approach. European Journal of Applied Mathematics , 14(4):369–386, August 2003. Publisher: Cambridge Universit y Press

  25. [26]

    Dosso and Barbara J

    Stan E. Dosso and Barbara J. Sotirin. Optimal array element loc alization. The Journal of the Acoustical Society of America, 106(6):3445–3459, December 1999

  26. [27]

    Array element localization of a bottom moored hydrophone array

    Matthew Barlee, Stan Dosso, and Philip Schey. Array element localization of a bottom moored hydrophone array. Canadian Acoustics, 30(4):3–14, December 2002. Number: 4

  27. [28]

    Dosso and Gordon R

    Stan E. Dosso and Gordon R. Ebbeson. Array element localizat ion accuracy and survey design. Canadian Acoustics , 34(4):3–13, December 2006. Number: 4

  28. [29]

    Mortenson, Tracianne B

    Michael C. Mortenson, Tracianne B. Neilsen, Mark K. Transtru m, and David P. Knobles. Accurate Broadband Gradi- ent Estimates Enable Local Sensitivity Analysis of Ocean Ac oustic Models. Journal of Theoretical and Computational Acoustics, 31(02):2250015, June 2023. Publisher: World Scientific Publ ishing Co

  29. [30]

    E. B. Tadmor and R. E. Miller. Modeling Materials: Continuum, Atomistic and Multiscale T echniques. Cambridge University Press, 2011

  30. [31]

    Introduction to Computational Materials Science: Fundame ntals to Applications

    Richard LeSar. Introduction to Computational Materials Science: Fundame ntals to Applications . Cambridge University Press, 2013

  31. [32]

    Stillinger and Thomas A

    Frank H. Stillinger and Thomas A. Weber. Computer simulat ion of local order in condensed phases of silicon. Physical Review B , 31:5262–5271, Apr 1985

  32. [33]

    Stillinger and Thomas A

    Frank H. Stillinger and Thomas A. Weber. Erratum: Computer s imulation of local order in condensed phases of silicon [Phys. Rev. B 31, 5262 (1985)]. Phys. Rev. B , 33:1451–1451, jan 1986

  33. [34]

    Shirodkar, Petr Plech´ aˇ c, Efthimios Kaxiras, Ryan S

    Mingjian Wen, Sharmila N. Shirodkar, Petr Plech´ aˇ c, Efthimios Kaxiras, Ryan S. Elliott, and Ellad B. Tadmor. A force- matching Stillinger-Weber potential for MoS 2 : Parameterization and Fisher information theory based sensitiv ity analysis. Journal of Applied Physics , 122(24):244301, December 2017

  34. [35]

    E. B. Tadmor, R. S. Elliott, J. P. Sethna, R. E. Miller, and C . A. Becker. The potential of atomistic simulations and the Knowledgebase of Interatomic Models. JOM, 63(7):17–17, Jul 2011

  35. [36]

    Elliott and Ellad B

    Ryan S. Elliott and Ellad B. Tadmor. Knowledgebase of Int eratomic Models (KIM) application programming interface (API). https://openkim.org/kim-api, 2011

  36. [37]

    Stillinger-Weber Model Driver for Monolayer M X2 systems v001

    Mingjian Wen. Stillinger-Weber Model Driver for Monolayer M X2 systems v001. OpenKIM, https://doi.org/10.25950/ eeedbbc4, 2018

  37. [38]

    Modified Stillinger-Weber potential (MX2) for monolayer MoS2 developed by Wen et al

    Mingjian Wen. Modified Stillinger-Weber potential (MX2) for monolayer MoS2 developed by Wen et al. (2017) v001. OpenKIM, https://doi.org/10.25950/eeedbbc4, 2018

  38. [39]

    Dataset of MoS2 monolayer from AIMD trajectory, J une 2024

    Mingjian Wen. Dataset of MoS2 monolayer from AIMD trajectory, J une 2024

  39. [40]

    A. P. Thompson, H. M. Aktulga, R. Berger, D. S. Bolintineanu , W. M. Brown, P. S. Crozier, P. J. in ’t Veld, A. Kohlmeyer, S. G. Moore, T. D. Nguyen, R. Shan, M. J. Stevens, J. Tranchida, C. Trott, and S. J. Plimpton. LAMMPS - a flexible simulation tool for particle-based materials modeling at the a tomic, meso, and continuum scales. Comp. Phys. Comm. , 27...

  40. [41]

    Stillinger, and Thomas A

    Mingjian Wen, Yaser Afshar, Frank H. Stillinger, and Thomas A . Weber. Stillinger-Weber (SW) Model Driver v005. OpenKIM, https://doi.org/10.25950/934dca3e, 2021

  41. [42]

    Stillinger-Weber potential for Si due to Stillinger and Weber (1985) v006,

    Amit K. Singh, Frank H. Stillinger, and Thomas A. Weber. Sti llinger-Weber potential for Si due to Stillinger and Weber (1985) v006. OpenKIM, https://doi.org/10.25950/dd263fe3, 2021

  42. [43]

    Elliott, and Ellad B

    Mingjian Wen, Yaser Afshar, Ryan S. Elliott, and Ellad B. Ta dmor. Kliff: A framework to develop physics-based and machine learning interatomic potentials. Computer Physics Communications , 272:108218, Mar 2022

  43. [44]

    Justo, Martin Z

    Jo˜ ao F. Justo, Martin Z. Bazant, Efthimios Kaxiras, V. V. B ulatov, and Sidney Yip. Interatomic potential for silicon defects and disordered phases. Physical Review B , 58:2539–2550, Aug 1998

  44. [45]

    Karls, Joao F

    Daniel S. Karls, Joao F. Justo, Martin Z. Bazant, Efthimios K axiras, Vasily V Bulatov, and Sidney Yip. Environment- Dependent Interatomic Potential (EDIP) model driver v002. Open KIM, https://doi.org/10.25950/75c4686e, 2018

  45. [46]

    EDIP model for Si developed by Justo et al. (1998) v002,

    Daniel S. Karls, Joao F. Justo, Martin Z. Bazant, Efthimios K axiras, Vasily V Bulatov, and Sidney Yip. EDIP model for Si developed by Justo et al. (1998) v002. OpenKIM, https://doi.org/10.25950/545ca247, 2018

  46. [47]

    Tadmor and Junhao Li

    Ellad B. Tadmor and Junhao Li. Elastic constants for cubi c crystals at zero temperature and pressure v006. OpenKIM, https://doi.org/10.25950/5853fb8f, 2019

  47. [48]

    Elastic constants for diamond Si at zero temperature v001

    Junhao Li and Ellad Tadmor. Elastic constants for diamond Si at zero temperature v001. OpenKIM, https://openkim. org/cite/TE_507832142782_001, 2019

  48. [49]

    The atomic simulation environment—a python library for working with atoms

    Ask Hjorth Larsen, Jens Jørgen Mortensen, Jakob Blomqvist, I vano E Castelli, Rune Christensen, Marcin Du/suppress lak, Jesper Friis, Michael N Groves, Bjørk Hammer, Cory Hargus, Eric D Hermes, Paul C Jennings, Peter Bjerre Jensen, James Kermode, John R Kitchin, Esben Leonhard Kolsbjerg, Joseph Kubal , Kristen Kaasbjerg, Steen Lysgaard, J´ on Bergmann Mar...