REVIEW 3 major objections 4 minor 44 references
Matching output–control pairs solves constrained calibration.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 19:22 UTC pith:XR46YFSB
load-bearing objection A correct but mostly definitional extension of the disintegration-based calibration program to known control-parameter marginals; the real question is whether the unpaired-data OT surrogate can be trusted, and the paper's evidence on that is thin. the 3 major comments →
Non-Parametric Model Calibration with Stochastic Control Parameters
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper's central claim is Theorem 1: if a candidate joint distribution on Λ×U has the same pushforward as the trial-generating distribution under the augmented map Q̃(λ,u)=(Q(λ,u),u), then that candidate automatically solves the constrained statistical calibration problem—its pushforward under Q matches the observed field data, and its marginal on U matches the known control distribution. This reduction turns the constrained problem into an ordinary unconstrained SCP on the augmented output space D×U, for which a disintegration-based estimator already exists. When samples are unpaired, the paper supplies the missing joint pushforward by solving an entropically regularized optimal transpor
What carries the argument
The central object is the augmented computer model Q̃(λ,u)=(Q(λ,u),u), which embeds the control parameter into the output space. Any probability measure that matches the trial-generating measure's pushforward under Q̃ automatically satisfies both the data-matching condition and the known-marginal condition. To handle unpaired data, the paper constructs a surrogate for the joint distribution of (Q,u) by solving the entropic Kantorovich problem with the model-aware cost c(q,u)=∫_Λ |q−Q(λ,u)|² ρ_Λ(λ)dλ, regularized by KL divergence to the independent coupling; the Sinkhorn–Knopp algorithm computes the coupling. The disintegration-based estimator is then applied on the augmented space, which req
Load-bearing premise
For unpaired data, the surrogate coupling obtained by optimal transport stands in for the unobserved joint distribution of outputs and control parameters, and the final calibration inherits this assumption without any way to verify it from the data alone.
What would settle it
Construct a synthetic problem where the true coupling has strong negative dependence between Q and u (e.g., u = 1 − Q), run both the paired and unpaired algorithms, and compare the estimated marginal distributions on Λ. If the unpaired estimate substantially differs from the paired estimate despite both matching the desired marginals, then the optimal transport surrogate fails to recover scientifically plausible dependence.
If this is right
- When paired data exist, the constrained calibration problem is solved by applying the known unconstrained algorithm directly to the augmented model, requiring no new estimation scheme.
- When only unpaired data exist, the model-aware optimal transport coupling provides a surrogate for the missing joint, and the resulting calibration preserves both the data distribution and the known control marginal by construction.
- The augmented approach imposes a dimension condition (m ≤ n) but relaxes the Jacobian rank condition to only the λ-derivatives of Q, which may ease verification of regularity conditions.
- In the examples, the calibration solution is stable across a broad range of the entropy regularization parameter ε, even though the transport coupling itself varies substantially, suggesting an unexpected robustness worth theoretical investigation.
- The method extends the scope of non-parametric calibration to problems with partially specified stochastic inputs, including physical settings where control parameters follow well-established distributions.
Where Pith is reading between the lines
- The surrogate coupling for unpaired data is fundamentally unidentifiable from the two marginals alone; any coupling with the given marginals yields a valid constrained SCP solution, but only the true coupling captures the actual physical dependence, so the calibrated distribution on Λ can be misleading even when both constraints are satisfied.
- Different choices of the transport cost (e.g., copula-based or domain-informed costs) would produce different—still constraint-satisfying—calibrations, meaning the optimal transport step is not just computational but encodes a modeling assumption about the joint structure.
- The known continuity results for the unconstrained SCP suggest a limiting argument: as the control distribution concentrates to a point, the constrained solution should degenerate into a conditional calibration at that fixed control value, bridging to classical fixed-control calibration.
- A testable extension is to run the unpaired algorithm on synthetic data where the true coupling is deliberately anti-correlated with the model-aware cost, then compare the resulting Λ-marginal against the paired-data benchmark to reveal potential bias from the surrogate coupling.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper extends non-parametric, measure-disintegration-based computer model calibration to the setting where the input space contains both calibration parameters with unknown distribution and control parameters with a known marginal distribution. The constrained SCP (Definition 3.1) requires the solution to match both the data pushforward and the known control marginal. The paper's formal device is to augment the computer model to \tilde Q=(Q,\pi_U) (Eq. (1)) and prove (Theorem 1) that matching the augmented pushforward of the trial-generating distribution is sufficient for both constraints. For paired data, the existing SCP algorithm is applied directly with \tilde Q. For unpaired data, the paper constructs a surrogate joint distribution \hat\Pi on D\times U via entropic optimal transport with cost c(q,u)=∫|q-Q(λ,u)|^2ρ_Λ(λ)dλ (Eq. (4)), samples synthetic pairs from \hat\Pi, and then runs the paired algorithm. Two synthetic experiments (a quadratic model and a heat-equation model with a surrogate) are used to argue that the unpaired method performs nearly as well as the paired method.
Significance. If the unpaired-data extension were valid, the paper would contribute a practical method for calibrating computer models with stochastic control parameters whose marginal is known but whose joint dependence with outputs is unobserved. The manuscript has several genuine strengths: it ships reproducible code, Theorem 1 is a clean and correct marginalization argument, Theorem 2 is a standard existence result, and the regularity conditions are discussed carefully in an appendix. However, the central empirical claim — that the unpaired method is nearly as good as paired calibration — is not supported by the evidence presented. The unpaired pipeline relies on an arbitrary optimal-transport surrogate for an unidentifiable joint distribution, and the paper offers no sensitivity analysis, no dependent-TGD experiments, and no quantitative comparison. These issues are load-bearing because the unpaired case is the main methodological novelty; the formal paired case is a straightforward application of existing methods.
major comments (3)
- [§3.3, Eq. (4)] In the unpaired case, Algorithm 3 replaces the unavailable augmented pushforward \tilde Q P^t_{\Lambda\times U} with the entropic-OT coupling \hat\Pi computed by Eq. (3) under cost (4). Since only the marginals P_D and P^t_U are identified from unpaired data, every coupling in F(P_D,P^t_U) satisfies the two constraints in Definition 3.1 by construction; the final \hat P_{\Lambda\times U} depends on which coupling is chosen. The paper does not show that \hat P_\Lambda is insensitive to this choice or close to the paired-data solution, and the authors themselves note that the Sinkhorn joint 'certainly differs from the true distribution.' This is load-bearing: the advertised scientific plausibility of the unpaired estimate is not established. I request (i) a sensitivity analysis with respect to the cost function and the regularization \epsilon, and (ii) experiments where the TGD has non-tri
- [§4, Figures 2, 5, 6] The central empirical claim that the unpaired method 'yields similar estimates as paired data' rests on two single-run synthetic examples with visual comparison only. No quantitative distance or discrepancy measure between the paired and unpaired \hat P_\Lambda is reported, and there are no repeated simulations with different random seeds or Sinkhorn draws. Since this claim is the main motivation for the unpaired algorithm, the evidence is insufficient as it stands. Please add quantitative comparisons (e.g., L1 or Wasserstein distance between estimated marginal densities of \Lambda), repeated-run variability, and ideally a check of whether the unpaired solution remains close to paired when the surrogate coupling is changed.
- [Theorem 1, Definition 3.1] Theorem 1 is correct, but it only reduces the constrained problem to exact matching of \tilde Q P^t_{\Lambda\times U}. In the unpaired pipeline this target is replaced by the surrogate \hat\Pi, so Theorem 1 does not validate Algorithm 3. The U-marginal agreement of the final solution is imposed by construction in the OT step and then reproduced by Algorithm 2; it is not a data-informed property. The paper should state this limitation explicitly and separate the formal contribution (Theorem 1 and the paired Algorithm 2) from the heuristic unpaired extension, which currently requires an unvalidated modeling assumption.
minor comments (4)
- [§4.2] The text says 'We return to the heat transfer problem of Example 4.2' but the running example is Example 1.1.
- [§3.3] The statement that without the KL term '\hat\Pi would overfit to the graph of Q and not admit a density in Lebesgue measure' is imprecise for the discrete implementation described in Algorithm 3. Clarify whether this refers to the continuous population problem or to the discrete Sinkhorn approximation.
- [Algorithms 1, 2] The histogram estimator has undefined 0/0 ratios in bins that contain no prior samples. The paper should state how zero-count bins are handled in the implementation.
- [§4.2] The heat-equation example uses an XGBoost surrogate for the PDE solver, and Section 5 acknowledges that this introduces additional statistical error and may violate regularity conditions. A diagnostic of surrogate accuracy (e.g., held-out error) would help the reader assess the example.
Circularity Check
No significant circularity; the known-marginal constraint is enforced by construction, not presented as an independently predicted quantity.
full rationale
The paper's core reduction, Theorem 1, is a transparent mathematical equivalence rather than a circular derivation: the augmented map is explicitly defined as Qtilde=(Q, pi_U), so matching Qtilde's pushforward is by definition equivalent to matching both the Q-pushforward and the U-marginal. This is a sound reduction, not a hidden reuse of the conclusion. In the paired case, Algorithm 2 directly applies the prior SCP method to samples from Qtilde P^t, so the constrained result follows from the stated assumptions without circularity. In the unpaired case, Algorithm 3 constructs a surrogate joint distribution Pi-hat by entropic optimal transport with fixed marginals P_D and P_U. Consequently, the final solution's U-marginal matches P_U by construction. The paper presents this as a constraint-satisfaction feature, not as a fitted prediction or as evidence that the surrogate coupling is the true joint distribution; indeed it explicitly states that the SK solution 'certainly differs from the true distribution.' The arbitrariness of the OT cost and the unidentifiability of the joint coupling are substantive statistical-modeling limitations, but they are not circularity: the final U-marginal is not claimed to be inferred from the data. The paper also acknowledges surrogate-model regularity concerns in Section 5. Some foundational citations, especially [9], are by overlapping authors and are load-bearing building blocks, but the present contribution is tested on synthetic examples and is not justified solely by a self-citation chain or by an imported uniqueness theorem. Overall, no step in the claimed derivation reduces to its own inputs in a way that would constitute circular reasoning.
Axiom & Free-Parameter Ledger
free parameters (3)
- Entropic OT regularization epsilon =
epsilon = 1 in both examples
- Histogram bin counts M, M1, M2 =
M=30 (or 20 in heat example), M1=M2=30
- Monte Carlo sample sizes K3, K4, J =
K3=15,000, K4=100,000, J=250,000-300,000
axioms (5)
- domain assumption The unconstrained SCP machinery of [9] - the disintegration formula, Algorithm 1 estimator, and its convergence - is correct and applies to the augmented map \tilde Q.
- domain assumption The regularity conditions in Appendix A hold for Lambda, Q, the prior, the TGD, and the relevant pushforwards (compactness, C^1 a.e., Jacobian rank, densities, boundary conditions).
- ad hoc to paper For unpaired data, the entropic OT coupling \hat\Pi with cost c(q,u) = \int_\Lambda |q-Q(\lambda,u)|^2 \rho_\Lambda(\lambda)d\lambda is an acceptable stand-in for the unobserved joint pushforward \tilde Q P^t_{\Lambda\times U}.
- domain assumption The reference measure for the KL penalty is the independent coupling P_D \times P^t_U, and the prior factorizes as \rho_\Lambda \rho^t_U.
- standard math Theorem 3's measurable selection result depends on the parametrized Kantorovich theorem of Bogachev-Malofeev [43] and lower semicontinuity of KL divergence [44].
read the original abstract
We present a method for calibrating a computer model using non-parametric techniques where the inputs are stochastic but include calibration parameters whose distributions are unknown and control parameters whose distributions are specified. Our solution gives a distributional estimate over the input space that is consistent with observed field data, while also preserving the distribution of the known marginal of the control parameters. This property is desirable since stochastic inputs often include physical processes affecting the experimental conditions, and a scientifically plausible calibration estimate should preserve well-established distributional properties of these inputs. The method builds on recently developed non-parametric computer model calibration techniques based on the disintegration of measure and Bayesian inference.
Figures
Reference graph
Works this paper leans on
-
[1]
https://books.google.ca/books?id=VGqbyIUVZw8C
Fang, K., Li, R., and Sudjianto, A.,Design and Modeling for Computer Experiments, Chapman & Hall/CRC Computer Science & Data Analysis, CRC Press, 2005. https://books.google.ca/books?id=VGqbyIUVZw8C
2005
-
[2]
and O’Hagan, A., Bayesian Calibration of Computer Models,Journal of the Royal Statistical Society Series B: Statistical Methodology, 63(3):425–464, 01 2002
Kennedy, M.C. and O’Hagan, A., Bayesian Calibration of Computer Models,Journal of the Royal Statistical Society Series B: Statistical Methodology, 63(3):425–464, 01 2002
2002
-
[3]
Breidt, J., Butler, T., and Estep, D., A measure-theoretic computational method for inverse sensitivity problems I: Method and analysis,SIAM Journal on Numerical Analysis, 49(5):1836–1859, 2011
2011
-
[4]
Butler, T., Estep, D., and Sandelin, J., A computational measure theoretic approach to inverse sensitivity problems II: A posteriori error analysis,SIAM Journal on Numerical Analysis, 50(1):22–45, 2012
2012
-
[5]
Butler, T., Estep, D., Tavener, S., Dawson, C., and Westerink, J., A measure-theoretic computational method for inverse sensitivity problems III: Multiple quantities of interest,SIAM/ASA Journal on Uncertainty Quantification, 2(1):174–202, 2014
2014
-
[6]
Butler, T., Jakeman, J., and Wildey, T., Combining push-forward measures and Bayes’ rule to construct consistent solutions to stochastic inverse problems,SIAM Journal on Scientific Computing, 40(2):A984–A1011, 2018
2018
-
[7]
Chi, J., Sliced inverse approach and domain recovery for stochastic inverse problems, PhD thesis, Colorado State University, 2021
2021
-
[8]
Bingham, D., Butler, T., and Estep, D., Inverse problems for physics-based process models,Annual Review of Statistics and Its Application, 11, 2024
2024
-
[9]
Shi, H., Yang, L., Chi, J., Butler, T., Wang, H., Bingham, D., and Estep, D., Nonparametric Bayesian Calibration of Computer Models,arXiv preprint arXiv:2509.22597, 2025, submitted to the SIAM/ASA Journal on Uncertainty Quantification
Pith/arXiv arXiv 2025
-
[10]
Yang, L., Infinite-dimensional stochastic inverse problems, PhD thesis, Colorado State University, 2018
2018
-
[11]
and Pollard, D., Conditioning as disintegration,Statistica Neerlandica, 51(3):287–317, 1997
Chang, J.T. and Pollard, D., Conditioning as disintegration,Statistica Neerlandica, 51(3):287–317, 1997
1997
-
[12]
Prasadan, A., Estep, D., and Bingham, D., Continuity of the solution of a non-parametric bayesian statistical calibration procedure,arXiv preprint arXiv:2603.20665, 2026, in revision with the SIAM/ASA Journal on Uncertainty Quantification
arXiv 2026
-
[13]
Higdon, D., Kennedy, M., Cavendish, J., Cafeo, J., and Ryne, R., Combining field data and computer simulations for calibra- tion and prediction,SISC, 26:448–466, 2004
2004
-
[14]
Higdon, D., Gattiker, J., Williams, B., and Rightley, M., Computer model calibration using high-dimensional output,JASA, 103:570–583, 2008
2008
-
[15]
Wang, S., Chen, W., and Tsui, K.L., Bayesian validation of computer models,Techno., 51:439–451, 2009
2009
-
[16]
and Wu, J., Efficient calibration for imperfect computer models,Ann
Tuo, R. and Wu, J., Efficient calibration for imperfect computer models,Ann. Statist., 43:2331–2352, 12 2015
2015
-
[17]
Plumlee, M., Bayesian calibration of inexact computer models,JASA, 112:1274–1285, 2017
2017
-
[18]
Grosskopf, M., Bingham, D., Adams, M., Hawkins, W., and Perez-Nunez, D., Generalized computer model calibration for radiation transport simulation,Techno, 63:27–39, 2021
2021
-
[19]
and Tuo, R., A review of computer model calibration,WIREs Comput
Sung, C. and Tuo, R., A review of computer model calibration,WIREs Comput. Stat., 16, 02 2024
2024
-
[20]
and Tarantola, A
Mosegaard, K. and Tarantola, A. Probabilistic approach to inverse problems. In: Lee, W., Kanamori, H., Jennings, P., and Kisslinger, C. (Eds.),International Handbook of Earthquake and Engineering Seismology, Part A, V ol. 81 of International Geophysics, pp. 237–265. Academic Press, 2002
2002
-
[21]
Mosegaard, K.,Monte Carlo Analysis of Inverse Problems, Flemming Nicolaisen, Copenhagen, 2006
2006
-
[22]
and Taylor, A., Multiple models to capture the variability in biological neurons and networks,Nature neuroscience, 14(2):133–138, 2011
Marder, E. and Taylor, A., Multiple models to capture the variability in biological neurons and networks,Nature neuroscience, 14(2):133–138, 2011
2011
-
[23]
NAS, 110(23):E2098–E2105, 2013
Britton, O., Bueno-Orovio, A., Van Ammel, K., Lu, H., Towart, R., Gallacher, D., and Rodriguez, B., Experimentally cali- brated population of models predicts and explains intersubject variability in cardiac cellular electrophysiology,Proceed. NAS, 110(23):E2098–E2105, 2013
2013
-
[24]
Schervish, M.,Theory of Statistics, Springer, New York, 1995
1995
-
[25]
(Eds.),Bayesian Nonparametrics, Cambridge Series in Statistical and Prob- abilistic Mathematics, Cambridge University Press, 2010
Hjort, N., Holmes, C., Muller, P., and Walker, S. (Eds.),Bayesian Nonparametrics, Cambridge Series in Statistical and Prob- abilistic Mathematics, Cambridge University Press, 2010. Calibration with Stochastic Control Parameters19
2010
-
[26]
and van der Vaart, A.,Fundamentals of Nonparametric Bayesian Inference, Cambridge University Press, New York, 2017
Ghosal, S. and van der Vaart, A.,Fundamentals of Nonparametric Bayesian Inference, Cambridge University Press, New York, 2017
2017
-
[27]
Li, Q., Wang, L., and Yang, Y ., Differential equation–constrained optimization with stochasticity,SIAM/ASA JUQ, 12(2):549– 578, 2024
2024
-
[28]
Vadeboncoeura, A., Girolamia, M., and Stuart, A., Efficient deconvolution in populational inverse problems,arXiv preprint arXiv:2505.19841v1, 2025
Pith/arXiv arXiv 2025
-
[29]
Prob., 30:110301, 2014
Calvetti, D., Kaipio, J., and Somersalo, E., Inverse problems in the Bayesian framework,Inv. Prob., 30:110301, 2014
2014
-
[30]
Stuart, A., Inverse problems: A Bayesian perspective,Acta Num., 19:451–559, 2010
2010
-
[31]
Tarantola, A.,Inverse Problem Theory and Methods for Model Parameter Estimation, SIAM, Philadelphia, 2005
2005
-
[32]
https://jmlr.org/papers/v23/19-843.html
Lin, T., Ho, N., Cuturi, M., and Jordan, M.I., On the complexity of approximating multimarginal optimal transport,Journal of Machine Learning Research, 23(65):1–43, 2022. https://jmlr.org/papers/v23/19-843.html
2022
-
[33]
Nelsen, R.B.,An introduction to copulas, Springer, 2006
2006
-
[34]
87, Springer, 2015
Santambrogio, F.,Optimal transport for applied mathematicians, V ol. 87, Springer, 2015
2015
-
[35]
338, Springer, 2008
Villani, C.,Optimal transport: old and new, V ol. 338, Springer, 2008
2008
-
[36]
Nutz, M., Introduction to entropic optimal transport,Lecture notes, Columbia University, 2021
2021
-
[37]
and Sinkhorn, R., Concerning nonnegative matrices and doubly stochastic matrices.,Pacific Journal of Mathematics, 21(2):343 – 348, 1967
Knopp, P. and Sinkhorn, R., Concerning nonnegative matrices and doubly stochastic matrices.,Pacific Journal of Mathematics, 21(2):343 – 348, 1967
1967
-
[38]
https://jmlr.org/papers/v22/20-451.html
Flamary, R., Courty, N., Gramfort, A., Alaya, M.Z., Boisbunon, A., Chambon, S., Chapel, L., Corenflos, A., Fatras, K., Fournier, N., Gautheron, L., Gayraud, N.T., Janati, H., Rakotomamonjy, A., Redko, I., Rolet, A., Schutz, A., Seguy, V ., Sutherland, D.J., Tavenard, R., Tong, A., and Vayer, T., Pot: Python optimal transport,Journal of Machine Learning Re...
2021
-
[39]
POT Python Optimal Transport,
Flamary, R., Vincent-Cuaz, C., Courty, N., Gramfort, A., Kachaiev, O., Quang Tran, H., David, L., Bonet, C., Cassereau, N., Gnassounou, T., Tanguy, E., Delon, J., Collas, A., Mazelet, S., Chapel, L., Kerdoncuff, T., Yu, X., Feickert, M., Krzakala, P., Liu, T., Fernandes Montesuma, E., Neike, N., Genest, B., Coeurjolly, D., and Corneli, M. POT Python Optim...
-
[40]
http://bobby.gramacy.com/surrogates/
Gramacy, R.B.,Surrogates: Gaussian Process Modeling, Design and Optimization for the Applied Sciences, Chapman Hall/CRC, Boca Raton, Florida, 2020. http://bobby.gramacy.com/surrogates/
2020
-
[41]
http://www.ctcms.nist.gov/fipy
Guyer, J.E., Wheeler, D., and Warren, J.A., FiPy: Partial differential equations with Python,Computing in Science & Engi- neering, 11(3):6–15, 2009. http://www.ctcms.nist.gov/fipy
2009
-
[42]
and Guestrin, C., XGBoost: A scalable tree boosting system, InProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, pp
Chen, T. and Guestrin, C., XGBoost: A scalable tree boosting system, InProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, pp. 785–794, New York, NY , USA, 2016. ACM
2016
-
[43]
and Malofeev, I., Nonlinear kantorovich problems with a parameter,The Bulletin of Irkutsk State University
Bogachev, V .I. and Malofeev, I., Nonlinear kantorovich problems with a parameter,The Bulletin of Irkutsk State University. Series Mathematics, 2022
2022
-
[44]
Posner, E., Random coding strategies for minimum entropy,IEEE Transactions on Information Theory, 21(4):388–391, 1975
1975
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.