REMAL: Residual Equilibrium Manifold Active Learning for Surrogate-Based Multidisciplinary Design Analysis
Pith reviewed 2026-06-27 05:08 UTC · model grok-4.3
The pith
REMAL learns the joint residual manifold of coupled systems with multitask Gaussian processes so that equilibria can be recovered by nonlinear least squares on the surrogate instead of separate fixed-point iterations at each design point.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that the equilibrium states of a coupled system lie on the zero contour of a joint residual function that can be approximated by multitask Gaussian process models; entropy-based active learning reduces uncertainty near that contour, after which any new design point's equilibrium is recovered by a single nonlinear least squares optimization performed entirely on the surrogate, with the predictive error provably bounded under mild assumptions.
What carries the argument
The residual equilibrium manifold, represented by multitask Gaussian process models whose uncertainty is reduced near the zero contour through entropy-based active learning, with equilibria recovered via nonlinear least squares on the surrogate.
If this is right
- Repeated fixed-point solves across a design space require only surrogate evaluations plus one optimization per point rather than repeated disciplinary iterations.
- The same trained surrogate supplies equilibrium states for outer-loop tasks such as optimization or uncertainty quantification without re-solving the coupling at each query.
- Error bounds on the recovered fixed points follow directly from the Gaussian process approximation and the active learning placement near the zero contour.
- The framework applies uniformly to the satellite, aerostructural, gas-turbine, and feedback-coupled turbine models examined.
Where Pith is reading between the lines
- The method could be combined with gradient-based outer optimizers that treat the surrogate-based equilibrium solve as a differentiable black-box step.
- If the residual manifold dimension grows, the multitask Gaussian process scaling may become the next computational bottleneck.
- Active learning focused on the zero contour might also improve surrogate accuracy for related tasks such as sensitivity analysis around equilibria.
Load-bearing premise
The joint residual manifold admits a useful approximation by multitask Gaussian processes whose uncertainty can be driven down near the zero contour by entropy-based selection.
What would settle it
A benchmark case in which the nonlinear least squares solution on the trained surrogate deviates from the true fixed point by more than the stated error bound even after the active learning budget is exhausted.
Figures
read the original abstract
Multidisciplinary design analysis of coupled engineering systems requires the computation of equilibrium states in which all disciplinary coupling variables are mutually consistent. Conventional fixed-point iteration resolves this consistency problem separately at each design point, which can become expensive when disciplinary evaluations are costly and many analyses are required in outer-loop tasks such as multidisciplinary design optimization, uncertainty quantification, or digital twin updating. This paper introduces REMAL, a residual manifold surrogate modeling framework for coupled systems. Instead of approximating each discipline independently or directly learning converged coupling variables, the proposed method learns a surrogate model of the joint residual manifold via multitask Gaussian process models. An entropy-based active learning strategy selects additional residual evaluations near uncertain zero-contour regions, and equilibrium states for new design inputs are recovered by solving a nonlinear least squares optimization problem using only the trained surrogate. The method is evaluated on four engineering coupled system benchmarks: a satellite model, an aerostructural model, a finite-element gas-turbine heat-transfer and economics model, and a modified turbine model with added feedback coupling. Across these cases, REMAL consistently demonstrates the cost effectiveness when repeated evaluations of the fixed point across the design space are necessary. Theoretically, we show that, under mild assumptions, REMAL's predictive fixed point error is bounded.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces REMAL, a surrogate modeling approach for multidisciplinary design analysis of coupled systems. It models the joint residual manifold of the coupled system using multitask Gaussian processes, applies an entropy-based active learning strategy to select residual evaluations near uncertain regions of the zero contour, and recovers equilibrium states for new design points by solving a nonlinear least squares problem on the trained surrogate. The method is demonstrated on four engineering benchmarks (satellite model, aerostructural model, finite-element gas-turbine heat-transfer/economics model, and modified turbine with feedback), claiming consistent cost-effectiveness for repeated fixed-point evaluations across a design space. A theoretical result is stated: under mild assumptions, the predictive fixed-point error is bounded.
Significance. If the central claims hold, REMAL provides a targeted surrogate strategy for residual manifolds that could reduce the expense of repeated equilibrium solves in outer-loop tasks such as MDO or UQ. Strengths include the explicit focus on the zero contour via multitask GPs and entropy-driven selection, evaluation across four distinct coupled-system benchmarks, and the attempt to supply a theoretical error bound rather than purely empirical claims. These elements address a practical need in surrogate-based MDA when disciplinary evaluations are costly.
major comments (3)
- [§3, §4] §3 (method) and §4 (theory): the central recovery step (NLS on the multitask-GP surrogate) and the claimed predictive-error bound both require that arg min_y ||ρ̂(x,y)|| closely approximates arg min_y ||ρ(x,y)|| after active learning. No diagnostic is reported that isolates zero-contour fidelity (e.g., contour error, fixed-point residual on a validation set, or distance between learned and true manifolds). Without such a check, systematic bias in the GP mean could produce incorrect equilibria even when predictive variance is low, directly undermining both the practical procedure and the bound.
- [§4] §4 (theoretical bound): the abstract states that a bound on predictive fixed-point error is shown under mild assumptions, yet the manuscript supplies neither the derivation steps nor an explicit statement of the assumptions. This prevents verification that the bound is non-vacuous and load-bearing for the method's guarantees.
- [Results] Results section (benchmark tables): quantitative error tables or residual norms at recovered equilibria are not provided; only qualitative statements of cost-effectiveness appear. This makes it impossible to assess whether the NLS recoveries are accurate to within the claimed bound or merely cheaper.
minor comments (2)
- [§3] Notation for the multitask GP kernel and the entropy acquisition function should be defined once in a single location with consistent symbols across equations.
- [Results] The four benchmarks would benefit from a short table summarizing coupling dimension, number of design variables, and disciplinary evaluation cost to allow direct comparison of cost savings.
Simulated Author's Rebuttal
We thank the referee for the detailed and constructive review. The comments highlight important aspects of validation, theoretical presentation, and quantitative reporting that we will address in revision.
read point-by-point responses
-
Referee: [§3, §4] §3 (method) and §4 (theory): the central recovery step (NLS on the multitask-GP surrogate) and the claimed predictive-error bound both require that arg min_y ||ρ̂(x,y)|| closely approximates arg min_y ||ρ(x,y)|| after active learning. No diagnostic is reported that isolates zero-contour fidelity (e.g., contour error, fixed-point residual on a validation set, or distance between learned and true manifolds). Without such a check, systematic bias in the GP mean could produce incorrect equilibria even when predictive variance is low, directly undermining both the practical procedure and the bound.
Authors: We agree that explicit verification of zero-contour fidelity is necessary to support both the recovery procedure and the error bound. In the revised manuscript we will add (i) fixed-point residual norms ||ρ(x, y*)|| evaluated on a held-out validation set of design points, (ii) contour-error visualizations comparing the learned and true zero sets where the true manifold is available, and (iii) quantitative distances between the learned and true manifolds for the benchmark problems. These diagnostics will be reported alongside the existing cost metrics. revision: yes
-
Referee: [§4] §4 (theoretical bound): the abstract states that a bound on predictive fixed-point error is shown under mild assumptions, yet the manuscript supplies neither the derivation steps nor an explicit statement of the assumptions. This prevents verification that the bound is non-vacuous and load-bearing for the method's guarantees.
Authors: Section 4 states the bound under the assumptions of Lipschitz continuity of the true residual and bounded GP predictive variance. In the revision we will (a) list these assumptions explicitly at the beginning of the section and (b) insert the principal derivation steps (including the triangle inequality application and the propagation of GP error to the fixed-point location) so that the result can be verified directly. revision: yes
-
Referee: [Results] Results section (benchmark tables): quantitative error tables or residual norms at recovered equilibria are not provided; only qualitative statements of cost-effectiveness appear. This makes it impossible to assess whether the NLS recoveries are accurate to within the claimed bound or merely cheaper.
Authors: We will augment the results section with tables that report, for each benchmark and each method, the mean and maximum residual norms ||ρ(x, y*)|| at the recovered equilibria on an independent test set of design points. These quantitative accuracy measures will be presented together with the existing computational-cost figures. revision: yes
Circularity Check
No significant circularity in derivation chain
full rationale
The paper introduces REMAL as a surrogate framework that models the joint residual manifold with multitask GPs, applies entropy-based active learning near zero contours, and recovers equilibria by NLS on the trained surrogate. The fixed-point error bound is presented as theoretically derived under mild assumptions rather than fitted or self-referential. No quoted steps reduce by construction to inputs (no self-definitional relations, no fitted parameters renamed as predictions, no load-bearing self-citations, and no ansatz smuggled via prior work). The central procedure and benchmarks remain independent of the paper's own fitted values.
Axiom & Free-Parameter Ledger
free parameters (1)
- multitask GP kernel hyperparameters
axioms (1)
- domain assumption Mild assumptions under which the predictive fixed-point error remains bounded
Forward citations
Cited by 1 Pith paper
-
A Bayesian latent Gaussian process framework for aerodynamic uncertainty quantification
A Bayesian latent GP calibration framework for aerodynamic surrogates marginalizes input uncertainty and matches output uncertainty statistics, achieving 94.2-95.8% coverage of true 95% intervals.
Reference graph
Works this paper leans on
-
[1]
A., Rosasco, L., and Lawrence, N
´Alvarez, M. A., Rosasco, L., and Lawrence, N. D. (2012). Kernels for vector-valued functions: A review.Foundations and Trends in Machine Learning, 4(3):195–266
2012
-
[2]
and Ghoreishi, S
Asadi, N. and Ghoreishi, S. F. (2024). Active learning for efficient data acquiring in coupled multidisciplinary systems.IFAC-PapersOnLine, 58(28):114–119
2024
-
[3]
R., Daulton, S., Letham, B., Wilson, A
Balandat, M., Karrer, B., Jiang, D. R., Daulton, S., Letham, B., Wilson, A. G., and Bakshy, E. (2020). BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization. InAdvances in Neural Information Processing Systems 33
2020
-
[4]
Baptista, R., Marzouk, Y., Willcox, K., and Peherstorfer, B. (2018). Optimal approxima- tions of coupling in multidisciplinary models.AIAA Journal, 56(6):2412–2428
2018
-
[5]
Berthelin, G., Dubreuil, S., Sala¨ un, M., Bartoli, N., and Gogu, C. (2022). Disciplinary proper orthogonal decomposition and interpolation for the resolution of parameterized multidisciplinary analysis.International Journal for Numerical Methods in Engineering, 123(15):3594–3626
2022
-
[6]
V., Chai, K., and Williams, C
Bonilla, E. V., Chai, K., and Williams, C. (2007). Multi-task gaussian process prediction. In Platt, J., Koller, D., Singer, Y., and Roweis, S., editors,Advances in Neural Information Processing Systems, volume 20. Curran Associates, Inc
2007
-
[7]
and Renganathan, A
Booth, A. and Renganathan, A. (2025). Two-stage design for failure probability estimation with gaussian process surrogates.Journal of Quality Technology, 57:1–17
2025
-
[8]
S., Gramacy, R., and Renganathan, A
Booth, A. S., Gramacy, R., and Renganathan, A. (2024). Actively learning deep gaussian process models for failure contour and probability estimation. InAIAA SCITECH 2024 Forum, page 0577
2024
-
[9]
S., Renganathan, S
Booth, A. S., Renganathan, S. A., and Gramacy, R. B. (2025). Contour location for reliability in airfoil siulation experiments using deep gaussian processes.The Annals of Applied Statistics, 19:191–211
2025
-
[10]
Cardoso, I., Dubreuil, S., Bartoli, N., Gogu, C., and Sala¨ un, M. (2024). Constrained efficient global multidisciplinary design optimization using adaptive disciplinary surrogate enrichment.Structural and Multidisciplinary Optimization, 67(2):23
2024
-
[11]
and Renganathan, A
Carlson, K. and Renganathan, A. (2025). Multiobjective aerodynamic design optimization of the nasa common research model.Aerospace Science and Technology, page 111120
2025
-
[12]
Chaudhuri, A., Lam, R., and Willcox, K. (2018). Multifidelity uncertainty propagation via adaptive surrogates in coupled multidisciplinary systems.AIAA Journal, 56(1):235–249
2018
-
[13]
Chevalier, C., Bect, J., Ginsbourger, D., Vazquez, E., Picheny, V., and Richet, Y. (2014). Fast parallel kriging-based stepwise uncertainty reduction with application to the identification of an excursion set.Technometrics, 56(4):455–465. 27
2014
-
[14]
A., Gramacy, R
Cole, D. A., Gramacy, R. B., Warner, J. E., Bomarito, G. F., Leser, P. E., and Leser, W. P. (2023). Entropy-based adaptive design for contour finding and estimating reliability. Journal of Quality Technology, 55(1):43–60
2023
-
[15]
Dubreuil, S., Bartoli, N., Gogu, C., and Lefebvre, T. (2020). Towards an efficient global multidisciplinary design optimization algorithm.Structural and Multidisciplinary Optimization, 62(4):1739–1765
2020
-
[16]
Forrester, A. I. J., S´ obester, A., and Keane, A. J. (2008).Engineering Design via Surrogate Modelling: A Practical Guide. Wiley, Hoboken, NJ
2008
-
[17]
I., Powell, W
Frazier, P. I., Powell, W. B., and Dayanik, S. (2008). A knowledge-gradient policy for sequential information collection.SIAM Journal on Control and Optimization, 47(5):2410– 2439
2008
-
[18]
Gallard, F., Vanaret, C., Gu´ enot, D., Gachelin, V., Lafage, R., Pauwels, B., Barjhoux, P.-J., and Gazaix, A. (2018). Gems: a python library for automation of multidisciplinary design optimization process generation. In2018 AIAA/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, page 0657
2018
-
[19]
Q., Bindel, D., and Wilson, A
Gardner, J., Pleiss, G., Weinberger, K. Q., Bindel, D., and Wilson, A. G. (2018). Gpytorch: Blackbox matrix-matrix gaussian process inference with gpu acceleration. In Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R., editors,Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc
2018
-
[20]
Ghoreishi, S. F. and Allaire, D. L. (2017). Adaptive uncertainty propagation for coupled multidisciplinary systems.AIAA Journal, 55(11):3940–3950
2017
-
[21]
Ghoreishi, S. F. and Imani, M. (2020). Bayesian optimization for efficient design of uncertain coupled multidisciplinary systems. In2020 American Control Conference (ACC), pages 3412–3418
2020
-
[22]
Ghoreishi, S. F. and Imani, M. (2021). Bayesian surrogate learning for uncertainty analysis of coupled multidisciplinary systems.Journal of Computing and Information Science in Engineering, 21(4):041009
2021
-
[23]
Gramacy, R. B. (2020).Surrogates: Gaussian process modeling, design and optimization for the applied sciences. CRC Press
2020
-
[24]
S., Hwang, J
Gray, J. S., Hwang, J. T., Martins, J. R. R. A., Moore, K. T., and Naylor, B. A. (2019). Openmdao: an open-source framework for multidisciplinary design, analysis, and optimization.Structural and Multidisciplinary Optimization, 59(4):1075–1104
2019
-
[25]
and Schuler, C
Hennig, P. and Schuler, C. (2011). Entropy search for information-efficient global optimization.Journal of Machine Learning Research, 13. 28
2011
-
[26]
and Mahadevan, S
Hu, Z. and Mahadevan, S. (2017). A surrogate modeling approach for reliability analysis of a multidisciplinary system with spatio-temporal output.Structural and Multidisciplinary Optimization, 56(3):553–569
2017
-
[27]
D., Friedman, S., Eldred, M., Tamellini, L., Gorodetsky, A
Jakeman, J. D., Friedman, S., Eldred, M., Tamellini, L., Gorodetsky, A. A., and Allaire, D. (2022). Adaptive experimental design for multi-fidelity surrogate modeling of multi-disciplinary systems.International Journal for Numerical Methods in Engineering, 123(12):2760–2790
2022
-
[28]
R., Schonlau, M., and Welch, W
Jones, D. R., Schonlau, M., and Welch, W. J. (1998). Efficient global optimization of expensive black-box functions.Journal of Global optimization, 13(4):455–492
1998
-
[29]
N., Berger, J
Kyzyurova, K. N., Berger, J. O., and Wolpert, R. L. (2018). Coupling computer models through linking their statistical emulators.SIAM/ASA Journal on Uncertainty Quantification, 6(3):1151–1171
2018
-
[30]
Marques, A., Lam, R., and Willcox, K. (2018). Contour location via entropy reduction leveraging multiple information sources. In Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R., editors,Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc
2018
-
[31]
Martins, J. R. and Ning, A. (2021).Engineering Design Optimization. Cambridge University Press
2021
-
[32]
Martins, J. R. R. A. and Lambe, A. B. (2013). Multidisciplinary design optimization: A survey of architectures.AIAA Journal, 51(9):2049–2075
2013
-
[33]
D., Beckman, R
McKay, M. D., Beckman, R. J., and Conover, W. J. (1979). A comparison of three methods for selecting values of input variables in the analysis of output from a computer code.Technometrics, 21(2):239–245
1979
-
[34]
and Wright, S
Nocedal, J. and Wright, S. J. (2006).Numerical optimization. Springer
2006
-
[35]
Ranjan, P., Bingham, D., and Michailidis, G. (2008). Sequential experiment design for contour estimation from complex computer codes.Technometrics, 50(4):527–541
2008
-
[36]
and Williams, K
Rasmussen, C. and Williams, K. (2006).Gaussian Processes for Machine Learning. the MIT Press
2006
-
[37]
Renganathan, A. (2024). Efficient reliability analysis with multifidelity gaussian processes and normalizing flows. InAIAA SCITECH 2024 Forum, page 0576
2024
-
[38]
Renganathan, A. and Booth, A. S. (2026). Surrogate-guided adaptive importance sampling for failure probability estimation.arXiv preprint arXiv:2603.20959
Pith/arXiv arXiv 2026
-
[39]
and Carlson, K
Renganathan, A. and Carlson, K. (2025). qpots: Efficient batch multiobjective bayesian optimization via pareto optimal thompson sampling. InInternational Conference on Artificial Intelligence and Statistics, pages 4051–4059. PMLR. 29
2025
-
[40]
Renganathan, S. A., Larson, J., and Wild, S. M. (2021). Lookahead acquisition functions for finite-horizon time-dependent bayesian optimization and application to quantum optimal control.arXiv preprint arXiv:2105.09824
arXiv 2021
-
[41]
A., Rao, V., and Navon, I
Renganathan, S. A., Rao, V., and Navon, I. M. (2023). Camera: A method for cost-aware, adaptive, multifidelity, efficient reliability analysis.Journal of Computational Physics, 472:111698
2023
-
[42]
and Mahadevan, S
Sankararaman, S. and Mahadevan, S. (2012). Likelihood-based approach to multidisci- plinary analysis under uncertainty.Journal of Mechanical Design, 134(3):031008
2012
-
[43]
Sanson, F., Le Maitre, O., and Congedo, P. M. (2019). Systems of gaussian process models for directed chains of solvers.Computer Methods in Applied Mechanics and Engineering, 352:32–55
2019
-
[44]
W., Mauery, T
Simpson, T. W., Mauery, T. M., Korte, J. J., and Mistree, F. (2001). Kriging models for global approximation in simulation-based multidisciplinary design optimization.AIAA Journal, 39(12):2233–2241
2001
-
[45]
Sobieszczanski-Sobieski, J. (1990). Sensitivity of complex, internally coupled systems. AIAA Journal, 28(1):153–160
1990
-
[46]
Swersky, K., Snoek, J., and Adams, R. P. (2013). Multi-task bayesian optimization. In Burges, C., Bottou, L., Welling, M., Ghahramani, Z., and Weinberger, K., editors, Advances in Neural Information Processing Systems, volume 26. Curran Associates, Inc
2013
-
[47]
Yao, W., Chen, X., Ouyang, Q., and van Tooren, M. (2012). A surrogate based multistage-multilevel optimization procedure for multidisciplinary design optimization. Structural and Multidisciplinary Optimization, 45(4):559–574. 30
2012
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.