REVIEW 4 major objections 6 minor 52 references
Adversarial Autoencoders in Operator Learning
T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read Adding an adversarial discriminator to DeepONets and Koopman autoencoders cuts prediction error by roughly 5% to 10% in the small-data regime.
desk verdict A sensible small-data regularization idea with public code, but single-run numbers don't yet establish the claimed 5–10% improvement. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The mechanism is the adversarial autoencoder: an encoder maps an input into a latent space, a decoder maps the latent point back to the original space, and a discriminator tries to tell true encodings apart from random points drawn from a distribution matched to the encodings. The encoder is trained to fool the discriminator, with the two trained on alternating epochs, and noise scaled to 0.025 times the standard deviation of the true encodings is added before the discriminator. The paper argues this encourages the encoder to use the latent space continuously, and for Koopman autoencoders it is applied on top of the usual reconstruction, consistency, and near-unitary terms for the learned linear operator.
What would settle it
Retrain the baseline and adversarial models on the same equations with identical hyperparameters, random seeds, and initialization, sweeping the discriminator noise scale and training-set size; if the adversarial errors are not consistently lower across seeds, the reported 5% to 10% gains would not replicate.
Extended reading notes
Core claim
The central discovery is that an adversarial discriminator on the latent representation reduces prediction error for two operator-learning autoencoders in the small-data regime. For DeepONets, the relative error on Burger's equation drops from 3.860e-2 to 3.707e-2 (a 4.0% improvement) and on the KdV equation from 3.059e-2 to 2.795e-2 (8.6%). For Koopman autoencoders, the pendulum error drops from 2.615e-3 to 1.922e-3 (26.5%), the Lorenz error from 5.874e-2 to 4.703e-2 (19.9%), and the fluid-attractor error from 2.833e-5 to 2.637e-5 (6.9%). The paper also reports that with larger training sets the adversarial addition no longer helped and sometimes performed slightly worse.
Load-bearing premise
The load-bearing premise is that the baseline and adversarial models differ only by the added discriminator, with the same hyperparameters, noise, and training budget, but the paper never explicitly states that this control was enforced, and the noise scale and data sizes appear to have been selected where gains showed up.
Editorial extensions
If this is right
- If the result holds, practitioners with limited training data can add a latent discriminator to DeepONets and Koopman autoencoders and expect a typical 5% to 10% error reduction without changing the operator architecture.
- The benefit appears only in the small-data regime, so the discriminator is acting as a regularizer rather than a universal accuracy booster.
- For Koopman autoencoders, a more continuous latent space could also make the learned linear dynamics better behaved for prediction and downstream control.
- The recommended practice would extend to other autoencoder-based operator learners, since the added component is generic to the latent-space bottleneck.
- Training cost increases slightly, but the regime where the trick helps is already cheap to train, so the trade-off is favorable in the paper's tests.
Reading between the lines
- The paper reports single runs without seed sweeps, so the natural next step is to measure the distribution of the improvement; adversarial training is sensitive to initialization, and the 5% to 10% figure may vary noticeably across seeds.
- The noise scale and data sizes were chosen after seeing where improvements appeared, so an out-of-sample test on a new equation with a fixed protocol would tell whether the gains generalize or are tuned to these benchmarks.
- A non-adversarial latent prior, such as a Gaussian penalty on the encodings, could be compared against the discriminator to test whether the mechanism is the adversarial game itself or simply regularization of the latent distribution.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes adding an adversarial discriminator to two autoencoder-based neural operator architectures, DeepONets and Koopman autoencoders, with the aim of improving accuracy in the small-data regime. The authors compare adversarial and non-adversarial variants on five benchmark differential equations (Burger, KdV, pendulum, Lorenz, and fluid attractor) and report lower test errors for the adversarial variant in all five cases. The paper provides no theoretical derivation; its contribution is an empirical claim that adversarial training yields roughly 5-10% accuracy improvements when training data are scarce.
Significance. If the empirical claim is validated, the modification is simple, architecture-agnostic, and potentially useful for operator learning with limited data. The paper reports consistent directional improvements across five external benchmarks and makes its code publicly available, which are strengths. However, the significance is currently limited by the absence of variance estimates, by the post hoc selection of the small-data regime, and by a headline quantification that does not match the reported tables. As it stands, the contribution is a suggestive empirical observation rather than an established result.
major comments (4)
- [Section 5, Tables 1 and 2] Each cell in Tables 1 and 2 is a single error value with no error bars, no repeated seeds, and no statement that the adversarial and baseline runs share identical hyperparameters, initializations, noise draws, and training budgets. Because the experiments deliberately use very small training sets (20-48 samples), seed-to-seed variance is expected to be large, especially for the chaotic Lorenz system, and a single favorable draw for the adversarial variant could plausibly produce improvements of the reported magnitude. Repeated-seed statistics (e.g., mean and standard deviation over at least 5-10 independent runs) are needed to support the claim that the adversarial addition routinely improves accuracy.
- [Section 5, data-size paragraph] The data sizes (25, 50, 20, 48, 40) are described as 'roughly the minimal amount of data needed' and as the regime where the adversarial addition helps, but this selection is made after observing the results. This post hoc choice invites selection bias: the comparison may only hold at data sizes chosen to make the effect appear. The authors should either pre-specify the data sizes or justify them independently, for example by reporting error-versus-data-size curves and showing where the adversarial benefit is stable.
- [Section 6 vs. Tables 1 and 2] The conclusion's 'about 5% to 10%' is internally inconsistent with the reported improvements: 4.0% for Burger, 8.6% for KdV, 26.5% for pendulum, 19.9% for Lorenz, and 6.9% for fluid attractor. Either the headline quantification should be corrected, or the paper should define an aggregate measure that makes '5% to 10%' meaningful.
- [Sections 2 and 5] The adversarial loss weight is only described qualitatively as 'weighted by the accuracy loss,' and the noise scale is fixed at 0.025 with no sensitivity analysis. Without evidence that the results are robust to reasonable variations of these hyperparameters, it is unclear whether the reported gains are specific to a single favorable configuration.
minor comments (6)
- [Abstract and Section 1] The text contains multiple grammatical errors, including 'DeepONets are Koopman autoencoders are both' and 'the use an adversarial addition'; these should be corrected with careful proofreading.
- [Section 4] The differential equations are typeset incorrectly (e.g., d2θ/d2t instead of d²θ/dt², ∂2u/∂2x instead of ∂²u/∂x²); these should be fixed for readability.
- [Section 3.2] In the loss formula, 'where E is the encoder, R is the encoder' should read 'R is the decoder'; the equation also deserves a number for reference.
- [Section 2] The binary cross-entropy formula is missing a minus sign; as written b·log(a) + (1−b)·log(1−a) is a log-likelihood to be maximized, not the usual loss to be minimized.
- [Section 3.1] The phrase 'Deep neural operators, which are abbreviated as DeepONets' does not match the standard expansion of DeepONet as Deep Operator Network (see reference [10]); please correct or justify this terminology.
- [Section 5] The description of the noise scale says the standard deviation of the true encodings scaled by 0.025, but it does not explain how the true encodings are estimated; please clarify this procedure.
Circularity Check
No circular derivation: the adversarial-accuracy claim is empirical against external equation solvers, with only minor non-load-bearing self-citations.
full rationale
The paper makes no analytical derivation whose conclusion is equivalent to its assumptions. Its central claim is an empirical comparison: adding an adversarial discriminator to DeepONets and Koopman autoencoders improves test error on Burger's equation, the KdV equation, the pendulum, the Lorenz system, and a fluid-attractor ODE. The targets are solutions obtained by independent numerical integrators (Runge-Kutta and standard PDE solvers), so the outcome is not determined by construction from any fitted parameter or self-citation. The only self-citations are methodological: the Koopman training loss is taken from the authors' prior work [40], and stochastic weight averaging is attributed to [45]. These choices apply equally to baseline and adversarial models, so they do not by themselves force the reported improvement. The selection of data sizes and noise scale is a modeling choice rather than a fitted parameter renamed as a prediction. Therefore the paper is not circular. Separate concerns about missing seed variance, uncontrolled hyperparameter comparison, and the discrepancy between the abstract's '5% to 10%' and the table's larger improvements are statistical-validity issues, not circularity, and do not change this verdict.
Assumptions & free parameters
free parameters (3)
- discriminator noise scale =
0.025
- adversarial loss weight =
not reported
- training sample sizes =
Burger 25, KdV 50, pendulum 20, Lorenz 48, fluid 40
assumptions (4)
- standard math Runge-Kutta method accurately solves the ODE/PDE initial value problems used to generate labels.
- domain assumption Randomly generated symbolic initial conditions are representative of the operator-learning task.
- domain assumption Adversarial training with binary cross entropy encourages the encoder to use the entire latent space continuously.
- domain assumption The unitary loss term and tridiagonal mask improve Koopman autoencoder training.
Cite this review
Pith. "Pith review of Adversarial Autoencoders in Operator Learning." pith.science (2026). https://pith.science/paper/PD23MQAI
@misc{pith2026241207811,
author = {Pith},
title = {Pith review of: Adversarial Autoencoders in Operator Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/PD23MQAI}},
note = {Machine review of arXiv:2412.07811}
}
read the original abstract
DeepONets and Koopman autoencoders are two prevalent neural operator architectures. These architectures are autoencoders. An adversarial addition to an autoencoder have improved performance of autoencoders in various areas of machine learning. In this paper, the use an adversarial addition for these two neural operator architectures is studied.
Figures
Reference graph
Works this paper leans on
-
[1]
Operator learning: Algorithms and analysis
Nikola B Kovachki, Samuel Lanthaler, and Andrew M Stuart. “Operator learning: Algorithms and analysis”. In:arXiv preprint arXiv:2402.15715(2024)
arXiv 2024
-
[2]
A mathematical guide to operator learning
Nicolas Boullé and Alex Townsend. “A mathematical guide to operator learning”. In:arXiv preprint arXiv:2312.14688 (2023)
arXiv 2023
-
[3]
NeuralNetworkApproximationstoSolutionOperatorsforPartialDifferential Equations
NickWinovich.“NeuralNetworkApproximationstoSolutionOperatorsforPartialDifferential Equations”. PhD thesis. Purdue University, 2021
work page 2021
-
[4]
Reducing the dimensionality of data with neural networks
Geoffrey E Hinton and Ruslan R Salakhutdinov. “Reducing the dimensionality of data with neural networks”. In:science 313.5786 (2006), pp. 504–507
work page 2006
-
[5]
David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. “Learning internal representa- tions by error propagation, parallel distributed processing, explorations in the microstructure of cognition, ed. de rumelhart and j. mcclelland. vol. 1. 1986”. In:Biometrika 71.599-607 (1986), p. 6. 11
work page 1986
-
[6]
Auto-encoding variational bayes
Diederik P Kingma. “Auto-encoding variational bayes”. In: arXiv preprint arXiv:1312.6114 (2013)
arXiv 2013
-
[7]
Alireza Makhzani et al. “Adversarial autoencoders”. In: arXiv preprint arXiv:1511.05644 (2015)
arXiv 2015
-
[8]
Binary cross entropy with deep learning technique for image classification
Usha Ruby and Vamsidhar Yendapalli. “Binary cross entropy with deep learning technique for image classification”. In:Int. J. Adv. Trends Comput. Sci. Eng9.10 (2020)
work page 2020
Show all 52 references
-
[9]
Generative deep learning: teaching machines to paint
David Foster. “Generative deep learning: teaching machines to paint”. In:Write, Compose, and Play (Japanese Version) O’Reilly Media Incorporated(2019), pp. 139–140
2019
-
[10]
LearningnonlinearoperatorsviaDeepONetbasedontheuniversalapproximation theorem of operators
LuLuetal.“LearningnonlinearoperatorsviaDeepONetbasedontheuniversalapproximation theorem of operators”. In:Nature Machine Intelligence 3.3 (Mar. 2021), pp. 218–229.issn: 2522-5839. doi: 10.1038/s42256-021-00302-5. url: http://dx.doi.org/10.1038/s42256- 021-00302-5
2021 doi
-
[11]
Error estimates for deep- onets: A deep learning framework in infinite dimensions
Samuel Lanthaler, Siddhartha Mishra, and George E Karniadakis. “Error estimates for deep- onets: A deep learning framework in infinite dimensions”. In:Transactions of Mathematics and Its Applications6.1 (2022), tnac001
2022
-
[12]
A physics-informed variational DeepONet for predicting crack path in quasi-brittle materials
Somdatta Goswami et al. “A physics-informed variational DeepONet for predicting crack path in quasi-brittle materials”. In:Computer Methods in Applied Mechanics and Engineering391 (2022), p. 114587
2022
-
[13]
Novel DeepONet architecture to predict stresses in elastoplastic structures with variable complex geometries and loads
Junyan He et al. “Novel DeepONet architecture to predict stresses in elastoplastic structures with variable complex geometries and loads”. In:Computer Methods in Applied Mechanics and Engineering 415 (2023), p. 116277
2023
-
[14]
Learning time-dependent PDE via graph neural networks and deep operator network for robust accuracy on irregular grids
Sung Woong Cho, Jae Yong Lee, and Hyung Ju Hwang. “Learning time-dependent PDE via graph neural networks and deep operator network for robust accuracy on irregular grids”. In: arXiv preprint arXiv:2402.08187(2024)
2024 arXiv
-
[15]
A comprehensive and fair comparison of two neural operators (with practical extensions) based on fair data
Lu Lu et al. “A comprehensive and fair comparison of two neural operators (with practical extensions) based on fair data”. In:Computer Methods in Applied Mechanics and Engineering 393 (2022), p. 114778
2022
-
[16]
Deep learning for universal linear embeddings of nonlinear dynamics
Bethany Lusch, J Nathan Kutz, and Steven L Brunton. “Deep learning for universal linear embeddings of nonlinear dynamics”. In:Nature communications9.1 (2018), p. 4950. 12
2018
-
[17]
A survey on the methods and results of data-driven koopman analysis in the visualization of dynamical systems
Nishaal Parmar, Hazem H Refai, and Thordur Runolfsson. “A survey on the methods and results of data-driven koopman analysis in the visualization of dynamical systems”. In:IEEE Transactions on Big Data8.3 (2020), pp. 723–738
2020
-
[18]
Learning data-driven stable Koopman operators
Giorgos Mamakoukas, Ian Abraham, and Todd D Murphey. “Learning data-driven stable Koopman operators”. In:Free radical biology & medicine.(2020)
2020
-
[19]
Data-driven nonlinear stabilization using koop- man operator
Bowen Huang, Xu Ma, and Umesh Vaidya. “Data-driven nonlinear stabilization using koop- man operator”. In:The Koopman Operator in Systems and Control: Concepts, Methodologies, and Applications(2020), pp. 313–334
2020
-
[20]
Data-driven approximation of the Koopman generator: Model reduc- tion, system identification, and control
Stefan Klus et al. “Data-driven approximation of the Koopman generator: Model reduc- tion, system identification, and control”. In:Physica D: Nonlinear Phenomena406 (2020), p. 132416
2020
-
[21]
Koopman invariant subspaces and finite linear representations of nonlinear dynamical systems for control
Steven L Brunton et al. “Koopman invariant subspaces and finite linear representations of nonlinear dynamical systems for control”. In:PloS one 11.2 (2016), e0150171
2016
-
[22]
Applied koopmanism
Marko Budišić, Ryan Mohr, and Igor Mezić. “Applied koopmanism”. In:Chaos: An Interdis- ciplinary Journal of Nonlinear Science22.4 (2012)
2012
-
[23]
Multiresolution dynamic mode decomposi- tion
J Nathan Kutz, Xing Fu, and Steven L Brunton. “Multiresolution dynamic mode decomposi- tion”. In:SIAM Journal on Applied Dynamical Systems15.2 (2016), pp. 713–735
2016
-
[24]
Learning Koopman invariant sub- spaces for dynamic mode decomposition
Naoya Takeishi, Yoshinobu Kawahara, and Takehisa Yairi. “Learning Koopman invariant sub- spaces for dynamic mode decomposition”. In:Advances in neural information processing sys- tems 30 (2017)
2017
-
[25]
Koopman-mode decomposition of the cylinder wake
Shervin Bagheri. “Koopman-mode decomposition of the cylinder wake”. In:Journal of Fluid Mechanics 726 (2013), pp. 596–623
2013
-
[26]
M Budišic, R Mohr, and I Mezic.The Koopman Operator in Systems and Control: Concepts, Methodologies, and Applications. 2020
2020
-
[27]
Learning compositional koopman operators for model-based control
Yunzhu Li et al. “Learning compositional koopman operators for model-based control”. In: arXiv preprint arXiv:1910.08264(2019)
2019 arXiv
-
[28]
Data-driven approximations of dy- namical systems operators for control
Eurika Kaiser, J Nathan Kutz, and Steven L Brunton. “Data-driven approximations of dy- namical systems operators for control”. In:The Koopman Operator in Systems and Control: Concepts, Methodologies, and Applications(2020), pp. 197–234. 13
2020
-
[29]
Deep learning of Koopman representation for control
Yiqiang Han, Wenjian Hao, and Umesh Vaidya. “Deep learning of Koopman representation for control”. In:2020 59th IEEE Conference on Decision and Control (CDC). IEEE. 2020, pp. 1890–1895
2020
-
[30]
Koopman-based control of a soft continuum manipulator under variable loading conditions
Daniel Bruder et al. “Koopman-based control of a soft continuum manipulator under variable loading conditions”. In:IEEE robotics and automation letters6.4 (2021), pp. 6852–6859
2021
-
[31]
Modeling and control of soft robots using the koopman operator and model predictive control
Daniel Bruder et al. “Modeling and control of soft robots using the koopman operator and model predictive control”. In:arXiv preprint arXiv:1902.02827(2019)
2019 arXiv
-
[32]
A data-driven koopman model predictive control framework for nonlinear partial differential equations
Hassan Arbabi, Milan Korda, and Igor Mezić. “A data-driven koopman model predictive control framework for nonlinear partial differential equations”. In:2018 IEEE Conference on Decision and Control (CDC). IEEE. 2018, pp. 6409–6414
2018
-
[33]
Model-based control using Koop- man operators
Ian Abraham, Gerardo De La Torre, and Todd D Murphey. “Model-based control using Koop- man operators”. In:arXiv preprint arXiv:1709.01568(2017)
2017 arXiv
-
[34]
Hamiltonian systems and transformation in Hilbert space
Bernard O Koopman. “Hamiltonian systems and transformation in Hilbert space”. In:Pro- ceedings of the National Academy of Sciences17.5 (1931), pp. 315–318
1931
-
[35]
Modern Koopman theory for dynamical systems
Steven L Brunton et al. “Modern Koopman theory for dynamical systems”. In:arXiv preprint arXiv:2102.12086 (2021)
2021 arXiv
-
[36]
What is the Koopman operator? a simplified treatment for discrete-time systems
Adam L Bruce, Vera M Zeidan, and Dennis S Bernstein. “What is the Koopman operator? a simplified treatment for discrete-time systems”. In:2019 American Control Conference (ACC). IEEE. 2019, pp. 1912–1917
2019
-
[37]
Understanding quantum physics: A user’s manual
Michael A Morrison. Understanding quantum physics: A user’s manual. 1990
1990
-
[38]
Hans C. Ohanian. Principles of Quantum Mechanics. 1989
1989
-
[39]
Griffiths
David J. Griffiths. Introduction to Quantum Mechanics. 2018
2018
-
[40]
Dustin Enyeart and Guang Lin.Loss Terms and Operator Forms of Koopman Autoencoders
-
[41]
Finite difference methods for ordinary and partial differential equations: steady-state and time-dependent problems
Randall J LeVeque. Finite difference methods for ordinary and partial differential equations: steady-state and time-dependent problems. SIAM, 2007
2007
-
[42]
A hierarchy of low-dimensional models for the transient and post- transient cylinder wake
Bernd R Noack et al. “A hierarchy of low-dimensional models for the transient and post- transient cylinder wake”. In:Journal of Fluid Mechanics497 (2003), pp. 335–363. 14
2003
-
[43]
Randall J LeVeque and Randall J Leveque.Numerical methods for conservation laws. Vol. 214. Springer, 1992
1992
-
[44]
Interaction of “solitons
Norman J Zabusky and Martin D Kruskal. “Interaction of “solitons" in a collisionless plasma and the recurrence of initial states”. In:Physical review letters15.6 (1965), p. 240
1965
-
[45]
Some Best Practices in Operator Learning
Dustin Enyeart and Guang Lin. Some Best Practices in Operator Learning. 2024. arXiv: 2412.06686 [cs.LG]. url: https://arxiv.org/abs/2412.06686
2024 arXiv
-
[46]
Automatic differentiation in PyTorch
Adam Paszke et al. “Automatic differentiation in PyTorch”. In:NIPS-W. 2017
2017
-
[47]
Programming pytorch for deep learning: Creating and deploying deep learning applications
Ian Pointer. Programming pytorch for deep learning: Creating and deploying deep learning applications. O’Reilly Media, 2019
2019
-
[48]
UvA Deep Learning Tutorials
Phillip Lippe. UvA Deep Learning Tutorials. https://uvadlc-notebooks.readthedocs.io/ en/latest/. 2024
2024
-
[49]
url: https://github.com/Lightning-AI/pytorch-lightning
Torch Lightning. url: https://github.com/Lightning-AI/pytorch-lightning
-
[50]
Hydra - A framework for elegantly configuring complex applications
Omry Yadan. Hydra - A framework for elegantly configuring complex applications. Github
-
[2019]
url: https://github.com/facebookresearch/hydra. 15
-
[2024]
url: https://arxiv.org/abs/2412.04578
arXiv: 2412.04578 [cs.LG]. url: https://arxiv.org/abs/2412.04578
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.