{"id":"072013d0-6904-41f8-8574-8ce671d4d955","arxiv_id":"2412.02984","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Koopman Model Averaging blends an ensemble of linear Koopman embeddings with data-based Bayesian weights and produces a single weighted linear model for prediction and control.","lead":"This paper trains several linear approximations of a nonlinear control system and combines them using Bayesian model averaging. The aim is to make predictions and controllers more reliable across operating conditions than relying on any single model.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim that KMA generalizes better than a single model is not established: the paper's only reported validation loss for the weighted model in Section V-B is worse than the single-model baseline (2.82e-5 vs 1.16e-5), and the claimed improvements rest on selected single trajectories…","rationale":"The reader's weakest assumption is the shared feature map, and I agree that is a real limitation: if the base model's embedding is poor, the ensemble cannot repair it. However, the more decisive problem is that the paper's own quantitative evidence does not support the central claim. In cartpole, the proposed method's reported validation loss is worse than the single-model baseline; in Duffing, the weighted model's validation loss is not reported. The claimed improvements are based on single-trajectory plots and control results with no error bars or repeated trials. Because the central claim is an empirical generalization claim, missing or contradictory quantitative evidence makes the current verdict unverified. I would move from CONDITIONAL to UNVERDICTED: the method is clearly described and could be validated, but until a systematic randomized comparison is provided, no reliable verdict can be reached.","tokens_in":9644,"tokens_out":10538,"duration_ms":108886,"concrete_test":"Re-run both examples with a systematic protocol: for 50 random data splits, train the normal NN baseline on the full D, train KMA per Algorithm 1, and evaluate on 100 held-out initial conditions. Report mean and standard deviation over splits of (i) validation loss of the actual weighted model on Da, (ii) multi-step prediction RMSE, and (iii) MPC tracking error. If KMA's means are not lower than the baseline's with non-overlapping confidence intervals—or if the Section V-B value of 2.82e-5 is indeed the weighted model's loss—the generalization claim should be rejected. Also report whether the test initial conditions are drawn from the same distribution as Da.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"The load-bearing issue is evidential rather than purely structural. The claim that the weighted model (28)-(29) has 'better predictive capability as well as generalizability compared to obtaining a single model only' is supported only by visual comparison of a few hand-selected trajectories. In the cartpole example (Section V-B), the reported validation loss of the proposed KMA model is 2.82e-5, whereas the normal NN baseline achieves 1.16e-5; this is direct quantitative evidence against the generalization claim. In the Duffing example (Section V-A), no validation loss for the actual weighted model is reported at all—only the base model's loss (1.81e-5) is compared with the baseline (7.60e-6), leaving the weighted model's generalization unquantified. Moreover, the KMA weights are computed on Da (Algorithm 1, Step 5), while the normal NN baseline is trained on all data including Da; hence Da functions as a validation set for KMA but as training data for the baseline, confounding the comparison. The conclusion that KMA 'achieves better state predictive accuracy as well as greater generalizability' is therefore not supported by the presented quantitative evidence.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a model-averaging method for Koopman operator-based modeling of unknown nonlinear control systems. A base neural-network Koopman model is trained on a subset of the data to obtain feature maps; an ensemble of linear embedding models is then fitted on different subsets via EDMD, and Bayesian Model Averaging (BMA) weights, computed on a calibration set D_a, are used to form a weighted linear model (Eqs. (28)-(29)). The authors claim that this weighted model has better state-prediction accuracy and generalizability than a single model, and they support the claim with two numerical examples (Duffing oscillator and cartpole) covering prediction, LQR, and MPC.","tokens_in":9864,"tokens_out":5458,"duration_ms":56353,"significance":"If the empirical claims held, the paper would present a simple, modular way to combine multiple Koopman models into a single linear embedding model while remaining model-uncertainty aware. The derivation of the weighted model from BMA is elementary and correct, the algorithm is explicitly stated, and the idea of averaging in the embedding space is reasonable. However, the evidence for the central generalization claim is thin: only two examples, single hand-selected trajectories, no error bars or statistical analysis, and in both examples the reported validation loss of the proposed model is worse than that of the normal NN baseline. The significance is therefore conditional on a substantially strengthened empirical evaluation.","major_comments":[{"comment":"The central claim that the proposed weighted model has better predictive capability and generalizability than a single model is not supported by the paper's own quantitative results. In the cartpole example, the validation loss of the proposed model is 2.82e-5, which is higher than the normal NN model's 1.16e-5, and the claimed advantage rests entirely on one hand-selected trajectory (IC 2) with no error bars, multiple seeds, or rollout statistics. The authors should report prediction errors (e.g., mean and standard deviation over many initial conditions) and establish statistical significance rather than relying on a single favorable trajectory.","section":"Section V-B, Fig. 2b"},{"comment":"The comparison between the proposed method and the baselines is confounded by the role of the data set D_a. The normal NN and EDMD baselines are trained on the entire data set D = (∪_i D_i) ∪ D_a, while the proposed KMA uses D_a only to compute the BMA weights (Algorithm 1, Step 5). Thus D_a acts as a validation/calibration set for KMA but as a training set for the baselines, making the reported validation losses and closed-loop comparisons not apples-to-apples. The authors should either train the baselines only on ∪_i D_i and use a separate held-out test set for all methods, or clearly define and use a test set that no method has seen during any stage of training or weight computation.","section":"Section V-A and Algorithm 1, Step 5"},{"comment":"The computation of the BMA weights is under-specified, which is load-bearing because the weighted model (28)-(29) is determined entirely by these weights. The text states only that 'PyMC is used to compute w_i' but does not specify the likelihood model, the priors, or how the Gaussian assumptions in Eqs. (24)-(25) are used, nor how the integrals in Eq. (16) are approximated. Without these details, the method is not reproducible. Please provide the exact probabilistic model, the estimation procedure (e.g., the LOO approximation used), and all relevant hyperparameters.","section":"Section IV-A, Eq. (19)"},{"comment":"The claimed superiority of the proposed model over the normal NN model is based on qualitative observations of single closed-loop or prediction trajectories. In the Duffing example, no validation loss for the actual weighted model is reported—only that of the base model—so the generalization claim for KMA is unquantified; the single MPC trajectory (Fig. 1e) is not enough to establish the advantage. The authors should provide quantitative performance metrics (e.g., tracking error, state prediction RMSE over an ensemble of initial conditions, and closed-loop cost) and, for Duffing, report the weighted model's validation loss as well.","section":"Section V-A, Fig. 1e and Section V-B, Fig. 2b"}],"minor_comments":[{"comment":"There is a typo in Eq. (12): the virtual system should be written as ξ_{k+1} = A ξ_k + B u_k, not ξ_{k+1} = A ξ_{k+1} + B u_k.","section":"Section II, Eq. (12)"},{"comment":"The expression for elpd_i is confusing: the integral over the true distribution p_t(\\tilde{q}_j) is not the quantity that is evaluated in practice, and the notation \\tilde{q}_j is not defined clearly. Please rewrite Eq. (19) using the usual definition of the expected log pointwise predictive density or the LOO approximation used to compute it.","section":"Section IV-A, Eq. (19)"},{"comment":"The phrase 'posterior model evidence' for p(M_i | D) is imprecise; Eq. (15) gives the posterior model probability. The marginal likelihood in Eq. (16) is the model evidence.","section":"Section IV-A"},{"comment":"The discretization used to obtain the difference equation from the continuous Duffing dynamics is not stated; the text says only that a first-order discretization is used with Δt = 0.01. Please give the explicit update formula.","section":"Section V-A"},{"comment":"The numerical setup is missing several training details needed for reproducibility: the values of the loss weights λ1 and λ2, the optimizer, learning rate, number of epochs, batch size, and random seeds. Please include these details or a pointer to released code.","section":"Section V-A and V-B"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is clearly written and the BMA-weighted model derivation is sound, but the empirical section is far too thin to support the advertised generalization claim. The reported validation losses for the proposed method are worse than the single-model baseline in both examples, and the single favorable trajectories may be selective. The authors should be asked to redo the evaluation with a clear train/calibration/test split, multiple random seeds, and statistical reporting. I do not see a fundamental unsoundness in the method itself."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The useful core is Eqs. (28)-(29): a weighted Koopman embedding whose matrices are convex combinations of ensemble members' A, B, C with pseudo-BMA weights, all members sharing one neural feature map. That combination is new as far as I can tell, and the derivation is clean. If you want a quick, implementable way to hedge over EDMD fits, this is a sensible recipe.\n\nThe paper is honest about what it does and does not do. It does not claim new theory, and the BMA machinery is standard. The algorithm is easy to reproduce from the text. Credit where due: the writing is clear, the setup is standard, and the authors correctly note that a single model may not generalize.\n\nThe soft spot is the evidence, and it is load-bearing. The claim that KMA has better predictive capability and generalizability is supported by selected trajectories only. The reported validation loss for KMA in the cartpole example is 2.82e-5 versus 1.16e-5 for the normal NN baseline, which is direct evidence against the claim. For Duffing, no validation loss for the weighted model is reported at all; only the base model's loss is compared with the baseline. The comparison is also confounded: the normal NN baseline is trained on all data including the calibration set Da, while KMA uses Da to compute weights, so Da is a validation set for KMA and training data for the baseline. That makes the better MPC tracking hard to interpret. Add no error bars, no code or data, and one or two hand-picked trajectories per claim.\n\nThe shared feature map is another structural risk. If the base neural-network embedding is poor, the EDMD fits and the weighted sum cannot repair it. The paper reports no sensitivity analysis on this choice. The math itself is elementary and correct; the method is plausible and likely useful for practitioners, but the empirical case needs work: more trials, proper held-out evaluation, error bars, and a baseline that respects the same data split. The conclusion overstates the case; the numerical section undermines rather than supports better generalization.\n\nFor a reader working on Koopman models with uncertainty, this is worth a skim, and the weighted linear embedding idea may be worth citing once it is better supported. A serious referee should look at it, but the verdict should be major revision, not accept. My take: send it to review, but put the onus on the authors to fix the evaluation.","headline":"A clean but thinly evidenced model-averaging recipe for Koopman embeddings; worth reviewing, but the generalization claim is contradicted by the paper's own validation numbers.","tokens_in":10417,"tokens_out":2389,"would_cite":false,"duration_ms":23866,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["93C10","93C55","37M99","62F15"],"pacs":[],"model":"deepseek-v4-flash","headline":"Weighted Koopman ensemble beats single-model prediction and control.","keywords":["Koopman operator","model averaging","Bayesian model averaging","neural network features","extended dynamic mode decomposition","model predictive control","nonlinear control systems","linear embedding model"],"falsifier":"Run the same Duffing and cartpole experiments but train the base model on a narrow slice of the state space (for example, initial conditions only in $[0,1]^2$, with the ensemble subsets drawn from the full $[-3,3]^2$ range) so that the shared $g$ is demonstrably poor outside the base region. If the weighted model still beats the single normal neural-network model on the full-range test data, the shared-feature assumption is not the limiting factor; if KMA degrades to the level of the individual EDMD fits, that would show the result hinges on having a good base feature map.","tokens_in":9364,"feed_emoji":"🎛️","tokens_out":8292,"duration_ms":73505,"temperature":0.7,"pith_summary":"The paper proposes Koopman Model Averaging (KMA), a data-driven way to build a Koopman-operator linear embedding model for nonlinear control systems that is more accurate and generalizes better than any single model trained on the same data. It trains one neural-network feature map on a first data subset, fits a small ensemble of Extended Dynamic Mode Decomposition models—each a linear embedding of the form $z_{k+1}=A_i z_k+B_i u_k$—on separate subsets, and then averages their parameters using Bayesian Model Averaging weights computed from holdout data. The result is a single weighted linear embedding model, equations (28)–(29), whose predicted next state is a posterior-weighted sum of the ensemble's outputs. In numerical tests on the Duffing oscillator and the cartpole, KMA tracks references and predicts states at least as well as, and in several cases better than, both a standard EDMD model and a normal neural-network model, even when the single model has lower validation loss. The authors' claim is that this weighting scheme compensates for each ensemble member's errors across different operating regimes, giving a practical surrogate for both prediction and controller design.","feed_headline":"Weighted Koopman ensemble beats single-model prediction and control","feed_subtitle":"Bayesian averaging of Koopman embeddings improves prediction and control without redesigning the controller.","key_machinery":"The central object is the weighted linear embedding model of Equations (28)–(29), whose parameters are Bayesian-Model-Averaging-weighted sums of the ensemble members' parameters. The key mechanism is that the posterior expectation of the next embedded state, $\\mathbb{E}[z_{k+1}\\mid D]=\\sum_i w_i(A_i z_k+B_i u_k)$, preserves the form of a single linear time-invariant system, so the averaged model still supports linear LQR and MPC designs. The feature map $g(\\cdot;\\theta_g)$ learned once by a neural-network base model on subset $D_1$ is shared by all ensemble members; the remaining members are obtained by closed-form EDMD fits on data subsets, and the weights $w_i$ are approximated by an AIC-type pseudo-BMA (expected log pointwise predictive density) computed on a holdout set.","core_discovery":"The central claim is that a weighted linear embedding model, formed as $z_{k+1}\\approx(\\sum_i w_i A_i)z_k+(\\sum_i w_i B_i)u_k$ and $x^{\\mathrm{pred}}_{k+1}=(\\sum_i w_i C_i A_i)z_k+(\\sum_i w_i C_i B_i)u_k$, where the weights $w_i$ approximate posterior model evidence, is a better and more generalizable predictor of nonlinear controlled dynamics than any single Koopman-based model in the ensemble. The paper shows that applying Bayesian Model Averaging to point estimates of the embedded state and the original state collapses the ensemble back into the same linear embedding structure used for LQR and MPC, so the model-uncertainty-aware average costs no additional control-design complexity. The Duffing oscillator and cartpole experiments support the claim: KMA matches or outperforms both an EDMD model and a neural-network-trained single model in state prediction, LQR stabilization, and reference-tracking MPC, despite the base model used in KMA having a larger validation loss than the normal neural-network model, which the authors interpret as better behavior on unseen operating regimes.","pith_inferences":["The success of KMA likely depends on the holdout set $D_a$ being representative of where the model will be used, since the pseudo-BMA weights are estimated on that set; a deployment distribution that shifts away from $D_a$ would require recomputing weights.","The same parameter-averaging recipe could be applied to bilinear Koopman embeddings or to models with nonlinear decoders, since only the linear-in-the-parameters structure is used in the derivation; this is a direct extension the paper does not test.","A natural next experiment is to compare KMA against other ensemble combination rules (plain averaging, stacking, or weights based on validation error) on the same benchmarks to isolate how much of the gain comes from Bayesian weighting versus from having multiple EDMD fits with a shared feature map.","The paper reports only point estimates of the posterior; extending to full predictive covariance would enable uncertainty-aware MPC, which would use the same Gaussian assumptions that are already stated."],"forward_implications":["Because the averaged model is itself linear time-invariant, any LQR, linear MPC, or other linear controller design can be applied directly to it; controller synthesis needs no change relative to single-model Koopman approaches.","The method turns a single expensive neural-network training into one base training plus several cheap closed-form EDMD fits, so improving generalization through ensembling does not require repeatedly solving the non-convex training problem.","The weighted parameters remain interpretable: each ensemble member contributes according to its posterior evidence on holdout data, which gives a principled way to discard or down-weight poorly performing models.","On the two tested systems, a model with a worse validation loss (the KMA base model) outperformed a better-fit single model in reference tracking and long-horizon prediction, indicating that validation loss on a finite data set is not a reliable proxy for closed-loop control performance."],"supporting_citations":[{"why":"Supplies the Extended Dynamic Mode Decomposition regression procedure used to fit each ensemble member's parameters and to define the linear embedding model structure.","marker":"[15]"},{"why":"Provides the Bayesian Model Averaging formalism, including the posterior model evidence and the weighted combination of model predictions that KMA adapts.","marker":"[14]"},{"why":"Supplies the AIC-type pseudo-BMA and expected log pointwise predictive density approximation used to compute the weights $w_i$ in Equation (19).","marker":"[25]"},{"why":"Establishes that EDMD-type approximations do not converge for control systems, the motivating gap that the model-uncertainty-aware averaging is designed to address.","marker":"[3]"},{"why":"Motivates the neural-network feature-map training used for the base model in Step 1 of Algorithm 1.","marker":"[21]"}],"fun_headline_variants":["Bayesian-averaged Koopman models improve prediction and control","Koopman ensemble via model averaging beats single-model control","Weighted Koopman ensemble matches or exceeds single models","Model averaging lifts Koopman prediction and control performance","KMA: Bayesian model averaging for robust Koopman control"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole method depends on the single feature map $g$ learned by the base model on the first data subset being a good enough embedding for all the operating regimes the other ensemble models are fit on; if that one map is poor outside the base training region, the later weighted-average fits cannot repair it, and the paper does not test how sensitive KMA is to this feature-map choice.","fun_headline_variants_meta":{"raw":{"variants":["Bayesian-averaged Koopman models improve prediction and control","Koopman ensemble via model averaging beats single-model control","Weighted Koopman ensemble matches or exceeds single models","Model averaging lifts Koopman prediction and control performance","KMA: Bayesian model averaging for robust Koopman control"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000242,"raw_usage":{"total_tokens":1514,"prompt_tokens":923,"completion_tokens":591,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":539,"completion_tokens_details":{"reasoning_tokens":508}},"tokens_in":539,"tokens_out":591,"duration_ms":6322,"temperature":1.0,"reasoning_tokens":508,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T22:53:58.626231+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same Duffing and cartpole experiments but train the base model on a narrow slice of the state space (for example, initial conditions only in $[0,1]^2$, with the ensemble subsets drawn from the full $[-3,3]^2$ range) so that the shared $g$ is demonstrably poor outside the base region. If the weighted model still beats the single normal neural-network model on the full-range test data, the shared-feature assumption is not the limiting factor; if KMA degrades to the level of the individual EDMD fits, that would show the result hinges on having a good base feature map.","supporting_citations":[{"cited_title":"A data driven approximation of the Koopman operator: Extending dynamic mode de- composition,","cited_arxiv_id":null,"evidence_quote":"Supplies the Extended Dynamic Mode Decomposition regression procedure used to fit each ensemble member's parameters and to define the linear embedding model structure."},{"cited_title":"Bayesian model averag- ing: A systematic review and conceptual classification,","cited_arxiv_id":null,"evidence_quote":"Provides the Bayesian Model Averaging formalism, including the posterior model evidence and the weighted combination of model predictions that KMA adapts."},{"cited_title":"Using Stack- ing to Average Bayesian Predictive Distributions (with Discussion),","cited_arxiv_id":null,"evidence_quote":"Supplies the AIC-type pseudo-BMA and expected log pointwise predictive density approximation used to compute the weights $w_i$ in Equation (19)."},{"cited_title":"Deep learning of Koopman repre- sentation for control,","cited_arxiv_id":null,"evidence_quote":"Motivates the neural-network feature-map training used for the base model in Step 1 of Algorithm 1."}],"review_version":1}