{"id":"fa30dee4-286b-4501-8cfc-4f96e082394a","arxiv_id":"2411.14839","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A Monte Carlo ensemble over DMD hyperparameters, called Bayesian Hankel-DMD, improves short-term ship motion forecasts and outputs uncertainty that tracks prediction error.","lead":"Researchers propose a Bayesian version of Hankel dynamic mode decomposition to predict ship motions in waves in real time, using a short window of past data and giving a forecast with uncertainty. Tested on simulated data for a destroyer in heavy beam-quartering seas, the method forecasts motions up to five wave periods and reports that uncertainty tracks forecast error.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's own Tables 3-4 show the Bayesian method is worse than deterministic for CFDShip-Iowa on JSD at all horizons and NAMMAE at one horizon, so the central 'improves across all metrics' claim is internally contradicted.","rationale":"Good-faith reading: the paper proposes a useful ensemble extension of Hankel-DMD, with clear methodology, reproducible metrics, and honest reporting of deterministic hyperparameter sweeps. The strongest load-bearing problem is not the mathematics of the ensemble but the scope of the headline claim. The authors themselves provide the falsifying evidence: in Table 4, for CFDShip-Iowa, the Bayesian JSD is higher (worse) than the best deterministic configuration at all three horizons, and in Table 3 NAMMAE is worse at one horizon. Since the conclusion explicitly claims improvement 'considering all the metrics,' the central claim cannot stand as written. This is an internal inconsistency, not a disagreement with external consensus. It is also independent of the prior-selection circularity noted by the reader: even on the exact data used to choose the priors, the claimed universal improvement is absent. The paper can be fixed by softening the conclusion, reporting per-case results, and adding a paired significance analysis. Because the method itself remains plausible and the issue is a calibration of claim, the conditional verdict stands. Prior-range generalization remains a secondary concern that should be addressed with an out-of-sample validation.","tokens_in":15989,"tokens_out":3954,"duration_ms":37391,"concrete_test":"Using the saved 250-starting-point forecasts, recompute Tables 3 and 4 and run paired Wilcoxon signed-rank tests for each solver x metric x horizon (lte = T, 2T, 5T). Report the number of cells where the Bayesian mean is worse and the p-values/effect sizes. If CFDShip-Iowa JSD remains worse or the comparisons are non-significant, the conclusion must be revised from 'significantly improves ... considering all metrics' to a per-solver, per-metric statement, and the prior-range selection should be validated on held-out data.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—Bayesian Hankel-DMD 'significantly improves the accuracy of the predictions compared to the deterministic method considering all the metrics' (Conclusion)—is contradicted by the paper's own summary tables. For CFDShip-Iowa, Bayesian JSD is worse than the best deterministic configuration at every horizon: 0.0250 vs 0.0141 (lte=T), 0.0293 vs 0.0217 (lte=2T), 0.0375 vs 0.0353 (lte=5T) (Table 4). Bayesian NAMMAE is also worse at lte=T: 0.2740 vs 0.2568 (Table 3). Only NRMSE improves in that block (Table 2). Thus the universal superiority claim is unsupported by the evidence as reported, regardless of generalization. A second, related weakness is that the uniform prior ranges (ltr/T ~ U(1,5), ld ~ U(0.5 ltr, 0.75 ltr)) in Section 5.2 are selected after inspecting deterministic performance on the same 250 test starting points (Section 5.1, Eq. 24), so the comparison is not an independent evaluation and the improvement could shrink on other seas, speeds, or hulls. But the immediate falsifier is internal: the aggregate tables do not support 'all metrics'.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a Bayesian extension of Hankel dynamic mode decomposition (Hankel-DMD) for nowcasting ship motions in waves, targeting digital-twin applications. The method treats two Hankel-DMD hyperparameters—the training window length ltr and the maximum delay length ld—as uniformly distributed random variables, computes many deterministic Hankel-DMD predictions via Monte Carlo sampling, and uses the sample mean and standard deviation as the forecast and its uncertainty. The approach is tested on course-keeping simulations of the 5415M destroyer in beam-quartering irregular waves (sea state 7 at Fr = 0.33) using data from three CFD/potential-flow solvers (CFDShip-Iowa, TEMPEST, ShipMo3D). The evaluation uses 250 prediction starting points and three error metrics (NRMSE, NAMMAE, JSD) over forecasting horizons up to five encounter periods. The paper reports that the Bayesian formulation improves prediction accuracy and provides uncertainty estimates that correlate with prediction error, while remaining fast enough for real-time use.","tokens_in":16186,"tokens_out":5522,"duration_ms":51981,"significance":"The digital-twin motivation and the real-time execution time are valuable. The method is data-lean, adaptive, and computationally cheap (about 0.05 s per deterministic prediction), and the evaluation across three independent solvers with a 250-start statistical protocol is a strength. If the claimed improvements and the uncertainty-error correlation were shown to be robust, the method would be a genuinely useful nowcasting component for naval digital twins. However, the paper's central claim of consistent improvement across all metrics is contradicted by its own summary tables, and the selection of the prior ranges on the same test data undermines the independence of the comparison. The work is therefore significant as a proposal, but it requires a re-analysis and more careful, defensible claims before it can be accepted as reported.","major_comments":[{"comment":"The conclusion states that 'the Bayesian Hankel-DMD significantly improves the accuracy of the predictions compared to the deterministic method considering all the metrics.' This is contradicted by the paper's own Tables 3 and 4. For CFDShip-Iowa, the Bayesian JSD is worse than the best deterministic configuration at every forecast horizon: 0.0250 vs 0.0141 at lte=T, 0.0293 vs 0.0217 at lte=2T, and 0.0375 vs 0.0353 at lte=5T. The Bayesian NAMMAE is also worse at lte=T (0.2740 vs 0.2568). Only NRMSE improves in that block. The authors should either revise the conclusion to a more limited claim (for example, improvement in the majority of cases or improvement in NRMSE) or provide a concrete explanation for why these counter-examples do not contradict the 'all metrics' statement.","section":"Section 6 (Conclusion) vs Tables 3–4"},{"comment":"The prior ranges used by the Bayesian method are selected after the full-factorial deterministic analysis on the same dataset, as stated in Section 4 ('their ranges of variation are defined after a full-factorial numerical experiment in order to identify reasonable promising combinations') and Section 5.2 (the ranges are obtained by 'integrating the insights on the hyperparameters derived from the deterministic analysis'). In addition, the 'best' deterministic baseline in Tables 2–4 is selected using the same 250 test starting points (Section 5.1). This creates a selection-on-test-data loop: both the Bayesian ensemble and the deterministic baseline are tuned to the same test set on which they are then compared. A fair evaluation would determine the prior ranges (or at least the deterministic baseline) on a training split and evaluate both methods on a separate validation set. Without this, the reported improvement may not generalize to other sea states, speeds, or hulls.","section":"Sections 4 and 5.2"},{"comment":"The claim that 'The Bayesian algorithm consistently outperforms the deterministic one across all three metrics' is not only contradicted by Tables 2–4, but the comparison is also asymmetrical: the Bayesian result is an average over a range of hyperparameters, while the deterministic result is the single best configuration found on the same test data. This mixes model averaging with parameter selection. The comparison should either be re-framed as 'Bayesian model average vs. a selected deterministic configuration' or, preferably, extended to compare the Bayesian average against the full distribution of deterministic configurations, so that the claimed advantage is not an artifact of comparing an average to a point selected under favorable conditions.","section":"Section 5.2 and Figures 9–11"},{"comment":"The paper claims a 'direct correlation' between the predicted standard deviation and the prediction accuracy ('The standard deviation of the forecast appears directly correlated with the accuracy of the prediction'). This is supported only by a qualitative comparison of one successful and one unsuccessful example. Since the uncertainty-reliability link is stated as one of the four digital-twin requirements (point iv in Section 1), a quantitative assessment is needed—for example, the correlation between the Bayesian standard deviation and the actual prediction error across the 250 starting points, or a calibration analysis of the stated 88.89% Chebyshev interval. Without such evidence, the reliability-information claim is not established.","section":"Section 5.2 and Figures 7–8"}],"minor_comments":[{"comment":"The table header reads 'NAMMAAE', which appears to be a typo for 'NAMMAE'.","section":"Table 3"},{"comment":"The method is called 'Bayesian' although the hyperparameters are only assigned fixed uniform priors and no data-updated posterior is computed; a more precise description would be 'Bayesian model averaging with prior-only weights' or 'ensemble-averaged DMD'.","section":"Section 5.2"},{"comment":"The captions refer to 'random time series', but the text identifies the panels as examples of successful and unsuccessful predictions; the captions should match the actual selection criterion.","section":"Figures 7 and 8"},{"comment":"Some symbols in the metric definitions are not explicitly defined at first use (e.g., Q, R, M, and the domain χ); please add definitions for completeness.","section":"Equations (20)–(23)"},{"comment":"The timing test is reported only for the largest configuration (ltr/T = 5, ld/T = 5); reporting the timing range across all tested configurations would strengthen the real-time claim.","section":"Section 5.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for Ocean Engineering and the underlying method is interesting, but the discrepancy between the conclusion and Tables 3–4 is substantial. The central 'all metrics' improvement claim is not supported by the paper's own data, and the prior selection on the test data introduces a circularity that needs to be addressed. If the authors can re-run the evaluation with a proper train/test split and revise the claims (or clearly explain why the counter-examples are not contradictory), the paper could become acceptable. I recommend major revision rather than rejection because the core methodology is sound and the issues are fixable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nYou should know before reading: the method is a reasonable and clearly explained Bayesian extension of Hankel-DMD, but the central claim that it 'significantly improves the accuracy ... considering all the metrics' does not survive contact with the paper's own tables. For CFDShip-Iowa, the Bayesian mean is worse than the best deterministic configuration on JSD at every horizon (0.0250 vs 0.0141 at 1T; 0.0293 vs 0.0217 at 2T; 0.0375 vs 0.0353 at 5T) and worse on NAMMAE at 1T and 2T. ShipMo3D also shows a JSD regression at 5T. The universal-superiority claim is unsupported, regardless of generalization.\n\nWhat is new and worth taking seriously: treating the hyperparameters (training length and delay length) as uniform random variables and averaging the DMD predictions via Monte Carlo is a sensible way to get a point forecast plus uncertainty. It is a modest extension of the authors' earlier deterministic Hankel-DMD work, but it is cleanly formulated and the paper is transparent about the statistical protocol: 250 starting points, three CFD solvers, three complementary error metrics. The runtime measurement (~0.05s per prediction) is useful for the digital-twin framing.\n\nThe bigger problem is that the Bayesian prior ranges are chosen from the deterministic analysis on the same dataset (Sections 5.1–5.2, Eq. 24). That makes the reported improvement partly a fitting exercise, not an independent test. The authors are transparent about this, but it means the headline result could shrink on other ships, speeds, or sea states. The claimed correlation between uncertainty and error is also anecdotal, based on a few example time series, not quantified across the 250 runs.\n\nWho gets value from this: naval hydrodynamics people working on nowcasting and digital twins, and anyone applying DMD to time series who wants a simple Bayesian ensemble. The paper deserves a serious referee, but not acceptance as-is. The authors should report all metric–solver–horizon combinations, run an independent validation set (at least a different sea state or speed), quantify the uncertainty–accuracy relationship, and soften the conclusion. With those changes I would be happy to see it in print, and I would cite it for the method.\n\nRecommendation: send to peer review with a request for major revision. Engage with it; the flaws are fixable and the core idea is useful.","headline":"A useful Bayesian DMD ensemble undermined by a claim its own tables contradict and a tuning-on-test-data setup — worth major revision, not rejection.","tokens_in":16816,"tokens_out":4053,"would_cite":true,"duration_ms":39248,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62F15","62M20","62P30"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a Bayesian extension of Hankel dynamic mode decomposition can nowcast a destroyer's motions in sea state 7 up to five wave encounter periods with better accuracy than the deterministic version, while producing…","keywords":["Bayesian dynamic mode decomposition","Hankel-DMD","ship motion nowcasting","digital twin","uncertainty quantification","sea state 7","data-driven reduced order model","5415M destroyer"],"falsifier":"Take the exact trained setup (same code, same prior ranges, same Monte Carlo count) and apply it to a different ship speed, heading, or sea state, comparing the Bayesian mean against the deterministic best configuration on prediction windows of one, two, and five encounter periods. If the Bayesian mean loses to the deterministic best on more than a small share of solver/metric/window combinations, or if the standard deviation shows no relation to actual error, the paper's core claim is contradicted.","tokens_in":15691,"feed_emoji":"🚢","tokens_out":10721,"duration_ms":97334,"temperature":0.7,"pith_summary":"The paper seeks to establish that a Bayesian extension of Hankel dynamic mode decomposition (Hankel-DMD) can act as a real-time, data-lean nowcasting engine for ship motions inside a naval digital twin. The core idea is to treat the method's two tuning parameters—how much recent history to use and how many time-delayed copies to embed—as random variables, and to report the mean and spread of predictions across many hyperparameter draws. The authors show on simulated course-keeping data for a 5415M destroyer in beam-quartering sea state 7 waves that this Bayesian mean outperforms the best deterministic hyperparameter choice for most error metrics and prediction windows, with forecasts staying accurate up to five wave encounter periods. If the method works as claimed, ships could continuously update short-term motion forecasts at very low cost while also getting a signal for when those forecasts are unreliable.","feed_headline":"Bayesian DMD nowcasts ship motions five wave periods ahead","feed_subtitle":"Uncertainty-aware forecasts beat deterministic DMD on all three CFD solvers for a destroyer in sea state 7.","key_machinery":"The central object is the Bayesian Hankel-DMD predictor. Standard Hankel-DMD augments the measured state vector (heave, roll, pitch, yaw, rudder angle, surge and sway velocities) with $s$ time-delayed copies, builds the data matrices $\\hat{X}$ and $\\hat{X}'$, and extracts DMD modes and eigenvalues via full SVD, giving a modal expansion $x(t)=\\sum_{k} \\phi_k b_k \\exp(\\omega_k t)$ that predicts the future. The Bayesian extension treats the two hyperparameters—observation duration $l_{tr}$ and delay duration $l_d$—as independent uniform random variables over ranges chosen from a prior deterministic hyperparameter study, and uses 100 Monte Carlo samples to integrate the prediction into a normal distribution with mean $\\bar{x}(t)$ and standard deviation $\\sigma_x(t)$. The mean is the improved forecast; the standard deviation is the claimed reliability estimate, visualized through Chebyshev's inequality with a coverage factor of two.","core_discovery":"The paper's central claim is that Bayesian Hankel-DMD significantly improves the accuracy of the predictions compared to the deterministic method, and that the standard deviation of the Bayesian prediction is correlated with the actual forecast error. The implementation replaces a single choice of observation length $l_{tr}$ and delay length $l_d$ with uniform priors ($l_{tr}/\\hat{T}$ between 1 and 5, $l_d/l_{tr}$ between half and three-quarters), draws 100 Monte Carlo samples, and uses the sample mean as the forecast and the sample spread as the uncertainty. On the 5415M course-keeping test case with three CFD solvers, the Bayesian mean improves over the best deterministic hyperparameter configuration for NRMSE and NAMMAE across all solvers and windows, and improves JSD in most combinations, with the CFDShip-Iowa JSD values being slightly worse in the reported tables. The paper also reports a computational cost of a few hundredths of a second per deterministic prediction, with the Monte Carlo runs embarrassingly parallel.","pith_inferences":["This suggests a general recipe for other DMD-based reduced-order models: if a few hyperparameters dominate accuracy, Bayesian marginalization over them can improve accuracy and yield an uncertainty estimate even without a deeper probabilistic model.","The most direct untested extension is to close the loop with the uncertainty: use the predicted standard deviation as an online trigger to re-train the ROM or shorten the forecast horizon, something the paper motivates but does not implement.","A fair generalization test would fix the prior ranges on one condition (for example, one solver) and evaluate on another; the cross-solver consistency reported in the paper makes this feasible and is the natural next experiment.","If the uncertainty estimates remain informative for full-scale motions, they could be fed into model predictive control or fatigue monitoring as risk weights, converting a point forecast into a decision input; the paper only hints at this possibility."],"forward_implications":["A ship operator could use the Bayesian mean forecast instead of a single deterministic DMD run and get better average accuracy at essentially the same real-time cost, since the Monte Carlo draws are independent and parallelizable.","The correlation between predicted standard deviation and actual error gives a principled way to flag low-confidence forecasts: when $\\sigma_x(t)$ rises, the forecast should not be trusted for high-regret decisions.","Because the ROM is rebuilt from the near past whenever a new prediction is needed, the method adapts as the ship's state evolves, matching the 'as-is' updating requirement of digital twins without a large training database.","The improvement is largest for the shortest prediction window and shrinks as the horizon approaches five encounter periods, suggesting the method is best suited for nowcasting rather than long-horizon seakeeping prediction.","For NRMSE and NAMMAE, all three independent solvers agree on the improvement, so the result is unlikely to be an artifact of one CFD code, though the underlying dynamics all come from simulations rather than full-scale measurements."],"supporting_citations":[{"why":"Establishes DMD as a data-driven approximation of the Koopman operator and supplies the modal expansion used for forecasting.","marker":"[20]"},{"why":"Introduces time-delay observables as a linearizing basis, motivating the Hankel augmentation used here.","marker":"[34]"},{"why":"Provides the theory and applications of time-delay observables for Koopman models, the foundation of Hankel-DMD.","marker":"[35]"},{"why":"Gives the exact-DMD algorithm with SVD-based pseudo-inverse that the paper implements.","marker":"[39]"},{"why":"Prior statistical assessment of DMD forecasting for ships in waves; the hyperparameter boxplot analysis extends this methodology.","marker":"[32]"},{"why":"Compares DMD and machine-learning forecasting of ship motions and supplies the NRMSE and NAMMAE error metrics used for evaluation.","marker":"[19]"},{"why":"One of the three CFD solvers used to generate the test data; supplies the TEMPEST dataset.","marker":"[36]"},{"why":"One of the three CFD solvers used to generate the test data; supplies the ShipMo3D dataset.","marker":"[37]"},{"why":"One of the three CFD solvers used to generate the test data; supplies the CFDShip-Iowa dataset.","marker":"[38]"},{"why":"Source of the Jensen-Shannon divergence metric used to compare predicted and reference probability distributions.","marker":"[45]"}],"fun_headline_variants":["Bayesian DMD boosts ship motion forecast accuracy","Real-time ship motion forecasts with quantified uncertainty","Bayesian DMD predicts ship motions up to 5 periods ahead","Uncertainty-aware Bayesian DMD improves ship motion nowcasts"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The prior ranges for the two hyperparameters were chosen after running the deterministic algorithm on these same datasets and seeing which ranges worked best, so the Bayesian improvement is only proven for conditions where those prior ranges happen to be right.","fun_headline_variants_meta":{"raw":{"variants":["Bayesian DMD boosts ship motion forecast accuracy","Real-time ship motion forecasts with quantified uncertainty","Bayesian DMD predicts ship motions up to 5 periods ahead","Uncertainty-aware Bayesian DMD improves ship motion nowcasts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000654,"raw_usage":{"total_tokens":3006,"prompt_tokens":967,"completion_tokens":2039,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":583,"completion_tokens_details":{"reasoning_tokens":1983}},"tokens_in":583,"tokens_out":2039,"duration_ms":16688,"temperature":1.0,"reasoning_tokens":1983,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T14:48:16.688574+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the exact trained setup (same code, same prior ranges, same Monte Carlo count) and apply it to a different ship speed, heading, or sea state, comparing the Bayesian mean against the deterministic best configuration on prediction windows of one, two, and five encounter periods. If the Bayesian mean loses to the deterministic best on more than a small share of solver/metric/window combinations, or if the standard deviation shows no relation to actual error, the paper's core claim is contradicted.","supporting_citations":[{"cited_title":"SIAM, 2016","cited_arxiv_id":null,"evidence_quote":"Establishes DMD as a data-driven approximation of the Koopman operator and supplies the modal expansion used for forecasting."},{"cited_title":"Brunton, Bingni W","cited_arxiv_id":null,"evidence_quote":"Introduces time-delay observables as a linearizing basis, motivating the Hankel augmentation used here."},{"cited_title":"Time-delay observables for Koopman: Theory and applications","cited_arxiv_id":null,"evidence_quote":"Provides the theory and applications of time-delay observables for Koopman models, the foundation of Hankel-DMD."},{"cited_title":"On the use of dynamic mode decomposition for time-series forecasting of ships operating in waves","cited_arxiv_id":null,"evidence_quote":"Prior statistical assessment of DMD forecasting for ships in waves; the hyperparameter boxplot analysis extends this methodology."},{"cited_title":"Data-driven forecasting of ship motions in waves using machine learning and dynamic mode decomposition","cited_arxiv_id":null,"evidence_quote":"Compares DMD and machine-learning forecasting of ship motions and supplies the NRMSE and NAMMAE error metrics used for evaluation."},{"cited_title":"TEMPEST level-0 theory","cited_arxiv_id":null,"evidence_quote":"One of the three CFD solvers used to generate the test data; supplies the TEMPEST dataset."},{"cited_title":"McTaggart","cited_arxiv_id":null,"evidence_quote":"One of the three CFD solvers used to generate the test data; supplies the ShipMo3D dataset."},{"cited_title":"Carrica, and Frederick Stern","cited_arxiv_id":null,"evidence_quote":"One of the three CFD solvers used to generate the test data; supplies the CFDShip-Iowa dataset."},{"cited_title":"Marlantes, Piotr J","cited_arxiv_id":null,"evidence_quote":"Source of the Jensen-Shannon divergence metric used to compare predicted and reference probability distributions."}],"review_version":1}