{"id":"3f6684b1-97a4-455d-be2f-f8cc04c5a155","arxiv_id":"2603.24466","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.5,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"FloTS, a normalizing-flow time-series model, best balances point accuracy and calibrated uncertainty for two-hour atmospheric seeing forecasts trained only on historical seeing.","lead":"Machine-learning models forecast atmospheric seeing up to two hours ahead from past seeing alone, with a normalizing-flow model (FloTS) giving the best accuracy-plus-uncertainty trade-off. The forecasts can help schedule telescope time and adapt free-space optical links when turbulence changes quickly.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.5","headline":"Temporal leakage from random split of overlapping windows is the load-bearing threat to the FloTS ranking claim.","rationale":"The reader correctly isolated the single assumption on which the strongest claim most critically depends. All other methodological choices (kernel form, flow architecture, calibration temperature, 2 h context length) are secondary once the evaluation set is contaminated by near-duplicates of the training set. Because the manuscript supplies neither a leakage-free split nor public code that would let a third party enforce one, the Conditionally-accepted status remains appropriate; the concrete temporal-block experiment is the minimal decisive check.","tokens_in":20619,"tokens_out":443,"duration_ms":5099,"concrete_test":"Re-partition the same 71 906 pairs by contiguous time blocks (or by calendar year) so that no training window overlaps any test window by more than one sample; retrain FloTS, LSTM, RNN and GP under identical hyperparameters and recompute the lead-time RMSE/correlation curves of Fig. 5 plus the calibrated PP-plots. If FloTS’s two-hour RMSE rises by more than ~0.03″ or its ranking versus LSTM reverses, the headline claim weakens.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim (FloTS best accuracy–uncertainty trade-off) rests on metrics computed after randomly shuffling 71 906 overlapping sliding-window pairs into 81.5/8.5/10 % train/val/test (§2.2). With a single-timestep stride, consecutive pairs share 11 of 12 input and 11 of 12 target steps; a test pair can therefore have nearly identical history and future already seen (or nearly seen) during training. Under that regime the reported RMSE ~0.20″, correlation curves, and PP-plot calibration for FloTS (and the LSTM/GP baselines) can be inflated by partial memorization rather than genuine two-hour forecasting skill. The paper never reports a chronological or blocked temporal split that would eliminate this leakage, so the ranking that underpins the abstract and §5 conclusion is not yet secured.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The manuscript develops short-term (up to 2 h) forecasts of atmospheric seeing from Maunakea DIMM data (2009–2024), resampled to 10 min. It compares deterministic RNN and LSTM encoder–decoder models with two probabilistic approaches: a Gaussian process with an exponential kernel and a novel masked-autoregressive normalizing-flow model (FloTS) conditioned by an LSTM context encoder. All models are trained only on historical seeing, with a 2 h input window selected by RMSE. Probabilistic forecasts are post-calibrated via temperature scaling of coverage (PP plots). On a held-out set the authors report RMSE ~0.20″ and competitive Pearson r for LSTM and FloTS at 2 h, with FloTS providing the best accuracy–uncertainty trade-off after calibration; two qualitative case studies illustrate the predictive densities.","tokens_in":20909,"tokens_out":1026,"duration_ms":11197,"significance":"If the ranking is robust under a leakage-free evaluation protocol, the work would be a useful contribution to operational astronomy and free-space optical communications: a practical, seeing-only probabilistic nowcast with explicit non-Gaussian uncertainty, calibrated coverage, and a clear comparison of statistical versus deep generative models. The introduction of FloTS for this domain, the systematic input-length study, and the transparent calibration analysis are genuine strengths. The result is incremental rather than transformative, but it is well motivated and of clear applied interest.","major_comments":[{"comment":"§2.2: After constructing 71 906 sliding-window pairs with a single-timestep stride (so consecutive pairs share 11/12 input and 11/12 target steps), the pairs are randomly shuffled into 81.5/8.5/10 % train/val/test. This protocol admits substantial temporal leakage: a test window can have nearly identical history and future already present (or nearly present) in training. Absolute RMSE, correlation, and PP-plot calibration for FloTS (and the LSTM/GP baselines) may therefore be inflated by partial memorization rather than genuine two-hour skill. The abstract and §5 claim that FloTS achieves the best accuracy–uncertainty balance rests on these metrics. A chronological or blocked temporal split (or at least a sensitivity check with non-overlapping windows / multi-day gaps) is required before the ranking can be considered secured.","section":null},{"comment":"§3.3 and §4: GP hyperparameters are fit on random batches of only 500 pairs and then averaged; the same overlapping-window construction is used for evaluation. Combined with the Gaussian assumption that the authors themselves show is imperfect (Fig. 4 and Appendix A, where even piecewise-linear temperature scaling fails to reach the ideal PP line), the GP baseline is under-powered relative to FloTS. A fairer comparison would use a larger inducing-point or sparse GP (or a multi-scale kernel) under the same leakage-free split; otherwise the claimed superiority of FloTS over GP is only partially supported.","section":null}],"minor_comments":[{"comment":"Section numbering: the introduction states that Section 4 presents both the statistical evaluation and the case studies, then ‘Finally, Section 5 summarizes’; the body has a free-standing CASE STUDY block after §4. Renumber for clarity.","section":null},{"comment":"Table 1 is referenced for RNN/LSTM hyperparameters but is empty/placeholder in the draft; supply the actual numbers (hidden units, dropout, lr, etc.).","section":null},{"comment":"Fig. 1 caption and text use both ‘FloTS’ and ‘FLOW’; standardize nomenclature.","section":null},{"comment":"Eqs. (1)–(3) and (7)–(12) mix encoder/decoder notation (h_t vs g_t, c_t vs d_t); a short clarifying sentence would help readers follow the autoregressive decoder.","section":null},{"comment":"No code or data-release statement is given; for a methods paper in astro-ph.IM a public repository (even if only the trained models and split indices) would strengthen reproducibility.","section":null},{"comment":"Minor typos: ‘Short-T erm’, ‘T urbulence’, ‘Universit´ e’, and duplicated Cherubini et al. 2021 entries in the reference list.","section":null}],"recommendation":"major_revision","confidential_remarks":"The temporal-leakage issue is the single load-bearing threat; if the authors re-run with a proper blocked split and the FloTS ranking survives, the paper is close to acceptable. Novelty of FloTS itself is modest (MAF + LSTM context is standard), but the application and calibration analysis are solid. Fit for an instrumentation/methods journal is good."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The one thing worth knowing is that this is a careful, usable comparison of short-term seeing forecasts on the Maunakea DIMM archive, with a normalizing-flow model (FloTS) that actually delivers calibrated non-Gaussian uncertainty, not just another point-forecast bake-off. That is new relative to the Milli/Masciadri/Giordano/Ni line of work, which stayed mostly deterministic or Gaussian.\n\nWhat they do well: they keep the setup minimal (historical seeing only, 10-min resampling, fixed 2 h horizon), report RMSE and Pearson r versus lead time for RNN, LSTM, GP, and FloTS, show PP-plots before and after a temperature-style calibration, and include two case-study nights with density heatmaps. The calibration discussion is honest; they even try a piecewise-linear alternative for the GP and admit the Gaussian assumption still does not fully match the data. LSTM wins pure point accuracy; FloTS is competitive and gives the richer predictive density. That trade-off claim is the paper’s real contribution.\n\nThe soft spot is real but not fatal. They generate ~72 k sliding-window pairs with single-timestep stride, then randomly shuffle into 81.5/8.5/10 % splits. Adjacent pairs share almost everything, so temporal leakage can inflate absolute RMSE (~0.20″ at 2 h) and make the ranking look cleaner than a pure chronological or blocked split would. The paper never reports the stricter split. GP hyperparameters are also fit on only 500 pairs. These are methodological gaps, not circularity or invented physics; the ranking could still hold, but we do not yet know. No code release compounds the issue.\n\nCitations look appropriate; math is standard MAF + LSTM context encoder and textbook GP. Free parameters are the usual architecture and kernel knobs, all disclosed.\n\nThis is for people who schedule telescopes or FSO links and want probabilistic short-term seeing tools, and for the site-characterization community that already reads the earlier ML-seeing papers. It deserves a serious referee who will demand a temporal-split ablation or public code. I would engage with it, cite the FloTS comparison once the leakage is checked, and send it to peer review rather than desk-reject.","headline":"Solid applied ML comparison for two-hour seeing forecasts; FloTS is a useful probabilistic addition, but the random split of overlapping windows is a real soft spot that leaves absolute numbers and the ranking claim provisional.","tokens_in":21481,"tokens_out":558,"would_cite":true,"duration_ms":7728,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"A normalizing-flow model forecasts atmospheric seeing two hours ahead with the best balance of accuracy and calibrated uncertainty.","keywords":["atmospheric seeing","optical turbulence","short-term forecasting","normalizing flows","Gaussian processes","LSTM","probabilistic machine learning","adaptive optics"],"falsifier":"Re-split the same Maunakea DIMM series into strictly chronological blocks with no temporal overlap between train and test, retrain all four models, and check whether FloTS still shows the lowest RMSE and the best-calibrated coverage at the two-hour horizon.","tokens_in":21546,"feed_emoji":"🔭","tokens_out":518,"duration_ms":5779,"temperature":0.7,"pith_summary":"Ground-based telescopes and free-space optical links are limited by optical turbulence that blurs light as it passes through the atmosphere. Adaptive optics can correct distortions after they appear, but they cannot prepare for sudden changes; operators therefore need short-horizon forecasts of the seeing parameter. This paper trains four models on nothing but historical seeing time series and asks which one best predicts the next two hours while also reporting trustworthy uncertainty. The authors introduce FloTS, a masked autoregressive normalizing flow conditioned on recent measurements, and compare it with Gaussian processes, RNNs and LSTMs. FloTS matches or nearly matches the best point-forecast accuracy while producing flexible, non-Gaussian predictive distributions that, after a simple calibration step, better cover the true outcomes. The result supplies a practical, data-only tool for scheduling observations and adapting optical links when physical weather models are too slow.","feed_headline":"Flow model predicts telescope seeing two hours ahead","feed_subtitle":"FloTS matches LSTM accuracy while giving better-calibrated uncertainty from seeing data alone","key_machinery":"FloTS: a masked autoregressive normalizing flow whose context encoder (an LSTM) maps the past seeing window into the parameters of a base Gaussian, which is then transformed into a flexible conditional distribution over the future two-hour window.","core_discovery":"When only historical seeing measurements are available, a normalizing-flow model for time series (FloTS) yields the best overall balance of point accuracy and well-calibrated predictive uncertainty for two-hour-ahead forecasts, outperforming Gaussian processes and matching the deterministic accuracy of LSTMs while supplying richer non-Gaussian uncertainty estimates.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["FloTS balances accuracy and uncertainty in 2-hour seeing forecasts","Normalizing flow predicts telescope seeing two hours ahead","FloTS matches LSTM accuracy with calibrated uncertainty","Flow model forecasts optical turbulence two hours out","Seeing data alone: FloTS outperforms GPs for short-term forecasts"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"Randomly shuffling overlapping two-hour windows into train, validation and test sets does not leak future information across the split and therefore does not inflate the reported forecast skill.","fun_headline_variants_meta":{"raw":{"variants":["FloTS balances accuracy and uncertainty in 2-hour seeing forecasts","Normalizing flow predicts telescope seeing two hours ahead","FloTS matches LSTM accuracy with calibrated uncertainty","Flow model forecasts optical turbulence two hours out","Seeing data alone: FloTS outperforms GPs for short-term forecasts"]},"model":"grok-4.5","effort":"low","cost_usd":0.00521,"raw_usage":{"total_tokens":1429,"prompt_tokens":745,"num_sources_used":0,"completion_tokens":63,"cost_in_usd_ticks":52100000,"prompt_tokens_details":{"text_tokens":745,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":621,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":745,"tokens_out":63,"duration_ms":5325,"temperature":1.0,"reasoning_tokens":621,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-13T18:49:24.625340+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Re-split the same Maunakea DIMM series into strictly chronological blocks with no temporal overlap between train and test, retrain all four models, and check whether FloTS still shows the lowest RMSE and the best-calibrated coverage at the two-hour horizon.","supporting_citations":[],"review_version":1}