{"id":"f90d9d02-516f-4b7d-a2ee-22762b740310","arxiv_id":"2501.17781","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A realtime causal bandpass filter plus echo-state network reports 24-month ENSO prediction skill, but the skill is measured on a filtered proxy index whose filter is tuned on the full data record.","lead":"This paper combines a new causal band-pass filter with reservoir computing to claim El Nino forecasts up to two years ahead using only past observations. The authors show skillful predictions of a filtered version of the Nino-3.4 index, but the evaluation has several methodological caveats.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Claimed 24-month ENSO skill is evaluated on a filtered proxy whose parameters were tuned on the full 1870–2022 record, including the 2001–2015 test window; refitting on pre-2001 data and scoring the standard Niño-3.4 index would settle whether the result survives.","rationale":"The reader's weakest assumption identifies exactly the load-bearing weakness: prediction skill is computed on a filtered proxy rather than the standard Niño-3.4 index, and the filter parameters are optimized over the full 1870–2022 record including the 2001–2015 evaluation period. Both points are supported by the manuscript. Section IV.A explicitly states that 'the target of prediction in this study is the filtered time series' and that 'a direct comparison of the prediction horizon to those of previous studies is not possible.' That admission alone undercuts the abstract's claim of predicting 'the multi-year dynamics of the El Niño-Southern Oscillation' on the standard index. The leakage concern is also concrete: the causal filter kernel uses only past values when applied, but its parameters were selected by Bayesian optimization over the whole record, so information from 2001–2015 could have shaped the filter and therefore the reported skill curve. The reservoir hyperparameters were selected on 1986–1995, which is cleaner, but the filter optimization is not described as being restricted to pre-2001 data. A clean out-of-sample test would refit all filter parameters using data through 2000 and score both the filtered series and the standard Niño-3.4 index. The paper does provide code and a reproducible protocol, which is a genuine strength, but the central claim as currently evidenced is not safe. The reader's REJECT verdict is therefore appropriate; no verdict change is needed from this stress-test pass.","tokens_in":11721,"tokens_out":3748,"duration_ms":39045,"concrete_test":"Refit the realtime filter (Table I parameters) using only data from 1870 through 2000, holding the rest of the protocol fixed; then repeat the 180 forecasts over 2001–2015 and compute C(µ). Additionally score the same forecasts against the conventional Niño-3.4 index (with its standard 5-month running mean), not just against y*. If C(24) on the standard index, or C(24) on y* with the pre-2001 filter, falls at or below 0.5, the two-year ENSO prediction claim is not supported out-of-sample.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that ENSO dynamics can be predicted 24 months ahead using only past data. Two linked conditions must hold: (i) the predicted quantity must be (or faithfully track) the conventional ENSO state, and (ii) all fitted quantities must use only information available before forecast initialization. The paper concedes in Section IV.A that the target is the filtered realtime SST anomaly, not the standard Niño-3.4 index, so direct comparison with prior ENSO prediction studies is explicitly not possible. The proxy justification is a maximum lag-correlation of 0.837 at a 5-month shift (Section III.A), but correlation does not imply the filtered series has the same forecast-relevant dynamics; a band-limited, phase-shifted version of the same series can yield higher correlation skill simply because high-frequency noise has been removed. More importantly, the filter parameters in Table I were found by maximizing Eq. (3) and the lag-correlation over the realtime SST anomaly without an explicit training/evaluation split; the description in Sections II.B and II.D indicates the full 1870–2022 record, which includes the 2001–2015 forecast evaluation window, was used. Even though the filter kernel itself is causal, its shape has therefore been optimized with information from the test period. This is a leakage path that can inflate the reported 29-month >0.5 skill and the derived 24-month claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes a causal 'realtime filter', a one-sided weighted moving average whose parameters are tuned by Bayesian optimization, to band-pass the monthly SST anomaly in the Nino-3.4 region; the filtered series is then used as input and output for an echo-state network with delay-coordinate embeddings. The authors report that the all-season correlation skill of the filtered target remains above 0.5 for 29 months over 180 hindcasts initialized monthly between January 2001 and December 2015, and after discounting a claimed ~5-month lag of the filter, they conclude that ENSO can be predicted 24 months ahead using only past data. The paper also provides code and a systematic hyperparameter optimization framework.","tokens_in":12089,"tokens_out":7337,"duration_ms":73438,"significance":"The paper offers a useful methodological contribution: a genuinely causal filter suitable for operational forecasting, combined with Bayesian optimization of filter and reservoir hyperparameters, and publicly available code. If the headline result were established on the standard Nino-3.4 index with a leakage-free calibration, a 24-month skill horizon would be noteworthy. As it stands, however, the evaluation is performed on a filtered proxy and the filter parameters are optimized on the full 1870-2022 record; both issues bear directly on the central claim and must be resolved before the result can be accepted.","major_comments":[{"comment":"The central claim of 24-month ENSO prediction is evaluated on the filtered realtime SST anomaly, not on the standard Nino-3.4 index or the raw monthly SST anomaly. Section IV.A concedes that 'a direct comparison of the prediction horizon to those of previous studies is not possible.' Because the filtered series is a smoothed, phase-shifted transformation of the input, correlation skill on this target can be high even when forecasts of the standard index are poor; the 0.837 maximum lag-correlation at five months does not establish that the two series have the same forecast-relevant dynamics. The abstract and conclusions should either report skill for the conventional Nino-3.4 index or substantially soften the claim that ENSO is predicted for two years.","section":"IV.A, III.B"},{"comment":"The filter parameters are selected by maximizing Eq. (3) together with the maximum lag-correlation 'when applied to the realtime SST anomaly', with no temporal split described; since the realtime SST anomaly spans 1870-2022, this optimization includes the 2001-2015 evaluation window. The filter kernel itself is causal, but its shape is nevertheless fit using information from the test period, which can inflate the reported 29-month above-0.5 skill. The authors should refit the filter on data prior to 2001 only and re-evaluate on 2001-2015.","section":"II.B, II.D, Table I"},{"comment":"The reservoir hyperparameters and the random seed x are chosen by maximizing C(24) over 120 sequences from January 1986 to December 1995, and the same metric C(mu) is then reported on the separate 2001-2015 period. While this is a legitimate out-of-sample split, the headline 'above 0.5 for 29 months' is a single post-selection realization; reporting only the selected model's curve without uncertainty bands or comparison to a persistence/climatology baseline makes it difficult to assess whether the apparent skill reflects genuine predictability of the filtered signal.","section":"II.D, III.B"}],"minor_comments":[{"comment":"The section heading 'Realtime fitering method' should be corrected to 'Realtime filtering method'.","section":"II.B"},{"comment":"The word 'ecch' in the caption should be 'each'.","section":"Fig. 5 caption"},{"comment":"The word 'architechture' should be 'architecture'.","section":"III.B"},{"comment":"The direction of the five-month shift is described inconsistently ('shifts backward' vs. 'shifted about 5 months into the future'); please clarify with an explicit definition of the lag variable and the implied correction to the prediction horizon.","section":"III.A, III.B"},{"comment":"Please specify the exact data period used for the filter optimization; if the full record is used, state it explicitly and discuss the implications for the reported skill.","section":"II.D"},{"comment":"Please define the range of lead months mu and explain how target years are assigned for lead times greater than 12 months.","section":"Eq. (7)"},{"comment":"The phrase 'Tempor al characteristics' should be 'Temporal characteristics'.","section":"Fig. 7 caption"}],"recommendation":"major_revision","confidential_remarks":"For the editor: the two major issues (proxy target and filter leakage) are fixable by re-analysis, and the causal-filter idea is worth publishing if the re-run supports it. My concern is scope: the current claims in the abstract and conclusions outrun the evidence, so I would require a substantial revision before further consideration."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The realtime filter is the genuinely new piece here, and it's a good idea. Avoiding future leakage in the preprocessing is a real step forward, and the dictionary-based objective for choosing the filter shape is not something I've seen before. The code is available, the writing is clear, and the authors are upfront that the target is not the conventional Niño-3.4 index. That honesty counts for something.\n\nBut the soft spots are load-bearing. First, skill is evaluated on the filtered realtime SST anomaly, not the standard index. The justification is a maximum lag-correlation of 0.837 at five months, but a smoothed, phase-shifted version of the same series will almost always produce higher correlation skill simply because high-frequency noise is gone. The paper itself concedes that direct comparison to prior work is impossible, so the claim of \"24-month ENSO prediction\" rests on the proxy being faithful, and the evidence for that is thin.\n\nSecond, and more seriously, the filter parameters in Table I are optimized over the entire 1870–2022 record, which includes the 2001–2015 evaluation period. The description in Section II.D says the filter is chosen to maximize the objective on \"the realtime SST anomaly\" without any training/test split. Even though the filter kernel is causal, its shape has seen the test data. That is a leakage path that can inflate the reported 29-month skill. Refitting the filter on data up to 2000 and re-running the evaluation would settle this quickly.\n\nThird, the headline skill comes from selecting the best random reservoir matrices among up to 1000 trials. The mean skill over 100 random matrices is much lower, which suggests the optimal is an outlier. That's not necessarily invalid—model selection on a validation period is legitimate—but it makes me worry about robustness. And there's no baseline comparison. No persistence, no climatology, no earlier ENSO forecast model scored on the same index. Without a baseline, a correlation of 0.5 at 24 months doesn't tell us much.\n\nThe reservoir hyperparameters are tuned on a separate 1986–1995 validation period, which is good, and the idea of using delay coordinates with the filter is sensible. The non-monotonic skill curve is odd but the authors offer a plausible explanation.\n\nSo: the method is worth a serious referee, but the current evidence does not support the headline claim. I would send it out, but I'd expect major revision—refit the filter on pre-2001 data, score the standard Niño-3.4 index, and show baselines.","headline":"The causal filter is a genuinely new idea, but the headline 24-month ENSO skill is measured on a filtered proxy whose parameters are tuned on the full record including the test window, so the central claim is not supported as presented.","tokens_in":12590,"tokens_out":4323,"would_cite":false,"duration_ms":41823,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A causal, data-driven band-pass filter combined with reservoir computing extends ENSO prediction to about two years.","keywords":["ENSO prediction","reservoir computing","echo state network","realtime band-pass filter","causal filtering","Bayesian optimization","sea surface temperature anomaly","multi-year climate forecasting"],"falsifier":"Evaluate the trained model directly against the conventional 5-month-running-mean Niño-3.4 index over 2001–2015: if the all-season correlation skill at 24 months falls below 0.5, the claimed two-year horizon does not transfer to the standard ENSO index. Alternatively, retrain the filter only on data before January 2001 and repeat the evaluation; a drop below 0.5 would indicate the test period influenced the reported skill.","tokens_in":11494,"feed_emoji":"🌊","tokens_out":6902,"duration_ms":60932,"temperature":0.7,"pith_summary":"This paper claims that the two-year-ahead prediction horizon for ENSO, previously reached by deep-learning models, can be obtained with a much simpler reservoir-computing setup once the input is filtered by a new type of band-pass filter that uses only past observations. The filter is a weighted moving average whose weights are tuned by Bayesian optimization, and its decisive property is causality: unlike conventional moving-average or Butterworth filtering, it never mixes future data into the training or prediction signal, so the whole workflow is usable in realtime operational forecasting. On 180 forecasts initialized monthly between January 2001 and December 2015, the average all-season correlation skill stays above 0.5 for 29 months, which the authors count as about 24 months after subtracting the roughly five-month lag the filter introduces. Because the predicted target is the filtered realtime sea-surface-temperature anomaly rather than the standard Niño-3.4 index, the authors are careful to say that a direct numerical comparison with earlier ENSO forecast studies is not possible.","feed_headline":"ENSO forecasts reach two years with a past-only filter","feed_subtitle":"A causal band-pass filter keeps future data out of the pipeline, making the two-year skill usable in realtime operations.","key_machinery":"The load-bearing object is the realtime filter, a causal weighted moving average with kernel $\\Psi(t) = (d_1 \\cos(t/(\\pi r_1)) + d_2 \\cos(t/(\\pi r_2))) (w-t)^c / w^c$ for $t \\in [0,w]$ and $\\Psi(t)=0$ for $t \\ge 0$; convolving it with the monthly SST anomaly suppresses fluctuations shorter than about three years and longer than about eight years while using no future values. Its parameters are selected to maximize a dictionary objective that counts how often discretized length-$L$ patterns (keys) are followed by a single value among a prescribed set, together with the lag correlation between original and filtered series. The filtered series is then fed, through an $M$-dimensional delay-coordinate vector with delay $\\Delta\\tau$, into an echo-state network whose recurrent and input matrices are kept fixed and whose output matrix is fitted by ridge regression; prediction proceeds by recursively feeding the output back as input. All filter and reservoir hyperparameters are calibrated by Bayesian optimization, which the paper argues is what lets the same workflow transfer to other phenomena.","core_discovery":"The central discovery claimed in the paper is that the main obstacle to long data-driven ENSO forecasts is not the predictor but the preprocessing: standard filters leak future information, and unfiltered data carry fast fluctuations that degrade reservoir training. The authors construct a causal realtime filter—a finite-support convolution kernel supported on $t \\in [0,w]$, so that the filtered value at time $t$ depends only on observations at $t, t-1, \\dots, t-w$—and tune its parameters together with the reservoir hyperparameters using Bayesian optimization. With the optimized filter and an echo-state network driven by delay-coordinate vectors of the filtered realtime SST anomaly, they report an average all-season correlation skill above 0.5 out to 29 months of lead time, and roughly 24 months once the filter-induced five-month lag is discounted. They interpret this as successfully predicting the multi-year dynamics of ENSO for two years using only past data.","pith_inferences":["Because the skill metric is computed on the filtered index, the paper does not establish the same two-year skill for the conventional Niño-3.4 index; a separate evaluation on the raw running-mean index would be needed before operational use.","The filter parameters were optimized over the full 1870–2022 record, which includes the 2001–2015 evaluation window, so part of the reported skill may reflect information from the test period; retraining the filter on data ending in 2000 would quantify this.","The causality property is generic, so the same filter-plus-reservoir workflow is a natural candidate for other oscillatory climate modes such as the Madden–Julian Oscillation or monsoon indices, where future-leaking filters have been a known obstacle."],"forward_implications":["The full pipeline—filter construction, model training, and forecasting—can in principle be run in realtime because no future information enters the filtered series.","On the filtered realtime SST anomaly, skillful forecasts (correlation above 0.5) extend to 29 months of lead time, or about 24 months after accounting for the filter's five-month backward shift.","Predictions initialized around 2010–2011 capture major warm and cold phases of the filtered index up to about three years ahead, while the strong 2015–2016 El Niño is reproduced with relatively low accuracy.","The same methodology, with its six filter parameters and nine reservoir parameters tuned by Bayesian optimization, is proposed as a general recipe for multi-year prediction of other high-dimensional climate time series."],"supporting_citations":[{"why":"Provides the monthly sea-surface temperature record (1870–2022) from which the realtime SST anomaly is computed.","marker":"[33]"},{"why":"Deep-learning baseline for multi-year ENSO forecasts that the authors compare their simpler reservoir-computing result against.","marker":"[17]"},{"why":"Echo-state-network ENSO prediction study whose skill metric and filtering rationale the paper builds on.","marker":"[32]"},{"why":"Supplies the delay-coordinate design rule for choosing delay time and dimension when the observable dimension is smaller than the attractor's.","marker":"[26]"},{"why":"Recent deep-learning ENSO forecast whose two-year horizon the paper matches with a simpler architecture on a filtered target.","marker":"[20]"},{"why":"Reservoir-computing analysis of ENSO limit-cycle signatures used as methodological precedent for applying echo-state networks to ENSO.","marker":"[16]"},{"why":"Bayesian optimization framework used to select filter and reservoir hyperparameters.","marker":"[41]"}],"fun_headline_variants":["Causal filter powers two-year ENSO forecasts","Realtime filter extends ENSO predictions to 24 months","Reservoir computing achieves 24-month ENSO lead time","Past-only filter enables two-year El Niño forecasts"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim's load-bearing premise is that the filtered realtime SST anomaly faithfully represents ENSO, a stand-in supported by a 0.837 lag correlation with the raw anomaly, while the filter parameters were tuned on a record that includes the test years.","fun_headline_variants_meta":{"raw":{"variants":["Causal filter powers two-year ENSO forecasts","Realtime filter extends ENSO predictions to 24 months","Reservoir computing achieves 24-month ENSO lead time","Past-only filter enables two-year El Niño forecasts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000421,"raw_usage":{"total_tokens":2126,"prompt_tokens":866,"completion_tokens":1260,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":482,"completion_tokens_details":{"reasoning_tokens":1196}},"tokens_in":482,"tokens_out":1260,"duration_ms":9896,"temperature":1.0,"reasoning_tokens":1196,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T04:32:12.442971+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Evaluate the trained model directly against the conventional 5-month-running-mean Niño-3.4 index over 2001–2015: if the all-season correlation skill at 24 months falls below 0.5, the claimed two-year horizon does not transfer to the standard ENSO index. Alternatively, retrain the filter only on data before January 2001 and repeat the evaluation; a drop below 0.5 would indicate the test period influenced the reported skill.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the monthly sea-surface temperature record (1870–2022) from which the realtime SST anomaly is computed."},{"cited_title":"\\ Ham , author J.-H","cited_arxiv_id":null,"evidence_quote":"Deep-learning baseline for multi-year ENSO forecasts that the authors compare their simpler reservoir-computing result against."},{"cited_title":"Hassanibesheli , author J","cited_arxiv_id":null,"evidence_quote":"Echo-state-network ENSO prediction study whose skill metric and filtering rationale the paper builds on."},{"cited_title":"Nakai \\ and\\ author Y","cited_arxiv_id":null,"evidence_quote":"Supplies the delay-coordinate design rule for choosing delay time and dimension when the observable dimension is smaller than the attractor's."},{"cited_title":"Wang , author S","cited_arxiv_id":null,"evidence_quote":"Recent deep-learning ENSO forecast whose two-year horizon the paper matches with a simpler architecture on a filtered target."},{"cited_title":"Guardamagna , author C","cited_arxiv_id":null,"evidence_quote":"Reservoir-computing analysis of ENSO limit-cycle signatures used as methodological precedent for applying echo-state networks to ENSO."},{"cited_title":"Akiba , author S","cited_arxiv_id":null,"evidence_quote":"Bayesian optimization framework used to select filter and reservoir hyperparameters."}],"review_version":1}