{"id":"f0c8ac15-db67-4a27-9363-fa62f2b9dafc","arxiv_id":"2501.00889","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"On synthetic noisy periodic series, time-series foundation models match or beat FFT/AR baselines only for short, smooth, high-sample-rate signals, and degrade sharply when periods grow and sampling gets sparse.","lead":"This paper tests six time-series foundation models on synthetic periodic signals with noise, comparing them to a Fourier-based method and a linear autoregressive model. It finds the models keep up on short, smooth, densely sampled series but fall behind on long-period, noisy, sparsely sampled ones.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The load-bearing weakness is §3.1's φ_n=0, t=0-aligned generation: set B's long-period contexts never test arbitrary phase, so the AR baseline's large margin over TSFMs may be an artifact of a favorable starting phase.","rationale":"The paper's central generalization is that TSFM forecasting quality degrades with long periods, noise, low sampling, and shape complexity, and that AR/FFT beat TSFMs on set B. This claim is empirical and rests entirely on the synthetic generator. The generator has free choices—amplitude/frequency ranges, phases, noise model—and the one that is most load-bearing and least justified is φ_n=0. It is load-bearing because set B is defined by periods longer than the context; with zero phases and t=0 sampling, the model never sees a long-period series from an arbitrary phase, which is the typical observation setting. The AR baseline's per-series fit gives it an additional advantage, but the phase alignment is the prior issue: it shapes what the baseline has to fit. The reader's weakest_assumption already identified zero phases; I agree. The proposed regeneration is a single, inexpensive experiment that would either corroborate the paper's conclusion or force a scoped claim. Secondary issues—underspecified amplitude/frequency distributions and no released code—are real but secondary; they would be addressed by the same code/data release. I do not see a fatal internal inconsistency, so the reader's CONDITIONAL verdict remains appropriate.","tokens_in":5324,"tokens_out":10467,"duration_ms":106088,"concrete_test":"Regenerate set B with φ_n drawn i.i.d. from Uniform(0,2π) (equivalently, shift each series by a random start time t0), keeping N, amplitude/frequency distributions, SNR, sampling ratios, context, and horizon identical; recompute Table 1 and the bottom row of Figure 1. If the AR margin over the best TSFM collapses or reverses, the central claim must be scoped to zero-phase generation; if the margin persists within Monte Carlo error, the phase concern is refuted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.1 (Eq. 1) fixes all phases φ_n to zero and samples every series from t=0. In set B, the noise-free period is deliberately far longer than the 512-point context, so the context contains at most a small fraction of a cycle for the lowest-frequency components, always beginning at a global zero crossing. A real periodic process observed at an arbitrary time has uniformly random phases; the synthetic generator excludes that entire class. The AR baseline is fit per series to the context via AIC, so it can exploit the specific phase-aligned structure (e.g., near-linear onset of low-frequency components) in a way that the zero-shot TSFMs cannot. The headline set-B result—AR mean MSE 13.32 versus best TSFM TimesFM 33.32—is therefore not clearly a measure of TSFM failure on long periods; it may be a measure of sensitivity to the artificial zero-phase start. This is an internal design choice, not a disagreement with external consensus, and it directly controls the difficulty gradient the paper claims to establish.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports an empirical evaluation of six time series foundation models (TimesFM, Lag-Llama, CHRONOS-small/base, MOIRAI-small/base) in zero-shot long-horizon forecasting on two synthetic datasets of noisy periodic series, compared against an FFT-based reconstruction and a linear AR model fitted per series. Set A consists of harmonic series with bounded periods; set B allows arbitrary rational frequencies, yielding often extremely long periods. The headline results, based primarily on Table 1 and breakdown plots, are that TSFMs are competitive on set A—with CHRONOS approaching AR performance—but deteriorate on set B, where AR substantially outperforms all TSFMs and FFT ranks second. The paper concludes that TSFM zero-shot forecasting degrades with longer periods, higher noise, lower sampling rates, and more complex shapes.","tokens_in":5542,"tokens_out":2950,"duration_ms":30385,"significance":"The study is a useful controlled investigation of a practically important question: how TSFMs behave on noisy periodic signals outside standard benchmark suites. Its strengths include the use of multiple TSFMs, two statistical baselines, a synthetic generator that permits systematic variation of noise, sampling rate, and shape complexity, and breakdowns by these factors in the figures. If the findings are robust, they provide concrete evidence of TSFM limitations on long-period, low-sampling-rate periodic series and a clear benchmark for future improvements. The manuscript is honest about its scope as a limited empirical study, and the negative result for long periods is potentially valuable to the forecasting community. However, the strength of the conclusions currently exceeds what the experimental design can support, for the reasons detailed below.","major_comments":[{"comment":"The generator fixes all phases φ_n to zero and samples every series from t = 0. In set B, where the noise-free period is often far longer than the 512-point context, this means every context begins at a global zero crossing and the low-frequency components appear as near-linear ramps over the context. Real periodic observations occur at arbitrary phases, so this design excludes an entire class of realistic signals. Because the AR baseline is fit per series to the context (via AIC), it can exploit the phase-aligned, locally linear onset of long-period components, whereas zero-shot TSFMs cannot. The set B result—AR mean MSE 13.32 versus best TSFM 33.32 in Table 1—may therefore reflect sensitivity to the artificial zero-phase start rather than a general failure on long-period series. Please rerun the experiments with uniformly random phases (and ideally random start times) and report whether the AR margin persists.","section":"Section 3.2, Eq. (3)"},{"comment":"The comparison is not symmetric: the AR model selects its order p per series using AIC on the observed context, making it an in-context fitted model, whereas the TSFMs are evaluated zero-shot with no adaptation to the input series. The central claim that AR 'outperforms all TSFMs' on set B is therefore not a comparison of like with like. Since the paper emphasizes zero-shot performance as the TSFM setting, the AR baseline should either be constrained to a fixed order chosen independently of the test series, or the TSFMs should be given an analogous per-series adaptation mechanism (e.g., a lightweight fine-tuning or in-context learning variant) before the superiority claim is made. This affects the interpretation of Table 1 and the conclusions in Section 5.","section":"Section 4, Table 1"},{"comment":"Table 1 reports mean and median MSE over 5,040 series per set, but no confidence intervals, standard errors, or significance tests are provided. Given the large gap between mean and median (e.g., set A CHRONOS-base mean 11.62 vs median 2.18), the error distribution is heavily skewed, and verbal claims such as 'follow closely', 'marginally better', and 'significant superiority' in Section 4 and Figure 1 are not statistically supported. Please add error bars, confidence intervals, or pairwise test results for the key comparisons, especially for the set B AR-vs-TSFM margin and the set A CHRONOS-vs-AR gap.","section":"Section 3.2"},{"comment":"The FFT baseline uses a fixed threshold of 20% of the peak FFT value, but no sensitivity analysis is reported. Since the FFT is a central benchmark and is used as evidence that 'the FFT, too, performs slightly better than the TSFMs on set B', the arbitrariness of this threshold matters. Please report how the FFT results, and the resulting rankings, change as the threshold is varied (e.g., 5%, 10%, 20%, 50%). If the ranking is insensitive, state that; if not, the current threshold choice is a free parameter that needs justification.","section":"Section 3.1"}],"minor_comments":[{"comment":"The distributions for the amplitudes An and the frequencies fn are not specified beyond 'randomly generated'; this prevents exact replication. Please provide the sampling distributions for both sets A and B, or provide code.","section":"Section 3.1"},{"comment":"The statement that set A has period 1/f1 ≤ 1 'since f1 is chosen to be an integer' is confusing because the time unit is unspecified and f1 is later described as 'principal frequencies f1 randomly chosen'. Please clarify the units and the exact range of f1.","section":"Section 3.1"},{"comment":"The sentence 'These challenges are further exacerbated with higher N, lower sampling rates, and lower SNR' is a qualitative summary; please point to the specific panels in Figure 1 that demonstrate this, or add quantitative effect sizes.","section":"Section 4"},{"comment":"The text says 'a median time-period of 10^9 s, compared to 0.1 s in set A', but no time unit is defined for the synthetic process and the values seem to depend on arbitrary frequency choices. Please report periods in samples or in normalized frequency units, or define the time axis explicitly.","section":"Section 4"},{"comment":"There is a grammatical error in the Introduction: 'Unlike traditional statistical foundation models lack well-defined assumptions...' should be 'Unlike traditional statistical models, foundation models lack well-defined assumptions...' or similar.","section":"Section 1"},{"comment":"The caption states 'best performing model in bold, best performing TSFM underlined', but the table as rendered does not show bold or underlining; please ensure the formatting is visible or use a separate column to mark these values.","section":"Table 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for an empirical ML venue and the central question is interesting, but the zero-phase generation and the per-series fitted AR baseline are load-bearing design choices that may dominate the main set-B result. I would like to see the authors address those with additional experiments, along with uncertainty quantification. The scope is appropriate for a workshop or conference preprint, but as a journal contribution it currently needs substantive revision rather than acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper does something straightforward and worth doing: it takes six time-series foundation models (TSFM), runs them zero-shot on two synthetic families of noisy periodic series, and compares them against a zero-shot FFT method and a per-series AR baseline. The main finding is credible in direction: on set A (bounded periods), the better TSFMs come close to AR and sometimes beat it at high sampling rates; on set B (arbitrarily large periods), AR clearly wins and the TSFMs degrade further with more components, lower SNR, and lower sampling. That is a clean, useful result for practitioners who need a boundary on when to trust TSFMs on periodic data.\n\nWhat is genuinely new is the specific comparison: the cited model papers do not report performance on controlled, noisy periodic series with long periods, and the paper isolates period length, noise, sampling rate, and shape complexity as separate axes. The experimental design is simple but the decomposition is helpful. The FFT baseline is a sensible zero-shot reference, and the AR baseline is a strong lower bound, though the asymmetry between a per-series fitted AR and zero-shot TSFMs is a real caveat: AR gets to adapt to each series; the TSFMs do not.\n\nThe soft spots are proportionate to the claims. The biggest one is the phase. Section 3.1 fixes all phases to zero and starts every series at t=0. In set B, where periods can be far longer than the 512-step context, the context then contains only a tiny fraction of a cycle, always beginning at a zero crossing. A real observation of a periodic process would start at an arbitrary phase. The per-series AR baseline can exploit the near-linear onset of those low-frequency components; the TSFMs, trained on varied real data, have no reason to. So the large AR margin on set B may be less about TSFMs failing on long periods and more about the artificial phase alignment. That does not destroy the paper, but it does mean the headline claim is overstated without a phase-randomized control.\n\nOther issues are minor by comparison. There are no confidence intervals on the MSE table, the FFT threshold is a fixed 20% of peak without justification, and code/data are not released. All of these are fixable. The paper is honest about its limitations and does not overclaim outside its synthetic scope.\n\nWho is this for? Researchers and engineers doing TSFM evaluation, especially anyone thinking about when zero-shot forecasting breaks down. It deserves a serious referee: the topic is timely, the design is reproducible in principle, and the phase issue is a well-defined fix rather than a conceptual dead end. I would send it to review, with a request to address the phase confound, add uncertainty bounds, and ideally release the generator.","headline":"A useful, narrowly scoped empirical study of TSFMs on noisy periodic series, but the zero-phase synthetic design undermines the headline claim about long-period degradation.","tokens_in":5987,"tokens_out":986,"would_cite":false,"duration_ms":11009,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Time-series foundation models lose to AR and FFT once periods grow longer than the context window.","keywords":["time series foundation models","zero-shot forecasting","noisy periodic time series","synthetic benchmark","Fourier transform baseline","autoregressive model","long-horizon forecasting","model evaluation"],"falsifier":"Re-run the same protocol with random nonzero phases and time-varying amplitudes, or with real recordings such as ECG, electricity load, or weather whose dominant periods exceed the 512-step context; if the linear AR model no longer beats the TSFMs on those series, the paper's central claim about long-period deterioration would be refuted.","tokens_in":5141,"feed_emoji":"📈","tokens_out":11012,"duration_ms":91084,"temperature":0.7,"pith_summary":"Time-series foundation models (TSFMs) promise zero-shot forecasting, and this paper tests that promise on noisy periodic data. Across 10,080 synthetic series, it compares five model families against two statistical baselines: an FFT-based spectral reconstruction and a linear autoregressive (AR) model. The central finding is a performance cliff: with bounded periods and dense sampling, the best TSFM (CHRONOS-base) nearly matches AR, but when periods exceed the 512-step context window, every TSFM trails both baselines, with AR ahead by a wide margin. The paper concludes that longer periods, higher noise, lower sampling rates, and more complex shapes each erode TSFM accuracy.","feed_headline":"AR and FFT beat time-series foundation models on long periods","feed_subtitle":"10,080 test series: foundation models lose their edge once the period exceeds the context window.","key_machinery":"The central machinery is a synthetic-data protocol plus two classical baselines. Each series is generated as $X(t)=\\sum_{n=1}^N A_n\\sin(2\\pi f_n t+\\phi_n)+w(t)$, where $w(t)$ is zero-mean Gaussian noise; set A fixes $f_n=nf_1$, giving a bounded period, while set B lets the $f_n$ be arbitrary positive rationals, so periods can exceed the 512-observation context. The FFT baseline applies a Hann window, clips Fourier coefficients below 20% of the peak, and reconstructs the series over context plus horizon; the AR baseline selects its lag order $p$ by the Akaike Information Criterion. The decisive mechanism is the mismatch between context length and period: set B series often do not complete one cycle inside the context, so pattern-based TSFMs cannot lock onto the period, whereas AR exploits short-lag linear dependence and FFT exploits global frequency content.","core_discovery":"The paper reports a controlled zero-shot, long-horizon comparison on 10,080 synthetic noisy periodic series. On set A, where the signal is a sum of harmonics and the period is at most one time unit, CHRONOS-base reaches a mean MSE of 11.62, close to the linear AR model's 10.83 and well ahead of the FFT baseline's 23.54. On set B, where frequencies are arbitrary rationals and the median period is around $10^9$ seconds, the pattern reverses: AR scores 13.32, FFT 32.42, and the best TSFM (TimesFM) 33.32. The paper's claim is that TSFM forecasting quality degrades systematically with longer periods, higher noise, lower sampling rates, and more complex shapes, and that this degradation is steeper than for classical statistical methods.","pith_inferences":["Editorial inference: the zero-phase, fixed-amplitude generator is a narrow slice of periodic data; real signals with phase drift, amplitude modulation, or nonstationary spectra could reorder the models, so the reported difficulty gradients are a lower bound on stress rather than a universal law.","Editorial inference: the FFT baseline's 20%-of-peak clipping threshold is a fixed free parameter, so the TSFM-vs-FFT gap on set B is conservative with respect to the spectral method; tuning the threshold per series would likely strengthen FFT.","Editorial inference: because AR with AIC selection is simple and needs no training, the practical takeaway extends beyond this benchmark: classical methods remain a strong default for long-period noisy series, and TSFM claims should be routinely benchmarked against them.","Editorial inference: extending the test to longer context windows or multivariate periodic series would separate a fundamental TSFM limitation from a context-length mismatch."],"forward_implications":["On series with known bounded periods and dense sampling, the best tested TSFMs are a viable zero-shot choice, with CHRONOS-base nearly matching the linear AR baseline on set A.","When the period can exceed the context window, the tested TSFMs should not be the default; AR or FFT-style spectral reconstruction gives lower error on this benchmark.","Improving sampling rate or smoothing the series (fewer sinusoidal components) should help TSFMs more than the statistical baselines, since TSFM degradation is concentrated in low-sampling, high-complexity regimes.","Model evaluation reports should include period relative to context length, noise level, sampling ratio, and shape complexity, because average benchmark error hides these failure modes.","Fine-tuning TSFMs on long-period data such as set B is a plausible path to closing the gap, as the paper explicitly suggests."],"supporting_citations":[{"why":"Survey used to select the state-of-the-art transformer-based self-supervised TSFMs that form the test set.","marker":"[3]"},{"why":"Supplies Lag-Llama, one of the TSFMs evaluated zero-shot.","marker":"[6]"},{"why":"Supplies the CHRONOS small and base models, the best-performing TSFM family in the study.","marker":"[8]"},{"why":"Supplies MOIRAI small and base models evaluated in the comparison.","marker":"[10]"},{"why":"Supplies TimesFM, the second-best TSFM family in set B and a key comparison point.","marker":"[11]"},{"why":"Provides the Hann window applied to the context before FFT reconstruction to reduce spectral leakage.","marker":"[12, 13]"},{"why":"Provides the FFT algorithm used to build the spectral reconstruction baseline.","marker":"[14]"},{"why":"Provides the Akaike Information Criterion used to select the order of the linear AR baseline.","marker":"[15]"}],"fun_headline_variants":["Foundation models lose edge as forecast horizon grows","Statistical baselines outdo TSFMs on long periodic series","Period length flips forecasting winner: AR/FFT win","TSFMs stumble on longer, noisier periodic signals","Context window limits TSFM forecast on long periods"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that synthetic series made from fixed-phase, fixed-amplitude sinusoids plus Gaussian noise represent the real-world periodic data the conclusions are meant to cover; phase is locked at zero and the amplitude/frequency sampling distributions are not specified, so the difficulty gradients may not transfer to real series with drift or nonstationary spectra.","fun_headline_variants_meta":{"raw":{"variants":["Foundation models lose edge as forecast horizon grows","Statistical baselines outdo TSFMs on long periodic series","Period length flips forecasting winner: AR/FFT win","TSFMs stumble on longer, noisier periodic signals","Context window limits TSFM forecast on long periods"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000181,"raw_usage":{"total_tokens":1263,"prompt_tokens":855,"completion_tokens":408,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":471,"completion_tokens_details":{"reasoning_tokens":328}},"tokens_in":471,"tokens_out":408,"duration_ms":3816,"temperature":1.0,"reasoning_tokens":328,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:39:27.995596+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same protocol with random nonzero phases and time-varying amplitudes, or with real recordings such as ECG, electricity load, or weather whose dominant periods exceed the 512-step context; if the linear AR model no longer beats the TSFMs on those series, the paper's central claim about long-period deterioration would be refuted.","supporting_citations":[{"cited_title":"Foundation models for time series analysis: A tutorial and survey,","cited_arxiv_id":null,"evidence_quote":"Survey used to select the state-of-the-art transformer-based self-supervised TSFMs that form the test set."},{"cited_title":"Information theory and an extension of the maximum likelihood principle,","cited_arxiv_id":null,"evidence_quote":"Provides the Akaike Information Criterion used to select the order of the linear AR baseline."}],"review_version":1}