{"id":"7cb66386-e317-489d-806a-f4d3289db668","arxiv_id":"2504.14677","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"Across ten temporal partitions of two datasets, incrementally fine-tuned Time-MoE and Chronos mostly beat their zero-shot baselines and show less plasticity loss than DLinear, PatchTST, and iTransformer, though improvement is not progressive.","lead":"This paper asks whether large time series foundation models can keep improving when they are periodically fine-tuned on new data, and compares them with smaller deep learning models on two real-world datasets. It reports that foundation models adapt better and lose less plasticity, but the evidence is thin, partly contradicts the headline claim, and lacks error bars or code.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central claim is undercut by the paper's own Rzero analysis: no consistent decrease in Rzero_p, so 'sustained improvement' is not established and the metrics do not directly measure temporal plasticity.","rationale":"The reader's CONDITIONAL verdict is reasonable, but the weakest assumption should not be contamination. Even with perfectly clean datasets, the central claim fails because the paper's own Rzero trend analysis explicitly states there is no consistent improvement over p. This is an internal inconsistency with the abstract rather than an external risk. The paper does include an honest caveat, which is to its credit, but the caveat is incompatible with the headline. I recommend rejecting the paper in its current form unless the claims are rewritten to match the evidence: incremental fine-tuning yields lower MSE than zero-shot on current partitions and approaches full-training performance for foundation models, but no progressive plasticity advantage is established. I agree partially with the reader because they noted the tension in the rationale but selected dataset contamination as the weakest assumption.","tokens_in":11613,"tokens_out":5766,"duration_ms":55145,"concrete_test":"Recompute the Rzero_p curves for Time-MoE and Chronos on both datasets (release checkpoints or use Figure 5 values). For each model, fit a linear regression of Rzero_p on p and compute the Spearman rank correlation plus the difference Rzero_0 vs Rzero_8. If the trend is not significantly negative, revise the abstract to 'improvement over zero-shot without progressive gains.' Separately, after each incremental update, evaluate the model on a held-out earlier partition and report the change in MSE; without this, the 'mitigating catastrophic forgetting' conclusion is unmeasured.","verdict_should_be":"REJECT","load_bearing_attack":"The load-bearing weakness is internal consistency, not external contamination. The abstract claims foundation models 'demonstrate sustained improvement in predictive accuracy,' and the conclusion claims they 'effectively mitigate plasticity loss and catastrophic forgetting during incremental fine-tuning.' For the first claim to hold, Rzero_p = MSE_inc_p/MSE_zero_p (Eq. 12) should trend downward with p. Section V-B (Metric Rzero_p) states the opposite: 'we do not observe a consistent decrease in Rzero_p with increasing p, indicating that our raw incremental learning approach does not progressively enhance foundation models' domain adaptation capabilities.' The figures show flat curves with a spike at p=4 on Flight. Rzero_p<1 only says a model fine-tuned on partition p beats its own zero-shot checkpoint on that partition; it does not show later checkpoints outperform earlier ones and does not measure retention of earlier partitions. Rfull_p (Eq. 13) similarly compares against full training on all data, so it partly reflects the pretraining advantage rather than a measured plasticity trajectory. Thus the paper's own data support a weaker claim—fine-tuning on a new partition can beat zero-shot—but not the headline. The contamination concern, if real, would only shrink the gains, so this internal mismatch is more decisive.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript studies whether time series foundation models can be incrementally fine-tuned on successively collected data without losing plasticity or suffering catastrophic forgetting. The authors propose a continual-learning pipeline with three evaluation ratios—Rzero_p (incremental fine-tuning vs. zero-shot), Rfull_p (incremental fine-tuning vs. full training), and Rfz_p (full training vs. zero-shot)—and apply it to Time-MoE, Chronos, and three smaller models on two real-world datasets, Flight and CD-Bike, with distribution shifts. The paper claims that foundation models demonstrate sustained improvement in predictive accuracy and effectively mitigate plasticity loss and catastrophic forgetting, while smaller models degrade. The contribution is framed as the first comprehensive evaluation of temporal plasticity for time series foundation models.","tokens_in":11857,"tokens_out":4615,"duration_ms":39278,"significance":"If the claims were fully supported, this would be a useful early result for practitioners deciding whether to keep foundation time series models current by periodic fine-tuning rather than full retraining. The paper contributes a concrete evaluation protocol, including algorithms and ratio-based metrics, and targets real distribution shifts, which is a relevant and understudied setting. It also ships the comparison across several public foundation-model checkpoints and smaller baselines, which is valuable for reproducibility. However, the headline conclusion is not supported by the paper's own reported Rzero_p analysis, and the proposed metrics do not directly measure forgetting or backward transfer. The evaluation also compares models under different prediction horizons and lacks statistical uncertainty. The study is therefore of moderate significance pending major revision.","major_comments":[{"comment":"The abstract and conclusion claim that foundation models demonstrate 'sustained improvement in predictive accuracy' and 'effectively mitigate plasticity loss and catastrophic forgetting,' but Section V-B explicitly states: 'we do not observe a consistent decrease in Rzero_p with increasing p, indicating that our raw incremental learning approach does not progressively enhance foundation models' domain adaptation capabilities.' Rzero_p < 1 only shows that a checkpoint fine-tuned on partition p beats that same model's zero-shot checkpoint on partition p; it does not establish that later checkpoints improve over earlier ones or that adaptation improves over time. The central claim must either be supported by direct evidence of progressive improvement (e.g., a declining trend in MSE_inc_p over p, or comparisons of checkpoints across partitions) or be weakened to the claim that incremental fine-tuning can beat zero-shot on each new partition.","section":"Abstract, §V-B, Eq. (12)"},{"comment":"Catastrophic forgetting is defined in Eq. (6) as performance deterioration on old distributions after training on new data, but no experiment measures performance on earlier partitions using later checkpoints. The metrics Rzero_p, Rfull_p, and Rfz_p all compare checkpoints against zero-shot or full-training references on the currently considered partition; they cannot detect whether knowledge from partitions q < p has been forgotten. To support the forgetting-related claims, the authors should add a backward-transfer evaluation, for example by evaluating the model after step p on test sets from all earlier partitions and reporting those errors.","section":"§III, Eq. (6); §V-B, Figs. 5–7"},{"comment":"The derivation of MSE as 1 + σ² + μ² assumes that the prediction Ŷ and the target Y are mutually independent, which is not true for a forecasting model whose predictions are functions of inputs correlated with the target. Even under a simple optimal conditional-mean predictor, E[ŶY] = E[Ŷ²], so the cross term does not vanish and the variance enters with the opposite sign relative to the stated formula. This derivation is not used directly in the ratio definitions, but it is presented as part of the proposed evaluation methodology and should be corrected or removed because, as written, it is mathematically incorrect.","section":"§IV, Eqs. (8)–(10)"},{"comment":"The setup states that all models use a prediction length of 96, 'except for Chronos, which has a maximum prediction length of 64 due to its architectural constraints.' This means Chronos is evaluated on a shorter forecast horizon than all other models, and MSE is generally lower for shorter horizons. Consequently, the cross-model comparisons involving Chronos in Figures 3–7 are not on equal footing and can be misleading. Please either evaluate all models at the same horizon (e.g., 64) or explain why the different horizons do not affect the conclusions.","section":"§V, Setups; Figs. 3–7"},{"comment":"The paper asserts that the Flight and CD-Bike datasets were 'carefully selected' and 'verified' not to be part of foundation models' pretraining corpora, but no verification procedure is described and both datasets come from the authors' own prior publications ([30] and [31]). Since the zero-shot MSE values are the baselines for Rzero_p and Rfz_p, dataset contamination would directly inflate the reported gains. The authors should provide a concrete and auditable contamination check, such as matching against public pretraining corpus lists, documenting dataset release dates relative to model training, or reporting zero-shot error patterns that would be inconsistent with memorization.","section":"§V, Datasets"}],"minor_comments":[{"comment":"There is a typo in the reference to Chronos: 'Chnoros' should be 'Chronos'.","section":"§II-A"},{"comment":"The section heading 'Loss of Plasiticity and Catastrophic Forgetting in Deep Learning' contains a typo: 'Plasiticity' should be 'Plasticity'.","section":"§II-B"},{"comment":"Algorithm 1 is described as an incremental fine-tuning algorithm but does not explicitly show the loop over partitions p = 0, 1, ..., P−1; the pseudocode should clarify that it is called once per partition with the previously fine-tuned model as input.","section":"§IV, Algorithm 1"},{"comment":"All figures use 'Data Subset Index' on the x-axis; the text calls these partitions p. Use one consistent term throughout.","section":"§V, Figures"},{"comment":"No code or configuration files are provided, which makes it difficult to reproduce the exact fine-tuning protocol, dataset splits, and model checkpoints. Releasing the evaluation code would strengthen the paper.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper's central claim is undercut by its own Rzero_p analysis, and the proposed metrics do not directly measure the forgetting or plasticity the title promises. The good news is that these are fixable by reframing the claims and adding a backward-transfer experiment. I would also ask for a formal response on the prediction-horizon mismatch for Chronos, since that affects all cross-model comparisons. The dataset-contamination concern is amplified by both datasets coming from the authors' own prior work; requesting an external audit or a documented contamination check would be appropriate. Overall, the topic is timely and the protocol is a useful starting point, but the current evidence does not support the headline conclusions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This paper is a genuine first attempt to ask whether time-series foundation models (Time-MoE, Chronos) can be kept current by incremental fine-tuning on new data without losing plasticity. The authors run a chronological partition experiment on two real datasets, compare incremental fine-tuning against full retraining and zero-shot, and find that the foundation models degrade less than small models like DLinear and PatchTST. That pattern is worth knowing about, and the Rzero/Rfull/Rfz ratios are a useful, simple way to frame the comparison.\n\nThe main problem is that the abstract overclaims. It says the models \"demonstrate sustained improvement in predictive accuracy,\" but in Section V-B the authors themselves report \"we do not observe a consistent decrease in Rzero_p with increasing p, indicating that our raw incremental learning approach does not progressively enhance foundation models' domain adaptation capabilities.\" Those two statements contradict each other. The flat or spiky curves in Figure 5 support the weaker claim: fine-tuning usually beats zero-shot on the same partition (Rzero<1), but it doesn't show that later checkpoints are better than earlier ones, nor does it measure retention on old partitions. So the \"sustained improvement\" claim fails, while the more modest claim about reduced plasticity loss may still hold.\n\nOther soft spots. There are no confidence intervals or multiple seeds, so we can't tell how stable the MSE differences are. Chronos is evaluated with prediction length 64 while everything else uses 96, which is a confound for comparing absolute performance. The authors say they \"verified\" the datasets are not in the pretraining corpora, but no procedure is given; if one of these leaked, the zero-shot baseline improves and the gains shrink. None of these are fatal; they're fixable.\n\nOn the positive side, the question is timely, the experimental design is transparent, and the authors make their measured statement in the body even if the abstract overreaches. The citation pattern looks fine; using their own prior datasets is a minor self-reference, not a flaw by itself.\n\nMy take: this deserves a serious referee, but it needs major revision before publication. The claims must match the evidence, seeds and error bars should be added, and the contamination check needs to be documented. As is, I'd treat it as a promising preliminary study rather than a definitive result.\n\nRecommendation: send to peer review, but with a strong expectation of substantial revision.","headline":"First look at temporal plasticity in time-series foundation models, but the headline claim of sustained improvement is undercut by the authors' own Rzero analysis; promising direction that needs major revision.","tokens_in":12393,"tokens_out":2259,"would_cite":true,"duration_ms":20301,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Foundation time series models keep improving as new data arrives.","keywords":["temporal plasticity","time series foundation models","continual learning","incremental fine-tuning","catastrophic forgetting","loss of plasticity","Time-MoE","Chronos"],"falsifier":"Search the pretraining corpora of Time-MoE and Chronos for the Flight and CD-Bike series, or rerun the incremental fine-tuning protocol on a dataset constructed after those models were trained; if the zero-shot baselines improve because the series appeared in pretraining, the reported $R^{\\mathrm{zero}}_p$ gains would shrink or invert.","tokens_in":11374,"feed_emoji":"📈","tokens_out":6459,"duration_ms":51373,"temperature":0.7,"pith_summary":"This paper tries to establish that large pre-trained time series foundation models can be updated continuously on newly collected data without losing the ability to learn or forgetting what they already know. The authors call this capacity temporal plasticity and study it with an incremental fine-tuning pipeline that splits each dataset into ten chronological subsets. On two real-world datasets, the foundation models Time-MoE and Chronos show sustained predictive improvement under this regime, while smaller models such as DLinear, PatchTST, and iTransformer deteriorate. If this is right, practitioners could keep foundation models current by periodically fine-tuning on the newest observations instead of retraining from scratch, and the field's effort is better spent on fine-tuning strategies than on bespoke small models.","feed_headline":"Foundation time series models keep improving as new data arrives","feed_subtitle":"Unlike small models, Time-MoE and Chronos avoid catastrophic forgetting and plasticity loss when fine-tuned on new data.","key_machinery":"The load-bearing mechanism is a continual-learning evaluation pipeline that splits each dataset chronologically into $P = 10$ subsets and compares three training regimes: zero-shot, incremental fine-tuning on each new subset, and full retraining on all data up to that point. Performance is captured by three ratios — $R^{\\mathrm{zero}}_p$, $R^{\\mathrm{full}}_p$, and $R^{\\mathrm{fz}}_p$ — which isolate whether incremental fine-tuning beats the zero-shot model, approaches the fully retrained model, and whether full training improves over zero-shot, respectively. These ratios, rather than any single accuracy number, are what the paper uses to diagnose plasticity loss, catastrophic forgetting, and temporal adaptability.","core_discovery":"The central claim is that foundation models for time series forecasting exhibit temporal plasticity: when fine-tuned sequentially on new temporal subsets of data, Time-MoE and Chronos continue to improve predictive accuracy relative to their zero-shot baselines, whereas conventional deep learning models suffer from plasticity loss and catastrophic forgetting. The paper evaluates this with ratio metrics $R^{\\mathrm{zero}}_p = \\mathrm{MSE}^{\\mathrm{inc}}_p / \\mathrm{MSE}^{\\mathrm{zero}}_p$ and $R^{\\mathrm{full}}_p = \\mathrm{MSE}^{\\mathrm{inc}}_p / \\mathrm{MSE}^{\\mathrm{full}}_p$ across $P = 10$ partitions, and reports that foundation models maintain $R^{\\mathrm{zero}}_p < 1$ on most subsets and stay close to full-training performance, while small models show large deteriorations, especially after abrupt distribution shifts. A further observation is that the mini, base, and large variants of the foundation models behave similarly, suggesting that beyond a certain scale the amount of pre-training data matters more than parameter count.","pith_inferences":["A testable extension is to run the same ratio-based pipeline on a dataset created after the models' pretraining cutoff; if the $R^{\\mathrm{zero}}_p < 1$ pattern persists, the mitigation claim is not a contamination artifact.","If pre-training data scale is what drives plasticity, then a smaller architecture trained on the same massive corpus should exhibit the same temporal plasticity as Time-MoE; that scaling comparison would isolate the cause.","The three-ratio framework could be reused to benchmark continual-learning fixes such as replay, regularization, or parameter isolation against the raw fine-tuning baseline, since the paper only studies the raw protocol."],"forward_implications":["Periodic fine-tuning on the newest data can keep a foundation forecaster current without full retraining, since incremental fine-tuning stays close to full-training performance.","The main research bottleneck shifts from designing domain-specific small models to designing fine-tuning and adaptation strategies for foundation models.","Abrupt distribution shifts still cause temporary performance spikes, so raw incremental fine-tuning is not yet a complete solution and robust adaptation under sudden shifts remains an open target.","Model scale beyond a threshold matters less than pre-training data scale, so future foundation models should emphasize diverse training corpora."],"supporting_citations":[{"why":"Supplies the Time-MoE foundation model whose pretrained parameters are used in the incremental and full training experiments.","marker":"[12]"},{"why":"Supplies the Chronos foundation model and its tokenization-based architecture, the second family of foundation models tested.","marker":"[14]"},{"why":"Establishes the loss-of-plasticity phenomenon that motivates the study and provides the framing the paper tests against foundation models.","marker":"[18]"},{"why":"Supplies the Flight dataset, one of the two real-world evaluation sets, with major distribution shifts including a COVID-19-related break.","marker":"[30]"},{"why":"Supplies the CD-Bike dataset, the proprietary bike-sharing evaluation set with comparatively minor distribution shifts.","marker":"[31]"},{"why":"Defines the DLinear baseline, one of the small models the paper contrasts with foundation models.","marker":"[32]"},{"why":"Defines the PatchTST baseline, another small model used in the comparison.","marker":"[33]"},{"why":"Defines the iTransformer baseline, the third small model in the comparison.","marker":"[9]"}],"fun_headline_variants":["Foundation models keep learning, small models forget","Time-MoE and Chronos improve with new data, unlike small models","Incremental fine-tuning boosts foundation time series models","Foundation models show plasticity, small models don't","Time series foundation models thrive on continual learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the two evaluation datasets were not part of the foundation models' pretraining corpora, so the zero-shot baselines used to compute the improvement ratios are uncontaminated.","fun_headline_variants_meta":{"raw":{"variants":["Foundation models keep learning, small models forget","Time-MoE and Chronos improve with new data, unlike small models","Incremental fine-tuning boosts foundation time series models","Foundation models show plasticity, small models don't","Time series foundation models thrive on continual learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000631,"raw_usage":{"total_tokens":2890,"prompt_tokens":893,"completion_tokens":1997,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":509,"completion_tokens_details":{"reasoning_tokens":1921}},"tokens_in":509,"tokens_out":1997,"duration_ms":12004,"temperature":1.0,"reasoning_tokens":1921,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:43:02.888538+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Search the pretraining corpora of Time-MoE and Chronos for the Flight and CD-Bike series, or rerun the incremental fine-tuning protocol on a dataset constructed after those models were trained; if the zero-shot baselines improve because the series appeared in pretraining, the reported $R^{\\mathrm{zero}}_p$ gains would shrink or invert.","supporting_citations":[{"cited_title":"Msgnet: Learning multi- scale inter-series correlations for multivariate time series forecasting,","cited_arxiv_id":null,"evidence_quote":"Supplies the Flight dataset, one of the two real-world evaluation sets, with major distribution shifts including a COVID-19-related break."},{"cited_title":"Rethinking urban mobility prediction: A multivariate time series forecasting approach,","cited_arxiv_id":null,"evidence_quote":"Supplies the CD-Bike dataset, the proprietary bike-sharing evaluation set with comparatively minor distribution shifts."},{"cited_title":"A time series is worth 64 words: Long-term forecasting with transformers,","cited_arxiv_id":null,"evidence_quote":"Defines the PatchTST baseline, another small model used in the comparison."},{"cited_title":"itrans- former: Inverted transformers are effective for time series forecasting,","cited_arxiv_id":null,"evidence_quote":"Defines the iTransformer baseline, the third small model in the comparison."}],"review_version":1}