pith. sign in

arxiv: 2605.06032 · v1 · submitted 2026-05-07 · 💻 cs.LG · cs.AI

Does Synthetic Data Help? Empirical Evidence from Deep Learning Time Series Forecasters

Hugo Cazaux , Eyj\'olfur Ingi \'Asgeirsson , Hlynur Stef\'ansson This is my paper

Pith reviewed 2026-05-08 14:12 UTC · model grok-4.3

classification 💻 cs.LG cs.AI
keywords synthetic datatime series forecastingdata augmentationdeep learningarchitecture comparisonchannel mixinglow-resource learning
0
0 comments X

The pith

Synthetic data augmentation improves time series forecasts only for channel-mixing architectures and in selected low-resource cases.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper asks whether synthetic data can reliably improve deep learning models that forecast time series, an approach that has succeeded in language modeling but remains untested at scale here. It answers by running more than four thousand controlled experiments that vary model architecture, dataset, synthetic generator, and training schedule. The results show a sharp split: models that mix information across channels gain from the extra data in most settings, especially when real observations are scarce, while models that keep channels separate lose performance. Only one generator type produces consistent gains, and the way synthetic data is introduced during training determines whether it helps or harms. These patterns translate into concrete rules for practitioners deciding when augmentation is worth the effort.

Core claim

Through more than four thousand runs the study establishes that synthetic time series augmentation yields architecture-dependent results. Channel-mixing models such as TimesNet and iTransformer improve in the majority of trials and can exceed full-data baselines when real data is reduced to ten percent on certain datasets. Channel-independent models such as DLinear and PatchTST are degraded in every tested configuration. Among the four generators examined only the Seasonal-Trend variant helps reliably across benchmarks, while hard curriculum switching raises mean squared error by twenty-four percent. Averaged over all architectures and settings, augmentation increases error in sixty-seven of

What carries the argument

The distinction between channel-mixing and channel-independent architectures as the decisive factor controlling whether synthetic time series signals raise or lower forecast accuracy.

If this is right

  • Channel-mixing architectures should be paired with synthetic augmentation to obtain performance gains in most settings.
  • Channel-independent architectures should avoid synthetic augmentation because it consistently raises forecast error.
  • Only the Seasonal-Trend generator can be used with confidence across the tested benchmarks.
  • Gradual annealing schedules must replace hard curriculum switching to prevent large error increases.
  • Low-resource regimes offer the largest potential payoff, where augmentation can let suitable models surpass full-data baselines.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Model selection and augmentation strategy should be chosen together rather than sequentially.
  • The conditional benefit pattern may extend to other sequential prediction tasks outside the seven datasets examined.
  • Practitioners facing new data regimes should run small validation trials before committing to augmentation.
  • Future generators could be designed to exploit the mixing bias that appears to drive the observed gains.

Load-bearing premise

The seven datasets, five architectures, four synthetic generators, and nine experiment groups are representative enough of real-world time series forecasting to support general guidelines on synthetic data use.

What would settle it

A new experiment on an additional dataset or architecture in which channel-independent models show net gains from augmentation or in which the Seasonal-Trend generator fails to help would contradict the reported architecture-conditional pattern.

Figures

Figures reproduced from arXiv: 2605.06032 by Eyj\'olfur Ingi \'Asgeirsson, Hlynur Stef\'ansson, Hugo Cazaux.

Figure 1
Figure 1. Figure 1: MSE improvement from synthetic augmentation (Group 3, view at source ↗
Figure 2
Figure 2. Figure 2: Low-resource augmentation (Group 4). Dashed line: real-only training at each sparsity level view at source ↗
Figure 3
Figure 3. Figure 3: Ablation results (Groups 5–8, s = 1, r = 1, averaged over all architectures). All y-axes show MSE improvement vs. the full-data baseline: lower (more negative) values indicate greater degradation. (a) Every bundle type degrades MSE on average; ST causes the least harm. (b) Curriculum strategies vs. static mixed at r = 1: gradual annealing performs the same as static mixing; the hard switch (A+H) is catastr… view at source ↗
Figure 4
Figure 4. Figure 4: One representative channel drawn from each bundle type at medium difficulty ( view at source ↗
read the original abstract

Synthetic data has transformed language model training, yet its role in time series forecasting remains poorly understood. We present a large-scale empirical study: nine experiment groups, 4,218 runs systematically evaluating synthetic time series augmentation across five architectures, four synthetic signals and seven datasets. The effect is sharply architecture-conditional: channel-mixing models (TimesNet, iTransformer) benefit in the majority of trials, while channel-independent models (DLinear, PatchTST) are consistently degraded. In selected low-resource settings the gains are striking: TimesNet trained on only 10\% of Weather data with synthetic augmentation surpasses the full-data baseline (4 of 16 sparsity-dataset combinations). Averaged across all architectures, augmentation hurts in 67\% of trials. We further find that only the Seasonal-Trend generator reliably helps across the tested benchmarks, and that hard curriculum switching is actively harmful (+24\% MSE degradation). These results provide concrete, actionable guidelines on how to use synthetic data: use synthetic augmentation with channel-mixing architectures, use gradual annealing schedules, and treat low-resource augmentation as architecture- and dataset-dependent. Code is available at \href{https://github.com/hugoiscracked/synthetic-ts/tree/main}

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper reports on a comprehensive empirical investigation into the utility of synthetic data for augmenting training of deep learning models for time series forecasting. It evaluates this across 4,218 runs involving five different architectures, four synthetic generators, seven real-world datasets, and varying levels of data sparsity. The key observations are that benefits are conditional on the model architecture, with channel-mixing models like TimesNet and iTransformer showing improvements in most cases, whereas channel-independent models like DLinear and PatchTST experience consistent degradation. Additional findings include notable performance gains in low-data regimes for specific combinations and the identification of the Seasonal-Trend generator as particularly effective, along with the negative impact of hard curriculum learning schedules.

Significance. This work has substantial significance for the field of time series forecasting by offering empirical evidence and practical guidelines on synthetic data augmentation. The large number of experiments (4,218 runs) and the public release of the code are notable strengths that support reproducibility and allow for further analysis. If the architecture-conditional effects are confirmed, it could influence how practitioners approach data augmentation in resource-constrained settings, potentially leading to more efficient model training strategies.

major comments (2)
  1. [§4 (Results)] §4 (Results): The aggregate claim that augmentation hurts in 67% of trials is presented without per-condition variance, confidence intervals, or statistical tests comparing channel-mixing vs. channel-independent groups; this weakens the robustness of the architecture-conditional conclusion given the cross-dataset variability.
  2. [§3 (Experimental Protocol)] §3 (Experimental Protocol): The description of data splits, sparsity implementation (e.g., random subsampling vs. contiguous), and the exact mixing ratio of synthetic to real samples is high-level; explicit details or pseudocode are needed to rule out confounds in the low-resource gains (e.g., TimesNet on 10% Weather data surpassing full baseline).
minor comments (2)
  1. [Abstract] Abstract: The mention of 'nine experiment groups' is not enumerated; adding a short list would improve immediate clarity for readers.
  2. [Discussion] The paper would benefit from a dedicated limitations subsection discussing the representativeness of the seven datasets and five architectures for broader real-world time series tasks.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback and the recommendation for minor revision. We address each major comment below and have updated the manuscript accordingly to improve clarity and robustness.

read point-by-point responses

  1. Referee: [§4 (Results)] The aggregate claim that augmentation hurts in 67% of trials is presented without per-condition variance, confidence intervals, or statistical tests comparing channel-mixing vs. channel-independent groups; this weakens the robustness of the architecture-conditional conclusion given the cross-dataset variability.

    Authors: We agree that additional statistical support would strengthen the architecture-conditional claims. The 67% aggregate is computed across all 4,218 runs, but in the revision we now include per-architecture and per-dataset means with standard deviations, 95% confidence intervals on the key differences, and a paired t-test confirming that channel-mixing models improve significantly more than channel-independent models (p < 0.01). These additions directly address cross-dataset variability while preserving the original aggregate observation. revision: yes

  2. Referee: [§3 (Experimental Protocol)] The description of data splits, sparsity implementation (e.g., random subsampling vs. contiguous), and the exact mixing ratio of synthetic to real samples is high-level; explicit details or pseudocode are needed to rule out confounds in the low-resource gains (e.g., TimesNet on 10% Weather data surpassing full baseline).

    Authors: We accept that the protocol description was insufficiently precise. Section 3 has been expanded to specify chronological 70/15/15 train/val/test splits, random (non-contiguous) subsampling for sparsity levels, and a default 1:1 synthetic-to-real mixing ratio (with explicit ratios listed per experiment group). Pseudocode for the full augmentation pipeline is now provided in Appendix A. The reported low-resource gains (including the TimesNet 10% Weather case) are averaged over five random seeds; we have added a note confirming that the same subsampling procedure is applied uniformly across all compared runs, ruling out the most obvious confounds. revision: yes

Circularity Check

0 steps flagged

No significant circularity

full rationale

This is a purely empirical study reporting direct experimental outcomes from 4,218 controlled runs across five architectures, four synthetic generators, seven datasets, and multiple sparsity levels. All central claims (architecture-conditional benefits, specific low-resource gains for TimesNet, 67% aggregate degradation, and generator reliability) are presented as measured MSE differences from the experiments, with no derivations, equations, predictions, or fitted parameters that reduce to inputs by construction. The protocol is self-contained, code is public, and findings rest on external benchmarks rather than self-referential definitions or self-citation chains.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claims rest entirely on the reported experimental outcomes from the 4,218 runs; no new theoretical entities, fitted constants, or unstated assumptions beyond standard experimental design choices are introduced.

axioms (1)
  • domain assumption The seven chosen datasets and five architectures sufficiently represent broader time series forecasting tasks for drawing general guidelines.
    General recommendations are extrapolated from these specific benchmarks without additional cross-validation on external domains.

pith-pipeline@v0.9.0 · 5526 in / 1353 out tokens · 74096 ms · 2026-05-08T14:12:39.740240+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

298 extracted references · 298 canonical work pages · 4 internal anchors

  1. [1]

    Killer Robots with lasers , Year =

    work page

  2. [2]

    CDIO Annual International Conference , pages = 12, owner =

    Joseph Timothy Foley , year = 2021, address =. CDIO Annual International Conference , pages = 12, owner =

    work page 2021

  3. [3]

    The Gnats and Gnus Document Preparation System , Author =

    work page

  4. [5]

    2011 , author =

    Introduction to Mechatronic Design , publisher =. 2011 , author =

    work page 2011

  5. [6]

    Cochran and Joseph T

    David S. Cochran and Joseph T. Foley and Zhuming Bi , Year =

    work page

  6. [7]

    Sacha Goedegebure and Andy Goralczyk and Enrico Valenza and Nathan Vegdahl and William Reynish and Brecht Van Lommel and Campbell Barton and Jan Morgenstern and Ton Roosendaal , title =

    work page

  7. [8]

    Kristján Gerhard and Joseph Timothy Foley , title =

    work page

  8. [9]

    Jevgenij Guls and \'

    work page

  9. [10]

    Storrs Hall , title =

    J. Storrs Hall , title =

    work page

  10. [12]

    Kreher and D.R

    D.L. Kreher and D.R. Stinson , title =. 2005 , url =

    work page 2005

  11. [13]

    The Great Moon Landing Hoax , author =

    work page

  12. [14]

    Freyja Yeatman Ómarsdóttir and Róbert Bjarnar Ólafsson and Joseph Timothy Foley , title =

    work page

  13. [15]

    2010 , publisher =

    Richard Ramirez , title =. 2010 , publisher =

    work page 2010

  14. [16]

    2015 , pages =

    Gunnar Óli Sölvason and Joseph Timothy Foley , title =. 2015 , pages =. doi:10.1016/j.procir.2015.07.061 , owner =

    work page doi:10.1016/j.procir.2015.07.061 2015

  15. [17]

    2015 , pages =

    Nam Pyo Suh , title =. 2015 , pages =

    work page 2015

  16. [18]

    Reliability, IEEE Transactions on , year =

    Nam Pyo Suh , title =. Reliability, IEEE Transactions on , year =

    work page

  17. [19]

    Research in engineering design , year =

    Nam Pyo Suh , title =. Research in engineering design , year =

    work page

  18. [20]

    Cochran and P

    Nam Pyo Suh and David S. Cochran and P. C. Lima , title =. 48th. 1998 , volume =

    work page 1998

  19. [21]

    2013 , pages =

    Mary Katheryn Thompson , title =. 2013 , pages =

    work page 2013

  20. [22]

    2013 , owner =

    Didier Verna , title =. 2013 , owner =

    work page 2013

  21. [23]

    Jon Warbrick , title =

    work page

  22. [24]

    2014 , owner =

    Joseph Wright , title =. 2014 , owner =

    work page 2014

  23. [25]

    Reykjavik University Website , key =

    work page

  24. [26]

    Semigroups of Recurrences , Author =

    work page

  25. [27]

    Oaho and Jeffrey D

    Alfred V. Oaho and Jeffrey D. Ullman and Mihalis Yannakakis , Note =. On Notions of Information Transfer in

    work page

  26. [28]

    Seminumerical Algorithms , Year =

    work page

  27. [29]

    Fundamental Algorithms , Chapter =

    work page

  28. [30]

    Lower Bounds for Wishful Research Results , Author =

    work page

  29. [31]

    1985 , Address =

    Structure and Interpretation of Computer Programs , Author =. 1985 , Address =

    work page 1985

  30. [32]

    1986 , Comment =

    Mach: A New Kernel Foundation for UNIX Development , Author =. 1986 , Comment =

    work page 1986

  31. [33]

    1984 , Owner =

    So long, and thanks for all the fish , Author =. 1984 , Owner =

    work page 1984

  32. [34]

    1985 , Address =

    Compilers: Principles, Techniques, and Tools , Author =. 1985 , Address =

    work page 1985

  33. [35]

    Annual Reviews of Computer Science , Publisher =

    Dataflow Architectures , Author =. Annual Reviews of Computer Science , Publisher =. 1986 , Address =

    work page 1986

  34. [36]

    1988 , Address =

    The Price of Asynchronous Parallelism: An Analysis of Dataflow Architectures , Author =. 1988 , Address =

    work page 1988

  35. [37]

    IEEE Transactions on Computers , Year =

    Design of A Massively Parallel Processor , Author =. IEEE Transactions on Computers , Year =

    work page

  36. [38]

    , Journal =

    Birnbaum, J.S and Worsley, W.S. , Journal =. Beyond. 1985 , Month = aug, Number =

    work page 1985

  37. [39]

    ACM Transactions on Computers , Year =

    Secure Communication Using Remote Procedure Calls , Author =. ACM Transactions on Computers , Year =

    work page

  38. [40]

    Communications of the ACM , Year =

    Grapevine: An Exercise in Distributed computing , Author =. Communications of the ACM , Year =

    work page

  39. [41]

    ACM Transactions on Computer Systems , Year =

    Implementing Remote Procedure Calls , Author =. ACM Transactions on Computer Systems , Year =

    work page

  40. [42]

    IEEE Transactions on Communications , Year =

    Pup: An Internetwork Architecture , Author =. IEEE Transactions on Communications , Year =

    work page

  41. [43]

    and others , Journal =

    Brown, Alan S. and others , Journal =. Data Base Management for. 1986 , Month = dec, Number =

    work page 1986

  42. [44]

    and others , Journal =

    Busch, J. and others , Journal =. 1987 , Month = dec, Number =

    work page 1987

  43. [45]

    45 , Note =

    de Finibus Bonorum et Malorum (The Extremes of Good and Evil) , Author =. 45 , Note =

    work page

  44. [46]

    Clegg, F. W. and others , Journal =. The. 1986 , Month = dec, Number =

    work page 1986

  45. [47]

    , Journal =

    Colwell, Robert P. , Journal =. A. 1988 , Month = aug, Number =

    work page 1988

  46. [48]

    IEEE Computer , Year =

    An Implementation Guide to a Proposed Standard for Floating-Point Arithmetic , Author =. IEEE Computer , Year =

    work page

  47. [49]

    , Institution =

    Corbato, F. , Institution =. A Paging Experiment with the. 1968 , Address =

    work page 1968

  48. [50]

    Conference Record of the 13

    Retargetable High-Level Alias Analysis , Author =. Conference Record of the 13. 1986 , Month = jan, Comment =

    work page 1986

  49. [51]

    Proceedings of the SIGPLAN '88 Conference on Programming Language Design and Implementation , Year =

    DOC: A Practical Approach to Source-Level Debugging of Globally Optimized Code , Author =. Proceedings of the SIGPLAN '88 Conference on Programming Language Design and Implementation , Year =

    work page

  50. [52]

    Coutant, D. S. and others , Journal =. Compilers for the New Generation of. 1986 , Month = jan, Number =

    work page 1986

  51. [53]

    Proceedings of the Third Intl

    Micro-Optimization of Floating-Point Operations , Author =. Proceedings of the Third Intl. Conf. on Architectural Support for Programming Languages and Operating Systems (ASPLOS-III) , Year =

    work page

  52. [54]

    , Publisher =

    Dally, William J. , Publisher =. A. 1987 , Address =

    work page 1987

  53. [55]

    Proceedings of the 14

    Architecture of a Message-Driven Processor , Author =. Proceedings of the 14. 1987 , Month = jun, Pages =

    work page 1987

  54. [56]

    Journal of Distributed Systems , Year =

    The Torus Routing Chip , Author =. Journal of Distributed Systems , Year =

    work page

  55. [57]

    IEEE Computer , Year =

    Data Flow Supercomputers , Author =. IEEE Computer , Year =

    work page

  56. [58]

    , Note =

    Efland, G. , Note =. The. 1988 , Comment =

    work page 1988

  57. [59]

    Ellis, J. R. , Publisher =. Bulldog: A Compiler for. 1986 , Address =

    work page 1986

  58. [60]

    Fisher, J. A. , Booktitle =. Very long instruction word architectures and the. 1983 , Month = jun, Pages =

    work page 1983

  59. [61]

    1988 , Address =

    A Reconfigurable Arithmetic Processor , Author =. 1988 , Address =

    work page 1988

  60. [62]

    Comuting Surveys , Year =

    Data Flow Analysis and Software Reliability , Author =. Comuting Surveys , Year =

    work page

  61. [63]

    and et al

    Fotland, David A. and et al. , Journal =. Hardware Design of the First. 1987 , Month = mar, Number =

    work page 1987

  62. [64]

    and others , Journal =

    Gassman, Gerald R. and others , Journal =. 1988 , Month = sep, Number =

    work page 1988

  63. [65]

    Proceedings of the SIGPLAN '86 symposium on Compiler Construction, ACM SIGPLAN Notices , Year =

    Efficient Instruction Scheduling for a Pipelined Architecture , Author =. Proceedings of the SIGPLAN '86 symposium on Compiler Construction, ACM SIGPLAN Notices , Year =

    work page

  64. [66]

    and Spector, A

    Gifford, D. and Spector, A. , Journal =. The. 1984 , Month = jul, Number =

    work page 1984

  65. [67]

    Communications of the ACM , Year =

    Monitors: An Operating System Structuring Concept , Author =. Communications of the ACM , Year =

    work page

  66. [68]

    1979 , Comment =

    Computer Arithmetic: Principles, Architecture, and Design , Author =. 1979 , Comment =

    work page 1979

  67. [69]

    Proceedings of the SIGPAL '88 Conference on Programming Language Design and Implementation , Year =

    An Efficient Approach for Data Flow Analysis in a Multiple Pass Global Optimizer , Author =. Proceedings of the SIGPAL '88 Conference on Programming Language Design and Implementation , Year =

    work page

  68. [70]

    James, D. V. , Journal =. Hewlett-. 1986 , Month = aug, Number =

    work page 1986

  69. [71]

    Proceedings of the SIGPLAN '86 Symp

    Effectiveness of a Machine-Level, Global Optimizer , Author =. Proceedings of the SIGPLAN '86 Symp. on Compiler Construction , Year =

    work page

  70. [72]

    1983 , Comment =

    Smalltalk-80 -- Bits of History, Words of Advice , Author =. 1983 , Comment =

    work page 1983

  71. [73]

    Communications of the ACM , Year =

    Logical Analysis of Programs , Author =. Communications of the ACM , Year =

    work page

  72. [74]

    Communications of the ACM , Year =

    Symbolic Execution and Program Testing , Author =. Communications of the ACM , Year =

    work page

  73. [75]

    Seminumerical Algorithms , Author =

    work page

  74. [76]

    Fundamental Algorithms , Author =

    work page

  75. [77]

    The Art of Computer Programming , Author =

    work page

  76. [78]

    Knvth , HowPublished =

    Jill C. Knvth , HowPublished =. The Programming of Computer Art , Address =

    work page

  77. [79]

    1986 , Month = jul, Note =

    The Gnats and Gnus Document Preparation System , Author =. 1986 , Month = jul, Note =

    work page 1986

  78. [80]

    Proceedings of the Ninth ACM Symposium on Operating Systems Principles , Year =

    Hints for Computer System Design , Author =. Proceedings of the Ninth ACM Symposium on Operating Systems Principles , Year =

    work page

  79. [81]

    Communications of the ACM , Year =

    Experience with Processes and Monitors in Mesa , Author =. Communications of the ACM , Year =

    work page

  80. [82]

    IEEE Journal on Selected Areas in Communications , Year =

    The Architecture of an Integrated Local Network , Author =. IEEE Journal on Selected Areas in Communications , Year =

    work page

Showing first 80 references.