Pith. sign in

REVIEW 4 major objections 5 minor 26 references

The Granularity Paradox: How Temporal Disaggregation Inflates In-Sample Fit and Compounds Out-of-Sample Error

T0 review · 4 major / 5 minor · reviewed 2026-07-11 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Finer time grains inflate in-sample fit but compound recursive forecast error over longer horizons; the damage tracks feedback topology, not model complexity.

desk verdict Clear single-series map of recursive compounding vs sample size, with a useful cumulative-metric warning; topology claim is suggestive but not yet structural. read the letter →

arxiv 2607.05450 v1 pith:PY4YXJG7 submitted 2026-07-05 cs.LG cs.AIstat.ME

Hugo Moreira This is my paper · ORCID
classification cs.LGcs.AIstat.ME
keywords ForecastingTemporalAggregationErrorPropagationDeepLearningGranularityParadoxPublicProcurementMulti-stepAheadCumulativeMetrics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

When you fix a planning window such as one year and choose how finely to slice the series, you face a trade-off the paper names the Granularity Paradox. Daily or weekly data give estimators more observations and prettier training diagnostics, but they force multi-step recursive models to chain hundreds of predictions, so small biases explode. Annual data remove that chaining yet starve the estimator of sample size. Across six grains and ten models on a long public-procurement series, recursive and seasonal methods collapse at high frequency while a plain linear regression of the series on time stays flat near 16–17% total percentage error. An LSTM only recovers after the sample becomes large enough to overcome the same penalty, tracing a U-shaped error curve. Pointwise scores such as RMSE hide the cumulative planning gap; a consensus–dissensus check against total percentage forecast error exposes which models are systematically biased. The practical message is that grain choice is a design decision that controls error dynamics, and evaluation without a goal-dependent cumulative metric systematically misranks models.

What carries the argument

The Granularity Paradox trade-off between sample size N and recursive horizon H, together with the consensus–dissensus diagnostic that compares directional changes in pointwise metrics (RMSE, MAE, R²) against cumulative Total Percentage Forecast Error (TPFE) across grains.

What would settle it

Repeat the six-grain, ten-model expanding-window backtest on several additional independent series (different CPV divisions or domains); if recursive models no longer show systematic TPFE degradation at fine grains relative to non-recursive baselines, or if the LSTM U-shape disappears, the structural claim fails.

Watch

Extended reading notes

Core claim

Finer temporal disaggregation improves in-sample fit and sample size yet degrades out-of-sample multi-step accuracy for recursive models because it lengthens the forecast horizon and compounds feedback error; the effect is driven by recursive topology rather than complexity, as linear regression remains stable across all six grains while recursive seasonal models and intermediate-grain LSTMs degrade.

Load-bearing premise

The directional patterns (recursive collapse at fine grains, LSTM U-shape, linear-regression flatness) are assumed to be structural features of model–grain interaction that will hold beyond the single 13-year IT-procurement series used in the benchmark.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper defines a “Granularity Paradox”: for a fixed planning horizon, finer temporal grains raise N and in-sample fit but lengthen the multi-step horizon H, so recursive/state-dependent models compound forecast error out of sample, while coarse grains avoid recursion but starve estimators. It formalizes recursive bias growth with a simple AR(1) sketch (Eqs. 6–7), introduces cumulative metrics (CFE/TAFE/TPFE) and a consensus–dissensus diagnostic that contrasts pointwise vs cumulative directional behaviour, and benchmarks 10 models (naïve, statistical, XGBoost, LSTM, N-BEATS) over six grains on a 13-year Portuguese public-procurement series (CPV 72) with 8-fold expanding-window backtesting. Empirically, Linear Regression stays near 16–17% TPFE across grains; recursive seasonal models collapse at Daily (Holt-Winters Test R² −151, TPFE 425.85%); LSTM shows a U-shaped TPFE path that recovers only at Daily; and pointwise metrics often improve while TPFE degrades for recursive models.

Significance. If the recursive-topology account generalizes, the paper makes a practically important point: temporal grain is an upstream design choice that can dominate architecture, and evaluation without a goal-dependent cumulative metric systematically misranks models for budgeting/volume planning. Strengths include a transparent multi-model, multi-grain table of 8-fold means; explicit naïve recursive controls; a clear non-recursive contrast (Linear Regression); and a usable consensus–dissensus diagnostic that does not require extra model runs. The AR(1) bias-compounding sketch is standard and supports the qualitative mechanism. The main scientific value is as a careful demonstration and evaluation methodology rather than as a fully established structural law of forecasting.

major comments (4)
  1. [Abstract; §9 Discussion; §10 Conclusion; §11] Abstract and §9–10 treat the LR-vs-recursive split, LSTM U-shape, and “recursive feedback topology, not model complexity” as structural properties of model–granularity interaction. All cells in Table 1 come from one CPV-72 series (§7, §11). The AR(1) sketch (Eqs. 6–7) is illustrative and does not prove scale-invariance of non-recursive linear projection under other DGPs. Either add multi-series evidence (other CPV divisions or public series) or systematically scope abstract/discussion/conclusion to a single-series demonstration with directional hypotheses, not a general structural claim.
  2. [§7 Empirical Evaluation; Table 1; §9 point 3] The LSTM U-shaped TPFE curve (Monthly 19.66% → Bi-Weekly 35.94% → Daily 4.35%, Test R² 0.66) is a headline result and the main evidence that high-capacity recursive models can “overcome” the H penalty. §7 states single-seed LSTM (seed 42), hyperparameters grid-searched on the first fold and held fixed across grains, and no multi-seed CIs or Diebold–Mariano tests. Fixed hyperparams can confound grain with capacity–data match; a single Daily seed can drive the recovery. Report multi-seed means/dispersion for LSTM/N-BEATS (and preferably XGBoost) at least at Monthly/Bi-Weekly/Daily, or move the U-shape from a confirmed threshold to a provisional pattern and soften abstract claims accordingly.
  3. [§7; §9.2; Table 1 Linear Regression rows] The topology claim rests on Linear Regression being non-recursive (“projects predictions directly as a function of time,” §9.2) while ARIMAX/SARIMAX/Holt-Winters/Persistence feed predictions back. The manuscript does not specify the Linear Regression feature set (time index only vs lags/exogenous), how multi-step forecasts are produced at each grain, or whether the same exogenous contract-value series used in ARIMAX/SARIMAX enters LR. Without that protocol, the flat 16.09–16.96% TPFE band cannot be cleanly attributed to absence of recursive feedback rather than to a different information set or direct multi-step setup. Add an explicit forecasting protocol for every model class (recursive vs direct, features, exogenous use).
  4. [§6 Eqs. 16–18; Table 1 Annual rows; §10] TPFE is defined as absolute cumulative error over the full test window scaled by total observed volume (Eq. 18) and is treated as the primary, goal-dependent criterion. For H=1 (Annual) TAFE equals the single-step absolute error, so cross-grain TPFE comparisons mix one-shot level error with multi-step cumulative bias. The paper already notes R² is undefined at H=1; it should also discuss whether Annual TPFE is commensurate with Daily TPFE for ranking, and whether CFE sign (over- vs under-prediction) matters for procurement budgeting. A short sensitivity (e.g., reporting signed CFE or horizon-normalized cumulative error) would strengthen the metric argument in §6 and §10.
minor comments (5)
  1. [Table 2] Table 2’s global log sparklines are described in prose but are hard to verify in the text-only manuscript; ensure the published version has readable glyphs and a self-contained caption defining the [0,1] maps and clipping (R² at −1, TPFE at 100%).
  2. [Abstract; §9.2] Abstract TPFE band for Linear Regression is “16.3–17.0%” while Table 1 and §9 give 16.09–16.96% (and “16.1–17.0%” in one place). Align all stated ranges with Table 1.
  3. [Table 1; §9.1] Holt-Winters Daily Test R² = −151 is extreme; a one-sentence check that seasonal period, initialization, and recursive update match the Daily grain (and that the figure is not a numerical overflow) would help readers trust the collapse narrative.
  4. [§4; §9.4] §4 related work is appropriate; a brief pointer to multi-horizon non-recursive DL (e.g., direct/MIMO or TFT-style) in the discussion would clarify that the paradox is about recursive deployment, not “deep learning” per se—consistent with your N-BEATS vs LSTM contrast.
  5. [§11] Reproducibility: Portal BASE is public and filters are described, but code is “on request.” Depositing the preprocessing, fold indices, and configs would match the paper’s methodological emphasis.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the Granularity Paradox is an empirical observation from independent backtesting, not a result forced by definition, fitted targets, or self-citation.

full rationale

The paper's central claims rest on an empirical 8-fold expanding-window benchmark of 10 models across six granularities on one public series, together with a standard AR(1) bias-propagation sketch (Eqs. 6–7) that is purely illustrative. TPFE/CFE/TAFE are defined directly from observed and forecasted sums (Eqs. 16–18) without any parameter fitted to the target claim; the consensus-dissensus labels are post-hoc directional comparisons of those metrics and do not enter model training. Linear Regression's flat TPFE band, Holt-Winters collapse, and LSTM U-shape are reported outcomes of the same held-out evaluation protocol, not quantities recovered by construction from inputs. All foundational citations (temporal aggregation, multi-step strategies, MAPA, hierarchical reconciliation, DeepAR, N-BEATS, etc.) are external; there are no self-citations, uniqueness theorems imported from the author, or ansätze smuggled via prior work by the same author. The single-series / single-seed limitations affect generalizability, not circularity of the derivation chain. The result is therefore self-contained against its own evaluation metrics and external literature.

Assumptions & free parameters 4 free parameters · 4 assumptions · 3 invented entities

The central claim rests on standard multi-step forecasting theory plus experimental design choices on one public series. Free parameters are hyperparameters and seeds held fixed; axioms are domain assumptions about recursive generation and cumulative planning objectives; invented entities are named constructs (Granularity Paradox, TPFE, consensus-dissensus) that reorganize known ideas rather than new physical objects.

free parameters (4)
  • LSTM / N-BEATS / XGBoost hyperparameters (grid on first fold, fixed across grains)
    Chosen on fold 1 and frozen so grain effects are not confounded with retuning; magnitudes (especially LSTM Daily TPFE) depend on this choice.
  • LSTM random seed = 42; N-BEATS Darts defaults
    Deep-learning results are single-seed point estimates; directional claims depend on these fixed seeds.
  • Expanding-window 8-fold backtesting scheme
    Fold count and expanding design determine reported means; not derived from theory.
  • Planning horizon H fixed to one year equivalent per grain (H=1,4,12,26,52,365)
    Defines the multi-step length that drives compounding; alternative horizons would change the trade-off surface.
assumptions (4)
  • domain assumption Recursive multi-step forecasts feed predictions back as inputs, so parameter bias compounds with horizon H (AR(1) sketch).
    Section 3.1; standard but load-bearing for attributing degradation to recursive topology.
  • domain assumption Cumulative planning error over a fixed physical horizon is the operational objective, so TPFE is the primary adequacy metric.
    Section 6 and Discussion; without this, pointwise metrics would not be 'masking'.
  • domain assumption Resampling the same micro-data to coarser grains preserves a meaningful comparable series for cross-grain comparison.
    Section 7 empirical design; aggregation method affects noise and seasonality.
  • ad hoc to paper Directional model–grain patterns on one CPV-72 series reflect structural topology effects rather than series idiosyncrasy.
    Stated in Limitations as expected to generalise; not independently established in the paper.
invented entities (3)
  • Granularity Paradox
    purpose: Name the N-vs-H trade-off where finer grain helps in-sample fit but hurts recursive multi-step out-of-sample accuracy.
    Branding of a known tension between sample size and recursive horizon; useful framing, not a new mechanism with external handle.
  • Total Percentage Forecast Error (TPFE) independent evidence
    purpose: Scale-free cumulative absolute error over the planning horizon for cross-grain comparison.
    Simple normalization of cumulative forecast error; standard idea under a new acronym.
  • Consensus-dissensus diagnostic independent evidence
    purpose: Flag when pointwise metrics improve while TPFE degrades across grains, indicating masked bias propagation.
    Post-hoc classification rule on metric directions; falsifiable on other datasets if both metric classes are reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Granularity Paradox: How Temporal Disaggregation Inflates In-Sample Fit and Compounds Out-of-Sample Error." pith.science (2026). https://pith.science/paper/PY4YXJG7

@misc{pith2026260705450,
  author       = {Pith},
  title        = {Pith review of: The Granularity Paradox: How Temporal Disaggregation Inflates In-Sample Fit and Compounds Out-of-Sample Error},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PY4YXJG7}},
  note         = {Machine review of arXiv:2607.05450}
}
read the original abstract

This paper explores the "Granularity Paradox" in time-series forecasting, wherein finer temporal disaggregation (e.g., Monthly to Weekly/Daily) improves in-sample diagnostics and dataset size (N), but degrades out-of-sample accuracy due to recursive error compounding over longer horizons (H). Conversely, coarse aggregation (Annual) eliminates recursive error propagation but reduces data available to estimators. We formalize this trade-off and benchmark 10 models - spanning na\"ive, statistical, machine learning, and deep learning architectures - across six granularities using a 13-year public procurement dataset. The empirical results reveal a non-monotonic threshold structure: recursive autoregressive and seasonal models degrade substantially under high-frequency forecasting (e.g., Holt-Winters reaches a Test R-squared of -151 and TPFE of 425.85% at the Daily grain), while the LSTM traces a U-shaped error curve, worsening from Monthly (19.66%) through Bi-Weekly (35.94%) before overcoming the error propagation penalty at Daily (TPFE of 4.35%, R-squared of 0.66). Linear Regression remains stable across all granularities (16.3-17.0% TPFE), confirming that the paradox is driven by recursive feedback topology, not model complexity. The results demonstrate that standard pointwise metrics (RMSE, MAE) systematically mask cumulative error propagation, and that evaluating forecasts without goal-dependent cumulative metrics produces misleading assessments of model adequacy. We introduce a consensus-dissensus diagnostic comparing the directional behaviour of pointwise metrics against cumulative TPFE across granularities, enabling the identification of models whose standard diagnostics mask systematic error propagation.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references

  1. [1]

    Hyndman, Nikolaos Kourentzes, and Fotios Petropoulos

    George Athanasopoulos, Rob J. Hyndman, Nikolaos Kourentzes, and Fotios Petropoulos. Forecasting with temporal hierarchies. European Journal of Operational Research, 262 0 (1): 0 60--74, 2017

    2017

  2. [2]

    Hyndman, Nikolaos Kourentzes, and Anastasios Panagiotelis

    George Athanasopoulos, Rob J. Hyndman, Nikolaos Kourentzes, and Anastasios Panagiotelis. Forecast reconciliation: A review. International Journal of Forecasting, 40 0 (2): 0 430--456, 2024

    2024

  3. [3]

    Atiya, and Antti Sorjamaa

    Souhaib Ben Taieb , Gianluca Bontempi, Amir F. Atiya, and Antti Sorjamaa. A review and comparison of strategies for multi-step ahead time series forecasting based on the NN5 forecasting competition. Expert Systems with Applications, 39 0 (8): 0 7067--7083, 2012

    2012

  4. [4]

    Random search for hyper-parameter optimization

    James Bergstra and Yoshua Bengio. Random search for hyper-parameter optimization. Journal of Machine Learning Research, 13: 0 281--305, 2012

    2012

  5. [5]

    Lazy learning for local modelling and control design

    Gianluca Bontempi, Mauro Birattari, and Hugues Bersini. Lazy learning for local modelling and control design. International Journal of Control, 72 0 (7-8): 0 643--658, 1999

    1999

  6. [6]

    George E. P. Box and Gwilym M. Jenkins. Time Series Analysis: Forecasting and Control. Holden-Day, San Francisco, revised edition, 1976

    1976

  7. [7]

    XGBoost : A scalable tree boosting system

    Tianqi Chen and Carlos Guestrin. XGBoost : A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , pages 785--794, 2016

    2016

  8. [8]

    Hamilton

    James D. Hamilton. Time Series Analysis. Princeton University Press, 1994

    1994

Show all 26 references
  1. [9]

    Darts: U ser-friendly modern machine learning for time series

    Julien Herzen, Francesco L \"a ssig, Samuele Giuliano Piazzetta, Thomas Neuer, L \'e o Tafti, Guillaume Raille, Tomas Van Pottelbergh, Marek Pasieka, Andrzej Skrodzki, and Nicolas Huguenin. Darts: U ser-friendly modern machine learning for time series. Journal of Machine Learn...

    2022

  2. [10]

    Long short-term memory

    Sepp Hochreiter and J \"u rgen Schmidhuber. Long short-term memory. Neural Computation, 9 0 (8): 0 1735--1780, 1997

    1997

  3. [11]

    Hyndman and George Athanasopoulos

    Rob J. Hyndman and George Athanasopoulos. Forecasting: Principles and Practice . OTexts, Melbourne, Australia, 3rd ed. edition, 2021

    2021

  4. [12]

    Hyndman and Anne B

    Rob J. Hyndman and Anne B. Koehler. Another look at measures of forecast accuracy. International Journal of Forecasting, 22 0 (4): 0 679--688, 2006

    2006

  5. [13]

    Nikolaos Kourentzes, Fotios Petropoulos, and Juan R. Trapero. Improving forecasting via multiple temporal aggregation. International Journal of Forecasting, 30 0 (3): 0 464--476, 2014

    2014

  6. [14]

    Ar k, Nicolas Loeff, and Tomas Pfister

    Bryan Lim, Sercan O. Ar k, Nicolas Loeff, and Tomas Pfister. Temporal fusion transformers for interpretable multi-horizon time series forecasting. International Journal of Forecasting, 37 0 (4): 0 1748--1764, 2021

    2021

  7. [15]

    Statistical and Machine Learning forecasting methods: Concerns and ways forward

    Spyros Makridakis, Evangelos Spiliotis, and Vassilios Assimakopoulos. Statistical and Machine Learning forecasting methods: Concerns and ways forward. PLOS ONE, 13 0 (3): 0 e0194889, 2018

    2018

  8. [16]

    The M4 competition: 100,000 time series and 61 forecasting methods

    Spyros Makridakis, Evangelos Spiliotis, and Vassilios Assimakopoulos. The M4 competition: 100,000 time series and 61 forecasting methods. International Journal of Forecasting, 36 0 (1): 0 54--74, 2020

    2020

  9. [17]

    Stock, and Mark W

    Massimiliano Marcellino, James H. Stock, and Mark W. Watson. A comparison of direct and iterated multistep AR methods for forecasting macroeconomic time series. Journal of Econometrics, 135 0 (1-2): 0 499--526, 2006

    2006

  10. [18]

    Syntetos, John E

    Konstantinos Nikolopoulos, Aris A. Syntetos, John E. Boylan, Fotios Petropoulos, and Vassilios Assimakopoulos. An aggregate--disaggregate intermittent demand approach ( ADIDA ) to forecasting. Journal of the Operational Research Society, 62 0 (3): 0 544--554, 2011

    2011

  11. [19]

    Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio

    Boris N. Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. N-BEATS : Neural basis expansion analysis for interpretable time series forecasting. In Proceedings of the 8th International Conference on Learning Representations ( ICLR ) , 2020

    2020

  12. [20]

    Forecast combinations for intermittent demand

    Fotios Petropoulos and Nikolaos Kourentzes. Forecast combinations for intermittent demand. Journal of the Operational Research Society, 66 0 (6): 0 914--924, 2015

    2015

  13. [21]

    Zied Babai, Aris Syntetos, and Yves Ducq

    Bahman Rostami-Tabar, M. Zied Babai, Aris Syntetos, and Yves Ducq. Demand forecasting by temporal aggregation. Naval Research Logistics, 60 0 (6): 0 479--498, 2013

    2013

  14. [22]

    DeepAR : Probabilistic forecasting with autoregressive recurrent networks

    David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. DeepAR : Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting, 36 0 (3): 0 1181--1191, 2020

    2020

  15. [23]

    Temporal aggregation of univariate and multivariate time series models: a survey

    Andrea Silvestrini and David Veredas. Temporal aggregation of univariate and multivariate time series models: a survey. Journal of Economic Surveys, 22 0 (3): 0 458--497, 2008

    2008

  16. [24]

    Tiao and William W

    George C. Tiao and William W. S. Wei. Effect of temporal aggregation on the dynamic relationship of two time series variables. Biometrika, 63 0 (3): 0 513--523, 1976

    1976

  17. [25]

    William W. S. Wei. Time Series Analysis: Univariate and Multivariate Methods. Pearson Addison Wesley, 2nd edition, 2006

    2006

  18. [26]

    Are transformers effective for time series forecasting? In Proceedings of the AAAI Conference on Artificial Intelligence , volume 37, pages 11121--11128, 2023

    Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AAAI Conference on Artificial Intelligence , volume 37, pages 11121--11128, 2023

    2023

Pith tools

Reviewed July 11, 2026 · model on record in the stance chip above.