Pith. sign in

REVIEW 2 major objections 23 references

tsbootstrap: Distribution-Free Uncertainty Quantification and Conformal Prediction for Time Series

T0 review · 2 major / 0 minor · reviewed 2026-07-10 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single library restores valid uncertainty intervals for dependent time series by combining dependence-aware bootstrap resampling with adaptive conformal calibrators.

desk verdict Solid JMLR-style software note: real tooling gap filled, honest Monte Carlo, no new theory, and the claims hold. read the letter →

arxiv 2607.06690 v1 pith:6C4AA4TK submitted 2026-07-07 stat.ME cs.AIcs.MSq-fin.STstat.AP

Sankalp Gilda This is my paper
classification stat.MEcs.AIcs.MSq-fin.STstat.AP MSC 62M1062F4062G15
keywords conformalpredictionuncertaintyquantificationbootstraptimeseriesdistribution-freeinferencesieveadaptive
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Ordinary bootstrap and conformal methods assume independent or exchangeable data, so they undercover badly on the autocorrelated streams that appear in finance, sensing, and demand. This paper presents tsbootstrap, software that unifies block, residual, sieve, and wild resampling with classical confidence intervals and adaptive conformal procedures (EnbPI, ACI, NexCP, AgACI) behind one typed API. A controlled Monte Carlo study shows that dependence-aware methods recover most of the coverage the IID bootstrap loses under short-memory linear dependence, with the sieve coming nearest to the nominal level. A compiled backend and streaming reduce keep the computation fast and memory-bounded. Practitioners who need calibrated intervals for real-world series can therefore choose methods that match the dependence structure instead of defaulting to procedures that fail under dependence.

What carries the argument

A typed specification object that selects the resampling method (block variants, residual, sieve, wild) or adaptive conformal calibrator, paired with a streaming reduce that never materializes the full O(Bn) replicate tensor.

What would settle it

Re-run the same Monte Carlo design on AR(1) and AR+ARCH series while deliberately misspecifying the automatic block length or BIC sieve order; if the sieve and block coverages then collapse toward the IID undercoverage numbers, the recovery claim is overturned.

Watch

Extended reading notes

Core claim

Under temporal dependence the IID bootstrap undercovers sharply (for example 27.8 percent empirical coverage for an AR(1) mean at 90 percent nominal), while block and especially sieve methods reduce that deficit and the sieve reaches nearest to nominal under short-memory linear dependence; the library makes those methods, plus adaptive conformal calibrators, available through a single specification-driven API.

Load-bearing premise

That the automatic block-length rule and BIC sieve-order selection used in the study are well-matched enough to the tested short-memory linear processes that the reported coverage recovery generalizes beyond those particular choices.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 0 minor

Summary. The manuscript presents tsbootstrap (v0.6.1), an MIT-licensed Python library that unifies dependence-aware time-series resampling (block, residual, sieve, wild) with classical bootstrap confidence intervals and adaptive conformal calibrators (EnbPI, ACI, NexCP, AgACI) under a single typed specification API. It positions the package against existing forecasting, conformal, and resampling libraries that supply only one of the two pieces. A controlled Monte Carlo coverage study (five DGPs, 5 000 replications, paired seeds, analytic autocovariance targets) shows that the IID bootstrap undercovers sharply under dependence while dependence-aware methods recover substantial coverage, with the sieve nearest to nominal under short-memory linear dependence (Table 1). Performance claims compare a compiled fixed-statistic backend against arch and document a streaming reduce that keeps peak extra memory O(B). Reproducibility infrastructure (CI matrix, mutation testing, sktime adapters, notebooks) is described.

Significance. If the claims hold, the work fills a genuine tooling gap: practitioners currently must stitch a general resampling engine to a separate conformal layer, or accept residual-bootstrap intervals tied to a particular forecaster. The coverage study is carefully designed (analytic targets, burn-ins, Davies–Harte ARFIMA, reported Monte Carlo SE ≤ 0.7 pp) and honestly discloses residual undercoverage and complete failure under long memory. Shipping a typed API, method metadata, recommender, streaming reduce, and public CI/mutation suite is a concrete contribution for applied time-series uncertainty quantification. The result is primarily software-engineering and empirical rather than theoretical novelty, but it is useful and well-scoped for a short software note.

major comments (2)
  1. Section 3 / Table 1: the coverage study evaluates only classical bootstrap percentile intervals for the mean; the adaptive conformal calibrators (EnbPI, ACI, NexCP, AgACI) that form half of the paper’s claimed contribution receive no analogous finite-sample coverage evaluation under the same DGPs. A short paired Monte Carlo (or at least a single non-exchangeable stream) for those calibrators would make the dual claim load-bearing rather than asserted by reference to the companion methods manuscript.
  2. Section 5: the speed comparison is restricted to a fixed-statistic compiled kernel versus arch’s general Python-callback path. The text correctly notes the asymmetry, yet the abstract and Table 2 still present “several times faster than arch” as a headline capability. Either restrict the claim to the fixed-statistic path throughout, or add a generality-matched timing (arbitrary statistic on the numpy backend) so readers can judge the practical trade-off.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: coverage targets are analytic from DGP autocovariances, methods use standard automatic selectors, and claims are empirical Monte Carlo rankings plus software engineering.

full rationale

This is a software note whose load-bearing empirical claim (Table 1) is a paired Monte Carlo coverage ranking of classical dependence-aware bootstraps against the IID baseline under four explicitly generated DGPs. Coverage targets are stated to be exact-analytic from each DGP’s closed-form autocovariance (or martingale-difference property for AR+ARCH) and cross-checked by independent Monte Carlo; they are not fitted to bootstrap output. Block lengths use the Politis–White/Patton automatic rule and sieve order uses BIC—standard selectors, not tuned to force the reported coverages. The companion methods manuscript (arXiv:2404.15227) is cited only for derivations; the present paper’s claims rest on the public Monte Carlo design, analytic targets, disclosed long-memory failure, and shipped code/CI. Performance claims are fixed-statistic benchmarks against arch, not theoretical predictions. No step reduces a claimed prediction or first-principles result to its own inputs by construction, self-definition, or load-bearing self-citation of an unverified uniqueness result. Score 0 is therefore appropriate.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

As a software + empirical-coverage paper the load-bearing content rests on standard resampling theory and the design choices of the Monte Carlo study rather than new physical entities or free parameters fitted to real data. The free parameters that do appear are the automatic tuning rules and the specific DGP coefficients used to generate the coverage table; the axioms are the classical bootstrap and conformal validity conditions under the stated dependence regimes.

free parameters (3)
  • Politis–White automatic block length (with Patton correction)
    Default block length used for all block methods in the coverage study; the reported recovery depends on this rule matching the short-memory DGPs.
  • BIC-selected sieve order
    Order of the AR sieve is chosen by BIC per series; coverage of the sieve column is conditional on this selector.
  • DGP coefficients (ϕ=0.9, ARCH parameters, d=0.4, etc.)
    Fixed by design to stress short-memory, volatility clustering, and long-memory regimes; not fitted, but the quantitative coverage numbers are specific to these choices.
assumptions (4)
  • domain assumption Block bootstrap consistency for stationary weakly dependent series under suitable block-length growth (Künsch 1989 and successors)
    Underpins the claim that moving/circular/stationary/tapered blocks restore validity; invoked throughout Sections 3–4.
  • domain assumption Sieve bootstrap validity for linear processes (Bühlmann 1997; Kreiss 1992)
    Justifies the sieve column of Table 1 and the recursive residual regeneration scheme.
  • domain assumption Finite-sample marginal coverage of the listed adaptive conformal procedures under their stated (possibly non-exchangeable) conditions
    EnbPI, ACI, NexCP, AgACI are used as black-box calibrators; validity is taken from the cited papers.
  • standard math Standard Monte Carlo design assumptions (paired seeds, analytic autocovariance targets, burn-in sufficiency)
    Required for the coverage percentages and SE claims in Section 3 / Table 1 to be interpretable.

how reviews work

0 comments
Cite this review

Pith. "Pith review of tsbootstrap: Distribution-Free Uncertainty Quantification and Conformal Prediction for Time Series." pith.science (2026). https://pith.science/paper/6C4AA4TK

@misc{pith2026260706690,
  author       = {Pith},
  title        = {Pith review of: tsbootstrap: Distribution-Free Uncertainty Quantification and Conformal Prediction for Time Series},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6C4AA4TK}},
  note         = {Machine review of arXiv:2607.06690}
}
abstract

Finance, sensing, and demand streams violate the exchangeability that IID conformal prediction and the IID bootstrap assume, and existing libraries implement either a general resampling engine or conformal calibration without the other. tsbootstrap provides block, residual, sieve, and wild resampling, classical bootstrap confidence intervals, and adaptive conformal calibrators (EnbPI, ACI, NexCP, AgACI) through a single typed API in which a specification object selects each method. In a controlled coverage study the IID bootstrap undercovers sharply under dependence; dependence-aware methods reduce the coverage deficit, the sieve nearest to nominal under short-memory linear dependence. On the shared fixed-statistic path a compiled backend runs several times faster than arch, and a streaming reduce avoids materializing the $O(Bn)$ replicate tensor, limiting peak extra memory to $O(B)$ for the statistic array. The software is MIT licensed (v0.6.1).

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

23 extracted references · 23 canonical work pages

  1. [1]

    F., Cand\`es, E

    Barber, R. F., Cand\`es, E. J., Ramdas, A., and Tibshirani, R. J. (2023). Conformal prediction beyond exchangeability. The Annals of Statistics, 51(2):816--845

    work page 2023

  2. [2]

    B\"uhlmann, P. (1997). Sieve bootstrap for time series. Bernoulli, 3(2):123--148

    work page 1997

  3. [3]

    Carlstein, E. (1986). The use of subseries values for estimating the variance of a general statistic from a stationary sequence. The Annals of Statistics, 14(3):1171--1179

    work page 1986

  4. [4]

    and Flachaire, E

    Davidson, R. and Flachaire, E. (2008). The wild bootstrap, tamed at last. Journal of Econometrics, 146(1):162--169

    work page 2008

  5. [5]

    Efron, B. (1979). Bootstrap methods: another look at the jackknife. The Annals of Statistics, 7(1):1--26

    work page 1979

  6. [6]

    Efron, B. (1987). Better bootstrap confidence intervals. Journal of the American Statistical Association, 82(397):171--185

    work page 1987

  7. [7]

    and Cand\`es, E

    Gibbs, I. and Cand\`es, E. (2021). Adaptive conformal inference under distribution shift. Advances in Neural Information Processing Systems, 34:1660--1672

    work page 2021

  8. [8]

    otze, F. and K\

    G\"otze, F. and K\"unsch, H. R. (1996). Second-order correctness of the blockwise bootstrap for stationary observations. The Annals of Statistics, 24(5):1914--1933

    work page 1996

Show all 23 references
  1. [9]

    Huang, J., Song, J., Zhou, X., Jing, B., and Wei, H. (2025). TorchCP: A Python library for conformal prediction. Journal of Machine Learning Research, 26(266):1--25

    2025

  2. [10]

    Kreiss, J.-P. (1992). Bootstrap procedures for AR( ) processes. In Bootstrapping and Related Techniques, pages 107--113. Springer

    1992

  3. [11]

    K\"unsch, H. R. (1989). The jackknife and the bootstrap for general stationary observations. The Annals of Statistics, 17(3):1217--1241

    1989

  4. [12]

    Liu, R. Y. (1988). Bootstrap procedures under some non-i.i.d.\ models. The Annals of Statistics, 16(4):1696--1708

    1988

  5. [13]

    Mammen, E. (1993). Bootstrap and wild bootstrap for high dimensional linear models. The Annals of Statistics, 21(1):255--285

    1993

  6. [14]

    and Politis, D

    Paparoditis, E. and Politis, D. N. (2001). Tapered block bootstrap. Biometrika, 88(4):1105--1119

    2001

  7. [15]

    Pascual, L., Romo, J., and Ruiz, E. (2004). Bootstrap predictive inference for ARIMA processes. Journal of Time Series Analysis, 25(4):449--465

    2004

  8. [16]

    N., and White, H

    Patton, A., Politis, D. N., and White, H. (2009). Correction to ``Automatic block-length selection for the dependent bootstrap'' by D. Politis and H. White. Econometric Reviews, 28(4):372--375

    2009

  9. [17]

    Politis, D. N. and Romano, J. P. (1992). A circular block-resampling procedure for stationary data. In Exploring the Limits of Bootstrap, pages 263--270. Wiley

    1992

  10. [18]

    Politis, D. N. and Romano, J. P. (1994). The stationary bootstrap. Journal of the American Statistical Association, 89(428):1303--1313

    1994

  11. [19]

    Politis, D. N. and White, H. (2004). Automatic block-length selection for the dependent bootstrap. Econometric Reviews, 23(1):53--70

    2004

  12. [20]

    Shao, X. (2010). The dependent wild bootstrap. Journal of the American Statistical Association, 105(489):218--235

    2010

  13. [21]

    Wu, C. F. J. (1986). Jackknife, bootstrap and other resampling methods in regression analysis. The Annals of Statistics, 14(4):1261--1295

    1986

  14. [22]

    and Xie, Y

    Xu, C. and Xie, Y. (2021). Conformal prediction interval for dynamic time-series. Proceedings of the 38th International Conference on Machine Learning, PMLR 139:11559--11569

    2021

  15. [23]

    Zaffran, M., F\'eron, O., Goude, Y., Josse, J., and Dieuleveut, A. (2022). Adaptive conformal predictions for time series. Proceedings of the 39th International Conference on Machine Learning, PMLR 162:25834--25866

    2022

Pith tools

Reviewed July 10, 2026 · model on record in the stance chip above.