REVIEW 3 major objections 2 minor 4 references
Conformal prediction for functional time series: Application to age-specific mortality rates
T0 review · 3 major / 2 minor · reviewed 2026-06-29 · grok-4.3
Pith's one-line read Conformal prediction constructs distribution-free prediction intervals for functional time series such as age-specific mortality rates.
desk verdict The paper applies split and sequential conformal prediction to functional mortality time series, but serial dependence likely undercuts the finite-sample coverage claims. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Split conformal prediction that partitions the data to calibrate empirical coverage on a validation set, and sequential conformal prediction that updates quantiles autoregressively, both applied to any base forecasting model for the functional observations.
What would settle it
On the Australian age- and sex-specific mortality holdout set, the empirical coverage probability after calibration deviates substantially from the nominal level or the mean interval score exceeds that of standard model-based intervals.
Extended reading notes
Core claim
The paper claims that split conformal prediction, by dividing data into training, validation, and test sets and calibrating tuning parameters on the validation set to align empirical coverage with nominal values, together with sequential conformal prediction that updates predicted quantiles via an autoregressive process, can produce prediction intervals for functional time series that achieve the desired coverage on holdout observations while remaining valid regardless of the underlying distribution or model.
Load-bearing premise
Splitting the data or sequential updating allows the empirical coverage on validation observations to be adjusted to the nominal level while the resulting intervals remain valid on the holdout functional data.
Editorial extensions
If this is right
- The calibrated intervals achieve empirical coverage close to the nominal level on unseen functional observations.
- The procedures apply to any chosen forecasting model without requiring distributional assumptions.
- Sequential conformal prediction produces intervals without needing to reserve separate validation data.
- Interval accuracy can be compared directly via coverage probability difference and mean interval score on the mortality application.
Reading between the lines
- The same calibration logic could be tested on functional series from other countries or domains such as economic indicators to check whether coverage holds under different dependence structures.
- If the functional observations satisfy a suitable exchangeability condition, the validity guarantee would extend beyond the Australian mortality example without further tuning.
- Combining the conformal intervals with existing functional time series models might reduce over- or under-coverage that arises when those models are misspecified.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces two conformal prediction variants for functional time series: split conformal (data split into training/validation/test, with validation used to calibrate tuning parameters for nominal coverage) and sequential conformal (autoregressive updating of quantiles without splitting). Both are applied to Australian age- and sex-specific log mortality rates as a model-agnostic, distribution-free alternative to parametric forecast uncertainty quantification, with evaluation via empirical coverage probability, coverage probability difference, and mean interval score.
Significance. If finite-sample validity can be established under temporal dependence, the methods would supply a useful nonparametric tool for interval forecasting in demography that avoids model misspecification risks. The empirical comparison on real mortality data and the explicit handling of functional observations are practical strengths; however, the core distribution-free claim requires additional justification to be load-bearing.
major comments (3)
- [Abstract; split conformal procedure] Abstract and the split-conformal description: the finite-sample marginal coverage guarantee of split conformal rests on exchangeability of calibration and test nonconformity scores. The manuscript applies the procedure directly to serially dependent functional observations (age-specific mortality curves with trends and autocorrelation) without stating mixing, blocking, or other conditions that would restore exchangeability; empirical coverage on the Australian data does not substitute for this guarantee.
- [Sequential conformal prediction] Sequential conformal section: the autoregressive update of predicted quantiles introduces a parametric structure on the quantile process. This appears to contradict the repeated claim of a 'model-agnostic and distribution-free' method; the manuscript should clarify whether the AR step preserves the distribution-free property or merely approximates it.
- [Numerical results / evaluation] Evaluation section: the reported metrics (empirical coverage, CPD, MIS) are computed on holdout functional curves, but no diagnostic is given for whether the observed coverage deviations arise from dependence violation versus other sources (e.g., choice of nonconformity score for functional data). A direct comparison against a blocked or mixing-adjusted conformal baseline would strengthen the central claim.
minor comments (2)
- [Method definitions] Notation for the functional observations and the nonconformity measure should be made fully explicit (e.g., how the functional residual is scalarized) to allow replication.
- [Split conformal procedure] The abstract states that tuning parameters are chosen by 'calibrating the empirical coverage probabilities to match their nominal values' on the validation set; this step should be described with an equation or algorithm box to avoid ambiguity about whether it is a simple grid search or a more involved optimization.
Simulated Author's Rebuttal
We thank the referee for the constructive and detailed comments, which help clarify the scope and limitations of our work. We address each major comment point by point below, indicating planned revisions where appropriate.
read point-by-point responses
-
Referee: [Abstract; split conformal procedure] Abstract and the split-conformal description: the finite-sample marginal coverage guarantee of split conformal rests on exchangeability of calibration and test nonconformity scores. The manuscript applies the procedure directly to serially dependent functional observations (age-specific mortality curves with trends and autocorrelation) without stating mixing, blocking, or other conditions that would restore exchangeability; empirical coverage on the Australian data does not substitute for this guarantee.
Authors: We agree that the standard finite-sample marginal coverage guarantee for split conformal prediction requires exchangeability between calibration and test scores, an assumption violated by serially dependent functional time series. The manuscript presents the method primarily as a practical, model-agnostic tool evaluated empirically on Australian mortality data rather than deriving new theoretical guarantees under dependence. We will revise the abstract, introduction, and split conformal section to explicitly note that the distribution-free guarantee holds under exchangeability and that the procedure is applied heuristically to dependent data, with performance assessed via empirical coverage. A brief discussion of mixing or blocking conditions as potential future extensions will be added. revision: yes
-
Referee: [Sequential conformal prediction] Sequential conformal section: the autoregressive update of predicted quantiles introduces a parametric structure on the quantile process. This appears to contradict the repeated claim of a 'model-agnostic and distribution-free' method; the manuscript should clarify whether the AR step preserves the distribution-free property or merely approximates it.
Authors: The sequential conformal procedure uses an autoregressive model solely to update the estimated quantiles over time in the absence of sample splitting. The conformal calibration step that produces the final intervals remains distribution-free conditional on those quantile estimates, without assuming a specific error distribution. However, the AR update does introduce a parametric modeling choice for the quantile dynamics. We will revise the sequential conformal section and related claims to distinguish these elements, stating that the method is distribution-free with respect to the prediction errors while acknowledging the parametric approximation in the sequential update step. revision: yes
-
Referee: [Numerical results / evaluation] Evaluation section: the reported metrics (empirical coverage, CPD, MIS) are computed on holdout functional curves, but no diagnostic is given for whether the observed coverage deviations arise from dependence violation versus other sources (e.g., choice of nonconformity score for functional data). A direct comparison against a blocked or mixing-adjusted conformal baseline would strengthen the central claim.
Authors: We agree that diagnostics separating dependence effects from other factors (such as nonconformity score choice) would be informative, and that a blocked or mixing-adjusted baseline could provide useful context. Our evaluation prioritizes real-data performance using standard metrics on holdout curves. Adding a full blocked conformal comparison would require new methodological development and extensive additional experiments beyond the paper's scope. We will revise the evaluation section to discuss potential sources of coverage deviations, including serial dependence, and explicitly note the absence of blocked baselines as a limitation with suggestions for future work. revision: partial
- Establishing finite-sample coverage guarantees under serial dependence for functional time series without additional assumptions such as mixing or blocking.
Circularity Check
No circularity: method applies standard split/sequential conformal to functional series without self-referential reduction.
full rationale
The paper presents split conformal (data partitioning into training/validation/test with empirical coverage calibration on validation) and sequential conformal (autoregressive quantile updates) as model-agnostic procedures. No quoted equation or step shows a prediction or coverage guarantee reducing by construction to a parameter fitted on the identical holdout data; the derivation chain rests on the exchangeability assumption of conformal prediction applied to the observed series, with external empirical evaluation on Australian mortality rates. No self-citation load-bearing, ansatz smuggling, or renaming of known results appears in the provided abstract or description.
Assumptions & free parameters
Cite this review
Pith. "Pith review of Conformal prediction for functional time series: Application to age-specific mortality rates." pith.science (2026). https://pith.science/paper/DN3LNWHG
@misc{pith2026260529296,
author = {Pith},
title = {Pith review of: Conformal prediction for functional time series: Application to age-specific mortality rates},
year = {2026},
howpublished = {\url{https://pith.science/paper/DN3LNWHG}},
note = {Machine review of arXiv:2605.29296}
}
read the original abstract
In demographic literature, forecast uncertainty is often quantified with a statistical model. This model-based approach may potentially suffer from drawbacks, namely model misspecification, selection effect, and lack of finite-sample validity. We introduce a model-agnostic and distribution-free procedure, conformal prediction, for constructing prediction intervals for a functional time series. In the family of conformal prediction, split conformal prediction divides the data into training, validation, and test sets. Within the validation set, we can select optimal tuning parameters by calibrating the empirical coverage probabilities to match their nominal values. With the selected optimal tuning parameters, we then construct the prediction intervals using the same forecasting model for the holdout data in the testing set. Without sample splitting, sequential conformal prediction sequentially updates the predicted quantiles via an autoregressive process. Using Australian age- and sex-specific log mortality rates, we evaluate and compare the interval forecast accuracy, as measured by empirical coverage probability, coverage probability difference and mean interval score, between the two variants of conformal prediction.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
Alho, J. M. and Spencer, B. D. (2005),Statistical Demography and Forecasting, Springer-Verlag, New York. Angelopoulos, A. N., Cand `es, E. J. and Tibshirani, R. J. (2023), Conformal PID control for time series prediction,in‘NIPS’23: Proceedings of the 37th International Conference on Neural Information Processing Systems’, pp. 23047–23074. URL: https://pr...
2005
-
[2]
and Myrskyl¨a, M
Duerst, R., Sch ¨oley, J., Hellstrand, J. and Myrskyl¨a, M. (2024), Calibrating probabilistic forecast paths on past forecast errors: An application to the Finnish total fertility rate, Working paper WP 2024-01, Max Planck Institute for Demographic Research. URL:https://www.demogr.mpg.de/papers/working/wp-2024-016.pdf Fontana, M., Zeni, G. and Vantini, S....
2024
-
[3]
and Shang, H
Paparoditis, E. and Shang, H. L. (2023), ‘Bootstrap prediction bands for functional time series’, Journal of the American Statistical Association: Theory and Method118(542), 972–986. Pilavakis, D., Paparoditis, E. and Sapatinas, T. (2019), ‘Moving block and tapered block bootstrap for functional time series with an application to the K-sample mean problem...
2023
-
[4]
Preston, S., Heuveline, P . and Guillot, M. (2001),Demography: Measuring and Modeling Population Processes, Blackwell Publishers, Malden, MA. Reiss, P . T., Goldsmith, J., Shang, H. L. and Ogden, R. T. (2017), ‘Methods for scalar-on-function regression’,International Statistical Review85(2), 228–249. Shafer, G. and Vovk, V . (2008), ‘A tutorial on conform...
Reviewed June 29, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.