Online Conformal Prediction for Non-Exchangeable Panel Data
Pith reviewed 2026-05-19 23:15 UTC · model grok-4.3
The pith
Online conformal prediction delivers stepwise and long-run coverage bounds for panel data by using contemporaneous outcomes from related units as calibration.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
When a forecast is needed for a target unit, the method treats currently observed outcomes from related units as a calibration panel and constructs prediction sets using two adaptive quantities: history-based similarity weights that favor units resembling the target and an adaptive miscoverage level that is revised whenever target feedback is revealed. This two-state design produces a stepwise coverage bound at each round together with a long-run coverage guarantee, and empirical tests on synthetic and real panels show improved coverage on the worst-covered units through adaptive rather than uniform interval widening.
What carries the argument
The two-state online design that pairs history-based similarity weights with an adaptive miscoverage level updated on target feedback, using contemporaneous outcomes from related units as the calibration panel.
If this is right
- Each prediction set satisfies a finite-sample coverage bound that holds at the individual time step.
- The long-run average coverage converges to the target level once sufficient target feedback has accumulated.
- Coverage for the worst-performing target units improves by allocating wider intervals adaptively instead of widening every set by the same amount.
- Similarity weights alone maintain protection when direct feedback from the target unit remains sparse or delayed.
- The adaptive miscoverage level tightens the sets further once direct observations of the target become available.
Where Pith is reading between the lines
- The same weighting-plus-adaptation structure could be tested on multivariate time series where related series serve as the calibration panel.
- Resource-constrained forecasting systems might use the similarity weights to decide which units to monitor more closely.
- Applying the method to panels that experience known structural breaks would test whether the weights adjust quickly enough to preserve the bounds.
- Pairing the framework with black-box predictors would check whether the coverage guarantees survive when the underlying model is itself learned online.
Load-bearing premise
Contemporaneous outcomes from related units can serve as a valid calibration panel for the target unit under temporal dependence and unit heterogeneity.
What would settle it
A simulation in which units exhibit strong temporal dependence and no measurable similarity, yet the realized coverage rate stays below the claimed stepwise bound across many rounds, would show the guarantee fails to hold.
Figures
read the original abstract
Panel data, in which multiple units are repeatedly observed over time, arise throughout science and engineering. Quantifying predictive uncertainty in such settings is challenging because conformal prediction, while distribution-free and model-agnostic, classically relies on exchangeability assumptions that fail under temporal dependence and unit heterogeneity. We propose a simple online conformal framework for non-exchangeable panel data. The method exploits a key feature of online panel prediction: when a forecast is required for one unit, contemporaneous outcomes from related units may already be observed and can serve as a calibration panel. At each round, prediction sets are formed using currently observed calibration units together with two adaptive quantities: history-based similarity weights that emphasize calibration units resembling the target, and an adaptive miscoverage level that is updated whenever target feedback is revealed. This two-state design yields a stepwise coverage bound and a long-run coverage guarantee. Empirically, across synthetic and real panel data sets, the method improves coverage on the worst-covered target units through adaptive interval-width allocation rather than uniform inflation. The two states are complementary: similarity weights protect coverage when target feedback is sparse, while the adaptive level further improves coverage as feedback accumulates.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes an online conformal prediction framework for non-exchangeable panel data arising from temporal dependence and unit heterogeneity. At each round, prediction sets for a target unit are constructed from contemporaneous outcomes of related units serving as a calibration panel, combined with two adaptive components: history-based similarity weights that emphasize resembling calibration units, and an adaptive miscoverage level updated only upon target feedback. The central claim is that this two-state design produces a stepwise coverage bound together with a long-run coverage guarantee. Empirical results across synthetic and real panel datasets indicate that the approach improves coverage for the worst-covered target units through adaptive interval-width allocation rather than uniform inflation.
Significance. If the coverage bounds hold under the paper's conditions, the work addresses a practically important gap by extending distribution-free conformal prediction to panel settings without requiring exchangeability. The complementary roles of the two states—similarity weights protecting coverage under sparse target feedback and the adaptive level improving it as feedback accumulates—represent a pragmatic strength. The empirical focus on worst-case units rather than average performance is a clear positive, as is the model-agnostic and online nature of the procedure.
major comments (2)
- [Abstract] Abstract and method description: the stepwise coverage bound is asserted to follow from the weighted calibration set formed by contemporaneous related units, yet the derivation does not explicitly establish that this set satisfies conditional exchangeability or a martingale property relative to the target unit's outcome once cross-unit temporal dependence or heterogeneity is present. If the similarity weights are heuristic (e.g., inverse-distance on past covariates), the empirical quantile need not dominate the target's conditional quantile, which is load-bearing for the first-step coverage claim.
- [Abstract] Abstract: the long-run coverage guarantee is stated without visible mixing, weak-dependence, or ergodicity conditions on the panel process. The skeptic concern that unstated assumptions on inter-unit dependence may be required is therefore a correctness-risk issue that must be resolved by either adding the precise conditions or showing that the adaptive update alone suffices.
minor comments (1)
- [Abstract] The abstract would benefit from a single sentence clarifying the exact functional form of the adaptive miscoverage level and how it is updated solely on target feedback.
Simulated Author's Rebuttal
We thank the referee for the careful and constructive review. The comments help us clarify the theoretical foundations of the stepwise and long-run guarantees. We respond to each major comment below and indicate the revisions we will make.
read point-by-point responses
-
Referee: [Abstract] Abstract and method description: the stepwise coverage bound is asserted to follow from the weighted calibration set formed by contemporaneous related units, yet the derivation does not explicitly establish that this set satisfies conditional exchangeability or a martingale property relative to the target unit's outcome once cross-unit temporal dependence or heterogeneity is present. If the similarity weights are heuristic (e.g., inverse-distance on past covariates), the empirical quantile need not dominate the target's conditional quantile, which is load-bearing for the first-step coverage claim.
Authors: We agree that the current derivation of the stepwise coverage bound would benefit from greater explicitness. The similarity weights are computed exclusively from historical data and are therefore measurable with respect to the filtration available at prediction time. The contemporaneous calibration outcomes then enter the weighted quantile. In the revision we will insert a supporting lemma (in the appendix) that formalizes the relevant martingale property: the weighted empirical quantile dominates the target non-conformity score in conditional probability, given the past, under the maintained panel-data assumptions. We will also state explicitly that the bound is marginal with respect to the conditional law of the target given history, and we will note the approximation error incurred when the weights are heuristic. These additions directly address the concern about conditional exchangeability and the role of the weights. revision: yes
-
Referee: [Abstract] Abstract: the long-run coverage guarantee is stated without visible mixing, weak-dependence, or ergodicity conditions on the panel process. The skeptic concern that unstated assumptions on inter-unit dependence may be required is therefore a correctness-risk issue that must be resolved by either adding the precise conditions or showing that the adaptive update alone suffices.
Authors: The long-run coverage guarantee is produced entirely by the adaptive miscoverage level, which is updated only when target feedback is received. This online adjustment controls the cumulative coverage error for the target unit alone and yields the desired long-run average by a standard supermartingale or counting argument; the argument does not invoke any mixing, weak-dependence, or ergodicity properties of the panel process. Inter-unit dependence is already accounted for in the finite-sample stepwise bounds via the similarity weights; it does not enter the long-run statement. In the revision we will restate the long-run theorem to make this separation of roles explicit and to confirm that no additional process-level conditions are required. revision: yes
Circularity Check
No circularity: coverage guarantees derived from adapted conformal arguments on panel structure
full rationale
The abstract and method description define history-based similarity weights and an adaptive miscoverage level as new quantities computed from observed data and target feedback. The stepwise coverage bound and long-run guarantee are presented as consequences of the two-state design applied to contemporaneous calibration units. No equations or self-citations are shown that reduce the bound to a fitted parameter or prior result by construction. The derivation remains independent of the target result itself and does not invoke uniqueness theorems or ansatzes from the authors' prior work. This is the standard case of an honest non-finding.
Axiom & Free-Parameter Ledger
Lean theorems connected to this paper
-
IndisputableMonolith/Cost/FunctionalEquation.leanwashburn_uniqueness_aczel unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
W-TQA maintains two online states... history-based similarity weights... adaptive nominal miscoverage level... weighted conformal threshold
-
IndisputableMonolith/Foundation/AbsoluteFloorClosure.leanreality_from_one_distinction unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
Theorem 5.1 (Stepwise weighted conformal bound) ... Δ_t := sum w_k dTV(P_t, Pswap_t,k)
What do these tags mean?
- matches
- The paper's claim is directly supported by a theorem in the formal canon.
- supports
- The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
- extends
- The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
- uses
- The paper appears to rely on the theorem as machinery.
- contradicts
- The paper's claim conflicts with a theorem or certificate in the canon.
- unclear
- Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.
Reference graph
Works this paper leans on
-
[1]
Anastasios Angelopoulos, Emmanuel Candes, and Ryan J Tibshirani. Conformal pid control for time series prediction.Advances in Neural Information Processing Systems, 36:23047–23074, 2023
work page 2023
-
[2]
Conformal prediction: A gentle introduction
Anastasios N Angelopoulos and Stephen Bates. Conformal prediction: A gentle introduction. Foundations and Trends in Machine Learning, 16(4):494–591, 2023
work page 2023
-
[3]
Online conformal prediction with decaying step sizes
Anastasios N Angelopoulos, Rina Foygel Barber, and Stephen Bates. Online conformal prediction with decaying step sizes. InInternational Conference on Machine Learning, pages 1616–1630. PMLR, 2024
work page 2024
-
[4]
Online conformal prediction via online optimization
Felipe Areces, Christopher Mohri, Tatsunori Hashimoto, and John Duchi. Online conformal prediction via online optimization. InInternational Conference on Machine Learning, pages 1604–1649. PMLR, 2025
work page 2025
-
[5]
Andreas Auer, Martin Gauch, Daniel Klotz, and Sepp Hochreiter. Conformal prediction for time series with modern hopfield networks.Advances in Neural Information Processing Systems, 36: 56027–56074, 2023
work page 2023
-
[6]
Unifying different theories of conformal prediction
Rina Foygel Barber and Ryan J Tibshirani. Unifying different theories of conformal prediction. Electronic Journal of Statistics, 20(1):1428–1474, 2026
work page 2026
-
[7]
Conformal prediction beyond exchangeability.The Annals of Statistics, 51(2):816–845, 2023
Rina Foygel Barber, Emmanuel J Candes, Aaditya Ramdas, and Ryan J Tibshirani. Conformal prediction beyond exchangeability.The Annals of Statistics, 51(2):816–845, 2023
work page 2023
-
[8]
Conformal predictions for longitudinal data.arXiv preprint arXiv:2310.02863, 2023
Devesh Batra, Salvatore Mercuri, and Raad Khraishi. Conformal predictions for longitudinal data.arXiv preprint arXiv:2310.02863, 2023
-
[9]
Improved online conformal prediction via strongly adaptive online learning
Aadyot Bhatnagar, Huan Wang, Caiming Xiong, and Yu Bai. Improved online conformal prediction via strongly adaptive online learning. InInternational Conference on Machine Learning, pages 2337–2363. PMLR, 2023
work page 2023
-
[10]
Tarun Chordia and Bhaskaran Swaminathan. Trading volume and cross-autocorrelations in stock returns.The Journal of Finance, 55(2):913–935, 2000
work page 2000
-
[11]
The total variation distance between high-dimensional gaussians with the same mean,
Luc Devroye, Abbas Mehrabian, and Tommy Reddad. The total variation distance between high-dimensional gaussians with the same mean.arXiv preprint arXiv:1810.08693, 2018
-
[12]
Robin Dunn, Larry Wasserman, and Aaditya Ramdas. Distribution-free prediction sets for two-layer hierarchical models.Journal of the American Statistical Association, 118(544): 2491–2502, 2023
work page 2023
-
[13]
HF Data Library: High-Frequency U.S
Ahmed Elkassabgi. HF Data Library: High-Frequency U.S. Equity Data (1-Minute OHLCV). Zenodo dataset, version 1.0, 2026. URLhttps://doi.org/10.5281/zenodo.19501605
-
[14]
Cheol S Eun and Sangdal Shim. International transmission of stock market movements.Journal of Financial and Quantitative Analysis, 24(2):241–256, 1989
work page 1989
-
[15]
António Farinhas, Chrysoula Zerva, Dennis Ulmer, and André F. T. Martins. Non-exchangeable conformal risk control. InThe Twelfth International Conference on Learning Representations, 2024
work page 2024
-
[16]
Retail forecasting: Research and practice
Robert Fildes, Shaohui Ma, and Stephan Kolassa. Retail forecasting: Research and practice. International Journal of Forecasting, 38(4):1283–1318, 2022
work page 2022
-
[17]
Stochastic online conformal prediction with semi-bandit feedback
Haosen Ge, Hamsa Bastani, and Osbert Bastani. Stochastic online conformal prediction with semi-bandit feedback. InInternational Conference on Machine Learning, pages 18954–18965. PMLR, 2025
work page 2025
-
[18]
Adaptive conformal inference under distribution shift
Isaac Gibbs and Emmanuel Candes. Adaptive conformal inference under distribution shift. In Advances in Neural Information Processing Systems, volume 34, pages 1660–1672, 2021. 11
work page 2021
-
[19]
Isaac Gibbs and Emmanuel J Candès. Conformal inference for online prediction with arbitrary distribution shifts.Journal of Machine Learning Research, 25(162):1–36, 2024
work page 2024
-
[20]
Leying Guan. Localized conformal prediction: A generalized inference framework for confor- mal prediction.Biometrika, 110(1):33–50, 2023
work page 2023
-
[21]
Yasushi Hamao, Ronald W Masulis, and Victor Ng. Correlations in price changes and volatility across international stock markets.The Review of Financial Studies, 3(2):281–307, 1990
work page 1990
-
[22]
Rohan Hore and Rina Foygel Barber. Conformal prediction with local weights: randomization enables local guarantees.arXiv preprint arXiv:2310.07850, 2023
-
[23]
Kewei Hou and Tobias J Moskowitz. Market frictions, price delay, and the cross-section of expected returns.The Review of Financial Studies, 18(3):981–1020, 2005
work page 2005
-
[24]
Lisa Kessler, Felix Rempe, and Klaus Bogenberger. Multi-sensor data fusion for accurate traffic speed and travel time reconstruction.Frontiers in Future Transportation, 2:766951, 2021
work page 2021
-
[25]
Kernel-based optimally weighted conformal time-series prediction
Jonghyeok Lee, Chen Xu, and Yao Xie. Kernel-based optimally weighted conformal time-series prediction. InThe Thirteenth International Conference on Learning Representations, 2025
work page 2025
-
[26]
Jing Lei, Max G’Sell, Alessandro Rinaldo, Ryan J Tibshirani, and Larry Wasserman. Distribution-free predictive inference for regression.Journal of the American Statistical Associ- ation, 113(523):1094–1111, 2018
work page 2018
-
[27]
Ang Li, William H K Lam, Mei Lam Tam, Ren Xin Zhong, and Wei Ma. Prediction of travel time on urban road links with and without point detectors.Asian Transport Studies, 8:100081, 2022
work page 2022
-
[28]
Zhen Lin, Shubhendu Trivedi, and Jimeng Sun. Conformal prediction with temporal quantile adjustments.Advances in Neural Information Processing Systems, 35:31017–31030, 2022
work page 2022
-
[29]
Spyros Makridakis, Evangelos Spiliotis, and Vassilios Assimakopoulos. The m5 competition: Background, organization, and implementation.International Journal of Forecasting, 38(4): 1325–1336, 2022
work page 2022
-
[30]
Inductive confidence machines for regression
Harris Papadopoulos, Kostas Proedrou, Vladimir V ovk, and Alex Gammerman. Inductive confidence machines for regression. InEuropean Conference on Machine Learning, pages 345–356. Springer, 2002
work page 2002
-
[31]
Distribution-free uncertainty quantification for classification under label shift
Aleksandr Podkopaev and Aaditya Ramdas. Distribution-free uncertainty quantification for classification under label shift. InUncertainty in Artificial Intelligence, pages 844–853. PMLR, 2021
work page 2021
-
[32]
Kamile Stankeviciute, Ahmed M Alaa, and Mihaela Van der Schaar. Conformal time-series forecasting.Advances in Neural Information Processing Systems, 34:6216–6228, 2021
work page 2021
-
[33]
Ryan J Tibshirani, Rina Foygel Barber, Emmanuel Candes, and Aaditya Ramdas. Conformal prediction under covariate shift.Advances in Neural Information Processing Systems, 32, 2019
work page 2019
-
[34]
Cambridge university press, 2018
Roman Vershynin.High-dimensional probability: An introduction with applications in data science, volume 47. Cambridge university press, 2018
work page 2018
-
[35]
Vladimir V ovk, Alexander Gammerman, and Glenn Shafer.Algorithmic learning in a random world. Springer, 2005
work page 2005
-
[36]
Mirror online conformal prediction with intermittent feedback.IEEE Signal Processing Letters, 2025
Bowen Wang, Matteo Zecchin, and Osvaldo Simeone. Mirror online conformal prediction with intermittent feedback.IEEE Signal Processing Letters, 2025
work page 2025
-
[37]
Conformal prediction interval for dynamic time-series
Chen Xu and Yao Xie. Conformal prediction interval for dynamic time-series. InInternational Conference on Machine Learning, pages 11559–11569. PMLR, 2021
work page 2021
-
[38]
Sequential predictive conformal inference for time series
Chen Xu and Yao Xie. Sequential predictive conformal inference for time series. InInternational Conference on Machine Learning, pages 38707–38727. PMLR, 2023. 12
work page 2023
-
[39]
Online Conformal Prediction with Adversarial Semi-bandit Feedback via Regret Minimization
Junyoung Yang, Kwang-Sung Kim, and Sangdon Park. Online conformal prediction with adversarial semi-bandit feedback via regret minimization.arXiv preprint arXiv:2604.17984, 2026
work page internal anchor Pith review Pith/arXiv arXiv 2026
-
[40]
Adaptive conformal predictions for time series
Margaux Zaffran, Olivier Feron, Yannig Goude, Julie Josse, and Aymeric Dieuleveut. Adaptive conformal predictions for time series. InInternational Conference on Machine Learning, pages 25834–25866. PMLR, 2022
work page 2022
-
[41]
Conformalized interactive imitation learning: Handling expert shift and intermittent feedback
Michelle D Zhao, Reid Simmons, Henny Admoni, Aaditya Ramdas, and Andrea Bajcsy. Conformalized interactive imitation learning: Handling expert shift and intermittent feedback. InThe Thirteenth International Conference on Learning Representations, 2025. 13 A Additional Problem Context A.1 Motivating examples Financial panels.Our primary motivation is financ...
work page 2025
-
[42]
conditional on (Ft,F + t−1), the unit-specific featuresZ1,t, . . . , ZN+1,t are independent across units and satisfy Zj,t ∼ N(µ j,Σ Z), whereµ j ∈R d1 andΣ Z ∈R d1×d1 is a common positive semidefinite covariance matrix
-
[43]
, ZN+1,t , Ft)conditional onF + t−1; 20
the idiosyncratic noises satisfy εj,t ∼ N(0, σ 2 ε), independently acrossj, and independently of(Z 1,t, . . . , ZN+1,t , Ft)conditional onF + t−1; 20
-
[44]
the baseline component is correctly specified, ˜g0 =g 0, and the loading error is linear: there exists a matrixB∈R d2×d1 such that, for everyz∈R d1, gF (z)−˜gF (z) =Bz. Under Assumption B.4, the fitted model is explicit and the pre-absolute residual is Uj,t :=Y j,t − ˆf( ¯Xj,t) =F ⊤ t BZj,t +ε j,t,ˆs j,t =|U j,t|. The common factor can induce cross-sectio...
-
[45]
using the same ridge point predictor, calibration split, and symmetric interval format as the other conformal methods. TQA-B maintains exponentially decayed mean absolute residuals with decay 0.8, predicts the target’s residual rank from this history, and applies the budgeting map to choose the queried miscoverage level αt. We then clip this TQA-B level t...
work page 1950
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.