REVIEW 4 major objections 5 minor 2 cited by
Locally Adaptive Conformal Inference for Operator Models
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read The paper claims that functionally valued, locally adaptive prediction sets for operator models can be built by localizing a projection-based depth score, with the coverage gap bounded by a sum of input-space distances under local…
desk verdict Genuinely new local-depth score idea, but the reported bands come from an unguaranteed sampler and under-cover on Air Quality; worth reviewing with revisions. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the local $\Phi$-score: for a test function $f_{n+1}$, the score of a residual $r$ is $D^\Phi(r|\hat{P}_{n+1}) = \inf_{\phi\in\Phi} D(\phi(r)| \phi_\#(\hat{P}_{n+1}))$, the worst-case univariate Tukey depth over a family of linear projections (random Gaussian slices, FPCA, or wavelets). The projected distribution $\phi_\#(\hat{P}_{n+1})$ is a locally weighted empirical measure with weights $w_t \propto \exp(-\lambda H(\varphi(f_t), \varphi(\tilde{f}_{n+1})))$ built around a knockoff $\tilde{f}_{n+1} = f_{n+1} + \varepsilon$, which lets the score inherit the local geometry of the residual cloud. Taking the unweighted conformal quantile of these localized scores turns the standard exchangeability argument into a finite-sample coverage statement with the explicit gap bound above. The depth-based worst-case projection is what makes the sets deform in anisotropic regions, while the knockoff centering keeps the coverage argument intact.
What would settle it
Simulate a residual process that violates local exchangeability—e.g., residuals drawn i.i.d. $\mathcal{N}(0,1)$ for inputs in one half of the domain and $\mathcal{N}(10,1)$ in the other half—and measure the functional coverage of the true (unsampled) LSCI sets over many replicates; if coverage falls below $1-\alpha - \frac{2}{n+1}\sum_t d(f_t, f_{n+1})$, Proposition 3.1 is false. Alternatively, on the air-quality data, re-run Algorithm 1 and check whether the sampled EC stays near 0.9; the paper's own Table 3 shows LSCI1 and LSCI3 dropping to about 0.66, exposing the sampler's dependence on independent FPCA coordinates.
Extended reading notes
Core claim
LSCI constructs the conformal set $C_\alpha(f_{n+1}) = \{\hat{\Gamma}_\theta(f_{n+1}) + r : D^\Phi(r|\hat{P}_{n+1}) \geq q_\alpha\}$, where $q_\alpha$ is the $\lfloor\alpha(n+1)\rfloor$-th smallest local $\Phi$-depth score of calibration residuals, and $\hat{P}_{n+1}$ is a kernel-weighted empirical measure centered on a knockoff of the test input. The central theoretical result, Proposition 3.1, bounds the coverage gap by $\frac{2}{n+1}\sum_{t=1}^n d(f_t, f_{n+1})$ under local exchangeability of the residual process, so that when residual laws drift smoothly, the depth-based sets achieve approximately nominal coverage and the bound does not depend on bandwidth or localizer. Empirically, the implicit sets are tighter (lower interval scores) than baselines that localize only the quantile, and they adapt to seasonal and regional structure in weather forecasting.
Load-bearing premise
The finite-sample guarantee stands or falls with the assumption that residuals change smoothly across input space in the local-exchangeability sense; when residuals jump abruptly or the pre-metric is misspecified, the bound is vacuous, and the sampled bands additionally rely on an independence assumption that the air-quality experiment shows can fail.
Editorial extensions
If this is right
- For any operator model, users can obtain function-valued prediction sets with finite-sample coverage control without distributional assumptions, provided residual laws vary smoothly over the input space.
- Localizing the score rather than the quantile yields sets that are tighter in directions of low residual variability, lowering interval scores relative to norm-based conformal baselines on the tested tasks.
- Because the coverage bound is independent of bandwidth and localizer, tuning those choices affects efficiency rather than the worst-case guarantee, so users can tune for tightness while keeping nominal coverage.
- The method extends to coverage-risk control: choosing the sampled band's quantile to meet an expected-coverage or coverage-risk target gives practically calibrated bands, subject to the sampler being faithful.
- The knockoff-centered local score keeps LSCI stable under predictor bias and certain covariate shifts, as shown in the synthetic experiments.
Reading between the lines
- The independence assumption on FPCA coordinates inside the sampler is the weakest link between the proven guarantee and the deployed bands; a copula-based or fully depth-aware sampler should fix the under-coverage seen in the air-quality experiment without changing the implicit set.
- The bound's linear sum over calibration points suggests that in large-$n$ regimes the coverage gap could accumulate even with small per-point distances; a sharper analysis might replace the sum by a local effective sample size and explain why the experiments show near-nominal coverage in practice.
- Nothing in the theory prevents learning the pre-metric $d$ or the feature map $\varphi$ from the calibration data, so the framework could be coupled with learned similarity to handle problems where the natural geometry of the input space is unknown.
- The same local $\Phi$-scoring mechanism could serve as a diagnostic tool: a test input whose local residual cloud is multimodal or highly non-elliptical would flag regions where a pointwise band under-represents uncertainty.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces Local Sliced Conformal Inference (LSCI), a method for constructing function-valued, locally adaptive prediction sets for operator models. LSCI defines a local Φ-depth conformity score from weighted calibration residuals, forms an implicit depth-based conformal region, and proves a finite-sample coverage gap bound under a local exchangeability assumption (Prop. 3.1). The paper evaluates LSCI on synthetic Gaussian-process tasks and on air quality, energy demand, and weather forecasting, reporting functional coverage, expected coverage, coverage risk, band width, and interval score for pointwise bands generated by a local-FPCA sampler.
Significance. If the formal coverage bound is correct, LSCI is a conceptually clean extension of conformal inference to functional outputs with anisotropic residual geometry, and the data-dependent gap bound in Prop. 3.1 is a useful contribution. The paper also provides a broad set of experiments and explicit robustness checks across localization kernels, feature maps, depth notions, and projection families. However, the practical claims currently depend on an unvalidated sampler: the certified guarantee applies to the implicit depth region, while nearly all reported metrics come from sampled pointwise bands, and the paper's own Air Quality results show those bands can substantially under-cover. Closing this gap is necessary before the method can be recommended as a calibrated UQ tool.
major comments (4)
- [Section 3.4, Algorithm 1, and Tables 1-3] The coverage guarantee in Section 3.3 applies only to the implicit region D^Φ_{γ(α)}(f_{n+1}) defined in Eqs. (10)-(11), not to the pointwise bands produced by Algorithm 1. All FC, EC, CR, BW, and IS numbers in Tables 1-3 are nevertheless computed from the sampled bands. Table 3 demonstrates the risk: LSCI1 and LSCI3 on Air Quality report FC near 0.887 but EC of only 0.676 and 0.659 against a nominal 0.9. The rejection step in Algorithm 1 only ensures accepted samples lie inside the depth region; it does not make the accepted sample distribution equal to, or even close to, the conditional law of the residuals, and no theorem controls the empirical quantiles of accepted samples. The paper should either provide a finite-sample guarantee for the sampled bands or clearly separate certified implicit-set metrics from approximate sampler-based metrics and avoid claiming the latter inherit the Prop. 3.1 guarantee.
- [Section 4.1, 'Baseline comparisons'] The sentence 'We enforce the desired guarantee by adjusting the pointwise empirical quantiles of the accepted samples' is ambiguous and potentially circular. If the adjustment uses test targets, then the EC/CR values in Tables 1-3 are not out-of-sample and should be removed. If it uses a calibration fold, the protocol is not specified, and the Air Quality rows show that the adjustment did not achieve EC near 0.9 even on that protocol. Please specify the exact adjustment procedure, including which data are used, how quantiles are modified, and why the reported metrics remain out-of-sample.
- [Section 3.3 and Appendix A.2] Equation (2) defines local exchangeability on an index set T with a pre-metric d : T×T → [0,∞), but Proposition 3.1 states the pre-metric on F×F and the proof applies Eq. (2) to indices {1,...,n+1} while writing d(f_i, f_{π(i)}). Unless the residual process is explicitly assumed to be indexed by feature values and the feature pre-metric is the index pre-metric, the proof does not follow from the local exchangeability condition stated in Section 2.1. Please state this assumption precisely or adjust the notation so that the index set and the feature space are reconciled.
- [Section 3.3 and Section 5] Proposition 3.1 is vacuous unless the residual process is locally exchangeable with respect to the chosen pre-metric d, and the paper gives no guidance for selecting d or for checking the assumption empirically. The bound is data-dependent, but its usefulness depends entirely on this choice; a practitioner who chooses a poor d obtains a vacuous bound. The paper's sensitivity analysis in Figure 2 and Tables 4-5 varies λ, H, φ, Φ, and D, but never varies d. Please add a discussion of how d should be chosen in practice, or provide a diagnostic for the local exchangeability assumption.
minor comments (5)
- [Section 4.1, Figure 2, Table 4] The number of slicing projections is called N in the text of Section 4.1, M in Algorithm 1, and 'components' in Table 2; please unify the notation.
- [Equation (8) and Algorithm 1, line 2] Equation (8) includes w_{n+1} in the normalization of the local empirical measure, but Algorithm 1 normalizes the weights only over the calibration set; please reconcile the two definitions.
- [Section 2.1 and Proposition 3.1] The local exchangeability pre-metric in Eq. (2) is not required to be symmetric, while Prop. 3.1 assumes symmetry; please state the added symmetry assumption explicitly in the proposition statement.
- [Table 3] The 'Supr.' row on Air Quality reports FC=0.000 with EC=0.879 and a very narrow band, which looks pathological; a brief explanation or a removal of that row would improve readability.
- [Section 2.2] The sentence 'This set will also cover at level 1−α' after the depth-score convention should explicitly state that the usual exchangeability argument applies with the lower-order-statistic convention; as written it is not obvious why the same guarantee holds.
Circularity Check
No significant circularity: the coverage bound is a direct consequence of the paper's explicit local-exchangeability assumption, and the sampled-band caveat is an acknowledged limitation rather than a reduction of outputs to inputs.
full rationale
LSCI's central theoretical claim (Eqs. 11–13, Prop. 3.1) is not circular: the coverage bound is a direct application of the paper's explicit local-exchangeability assumption (Eq. 2) to the transposition π_t, combined with the generic non-exchangeable conformal bound of Barber et al. (2023). The bound is conditional on the stated modeling premise and contains no fitted parameter that is later reported as a prediction. The empirical section's use of Algorithm 1 to produce pointwise bands is explicitly flagged in Section 5 as an approximation whose guarantee differs from that of the implicit depth region; this is an acknowledged limitation and a correctness/robustness concern, not a reduction of output to input. The phrase 'we enforce the desired guarantee by adjusting the pointwise empirical quantiles of the accepted samples' appears in the context of tuning on calibration data ('All methods are tuned on the calibration data to achieve their respective conformal guarantees'), so there is no demonstrated fitted-input-called-prediction loop. Self-citations (Harris & Sriver 2024; Harris et al. 2021) are used only for a sampling heuristic and a depth notion, not as the load-bearing justification of the coverage theorem.
Assumptions & free parameters
free parameters (7)
- localization bandwidth lambda =
chosen by cross-validation on a tuning fold; values 0.5, 1, 2 explored
- knockoff scale constant c =
c in (0, 0.05) with sigma^2 = c^2 IQR(f_t)^2
- number of slice projections M (N in experiments) =
32, 128 in practice; 1, 10, 100, 200 in experiments
- number of local FPCA components J =
not specified; M = 32 or 128 used for 1D and 2D tasks
- residual sample size n_s =
50 to 5000 in experiments
- pointwise empirical quantile adjustment for EC/CR =
adjusted to achieve EC or CR near 0.9
- k in k-NN localizer =
500 in main experiments
assumptions (5)
- domain assumption The response satisfies g_t = Gamma(f_t) + r_t, where Gamma is the unknown population operator and residuals (r_t) are a locally exchangeable error process (Eqn. 4, Section 3).
- domain assumption The residual process is locally exchangeable in the input space (F, d): d_TV(Y_A, Y_pi(A)) <= sum d(t, pi(t)) for every finite subset A and injective map pi (Eqn. 2, Section 2.1).
- standard math F, G are subsets of L^2(Omega) with Omega compact, and the target distribution admits unimodal, convex level sets so that Phi-depth central regions reflect location, scale, and shape (Section 3.1).
- domain assumption The knockoff draw f_tilde = f_{n+1} + epsilon with epsilon ~ GP(0, K_sigma) preserves the conformal argument (Hore & Barber 2023).
- domain assumption The weighted empirical measure phi_hat(P_{n+1}) (Eqn. 8) provides stable estimates of the local projected residual law as n grows, so depth quantiles are consistent.
Cite this review
Pith. "Pith review of Locally Adaptive Conformal Inference for Operator Models." pith.science (2026). https://pith.science/paper/5W2QABIN
@misc{pith2026250720975,
author = {Pith},
title = {Pith review of: Locally Adaptive Conformal Inference for Operator Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/5W2QABIN}},
note = {Machine review of arXiv:2507.20975}
}
read the original abstract
Operator models are regression algorithms between Banach spaces of functions. They have become an increasingly critical tool for spatiotemporal forecasting and physics emulation, especially in high-stakes scenarios where robust, calibrated uncertainty quantification is required. We introduce Local Sliced Conformal Inference (LSCI), a distribution-free framework for generating function-valued, locally adaptive prediction sets for operator models. We prove finite-sample validity and derive a data-dependent upper bound on the coverage gap under local exchangeability. On synthetic Gaussian-process tasks and real applications (air quality monitoring, energy demand forecasting, and weather prediction), LSCI yields tighter sets with stronger adaptivity compared to conformal baselines. We also empirically demonstrate robustness against biased predictions and certain out-of-distribution noise regimes.
Figures
Forward citations
Cited by 2 Pith papers
-
Flow-Based Conformal Predictive Distributions
Differentiable nonconformity scores induce flows that sample conformal prediction set boundaries, and mixing flows across levels produces conformal predictive distributions whose quantiles match the sets.
-
Beyond Predicting Responses: Conformal Inference for Latent Distributional Parameters
LatentCP builds finite-sample-valid uncertainty sets for instance-specific latent distributional parameters by inverting a conformal response set through a known forward model, with an e-value based multilevel aggrega...
Reference graph
Works this paper leans on
-
[1]
Sincedis symmetric, this becomes dTV(R,R (t))≤2d(f t,fn+1)
Hence dTV(R,R (t))≤d(f t,fn+1) +d(fn+1,ft). Sincedis symmetric, this becomes dTV(R,R (t))≤2d(f t,fn+1). Using the unweighted conformal rank averaging weightsat = 1/(n+ 1)therefore gives ∆n = n∑ t=1 1 n+ 1dTV(R,R (t))≤ 2 n+ 1 n∑ t=1 d(ft,fn+1). A.3 Simulation details MetricsWe report the following metrics. Let Bi(u) = [Li(u),Ui(u)]denote a prediction band ...
work page 2024
-
[2]
= (v1,v 2)}, with isotropic length-scalesℓf = 0.15andℓ g = 0.08and jitterλ= 10 −6. Both processes use time-varyinglocalscales of the form σf t (u) =σ f [ 1 +αfκ ( u−c(t) w )] , σ g t (u) =σ g [ 1 +αgκ ( u−c(t) w )] , where σf = 0.35, σg = 0.40are base levels, αf,αg > 0set the contrast, w = (0.06, 0.06)is the (axis-wise) width, andκ is a smooth, nonnegativ...
work page 2000
-
[10]
Emilia Magnani, Nicholas Krämer, Runa Eschenhagen, Lorenzo Rosasco, and Philipp Hennig. Approximate bayesian neural operators: Uncertainty quantification for parametric pdes.arXiv preprint arXiv:2208.01565,
-
[11]
Linearization Turns Neural Operators into Function-Valued Gaussian Processes
Emilia Magnani, Marvin Pförtner, Tobias Weber, and Philipp Hennig. Linearization turns neural operators into function-valued gaussian processes.arXiv preprint arXiv:2406.05072,
-
[12]
Amirhossein Mollaali, Gabriel Zufferey, Gonzalo Constante-Flores, Christian Moya, Can Li, Guang Lin, and Meng Yue. Conformalized prediction of post-fault voltage trajectories using pre-trained and finetuned attention-driven neural operators.arXiv preprint arXiv:2410.24162,
-
[14]
Christian Moya, Amirhossein Mollaali, Zecheng Zhang, Lu Lu, and Guang Lin
doi: 10.1038/s41467-023-38874-y. Christian Moya, Amirhossein Mollaali, Zecheng Zhang, Lu Lu, and Guang Lin. Conformalized-deeponet: A distribution-free framework for uncertainty quantification in deep operator networks.Physica D: Nonlinear Phenomena, 471:134418,
-
[15]
14 Jaideep Pathak, Shashank Subramanian, Peter Harrington, Sanjeev Raja, Ashesh Chattopadhyay, Morteza Mardani, Thorsten Kurth, David Hall, Zongyi Li, Kamyar Azizzadenesheli, et al. Fourcastnet: A global data-driven high-resolution weather model using adaptive fourier neural operators.arXiv preprint arXiv:2202.11214,
-
[17]
Bayesian deep operator learning for homogenized to fine-scale maps for multiscale PDE
Zecheng Zhang, Christian Moya, Wing Tat Leung, Guang Lin, and Hayden Schaeffer. Bayesian deep operator learning for homogenized to fine-scale maps for multiscale pde.arXiv preprint arXiv:2308.14188,
Show all 20 references
-
[2000]
A Appendix A.1 Local Exchangeability Let( Yt)t∈T denote a stochastic process onR with finite first and second moments.(Yt)t∈T is exchangeable if (Yt)t∈T =D (Yt)π(t)∈T, for all injective mapsπ :T →T , i.e., for all permutations of the indexing set Campbell et al. (2019). Exchan...
2019
-
[2004]
Probabilisticneuraloperatorsforfunctionaluncertainty quantification.arXiv preprint arXiv:2502.12902,
ChristopherBülte, PhilippScholl, andGittaKutyniok. Probabilisticneuraloperatorsforfunctionaluncertainty quantification.arXiv preprint arXiv:2502.12902,
-
[2009]
Calibrated uncertainty quantification for operator learning via conformal prediction.arXiv preprint arXiv:2402.01960,
Ziqi Ma, Kamyar Azizzadenesheli, and Anima Anandkumar. Calibrated uncertainty quantification for operator learning via conformal prediction.arXiv preprint arXiv:2402.01960,
-
[2013]
General notions of depth for functional data.arXiv preprint arXiv:1208.1981,
Karl Mosler and Yulia Polyakova. General notions of depth for functional data.arXiv preprint arXiv:1208.1981,
1981 arXiv
-
[2015]
Fourier neural operator for parametric partial differential equations.arXiv preprint arXiv:2010.08895,
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations.arXiv preprint arXiv:2010.08895,
2010 arXiv
-
[2019]
Scientific machine learning for closure models in multiscale problems: A review.arXiv preprint arXiv:2403.02913,
Benjamin Sanderse, Panos Stinis, Romit Maulik, and Shady E Ahmed. Scientific machine learning for closure models in multiscale problems: A review.arXiv preprint arXiv:2403.02913,
-
[2020]
Rohan Hore and Rina Foygel Barber
doi: 10.1002/qj.3803. Rohan Hore and Rina Foygel Barber. Conformal prediction with local weights: randomization enables local guarantees.arXiv preprint arXiv:2310.07850,
-
[2021]
The ERA5 global reanalysis.Quarterly Journal of the Royal Meteorological Society, 146(730):1999–2049,
13 Hans Hersbach, Bill Bell, Paul Berrisford, Shoji Hirahara, András Horányi, Joaquín Muñoz-Sabater, Julien Nicolas, Céline Peubey, Raluca Radu, Dinand Schepers, Adrian Simmons, Cosimo Soci, Saleh Abdalla, Xavier Abellan, Gianpaolo Balsamo, Peter Bechtold, Gionata Biavati, Jea...
1999
-
[2022]
Variational bayes deep operator network: a data-driven bayesian solver for parametric differential equations.arXiv preprint arXiv:2206.05655,
Shailesh Garg and Souvik Chakraborty. Variational bayes deep operator network: a data-driven bayesian solver for parametric differential equations.arXiv preprint arXiv:2206.05655,
-
[2023]
Quantifying uncertainty in climate projections with conformal ensembles
Trevor Harris and Ryan Sriver. Quantifying uncertainty in climate projections with conformal ensembles. arXiv preprint arXiv:2408.06642,
-
[2024]
Baiting Chen, Zhimei Ren, and Lu Cheng
doi: 10.1038/s41612-024-00638-w. Baiting Chen, Zhimei Ren, and Lu Cheng. Conformalized time series with semantic features.Advances in Neural Information Processing Systems, 37:121449–121474,
-
[2025]
Local exchange- ability.arXiv preprint arXiv:1906.09507,
Trevor Campbell, Saifuddin Syed, Chiao-Yu Yang, Michael I Jordan, and Tamara Broderick. Local exchange- ability.arXiv preprint arXiv:1906.09507,
1906 arXiv
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.