Pith. sign in

REVIEW 3 major objections 5 minor 47 references

Parametric Scaling Law of Tuning Bias in Conformal Prediction

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Using the same data for tuning and calibration in conformal prediction introduces a tuning bias bounded by a parametric scaling law: it grows with parameter-space complexity and shrinks with calibration-set size.

desk verdict Empirically plausible and the finite-parameter bound is fine, but the infinite-parameter scaling law rests on a false VC-dimension lemma and on a gap between coverage-gap bounds and actual tuning bias. read the letter →

arxiv 2502.03023 v2 pith:P2BPKNWT submitted 2025-02-05 cs.LG math.STstat.MEstat.TH

Hao Zeng , Kangdao Liu , Bingyi Jing , Hongxin Wei This is my paper · ORCID
classification cs.LGmath.STstat.MEstat.TH MSC 62G1562G2068T05
keywords conformalpredictiontuningbiasexchangeabilitycoveragegapempiricalprocessVCdimensionparametricscalinglawconfidencecalibration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks when it is safe to use the same dataset for tuning and calibration in conformal prediction, a practice that breaks the exchangeability assumption behind coverage guarantees. Its central claim is that the resulting coverage gap, called the tuning bias, obeys a parametric scaling law: it grows with the complexity of the parameter space and shrinks with the size of the calibration set. For a finite set of $|\Lambda|$ parameter choices the paper proves a bound of order $\sqrt{\log(2|\Lambda|)/(2n)}$, and for a $d$-dimensional parameter space a bound of order $\sqrt{(d+1)/n}$. If the claim holds, practitioners in data-scarce settings can reuse one dataset for simple tuning at small coverage cost, and the main danger is high-dimensional parameter spaces such as vector scaling or full-layer fine-tuning.

What carries the argument

The load-bearing object is the threshold class $H_\Lambda = \{1\{S_\lambda(x,y)\le t\}: \lambda\in\Lambda,\ t\in\mathbb R\}$, the indicator functions of events that a tuned score falls below a threshold. Tuning bias is bounded by the expected supremum of the empirical process indexed by this class, $E\sup_{g\in H_\Lambda} |(1/n)\sum_i g(x_i,y_i) - E[g(x_{\text{test}},y_{\text{test}})\mid D_{\text{cal}}]|$. The finite-parameter case is controlled by the Dvoretzky–Kiefer–Wolfowitz inequality plus a union bound over $|\Lambda|$; the infinite-parameter case is controlled by the VC dimension of $H_\Lambda$, with the paper claiming $\mathrm{VC}(H_\Lambda)\le d+1$ for $\Lambda\subset\mathbb R^d$ under continuity and boundedness. The same machinery yields corollaries for RAPS, score selection, temperature versus vector scaling, and order-preserving regularization, which shrinks the effective parameter space.

What would settle it

Find a score function $S_\lambda(x,y)$ that is continuous and bounded in $\lambda\in\mathbb R$ and three fixed samples whose eight label patterns are all realized by varying $\lambda$ and the threshold $t$ (for instance $S_\lambda(x_i)=\cos(a_i\lambda)$ with well-separated frequencies $a_i$). Exhibiting such a triple refutes the lemma behind the $\sqrt{(d+1)/n}$ bound, so the infinite-parameter scaling law would not follow from the proof as written.

Watch

Extended reading notes

Core claim

The discovery is a formal quantification of tuning bias: when the tuning parameter $\hat\lambda$ is picked and the threshold $\hat t$ is calibrated on the same i.i.d. sample, the extra coverage gap is bounded by the supremum of empirical-process fluctuations over the class $H_\Lambda=\{1\{S_\lambda(x,y)\le t\}:\lambda\in\Lambda,\ t\in\mathbb R\}$. For finite $\Lambda$, a union bound with the Dvoretzky–Kiefer–Wolfowitz inequality gives $\text{TuningBias}(C) \le \sqrt{\log(2|\Lambda|)/(2n)} + 1/(\sqrt{2n}\sqrt{\log(2|\Lambda|)})$. For infinite $\Lambda\subset\mathbb R^d$, a VC-dimension bound gives $\text{TuningBias}(C) \le C\sqrt{(d+1)/n}$. The proof identifies parameter-space complexity, not the number of parameters per se, as the driver of tuning bias, and the applications show that temperature scaling has a smaller bound than vector scaling, with zero tuning bias in binary classification. The authors take these bounds as rigorous support for the scaling law they observe empirically across RAPS, SAPS, score aggregation, C-Adapter, and confidence calibration methods.

Load-bearing premise

The infinite-parameter bound relies on the assumption that a continuous family of score functions cannot fit arbitrary label patterns on more than $d+1$ points; oscillatory families can violate this, and the paper's proof of this assumption is not valid in general.

Editorial extensions

If this is right

  • Same-set tuning is safe only when the parameter space is simple: the bound is practically small for a handful of candidates and thousands of calibration points.
  • High-dimensional tuning methods, such as vector scaling with $2K$ parameters, carry a coverage penalty that grows like $\sqrt{(d+1)/n}$, matching the empirical failures the paper reports.
  • Temperature scaling dominates vector scaling in tuning bias; in binary classification temperature scaling has exactly zero tuning bias, so no hold-out split is needed for that case.
  • Order-preserving regularization reduces the effective parameter dimension (from $2K$ to $2$ for vector scaling, and from $K^2+K$ to $K+2$ for matrix scaling) and therefore lowers the tuning bias bound.
  • Increasing calibration size $n$ reduces the bias at rate $1/\sqrt n$; the settings where data reuse is most tempting are exactly the settings where the bound is largest.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Implicit consequence: the proved bounds are uniform over score distributions and likely loose for typical neural-network scores, so a data-dependent or Rademacher-style bound could show same-set reuse is safe in even more cases, or reveal settings where the empirical process is larger than $\sqrt{d/n}$.
  • Testable design rule: compare the estimated tuning-bias bound for a specific method and calibration size against the coverage slack $\varepsilon_{\alpha,n}$ already tolerated; reuse is justified when the bound is a small fraction of that slack.
  • Testable extension: the theory should extend to early stopping and model selection in regression, which the paper treats only empirically; verifying the same scaling there would test whether the law is universal across conformal prediction pipelines.
  • Computational check: for random low-dimensional score families, search for the largest shattered set; a counterexample to the $d+1$ VC-dimension lemma would confine the scaling law to the finite-parameter regime.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper studies the coverage gap ('tuning bias') incurred when the same dataset is used both to tune parameters of a conformal score function and to calibrate the conformal threshold. The authors report experiments on CIFAR-100 and ImageNet with RAPS, SAPS, score aggregation, temperature scaling, vector scaling, C-Adapter, and ConfTr fine-tuning, observing that tuning bias is small for low-complexity tuning and that it increases with the number of parameters and decreases with calibration-set size. The theoretical part models tuning as constrained ERM and bounds the coverage gap by an empirical process supremum R_Lambda plus a split-conformal term epsilon_alpha,n (Theorem 4.1). Finite parameter spaces are handled by a DKW union bound (Proposition 4.2), and infinite Lambda subset R^d by a VC-dimension bound (Proposition 4.6), leading to Corollary 4.7 comparing temperature scaling with vector scaling and Proposition 5.2 on order-preserving regularization. I find the empirical study informative but the theoretical core contains unsupported and false steps, detailed below.

Significance. If the theoretical claims were correct, the paper would provide a useful justification for reusing calibration data for low-complexity tuning and a first rigorous explanation of the observed scaling behavior. The empirical study is extensive and reproducible (code link is provided), and the finite-Lambda DKW argument is a sensible use of standard inequalities; the exact zero tuning bias for binary temperature scaling (Proposition 5.1) is a nice structural observation. However, the headline theoretical contributions are not established: the infinite-dimensional bound rests on a false VC-dimension lemma, and the comparison corollaries infer an ordering of biases from an ordering of upper bounds. Since these flaws are load-bearing for the claimed parametric scaling law, the paper in its current form does not support its abstract.

major comments (3)
  1. [Section 4.1 (after Theorem 4.1)] The statement 'TuningBias(C) <= ER_Lambda' does not follow from Theorem 4.1. The theorem gives CovGap(C) <= ER_Lambda + epsilon_alpha,n, while TuningBias(C) = CovGap(C) - CovGap(C_holdout). The argument would require CovGap(C_holdout) >= epsilon_alpha,n, but the split-conformal gap is only bounded above by about 1/(n+1) (Theorem 2.2) and can be much smaller; epsilon_alpha,n is an upper-bound constant, not the exact hold-out gap. Consequently Proposition 4.2 and Proposition 4.6 are not derived as stated; at best one obtains TuningBias(C) <= ER_Lambda + epsilon_alpha,n, an additive O(1/n) term. This does not change the asymptotic rate for fixed |Lambda|, but the displayed bounds should be corrected.
  2. [Appendix D, Lemma D.5 (and Lemma D.3)] Lemma D.5 is false as stated. Take Lambda = R and S_lambda(x) = sin(lambda x). For any finite set of rationally independent real numbers x_1,...,x_m (with 1, x_1,...,x_m linearly independent over Q), Kronecker's theorem implies that (lambda x_1 mod 2pi, ..., lambda x_m mod 2pi) is dense in the m-torus. Hence for every sign pattern there exists a lambda with sin(lambda x_i) > 0 on the chosen positive set and sin(lambda x_i) < 0 on the chosen negative set; the threshold t = 0 then realizes the pattern with 1{sin(lambda x_i) <= 0}. Thus H_Lambda shatters m points for every m and VC(H_Lambda) = infinity, although S_lambda is continuous and bounded in lambda. The same counterexample refutes the one-dimensional Lemma D.3, which claims VC(H_Lambda) <= 2. Since Proposition 4.6 is proved as a direct application of Lemmas D.5 and D.6, the bound C sqrt((d+1)/n) is unsupported, and Corollary 4.7 inherits this failure.
  3. [Corollary 4.7 and Proposition 5.2] Even if Proposition 4.6 were valid, Corollary 4.7 does not follow. The proposition supplies upper bounds TuningBias(C_TS) <= C sqrt(2/n) and TuningBias(C_VS) <= C sqrt((2K+1)/n); a smaller upper bound for TS does not imply TuningBias(C_TS) <= TuningBias(C_VS). The same invalid transfer appears in Appendix K in the proof of Proposition 5.2: from H_Lambda,op subset H_Lambda the authors conclude that the empirical process supremum over the smaller class is no larger, 'which means' TuningBias(C_op) <= TuningBias(C). This requires a direct argument about the tuning biases themselves, not merely about their upper bounds. Proposition 5.1's claim 0 <= TuningBias(C_VS) also lacks a lower-bound proof; a finite-sample difference of coverage gaps can be negative.
minor comments (5)
  1. [Appendix B.5 and Figure 5 caption] The Figure 5 caption says the tuning bias of ConfTr (ft.) is 'generally small', while the plotted gaps and the Appendix B.5 text describe it as 'significantly different' from the hold-out setting; these statements should be reconciled.
  2. [Appendix K] Lemma K.3 (the characterization of order-preserving matrix scaling) is stated without a proof; the surrounding text only remarks on it, so either a proof or an explicit conjecture label is needed.
  3. [Section 3.2 and Figure 2] The main text does not clearly state which dataset is used for the scaling-law experiments; Figure 2 caption says ImageNet, whereas Section 3.1 is introduced with CIFAR-100. Please clarify the dataset for each experiment.
  4. [General presentation] There are several typos and corrupted labels, including 'tunig' in Appendix B.2, 'Paramters' in Figure 3d caption, and 'Image/glyph1197et' in Appendix B.1 captions.
  5. [Theorem 4.1] The theorem uses T subset R in the definition of R_Lambda without defining T; specify the admissible threshold set explicitly.

Circularity Check

0 steps flagged · score 2.0 of 10

No load-bearing circularity: the scaling-law bounds are derived from external DKW, Gaussian-tail, and VC-dimension inequalities, with self-citations used only as illustrative examples.

full rationale

The derivation chain is not circular. Theorem 4.1 bounds CovGap(C) by an empirical-process supremum ER_Lambda plus the split-conformal discretization gap eps_alpha,n; Proposition 4.2 then follows from the DKW inequality, a union bound over the finite parameter space, and a Gaussian-tail estimate, all external and un-fitted. Proposition 4.6 is intended to follow from a VC-dimension bound on H_Lambda together with Vershynin's empirical-process inequality (Lemma D.6), again with a universal constant that is not estimated from the experiments. The empirical observations in Section 3 are not used as inputs to these proofs, so the agreement in monotonicity between the bounds and the data is not a reduction of the result to its inputs. The self-citations (Xi et al. 2024; Liu et al. 2025) appear only as examples of confidence-calibration and C-Adapter tuning methods, and not as the source of the main inequalities or as any uniqueness theorem; they are therefore minor and non-load-bearing. The principal weaknesses are correctness gaps rather than circularity: Lemma D.5's assertion that VC(H_Lambda) <= d+1 for every continuous bounded family is not justified by continuity alone, the passage from Theorem 4.1 and Definition 2.3 to TuningBias(C) <= ER_Lambda silently assumes a comparison between CovGap(C_hold-out) and eps_alpha,n, and Corollary 4.7 compares upper bounds rather than actual tuning biases. These are invalid or missing proof steps, not cases where a prediction is equivalent to an input by construction, so they do not raise the circularity score. The limitation statement that the bound 'is not tight without more assumptions on the modelling' further confirms the paper offers an upper bound rather than a fitted identity. Overall, no significant circularity is present; the score of 2 reflects only the presence of minor, non-load-bearing self-citations.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No fitted parameters are introduced; the empirical scaling-law direction matches the upper bounds but no constants are estimated. The main added assumptions are the VC-dimension lemma and the unproved transitivity from bound ordering to quantity ordering.

assumptions (4)
  • domain assumption Samples in Dcal ∪ Dtest are i.i.d.
    Invoked in Theorem 4.1 to identify the coverage gap as ER_Λ plus ε_α,n; standard split-conformal assumption.
  • standard math DKW inequality, Gaussian tail bound, and VC empirical-process bound (Vershynin Theorem 8.3.23)
    Used in proofs of Proposition 4.2 and Lemma D.6 to convert empirical-process suprema into O(sqrt(d/n)) or sqrt(log M/n) bounds.
  • ad hoc to paper For continuous, bounded S_λ, the class H_Λ = {1{S_λ(x,y) ≤ t}: λ ∈ R^d, t ∈ R} has VC dimension at most d+1
    Lemma D.5 is the bridge to Proposition 4.6. Its proof argues by continuity that d+2 points cannot be shattered; this is not a valid general argument and the statement is doubtful for oscillatory score functions. If false, the infinite-parameter scaling law lacks proof.
  • ad hoc to paper The ordering of the empirical-process upper bounds transfers to the ordering of the tuning biases themselves
    Used in Corollary 4.7 and Proposition 5.2 to conclude TuningBias(TS) ≤ TuningBias(VS) and TuningBias(Cop) ≤ TuningBias(C). The paper provides no lower bounds, so this transitivity is not justified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Parametric Scaling Law of Tuning Bias in Conformal Prediction." pith.science (2026). https://pith.science/paper/P2BPKNWT

@misc{pith2026250203023,
  author       = {Pith},
  title        = {Pith review of: Parametric Scaling Law of Tuning Bias in Conformal Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P2BPKNWT}},
  note         = {Machine review of arXiv:2502.03023}
}
read the original abstract

Conformal prediction is a popular framework of uncertainty quantification that constructs prediction sets with coverage guarantees. To uphold the exchangeability assumption, many conformal prediction methods necessitate an additional holdout set for parameter tuning. Yet, the impact of violating this principle on coverage remains underexplored, making it ambiguous in practical applications. In this work, we empirically find that the tuning bias - the coverage gap introduced by leveraging the same dataset for tuning and calibration, is negligible for simple parameter tuning in many conformal prediction methods. In particular, we observe the scaling law of the tuning bias: this bias increases with parameter space complexity and decreases with calibration set size. Formally, we establish a theoretical framework to quantify the tuning bias and provide rigorous proof for the scaling law of the tuning bias by deriving its upper bound. In the end, we discuss how to reduce the tuning bias, guided by the theories we developed.

Figures

Figures reproduced from arXiv: 2502.03023 by the authors.

Figure 1
Figure 1. Tuning biases of various methods in conformal pre￾diction, using ResNet-18 on CIFAR-100 at α = 0.1. The APS is used except for RAPS, SAPS and SA, with a calibration set size of 1000. The “hold-out” and “same” denote that we use separate/same datasets for parameter tuning and calibration, respectively. Tuning bias is the difference between the coverage gaps of models tuned in the setting of hold-out and same. In part… view at source ↗
Figure 2
Figure 2. Parametric scaling law, on (a) the complexity of parameter space and (b) the size of the calibration set, using ResNet-18 on ImageNet using APS. The calibration set size for Figure (a) is 6000. The ‘hold-out’ and ‘same’ denote that we use separate/same datasets for parameter tuning and calibration, respectively. Tuning bias is the difference between the coverage gaps of models tuned in the setting of hold-out and sa… view at source ↗
Figure 3
Figure 3. Additional results for confidence calibration across various datasets and score functions. The coverage gaps are obtained on the conformal prediction with a hold-out set or the same set as calibration set to parameter tuning. Figures 3a and 3b show the tuning bias with temperature scaling, using APS and THR as the score functions, respectively. The coverage gaps are obtained on the conformal prediction with a hold-o… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Additional results for score function tuning, aggregation, and C-Adapter across various datasets and score functions. The coverage gaps are obtained on the conformal prediction with a hold-out set or the same set as calibration set to parameter tuning. Figures 4a and 4…
Figure 5
Figure 5. Figure 5: Additional results for ConfTr(ft.) across various datasets and score functions. The coverage gaps are obtained on the conformal prediction with a hold-out set or the same set as calibration set to parameter tuning. Figures 5a and 5b show the tuning bias with ConfTr(ft.…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 34 canonical work pages

  1. [1]

    N., Bates, S., Jordan, M., and Malik, J

    Angelopoulos, A. N., Bates, S., Jordan, M., and Malik, J. Uncertainty sets for image classifiers using conformal prediction. In International Conference on Learning Representations , 2021

    work page 2021

  2. [2]

    Efficient and differentiable conformal prediction with general function classes

    Bai, Y., Mei, S., Wang, H., Zhou, Y., and Xiong, C. Efficient and differentiable conformal prediction with general function classes. In International Conference on Learning Representations , 2022

    work page 2022

  3. [3]

    F., Cand \`e s, E

    Barber, R. F., Cand \`e s, E. J., Ramdas, A., and Tibshirani, R. J. Conformal prediction beyond exchangeability. The Annals of Statistics, 51: 0 816--845, 2023

    work page 2023

  4. [4]

    On Temperature Scaling and Conformal Prediction of Deep Classifiers

    Dabah, L. and Tirer, T. On temperature scaling and conformal prediction of deep classifiers. arXiv preprint arXiv: 2402.05806, 2024

    work page Pith review arXiv 2024

  5. [5]

    Asymptotic minimax character of the sample distribution function and of the classical multinomial estimator

    Dvoretzky, A., Kiefer, J., and Wolfowitz, J. Asymptotic minimax character of the sample distribution function and of the classical multinomial estimator. The Annals of Mathematical Statistics, 27: 0 642--669, 1956

    work page 1956

  6. [6]

    UTOPIA : Universally trainable optimal prediction intervals aggregation

    Fan, J., Ge, J., and Mukherjee, D. UTOPIA : Universally trainable optimal prediction intervals aggregation. arXiv preprint arXiv: 2306.16549, 2024

    arXiv 2024

  7. [7]

    and Ramdas, A

    Gasparin, M. and Ramdas, A. Conformal online model aggregation. arXiv preprint arXiv: 2403.15527, 2024

    arXiv 2024

  8. [8]

    Optimal aggregation of prediction intervals under unsupervised domain shift

    Ge, J., Mukherjee, D., and Fan, J. Optimal aggregation of prediction intervals under unsupervised domain shift. In Neural Information Processing Systems , volume 37. Curran Associates, Inc., 2024

    work page 2024

Show all 47 references
  1. [9]

    J., and Cand \`e s, E

    Gibbs, I., Cherian, J. J., and Cand \`e s, E. J. Conformal prediction with conditional guarantees. arXiv preprint arXiv: 2305.12616, 2024

    2024 arXiv

  2. [10]

    Guo, C., Pleiss, G., Sun, Y., and Weinberger, K. Q. On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning , pp.\ 1321--1330. PMLR, 2017

    2017

  3. [11]

    K., and Ramdas, A

    Gupta, C., Kuchibhotla, A. K., and Ramdas, A. Nested conformal prediction and quantile out-of-bag ensemble methods. Pattern Recognition, 127: 0 108496, 2022

    2022

  4. [12]

    Guyon, I., Vapnik, V., Boser, B., Bottou, L., and Solla, S. A. Structural risk minimization for character recognition. In Neural Information Processing Systems , volume 4. Curran Associates, Inc., 1991

    1991

  5. [13]

    Deep residual learning for image recognition

    He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pp.\ 770--778, 2016

    2016

  6. [14]

    Conformal prediction for deep classifier via label ranking

    Huang, J., Xi, H., Zhang, L., Yao, H., Qiu, Y., and Wei, H. Conformal prediction for deep classifier via label ranking. In Proceedings of the 41st International Conference on Machine Learning , Proceedings of Machine Learning Research , pp.\ 20331--20347. PMLR, 2024

    2024

  7. [15]

    Uncertainty quantification over graph with conformalized graph neural networks

    Huang, K., Jin, Y., Candes, E., and Leskovec, J. Uncertainty quantification over graph with conformalized graph neural networks. In Neural Information Processing Systems , volume 36, pp.\ 26699--26721. Curran Associates, Inc., 2023

    2023

  8. [16]

    Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.6980, 2017

    2017 arXiv

  9. [17]

    J., and Hassani, H

    Kiyani, S., Pappas, G. J., and Hassani, H. Conformal prediction with learned features. In Proceedings of the 41st International Conference on Machine Learning , 2024 a

    2024

  10. [18]

    J., and Hassani, H

    Kiyani, S., Pappas, G. J., and Hassani, H. Length optimization in conformal prediction. In Neural Information Processing Systems , volume 37. Curran Associates, Inc., 2024 b

    2024

  11. [19]

    Rademacher penalties and structural risk minimization

    Koltchinskii, V. Rademacher penalties and structural risk minimization. IEEE Transactions on Information Theory, 47: 0 1902--1914, 2001

    1902

  12. [20]

    Learning multiple layers of features from tiny images

    Krizhevsky, A. Learning multiple layers of features from tiny images. Master's thesis, University of Toronto, 2009

    2009

  13. [21]

    J., and Wasserman, L

    Lei, J., G'Sell, M., Rinaldo, A., Tibshirani, R. J., and Wasserman, L. Distribution-free predictive inference for regression. Journal of the American Statistical Association, 113: 0 1094--1111, 2018

    2018

  14. [22]

    Liang, R., Zhu, W., and Barber, R. F. Conformal prediction after efficiency-oriented model selection. arXiv preprint arXiv: 2408.07066, 2024

    2024 arXiv

  15. [23]

    Conformal inference is (almost) free for neural networks trained with early stopping

    Liang, Z., Zhou, Y., and Sesia, M. Conformal inference is (almost) free for neural networks trained with early stopping. In Proceedings of the 40th International Conference on Machine Learning , pp.\ 20810--20851. PMLR, 2023

    2023

  16. [24]

    Structural risk minimization of rough set-based classifier

    Liu, J., Bai, M., Jiang, N., and Yu, D. Structural risk minimization of rough set-based classifier. Soft Computing, 24: 0 2049--2066, 2020

    2020

  17. [25]

    M., and Wei, H

    Liu, K., Zeng, H., Huang, J., Zhuang, H., Vong, C. M., and Wei, H. C-adapter: Adapting deep classifiers for efficient conformal prediction sets. In The Thirteenth International Conference on Learning Representations , 2025

    2025

  18. [26]

    and Zhou, Z

    Luo, R. and Zhou, Z. Weighted aggregation of conformity scores for classification. arXiv preprint arXiv: 2407.10230, 2024

    2024 arXiv

  19. [27]

    The tight constant in the Dvoretzky-Kiefer-Wolfowitz inequality

    Massart, P. The tight constant in the Dvoretzky-Kiefer-Wolfowitz inequality. The Annals of Probability, 18: 0 1269--1283, 1990

    1990

  20. [28]

    D., Hassani, H., and Pappas, G

    Noorani, S., Romero, O., Fabbro, N. D., Hassani, H., and Pappas, G. J. Conformal risk minimization with variance reduction. arXiv preprint arXiv: 2411.01696, 2024

    2024 arXiv

  21. [29]

    I., Orenstein, P., Ramos, T., and Romano, J

    Oliveira, R. I., Orenstein, P., Ramos, T., and Romano, J. V. Split conformal prediction and non-exchangeable data. Journal of Machine Learning Research, 25: 0 1--38, 2024

    2024

  22. [30]

    Inductive conformal prediction: Theory and application to neural networks

    Papadopoulos, H. Inductive conformal prediction: Theory and application to neural networks. In Tools in Artificial Intelligence. IntechOpen, 2008

    2008

  23. [31]

    PyTorch : An imperative style, high-performance deep learning library

    Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., K \"o pf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S. PyTorch : An i...

    2019

  24. [32]

    Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods

    Platt, J. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in Large Margin Classifiers, 10: 0 61--74, 1999

    1999

  25. [33]

    SAT : Data-light uncertainty set merging via synthetics, aggregation, and test inversion

    Qin, S., He, J., Gang, B., and Xia, Y. SAT : Data-light uncertainty set merging via synthetics, aggregation, and test inversion. arXiv preprint arXiv: 2410.12201, 2024

    2024

  26. [34]

    Romano, Y., Sesia, M., and Cand \`e s, E. J. Classification with valid and adaptive coverage. arXiv preprint arXiv: 2006.02544, 33: 0 3581--3591, 2020

    2006 arXiv

  27. [35]

    Least ambiguous set-valued classifiers with bounded error levels

    Sadinle, M., Lei, J., and Wasserman, L. Least ambiguous set-valued classifiers with bounded error levels. arXiv preprint arXiv: 1609.00451, 114: 0 223--234, 2019

    2019 arXiv

  28. [36]

    Structural risk minimization over data-dependent hierarchies

    Shawe-Taylor , J., Bartlett, P., Williamson, R., and Anthony, M. Structural risk minimization over data-dependent hierarchies. IEEE Transactions on Information Theory, 44: 0 1926--1940, 1998

    1926

  29. [37]

    D., Cemgil, A

    Stutz, D., Dvijotham, K. D., Cemgil, A. T., and Doucet, A. Learning optimal conformal classifiers. In International Conference on Learning Representations . arXiv, 2022

    2022

  30. [38]

    van der Vaart , A. W. and Wellner, J. A. Weak Convergence and Empirical Processes. Springer Series in Statistics . Springer New York, New York, 1996. ISBN 978-1-4757-2545-2

    1996

  31. [39]

    Principles of risk minimization for learning theory

    Vapnik, V. Principles of risk minimization for learning theory. In Neural Information Processing Systems , volume 4. Curran Associates, Inc., 1991

    1991

  32. [40]

    An overview of statistical learning theory

    Vapnik, V. An overview of statistical learning theory. IEEE Transactions on Neural Networks, 10: 0 988--999, 1999

    1999

  33. [41]

    Vapnik, V. N. and Chervonenkis, A. Ya . On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability & Its Applications, 16: 0 264--280, 1971

    1971

  34. [42]

    High-Dimensional Probability: An Introduction with Applications in Data Science

    Vershynin, R. High-Dimensional Probability: An Introduction with Applications in Data Science. Cambridge Series in Statistical and Probabilistic Mathematics . Cambridge University Press, Cambridge, 1th edition, 2018. ISBN 978-1-108-41519-4. doi:10.1017/9781108231596

    2018 doi

  35. [43]

    Conditional validity of inductive conformal predictors

    Vovk, V. Conditional validity of inductive conformal predictors. arXiv preprint arXiv: 1209.2673, pp.\ 475--490, 2012

    2012 arXiv

  36. [44]

    Algorithmic Learning in a Random World

    Vovk, V., Gammerman, A., and Shafer, G. Algorithmic Learning in a Random World. Springer, New York, 2005. ISBN 978-0-387-00152-4 978-0-387-25061-8

    2005

  37. [45]

    Does confidence calibration improve conformal prediction? arXiv preprint arXiv: 2402.04344, 2024

    Xi, H., Huang, J., Liu, K., Feng, L., and Wei, H. Does confidence calibration improve conformal prediction? arXiv preprint arXiv: 2402.04344, 2024

    2024 arXiv

  38. [46]

    and Kuchibhotla, A

    Yang, Y. and Kuchibhotla, A. K. Selection and aggregation of conformal prediction sets. Journal of the American Statistical Association, Online: 0 1--13, 2024

    2024

  39. [47]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.