Pith. sign in

REVIEW 2 major objections 5 minor 2 cited by

Valid Selection among Conformal Sets

T0 review · 2 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A stability condition on the selection rule transfers the marginal coverage of each conformal predictor to the chosen one, so a user can pick the smallest set per input $X$ with a certified $1-\alpha$ guarantee.

desk verdict The stable-selection framework (MinSE, Theorem 1, Corollary 1) is a genuine and sound contribution, but the paper's most practically useful result, the effective-rank recalibration (Theorem 2), has a proof with a backwards inequality, so its coverage guarantee is currently unsupported. read the letter →

arxiv 2506.20173 v1 pith:Z3XOVXV4 submitted 2025-06-25 stat.ML cs.AIcs.LGstat.MEstat.OT

Mahmoud Hegazy , Liviu Aolaritei , Michael I. Jordan , Aymeric Dieuleveut This is my paper · ORCID
classification stat.MLcs.AIcs.LGstat.MEstat.OT MSC 62G1562F25
keywords conformalpredictionvalidselectionalgorithmicstabilitypost-selectioncoveragesmallestsetsplitonlineeffectiveranks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper takes up a practical annoyance of conformal prediction: when several valid prediction sets are available, selecting the most attractive one, typically the smallest, breaks the marginal coverage guarantee because the selection is informed by the same data that certified the sets. The authors claim the loss can be repaired by making the selection rule stable in a precise sense: a randomized rule whose output distribution, conditional on the observed set sizes, is within an $(e^\eta,\tau)$-indistinguishability budget of a fixed reference distribution. Under that condition the selected set covers the true label with probability at least $1-\alpha e^\eta-\tau$, so building the individual sets at the adjusted level $1-(\alpha-\tau)e^{-\eta}$ restores nominal $1-\alpha$ coverage. In the split-conformal regime they go further and prove that exact $1-\alpha$ coverage can be recovered by recalibrating on effective ranks, provided the selection rule is independent of the calibration data. The payoff, if the paper is right, is that a user can choose among conformal predictors pointwise, per test input $X$, without sacrificing the coverage guarantee.

What carries the argument

The load-bearing object is $(\eta,\tau)$-conditional indistinguishability and the stability notion built on it: a randomized selection algorithm is stable if, conditional on the feature and the vector of set sizes, its output distribution is within factor $e^\eta$ and additive slack $\tau$ of a fixed reference random index $S_0$. Stability lets the proof attach a 'shadow' reference output to the selection rule and push the coverage event through the indistinguishability inequality. The second machinery, for the split conformal setting, is the effective rank: the rank of the $i$-th calibration point's non-conformity score under the predictor selected for that point. These effective ranks are exchangeable with the test point's effective rank when the selector is independent of the calibration data, so forming the set at the $\lceil(1-\alpha)(m+1)\rceil$-th order statistic of the effective ranks reproduces the textbook split-conformal rank argument.

What would settle it

Fix the split-conformal setup with two predictors and arrange the auxiliary dataset so that its proxy set sizes rank the predictors in the opposite order to the sizes obtained from the calibration quantiles (for instance, by using different score normalizations in the two datasets). Run the effective-rank recalibration on fresh test points: if the empirical coverage of the selected set falls below $1-\alpha$, the independence premise is not merely technical but load-bearing, and Theorem 2 as a practical recipe fails. A second decisive check is to rerun the reported real-data experiments while varying the unreported preliminary rate $\tilde\alpha$; if any choice breaks the nominal $1-\alpha$ coverage, the practical version of the claim fails.

Watch

Extended reading notes

Core claim

The central claim is a transfer principle. Corollary 1 states that if the selection rule $\hat S$ is $(\eta,\tau)$-stable, then $\mathbb{P}\{Y \in C^\alpha_{\hat S(\xi,\varepsilon)}(X)\} \ge 1-\alpha e^\eta-\tau$, so the coverage of each individual conformal set passes through the selection with only a multiplicative $e^\eta$ inflation and an additive $\tau$ loss. The paper introduces MinSE, the Minimum Stable Expectation mechanism, a linear program that chooses the selection distribution minimizing expected selected size subject to the stability constraints, and proves it optimal among all $(\eta,\tau)$-stable rules. It then shows the same principle yields long-run coverage in the online setting through AdaCOMA, and that in the split conformal setting exact coverage can be restored by using the selected predictor's calibration rank as a meta-score, taking the usual quantile of these effective ranks (Theorem 2).

Load-bearing premise

For the batch theorems the selection rule must satisfy Definition 2 stability: conditional on the observed set sizes there must exist a reference random index whose distribution is within the multiplicative-additive budget $(e^\eta,\tau)$ of the rule's own selection probabilities, and for the exact recalibration result the rule must be conditionally independent of the calibration data, which forces the selector to use only an auxiliary dataset.

Editorial extensions

If this is right

  • A practitioner with several conformal predictors can combine them pointwise, picking the smallest set for each $X$ with a randomized stable rule, and still certify the nominal $1-\alpha$ marginal coverage after inflating the individual levels to $1-(\alpha-\tau)e^{-\eta}$.
  • MinSE is a near-optimal way to do this: among all rules satisfying the same stability budget it achieves the smallest expected selected size almost surely (given a suitable prior), and the worst-case bound $\alpha e^\eta+\tau$ is tight, as the oracle example shows.
  • In the split conformal setting, effective-rank recalibration removes the inflation entirely under the independence condition, delivering exactly the standard $1-\alpha$ guarantee; the paper's experiments report that this version (Recal) gives the shortest average intervals among the compared methods.
  • In the online setting, AdaCOMA inherits COMA's historically learned weights as a prior but conditions the selection on the current set sizes, gaining pointwise adaptability while keeping the long-run coverage statement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same transfer principle should apply to any family of data-dependent confidence intervals beyond conformal sets, suggesting a general recipe for repairing selection among valid confidence statements by paying a small multiplicative randomization budget.
  • Editorial inference: the independence condition behind Theorem 2 tells practitioners to spend a slice of the calibration budget on a proxy dataset for the selector; the unreported preliminary rate $\tilde\alpha$ for the proxy quantiles is then a hidden tuning knob, and testing the method's sensitivity to it is the most direct check of the practical claim.
  • Editorial inference: one could try to lift effective-rank recalibration from split conformal to cross-conformal or jackknife+ constructions, where exchangeability of the meta-scores is not automatic, and that would require new arguments rather than a direct application of Theorem 2.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes a stability-based framework for selecting among multiple conformal prediction sets while preserving finite-sample coverage. After introducing an (η,τ,ν)-stability notion adapted from Zrnic and Jordan, Theorem 1 gives a generic post-selection coverage bound, and Corollary 1 specializes it to conformal sets. Several stable selection mechanisms are proposed (Laplace, exponential, MinSE), with an optimality result for MinSE and extensions to adaptive, derandomized, and conditionally-valid selection. The framework is then extended to the online setting via AdaCOMA. Finally, Section 5 introduces a recalibration method (Recal) based on effective ranks in split conformal prediction, with Theorem 2 claiming exact 1−α coverage after an independence-compliant selection rule; experiments report Recal as the best-performing method. The stability-based results (Sections 3–4) appear self-contained and correct on inspection; the critical problem is that Theorem 2, the foundation of the recommended Recal method, is false as stated.

Significance. If the stability-based results were the whole paper, the contribution would be solid: Theorem 1 and Corollary 1 provide a clean transfer of marginal coverage under a weak stability condition, MinSE is a well-motivated and provably optimal-in-class mechanism, and the online AdaCOMA extension is a natural and useful combination with COMA. The main significance claim, however, is attached to the Recal method, which is presented as achieving tight post-selection coverage and is the best performer in the experiments. Because Theorem 2 is false, the Recal guarantee is invalid, and the empirical results for Recal in Section 6 and Appendix B are not backed by any valid theory. This substantially reduces the significance of the manuscript in its current form.

major comments (2)
  1. [Theorem 2 (Section 5; proof in Appendix A.3)] The proof of Theorem 2 is invalid and the theorem is false as stated. The proof asserts 'ℛ⊂ℛ+ implies that ^R+_{(m)}≤ ^R_{(m)}' and later uses ^R+_{(i)}≤ ^R_{(i)} to replace the threshold. However, for each i≤m, adding the test score to the calibration set can only increase (or keep) the rank of a calibration point, so pointwise R_i≤R^+_i, which implies R_{(i)}≤R^+_{(i)} for every i — the opposite of the direction used in the proof. Consequently, the step replacing ^R+_{(i)} by ^R_{(i)} is a decrease of the threshold, not an increase, and the coverage event of the final set is not implied by the exchangeability of ℛ+. This is not a minor gap: the claim is false. A concrete counterexample is obtained with m=2, α=0.4 (so τ_α=⌈0.6·3⌉=2), and a constant selection rule k_1=1, k_2=2, k_test=1, which is independent of D_cal and hence satisfies Theorem 2's assumption. If the score values s_{1,1},s_{1,2},s_{1,test},s_{2,1},s_{2,2} are iid continuous, then R_1 and R_2 are the binary ranks of the two calibration points under predictors 1 and 2, and the threshold is ^R_{(2)}=max(R_1,R_2). Conditional on max(R_1,R_2)=2, coverage occurs with probability 2/3; conditional on max(R_1,R_2)=1, it occurs with probability 1/3; and P(max(R_1,R_2)=1)=1/4. Thus the unconditional coverage is 3/4·2/3+1/4·1/3=7/12≈0.583, which is strictly less than 1−α=0.6, violating the theorem's conclusion.
  2. [Sections 5–6 (Recal and experiments)] Because Theorem 2 is the sole theoretical justification for the Recal procedure, the claimed post-selection coverage guarantee for Recal is unsupported, and the empirical coverage reported in Figure 2 and Appendix B is purely anecdotal. The counterexample in the previous comment shows that a selection rule satisfying the theorem's independence assumption can undercover; therefore the Recal method, as described, does not provide a valid distribution-free guarantee. The authors would need to either remove Recal from the paper, replace it with a different recalibration method whose guarantee can actually be proved, or substantially revise the theory. As submitted, the paper's best-performing experimental method rests on a false theorem.
minor comments (5)
  1. [Appendix A.2 (proofs of Corollary 2 and Proposition 5)] In the display for Corollary 2, the equality 'P{Y_t ∉ C^(t)_Ŝ} = E[1{...}] = ∑_i p_i(ξ_t) 1{Y_t ∉ C^(t)_i}' omits the expectation operator around the random sum; as printed it equates a deterministic probability with a random variable. The same issue appears in Proposition 5. This should be corrected to 'E[1{...}] = E[∑_i p_i(ξ_t) 1{Y_t ∉ C^(t)_i}]'.
  2. [Section 5 (Construction of an independent Ŝ)] The preliminary miscoverage rate ᾱ used to compute proxy quantiles from the auxiliary dataset D_aux is never reported in Section 6 or Appendix B, and the size of D_aux is not specified. This makes the exact Recal variant used in the experiments unclear and the results difficult to reproduce.
  3. [Figure 2 caption] The legend in Figure 2 lists 'AdaMinSE α′=0.50', while the experimental text states 'AdaMinSE with α′=0.05'; these should be reconciled.
  4. [Appendix A.3 (proof of Theorem 2)] The notation 'ℛ⊂ℛ+' is undefined and, if read as a multiset inclusion, is false: the elements of ℛ are not the same as the corresponding elements of ℛ+ because the ranks R_i and R^+_i are computed against different score sets. The proof should state the intended relationship explicitly, though the correct pointwise relationship points in the opposite direction from the one used.
  5. [Lemma 1 statement] The noise distribution 'ε∼(Lap(1/η))^{⊗K}' is nonstandard; it would be clearer to write ε_1,...,ε_K i.i.d. with distribution Lap(1/η).

Circularity Check

0 steps flagged · score 0.0 of 10

No material circularity: the coverage guarantees are derived from explicit stability and exchangeability assumptions, not from the target coverage claims.

full rationale

The paper's main guarantees are self-contained derivations from stated premises. Theorem 1 and Corollary 1 follow from the indistinguishability condition in Definition 2 via Lemma 4 of Zrnic and Jordan [37]; that lemma is a published, parameter-free result whose stated assumptions do not include conformal coverage, and the authors supply the conformal-specific reduction, so the self-citation is real evidence rather than a circular premise. Lemmas 1-3 and the MinSE/AdaMinSE constructions are proven stable directly from the same definition; in AdaMinSE, the constraint e^eta alpha' + tau <= alpha is a design constraint that directly implies the stated coverage, so Proposition 2 is a verification of the construction rather than a fitted prediction dressed up as a finding. The tightness examples in Section 3.3 are explicitly worst-case constructions whose miscoverage is computed to equal the bound; they are not retrospective fits. Theorem 2 is an exchangeability argument on effective ranks with an independent selection rule, and the use of D_aux is presented as a sufficient condition for that independence; the coverage statement is not obtained by tuning to empirical coverage. The experiments benchmark against external baselines (Yang-Kuchibhotla, Liang et al., COMA/Gasparin-Ramdas) and do not fit parameters to force nominal coverage. The only flagged issue, the order-statistic inequality in Appendix A.3 relating \hat{R}^+_{(i)} and \hat{R}_{(i)}, is a proof-correctness concern rather than a circular reduction of the conclusion to its inputs; it therefore does not change the circularity score.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The paper imports exchangeability, per-set coverage guarantees, and the Zrnic-Jordan stability lemma as background, then derives the stable selection bounds and the effective-rank recalibration from them. The main costs are structural: the user must choose η, τ, α', and in Section 5 the unspecified α̃, and the practical value of the recalibration depends on the proxy auxiliary data reflecting true set sizes. No new physical or probabilistic entities are introduced beyond the algorithmic constructions.

free parameters (4)
  • η (stability parameter) = 1 (batch), ln(2) (Figure 1), tuned online
    Controls the per-set coverage inflation factor e^η in the selection guarantee (Corollary 1). Chosen by the user, not fitted to data, but it directly sets the coverage price of selection.
  • τ (stability slack) = 0 in all experiments
    Allows trading an additive slack τ against less inflation in the stability bound; set to zero in all experiments.
  • α' (pre-selection miscoverage level) = 0.05 (AdaMinSE), 0.02 (Recal), 0.09 (online ACI)
    Input to AdaMinSE and to the individual conformal predictors, chosen per experiment to target 0.1 post-selection miscoverage.
  • α̃ (proxy quantile level for auxiliary data) = not reported
    Defines the proxy conformal sets from D_aux used by the selection rule in Section 5; the paper never states the value used in the Recal experiments, which is a reproducibility gap.
assumptions (6)
  • domain assumption Batch calibration and test points are exchangeable (or i.i.d.).
    Stated in Section 2; the basis for the marginal coverage guarantee (1) and for the rank argument in Theorem 2.
  • domain assumption Each individual conformal predictor satisfies its nominal coverage guarantee (marginal in batch, asymptotic (2) online).
    Imported standard conformal prediction output; Corollary 1 and Theorem 1 take (3) as their starting point.
  • standard math Zrnic and Jordan [37, Lemma 1]: stability of a randomized algorithm implies joint indistinguishability of (input, output).
    Used as the black box in the proof of Theorem 1 (Appendix A.1); published and peer-reviewed lemma.
  • domain assumption The selection noise ε_i is independent across calibration points and independent of the data.
    Needed in Theorem 2 so that the effective ranks R̂_i form an exchangeable sequence; stated before the theorem.
  • standard math Laplace density ratio bound: for X~Lap(1/η), p(x-1)/p(x) ≤ e^η.
    Used in the proof of Lemma 1.
  • domain assumption The online environment is adversarial or fixed, and the only randomness is the selection noise.
    Assumed in Section 4 when converting per-step selection probabilities into the averaged guarantee (10); this is why (10) is weaker than (2).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Valid Selection among Conformal Sets." pith.science (2026). https://pith.science/paper/Z3XOVXV4

@misc{pith2026250620173,
  author       = {Pith},
  title        = {Pith review of: Valid Selection among Conformal Sets},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z3XOVXV4}},
  note         = {Machine review of arXiv:2506.20173}
}
read the original abstract

Conformal prediction offers a distribution-free framework for constructing prediction sets with coverage guarantees. In practice, multiple valid conformal prediction sets may be available, arising from different models or methodologies. However, selecting the most desirable set, such as the smallest, can invalidate the coverage guarantees. To address this challenge, we propose a stability-based approach that ensures coverage for the selected prediction set. We extend our results to the online conformal setting, propose several refinements in settings where additional structure is available, and demonstrate its effectiveness through experiments.

Figures

Figures reproduced from arXiv: 2506.20173 by the authors.

Figure 1
Figure 1. (Left-example 2) Coverage of the coin flips example after selection using MinSE with parameters 𝜂 ∈ {0.1, 2} and 𝜏 = 0. Before selection, the oracle returns the full set [0, 1] with probability 1 − 𝛼 exp (−𝜂). (Right-example 3) Second and third figures reflecting adaptive conformal intervals, with miscoverage 𝛼 = 0.05, obtained using the models 𝑓(𝑋) = 𝑋 and 𝑓(𝑋) = −𝑋. Fifth figure shows the stable selection applied … view at source ↗
Figure 2
Figure 2. Marginal coverage and average lengths, for real datasets (Bike Sharing, Abalone, [PITH_FULL_IMAGE:figures/full_fig_p015_2.png] view at source ↗
Figure 3
Figure 3. Homogeneous synthetic results. Each plot shows coverage (left) and interval length [PITH_FULL_IMAGE:figures/full_fig_p027_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Heterogeneous synthetic results. Each plot shows coverage (left) and interval length [PITH_FULL_IMAGE:figures/full_fig_p028_4.png]
Figure 5
Figure 5. Figure 5: Real data results under the homogeneous setting [PITH_FULL_IMAGE:figures/full_fig_p029_5.png]
Figure 6
Figure 6. Figure 6: Real data results under the heterogeneous setting. [PITH_FULL_IMAGE:figures/full_fig_p029_6.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Approximate full conformal prediction in an RKHS

    stat.ML 2026-01 conditional novelty 6.0 of 10

    For RKHS/Tikhonov predictors, computable approximations to the full conformal region contain it (hence cover at level 1−α) with explicit thickness rates, improved from O(1/(λn)) to O(1/(λ³n²)) via influence functions.

  2. Improving Coverage in Combined Prediction Sets with Weighted p-values

    cs.LG 2025-05 conditional novelty 6.0 of 10

    Weighted averaging of conformal p-values with a learned scaling correction yields valid combined prediction sets for data-dependent weights, improving worst-slice coverage.

Reference graph

Works this paper leans on

40 extracted references · 31 canonical work pages · cited by 2 Pith papers

  1. [1]

    A. N. Angelopoulos, R. F. Barber, and S. Bates. Theoretical foundations of conformal prediction. arXiv preprint arXiv:2411.11824, 2024. 15

    arXiv 2024

  2. [2]

    Bai and Y

    T. Bai and Y. Jin. Optimized conformal selection: Powerful selective inference after con- formity score optimization.arXiv preprint arXiv:2411.17983, 2024

    arXiv 2024

  3. [3]

    R. F. Barber, E. J. Candes, A. Ramdas, and R. J. Tibshirani. Predictive inference with the jackknife. The Annals of Statistics, 49(1):486–507, 2021

    work page 2021

  4. [4]

    Bassily and Y

    R. Bassily and Y. Freund. Typical stability.arXiv preprint arXiv:1604.03336, 2016

    arXiv 2016

  5. [5]

    Bassily, K

    R. Bassily, K. Nissim, A. Smith, T. Steinke, U. Stemmer, and J. Ullman. Algorithmic sta- bility for adaptive data analysis. InProceedings of the forty-eighth annual ACM symposium on Theory of Computing, pages 1046–1059, 2016

    work page 2016

  6. [6]

    Bhatnagar, H

    A. Bhatnagar, H. Wang, C. Xiong, and Y. Bai. Improved online conformal prediction via strongly adaptive online learning. InInternational Conference on Machine Learning, pages 2337–2363. PMLR, 2023

    work page 2023

  7. [7]

    Bousquet and A

    O. Bousquet and A. Elisseeff. Stability and generalization.The Journal of Machine Learn- ing Research, 2:499–526, 2002

    work page 2002

  8. [8]

    P. S. Bradley, K. P. Bennett, and A. Demiriz. Constrained k-means clustering.Microsoft Research, Redmond, 20(0):0, 2000

    work page 2000

Show all 40 references
  1. [9]

    De Rooij, T

    S. De Rooij, T. Van Erven, P. D. Grünwald, and W. M. Koolen. Follow the leader if you can, hedge if you must.The Journal of Machine Learning Research, 15(1):1281–1316, 2014

    2014

  2. [10]

    Dwork, V

    C. Dwork, V. Feldman, M. Hardt, T. Pitassi, O. Reingold, and A. L. Roth. Preserving statistical validity in adaptive data analysis. InProceedings of the forty-seventh annual ACM symposium on Theory of computing, pages 117–126, 2015

    2015

  3. [11]

    Dwork, F

    C. Dwork, F. McSherry, K. Nissim, and A. Smith. Calibrating noise to sensitivity in private data analysis. InTheory of Cryptography: Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA, March 4-7, 2006. Proceedings 3, pages 265–284. Springer, 2006

    2006

  4. [12]

    Dwork, A

    C. Dwork, A. Roth, et al. The algorithmic foundations of differential privacy.Foundations and Trends® in Theoretical Computer Science, 9(3–4):211–407, 2014

    2014

  5. [13]

    Elisseeff, T

    A. Elisseeff, T. Evgeniou, M. Pontil, and L. P. Kaelbing. Stability of randomized learning algorithms. Journal of Machine Learning Research, 6(1), 2005

    2005

  6. [14]

    Feldman, L

    S. Feldman, L. Ringel, S. Bates, and Y. Romano. Achieving risk control in online learning settings. arXiv preprint arXiv:2205.09095, 2022

    2022 arXiv

  7. [15]

    Freund and R

    Y. Freund and R. E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting.Journal of computer and system sciences, 55(1):119–139, 1997

    1997

  8. [16]

    Gasparin and A

    M. Gasparin and A. Ramdas. Conformal online model aggregation. arXiv preprint arXiv:2403.15527, 2024. 16

    2024

  9. [17]

    Gasparin and A

    M. Gasparin and A. Ramdas. Merging uncertainty sets via majority vote.arXiv preprint arXiv:2401.09379, 2024

    2024 arXiv

  10. [18]

    Gibbs and E

    I. Gibbs and E. Candes. Adaptive conformal inference under distribution shift.Advances in Neural Information Processing Systems, 34:1660–1672, 2021

    2021

  11. [19]

    Gibbs and E

    I. Gibbs and E. J. Candès. Conformal inference for online prediction with arbitrary distri- bution shifts. Journal of Machine Learning Research, 25(162):1–36, 2024

    2024

  12. [20]

    Conformalpredictionwithconditionalguarantees

    I.Gibbs, J.J.Cherian, andE.J.Candès. Conformalpredictionwithconditionalguarantees. Journal of the Royal Statistical Society Series B: Statistical Methodology, page qkaf008, 2025

    2025

  13. [21]

    Hajihashemi and Y

    E. Hajihashemi and Y. Shen. Multi-model ensemble conformal prediction in dynamic environments. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  14. [22]

    Harries, N

    M. Harries, N. S. Wales, et al. Splice-2 comparative evaluation: Electricity pricing. 1999

    1999

  15. [23]

    Head, MechCoder, G

    T. Head, MechCoder, G. Louppe, I. Shcherbatyi, fcharras, Z. Vinícius, cmmalone, C. Schröder, nel215, N. Campos, T. Young, S. Cereda, T. Fan, rene rex, K. K. Shi, J. Schwabedal, carlosdanielcsantos, Hvass-Labs, M. Pak, SoManyUsernamesTaken, F. Call- away, L. Estève, L. Besson, ...

    2018 doi

  16. [24]

    Jin and E

    Y. Jin and E. J. Candès. Selection by prediction with conformal p-values. Journal of Machine Learning Research, 24(244):1–41, 2023

    2023

  17. [25]

    Kelly, R

    M. Kelly, R. Longjohn, and K. Nottingham. The UCI machine learning repository.https: //archive.ics.uci.edu, 2023. Accessed: 2023-10-05

    2023

  18. [26]

    Levy-Kramer

    J. Levy-Kramer. k-means-constrained, Apr. 2018. URL https://github.com/joshlk/ k-means-constrained

    2018

  19. [27]

    Liang, W

    R. Liang, W. Zhu, and R. F. Barber. Conformal prediction after efficiency-oriented model selection. arXiv preprint arXiv:2408.07066, 2024

    2024 arXiv

  20. [28]

    McSherry and K

    F. McSherry and K. Talwar. Mechanism design via differential privacy. In48th Annual IEEE Symposium on Foundations of Computer Science (FOCS’07), pages 94–103. IEEE, 2007

    2007

  21. [29]

    R. K. Pace and R. Barry. Sparse spatial autoregressions.Statistics & Probability Letters, 33(3):291–297, 1997

    1997

  22. [30]

    Pedregosa, G

    F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, et al. Scikit-learn: Machine learning in python. the Journal of machine Learning research, 12:2825–2830, 2011. 17

    2011

  23. [31]

    Shafer and V

    G. Shafer and V. Vovk. A tutorial on conformal prediction.Journal of Machine Learning Research, 9(3), 2008

    2008

  24. [32]

    R. S. Tsay.Analysis of financial time series. John wiley & sons, 2005

    2005

  25. [33]

    V. Vovk, A. Gammerman, and G. Shafer.Algorithmic learning in a random world, vol- ume 29. Springer, 2005

    2005

  26. [34]

    E. W. Weisstein. Bonferroni correction.https://mathworld. wolfram. com/, 2004

    2004

  27. [35]

    Yang and A

    Y. Yang and A. K. Kuchibhotla. Selection and aggregation of conformal prediction sets. Journal of the American Statistical Association, pages 1–13, 2024

    2024

  28. [36]

    Zaffran, O

    M. Zaffran, O. Féron, Y. Goude, J. Josse, and A. Dieuleveut. Adaptive conformal predic- tions for time series. InInternational Conference on Machine Learning, pages 25834–25866. PMLR, 2022

    2022

  29. [37]

    Zrnic and M

    T. Zrnic and M. I. Jordan. Post-selection inference via algorithmic stability.The Annals of Statistics, 51(4):1666–1691, 2023. 18 Contents 1 Introduction 1 2 Preliminaries in Conformal Prediction 3 3 Smallest Confidence Set Selection 4 3.1 Valid Selection via Algorithmic Stabi...

    2023

  30. [38]

    Here, we provide additional results, without this splitting step

    First, in Section 6, we preprocessed the dataset, both synthetic and real, by splitting them to 5disjoint equal subsets using constrained K-means [8, 26, Code under BSD 3-Clause License]. Here, we provide additional results, without this splitting step. We call the setting wit...

  31. [39]

    BayesSearchCV

    Second, we repeat the experiments on synthetic data, while varying the number datapoints in the calibration dataset. This mainly affects the results of YK-Adjust as its performance improves with larger calibration datasets. For the real dataset experiments (Abalone, Bike Shari...

  32. [50]

    The target miscoverage is𝛼 = 0.1

    for the standard error). The target miscoverage is𝛼 = 0.1. Dataset Method Avg. Miscoverage Avg. Length ELEC COMA (AdaHedge) 0.0942± 0.002 0 .69± 0.06 AdaCOMA (AdaHedge) 0.0959± 0.001 0 .32± 0.09 COMA (Hedge) 0.0942± 0.002 0 .69± 0.06 AdaCOMA (Hedge) 0.0963± 0.001 0 .31± 0.01 A...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.