Performative Risk Control: Calibrating Models for Reliable Deployment under Performativity

Baiting Chen; Qi Lei; Victor Li; Yuzhen Mao; Zhun Deng

REVIEW 3 major objections 3 minor 1 cited by

Performative Risk Control: Calibrating Models for Reliable Deployment under Performativity

T0 review · 3 major / 3 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A new calibration procedure proves finite-sample risk control even when predictions reshape the data.

desk verdict Genuinely new framework for risk control under performativity, but the main proof has a fixable gap; send to peer review. read the letter →

arxiv 2505.24097 v1 pith:QABTJI6J submitted 2025-05-30 stat.ML cs.LG

Victor Li , Baiting Chen , Yuzhen Mao , Qi Lei , Zhun Deng This is my paper

classification stat.MLcs.LG

keywords performativepredictionriskcontrolfinite-sampleguaranteesconformalanytimesafetyquantilemeasurescreditscoringthresholdcalibration

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Safely calibrating a black-box model usually assumes the calibration data look like the deployment data. This paper treats the harder case where the act of deploying a threshold changes the distribution of future data — a phenomenon called performativity, common in credit scoring and other policy settings. It introduces Performative Risk Control, an iterative procedure that at each round uses samples drawn under the current threshold to pick a less conservative threshold, adding a 'performativity guard' that anticipates how the distribution will shift. Under the assumption that the loss distribution moves Lipschitz-continuously with the threshold (speed at most γ), choosing the guard τ ≥ γ yields a finite-sample guarantee, holding with probability 1−δ, that every intermediate threshold and the final threshold keep the expected risk at or below the user level α, while the final threshold is never much more conservative than needed. The same guarantees extend to quantile-based risk measures such as CVaR.

What carries the argument

The engine is the update rule $\lambda_t = \inf\{\lambda \in \Lambda : \hat{R}_n(\lambda_{t-1}, \lambda) + c(n, \delta/\tilde{T}) + \tau(\lambda_{t-1} - \lambda) \le \alpha\}$, together with a stopping rule that terminates when $\lambda_{t-1} - \lambda_t < \Delta\lambda$. Here $\hat{R}_n$ is the empirical loss evaluated at the candidate threshold on samples drawn under the previous threshold, $c(n, \delta/\tilde{T})$ is a pointwise confidence width from a concentration bound (Hoeffding, Bernstein, Hoeffding–Bentkus, or CLT), and the term $\tau(\lambda_{t-1} - \lambda)$ is the performativity guard. The guard is what converts the assumption that the loss distribution moves at speed at most $\gamma$ into an upper bound on the performative error, so that $\tau \ge \gamma$ makes each deployed threshold safe against the distribution shift it induces. The stopping rule and the joint choice of $\tilde{T}$ and $\Delta\lambda$ ensure the process terminates and the final threshold is tight.

What would settle it

Run Algorithm 1 on a constructed distribution map where a small threshold decrease causes a mass of loss values to jump from 0 to 1 (a super-Lipschitz shift with no finite γ), with τ set to any finite value; if the middle iterations or final threshold ever show empirical risk above α at a rate exceeding δ, the anytime-safety claim is falsified. Alternatively, on any real dataset, measure W1(D_l(λ1), D_l(λ2)) for nearby thresholds and check whether it exceeds the assumed γ|λ1−λ2|.

Watch

Extended reading notes

Core claim

The paper's central claim is that a simple, iterative, defensively conservative threshold update converts static-distribution risk control into performative risk control without sacrificing finite-sample guarantees. Starting from a guaranteed-safe threshold λ_safe, each round samples data from the distribution induced by the previously deployed threshold, forms an empirical upper confidence bound for the loss evaluated at candidate thresholds, and adds the term τ(λ_{t−1}−λ) — the performativity guard — to bound the error that arises because deploying λ moves the distribution itself. The procedure then moves to the most aggressive threshold whose guarded upper bound still lies at or below α, and stops when further progress would be smaller than a precomputed increment Δλ. Provided the distribution map is (γ,1,ℓ(·,λ))-sensitive for all λ (loss distributions shift by at most γ in 1-Wasserstein distance per unit threshold change), the loss is continuous and monotone in λ, and τ ≥ γ, Algorithm 1 proves with probability 1−δ that the risk stays at or below α at every step along the trajectory and that the returned threshold satisfies R(λ_T) ≥ α − Δα. The proof rests on a lemma bounding the performative error by γ times the threshold gap, a UCB-calibration argument adapted from the static risk-control literature, and a union bound over the at most T̃ iterations.

Load-bearing premise

The load-bearing premise is that the loss distribution changes at a bounded rate — at most γ per unit change in the threshold, measured in 1-Wasserstein distance — so the shift induced by any deployment step is no larger than the guard τ times the step size.

Editorial extensions

If this is right

An institution can deploy a sequence of increasingly aggressive thresholds, each guaranteed (with probability $1-\delta$) to keep expected risk at or below $\alpha$, even when applicants or agents adapt their behavior to the deployed threshold.
The user-specified tightness $\Delta\alpha$ can be driven to zero as the sample size grows, so the final threshold is asymptotically as aggressive as the risk budget allows.
The anytime-safety property means the guarantee holds for every intermediate deployment, not just the final one, which is essential for monitoring policies that are rolled out incrementally.
Because the framework only post-processes a black-box predictor and never refits it, the same procedure applies to any threshold-indexed prediction rule with a monotone loss, including prediction sets and standard classifiers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The conservative choice $\tau \ge \gamma$ with $\gamma$ estimated from a single balanced dataset may be loose in practice; estimating $\gamma$ per iteration or from streaming deployment data could yield smaller $\Delta\alpha$ while preserving the proof structure — a modification the authors do not investigate.
The monotonicity assumption on the loss confines the method to parameters where larger values are uniformly safer; for non-monotone post-processing knobs such as temperature with an optimum in the interior, a two-sided or signed guard would be needed.
The sensitivity assumption is stated in Wasserstein-1 distance on losses, so a practical validation route is to estimate the Lipschitz constant empirically from repeated deployments and check whether observed distribution shifts stay within the assumed bound.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's the one thing you should know: this paper has a genuinely new idea—finite-sample risk control under performative distribution shift—but the proof of the main theorem has a concrete gap that needs repair before the claims are fully supported. I think the gap is fixable, so it's a conditional revise, not a reject.

What's new: PRC extends the UCB calibration framework of Bates et al. to a setting where the distribution depends on the deployed threshold. The iterated procedure with a Lipschitz-sensitivity assumption on the distribution map is a natural and useful generalization, and the anytime-safety guarantee—risk controlled at every iterate, not just the final one—is a nice property for policy deployment. The quantile-risk extension (CVaR et al.) is a plus. The experiments on credit scoring are consistent with the claims and reproducible.

Now the soft spots, in order of importance. The gap: in Theorem A.5, Case 2, the proof bounds R(λhat_t) using Proposition A.4, but that proposition only certifies the standard UCB threshold λ*_+ = inf{λ' : \hat R_n^+(λ_prev, λ') < α}. Algorithm 1 selects a different object—the minimizer of \hat R_n(λ_prev,λ) + c + τ(λ_prev−λ). The citation therefore doesn't apply as written. The standard repair is to run the UCB argument on the augmented risk f(λ) = R(λ_prev,λ) + τ(λ_prev−λ), whose natural upper confidence bound has the same form, and then τ(λ_prev−λ) is already inside the UCB. That's a local fix but it's load-bearing: without it, the anytime-safety and the proof of part (i) are not rigorous. I checked the rest of the chain; part (iii) uses pointwise bounds directly and looks fine.

Second, Assumption 3.2 (γ-sensitivity of the loss distribution) is simply assumed. In the experiments, γ is estimated from the same balanced dataset used to evaluate the algorithm, so the reported failure probabilities aren't a pure test of the method. That's not a flaw in the theory, just a caveat on the empirical claims. The paper does say γ is a domain-knowledge parameter, which is honest.

Third, the 'first to study risk control under performativity' claim should be delimited against strategic conformal prediction (Csillag et al.) and distribution-shift-robust conformal methods. The current related work mentions them but doesn't draw the boundary sharply; a referee should ask for that.

Who is this for: anyone working on calibration, conformal risk control, or performative prediction. It's a solid subfield contribution, not a breakthrough, but the framework is likely to be cited. My recommendation: send to peer review. The gap is specific and addressable; a serious referee could get the authors to fix it in a revision. I'd accept it as a conditional, expecting a corrected proof and a sharper comparison.

Referee Report

3 major / 3 minor

Summary. The paper introduces Performative Risk Control (PRC), an iterative post-hoc calibration procedure for a threshold parameter λ so that the performative risk R(λ)=E_{z∼D(λ)}ℓ(z,λ) is controlled. Algorithm 1 samples from the distribution induced by the current threshold and sets the next threshold as the infimum of the set {λ : \hat R_n(\hat λ_{t−1},λ)+c(n,δ/\tilde T)+τ(\hat λ_{t−1}−λ)≤α}, stopping when the progress is below Δλ. Theorem 3.3 claims that, under a Wasserstein-Lipschitz condition on D(·) with constant γ and τ≥γ, with probability 1−δ all iterates satisfy safety of the deployed pair R(\hat λ_{t−1},\hat λ_t)≤α, safety at anytime R(\hat λ_t)≤α, and tightness of the final threshold R(\hat λ_T)≥α−Δα. The paper also extends the procedure to quantile-based risk measures (Theorem 4.2) and reports experiments on credit default risk for expected type-II error and 90%-CVaR.

Significance. If the main theorem is established, this is a genuinely useful bridge between conformal risk control and performative prediction: it gives an anytime-safe, iteratively refined calibration procedure with a clean finite-sample guarantee in a setting where the data distribution reacts to the deployed decision rule. The paper has concrete strengths: it provides several explicit confidence-width constructions (Hoeffding, Bernstein, Hoeffding-Bentkus, CLT), a nontrivial quantile-risk extension, and experiments that simulate the performative shift and attempt to verify the sensitivity constant. The central proof is, however, incomplete as written: the key step applies a UCB-calibration proposition to an iterate that is not the plain UCB threshold, and the pointwise confidence statement in Eq. (1) is data-dependent. These are specific, addressable gaps rather than demonstrated counterexamples, so the contribution is promising but not yet proven.

major comments (3)

[App. A.4, Theorem A.5 and Proposition A.4] The citation of Proposition A.4 in the second inequality of Theorem A.5, Case 2 is not available as written. Proposition A.4 proves safety at λ_*^+:=inf{λ′: \hat R_n^+(λ,λ′;δ′)<α}, whereas Algorithm 1 selects \hat λ_t as the infimum over {λ: V(\hat λ_{t−1},λ,δ)≤α}, with V containing the additional term τ(\hat λ_{t−1}−λ). The proof therefore jumps from R(\hat λ_t) ≤ R(\hat λ_{t−1},\hat λ_t)+γM|\hat λ_t−\hat λ_{t−1}| to ≤ \hat R_n^+(\hat λ_{t−1},\hat λ_t,δ/\tilde T)+γM|\hat λ_t−\hat λ_{t−1}| without a valid probabilistic statement at the data-dependent point \hat λ_t. A standard repair is to run the UCB argument on the augmented risk f(λ)=R(\hat λ_{t−1},λ)+τ(\hat λ_{t−1}−λ), whose natural upper confidence bound is \hat f^+(λ)=\hat R_n^+(\hat λ_{t−1},λ,δ/\tilde T)+τ(\hat λ_{t−1}−λ), and to prove that the inf-threshold of \hat f^+ controls f. Until this argument is supplied, the anytime-safety and tightness guarantees in Theorem 3.3 are unsupported.
[Eq. (1), §3.2 and App. A.2] The pointwise confidence statement in Eq. (1) is formulated for "bλ_{t−1} and λ′≥bλ_t for all t encountered in the procedure," so the event itself depends on the algorithm's data-dependent iterates. This is not a well-defined pointwise coverage statement, and it cannot be used as a hypothesis in Proposition A.4 without an explicit union bound over the thresholds that may be visited. The construction of c(n,δ′) in App. A.2 uses worst-case variance but does not prove a uniform-in-λ bound with failure probability δ/\tilde T when the iterates are stopping times. The proof should either state a simultaneous confidence band over a fixed grid or over all λ′∈Λ with total error δ/\tilde T, or justify the stopping-time argument directly.
[App. B.1 and §5] The empirical claims are conditioned on an estimated sensitivity constant: γ is bounded by p·C with C read off a 20-bin histogram of the balanced subset, and the same balanced subset is used to evaluate the algorithm. The theorem requires the exact γ, and if the estimated γ is smaller than the true value, the anytime guarantee can fail. This is not a circularity in the theorem, because τ is not fitted to the target guarantee; however, it is a mismatch between the strength of the claimed finite-sample guarantee and the experimental verification. The paper should either state the empirical results as heuristic or provide a valid confidence upper bound for γ derived from a separate calibration set.

minor comments (3)

[Section 1 and App. A.2] There are typos in "initate" (Section 1) and "Hoeffding-Benkus" (App. A.2 heading and text; should be Hoeffding-Bentkus).
[Notation, Theorem 3.3] The shorthand R(λ) is used in Theorem 3.3 without a formal definition; the paper should define R(λ):=R(λ,λ) at first use, since both R(λ) and R(λ,λ′) appear throughout.
[Theorem 4.2] The proof of Theorem 4.2 is one sentence saying it follows from App. A.4; given the additional CDF confidence bands and the Hölder conditions, a more detailed proof would help the reader verify that the quantile confidence width satisfies the same union-bound requirements.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular reduction found: the PRC guarantees follow from the stated γ-sensitivity and confidence-width assumptions; the flagged Proposition A.4 citation issue is a write-up gap, not a circular reduction.

full rationale

The central derivation is self-contained rather than circular. Assumption 3.2 (Lipschitz distribution mapping with sensitivity γ) and the confidence widths c(n,δ′) are stated inputs; Algorithm 1 builds V from the empirical loss, a concentration width, and the user-chosen performativity guard τ(bλ_{t−1}−λ) with τ ≥ γ. Theorem 3.3's safety guarantees follow by bounding the performative shift with Lemma A.3 and using the pointwise UCB/LCB from Eq. (1), while the tightness bound R(bλ_T) ≥ α − Δα is an algebraic consequence of the stopping rule (progress ≤ Δλ) and the definition Δα = 2τΔλ + 2c(n,δ/T̃), not a fitted quantity. No parameter is fitted to the target guarantee: τ is user-chosen before deployment, and the experimental estimate of γ is used only to select τ in the simulation, not inside the theorem. The quantile extension follows the CDF-confidence technique of Snell et al. (2022) and Zollo et al. (2023), which include an author of this paper, but those are external published results and are not the foundation of the main expected-risk guarantee; at most this is a minor, non-load-bearing self-citation. I note one rigor gap that is not circularity: Theorem A.5's Case 2 attributes the inequality R(bλ_{t−1},bλ_t) ≤ bR+_n(bλ_{t−1},bλ_t,δ/T̃) to Proposition A.4, which is stated for the plain UCB threshold λ*_+, not for the minimizer bλ_t of V. The needed inequality is available directly from the pointwise confidence property in Eq. (1), so the argument is repairable without making the conclusion an input. This is a write-up gap, not a self-definitional or fitted-input circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The framework rests on a Lipschitz condition for the performative distribution map and on pointwise confidence bounds. No new physical entities or mechanisms are introduced. The key unproven ingredient is the simultaneous validity of the confidence width at data-dependent thresholds.

free parameters (2)

τ (performative guard) = not fitted; user-specified, requires τ≥γ
The algorithm's safety and tightness depend on a guard multiplier τ that must upper-bound the distribution-map sensitivity γ, which is unknown and must be estimated or guessed.
γ (sensitivity constant) = estimated in experiments from histogram (γ=pC≤1.38 for expected-risk experiment, γ≤0.205 for CVaR)
The central assumption is a Lipschitz bound on the loss-distribution map; its constant γ is a property of the environment, not derived. In experiments it is estimated on the same balanced subset used for calibration and validation.

assumptions (4)

domain assumption Loss ℓ(z,λ) is non-increasing in λ for every z, with ℓ(z,λ_safe)=0.
Section 2.1; this monotonicity makes risk a non-increasing function and enables the threshold crossing arguments.
domain assumption Distribution map D(·) is (γ,1,ℓ(·,λ))-sensitive for all λ (Assumption 3.2).
The whole performative guard τ(λ_{t-1}−λ) relies on this 1-Wasserstein Lipschitz condition; if it fails, R(λ_t) can exceed α regardless of sample size.
ad hoc to paper The confidence width c(n,δ') as constructed in App. A.2 provides pointwise coverage (Eq. 1) at any fixed λ'.
The construction uses max over λ' of pointwise bounds, but the paper never proves a simultaneous band; at a data-dependent threshold this coverage is not automatically valid. The proof would need a counterfactual argument at a fixed crossing point.
standard math Loss continuity in λ and standard concentration inequalities (Hoeffding, Bernstein, Hoeffding-Bentkus, CLT).
Continuity is used to ensure the infimum is attained and the crossing arguments work; the concentration inequalities are standard but are invoked pointwise, not uniformly.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Performative Risk Control: Calibrating Models for Reliable Deployment under Performativity." pith.science (2026). https://pith.science/paper/QABTJI6J

@misc{pith2026250524097,
  author       = {Pith},
  title        = {Pith review of: Performative Risk Control: Calibrating Models for Reliable Deployment under Performativity},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QABTJI6J}},
  note         = {Machine review of arXiv:2505.24097}
}

read the original abstract

Calibrating blackbox machine learning models to achieve risk control is crucial to ensure reliable decision-making. A rich line of literature has been studying how to calibrate a model so that its predictions satisfy explicit finite-sample statistical guarantees under a fixed, static, and unknown data-generating distribution. However, prediction-supported decisions may influence the outcome they aim to predict, a phenomenon named performativity of predictions, which is commonly seen in social science and economics. In this paper, we introduce Performative Risk Control, a framework to calibrate models to achieve risk control under performativity with provable theoretical guarantees. Specifically, we provide an iteratively refined calibration process, where we ensure the predictions are improved and risk-controlled throughout the process. We also study different types of risk measures and choices of tail bounds. Lastly, we demonstrate the effectiveness of our framework by numerical experiments on the task of predicting credit default risk. To the best of our knowledge, this work is the first one to study statistically rigorous risk control under performativity, which will serve as an important safeguard against a wide range of strategic manipulation in decision-making processes.

Figures

Figures reproduced from arXiv: 2505.24097 by the authors.

**Figure 2.** Type II error control in credit scoring. We use the same coloring scheme to label trajectories [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗

**Figure 3.** 90%-CVaR type II error control in credit scoring. These plots correspond to the bottom two plots in Fig.2 but for the 90%-CVaR risk measure. The left plot shows τ = 2 instead of τ = 1. 6 Conclusion We have introduced a theoretically sound and empirically effective framework for risk control under performativity, a previously unexplored topic in the risk control literature. To mitigate performativity, our framework i… view at source ↗

Figures from the paper (7 more)

**Figure 4.** Figure 4: A comparison of Hoeffding and the risk level-dependent bounds. Plots 2c(n, δ′ ) vs. α for n = 2000, T˜ = 100, δ = 0.1, and δ ′ = δ/T˜. 2c(n, δ′ ) represents the lower limit of achievable tightness ∆α. Lower confidence widths are better. Next, note that our algorithm is…

**Figure 5.** Figure 5: Histogram of predictions f (x) on the base distribution. be 3.20 and 0.432, respectively. Importantly, we calculate C using the balanced subset rather than the training set. We use 20 bins for the histogram (see [PITH_FULL_IMAGE:figures/full_fig_p024_5.png]

**Figure 6.** Figure 6: The relative frequency of runs that end with a [PITH_FULL_IMAGE:figures/full_fig_p025_6.png]

**Figure 7.** Figure 7: Analog of the bottom left plot in Fig [PITH_FULL_IMAGE:figures/full_fig_p025_7.png]

**Figure 8.** Figure 8: The relative frequency of runs that end with a [PITH_FULL_IMAGE:figures/full_fig_p027_8.png]

**Figure 9.** Figure 9: Analog of the bottom left plot in Fig [PITH_FULL_IMAGE:figures/full_fig_p027_9.png]

**Figure 10.** Figure 10: Analog of the bottom left plot in Fig [PITH_FULL_IMAGE:figures/full_fig_p028_10.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Robust Conformal CBF and CLF Controllers via Iterative Policy Updates
eess.SY 2026-06 conditional novelty 6.0 of 10

An iterative conformal-prediction update rule transfers probabilistic safety/stability guarantees across changing robust CBF/CLF policies despite policy-induced distribution shift.

Reference graph

Works this paper leans on

25 extracted references · 12 canonical work pages · cited by 1 Pith paper

[1]

L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al

Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. (2023). Gpt-4 technical report. arXiv preprint arXiv:2303.08774

arXiv 2023
[2]

Angelopoulos, A. N. and Bates, S. (2021). A gentle introduction to conformal prediction and distribution-free uncertainty quantification. arXiv preprint arXiv:2107.07511

arXiv 2021
[3]

N., Bates, S., Candès, E

Angelopoulos, A. N., Bates, S., Candès, E. J., Jordan, M. I., and Lei, L. (2022a). Learn then test: Calibrating predictive algorithms to achieve risk control

work page 2022
[4]

N., Bates, S., Fisch, A., Lei, L., and Schuster, T

Angelopoulos, A. N., Bates, S., Fisch, A., Lei, L., and Schuster, T. (2022b). Conformal risk control. arXiv preprint arXiv:2208.02814

arXiv 2022
[5]

F., Candes, E

Barber, R. F., Candes, E. J., Ramdas, A., and Tibshirani, R. J. (2023). Conformal prediction beyond exchangeability. The Annals of Statistics , 51(2):816--845

2023
[6]

Bates, S., Angelopoulos, A., Lei, L., Malik, J., and Jordan, M. (2021). Distribution-free, risk-controlling prediction sets. Journal of the ACM (JACM) , 68(6):1--34

work page 2021
[7]

Brown, G., Hod, S., and Kalemaj, I. (2022). Performative prediction in a stateful world. In International conference on artificial intelligence and statistics , pages 6045--6061. PMLR

work page 2022
[8]

Cauchois, M., Gupta, S., Ali, A., and Duchi, J. C. (2024). Robust validation: Confident predictions even when distributions shift. Journal of the American Statistical Association , pages 1--66

work page 2024

Show all 25 references

[9]

Chen, Y., Tang, W., Ho, C.-J., and Liu, Y. (2023). Performative prediction with bandit feedback: Learning through reparameterization. arXiv preprint arXiv:2305.01094

2023 arXiv
[10]

J., and Goedert, G

Csillag, D., Struchiner, C. J., and Goedert, G. T. (2024). Strategic conformal prediction

2024
[11]

and Cukierski, W

Fusion, C. and Cukierski, W. (2011). Give me some credit. https://kaggle.com/competitions/GiveMeSomeCredit. Kaggle

2011
[12]

and Candes, E

Gibbs, I. and Candes, E. (2021). Adaptive conformal inference under distribution shift. Advances in Neural Information Processing Systems , 34:1660--1672

2021
[13]

and Mendler-D \"u nner, C

Hardt, M. and Mendler-D \"u nner, C. (2023). Performative prediction: Past and future. arXiv preprint arXiv:2310.16608

2023 arXiv
[14]

Jagadeesan, M., Zrnic, T., and Mendler-D \"u nner, C. (2022). Regret minimization with performative feedback. In International Conference on Machine Learning , pages 9760--9785. PMLR

2022
[15]

and Pontil, M

Maurer, A. and Pontil, M. (2009). Empirical bernstein bounds and sample variance penalization

2009
[16]

P., Perdomo, J

Miller, J. P., Perdomo, J. C., and Zrnic, T. (2021). Outside the echo chamber: Optimizing the performative risk. In International Conference on Machine Learning , pages 7710--7720. PMLR

2021
[17]

Perdomo, J., Zrnic, T., Mendler-D \"u nner, C., and Hardt, M. (2020). Performative prediction. In International Conference on Machine Learning , pages 7599--7609. PMLR

2020
[18]

and Vovk, V

Shafer, G. and Vovk, V. (2008). A tutorial on conformal prediction. Journal of Machine Learning Research , 9(12):371--421

2008
[19]

C., Zollo, T

Snell, J. C., Zollo, T. P., Deng, Z., Pitassi, T., and Zemel, R. (2022). Quantile risk control: A flexible framework for bounding the probability of high-loss predictions. arXiv preprint arXiv:2212.13629

2022 arXiv
[20]

M., Hauth, A., Millican, K., et al

Team, G., Anil, R., Borgeaud, S., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., Millican, K., et al. (2023). Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805

2023 arXiv
[21]

I., Burnell, R., Bai, L., Gulati, A., Tanzer, G., Vincent, D., Pan, Z., Wang, S., et al

Team, G., Georgiev, P., Lei, V. I., Burnell, R., Bai, L., Gulati, A., Tanzer, G., Vincent, D., Pan, Z., Wang, S., et al. (2024). Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530

2024 arXiv
[22]

J., Foygel Barber, R., Candes, E., and Ramdas, A

Tibshirani, R. J., Foygel Barber, R., Candes, E., and Ramdas, A. (2019). Conformal prediction under covariate shift. Advances in neural information processing systems , 32

2019
[23]

Vaart, A. W. v. d. (1998). Asymptotic Statistics . Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press

1998
[24]

Vovk, V., Gammerman, A., and Shafer, G. (2005). Algorithmic Learning in a Random World . Springer-Verlag, Berlin, Heidelberg

2005
[25]

P., Morrill, T., Deng, Z., Snell, J

Zollo, T. P., Morrill, T., Deng, Z., Snell, J. C., Pitassi, T., and Zemel, R. (2023). Prompt risk control: A rigorous framework for responsible deployment of large language models. arXiv preprint arXiv:2311.13628

2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al

Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. (2023). Gpt-4 technical report. arXiv preprint arXiv:2303.08774

arXiv 2023

[2] [2]

Angelopoulos, A. N. and Bates, S. (2021). A gentle introduction to conformal prediction and distribution-free uncertainty quantification. arXiv preprint arXiv:2107.07511

arXiv 2021

[3] [3]

N., Bates, S., Candès, E

Angelopoulos, A. N., Bates, S., Candès, E. J., Jordan, M. I., and Lei, L. (2022a). Learn then test: Calibrating predictive algorithms to achieve risk control

work page 2022

[4] [4]

N., Bates, S., Fisch, A., Lei, L., and Schuster, T

Angelopoulos, A. N., Bates, S., Fisch, A., Lei, L., and Schuster, T. (2022b). Conformal risk control. arXiv preprint arXiv:2208.02814

arXiv 2022

[5] [5]

F., Candes, E

Barber, R. F., Candes, E. J., Ramdas, A., and Tibshirani, R. J. (2023). Conformal prediction beyond exchangeability. The Annals of Statistics , 51(2):816--845

2023

[6] [6]

Bates, S., Angelopoulos, A., Lei, L., Malik, J., and Jordan, M. (2021). Distribution-free, risk-controlling prediction sets. Journal of the ACM (JACM) , 68(6):1--34

work page 2021

[7] [7]

Brown, G., Hod, S., and Kalemaj, I. (2022). Performative prediction in a stateful world. In International conference on artificial intelligence and statistics , pages 6045--6061. PMLR

work page 2022

[8] [8]

Cauchois, M., Gupta, S., Ali, A., and Duchi, J. C. (2024). Robust validation: Confident predictions even when distributions shift. Journal of the American Statistical Association , pages 1--66

work page 2024