RECORD
Valid prediction intervals for regression problems
Machine Learning · Submitted 1 Jul 2021
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim We illustrate how conformal prediction can be used as a general calibration procedure for methods that deliver poor results without a calibration step.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Over the last few decades, various methods have been proposed for estimating prediction intervals in regression settings, including Bayesian methods, ensemble methods, direct interval estimation methods and conformal prediction methods. An important issue is the calibration of these methods: the generated prediction intervals should have a predefined coverage level, without being overly conservative. In this work, we review the above four classes of methods from a conceptual and experimental point of view. Results on benchmark data sets from various domains highlight large fluctuations in performance from one data set to another. These observations can be attributed to the violation of certain assumptions that are inherent to some classes of methods. We illustrate how conformal prediction can be used as a general calibration procedure for methods that deliver poor results without a calibration step.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Over the last few decades, various methods have been proposed for estimating prediction intervals in regression settings, including Bayesian methods, ensemble methods, direct interval estimation methods and conformal prediction methods. | background 1.00 | 0.03 | 0.03 | asserted 1.00 |
| s1 | An important issue is the calibration of these methods: the generated prediction intervals should have a predefined coverage level, without being overly conservative. | background 0.98 | 0.05 | 0.04 | asserted 1.00 |
| s2 | In this work, we review the above four classes of methods from a conceptual and experimental point of view. | contribution 0.98 | 0.88 | 0.04 | asserted 0.91 |
| s3 | Results on benchmark data sets from various domains highlight large fluctuations in performance from one data set to another. | result 1.00 | 0.83 | 0.18 | asserted 1.00 |
| s4 | These observations can be attributed to the violation of certain assumptions that are inherent to some classes of methods. | interpretation 1.00 | 0.40 | 0.22 | hedged possibility 0.50 |
| s5 | We illustrate how conformal prediction can be used as a general calibration procedure for methods that deliver poor results without a calibration step. | contribution 0.81 | 0.82 | 0.38 | asserted 0.99 |
What was examined, and what was not
- arXiv listing abstract v4
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Over the last few decades, various methods have been proposed for estimating prediction intervals in regression settings, including Bayesian methods, ensemble methods, direct interval estimation methods and conformal prediction methods. An important issue is the calibration of these methods: the generated prediction intervals should have a predefined coverage level, without being overly conservative. In this work, we review the above four classes of methods from a conceptual and experimental point of view. Results on benchmark data sets from various domains highlight large fluctuations in performance from one data set to another. These observations can be attributed to the violation of certain assumptions that are inherent to some classes of methods. We illustrate how conformal prediction can be used as a general calibration procedure for methods that deliver poor results without a calibration step.
Human review
How the literature uses it
What this paper does with the arXiv papers it cites
What it cites · 1 sentence without a confident label
Cites as support 1 paper
-
This is motivated by a prior observation that random initialization often leads to superior performance when compared to other ensemble techniques [ref] .
Why M Heads are Better than One: Training a Diverse Ensemble of Deep Networks · Deep ensembles · 0.99
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.