Decision Making Needs Uncertainty Quantification [Lecture Notes]

Osvaldo Simeone

For a risk-averse agent, a prediction set with a coverage guarantee is a lossless replacement for the full posterior distribution.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 02:08 UTC pith:YLDFHJA3

load-bearing objection A transparent, well-organized lecture note that unifies three uncertainty-representation toolboxes; no new theorems, but the pedagogical synthesis is genuinely useful.

arxiv 2607.14407 v2 pith:YLDFHJA3 submitted 2026-07-15 cs.IT cs.AIcs.LGmath.IT

Decision Making Needs Uncertainty Quantification [Lecture Notes]

Osvaldo Simeone This is my paper

classification cs.IT cs.AIcs.LGmath.IT

keywords uncertainty quantificationdecision theoryprediction setsrisk-averse decision makingvalue-at-riskcalibrationdistributionally robust optimizationBayesian inference

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This lecture note asks a practical question: what exactly must a decision-making system know about an uncertain state in order to act optimally, and in what form should that knowledge be represented? For a known environment, the answer depends on risk attitude: a risk-neutral agent needs the posterior distribution over the state, whereas a risk-averse agent—one maximizing value-at-risk—needs only a prediction set that covers the true state with high probability, combined with a worst-case decision rule, with no loss of optimality. When the environment is unknown, the note shows that three standard tools—predictor calibration, credal sets with distributionally robust optimization, and Bayesian inference over model parameters—are complementary ways to supply a certified uncertainty interface. The unifying claim is that trustworthy decisions require an uncertainty representation matched to the objective and backed by a guarantee that certifies the utility actually obtained.

Core claim

The central discovery is that the posterior distribution p(s|o) is a sufficient statistic of the observation for risk-neutral decision-making, while for risk-averse (value-at-risk) decision-making, an optimally chosen level-α prediction set C_α(o) with coverage probability at least 1−α is a sufficient interface: the max-min action over that set equals the value-at-risk optimal action (Proposition 1). The proof rests on the identity that value-at-risk equals the best worst-case utility achievable over any covering set. In unknown environments, the note establishes analogues: a calibrated predictor makes the plug-in action optimal on average, a credal set covering the true posterior with proba

What carries the argument

The load-bearing objects are the posterior distribution p(s|o), the level-α prediction set C_α(o) satisfying the coverage condition (9), the max-min decision rule (10), and the identity (17) that value-at-risk equals the maximum of the worst-case utility over all covering sets. For the data-driven case, the credal set P(D_o) around the empirical histogram, with coverage condition (33) and total-variation radius (34), converts the max-min value into a high-confidence utility certificate.

Load-bearing premise

The data-driven results rest on the assumption that the credal set covers the true posterior with high probability (eq. 33), a guarantee that is proven only for finite discrete variables through a total-variation radius; for continuous states, large alphabets, or model misspecification, no such certificate is supplied.

What would settle it

Simulate a decision problem with a finite discrete state and observation space and a known posterior, then check whether a prediction set of size 1−α that deliberately avoids the posterior's high-density region can yield a max-min action strictly worse than the value-at-risk optimal action; the proposition asserts this cannot happen, so a counterexample would violate the coverage condition (9). More directly, in the data-driven case, generate a dataset from a distribution with a state alphabet large enough that the total-variation bound (34) is loose, and test whether the realised disappointme

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

A production system making risk-averse decisions can replace a full posterior computation with a prediction set and a max-min choice, saving cost without sacrificing optimality in the single-decision setting.
Conformal prediction sets, which guarantee marginal coverage from data alone, inherit a decision-theoretic justification as practical substitutes for the oracle sets in Proposition 1.
Point estimates that ignore epistemic uncertainty are systematically over-optimistic; any data-driven decision system should carry a coverage certificate to bound disappointment.
The paper supplies a common vocabulary for calibration, distributionally robust optimization, and Bayesian inference, showing they solve the same problem under different knowledge assumptions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The oracle prediction set C*_α(o) in Proposition 1 requires knowing the exact conditional posterior, so the result is primarily a characterisation; in unknown environments the practical interface must rely on data-driven sets with marginal rather than conditional coverage.
The coverage certificate for credal sets is instantiated only for finite discrete state and observation spaces; for continuous states or large alphabets, analogous finite-sample certificates would need different concentration inequalities.
The logic connecting coverage to a value-at-risk guarantee suggests a natural extension to other risk measures such as conditional value-at-risk or spectral risk measures, where a set-based interface might again be sufficient.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

A transparent, well-organized lecture note that unifies three uncertainty-representation toolboxes; no new theorems, but the pedagogical synthesis is genuinely useful.

read the letter

Short version: this is a lecture note, and it is honest about being one. It develops a single decision-theoretic setup and shows that the right uncertainty representation depends on the agent's risk attitude and knowledge profile. The four propositions are correctly proved under their stated assumptions, and the paper clearly attributes the substantive ones to prior work — Prop. 2 is [6, Prop. 1], the conformal analog is [5], and the DRO optimality is [12]. The Bayesian section is standard textbook material. What's new is the arrangement: putting calibration, conformal prediction, credal sets/DRO, and Bayesian inference on a common decision-focused axis is a useful teaching device, and the exposition is clear enough that a first-year graduate student could follow it.

I agree with the stress-test: no internal inconsistency or load-bearing error. Prop. 1's 'without loss' claim is conditional on knowing the exact posterior to construct the optimal prediction set, and the paper says so. The finite-data coverage certificate (33) is only instantiated for finite discrete alphabets via the TV bound (34); continuous states and large alphabets aren't covered, and the paper doesn't pretend otherwise. These are scope limitations, not flaws.

The soft spots are proportional to the paper's ambition. It's not a research contribution; it's a synthesis. If you want new algorithms or new geometry, this isn't the place. But as lecture notes for a course on decision-oriented UQ, it's genuinely well done. The citation pattern is clean — the only self-citation is the author's own textbook for background, which is appropriate. The discussion of calibration's relationship to informativeness (Example 1) is a nice touch.

Who should read it: anyone teaching a course spanning Bayesian decision theory, calibration, and robust optimization, or a student who needs a bird's-eye view before diving into the literature. I'd bring it to a reading group as a pedagogical artifact, but I wouldn't cite it in a research paper.

If this crosses your desk for peer review, don't desk-reject it if the venue welcomes tutorials or lecture notes. A serious referee should check that the attributions are right and the pedagogy is sound — and they are, in my reading. For a research venue expecting novelty, it doesn't fit. But for a teaching-oriented venue, it deserves space.

Referee Report

0 major / 4 minor

Summary. This lecture note studies a single-shot decision problem with hidden state s, observation o, action a, and utility u(s,a). The paper asks which representation of uncertainty suffices for optimal action under different knowledge profiles. In a known environment, it shows that a risk-neutral agent needs only the posterior p(s|o) (Section IV-A), while a risk-averse agent optimizing value-at-risk can act via a level-α prediction set and a max-min decision rule without loss of optimality (Lemma 1 and Proposition 1). In an unknown environment, it surveys three routes: distribution-calibrated fixed predictors (Proposition 2), non-parametric histogram-based credal sets with distributionally robust optimization and a finite-sample TV-radius coverage certificate (Proposition 3), and Bayesian model averaging (Section IV-F). The common conclusion is that a reliable decision system needs an uncertainty representation matched to the decision objective together with a certificate of actual achievable utility.

Significance. The paper is a clear and useful pedagogical synthesis connecting Bayesian decision theory, calibration, conformal prediction, distributionally robust optimization, and Bayesian inference within one decision-theoretic framework. Its main mathematical claims are elementary and, as far as I can check, correctly proven: Proposition 1 is a valid quantile/worst-case equivalence, Lemma 1 is a straightforward coverage argument, Proposition 2 follows from the calibration definition, and Proposition 3 follows from the coverage assumption. A strength is that the paper is explicit about the scope of each certificate: the no-loss optimality statement for prediction sets is made for a known environment with oracle access to the exact posterior, and the data-driven results provide coverage/disappointment certificates rather than unconditional optimality. The paper does not claim new theorems, but the unification is valuable for teaching and for framing research. The main limitations—oracle dependence of C*_α(o) in Proposition 1 and the restriction of the finite-sample coverage result to finite discrete alphabets in Section IV-E—are acknowledged in the text or are evident from the setting, and the

minor comments (4)

[Sec. IV-E, Eq. (33)-(34)] The sub-dataset size n_o is random and can be zero for an observation o not present in D. The histogram (28), the radius (34), and the credal set (31) are undefined in that case. Since the coverage condition is written as Pr_{D_o∼p(s|o)^{⊗ n_o}} with n_o itself data-dependent, please clarify that the statement is conditional on n_o≥1, or define an explicit fallback (e.g., a degenerate full simplex) for n_o=0. This is a technical scope gap, not a flaw in the conditional finite-sample bound.
[Sec. II / Sec. IV-E, p.2 and p.4] There are several typographical errors: 'whygood' (end of first paragraph of Section I), 'free of of any bias' (paragraph before Eq. (29)), and 'motivatescalibration' (Section IV-C bullet). Also, 'the histogram is free of any bias' is imprecise as a standalone statement: the empirical conditional distribution is unbiased as a point estimate, but the plug-in decision rule is optimistically biased, as the paper itself immediately explains. A rewording would avoid confusion.
[Sec. IV-D, Prop. 2] The notation 'a*(ˆp) ∈ arg max_{a(ˆp)} E_o[U(a(ˆp)|o)]' is slightly ambiguous because a(ˆp) denotes both a function of the predictor and the action. It would be clearer to define the admissible policy class explicitly, e.g., P = {a: O→A : a(o)=φ(ˆp(s|o)) for some φ}, and then state the optimality over φ. The proof is correct, but this notation will be hard for the intended first-year graduate reader.
[Sec. IV-B, Eq. (16)] Proposition 1's optimized prediction set C*_α(o) requires exact knowledge of p(s|o) to solve the optimization over all covering subsets. The paper does state that this is the known-environment case, but it would be helpful to add one sentence explicitly noting that the 'without loss of optimality' statement is an oracle result and that the later data-driven sections provide certificates, not no-loss optimality for this criterion.

Circularity Check

0 steps flagged

No significant circularity: the posterior, prediction-set, and credal-set interfaces are derived from explicit assumptions and external bounds; no fitted quantity is renamed as a prediction.

full rationale

The derivation chain is self-contained. Proposition 1 is a direct quantile/worst-case equivalence over sets with coverage, proved from the definition of value-at-risk rather than assumed; the optimized prediction set C*_alpha(o) is defined in terms of posterior coverage and utility, and the proof establishes equality with the VaR-optimal action. Lemma 1 and Proposition 3 are conditional guarantees that follow from the relevant coverage events by inclusion, with the TV radius in (34) supplied by an external finite-sample bound (Weissman et al.). Proposition 2 is a formal implication of distribution calibration (23), an independently defined property, and is attributed to external work [6]. The only self-citation is the author's own textbook [2] for background on machine learning and Bayesian inference; it is not load-bearing. The acknowledged limitations (the oracle dependence of C*_alpha on p(s|o), the finite-alphabet and finite-sample coverage certificate in Section IV-E, and the 'assuming the validity of the assumed model' caveat in Section IV-F) are scope restrictions, not inputs disguised as predictions. No equation reduces to its own inputs by construction, and no fitted parameter is relabeled as a forecast.

Axiom & Free-Parameter Ledger

3 free parameters · 8 axioms · 0 invented entities

No new physical entities or fitted constants are introduced. The central claims are conditional mathematical statements parameterized by user-chosen α and δ; the one designed number is the DRO radius r, chosen via a stated deviation bound to satisfy coverage. The main exogenous assumptions are exact knowledge of the environment in the first two settings and exact validity of the model class/prior in the Bayesian setting.

free parameters (3)

risk level α
User-chosen quantile level in (7)–(9) and in the optimized prediction set (16); all risk-averse results are parameterized by it.
confidence level δ
User-chosen confidence in the coverage condition (33) and in the DRO radius formula (34); controls Proposition 3's certificate.
DRO radius r = sqrt((|S| log 2 + log(1/δ)) / (2 n_o))
Chosen from a deviation bound to satisfy the coverage condition (33). Not fitted to data, but the Proposition 3 guarantee only holds for this or a comparable coverage-ensuring radius.

axioms (8)

domain assumption Environment distribution factorizes as p(s,o)=p(s)p(o|s) and the agent observes only o, not s.
Problem statement in Sec. III, eq. (1). All subsequent optimality claims live inside this single-shot hidden-state model.
domain assumption The utility u(s,a) is known and fixed.
Sec. III defines utility as given; if the utility is unknown or misspecified, the optimal interfaces are not well-defined.
domain assumption Known-environment sections (IV-A, IV-B) assume exact knowledge of p(s) and p(o|s).
Needed to compute the exact posterior p(s|o), the coverage sets, and the value-at-risk.
domain assumption The dataset D in (19) is i.i.d. from p(s,o).
Used for histogram estimates, deviation bounds, and Bayesian posterior updates in IV-E and IV-F.
domain assumption The finite-sample DRO section assumes finite alphabets S and O.
Required for the histogram estimator (28) and for the TV deviation bound (34); no continuous-state analogue is provided.
domain assumption The credal set P(D_o) is constructed so that the coverage condition (33) holds.
Proposition 3's controlled-disappointment guarantee is conditional on this coverage; the text only instantiates it for the TV radius (34).
domain assumption The parametric Bayesian section assumes the model family p(s|o,θ) and prior p(θ) are correctly specified.
Sec. IV-F explicitly acknowledges model bias; the claimed reliability of the predictive expected utility depends on this correctness.
standard math Law of iterated expectations, Bayes' rule, the CLT, and the Weissman et al. L1 deviation bound are used without reproof.
Invoked in Secs. IV-A, IV-D, and IV-E; standard background for the derivations.

pith-pipeline@v1.3.0-alltime-deepseek · 9635 in / 11454 out tokens · 107461 ms · 2026-08-02T02:08:41.827409+00:00 · methodology

0 comments

read the original abstract

Many signal processing systems ultimately exist to {act}. Whenever the state variable that determines the action to be taken by a decision maker, or agent, is uncertain, the way that uncertainty is represented decides how well the agent performs and how much its performance can be trusted. This lecture note develops, from first principles and within a single decision-theoretic setting, the link between the {objective} and the knowledge of an agent and the form of uncertainty representation that is sufficient to act optimally. To start, assuming a known environment distribution, we show that a risk-neutral agent needs the posterior distribution over the state, whereas a risk-averse agent can rely without loss of optimality on a {prediction set} and a worst-case decision rule. We then turn to the case in which the environment is unknown, and identify three complementary approaches to address the resulting epistemic uncertainty: calibration of a fixed predictor, credal (ambiguity) sets with distributionally robust optimization, and Bayesian inference over model parameters. The common thread is that reliable decisions require an uncertainty representation matched to the decision objective and to the knowledge profile of the agent, together with a guarantee that certifies the utility the agent will actually obtain.

Figures

Figures reproduced from arXiv: 2607.14407 by Osvaldo Simeone.

**Figure 2.** Figure 2: The optimal interface between the agent and its environment across [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

**Figure 3.** Figure 3: Risk-neutral versus risk-averse decision making for a fixed obser [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

discussion (0)

Reference graph

Works this paper leans on

12 extracted references · 2 linked inside Pith

[1]

Towards a science of ai agent reliability,

S. Rabanser, S. Kapoor, P. Kirgis, K. Liu, S. Utpala, and A. Narayanan, “Towards a science of ai agent reliability,”arXiv preprint arXiv:2602.16666, 2026

Pith/arXiv arXiv 2026
[2]

Simeone,Machine Learning for Engineers

O. Simeone,Machine Learning for Engineers. Cambridge University Press, 2022

2022
[3]

V ovk, A

V . V ovk, A. Gammerman, and G. Shafer,Algorithmic Learning in a Random World. Springer, 2005

2005
[4]

Conformal prediction: A gentle introduction,

A. N. Angelopoulos and S. Bates, “Conformal prediction: A gentle introduction,”Foundations and Trends in Machine Learning, vol. 16, no. 4, pp. 494–591, 2023

2023
[5]

Decision theoretic foundations for conformal prediction: Optimal uncertainty quantification for risk-averse agents,

S. Kiyani, G. J. Pappas, A. Roth, and H. Hassani, “Decision theoretic foundations for conformal prediction: Optimal uncertainty quantification for risk-averse agents,” inProc. Int. Conf. Machine Learning (ICML), ser. PMLR, vol. 267, 2025, pp. 30 943–30 965, arXiv:2502.02561

Pith/arXiv arXiv 2025
[6]

Calibrating predictions to decisions: A novel approach to multi-class calibration,

S. Zhao, M. Kim, R. Sahoo, T. Ma, and S. Ermon, “Calibrating predictions to decisions: A novel approach to multi-class calibration,” Advances in Neural Information Processing Systems, vol. 34, pp. 22 313–22 324, 2021

2021
[7]

On calibration of modern neural networks,

C. Guo, G. Pleiss, Y . Sun, and K. Q. Weinberger, “On calibration of modern neural networks,” inProc. Int. Conf. Machine Learning (ICML), 2017, pp. 1321–1330

2017
[8]

Metrics of calibration for probabilistic predictions,

I. Arrieta-Ibarra, P. Gujral, J. Tannen, M. Tygert, and C. Xu, “Metrics of calibration for probabilistic predictions,”Journal of Machine Learning Research, vol. 23, no. 351, pp. 1–54, 2022

2022
[9]

Robust solutions of optimization problems affected by uncertain probabilities,

A. Ben-Tal, D. den Hertog, A. De Waegenaere, B. Melenberg, and G. Rennen, “Robust solutions of optimization problems affected by uncertain probabilities,”Management Science, vol. 59, no. 2, pp. 341– 357, 2013

2013
[10]

Walley,Statistical Reasoning with Imprecise Probabilities

P. Walley,Statistical Reasoning with Imprecise Probabilities. Chapman and Hall, 1991

1991
[11]

Inequalities for theL 1 deviation of the empirical distribution,

T. Weissman, E. Ordentlich, G. Seroussi, S. Verdu, and M. J. Weinberger, “Inequalities for theL 1 deviation of the empirical distribution,” Hewlett- Packard Laboratories, Tech. Rep. HPL-2003-97R1, 2003

2003
[12]

From data to decisions: Distributionally robust optimization is optimal,

B. P. G. Van Parys, P. Mohajerin Esfahani, and D. Kuhn, “From data to decisions: Distributionally robust optimization is optimal,”Management Science, vol. 67, no. 6, pp. 3387–3402, 2021

2021

[1] [1]

Towards a science of ai agent reliability,

S. Rabanser, S. Kapoor, P. Kirgis, K. Liu, S. Utpala, and A. Narayanan, “Towards a science of ai agent reliability,”arXiv preprint arXiv:2602.16666, 2026

Pith/arXiv arXiv 2026

[2] [2]

Simeone,Machine Learning for Engineers

O. Simeone,Machine Learning for Engineers. Cambridge University Press, 2022

2022

[3] [3]

V ovk, A

V . V ovk, A. Gammerman, and G. Shafer,Algorithmic Learning in a Random World. Springer, 2005

2005

[4] [4]

Conformal prediction: A gentle introduction,

A. N. Angelopoulos and S. Bates, “Conformal prediction: A gentle introduction,”Foundations and Trends in Machine Learning, vol. 16, no. 4, pp. 494–591, 2023

2023

[5] [5]

Decision theoretic foundations for conformal prediction: Optimal uncertainty quantification for risk-averse agents,

S. Kiyani, G. J. Pappas, A. Roth, and H. Hassani, “Decision theoretic foundations for conformal prediction: Optimal uncertainty quantification for risk-averse agents,” inProc. Int. Conf. Machine Learning (ICML), ser. PMLR, vol. 267, 2025, pp. 30 943–30 965, arXiv:2502.02561

Pith/arXiv arXiv 2025

[6] [6]

Calibrating predictions to decisions: A novel approach to multi-class calibration,

S. Zhao, M. Kim, R. Sahoo, T. Ma, and S. Ermon, “Calibrating predictions to decisions: A novel approach to multi-class calibration,” Advances in Neural Information Processing Systems, vol. 34, pp. 22 313–22 324, 2021

2021

[7] [7]

On calibration of modern neural networks,

C. Guo, G. Pleiss, Y . Sun, and K. Q. Weinberger, “On calibration of modern neural networks,” inProc. Int. Conf. Machine Learning (ICML), 2017, pp. 1321–1330

2017

[8] [8]

Metrics of calibration for probabilistic predictions,

I. Arrieta-Ibarra, P. Gujral, J. Tannen, M. Tygert, and C. Xu, “Metrics of calibration for probabilistic predictions,”Journal of Machine Learning Research, vol. 23, no. 351, pp. 1–54, 2022

2022

[9] [9]

Robust solutions of optimization problems affected by uncertain probabilities,

A. Ben-Tal, D. den Hertog, A. De Waegenaere, B. Melenberg, and G. Rennen, “Robust solutions of optimization problems affected by uncertain probabilities,”Management Science, vol. 59, no. 2, pp. 341– 357, 2013

2013

[10] [10]

Walley,Statistical Reasoning with Imprecise Probabilities

P. Walley,Statistical Reasoning with Imprecise Probabilities. Chapman and Hall, 1991

1991

[11] [11]

Inequalities for theL 1 deviation of the empirical distribution,

T. Weissman, E. Ordentlich, G. Seroussi, S. Verdu, and M. J. Weinberger, “Inequalities for theL 1 deviation of the empirical distribution,” Hewlett- Packard Laboratories, Tech. Rep. HPL-2003-97R1, 2003

2003

[12] [12]

From data to decisions: Distributionally robust optimization is optimal,

B. P. G. Van Parys, P. Mohajerin Esfahani, and D. Kuhn, “From data to decisions: Distributionally robust optimization is optimal,”Management Science, vol. 67, no. 6, pp. 3387–3402, 2021

2021