Pith. sign in

REVIEW 4 major objections 4 minor 62 references

On Calibration in Multi-Distribution Learning

T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The minimax-optimal predictor in multi-distribution learning is calibrated only for the maximum-entropy distribution; for others, calibration error is generic and a calibration-refinement trade-off persists.

desk verdict The calibration bound is real, but the 'fundamental' framing is not supported—a symmetric counterexample kills the universal impossibility. read the letter →

arxiv 2412.14142 v1 pith:SJ5NM75N submitted 2024-12-18 cs.LG

Rajeev Verma , Volker Fischer , Eric Nalisnick This is my paper · ORCID
classification cs.LG
keywords multi-distributionlearningcalibrationgeneralizedentropyproperscoringlossescalibration-refinementtrade-offdistributionallyrobustoptimizationmin-maxfairnessdecision-making
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Multi-distribution learning (MDL) promises a single predictor that performs uniformly well over a set of distributions, with applications in robustness and fairness. This paper asks whether that predictor can also be trusted as a calibrated confidence measure for every distribution in the set. It shows that the minimax-optimal MDL predictor—the conditional distribution of the maximum-generalized-entropy distribution $Q^*$—is perfectly calibrated for $Q^*$ but generically cannot be for any other distribution $Q$, with calibration error bounded by the gap in generalized entropies. Consequently, calibration errors are non-uniform across distributions and there is an inherent calibration-refinement trade-off at optimality: improving refinement for one distribution is compensated by worse calibration. If correct, this means MDL's worst-case loss guarantee does not automatically translate into reliable, equitable decisions for arbitrary downstream costs.

What carries the argument

The load-bearing object is the generalized entropy $H_\ell(Q(y|x)) = \inf_{h \in \Delta^{|Y|}} \mathbb{E}_Q[\ell(y,h(x))]$ of a proper scoring loss $\ell$, together with the saddle point $(Q^*, h^*)$ of the MDL zero-sum game in which $Q^* = \arg\max_{Q\in\mathcal{Q}} H_\ell(Q(y|x))$ and $h^*(x) = Q^*(y|x)$. The argument runs through the calibration-refinement decomposition $\mathbb{E}_Q[\ell(y,h(x))] = \mathbb{E}_Q[d_\ell(Q(y|h(x)), h(x))] + \mathbb{E}_Q[H_\ell(Q(y|h(x)))]$, where $d_\ell$ is the Bregman divergence associated with $\ell$; this identity converts the saddle-point property into bounds on per-distribution calibration error and exposes the trade-off.

What would settle it

Compute per-distribution expected calibration error for the max-entropy MDL predictor on a simple two-distribution envelope (e.g., two Gaussian classes with different priors): the claim predicts errors that are nonzero for any $Q \neq Q^*$ and scale with the generalized-entropy gap, so finding an envelope where all $Q$ have zero calibration error would falsify the central assertion.

Watch

Extended reading notes

Core claim

The paper's central claim is that the MDL minimax problem has a unique saddle point $(Q^*, h^*)$ where $Q^*$ maximizes the generalized entropy $H_\ell(Q(y|x))$ over the distribution set and $h^*(x) = Q^*(y|x)$. At this optimal point, the predictor is perfectly canonically calibrated for $Q^*$ but, absent special distributional coincidences, not for any other $Q$ in the set. Using the calibration-refinement decomposition of proper scoring losses—risk equals expected Bregman divergence between $Q(y|h^*(x))$ and $h^*(x)$ plus expected generalized entropy—the paper bounds the calibration error for each $Q$ by $\mathbb{E}_{Q^*}[H_\ell(Q^*(y|x))] - \mathbb{E}_Q[H_\ell(Q(y|x))]$. This bound is zero only when $Q$ collapses to $Q^*$, so calibration errors form a non-uniform pattern across the envelope. The authors conclude that the same forecast has different conditional-label meaning under different distributions, and a decision rule built on the calibrated forecast is worst-case optimal only for cost functions consistent with the training loss.

Load-bearing premise

The central claim rests on the minimax theorem applying to the MDL game—the distribution set compact and convex, the predictor free to output any label distribution, and the loss proper, convex, and continuous—so that the optimum is exactly the maximum-entropy predictor, and on the absence of distributional coincidences that would make other distributions calibrated with that predictor.

Editorial extensions

If this is right

  • For every distribution $Q$ in the envelope, the MDL-optimal predictor's calibration error is at most $\mathbb{E}_{Q^*}[H_\ell(Q^*(y|x))] - \mathbb{E}_Q[H_\ell(Q(y|x))]$, so the error vanishes only as $Q$ approaches $Q^*$.
  • A decision rule that thresholds the calibrated forecast is worst-case optimal over the whole set only when the downstream cost function is consistent with the training loss; otherwise some distributions will receive systematically suboptimal decisions.
  • In distributionally robust optimization, selecting an ambiguity-set divergence that is $\kappa$-Lipschitz with respect to the generalized entropy bounds the calibration disparity across the set by $\kappa\epsilon$.
  • In min-max fairness over $k$ distinct populations, the optimal predictor can fall inside the convex hull of the population distributions, so it is calibrated for none of the $k$ groups and needs post-processing.
  • At the MDL optimum, any gain in refinement for one distribution is compensated by an increase in its calibration error, so no predictor can be simultaneously perfectly calibrated and maximally refined for every distribution in the set.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's claims: the trade-off suggests that post-processing an MDL predictor to be calibrated on one distribution will push miscalibration onto other distributions, so fixes like temperature scaling cannot remove the disparity; this is testable by measuring per-distribution calibration error before and after such post-processing.
  • A further consequence is that the choice of training loss in MDL is itself a fairness decision, because the set of downstream cost functions for which the worst-case guarantee survives is determined by which loss was used to identify the max-entropy distribution.
  • The bounds may transfer to risk-measure formulations such as CVaR, where the envelope is defined by density constraints; in that case one would predict that calibration disparity grows with the allowed density ratio, a statement that empirical studies on robust optimization could check.
  • One could view the entropy-gap bound as an uncertainty principle for MDL: the product of worst-case calibration error and worst-case refinement loss across the envelope has a positive lower bound whenever the envelope contains distributions with different generalized entropies.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper studies the calibration properties of predictors obtained from multi-distribution learning (MDL). It argues that, at the MDL-optimal saddle point, the optimal predictor is the conditional distribution Q*(y|x) of the distribution Q* that maximizes the generalized entropy of the loss, and that this predictor cannot be perfectly canonically calibrated for all distributions in the set. The authors derive an upper bound on the calibration error for any Q in terms of a generalized entropy gap (Proposition 4.1), claim a non-uniform calibration disparity (Corollary 4.2), assert a fundamental calibration-refinement trade-off even at optimality (Corollary 4.3), and discuss decision-theoretic consequences (Proposition 4.4) and implications for distributionally robust optimization and fairness (Section 6).

Significance. If the main claims were correct, the paper would provide a useful warning about a fundamental limitation of MDL. The calibration error upper bound in Proposition 4.1 is a valid and instructive observation, and the paper is right that MDL can lead to calibration disparities. However, the central impossibility claim---that perfect calibration for all Q is impossible and that a calibration-refinement trade-off is fundamental---is not established and is in fact false in general, as shown by a symmetric counterexample. The correct statement is a possibility result: calibration failures can occur, but they need not. The paper also contains a circular argument in Proposition 4.4 and an unsupported corollary for DRO. The contribution is therefore significantly weaker than claimed, but the upper-bound result and the articulation of conditions under which disparities arise may be salvageable after substantial revision.

major comments (4)
  1. [Section 4, Proposition 4.1 and Corollary 4.3] The claim that h* = Q*(y|x) 'cannot be perfectly canonically calibrated' for Q != Q* is false. Proposition 4.1 only gives an upper bound: E_Q[d_l(Q(y|h*), h*)] <= E_Q*[H_l(Q*(y|x))] - E_Q[H_l(Q(y|x))]. The right-hand side can be zero for Q != Q* whenever the two distributions have the same generalized entropy, and the bound then forces zero calibration error. A concrete counterexample is X={x1,x2}, Y={0,1}, uniform marginals, log loss, Q1(y=1|x1)=0.1, Q1(y=1|x2)=0.9, Q2(y=1|x1)=0.9, Q2(y=1|x2)=0.1. Both Q1 and Q2 have identical generalized entropy H(0.1), and the max-entropy distribution over their convex hull is Q*(y|x)=0.5 for both x. Thus h* is the constant 0.5, and for each Q_i the marginal label distribution is Q_i(y=1)=0.5, so Q_i(y|h*=0.5)=0.5 and h* is perfectly calibrated for both distributions. This directly contradicts the universal 'cannot be calibrated' statement and the 'different meaning' conclusion of Corollary 4.3. The correct statement is a possibility result, not a fundamental limitation.
  2. [Section 4, Corollary 4.3 proof] The proof of the claimed calibration-refinement trade-off is not valid. It asserts 'a < b' where a = E_Q[H_l(Q(y|x))] and b = E_Q[l(y,h*)], but this inequality is false for Q = Q*, where h* = Q*(y|x) and a = b. The subsequent argument that a more informative predictor 'will be compensated by increase in the calibration error' is informal: the calibration-refinement decomposition alone does not imply that decreasing refinement forces an increase in calibration error, since the total risk need not be fixed. A formal derivation of a trade-off would require showing that the gap b - a is strictly positive for Q != Q* and that changes in refinement are absorbed by calibration error in a controlled way; the current proof does neither.
  3. [Section 4, Proposition 4.4 and Appendix B.2] Proposition 4.4 is circular as stated. The proof in B.2 shows that the decision rule delta is optimal on average for Q*, and then says 'If we further assume that Q* also maximizes the generalized entropy with respect to the cost function c, under this assumption it can be said that delta achieves the value sup_Q min_delta E_Q[c(delta(h(x)),y)]'. But the assumption that Q* maximizes generalized entropy for c is essentially the desired conclusion that the worst-case decision optimality holds over Q. Without independent conditions relating c to l and to the ambiguity set Q, the proposition merely restates the assumption that the cost function is 'consistent' with the loss. The manuscript needs to state sufficient conditions under which the saddle-point for l is also a saddle-point for c.
  4. [Section 6, Corollary 6.1] Corollary 6.1 is not established. It claims that an ambiguity set defined by a divergence d that is k-Lipschitz with respect to the generalized entropy function 'guarantees robust and equitable decisions', but the proof only notes that |H_l(P)-H_l(Q)| <= k*epsilon bounds the entropy gap in the calibration error bound. This bounds the calibration error, not the decision-theoretic optimality claimed in Proposition 4.4. Moreover, the statement that 'every proper scoring loss function is associated with a Bregman divergence that satisfies this continuity property' is unsubstantiated: the Bregman divergence is on prediction space, not on the space of distributions, and Lipschitz continuity of the entropy with respect to that divergence would need proof. The remark that KL is Lipschitz with respect to Shannon entropy is not true on the full simplex, since the gradient of entropy is unbounded near the boundary.
minor comments (4)
  1. [Section 4, Proposition 4.1 proof] In the proof of Proposition 4.1, the displayed relation 'EQ[H_l(Q*(y|x))] <= EQ*[H_l(Q*(y|x))]' appears to contain a typo: the left argument should presumably be Q(y|h*(x)) or Q(y|x), not Q*(y|x). This makes the subsequent sentence difficult to interpret.
  2. [Section 4, Corollary 4.2] Corollary 4.2 states that calibration errors form 'a non-increasing sequence converging to 0' with lim_{Q->Q*}, but no sequence is defined and no topology on the set of distributions Q is specified. As written, this is not a precise mathematical statement; it should be rephrased in terms of a chosen metric or family of distributions approaching Q*.
  3. [General] There are several presentation issues: the phrase 'there exists Q* that attains the said quantity sup inf = inf' in Proposition 3.2 is imprecise about the order of quantifiers; Figure 1's caption does not explain the plotted quantities; and the text switches between 'h*' and 'h' inconsistently in places. These should be corrected.
  4. [Section 5] The related work in Section 5 mentions the concurrent work of Fröhlich and Williamson (2024c) but does not carefully delineate which of the paper's claims are novel relative to that work, especially concerning maximum generalized entropy predictors and limitations for arbitrary cost functions. A more detailed comparison would help the reader assess the novelty.

Circularity Check

1 steps flagged · score 3.0 of 10

Core entropy-gap bound is external and not fitted; the main circularity is in Proposition 4.4, where the decision-theoretic conclusion is built into the assumed condition.

  1. self definitional [Section 4, Proposition 4.4 and proof in Appendix B.2]
    "Given Q and a predictor h calibrated with respect to Q*∈Q with the maximum generalized entropy for a loss function ℓ, a decision rule δ : h (x) ↦→ arg min_a∈A E_y∼h(x)[c (a,y)] with the action space A and a cost function c : A×Y→ R+ is optimal in the worst case sense over Q as long as the cost function c is consistent with the loss function ℓ. ... If we further assume that Q* also maximizes the generalized entropy with respect to the cost function c, under this assumption it can be said that δ achieves the value sup_Q min_δ E_Q[c (δ(h(x)),y)]—a worst-case decision optimality against Q."

    The proposition's conclusion is that δ is worst-case optimal over Q for the decision-maker's cost c. The proof obtains this by 'further assum[ing] that Q* also maximizes the generalized entropy with respect to the cost function c'. That assumption is precisely the statement that the adversarial distribution for the decision problem under c is the same Q* that is adversarial for the original MDL loss ℓ, which is what 'δ is worst-case optimal over Q for c' requires. The phrase 'consistent with the loss function ℓ' is left undefined in the proposition and is unpacked in the proof as this assumption.

full rationale

The central calibration-entropy bound in Proposition 4.1 is obtained from the external calibration-refinement decomposition (Lemma 2.5, Bröcker) together with the saddle-point inequality; it involves no fitted parameters and no self-citation chain. That derivation is self-contained, so the overall circularity is not high. However, the paper's stronger assertion in Proposition 4.1 that 'barring any distributional assumptions', h* 'cannot be perfectly canonically calibrated' for Q is not entailed by the upper bound, since the right-hand side of the bound can vanish when two distributions share the same generalized entropy; this is a correctness problem rather than a circularity, but it weakens the universal 'different meaning' conclusion in Corollary 4.3. The load-bearing circular step is Proposition 4.4, where worst-case decision optimality is asserted only under an undefined 'consistent' condition that the proof reveals to be the assumption that Q* also maximizes generalized entropy with respect to the cost function c. That assumption is nearly identical to the conclusion, so this secondary result reduces to its own premise. No self-citation or imported uniqueness theorem carries the core argument.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

No free parameters are fitted. The central derivations rely on standard calibration decomposition, the minimax theorem, and the robust-Bayes max-entropy result. Two assumptions specific to this paper are the Lipschitz-continuity of the divergence in Corollary 6.1 and the undefined consistency condition in Proposition 4.4.

assumptions (5)
  • standard math Proper scoring risk decomposes into calibration error plus refinement error (Bröcker 2009), and each proper loss has a concave generalized entropy with a Bregman divergence.
    Used throughout, especially in Lemma 2.5 and Proposition 4.1.
  • domain assumption The von Neumann minimax theorem applies to the MDL game, so sup_Q inf_h E_Q[l] = inf_h sup_Q E_Q[l] and the saddle point has h*=Q*(y|x).
    Stated in Section 3, paragraph after Proposition 3.2, requiring Q compact and convex, H full simplex, loss convex and continuous.
  • domain assumption Maximum generalized entropy is robust Bayes (Grünwald and Dawid 2004), so the worst-case distribution Q* maximizes H_l(Q(y|x)).
    Used to identify the MDL optimal predictor in Section 3 and in Proposition 4.1.
  • ad hoc to paper The chosen divergence d in Corollary 6.1 is Lipschitz continuous with respect to the generalized entropy function of the loss.
    This is asserted to guarantee robust and equitable decisions in DRO; no proof is given, and the paper's KL example is not generally true.
  • ad hoc to paper In Proposition 4.4, the decision cost function c is 'consistent' with the loss l, interpreted as Q* also maximizing the generalized entropy with respect to c.
    The consistency condition is never defined in Section 4 or Appendix B.2; the proof assumes the cost's worst-case distribution is Q*.

how reviews work

0 comments
Cite this review

Pith. "Pith review of On Calibration in Multi-Distribution Learning." pith.science (2026). https://pith.science/paper/SJ5NM75N

@misc{pith2026241214142,
  author       = {Pith},
  title        = {Pith review of: On Calibration in Multi-Distribution Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SJ5NM75N}},
  note         = {Machine review of arXiv:2412.14142}
}
read the original abstract

Modern challenges of robustness, fairness, and decision-making in machine learning have led to the formulation of multi-distribution learning (MDL) frameworks in which a predictor is optimized across multiple distributions. We study the calibration properties of MDL to better understand how the predictor performs uniformly across the multiple distributions. Through classical results on decomposing proper scoring losses, we first derive the Bayes optimal rule for MDL, demonstrating that it maximizes the generalized entropy of the associated loss function. Our analysis reveals that while this approach ensures minimal worst-case loss, it can lead to non-uniform calibration errors across the multiple distributions and there is an inherent calibration-refinement trade-off, even at Bayes optimality. Our results highlight a critical limitation: despite the promise of MDL, one must use caution when designing predictors tailored to multiple distributions so as to minimize disparity.

Figures

Figures reproduced from arXiv: 2412.14142 by the authors.

Figure 1
Figure 1. Calibration disparity intuition in MDL: For a forecast hx, the calibration error is defined by the (generalized) entropy function Hℓ as a hyperplane at hx evaluated at Q (y | hx) minus Hℓ (Q (y | hx)). To summarize, by the definition of Q∗ , the upper￾bound on the calibration error is positive, and under only the distributional assumptions the calibration error can be perfectly assumed to be zero. Thus, the generali… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

62 extracted references · 58 canonical work pages

  1. [1]

    On the coherence of expected shortfall, 2002

    Carlo Acerbi and Dirk Tasche. On the coherence of expected shortfall, 2002

    work page 2002

  2. [2]

    Out of distribution generalization in machine learning

    Martin Arjovsky. Out of distribution generalization in machine learning. PhD thesis, New York University, 2020

    work page 2020

  3. [3]

    Coherent measures of risk

    Philippe Artzner, Freddy Delbaen, Jean-Marc Eber, and David Heath. Coherent measures of risk. Mathematical Finance, 1999

    work page 1999

  4. [4]

    Exploiting task relatedness for multiple task learning

    Shai Ben-David and Reba Schuller. Exploiting task relatedness for multiple task learning. In Learning Theory and Kernel Machines: Conference on Learning Theory and Kernel Workshop, COLT/Kernel, 2003

    work page 2003

  5. [5]

    Ben-Tal, L

    A. Ben-Tal, L. El Ghaoui, and A. Nemirovski. Robust Optimization. Princeton University Press, 2009

    work page 2009

  6. [6]

    Robust solutions of optimization problems affected by uncertain probabilities

    Aharon Ben-Tal, Dick Den Hertog, Anja De Waegenaere, Bertrand Melenberg, and Gijs Rennen. Robust solutions of optimization problems affected by uncertain probabilities. Management Science, 2013

    work page 2013

  7. [7]

    Generalizing from several related classification tasks to a new unlabeled sample

    Gilles Blanchard, Gyemin Lee, and Clayton Scott. Generalizing from several related classification tasks to a new unlabeled sample. Advances in Neural Information Processing Systems, 2011

    work page 2011

  8. [8]

    Domain generalization by marginal transfer learning

    Gilles Blanchard, Aniket Anand Deshmukh, Urun Dogan, Gyemin Lee, and Clayton Scott. Domain generalization by marginal transfer learning. Journal of Machine Learning Research, 2021

    work page 2021

Show all 62 references
  1. [9]

    Collaborative pac learning

    Avrim Blum, Nika Haghtalab, Ariel D Procaccia, and Mingda Qiao. Collaborative pac learning. Advances in Neural Information Processing Systems, 2017

    2017

  2. [10]

    Communication-aware collaborative learning

    Avrim Blum, Shelby Heinecke, and Lev Reyzin. Communication-aware collaborative learning. In AAAI Conference on Artificial Intelligence, 2021

    2021

  3. [11]

    The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming

    Lev M Bregman. The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR computational mathematics and mathematical physics, 1967

    1967

  4. [12]

    Reliability, sufficiency, and the decomposition of proper scores

    Jochen Br \"o cker. Reliability, sufficiency, and the decomposition of proper scores. Quarterly Journal of the Royal Meteorological Society: A journal of the atmospheric sciences, applied meteorology and physical oceanography, 2009

    2009

  5. [13]

    Gender shades: Intersectional accuracy disparities in commercial gender classification

    Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on Fairness, Accountability and Transparency, 2018

    2018

  6. [14]

    Philip Dawid

    A. Philip Dawid. The well-calibrated bayesian. Journal of the American Statistical Association, 1982

    1982

  7. [15]

    Degroot and Stephen E

    Morris H. Degroot and Stephen E. Fienberg. The comparison and evaluation of forecasters. The Statistician, 1983

    1983

  8. [16]

    Four facets of forecast felicity: Calibration, predictiveness, randomness and regret

    Rabanus Derr and Robert C Williamson. Four facets of forecast felicity: Calibration, predictiveness, randomness and regret. arXiv preprint arXiv:2401.14483, 2024

    2024 arXiv

  9. [17]

    Statistics of robust optimization: A generalized empirical likelihood approach

    John C Duchi, Peter W Glynn, and Hongseok Namkoong. Statistics of robust optimization: A generalized empirical likelihood approach. Mathematics of Operations Research, 2021

    2021

  10. [18]

    Outcome indistinguishability

    Cynthia Dwork, Michael P Kim, Omer Reingold, Guy N Rothblum, and Gal Yona. Outcome indistinguishability. In ACM SIGACT Symposium on Theory of Computing, 2021

    2021

  11. [19]

    u gelgen, Hamed Hassani, George J Pappas, and Bernhard Sch \

    Cian Eastwood, Alexander Robey, Shashank Singh, Julius Von K \"u gelgen, Hamed Hassani, George J Pappas, and Bernhard Sch \"o lkopf. Probable domain generalization via quantile risk minimization. Advances in Neural Information Processing Systems, 2022

    2022

  12. [20]

    The foundations of cost-sensitive learning

    Charles Elkan. The foundations of cost-sensitive learning. In International Joint Conference on Artificial Intelligence, 2001

    2001

  13. [21]

    Data models with two manifestations of imprecision

    Christian Fr \"o hlich and Robert C Williamson. Data models with two manifestations of imprecision. arXiv preprint arXiv:2404.09741, 2024 a

    2024 arXiv

  14. [22]

    Risk measures and upper probabilities: Coherence and stratification

    Christian Fr \"o hlich and Robert C Williamson. Risk measures and upper probabilities: Coherence and stratification. Journal of Machine Learning Research, 2024 b

    2024

  15. [23]

    Scoring rules and calibration for imprecise probabilities

    Christian Fr \"o hlich and Robert C Williamson. Scoring rules and calibration for imprecise probabilities. arXiv preprint arXiv:2410.23001, 2024 c

    2024 arXiv

  16. [24]

    Williamson

    Christian Fr \"o hlich and Robert C. Williamson. Risk measures and upper probabilities: Coherence and stratification. Journal of Machine Learning Research, 2024

    2024

  17. [25]

    Strictly proper scoring rules, prediction, and estimation

    Tilmann Gneiting and Adrian E Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 2007

    2007

  18. [26]

    Omnipredictors

    Parikshit Gopalan, Adam Tauman Kalai, Omer Reingold, Vatsal Sharan, and Udi Wieder. Omnipredictors. arXiv preprint arXiv:2109.05389, 2021

    2021 arXiv

  19. [27]

    Igor I. Gorban. The Statistical Stability Phenomenon. Springer Cham, 2017

    2017

  20. [28]

    Safe probability

    Peter Gr \"u nwald. Safe probability. Journal of Statistical Planning and Inference, 2018

    2018

  21. [29]

    The minimum description length principle

    Peter D Gr \"u nwald. The minimum description length principle. 2007

    2007

  22. [30]

    Game theory, maximum entropy, minimum discrepancy and robust bayesian decision theory

    Peter D Gr \"u nwald and A Philip Dawid. Game theory, maximum entropy, minimum discrepancy and robust bayesian decision theory. 2004

    2004

  23. [31]

    On calibration of modern neural networks

    Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International Conference on Machine Learning, 2017

    2017

  24. [32]

    On-demand sampling: Learning optimally from multiple distributions

    Nika Haghtalab, Michael Jordan, and Eric Zhao. On-demand sampling: Learning optimally from multiple distributions. Advances in Neural Information Processing Systems, 2022

    2022

  25. [33]

    The reference class problem is your problem too

    Alan H\' a jek. The reference class problem is your problem too. Synthese, 2007

    2007

  26. [34]

    Fairness without demographics in repeated loss minimization

    Tatsunori Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. Fairness without demographics in repeated loss minimization. In International Conference on Machine Learning, 2018

    2018

  27. [35]

    Multicalibration: Calibration for the ( C omputationally-identifiable) masses

    Ursula Hebert-Johnson, Michael Kim, Omer Reingold, and Guy Rothblum. Multicalibration: Calibration for the ( C omputationally-identifiable) masses. In International Conference on Machine Learning, 2018

    2018

  28. [36]

    On the richness of calibration

    Benedikt H \"o ltgen and Robert C Williamson. On the richness of calibration. In ACM Conference on Fairness, Accountability, and Transparency, 2023

    2023

  29. [37]

    Toward efficient agnostic learning

    Michael J Kearns, Robert E Schapire, and Linda M Sellie. Toward efficient agnostic learning. In Workshop on Computational Learning Theory, 1992

    1992

  30. [38]

    U-calibration: Forecasting for an unknown agent

    Bobby Kleinberg, Renato Paes Leme, Jon Schneider, and Yifeng Teng. U-calibration: Forecasting for an unknown agent. In Conference on Learning Theory, 2023

    2023

  31. [39]

    Novel decompositions of proper scoring rules for classification: score adjustment as precursor to calibration

    Meelis Kull and Peter Flach. Novel decompositions of proper scoring rules for classification: score adjustment as precursor to calibration. 2015

    2015

  32. [40]

    On law invariant coherent risk measures

    Shigeo Kusuoka. On law invariant coherent risk measures. Springer Japan, 2001

    2001

  33. [41]

    Domain adaptation with multiple sources

    Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh. Domain adaptation with multiple sources. In Advances in Neural Information Processing Systems, 2008

    2008

  34. [42]

    Minimax pareto fairness: A multi objective perspective

    Natalia Martinez, Martin Bertran, and Guillermo Sapiro. Minimax pareto fairness: A multi objective perspective. In International Conference on Machine Learning, 2020

    2020

  35. [43]

    Stochastic optimization for spectral risk measures

    Ronak Mehta, Vincent Roulet, Krishna Pillutla, Lang Liu, and Zaid Harchaoui. Stochastic optimization for spectral risk measures. In International Conference on Artificial Intelligence and Statistics, 2023

    2023

  36. [44]

    Si Yi Meng and Robert M. Gower. A model-based method for minimizing cvar and beyond. In International Conference on Machine Learning, 2023

    2023

  37. [45]

    Agnostic federated learning

    Mehryar Mohri, Gary Sivek, and Ananda Theertha Suresh. Agnostic federated learning. In International Conference on Machine Learning, 2019

    2019

  38. [46]

    Improved algorithms for collaborative pac learning

    Huy Nguyen and Lydia Zakynthinou. Improved algorithms for collaborative pac learning. In Advances in Neural Information Processing Systems, 2018

    2018

  39. [47]

    Calibration for decision making: A principled approach to trustworthy ml, 2024

    Georgy Noarov and Aaron Roth. Calibration for decision making: A principled approach to trustworthy ml, 2024. Blog post

    2024

  40. [48]

    Dissecting racial bias in an algorithm used to manage the health of populations

    Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. Dissecting racial bias in an algorithm used to manage the health of populations. Science, 2019

    2019

  41. [49]

    Proper scoring rules and bregman divergence

    Evgeni Y Ovcharov. Proper scoring rules and bregman divergence. 2018

    2018

  42. [50]

    Frameworks and results in distributionally robust optimization

    Hamed Rahimian and Sanjay Mehrotra. Frameworks and results in distributionally robust optimization. Open Journal of Mathematical Optimization, 2022

    2022

  43. [51]

    On measures of entropy and information

    Alfr \'e d R \'e nyi. On measures of entropy and information. 1961

    1961

  44. [52]

    Convex analysis

    RT Rockafellar. Convex analysis. Princeton Mathematical Series, 1970

    1970

  45. [53]

    Coherent approaches to risk in optimization under uncertainty

    Terry Rockafellar. Coherent approaches to risk in optimization under uncertainty. 2007

    2007

  46. [54]

    Multi-group agnostic pac learnability

    Guy N Rothblum and Gal Yona. Multi-group agnostic pac learnability. In International Conference on Machine Learning, 2021

    2021

  47. [55]

    Hashimoto, and Percy Liang

    Shiori Sagawa*, Pang Wei Koh*, Tatsunori B. Hashimoto, and Percy Liang. Distributionally robust neural networks. In International Conference on Learning Representations, 2020

    2020

  48. [56]

    Conditional Value-at-Risk (CVaR)

    Gaia Serraino and Stanislav Uryasev. Conditional Value-at-Risk (CVaR). Springer US, 2013

    2013

  49. [57]

    Evaluating model calibration in classification

    Juozas Vaicenavicius, David Widmann, Carl Andersson, Fredrik Lindsten, Jacob Roll, and Thomas Sch\" o n. Evaluating model calibration in classification. In International Conference on Artificial Intelligence and Statistics, 2019

    2019

  50. [58]

    A theory of the learnable

    Leslie G Valiant. A theory of the learnable. Communications of the ACM, 1984

    1984

  51. [59]

    The Wasserstein distances

    C \'e dric Villani. The Wasserstein distances. Springer Berlin Heidelberg, Berlin, Heidelberg, 2009

    2009

  52. [60]

    Optimal multi-distribution learning

    Zihan Zhang, Wenhao Zhan, Yuxin Chen, Simon S Du, and Jason D Lee. Optimal multi-distribution learning. In Conference on Learning Theory, 2024

    2024

  53. [61]

    Calibrating predictions to decisions: A novel approach to multi-class calibration

    Shengjia Zhao, Michael Kim, Roshni Sahoo, Tengyu Ma, and Stefano Ermon. Calibrating predictions to decisions: A novel approach to multi-class calibration. Advances in Neural Information Processing Systems, 2021

    2021

  54. [62]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.