Algorithmic Expert Aggregation

Hanrui Zhang; Wei Tang

Algorithmic Expert Aggregation

T0 review · 0 major / 4 minor · reviewed 2026-07-10 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read When output experts may randomize, both finding an undominated calibrated refinement of a target and nearly minimizing proper loss among such refinements can be done efficiently; deterministic outputs make even the search problem NP-hard.

desk verdict Clean algorithmic dichotomy for calibrated expert aggregation: poly-time randomized search/OPT via observable cones, NP-hard deterministic even with two experts. read the letter →

arxiv 2607.08744 v1 pith:DEFZQYWS submitted 2026-07-09 cs.GT cs.DS

Wei Tang , Hanrui Zhang This is my paper · ORCID

classification cs.GTcs.DS

keywords expertaggregationBlackwelldominancecalibrationforecastproperlosscomputationalcomplexityobservablelinearinformation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper reframes forecast aggregation as expert aggregation: given calibrated input experts and a known prior over states, but without the latent Bayes probabilities, build a new calibrated expert that Blackwell-refines a target and cannot be improved further with the available information. Constructibility is completely determined by an observable linear system whose nonnegative cone tells exactly which output prediction components can be certified. With randomization allowed, both the search for an undominated refinement and additive approximation of proper-loss optimization admit polynomial-time algorithms. Forcing the output expert to be deterministic collapses the problem to an integrality question that is already NP-hard with two experts and a constant base-rate target, and admits no multiplicative PTAS for Brier loss unless P equals NP. The result therefore draws a sharp computational line between randomized and deterministic calibrated aggregation.

What carries the argument

The observable nonnegative cone K = Row(A) ∩ R₊ⁿ, where A is the signal-state matrix whose rows are the input experts’ state-wise reporting-probability vectors; membership in K exactly certifies which output atoms can be given identifiable posterior means, and every algorithmic and hardness result is built on linear optimization or binary-vector search inside this cone.

What would settle it

Exhibit two distinct Bayes-probability vectors that induce identical input-expert reports yet assign different conditional outcome means to some nonnegative vector outside the claimed observable cone; such a counter-example would falsify the completeness of the linear characterization.

Watch

Extended reading notes

Core claim

A calibrated output expert is constructible from the observed inputs if and only if its reporting components lie in the observable nonnegative cone generated by the row space of the input experts’ state-wise reporting matrix; under this characterization, randomized aggregation admits efficient algorithms for both undominated refinement search and additive proper-loss optimization, while deterministic aggregation is computationally intractable even in the simplest two-expert setting.

Load-bearing premise

The claim that every constructible expert is completely characterized by components lying inside the observable nonnegative cone rests on the premise that linear consequences of the input experts’ calibration equations recover every identifiable outcome mass.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The new piece is the expert-aggregation problem itself: take reduced-form calibrated experts plus the prior, never see the latent Bayes probabilities, and build a new calibrated expert that Blackwell-refines a target and is undominated (or nearly loss-optimal) among constructible ones. The linear characterization is clean—observable space S = Row(A) from the input reporting vectors, cone K = S ∩ R_{+}^{n}, and constructible experts are exactly those assembled from atoms in K. That turns the whole thing into linear algebra plus martingale couplings.

What they deliver is a sharp randomized/deterministic split. Randomized Search-Aggregation is poly-time: extreme-ray reduction to a finite prediction set, lex objective whose optimum is undominated, small-support vertex bound n+2(M+1) via pruned binding constraints in the exponential ExProg, then Stern-Brocot search with FeasProg oracles that recover the bounded-rational predictions. OPT gets an additive FPTAS by piecewise-linear upper approximation of the Bayes risk plus source-labelled LPs that enforce Blackwell exactly. Deterministic is hard even for two experts and a constant base-rate target: they encode Restricted SubsetSum into the observable binary vectors of an amplified hard instance, so Search-DetAgg is NP-hard and Brier OPT-DetAgg has no multiplicative PTAS. Proofs look complete and self-contained under the finite-support rational model; the stress-test found no load-bearing gap, and I agree.

Soft spots are minor and mostly modeling. Everything is finite-support and rational-input; continuous or irrational cases are left open. Regularity of the proper loss is needed for the FPTAS, but that is standard and covers Brier/log. The multi-target extension is sketched and works for fixed |T|. No experiments, no code, but this is pure theory and does not claim otherwise. Citations sit correctly inside robust aggregation and information design.

This is for people who care about algorithmic information design, modular expert systems, or Blackwell orderings under partial observability. It deserves a serious referee. I would bring it to reading group and expect to cite the dichotomy.

Referee Report

0 major / 4 minor

Summary. The paper introduces expert aggregation: given a known prior over a finite state space and a collection of calibrated (posterior-consistent) input experts, but without access to the latent Bayes probabilities, construct a new calibrated expert that is undominated under Blackwell dominance among all experts that can be assembled from the observable information, optionally refining a designated target expert, and (when a proper loss is given) nearly minimizing expected proper loss among such refinements. Constructibility is characterized exactly by membership of reporting components in the observable nonnegative cone K = Row(A) ∩ R₊ⁿ induced by the input experts’ state-wise reporting vectors. The main algorithmic dichotomy is that randomized outputs admit a polynomial-time algorithm for Search-Aggregation (via extreme-ray reduction, lexicographic optimization, small-support vertex counting, and Stern-Brocot feasibility oracles) and an additive FPTAS for OPT-Aggregation (via source-labelled martingale LPs with piecewise-linear upper approximations of regular proper losses), while deterministic outputs are NP-hard for Search-DetAgg even with two experts and a constant base-rate target, and admit no multiplicative PTAS for Brier-loss OPT-DetAgg unless P=NP (via an amplified SubsetSum construction that forces nontrivial binary vectors into the observable space).

Significance. If the results hold, the paper cleanly separates a new, modular notion of expert aggregation (calibrated, Blackwell-undominated, and reusable without access to the original experts) from classical and robust forecast aggregation. The randomized side supplies concrete poly-time and FPTAS algorithms that work from only the prior and the input reporting rules; the deterministic side supplies matching hardness even in highly restricted two-expert instances. The linear-observability characterization (Proposition 3.1, Lemmas 3.2–3.4) and the subsequent reductions (extreme-ray finite support, n+2(M+1) sparsity, Stern-Brocot recovery, source-labelled martingales, amplified SubsetSum) are self-contained and free of circularity. These contributions are of clear interest to algorithmic game theory, information design, and forecast aggregation.

minor comments (4)

The numerical bound (nW)^{10n^{3}} in Lemma 4.10 is extremely loose; a short remark that any poly(n) exponent suffices for the Stern-Brocot argument would improve readability without changing the claim.
Figure 1 and the accompanying three-state example are helpful; a second small worked example that also illustrates the source-labelled LP (LPε) would make the FPTAS more accessible.
Definition 5.1 (regular proper loss) is used only for the FPTAS; a one-sentence pointer that Brier and log loss satisfy it (via Corollary 5.2) already appears, but placing it earlier would help readers who skip the appendix.
A few typographical slips remain (e.g., “Aggrega tion”, “ExRayDec”, occasional missing spaces after periods). A light copy-edit pass would clean them.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: constructibility, algorithms, and hardness are self-contained reductions from stated definitions and linear algebra, not fits or self-citation chains.

full rationale

The paper is a pure theory/algorithms work. Constructible experts are defined (Def. 3.3) as assemblies of atoms from the observable cone K = Row(A) ∩ R⁺ⁿ induced by input calibration equations Ay = b; Prop. 3.1 and Lem. 3.2–3.4 then prove that linear observability of outcome mass is exactly membership in S and that such atoms remain posterior-consistent for every latent q compatible with the inputs. That is a definition-plus-proof, not a claim that recovers a fitted target by construction. Search-Aggregation is reduced to finite-support lexicographic optimization over extreme rays of K, with sparsity from vertex counting in ExProg and exact recovery of bounded rationals via Stern-Brocot oracles on FeasProg; OPT-Aggregation is reduced to a source-labelled LP with a piecewise-linear upper envelope of a regular proper loss. Deterministic hardness is a standard SubsetSum reduction that encodes nontrivial binary vectors in Shard. None of these steps renames a fitted parameter as a prediction, imports uniqueness from the authors’ prior work as an external fact, or smuggles an ansatz via self-citation. Background citations (Blackwell, proper losses, LP complexity) are external and non-load-bearing for the dichotomy. Score 0 is therefore appropriate.

Assumptions & free parameters 0 free parameters · 5 assumptions · 2 invented entities

The work rests on standard mathematical objects (Blackwell order, proper losses, polyhedral cones, linear programming) plus domain modeling choices that experts are reduced-form calibrated maps and that the aggregator sees only the prior and those maps. No free parameters are fitted to data; the only numerical constants that appear (e.g., block size L=m, η=1/16 in the hardness construction) are chosen for the reduction and do not affect the asymptotic claims.

assumptions (5)

standard math Blackwell dominance of prediction distributions is equivalent to mean-preserving spread / convex order of the induced posteriors (Definition 2.1, Lemma 5.3).
Classical comparison of experiments; used throughout to rank experts.
domain assumption Input experts report Bayesian posterior means (are calibrated) and are given as reduced-form stochastic maps Ω → Δ([0,1]).
Stated in Section 2 and used to obtain the linear system Ay=b.
domain assumption The aggregator observes the prior λ and the input experts but not the latent Bayes probabilities (q_i).
Core information constraint of the model (Introduction and Section 2).
ad hoc to paper A proper loss is regular if its Bayes risk admits a polynomial-size piecewise-linear upper approximation (Definition 5.1).
Needed for the additive FPTAS; satisfied by common losses (Brier, log) under mild smoothness (Corollary 5.2).
standard math All input numbers are rational with numerator/denominator bounded by W (standard encoding).
Used for the Stern-Brocot search and bit-complexity claims (Lemma 4.10).

invented entities (2)

Observable linear space S = Row(A) and observable cone K = S ∩ R_+^n independent evidence
purpose: Characterize exactly which state-wise reporting vectors have identifiable outcome mass and can therefore be used as atoms of a constructible expert.
Defined from the input experts' reporting matrix; the paper proves (Proposition 3.1, Lemmas 3.2–3.4) that constructibility is equivalent to decomposing the all-ones vector into nonzero elements of K.
Expert aggregation problem (Search-Aggregation / OPT-Aggregation) independent evidence
purpose: Formalize the task of producing a calibrated, Blackwell-undominated (or loss-optimal) output expert from reduced-form inputs.
New problem statement relative to classical forecast aggregation; the algorithmic results are stated for these problems.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Algorithmic Expert Aggregation." pith.science (2026). https://pith.science/paper/DEFZQYWS

@misc{pith2026260708744,
  author       = {Pith},
  title        = {Pith review of: Algorithmic Expert Aggregation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DEFZQYWS}},
  note         = {Machine review of arXiv:2607.08744}
}

abstract

Forecast aggregation aims to combine information from multiple Bayesian experts' forecasts into an aggregate forecast. In much of this literature, however, the aggregate forecast is optimized for a particular loss or robustness criterion and need not itself be calibrated with respect to the outcome. We introduce and study expert aggregation, where the goal is instead to aggregate Bayesian experts into a new expert that continues to provide calibrated forecasts. In particular, we consider a setting where each input expert reports calibrated predictions, and the aggregator observes the prior distribution over states, and the input experts, but not the underlying Bayes probabilities of the states. We ask whether one can (i) construct a calibrated output expert that Blackwell refines a target expert and cannot be further Blackwell improved using the available information; and (ii) when a proper loss is specified, compute a nearly loss-optimal expert among all such refinements. We formulate calibrated experts as reduced-form information structures and measure refinement by Blackwell dominance of the induced prediction distributions. We characterize the constructible output experts through observable linear information: the input experts generate a linear system whose row space determines which calibrated output predictions are identifiable, and a new expert is constructible exactly when its predictions lie in the associated observable nonnegative cone. We establish a sharp algorithmic picture. When randomized output experts are allowed, both questions above admit efficient algorithms. In contrast, deterministic output experts are computationally intractable: deciding whether a deterministic calibrated refinement exists is $\mathsf{NP}$-hard, and deterministic proper-loss optimization admits no multiplicative PTAS unless $\mathsf{P}=\mathsf{NP}$.

Figures

Figures reproduced from arXiv: 2607.08744 by the authors.

**Figure 2.** Proof flow for Theorem 4.1. The abstract lexicographical program AbsProg identifies the canonical undominated output expert. The source-labelled prototype LexProg motivates the feasibility oracle, while ExProg and Proposition 4.9 provide the structural sparsity of prediction levels needed to make the algorithm polynomial-time. Theorem 4.1. There is a polynomial-time algorithm (Algorithm 1) which solves the SearchAg… view at source ↗

**Figure 3.** Graphical illustration of type-2 constraints in ExProg. For readability, we omit the superscript fin and write pu instead of p fin u . The dashed vertical lines mark turning points of Ig and Ifτ . Note that between pτ,2 and pτ,3, 4 type-2 constraints are binding, but only the two corresponding to p3 and p6 are essential. • The n constraints induced by P u,v αu,v · ru,v = 1. These constraints are always binding. We d… view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

95 extracted references · 95 canonical work pages

[1]

The Thirty Ninth Annual Conference on Learning Theory , pages=

Calibeating Made Simple , author=. The Thirty Ninth Annual Conference on Learning Theory , pages=. 2026 , organization=

work page 2026
[2]

Proceedings of the 2026 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA) , pages=

Collaborative prediction: Tractable information aggregation via agreement , author=. Proceedings of the 2026 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA) , pages=. 2026 , organization=

work page 2026
[3]

Proceedings of the 57th Annual ACM Symposium on Theory of Computing , pages=

Tractable agreement protocols , author=. Proceedings of the 57th Annual ACM Symposium on Theory of Computing , pages=

work page
[4]

14th Innovations in Theoretical Computer Science Conference (ITCS 2023) , pages=

False Consensus, Information Theory, and Prediction Markets , author=. 14th Innovations in Theoretical Computer Science Conference (ITCS 2023) , pages=. 2023 , organization=

work page 2023
[5]

The Annals of Statistics , volume=

Agreeing to Disagree , author=. The Annals of Statistics , volume=

work page
[6]

Proceedings of the thirty-seventh annual ACM symposium on Theory of computing , pages=

The complexity of agreement , author=. Proceedings of the thirty-seventh annual ACM symposium on Theory of computing , pages=

work page
[7]

Proceedings of the 24th ACM Conference on Economics and Computation , pages=

Agreement implies accuracy for substitutable signals , author=. Proceedings of the 24th ACM Conference on Economics and Computation , pages=

work page
[8]

Proceedings of the AAAI Conference on Artificial Intelligence , volume=

Information elicitation mechanisms for statistical estimation , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

work page

Show all 95 references

[9]

A definition , author=

Increasing risk: I. A definition , author=. Uncertainty in economics , pages=. 1978 , publisher=

1978
[10]

The annals of mathematical statistics , pages=

Equivalent comparisons of experiments , author=. The annals of mathematical statistics , pages=. 1953 , publisher=

1953
[11]

American Economic Review , volume=

Comparisons of signals , author=. American Economic Review , volume=. 2024 , publisher=

2024
[12]

Econometrica , volume=

Information hierarchies , author=. Econometrica , volume=. 2022 , publisher=

2022
[13]

Proceedings of the forty-eighth annual ACM symposium on Theory of Computing , pages=

Algorithmic bayesian persuasion , author=. Proceedings of the forty-eighth annual ACM symposium on Theory of Computing , pages=
[14]

Journal of Economic Literature , volume=

Information design: A unified perspective , author=. Journal of Economic Literature , volume=. 2019 , publisher=

2019
[15]

American Economic Review , volume=

Bayesian persuasion , author=. American Economic Review , volume=. 2011 , publisher=

2011
[16]

2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS) , pages=

Informational substitutes , author=. 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS) , pages=. 2016 , organization=

2016
[17]

Calcul des rouages par approximation: nouvelle m

Brocot, Achille , year=. Calcul des rouages par approximation: nouvelle m
[18]

Ueber eine zahlentheoretische Funktion , author=
[19]

2006 , publisher=

Algorithm design , author=. 2006 , publisher=

2006
[20]

Get Online Inferences from a Custom Trained Model , howpublished =
[21]

arXiv preprint arXiv:2108.07258 , year=

On the opportunities and risks of foundation models , author=. arXiv preprint arXiv:2108.07258 , year=

arXiv
[22]

25th USENIX security symposium (USENIX Security 16) , pages=

Stealing machine learning models via prediction \ APIs \ , author=. 25th USENIX security symposium (USENIX Security 16) , pages=
[23]

About Expected Click-Through Rate in Google Ads , howpublished =
[24]

Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining , pages=

Ad click prediction: a view from the trenches , author=. Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining , pages=
[25]

Proceedings of the eighth international workshop on data mining for online advertising , pages=

Practical lessons from predicting clicks on ads at facebook , author=. Proceedings of the eighth international workshop on data mining for online advertising , pages=
[26]

International journal of forecasting , volume=

Combining forecasts: A review and annotated bibliography , author=. International journal of forecasting , volume=. 1989 , publisher=

1989
[27]

Journal of the Royal Statistical Society: Series D (The Statistician) , volume=

The comparison and evaluation of forecasters , author=. Journal of the Royal Statistical Society: Series D (The Statistician) , volume=. 1983 , publisher=

1983
[28]

Econometrica , volume=

Do markets favor agents able to make accurate predictions? , author=. Econometrica , volume=. 2000 , publisher=

2000
[29]

arXiv preprint arXiv:2601.19862 , year=

Calibration without Ground Truth , author=. arXiv preprint arXiv:2601.19862 , year=
[30]

Neural networks , volume=

Stacked generalization , author=. Neural networks , volume=. 1992 , publisher=

1992
[31]

International workshop on multiple classifier systems , pages=

Ensemble methods in machine learning , author=. International workshop on multiple classifier systems , pages=. 2000 , organization=

2000
[32]

Machine learning , volume=

Random forests , author=. Machine learning , volume=. 2001 , publisher=

2001
[33]

Machine learning , volume=

Bagging predictors , author=. Machine learning , volume=. 1996 , publisher=

1996
[34]

The Annals of Mathematical Statistics , pages=

The opinion pool , author=. The Annals of Mathematical Statistics , pages=. 1961 , publisher=

1961
[35]

Management Science , volume =

Hidden experts in the crowd: Using meta-predictions to leverage expertise in single-question prediction problems , author =. Management Science , volume =. 2022 , month = jan, publisher =

2022
[36]

Management Science , volume =

Extracting the wisdom of crowds when information is shared , author =. Management Science , volume =. 2019 , month = may, publisher =

2019
[37]

Prelec, Drazen , journal =. A. 2004 , month = oct, publisher =

2004
[38]

Nature , volume =

A solution to the single-question crowd wisdom problem , author =. Nature , volume =. 2017 , month = jan, publisher =

2017
[39]

Journal of the American Statistical Association , volume =

Modeling probability forecasts via information diversity , author =. Journal of the American Statistical Association , volume =. 2016 , month = dec, publisher =

2016
[40]

Statistics & Probability Letters , volume =

Ernst, Philip and Pemantle, Robin and Satop. Statistics & Probability Letters , volume =. 2016 , month = dec, publisher =

2016
[41]

International Journal of Forecasting , volume =

Combining multiple probability predictions using a simple logit model , author =. International Journal of Forecasting , volume =. 2014 , month = apr, publisher =

2014
[42]

SIAM Journal on Algebraic Discrete Methods , volume =

A characterization of weighted arithmetic means , author =. SIAM Journal on Algebraic Discrete Methods , volume =. 1980 , month = sep, publisher =

1980
[43]

A characterization theorem for externally

Genest, Christian , journal =. A characterization theorem for externally. 1984 , month = sep, publisher =

1984
[44]

Journal of Economic Theory , volume =

Combining forecasts in the presence of ambiguity over correlation structures , author =. Journal of Economic Theory , volume =. 2022 , month = jan, publisher =

2022
[45]

Theoretical Economics , volume =

A maximum likelihood approach to combining forecasts , author =. Theoretical Economics , volume =
[46]

Proceedings of the National Academy of Sciences , volume =

Robust Forecast Aggregation , author =. Proceedings of the National Academy of Sciences , volume =. 2018 , month = dec, publisher =

2018
[47]

Management Science , volume =

Prior-independent optimal auctions , author =. Management Science , volume =. 2020 , month = oct, publisher =

2020
[48]

Communications of the

Algorithmic game theory , author =. Communications of the. 2010 , month = jul, publisher =

2010
[49]

Management Science , volume =

A multiplicative formula for aggregating probability assessments , author =. Management Science , volume =. 1982 , month = oct, publisher =

1982
[50]

American Economic Journal: Microeconomics , volume =

A Random Dictator Is All You Need , author =. American Economic Journal: Microeconomics , volume =. 2025 , month = feb, doi =

2025
[51]

Soviet Mathematics Doklady , volume =

A polynomial time algorithm in linear programming , author =. Soviet Mathematics Doklady , volume =
[52]

Proceedings of the 29th AAAI Conference on Artificial Intelligence , series =

Elicitation for aggregation , author =. Proceedings of the 29th AAAI Conference on Artificial Intelligence , series =. 2015 , publisher =

2015
[53]

Proceedings of the 9th AAAI Conference on Human Computation and Crowdsourcing , series =

Forecast aggregation via peer prediction , author =. Proceedings of the 9th AAAI Conference on Human Computation and Crowdsourcing , series =. 2021 , publisher =

2021
[54]

Proceedings of the 22nd ACM Conference on Economics and Computation , series =

Robust Merging of Information , author =. Proceedings of the 22nd ACM Conference on Economics and Computation , series =. 2021 , publisher =

2021
[55]

Proceedings of the 25th ACM Conference on Economics and Computation , series =

The Surprising Benefits of Base Rate Neglect in Robust Aggregation , author =. Proceedings of the 25th ACM Conference on Economics and Computation , series =. 2024 , publisher =

2024
[56]

Proceedings of the 26th ACM Conference on Economics and Computation , series =

Algorithmic Robust Forecast Aggregation , author =. Proceedings of the 26th ACM Conference on Economics and Computation , series =. 2025 , publisher =

2025
[57]

Are you smarter than a random expert?

Neyman, Eric and Roughgarden, Tim , booktitle =. Are you smarter than a random expert?. 2022 , publisher =

2022
[58]

Proceedings of the 45th Annual ACM Symposium on Theory of Computing , series =

Prior-independent mechanisms for scheduling , author =. Proceedings of the 45th Annual ACM Symposium on Theory of Computing , series =. 2013 , publisher =

2013
[59]

Proceedings of the ACM Web Conference 2024 , series =

Robust Decision Aggregation with Second-order Information , author =. Proceedings of the ACM Web Conference 2024 , series =. 2024 , publisher =

2024
[60]

arXiv preprint arXiv:2604.24517 , year=

Prior-Agnostic Robust Forecast Aggregation , author=. arXiv preprint arXiv:2604.24517 , year=

arXiv
[61]

Proceedings of the ACM Web Conference 2025 , series =

Robust Aggregation with Adversarial Experts , author =. Proceedings of the ACM Web Conference 2025 , series =. 2025 , publisher =

2025
[62]

Proceedings of the 11th ACM Conference on Electronic Commerce , series =

Revenue maximization with a single sample , author =. Proceedings of the 11th ACM Conference on Electronic Commerce , series =. 2010 , publisher =

2010
[63]

Proceedings of the 16th ACM Conference on Economics and Computation , series =

Randomization beats second price as a prior-independent auction , author =. Proceedings of the 16th ACM Conference on Economics and Computation , series =. 2015 , publisher =

2015
[64]

Proceedings of the 61st Annual Symposium on Foundations of Computer Science , series =

Benchmark design and prior-independent optimization , author =. Proceedings of the 61st Annual Symposium on Foundations of Computer Science , series =. 2020 , organization =

2020
[65]

Proceedings of the 23rd ACM Conference on Economics and Computation , series =

On the robustness of second-price auctions in prior-independent mechanism design , author =. Proceedings of the 23rd ACM Conference on Economics and Computation , series =. 2022 , publisher =

2022
[66]

Proceedings of the 22nd Annual ACM-SIAM Symposium on Discrete Algorithms , series =

Near-optimal no-regret algorithms for zero-sum games , author =. Proceedings of the 22nd Annual ACM-SIAM Symposium on Discrete Algorithms , series =. 2011 , organization =

2011
[67]

Robust forecast aggregation via additional queries , author =
[68]

Beyond majority voting:

Ai, Rui and Pan, Yuqi and Simchi-Levi, David and Tambe, Milind and Xu, Haifeng , howpublished =. Beyond majority voting:
[69]

The wisdom of the crowd and higher-order beliefs , author =
[70]

Correlation uncertainty: a decision-theoretic approach , author =
[71]

Measuring Informativeness Gap of (Mis) Calibrated Predictors , author =
[72]

Last-iterate convergence: Zero-sum games and constrained min-max optimization , author =
[73]

Near-optimal prior-independent online algorithms and the ski-rental problem , author =
[74]

Online learning and solving infinite games with an

Assos, Amit and Attias, Idan and Dagan, Yuval and Daskalakis, Constantinos and Fishelson, Maxwell , institution =. Online learning and solving infinite games with an
[75]

1963 , publisher =

Linear programming and extensions , author =. 1963 , publisher =

1963
[76]

The Oxford Handbook of Probability and Philosophy , editor =

Probabilistic opinion pooling , author =. The Oxford Handbook of Probability and Philosophy , editor =. 2016 , pages =

2016
[77]

Games and Economic Behavior , volume=

Calibrated learning and correlated equilibrium , author=. Games and Economic Behavior , volume=
[78]

Journal of the American statistical Association , volume=

The well-calibrated Bayesian , author=. Journal of the American statistical Association , volume=. 1982 , publisher=

1982
[79]

Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

Deepseekmoe: Towards ultimate expert specialization in mixture-of-experts language models , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=
[80]

arXiv preprint arXiv:2412.19437 , year=

Deepseek-v3 technical report , author=. arXiv preprint arXiv:2412.19437 , year=

arXiv
[81]

Neural computation , volume=

Adaptive mixtures of local experts , author=. Neural computation , volume=. 1991 , publisher=

1991
[82]

2016 , publisher=

Handbook of computational social choice , author=. 2016 , publisher=

2016
[83]

, author=

Learning from crowds. , author=. Journal of machine learning research , volume=
[84]

2006 , publisher=

Prediction, learning, and games , author=. 2006 , publisher=

2006
[85]

Information and computation , volume=

The weighted majority algorithm , author=. Information and computation , volume=. 1994 , publisher=

1994
[86]

Machine learning , volume=

The strength of weak learnability , author=. Machine learning , volume=. 1990 , publisher=

1990
[87]

The annals of statistics , volume=

Additive logistic regression: a statistical view of boosting (with discussion and a rejoinder by the authors) , author=. The annals of statistics , volume=. 2000 , publisher=

2000
[88]

Information and computation , volume=

Boosting a weak learning algorithm by majority , author=. Information and computation , volume=. 1995 , publisher=

1995
[89]

Journal of computer and system sciences , volume=

A decision-theoretic generalization of on-line learning and an application to boosting , author=. Journal of computer and system sciences , volume=. 1997 , publisher=

1997
[90]

Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=

Probabilistic forecasts, calibration and sharpness , author=. Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=. 2007 , publisher=

2007
[91]

Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=

Combining probability forecasts , author=. Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=. 2010 , publisher=

2010
[92]

Journal of the operational research society , volume=

The combination of forecasts , author=. Journal of the operational research society , volume=. 1969 , publisher=

1969
[93]

2002 , publisher=

Computers and intractability , author=. 2002 , publisher=

2002
[94]

1998 , publisher=

Theory of linear and integer programming , author=. 1998 , publisher=

1998
[95]

1994 , publisher=

Concrete Mathematics: A Foundation for Computer Science , author=. 1994 , publisher=

1994

Pith tools

Reviewed July 10, 2026 · model on record in the stance chip above.

[1] [1]

The Thirty Ninth Annual Conference on Learning Theory , pages=

Calibeating Made Simple , author=. The Thirty Ninth Annual Conference on Learning Theory , pages=. 2026 , organization=

work page 2026

[2] [2]

Proceedings of the 2026 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA) , pages=

Collaborative prediction: Tractable information aggregation via agreement , author=. Proceedings of the 2026 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA) , pages=. 2026 , organization=

work page 2026

[3] [3]

Proceedings of the 57th Annual ACM Symposium on Theory of Computing , pages=

Tractable agreement protocols , author=. Proceedings of the 57th Annual ACM Symposium on Theory of Computing , pages=

work page

[4] [4]

14th Innovations in Theoretical Computer Science Conference (ITCS 2023) , pages=

False Consensus, Information Theory, and Prediction Markets , author=. 14th Innovations in Theoretical Computer Science Conference (ITCS 2023) , pages=. 2023 , organization=

work page 2023

[5] [5]

The Annals of Statistics , volume=

Agreeing to Disagree , author=. The Annals of Statistics , volume=

work page

[6] [6]

Proceedings of the thirty-seventh annual ACM symposium on Theory of computing , pages=

The complexity of agreement , author=. Proceedings of the thirty-seventh annual ACM symposium on Theory of computing , pages=

work page

[7] [7]

Proceedings of the 24th ACM Conference on Economics and Computation , pages=

Agreement implies accuracy for substitutable signals , author=. Proceedings of the 24th ACM Conference on Economics and Computation , pages=

work page

[8] [8]

Proceedings of the AAAI Conference on Artificial Intelligence , volume=

Information elicitation mechanisms for statistical estimation , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

work page