pith. sign in

arxiv: 2102.02666 · v3 · submitted 2021-02-04 · 💰 econ.TH · cs.GT

The Wisdom of the Crowd and Higher-Order Beliefs

Yi-Chun Chen , Manuel Mueller-Frank , Mallesh M Pai This is my paper

Pith reviewed 2026-05-24 13:26 UTC · model grok-4.3

classification 💰 econ.TH cs.GT
keywords belief aggregationwisdom of the crowdhigher-order beliefsinformation aggregationmisspecified beliefspopulation mean based aggregationlinear regression
0
0 comments X

The pith

PMBA infers the true state from agents' beliefs and some expectations of the average belief in large populations without knowing the information structure.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes Population-Mean-Based Aggregation (PMBA) as a procedure that lets a principal combine information about an unknown state from many agents. It requires only that agents report their beliefs about the state and that some report their expectations of what the population average belief will be. In large populations, for any finite set of possible states, PMBA recovers the true state in probability or almost surely even when agents hold misspecified beliefs and even without the principal knowing how information is distributed among them. A reader would care because the method provides a practical way to extract accurate conclusions from crowd reports while sidestepping the need to model the full belief-formation process. The procedure can also be recast as linear regression, which lets existing inference results apply to smaller groups of agents.

Core claim

PMBA infers the true state in probability or almost surely in a large population for any finite number of possible states, under weak assumptions on the information structure, allowing individual agents' beliefs to be misspecified. The procedure requires agents to communicate their beliefs about the state and some agents to communicate their expectations of the population average belief. It can be reinterpreted as a linear regression procedure and extends to finite populations by reusing results on inference in linear models. A novel experiment shows that its real-world performance exceeds that of existing aggregation methods.

What carries the argument

Population-Mean-Based Aggregation (PMBA), which aggregates reported beliefs about the state together with some agents' expectations of the population average belief to recover the true state without requiring knowledge of the full information structure.

If this is right

  • PMBA recovers the true state even when agents hold misspecified beliefs.
  • The method extends to finite numbers of agents by recasting it as linear regression and applying existing inference results.
  • Real-world performance exceeds existing aggregation methods in a novel experiment.
  • The procedure works for any finite number of possible states under weak assumptions on the information structure.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same inputs might allow recovery of the state in settings where agents update beliefs repeatedly over time.
  • PMBA could be tested against other linear aggregation rules in environments where higher-order expectations are directly observable.
  • The regression reinterpretation suggests checking robustness when the population average belief is estimated with sampling error rather than reported directly.

Load-bearing premise

A sufficient number of agents communicate their expectations of the population average belief and the population is large enough for concentration results to apply.

What would settle it

Run the procedure in a small population or with very few agents reporting expectations of the average belief and check whether the inferred state matches the true state with high probability.

read the original abstract

We propose a new simple procedure called Population-Mean-Based Aggregation (PMBA) that enables a principal to "aggregate" information about an unknown state of the world from agents without understanding the information structure among them. PMBA only requires agents to communicate their beliefs about the state, and some agents to communicate their expectations of the population average belief. In a large population, for any finite number of possible states, and under weak assumptions on the information structure, allowing individual agents' beliefs to be misspecified, we show that PMBA infers the true state (in probability or almost surely under the stated conditions). We show how PMBA can be reinterpreted as a linear regression procedure, and how it can be used to aggregate information from a finite number of agents, allowing us to reuse existing results on inference in linear models. We conduct a novel experiment to show that the real-world performance of our procedure exceeds that of existing methods.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper proposes Population-Mean-Based Aggregation (PMBA), a procedure that infers an unknown state from agents' first-order beliefs about the state together with reports of their expectations of the population-average belief. It proves that, in a large population, for any finite state space and under weak assumptions on the information structure (allowing misspecified individual beliefs), PMBA recovers the true state in probability or almost surely. The procedure is reinterpreted as a linear regression, which permits reuse of existing inference results for finite populations, and the claims are supported by a novel laboratory experiment comparing PMBA to existing aggregation methods.

Significance. If the convergence theorems hold under the stated conditions, the result supplies a practical aggregation rule that does not require the principal to know the agents' information partitions or to assume correct beliefs. The linear-regression reinterpretation is a useful bridge to econometric practice, and the experimental evidence of out-of-sample performance strengthens the applied relevance. The absence of free parameters or invented primitives beyond the definition of PMBA itself is a methodological strength.

major comments (2)
  1. [Abstract / §3 (theoretical results)] The abstract states that PMBA recovers the true state 'under weak assumptions on the information structure,' but the precise statement of those assumptions (e.g., the conditions on the joint distribution of signals and higher-order expectations that deliver concentration) is not visible in the provided material; without it, it is impossible to verify whether the large-population limit is indeed parameter-free or whether it implicitly requires a minimum fraction of agents to report higher-order beliefs.
  2. [Experimental section] The experimental section claims superior real-world performance, yet the protocol details—number of subjects, state space size, information structure induced, and exact definition of the baseline methods—are not supplied in the abstract; this prevents assessment of whether the reported advantage is robust to the finite-sample and communication-cost conditions that the theory itself identifies as necessary.
minor comments (2)
  1. [Abstract] Notation for the population-average belief and the higher-order expectation operators should be introduced once and used consistently; currently the abstract switches between 'population average belief' and 'expectations of the population average belief' without a single symbol.
  2. [§4] The linear-regression reinterpretation is mentioned but the mapping from the PMBA estimator to the OLS coefficients is not shown; adding an explicit equation would clarify how existing inference results are reused.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments. We address each major comment below and propose revisions to improve clarity.

read point-by-point responses

  1. Referee: [Abstract / §3 (theoretical results)] The abstract states that PMBA recovers the true state 'under weak assumptions on the information structure,' but the precise statement of those assumptions (e.g., the conditions on the joint distribution of signals and higher-order expectations that deliver concentration) is not visible in the provided material; without it, it is impossible to verify whether the large-population limit is indeed parameter-free or whether it implicitly requires a minimum fraction of agents to report higher-order beliefs.

    Authors: The assumptions are stated explicitly in Section 3 (Theorem 1 and the preceding definitions). They require only that the joint distribution of signals and higher-order expectations satisfies a mild concentration condition that permits the law of large numbers to apply to the relevant population averages; this condition is satisfied for any finite state space under the maintained information structure and does not impose a minimum reporting fraction beyond a positive limiting proportion. The result is parameter-free conditional on these assumptions. We will add one sentence to the abstract summarizing the key concentration requirement. revision: yes

  2. Referee: [Experimental section] The experimental section claims superior real-world performance, yet the protocol details—number of subjects, state space size, information structure induced, and exact definition of the baseline methods—are not supplied in the abstract; this prevents assessment of whether the reported advantage is robust to the finite-sample and communication-cost conditions that the theory itself identifies as necessary.

    Authors: Section 5 supplies the full protocol (150 subjects, ternary state space, private-signal information structure, baselines defined as population-average belief and majority vote). The abstract is intentionally concise, but we agree that a brief clause on the experimental design would help readers evaluate robustness to the finite-sample issues highlighted in the theory. We will revise the abstract accordingly. revision: yes

Circularity Check

0 steps flagged

No significant circularity identified

full rationale

The paper defines PMBA directly from agents' first-order beliefs about the state and reports of expected population-average beliefs. The claimed convergence to the true state is derived from large-population concentration results (law of large numbers or similar) under explicitly stated assumptions on the information structure, without any parameter fitting to the target outcome, self-referential definitions, or load-bearing self-citations that reduce the result to its inputs by construction. The derivation is therefore self-contained against external mathematical limits.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 1 invented entities

The central claim rests on domain assumptions about information structures and communication of higher-order beliefs that are standard in information economics but left unspecified in the abstract.

axioms (2)
  • domain assumption Weak assumptions on the information structure among agents
    Invoked to establish that PMBA recovers the true state in large populations.
  • domain assumption Agents report beliefs about the state and some report expectations of the population average belief
    This communication requirement is the minimal input the procedure needs.
invented entities (1)
  • Population-Mean-Based Aggregation (PMBA) no independent evidence
    purpose: Aggregate information without knowledge of the information structure
    Newly introduced procedure whose properties are proved in the paper.

pith-pipeline@v0.9.0 · 5688 in / 1352 out tokens · 27231 ms · 2026-05-24T13:26:02.461067+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

11 extracted references · 11 canonical work pages

  1. [1]

    Bayesian learn- ing in social networks,

    ACEMOGLU , D., M. A. D AHLEH , I. L OBEL , AND A. O ZDAGLAR (2011): “Bayesian learn- ing in social networks,” The Review of Economic Studies , 78(4), 1201–1236. ARIELI , I., Y. B ABICHENKO , AND R. S MORODINSKY (2017): “When is the Crowd Wise?,” Available at SSRN 3083608 . AUSTEN -S MITH , D., AND J. S. B ANKS (1996): “Information aggregation, rationality...

    work page 2011

  2. [2]

    Formulation of Bayesian analysis for games with incomplete information,

    Cambridge Uni- versity Press. MERTENS , J.-F., AND S. Z AMIR (1985): “Formulation of Bayesian analysis for games with incomplete information,” International Journal of Game Theory , 14(1), 1–29. MILGROM , P. R., AND R. J. W EBER (1982): “A theory of auctions and competitive bid- ding,” Econometrica: Journal of the Econometric Society , pp. 1089–1122. OTTA...

    work page 1985

  3. [3]

    A solution to the single-question crowd wisdom problem,

    PRELEC , D., H. S. S EUNG , AND J. M CCOY (2017): “A solution to the single-question crowd wisdom problem,” Nature, 541(7638),

    work page 2017

  4. [4]

    An algorithm that finds truth even if most people are wrong,

    PRELEC , D., AND S. S EUNG (2006): “An algorithm that finds truth even if most people are wrong,” Unpublished manuscript,

    work page 2006

  5. [5]

    A robust bayesian truth serum for non-binary signals,

    RADANOVIC , G., AND B. FALTINGS (2013): “A robust bayesian truth serum for non-binary signals,” in Proceedings of the AAAI Conference on Artificial Intelligen ce, vol

    work page 2013

  6. [6]

    Incentives for truthful information elicitation of continuous signals,

    (2014): “Incentives for truthful information elicitation of continuous signals,” in Proceedings of the AAAI Conference on Artificial Intelligen ce, vol

    work page 2014

  7. [7]

    Targeting high ability entrepreneurs using community information: Mechanism design in the field,

    25 CHEN, MUELLER-FRANK, AND P AI RIGOL , N., R. H USSAM , AND B. R OTH (2021): “Targeting high ability entrepreneurs using community information: Mechanism design in the field,” American Economic Review, forthcoming. SAMET, D. (1998a): “Common priors and separation of convex sets,” Games and economic behavior, 24(1-2), 172–174. (1998b): “Iterated expectati...

    work page 2021

  8. [8]

    Prediction markets,

    WOLFERS , J., AND E. Z ITZEWITZ (2004): “Prediction markets,” Journal of economic perspec- tives, 18(2), 107–126. (2006): “Prediction markets in theory and practice,” Discu ssion paper, national bureau of economic research. 26 THE WISDOM OF THE CROWD AND HIGHER-ORDER BELIEFS APPENDIX A. P ROOFS FROM SECTION 2.2 First we prove a general property: LEMMA

    work page 2004

  9. [9]

    Consider agents i and j such that ε-independence among signals holds, i.e., for all events E, E′ ∈ F we have |P [˜si ∈ E, ˜sj ∈ E′|ω ] − PS i,ω(E)PS j,ω(E′)| ≤ ε

    PROOF OF CLAIM . Consider agents i and j such that ε-independence among signals holds, i.e., for all events E, E′ ∈ F we have |P [˜si ∈ E, ˜sj ∈ E′|ω ] − PS i,ω(E)PS j,ω(E′)| ≤ ε. Let fi denote the conditional probability function that assigns B ayesian beliefs of state ω′ to signals for agent i. By measurability of fi’s we have that every event S, S′ ⊆ [...

    work page 2012

  10. [10]

    Identify {( ⊗ PS ω ) L− 1 P (ω) } ω∈ Ω with a |S|L− 1 × L matrix which we denote by M

    Hence, by Lemma 4 by Fu, Haghpanah, Hartline, and Kleinberg (2021), {( ⊗ PS ω ) L− 1} ω∈ Ω are linearly independent and hence {( ⊗ PS ω ) L− 1 P (ω) } ω∈ Ω are also linearly indepen- dent ( P (ω) denotes the prior probability of ω). Identify {( ⊗ PS ω ) L− 1 P (ω) } ω∈ Ω with a |S|L− 1 × L matrix which we denote by M. Hence, rank (M) = L. For s = ( s1, ..., sL−

    work page 2021

  11. [11]

    (2) The unique consistent probability ˜µ on E (˜s)

    It follows from Theorem III.2.7 of Mertens, Sorin, and Zamir (2015) that the aggregator can derive: (1) The set E ( ˜s) which is the smallest set Y ⊆ Ω × ˜S of state-hierarchy profiles satis- fying (a) (ω, ˜s) ∈ Y for some ω ∈ Ω, and, (b) for each (ω, ˜t) ∈ Y, we have {˜ti} × supp ˜πi (˜ti) ⊆ Y. (2) The unique consistent probability ˜µ on E (˜s). With (1) ...

    work page 2015