Pith. sign in

REVIEW 2 major objections 6 minor 37 references

A proper betting strategy tied to any proper scoring rule converts an accuracy edge over the market into guaranteed expected profit in liquid prediction markets, and is essentially the only strategy that does so.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 16:05 UTC pith:ZRMICGJL

load-bearing objection Clean generalization of the AMM accuracy–profit link to arbitrary price impact, with a usable uniqueness theorem; live ROI is illustrative, not the load-bearing claim. the 2 major comments →

arxiv 2607.06166 v2 pith:ZRMICGJL submitted 2026-07-07 cs.AI cs.CEcs.GT

When do prophets profit in prediction markets?

Anri Gu , Nicole Kagan , Alec Sun , Jibang Wu , Haifeng Xu This is my paper
classification cs.AI cs.CEcs.GT
keywords prediction marketsproper scoring rulesproper bettingBregman divergencecentral limit order booksautomated market makersliquidity lossforecasting personas
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Classical prediction-market theory equates better forecasts with trading profit, but only for automated market makers. On the central limit order books that dominate real exchanges, accurate forecasters often lose money and crude heuristics can still make money. This paper closes that gap: for every strictly proper scoring rule it defines a matching “proper” bet that depends only on the forecaster’s probability and the market price, and proves that the bet’s expected profit equals the score gap plus a Bregman divergence term minus liquidity loss. Whenever the forecast beats the market under that scoring rule and liquidity is deep enough for the divergence to cover slippage, expected profit is positive. The same strategy is essentially unique among strategies that enjoy this robust guarantee, and the equivalence also yields a new characterization of proper scoring rules themselves. Offline tests on thousands of AI forecasts and a live month-long deployment confirm that proper betting is the only allocation that reliably turns accuracy into returns.

Core claim

For any strictly proper scoring rule with convex potential G, the proper position s_G(p,q)=∇G(p)−∇G(q) has expected profit exactly equal to the score gap between forecast p and market price q, plus the Bregman divergence D_G(q,p), minus liquidity loss. Hence the strategy is robustly profitable whenever p outperforms q under the scoring rule and divergence offsets slippage; moreover any robustly profitable strategy is essentially a rescaling or constant shift of this proper bet.

What carries the argument

Proper betting: the position s_G(p,q)=∇G(p)−∇G(q) induced by the potential of a proper scoring rule. Its profit identity (score gap + Bregman divergence − liquidity loss) generalizes the classical AMM guarantee and pins down both existence and uniqueness of robustly profitable strategies.

Load-bearing premise

The market must be liquid enough that the Bregman divergence between forecast and price covers the slippage cost of the proper bet; accuracy alone does not guarantee that.

What would settle it

On a liquid order-book market, take a forecast that beats the market under a chosen proper scoring rule yet still loses money under the corresponding proper bet while a clearly different normalized strategy earns positive expected profit; that would refute the robust-profitability and uniqueness claims.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Accuracy measured by a proper scoring rule can be turned into expected trading profit by a single explicit position that needs only p and q, without knowledge of the true probability.
  • Heuristic rules such as Kelly or max-margin can lose even with a score edge, and can win without one, because they ignore the Bregman term.
  • In automated market makers the proper bet recovers the classical “move price to your belief” trade and the score-gap profit formula as the special case where divergence exactly cancels liquidity loss.
  • Different scoring rules (Brier, log, spherical) weight disagreement differently, so the best proper strategy depends on how a forecaster’s errors concentrate across margins.
  • The same decomposition extends to empirical scores, bid–ask spreads, and multi-period rebalancing, giving operational rules for live books.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Exchanges could publish the proper-bet map for a chosen scoring rule as a default “informed order” type, making accuracy-to-profit conversion the default rather than an expert skill.
  • Persona-dependent rule choice suggests an adaptive meta-layer that estimates a forecaster’s margin–accuracy profile and switches among Brier, log, or spherical weights in real time.
  • The uniqueness result implies that any profitable black-box trading bot whose edge is purely informational must be approximately implementing some proper bet, which could be audited by recovering the implied potential G.
  • Market designers choosing liquidity subsidies or fee schedules are effectively choosing how large the Bregman term must be before informed traders can profit, shaping who participates.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper resolves the gap between classical AMM theory (accuracy equals profit) and modern CLOB prediction markets by defining, for any strictly proper scoring rule S with potential G, a proper betting strategy s_G(p,q)=∇G(p)−∇G(q). Theorem 1 decomposes expected profit into score gap + Bregman divergence − liquidity loss, so the strategy is robustly profitable whenever the forecast beats the market under S and D_G offsets L_ρ. Theorem 2 shows any robustly profitable strategy is essentially the same as s_G (up to rescaling and constant shift). Proposition 2 recharacterizes proper scoring rules via robust profitability; Corollary 1 recovers the classical AMM guarantee as the special case where Bregman cancels liquidity loss. Extensions cover empirical scores, bid–ask spreads, and sequential trading. Empirically, proper betting is the only strategy that reliably converts AI forecast accuracy into ROI on thousands of Kalshi markets; persona analysis links model behavior to preferred scoring rules; a 26-day live Kalshi deployment with Gemini 3 yields +80.33% ROI and Sharpe 3.35.

Significance. If the results hold, the paper supplies the missing formal bridge between proper scoring rules and profitability on general (including CLOB) prediction markets, strictly generalizing Hanson’s AMM theory. The profit decomposition cleanly explains both the accuracy–profit paradox and how uninformed strategies can still earn money via Bregman divergence. The uniqueness result and the new characterization of proper scores are of independent theoretical interest. Empirically, the work is strengthened by large-scale AI forecast evaluation, a falsifiable persona taxonomy, and a real-capital live deployment with documented trades—rare for theory papers in this area. The algebraic identities follow from standard convex-analysis representations (McCarthy/Gneiting–Raftery) without free constants fitted to ROI, which is a genuine strength.

major comments (2)
  1. Theorem 1’s robust-profitability guarantee is conditional on D_G(q,p) ≥ L_ρ(s_G;q). Offline experiments in §4.1 explicitly set zero price impact (L_ρ=0), so they test only the frictionless special case of the decomposition, not the full claim. The live deployment (§4.3) reports ROI and a Brier decomposition (ΔS=+0.7205, D=+0.0828) but does not measure or bound realized liquidity loss / slippage against D_G. Either report order-book depth / fill-cost statistics that verify the offset condition on traded markets, or state more sharply that the offline tables and live ROI are evidence for the score-gap+Bregman mechanism under low friction, not a direct test of robust profitability under nontrivial L_ρ.
  2. Definition 3 and Theorem 2 define “essentially the same” via rescaling, constant shift by λ1, and non-vanishing difference along some sequence p_t→q. The proof construction (normalize to 1^⊥, subsequence limits ŝ,s*, pick v with v·s*>c>0>v·ŝ, set p*_t=q+v) is standard and appears correct for frictionless markets. However, the manuscript never checks whether natural practical modifications—e.g., clipping bets inside the bid–ask no-trade zone of §C.1, or discrete share/tick constraints in the live executor—remain “essentially the same” or can break robust profitability. A short remark or corollary clarifying which real-market truncations preserve the guarantee would make the uniqueness claim operationally usable.
minor comments (6)
  1. Examples 1–3 are helpful but use hand-picked p,q,p* triples; a one-line note that they are illustrative (not sampled from real books) would avoid over-reading.
  2. Table 2 vs Table 6: the main text shows a 200-event subset while Appendix D.2 reports the full standardized set; cross-reference the subset construction more clearly in §4.1 so readers know which numbers are comparable.
  3. Persona generation (§D.4) uses Beta modes, slopes, and a0 calibrated to a fixed ±0.05 Brier gap. These free parameters are fine for a controlled experiment but should be listed explicitly in the main text near Figure 1 so the taxonomy is reproducible without the appendix.
  4. Live eligibility filters (2–14 day horizon, ≥10¢ move, MENTIONS exclusion, resolution-clarity drop) are material; a short sensitivity note (e.g., ROI if the 10¢ filter is removed) would strengthen §4.3.
  5. Notation: ρ is introduced as price-impact and later as post-trade spot; a single sentence that ρ(s) is the marginal price path and L_ρ is its integral excess cost would help readers less familiar with AMM/CLOB abstractions.
  6. Typos / polish: “them arket score” spacing in §2; arXiv id in the header is 2607.06166 while the live dates are April/May 2026—confirm consistency of the preprint versioning note.

Circularity Check

0 steps flagged

No significant circularity: Theorems 1–2 and the profit decomposition are algebraic identities from convex potentials, not fits or self-citation chains.

full rationale

The load-bearing chain is Definition 2 (s_G := ∇G(p)−∇G(q)), Lemma 1 (algebraic Bregman identity equating idealized profit to realized score gap plus D_G), Theorem 1 (expectation plus liquidity loss), and Theorem 2 (uniqueness via normalization on 1^⊥ and explicit adversarial construction of p*_t). Each step is derived from the McCarthy characterization of proper scoring rules and the definition of Bregman divergence; nothing is fitted to the target ROI, and no free constant is later re-labeled a prediction. Corollary 1 recovers the classical AMM guarantee as the special case D_G = L_ρ, which is an independent check against Hanson/Chen–Pennock rather than a circular rename. Empirical ROI tables and the Kalshi live run measure held-out resolved outcomes under stated strategies; they do not force the formal claims. The only self-reference is Prophet Arena (Yang et al. 2025, overlapping coauthors) as a data pipeline—infrastructure, not a uniqueness theorem or ansatz underwriting Theorems 1–2. Under the analyzer rules this is ordinary self-citation, not circularity. Score 0 with empty steps is the honest finding.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 2 invented entities

The theory rests on standard convex analysis and the classical characterization of proper scoring rules, plus domain assumptions about price impact and arbitrage-free quotes. No new physical entities are postulated. Practical claims add market-liquidity and experimental-design choices (persona generators, live eligibility filters) that are free parameters of the evaluation, not of the theorems.

free parameters (3)
  • Persona margin/accuracy schedules (Beta modes, slopes, a0)
    Synthetic personas are calibrated via Brent search so all share a fixed ±0.05 quadratic gap; these knobs shape which proper rule wins in Regime A/B and are not predicted by theory.
  • Live eligibility filters (2–14 day horizon, ≥10¢ move, category exclusions)
    Section D.7.2 choices determine which Kalshi markets enter the live book and therefore the reported +80.33% ROI.
  • Budget normalization / sequential Kelly leverage when f>1
    Offline strategies normalize Σ|w_i|=1; Kelly allows leverage above 1. These implementation choices affect baseline ROI comparisons.
axioms (5)
  • standard math McCarthy characterization: S is (strictly) proper iff S(p,y)=G(p)+∇G(p)·(1_y−p) for (strictly) convex G
    Proposition 1; used as the definitional backbone of proper bets and Lemma 1.
  • domain assumption Price-impact ρ is monotone: (ρ(s)−ρ(s'))·(s−s')≥0, with ρ(0)=q, so liquidity loss L_ρ≥0
    Section 2 market model; needed for the profit definition and non-negativity of slippage.
  • domain assumption Arbitrage-free quotes lie on (or are reduced to) the simplex / bid-ask with q++q−≥1
    Section 2 and C.1; allows interpreting q as consensus belief and extending proper bets coordinate-wise.
  • ad hoc to paper Robust profitability requires D_G(q,p) to offset L_ρ(s_G;q)
    Theorem 1’s sufficient condition for converting score edge into guaranteed profit; not automatic in thin books.
  • ad hoc to paper Two strategies are ‘essentially the same’ after rescaling, constant shift by λ1, and agreement along sequences p_t→q
    Definition 3; load-bearing for the uniqueness claim in Theorem 2.
invented entities (2)
  • Proper betting strategy s_G(p,q)=∇G(p)−∇G(q) independent evidence
    purpose: Canonical map from forecast and market price to positions that inherit proper-score edges as profit.
    Mathematical construction, not a physical postulate; independent evidence is the profit identity and empirical ROI comparisons.
  • Forecasting personas (Conservative, Aggressive, Dispersed, Brittle) no independent evidence
    purpose: Taxonomy linking margin distribution and win-rate slope to which proper rule is preferred.
    Synthetic/analytical labels fitted to model behavior; useful for interpretation but not independently measured outside this evaluation design.

pith-pipeline@v1.1.0-grok45 · 35361 in / 3637 out tokens · 47710 ms · 2026-07-14T16:05:26.947207+00:00 · methodology

0 comments
read the original abstract

Prediction markets aggregate dispersed beliefs into prices that act as probabilistic forecasts of uncertain events. Classical theory establishes a clean equivalence between forecasting accuracy and trading profit, but only for the specific automated market maker (AMM) design. However, the largest exchanges today are based on central limit order books in which informed forecasters routinely lose money while uninformed strategies can profit on simple heuristics. We resolve this discrepancy by establishing a formal equivalence between predictive accuracy and profitability. For any strictly proper scoring rule $S$, we exhibit a "proper" betting strategy that depends only on the forecaster's prediction $\mathbf{p}$ and the market price $\mathbf{q}$, and earns positive expected profit whenever $\mathbf{p}$ outperforms $\mathbf{q}$ under $S$ and the market has sufficient liquidity. Moreover, this proper betting is essentially the only strategy with such robust profitability guarantee. The proof rests on a decomposition of expected profit that strictly generalizes the classical AMM guarantee and also explains how strategies can profit without an accuracy edge. Empirically, across thousands of forecasts by AI models, proper betting is the only strategy that reliably converts accuracy into profit, and we further identify systematic forecasting personas and show how the optimal proper strategy varies across them. A month-long live deployment on Kalshi achieves $+80.33\%$ return on investment with a Sharpe ratio of $3.35$.

Figures

Figures reproduced from arXiv: 2607.06166 by Alec Sun, Anri Gu, Haifeng Xu, Jibang Wu, Nicole Kagan.

Figure 1
Figure 1. Figure 1: Schematics for synthetic forecasting personas and their prediction profiles. analysis; although rare, such high-margin bets can occasionally be correct and highly profitable, which we examine further in Section D.8. We run the experiment in two regimes—one where the model beats the market and one where it loses—to examine how accuracy interacts with the choice of proper betting strategy and the resulting i… view at source ↗
Figure 2
Figure 2. Figure 2: Mapping real models to personas. losses. A symmetric ±0.05 quadratic score gap (market quadratic score = 0.839) isolates the effect of persona and strategy while holding average accuracy constant. As shown in [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Live deployment results for the Gemini 3 trading agent on Kalshi prediction markets. way to summarize how model behavior translates into returns. This abstraction lets us reason about profitability and select or adapt betting strategies based on these behavioral patterns. 4.3 Live deployment under active portfolio management We further evaluate proper betting in a live portfolio setting using the momentum-… view at source ↗
Figure 4
Figure 4. Figure 4: Capital allocation (bet weight in shares) as a function of signed margin and market price (q, y-axis) for the three proper betting strategies. Warmer colors indicate larger bets. Grey regions are infeasible (p /∈ (0, 1)). Contour lines mark constant weight levels. D.6 Mapping empirical forecasters to personas As shown in [PITH_FULL_IMAGE:figures/full_fig_p025_4.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

37 extracted references · 1 linked inside Pith

  1. [1]

    Proceedings of the National Academy of Sciences , volume =

    John McCarthy , title =. Proceedings of the National Academy of Sciences , volume =

  2. [2]

    Savage , journal =

    Leonard J. Savage , journal =. Elicitation of Personal Probabilities and Expectations , urldate =

  3. [3]

    2025 , eprint=

    LLM-as-a-Prophet: Understanding Predictive Intelligence with Prophet Arena , author=. 2025 , eprint=

    2025

  4. [4]

    Neural Information Processing Systems (NeurIPS 2025) Workshop: Generative AI in Finance , year=

    The Losing Winner: An LLM Agent that Predicts the Market but Loses Money , author=. Neural Information Processing Systems (NeurIPS 2025) Workshop: Generative AI in Finance , year=

    2025

  5. [5]

    the bell system technical journal , volume=

    A new interpretation of information rate , author=. the bell system technical journal , volume=. 1956 , publisher=

    1956

  6. [6]

    Monthly Weather Review , volume=

    Verification of forecasts expressed in terms of probability , author=. Monthly Weather Review , volume=. 1950 , publisher=

    1950

  7. [7]

    Journal of the Royal Statistical Society: Series B (Methodological) , volume=

    Rational decisions , author=. Journal of the Royal Statistical Society: Series B (Methodological) , volume=

  8. [8]

    Proceedings of the National Academy of Sciences Symposium on Information Theory , pages=

    Measures of the value of information , author=. Proceedings of the National Academy of Sciences Symposium on Information Theory , pages=

  9. [9]

    Foundations of Statistical Inference , pages=

    Elicitation of personal probabilities and expectations , author=. Foundations of Statistical Inference , pages=

  10. [10]

    Belief states: A preliminary empirical study , author=

  11. [11]

    Journal of the American statistical Association , volume=

    Strictly proper scoring rules, prediction, and estimation , author=. Journal of the American statistical Association , volume=. 2007 , publisher=

    2007

  12. [12]

    Proceedings of the 9th ACM Conference on Electronic Commerce , pages=

    Eliciting properties of probability distributions , author=. Proceedings of the 9th ACM Conference on Electronic Commerce , pages=

  13. [13]

    Proceedings of the 23rd ACM Conference on Economics and Computation , pages=

    Optimization of scoring rules , author=. Proceedings of the 23rd ACM Conference on Economics and Computation , pages=

  14. [14]

    Proceedings of the 11th ACM conference on Electronic commerce , pages=

    A new understanding of prediction markets via no-regret learning , author=. Proceedings of the 11th ACM conference on Electronic commerce , pages=

  15. [15]

    Journal of economic perspectives , volume=

    Prediction markets , author=. Journal of economic perspectives , volume=. 2004 , publisher=

    2004

  16. [16]

    2008 , publisher=

    The Promise of Prediction Markets , author=. 2008 , publisher=

    2008

  17. [17]

    Proceedings of the 5th ACM Conference on Electronic Commerce (EC) , pages=

    Combinatorial information market design , author=. Proceedings of the 5th ACM Conference on Electronic Commerce (EC) , pages=

  18. [18]

    Journal of Prediction Markets , volume=

    Logarithmic market scoring rules for modular combinatorial information aggregation , author=. Journal of Prediction Markets , volume=

  19. [19]

    Proceedings of the 23rd Conference on Uncertainty in Artificial Intelligence (UAI) , pages=

    A utility framework for bounded-loss market makers , author=. Proceedings of the 23rd Conference on Uncertainty in Artificial Intelligence (UAI) , pages=

  20. [20]

    ACM Transactions on Economics and Computation (TEAC) , volume=

    Efficient market making via convex optimization, and a connection to online learning , author=. ACM Transactions on Economics and Computation (TEAC) , volume=. 2013 , publisher=

    2013

  21. [21]

    Proceedings of the 10th ACM conference on Electronic commerce , pages=

    Information aggregation in dynamic markets with strategic traders , author=. Proceedings of the 10th ACM conference on Electronic commerce , pages=

  22. [22]

    Algorithmica , volume=

    Gaming prediction markets: Equilibrium strategies with a market maker , author=. Algorithmica , volume=. 2010 , publisher=

    2010

  23. [23]

    Handbook of experimental economics results , volume=

    Results from a dozen years of election futures markets research , author=. Handbook of experimental economics results , volume=. 2008 , publisher=

    2008

  24. [24]

    Public Opinion Quarterly , volume=

    Forecasting elections: Comparing prediction markets, polls, and their biases , author=. Public Opinion Quarterly , volume=. 2009 , publisher=

    2009

  25. [25]

    Advances in Neural Information Processing Systems , volume=

    Approaching human-level forecasting with language models , author=. Advances in Neural Information Processing Systems , volume=

  26. [26]

    Advances in Neural Information Processing Systems , volume=

    Forecasting future world events with neural networks , author=. Advances in Neural Information Processing Systems , volume=

  27. [27]

    Stochastic optimization models in finance , pages=

    Portfolio choice and the Kelly criterion , author=. Stochastic optimization models in finance , pages=. 1975 , publisher=

    1975

  28. [28]

    2011 , publisher=

    The Kelly capital growth investment criterion: Theory and practice , author=. 2011 , publisher=

    2011

  29. [29]

    Management Science , volume=

    Growth versus security in dynamic investment analysis , author=. Management Science , volume=. 1992 , publisher=

    1992

  30. [30]

    Mathematical finance , volume=

    Universal portfolios , author=. Mathematical finance , volume=. 1991 , publisher=

    1991

  31. [31]

    arXiv preprint arXiv:2412.18544 , year=

    Consistency checks for language model forecasters , author=. arXiv preprint arXiv:2412.18544 , year=

    Pith/arXiv arXiv

  32. [32]

    American Economic Review , volume=

    Price reaction to information with heterogeneous beliefs and wealth effects: Underreaction, momentum, and reversal , author=. American Economic Review , volume=. 2015 , publisher=

    2015

  33. [33]

    International joint conference on artificial intelligence , volume=

    The foundations of cost-sensitive learning , author=. International joint conference on artificial intelligence , volume=. 2001 , organization=

    2001

  34. [34]

    Who Profits from Prediction Markets? Execution, not Information , publisher =

    Della Vedova, Joshua , year =. Who Profits from Prediction Markets? Execution, not Information , publisher =

  35. [35]

    The American Economic Review , volume=

    The Use of Knowledge in Society , author=. The American Economic Review , volume=. 1945 , publisher=

    1945

  36. [36]

    2023 , howpublished =

    2023

  37. [37]

    Available at SSRN , year=

    Price discovery and trading in modern prediction markets , author=. Available at SSRN , year=