Empirical Validation of the Independent Chip Model

Juho Kim

REVIEW 3 major objections 5 minor 16 references

Empirical Validation of the Independent Chip Model

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The standard poker tournament chip-value model overestimates short stacks and underestimates large stacks.

desk verdict A useful new dataset and a first large-scale ICM test, but the headline stack-size bias rests on t-tests that ignore clustering the authors themselves acknowledge, so the current p-values don't hold up. read the letter →

arxiv 2506.00180 v2 pith:CIOAQDZY submitted 2025-05-30 cs.GT cs.LG

Juho Kim This is my paper

classification cs.GTcs.LG

keywords independentchipmodelpokertournamentsprizeequityempiricalvalidationstack-sizebiasresidualanalysistournamentdataset

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tests the independent chip model (ICM), the standard way to convert poker tournament chip counts into expected prize money, against real tournament results. Using a newly assembled dataset drawn from more than ten thousand poker tournaments, it compares the ICM with a naive baseline and finds the ICM more accurate. It then groups players by stack size and reports that the ICM is wrong at the extremes: players with the largest stacks win more prize money than the model predicts, while short-stacked players win less. The paper reads this as large-scale evidence for one side of a dispute left open by two earlier small studies.

What carries the argument

The independent chip model (ICM): an algorithm that takes each player's chip share $x_i$ and the tournament payout structure as inputs and computes expected prize money by assuming first place goes to player $i$ with probability $x_i$, and that each subsequent finishing position is drawn proportionally to chip share among the players not yet placed (Equations 1--3). The argument is carried by pairing end-of-day chip-count snapshots with actual payouts, then stratifying the ICM's residuals by stack-size quartiles in the same way the earlier peer-reviewed study did. Events with 11 or more players are evaluated with a Monte Carlo variant of the ICM (standard error tolerance 0.001, at most 10,000 simulations, at least 100 runs), and payouts are normalized per event so that estimates and outcomes are comparable.

What would settle it

Recompute Experiment 2 with standard errors clustered by tournament event or with a permutation test that shuffles stack ranks within each event. If the large-stack residual ($+5.59\times10^{-3}$) and the small-stack residual ($-4.44\times10^{-3}$) lose significance at the 95% level under that dependence-corrected test, the paper's central stack-size bias claim would be falsified.

Watch

Extended reading notes

Core claim

The paper's central finding is that the ICM, which assigns each player a prize expectation by treating every chip as equally valuable in a sequence of conditional draws, is not uniformly accurate across stack sizes. In Experiment 2, stratifying 9,962 players from 1,504 final-table-sized events into stack-size quartiles yields a mean residual (observed minus estimated prize share) of $+5.59\times10^{-3}$ for large stacks ($p=0.004$) and $-4.44\times10^{-3}$ for small stacks ($p<0.001$), with no statistically significant deviation for medium stacks. Positive residuals mean large-stack players systematically outperform their ICM expectations; negative residuals mean short-stack players underperform theirs. The authors interpret the results as agreeing with one of the two earlier studies on both ends of the stack distribution, while the other study had found short stacks performing close to expectation.

Load-bearing premise

The load-bearing assumption is that player-level observations are independent in the significance tests, even though the paper grants that prize shares within one tournament sum to one; if that dependence is taken into account, the reported $p$-values would be higher.

Editorial extensions

If this is right

If the ICM is more accurate than the proposed baseline, then even a simple comparison algorithm cannot replace it, and the ICM remains the reference point for tournament equity estimation.
Because the ICM underpredicts large-stack prize money, chip leaders have more tournament equity than ICM-based strategy software assumes, so decisions that put a large stack at risk may be better than ICM-based charts suggest.
Because the ICM overpredicts short-stack prize money, short-stacked players have less equity than the model assumes, so shoving ranges derived from ICM may be too loose.
The released dataset gives future estimators a large-scale benchmark, so new stack-size-aware or player-aware value models can be trained and evaluated against the same kind of evidence.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An implication the paper leaves implicit is that a stack-size correction term could be fit to the residuals and tested out of sample; if the large-stack and small-stack effects persist in held-out tournaments, the bias is structural to the ICM family rather than an artifact of this dataset.
If the bias reflects real tournament dynamics, then hand recommendations from ICM-based solvers are systematically conservative for chip leaders and too aggressive for short stacks; this inference about strategy is not something the paper measures directly.
Because the paper concedes that its independence assumption is violated by the within-event sum-to-one constraint, a clustered or permutation-based reanalysis would likely widen the error bars; the direction of the bias could survive, but the reported $p$-values of $0.004$ and $<0.001$ are optimistic as stated.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The thing to know: this is the first large-scale empirical test of the Independent Chip Model, using a new scraped dataset of 33,478 player-event observations. The headline result—ICM overestimates short stacks and underestimates large stacks—is plausible and matches Henke's unpublished thesis. But the statistical evidence as reported is not trustworthy. The t-tests treat players as independent even though values within an event sum to one. The authors acknowledge this in Section III-D and say p-values would be higher, yet they don't correct for it. The stress-test note is right: with roughly 6.6 players per event, ignoring clustering can inflate the effective sample size by that factor. Scaling the standard errors by sqrt(6.6) pushes both the large-stack (p=0.004) and small-stack (p<0.001) results above 0.05. The central claim currently rests on an unaddressed dependency.

What the paper does well: it contributes a genuinely new dataset, releases it on Zenodo, and describes the preprocessing transparently. The comparison with Scott et al. and Henke is careful, and the authors are honest about the independence violation. Experiment 1 shows ICM beats a deliberately naive baseline—not a strong test, but the result is what you'd expect. The medium-stack null finding is less affected by clustering, but it's also not the headline.

Soft spots beyond the t-tests: the data are end-of-day chip counts from two sources, filtered non-randomly to 10 or fewer players, so generalizing to live tournaments is a stretch. No code is shipped, which limits reproducibility. The baseline in Experiment 1 is too weak to tell us much about ICM accuracy beyond “better than a dumb rule.”

This paper is for anyone working on tournament poker theory or chip-value models. The dataset itself is a useful resource, and the Scott/Henke debate is worth settling properly. It deserves a serious referee because it’s the first large-scale empirical look at ICM, and the flaws are addressable—a clustered or permutation reanalysis would do a lot.

My recommendation: send it for review, but make clear the statistical analysis needs to be redone. In current form, the headline claim is not supported.

Referee Report

3 major / 5 minor

Summary. The manuscript scrapes a new dataset of poker tournament end-of-day chip counts and payouts (9,958 events total; 2,500 usable snapshots with 33,478 player records after preprocessing) and evaluates the independent chip model on it. Experiment 1 (Section III-B) compares the ICM's mean squared error with a rank-based baseline and reports that the ICM is significantly more accurate. Experiment 2 (Section III-C) restricts the data to 1,504 events with at most 10 players, groups players into large, medium, and small stacks by within-event quartiles, and uses one-sample t-tests on residual means; it reports that the ICM underestimates large stacks (mean residual 5.59e-3, p=0.004) and overestimates small stacks (mean residual -4.44e-3, p<0.001), with no significant medium-stack bias. Section III-D acknowledges that the t-test independence assumption is violated because residuals in each event sum to one, but no correction is applied.

Significance. If the stack-size bias findings survive a statistically valid reanalysis, the paper would be the first large-scale real-world validation of the ICM and would help resolve a genuine discrepancy between Scott et al. and Henke. The dataset release is a useful community resource, and the authors are transparent about the violated independence assumption and about the deliberately simple baseline in Experiment 1. However, the headline statistical evidence is currently not established because the tests ignore dependence within events, and Experiment 1's baseline is weak by construction. The empirical contribution is therefore conditional on a corrected analysis.

major comments (3)

[Section III-D and IV-B / Table III] The stack-size conclusion rests on one-sample t-tests that treat 9,962 player observations as independent. Section III-D concedes that this assumption is false because residuals within each event sum to one, so they are negatively correlated. With 1,504 events, the effective number of independent units is far smaller than 9,962, and a cluster-robust standard error at the event level, or a permutation test that shuffles stack labels within events, could materially change the reported p-values. The direction of the change is not obvious, so the reported p-values for large stacks (0.004) and small stacks (<0.001) cannot be interpreted as evidence of bias until such a reanalysis is provided. This is a required correction, not a robustness check.
[Section III-B and IV-A] Experiment 1 compares the ICM against a baseline that, by design, assigns each player the payout corresponding to his chip-count rank and ignores chip magnitudes. It is therefore essentially built into the comparison that the ICM will have lower MSE, and the p-value below 10^-354 is not evidence of real-world accuracy. The abstract's phrase 'verify that the ICM performs more accurately than a baseline' overstates what the experiment shows. Please reframe Experiment 1 as a sanity check or compare against a more natural null model (e.g., chip-proportional expected payout), and compute the MSE comparison at the event level to avoid the same non-independence problem.
[Section III-C / Table III] The three separate one-sample t-tests are not independent of the grouping: because each event's residuals sum to zero, the weighted mean of the large-, medium-, and small-stack group means is mechanically zero, so a positive large-stack mean and a negative small-stack mean are complementary. A more direct analysis would regress player-level residuals on within-event stack-size rank or percentile, with event-level cluster-robust standard errors; this would also estimate the size of the bias rather than reporting three binary significance claims.

minor comments (5)

[Section II, Eq. (1)] Please define x_i explicitly as chip_i divided by total chips in play; the notation currently appears without a formal definition.
[Section III-B] Please clarify the pairing in the 'one-sided paired t-test': is each pair a player-event observation, and are the MSEs computed per player or per event?
[Section III-D] The sentence 'the p-values and confidence intervals would be higher than what is reported' should say 'larger p-values and wider confidence intervals' to avoid ambiguity.
[Section V-B] The statement that the lack of a significant medium-stack residual 'can be interpreted as the ICM giving accurate predictions' should be softened; absence of evidence is not evidence of absence.
[Figure 2] Adding a horizontal zero line to Figure 2 would make the magnitudes and the medium-stack mean easier to read.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the ICM is evaluated against external tournament outcomes with no fitted parameters, self-citations, or definitional reductions.

full rationale

This paper performs an empirical validation of the independent chip model against real tournament results. The ICM is defined externally from prior literature (Acevedo [10]) and is not fitted to the dataset. Experiment 1 compares the ICM against a deliberately simple baseline using mean squared error on observed prize payouts; the baseline is not derived from the ICM, and the comparison is a straightforward accuracy measurement. Experiment 2 stratifies residuals (observed minus ICM-estimated payouts) by stack-size quartile and tests whether their means differ from zero. No parameter is estimated from the data and then relabeled as a prediction; the residuals are computed directly from the model's fixed outputs and observed outcomes. There are no self-citations, and the cited prior works (Scott et al., Henke) are independent empirical studies used only for comparison. The authors' explicit acknowledgment in Section III-D that the t-test independence assumption is violated because player values in the same event sum to one is a statistical validity concern, not a circularity concern; it affects the reliability of the p-values but does not mean any result is equivalent to its inputs by construction. The central claims are therefore self-contained empirical findings rather than definitional or fitted reductions.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper introduces no fitted coefficients or new theoretical entities. Its claims rest on hand-chosen grouping and Monte Carlo parameters, the acknowledged independence assumption, and the accuracy of scraped tournament data.

free parameters (3)

Quartile grouping rule for stack-size cohorts = top 25%, middle 50%, bottom 25% of chip counts per event
Hand-chosen to follow Scott et al.; determines which players are labeled large, medium, or small and directly shapes the residual means in Experiment 2.
Maximum player count for Experiment 2 = 10
Chosen to match Scott et al. and traditional final-table ICM use; filters 2,500 snapshots down to 1,504 and 9,962 players.
Monte Carlo ICM parameters = standard error tolerance 0.001, max 10,000 simulations, min 100 runs
Chosen to keep ICM computation tractable for events with 11 or more players; may introduce small approximation error in the expected values used for residuals.

assumptions (3)

domain assumption Player outcomes and algorithm outputs are independent across observations in the t-tests.
Stated in Section III-D; the authors acknowledge it is violated because each event's player values sum to one, yet the reported p-values rely on it.
domain assumption The implemented ICM (exact or Monte Carlo) correctly computes the model's expected values from chip counts and payouts.
Residuals in Experiment 2 and MSE in Experiment 1 are computed against these expected values; implementation error would change both conclusions.
domain assumption End-of-day chip count snapshots and final results scraped from WSOP.com and PokerNews are accurate and representative.
The dataset is scraped and deduplicated; errors or non-random coverage would bias both experiments.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Empirical Validation of the Independent Chip Model." pith.science (2026). https://pith.science/paper/CIOAQDZY

@misc{pith2026250600180,
  author       = {Pith},
  title        = {Pith review of: Empirical Validation of the Independent Chip Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CIOAQDZY}},
  note         = {Machine review of arXiv:2506.00180}
}

read the original abstract

The independent chip model (ICM) forms a cornerstone of all modern poker tournament strategy. However, despite its prominence, the ICM's performance in the real world has not been sufficiently scrutinized, especially at a large scale. In this paper, we introduce our new dataset of poker tournaments, consisting of results of over ten thousand events. Then, using this dataset, we perform two experiments as part of a large-scale empirical validation of the ICM. First, we verify that the ICM performs more accurately than a baseline we propose. Second, we obtain empirical evidence of the ICM underestimating the performances of players with larger stacks while overestimating those who are short-stacked. Our contributions may be useful to future researchers developing new algorithms for estimating a player's value in poker tournaments.

Figures

Figures reproduced from arXiv: 2506.00180 by the authors.

**Figure 2.** The mean residuals of the ICM on the poker tournament data stratified [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

16 extracted references · 11 canonical work pages

[1]

H. W. Kuhn, 9. A SIMPLIFIED TWO-PERSON POKER . Princeton: Princeton University Press, 1951, pp. 97–104. [Online]. Available: https://doi.org/10.1515/9781400881727-010

doi:10.1515/9781400881727-010 1951
[2]

A parameterized family of equilibrium profiles for three-player kuhn poker,

D. Szafron, R. Gibson, and N. Sturtevant, “A parameterized family of equilibrium profiles for three-player kuhn poker,” in Proceedings of the 2013 International Conference on Autonomous Agents and Multi-Agent Systems, ser. AAMAS ’13. Richland, SC: International Foundation for Autonomous Agents and Multiagent Systems, 2013, p. 247–254

work page 2013
[3]

Aivat: a new variance reduction technique for agent evaluation in imperfect information games,

N. Burch, M. Schmid, M. Morav ´c´ık, D. Morill, and M. Bowling, “Aivat: a new variance reduction technique for agent evaluation in imperfect information games,” in Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational ...

work page 2018
[4]

Heads-up limit hold’em poker is solved,

M. Bowling, N. Burch, M. Johanson, and O. Tammelin, “Heads-up limit hold’em poker is solved,” Commun. ACM , vol. 60, no. 11, pp. 81–88, 2017. [Online]. Available: https://doi.org/10.1145/3131284

doi:10.1145/3131284 2017
[5]

DeepStack: Expert-level artificial intelligence in heads-up no-limit poker,

M. Morav ˇc´ık, M. Schmid, N. Burch, V . Lis ´y, D. Morrill, N. Bard, T. Davis, K. Waugh, M. Johanson, and M. Bowling, “DeepStack: Expert-level artificial intelligence in heads-up no-limit poker,” Science, vol. 356, no. 6337, pp. 508–513, 2017. [Online]. Available: https://www.science.org/doi/abs/10.1126/science.aam6960

doi:10.1126/science.aam6960 2017
[6]

Superhuman AI for heads-up no-limit poker: Libratus beats top professionals,

N. Brown and T. Sandholm, “Superhuman AI for heads-up no-limit poker: Libratus beats top professionals,” Science, vol. 359, no. 6374, pp. 418–424, 2018. [Online]. Available: https://www.science.org/doi/ abs/10.1126/science.aao1733

doi:10.1126/science.aao1733 2018
[7]

Superhuman AI for multiplayer poker,

——, “Superhuman AI for multiplayer poker,” Science, vol. 365, no. 6456, pp. 885–890, 2019. [Online]. Available: https://www.science.org/ doi/abs/10.1126/science.aay2400

doi:10.1126/science.aay2400 2019
[8]

Poker playing as a dramaturgical event— Bull power, the meaning and commitment for efficacious gamesmanship,

S. H. Boyd, “Poker playing as a dramaturgical event— Bull power, the meaning and commitment for efficacious gamesmanship,” 1975

work page 1975

Show all 16 references

[9]

Social and Psychological Challenges of Poker,

K. Siler, “Social and Psychological Challenges of Poker,” Journal of Gambling Studies , vol. 26, no. 3, pp. 401–420, Sep 2010. [Online]. Available: https://doi.org/10.1007/s10899-009-9168-2

2010 doi
[10]

Acevedo, Modern Poker Theory: Building an unbeatable strategy based on GTO principles

M. Acevedo, Modern Poker Theory: Building an unbeatable strategy based on GTO principles . D&B Publishing, 2019. [Online]. Available: https://books.google.ca/books?id=fQ6oDwAAQBAJ

2019
[11]

A near-optimal strategy for a heads-up no-limit texas hold’em poker tournament,

P. B. Miltersen and T. B. Sørensen, “A near-optimal strategy for a heads-up no-limit texas hold’em poker tournament,” in Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems , ser. AAMAS ’07. New York, NY , USA: Association for Comp...

2007
[12]

Computing an approximate jam/fold equilibrium for 3-player no-limit texas hold’em tournaments,

S. Ganzfried and T. Sandholm, “Computing an approximate jam/fold equilibrium for 3-player no-limit texas hold’em tournaments,” in Pro- ceedings of the 7th International Joint Conference on Autonomous Agents and Multiagent Systems - Volume 2, ser. AAMAS ’08. Richland, SC: Inter...

2008
[13]

Computing equilibria in multiplayer stochastic games of im- perfect information,

——, “Computing equilibria in multiplayer stochastic games of im- perfect information,” in Proceedings of the 21st International Joint Conference on Artificial Intelligence , ser. IJCAI’09. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 2009, p. 140–146

2009
[14]

In the money: An analysis of monetary value of chips and player outcomes in no limit texas hold’em poker tournaments,

R. H. Scott, III, M. Sher, and M. T. Paz, “In the money: An analysis of monetary value of chips and player outcomes in no limit texas hold’em poker tournaments,” The Journal of Gambling Business and Economics , vol. 17, no. 1, pp. 75–98, 2024

2024
[15]

The independent chip model and risk aversion,

G. T. Gilbert, “The independent chip model and risk aversion,” 2009. [Online]. Available: https://arxiv.org/abs/0911.3100

2009 arXiv
[16]

Is poker different from flipping coins?

T. Henke, “Is poker different from flipping coins?” Master’s thesis, Washington University in St. Louis, 2007

2007

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

H. W. Kuhn, 9. A SIMPLIFIED TWO-PERSON POKER . Princeton: Princeton University Press, 1951, pp. 97–104. [Online]. Available: https://doi.org/10.1515/9781400881727-010

doi:10.1515/9781400881727-010 1951

[2] [2]

A parameterized family of equilibrium profiles for three-player kuhn poker,

D. Szafron, R. Gibson, and N. Sturtevant, “A parameterized family of equilibrium profiles for three-player kuhn poker,” in Proceedings of the 2013 International Conference on Autonomous Agents and Multi-Agent Systems, ser. AAMAS ’13. Richland, SC: International Foundation for Autonomous Agents and Multiagent Systems, 2013, p. 247–254

work page 2013

[3] [3]

Aivat: a new variance reduction technique for agent evaluation in imperfect information games,

N. Burch, M. Schmid, M. Morav ´c´ık, D. Morill, and M. Bowling, “Aivat: a new variance reduction technique for agent evaluation in imperfect information games,” in Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational ...

work page 2018

[4] [4]

Heads-up limit hold’em poker is solved,

M. Bowling, N. Burch, M. Johanson, and O. Tammelin, “Heads-up limit hold’em poker is solved,” Commun. ACM , vol. 60, no. 11, pp. 81–88, 2017. [Online]. Available: https://doi.org/10.1145/3131284

doi:10.1145/3131284 2017

[5] [5]

DeepStack: Expert-level artificial intelligence in heads-up no-limit poker,

M. Morav ˇc´ık, M. Schmid, N. Burch, V . Lis ´y, D. Morrill, N. Bard, T. Davis, K. Waugh, M. Johanson, and M. Bowling, “DeepStack: Expert-level artificial intelligence in heads-up no-limit poker,” Science, vol. 356, no. 6337, pp. 508–513, 2017. [Online]. Available: https://www.science.org/doi/abs/10.1126/science.aam6960

doi:10.1126/science.aam6960 2017

[6] [6]

Superhuman AI for heads-up no-limit poker: Libratus beats top professionals,

N. Brown and T. Sandholm, “Superhuman AI for heads-up no-limit poker: Libratus beats top professionals,” Science, vol. 359, no. 6374, pp. 418–424, 2018. [Online]. Available: https://www.science.org/doi/ abs/10.1126/science.aao1733

doi:10.1126/science.aao1733 2018

[7] [7]

Superhuman AI for multiplayer poker,

——, “Superhuman AI for multiplayer poker,” Science, vol. 365, no. 6456, pp. 885–890, 2019. [Online]. Available: https://www.science.org/ doi/abs/10.1126/science.aay2400

doi:10.1126/science.aay2400 2019

[8] [8]

Poker playing as a dramaturgical event— Bull power, the meaning and commitment for efficacious gamesmanship,

S. H. Boyd, “Poker playing as a dramaturgical event— Bull power, the meaning and commitment for efficacious gamesmanship,” 1975

work page 1975