Pith. sign in

REVIEW 2 major objections 1 minor 3 cited by

PokerSkill: LLMs Can Play Expert-Level Poker without Training or Solvers

T0 review · 2 major / 1 minor · reviewed 2026-06-29 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Combining human poker rules with LLMs produces competitive play without training or solvers.

desk verdict The paper's core claim is that a human-expert skill library plus deterministic retrieval lets LLMs reach competitive poker without training or solvers, but the abstract gives no hand counts, variance controls, or retrieval-coverage checks to back it up. read the letter →

arxiv 2605.30094 v1 pith:TQHJZYLN submitted 2026-05-28 cs.AI cs.GT

Boning Li , Baoxiang Wang , Longbo Huang This is my paper
classification cs.AIcs.GT
keywords pokerlargelanguagemodelsimperfectinformationgamestraining-freeagentssolver-freemethodsstrategicdecisionmakinggametheoryoptimalplay
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper establishes that large language models reach competitive poker performance in this imperfect-information game when guided by a structured set of human-expert rules rather than used alone or trained from scratch. A deterministic context engine pulls only the relevant skill fragments for the current state from a layered library, restricting the model to sensible actions. This hybrid reduces losses against the GTOWizard benchmark by 49 to 61 percent relative to direct prompting and beats the strong bot Slumbot. The authors report that neither the rules nor the LLM suffices by itself, yet the pairing yields an agent that needs neither training nor solver access. The result indicates a route to capable strategic agents in similar domains without the heavy computation required for equilibrium methods.

What carries the argument

The deterministic context engine that retrieves relevant fragments from the human-expert layered skill library to ground and constrain LLM action selection in poker states.

What would settle it

A large-scale evaluation of the agent against a stronger equilibrium solver that checks whether the reported mbb/hand loss range holds or widens substantially on particular hand categories or game variants.

Watch

Extended reading notes

Core claim

PokerSkill is a training-free and solver-free framework that uses detailed rule-based poker skills as a structured action-grounding interface for LLMs. A deterministic context engine analyzes the current state and retrieves only the relevant fragments from a layered skill library designed by human poker experts, constraining the LLM's choice to reasonable actions. Against GTOWizard, GPT-5.5 XHigh with PokerSkill achieves -57 ± 21 mbb/hand, Claude Opus 4.6 achieves -80 ± 29 mbb/hand and Claude Opus 4.7 achieves -87 ± 64 mbb/hand, reducing losses by 49-61% compared to default-prompt baselines and outperforming Slumbot.

Load-bearing premise

The human-expert skill library comprehensively covers every relevant game situation and the context engine retrieves the correct fragments without omission or bias.

Editorial extensions

If this is right

  • LLMs guided by expert rules can reach performance levels previously requiring millions of core-hours of counterfactual regret minimization.
  • Pure rule-based agents remain below equilibrium play while pure LLMs also fall short, but the combination closes much of the gap.
  • The resulting agent needs no game-specific training and no solver queries during play.
  • This is presented as the first demonstration of competitive LLM performance in a complex imperfect-information game without those resources.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The method could be tested in other imperfect-information settings such as bridge or multi-agent negotiation.
  • An adaptive version of the skill library might further improve results if the deterministic retrieval misses edge cases.
  • The work implies that explicit expert structure can address LLM weaknesses in long-horizon strategic planning.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 1 minor

Summary. The paper introduces PokerSkill, a training-free and solver-free framework that combines a human-expert-designed layered rule-based skill library with LLMs via a deterministic context engine for action grounding in poker; it reports that GPT-5.5 XHigh with PokerSkill achieves -57 ±21 mbb/hand against GTOWizard (with similar results for Claude variants), reducing losses by 49-61% vs. default LLM baselines and outperforming Slumbot, claiming this is the first LLM competitive performance in a complex imperfect-information game without training or solvers.

Significance. If the central results hold after verification, the work would be significant for demonstrating a hybrid approach that leverages interpretable rule-based skills to constrain LLMs without game-specific training or equilibrium solvers, potentially generalizable to other domains; the public code release at https://github.com/lbn187/PokerSkill is a clear strength supporting reproducibility.

major comments (2)
  1. [Abstract] Abstract: the performance numbers (e.g., -57 ±21 mbb/hand, -80 ±29 mbb/hand) are reported with error bars and comparisons to GTOWizard and Slumbot, but without any mention of total hands played, variance-reduction methods, or controls for selection effects in the skill library, which directly limits verification of the 49-61% loss-reduction claim and the assertion that the combination outperforms both components alone.
  2. [Abstract] Abstract / context-engine description: the strongest claim—that rule-based skills plus LLM yield competitive play precisely because the library supplies all relevant fragments and the engine retrieves them without omission or bias—is load-bearing, yet the manuscript provides no coverage metric, retrieval-error audit, or ablation study isolating cases where the deterministic engine might fail (e.g., unusual board textures or multi-way pots), leaving open whether performance gains can be attributed to the claimed mechanism.
minor comments (1)
  1. [Abstract] Abstract: model identifiers such as 'GPT-5.5 XHigh' and 'Claude Opus 4.6/4.7' should be defined or footnoted for clarity, as they do not correspond to standard public releases.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their insightful comments on our manuscript. We address each major comment below and indicate the revisions we will make to improve the clarity and verifiability of our results.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the performance numbers (e.g., -57 ±21 mbb/hand, -80 ±29 mbb/hand) are reported with error bars and comparisons to GTOWizard and Slumbot, but without any mention of total hands played, variance-reduction methods, or controls for selection effects in the skill library, which directly limits verification of the 49-61% loss-reduction claim and the assertion that the combination outperforms both components alone.

    Authors: We agree that the abstract would benefit from additional context on the experimental methodology. The full manuscript details the evaluation over a large number of hands with standard error for the reported metrics, and the skill library was designed by experts to cover typical poker scenarios without selection bias. To address this, we will revise the abstract to include a note on the scale of the experiments and the variance estimation approach. We will also ensure the main text explicitly discusses the controls used in the skill library design. revision: yes

  2. Referee: [Abstract] Abstract / context-engine description: the strongest claim—that rule-based skills plus LLM yield competitive play precisely because the library supplies all relevant fragments and the engine retrieves them without omission or bias—is load-bearing, yet the manuscript provides no coverage metric, retrieval-error audit, or ablation study isolating cases where the deterministic engine might fail (e.g., unusual board textures or multi-way pots), leaving open whether performance gains can be attributed to the claimed mechanism.

    Authors: The paper emphasizes that the deterministic context engine retrieves relevant skills based on explicit state analysis, and the results show the hybrid approach outperforms both rule-based and LLM-only baselines. We acknowledge that explicit coverage metrics would further support the mechanism. In the revised manuscript, we will include a dedicated analysis of the engine's retrieval coverage, including audits for various board textures and pot scenarios, to demonstrate the absence of systematic omission or bias. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity; empirical results rest on external benchmarks and human-designed inputs

full rationale

The paper presents an empirical framework (PokerSkill) that combines an externally authored human-expert skill library with LLMs via a deterministic retrieval engine. All reported performance numbers are measured against the independent third-party GTOWizard benchmark and compared to external baselines (Slumbot, default-prompt LLMs). No equations, fitted parameters, or self-referential derivations appear; the skill library is stated to be 'entirely designed by human poker experts' and the evaluation does not reduce any claimed quantity to quantities defined inside the method itself. This is the normal case of a self-contained empirical claim against external references.

Assumptions & free parameters 0 free parameters · 1 assumptions · 0 invented entities

The central claim depends on the completeness and correctness of the human-designed skill library and the accuracy of the context engine's retrieval; these are not derived from data or first principles within the paper.

assumptions (1)
  • domain assumption A layered skill library written by human poker experts contains the necessary knowledge to constrain LLM decisions to expert-level actions across all states.
    The framework treats this library as the source of strategic grounding; its quality is presupposed rather than proven.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PokerSkill: LLMs Can Play Expert-Level Poker without Training or Solvers." pith.science (2026). https://pith.science/paper/TQHJZYLN

@misc{pith2026260530094,
  author       = {Pith},
  title        = {Pith review of: PokerSkill: LLMs Can Play Expert-Level Poker without Training or Solvers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TQHJZYLN}},
  note         = {Machine review of arXiv:2605.30094}
}
abstract

Poker is a landmark challenge for artificial intelligence. The dominant approach relies on equilibrium solvers built on counterfactual regret minimization, requiring millions of core-hours of training. Large Language Models (LLMs) possess extensive poker knowledge but perform far below solver-based agents when asked to play directly. Traditional rule-based poker agents are interpretable and training-free, but their strategic ceiling remains far below equilibrium play. We introduce \textbf{PokerSkill}, a training-free and solver-free framework that bridges this gap by using detailed rule-based poker skills as a structured action-grounding interface for LLMs. A deterministic context engine analyzes the current state and retrieves only the relevant fragments from a layered skill library, which is entirely designed by human poker experts, constraining the LLM's choice to reasonable actions. Against GTOWizard, a state-of-the-art GTO benchmark, GPT-5.5 XHigh with PokerSkill achieves $-57 \pm 21$ mbb/hand, Claude Opus 4.6 achieves $-80 \pm 29$ mbb/hand and Claude Opus 4.7 achieves $-87\pm 64$ mbb/hand, reducing losses by 49--61\% compared to default-prompt baselines and outperforming the strong bot Slumbot. Our key finding is that rule-based skills alone do not constitute a strong strategy, and LLMs alone cannot play well, but their combination yields an agent that requires neither training nor solver access yet competes with systems built on millions of core-hours of computation. To our knowledge, this is the first demonstration of an LLM achieving competitive performance in a complex imperfect-information game without game-specific training or solver queries. Code is available at https://github.com/lbn187/PokerSkill.

Figures

Figures reproduced from arXiv: 2605.30094 by the authors.

Figure 1
Figure 1. Evolution of poker AI systems across three paradigms. Rule-based agents (green) offer interpretabil [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of PokerSkill versus the default prompting baseline. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. AIVAT loss rate against GTOWizard Benchmark. PokerSkill substantially closes the gap between [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. AV-AIVAT: 74x Cheaper Agent Evaluation with Certified Anytime-Valid Stopping in Imperfect-Information Games

    cs.GT 2026-08 conditional novelty 7.0 of 10

    Combining AIVAT variance reduction with anytime-valid confidence sequences lets poker agent evaluations stop at a median 74x fewer hands at plus or minus 1 BB, with exact finite-sample certification currently limited ...

  2. Agents That Certify Their Own Exploits: Confidence-Scheduled Restricted Responses for Safe Opponent Exploitation

    cs.GT 2026-07 accept novelty 6.5 of 10

    CS-RNR certifies each candidate exploit by full-tree best response before atomic deployment, so model error can cost gain but not the reference-relative safety budget.

  3. Correlated Chance Sampling for Monte Carlo Counterfactual Regret Minimization

    cs.GT 2026-07 conditional novelty 6.0 of 10

    Persistent randomized Weyl streams at each chance node cut MCCFR exploitability 19–34% on Kuhn and Leduc poker with local O(log N/N) frequency guarantees and no new hyperparameters.

Reference graph

Works this paper leans on

23 extracted references · cited by 3 Pith papers

  1. [1]

    Other opponents (Slumbot, rule-based bots, other LLMs) would require raw outcome evaluation, demanding∼30×more hands for equivalent statistical power

    AIV AT availability.GTOWizard is the only publicly available HUNL benchmark that provides AIV AT variance-reduced evaluation. Other opponents (Slumbot, rule-based bots, other LLMs) would require raw outcome evaluation, demanding∼30×more hands for equivalent statistical power

  2. [2]

    It beats the 2018 ACPC champion Slumbot by 194±41 mbb/hand over 150,000 hands

    Benchmark strength.GTOWizard represents current state-of-the-art GTO solving. It beats the 2018 ACPC champion Slumbot by 194±41 mbb/hand over 150,000 hands. Evaluating against a weaker opponent would be less informative about the ceiling of LLM poker play

    2018

  3. [3]

    top pair

    Economic constraints.Frontier LLM inference with extended reasoning is expensive. The total evaluation cost for results reported in this paper exceeds $2,500. Multi-opponent evaluation without AIV AT would multiply this cost by∼30×, making comprehensive ablation studies economically impractical. F.3 Experimental Details LLM Model Cost/hand Settings GPT-5....

  4. [5]

    - High-card / paired boards: c-bet ~25% pot at high frequency

    Flop C-BET: As AGGRESSOR, c-bet is default on most boards. - High-card / paired boards: c-bet ~25% pot at high frequency. - Default dry boards: c-bet ~65% pot at moderate frequency. - Wet / low connected boards (e.g. T98, 753): OOP aggressor -> range check in SRP/3BP. IP aggressor -> 33 lower freq with larger size. - In 4BP+: range c-bet ~20% pot on almos...

  5. [6]

    - Pure trash (no pair, no draw) -> FOLD to any bet

    HAND STRENGTH FIRST: Check [YOUR HAND STRENGTH] below BEFORE applying MDF/pot odds. - Pure trash (no pair, no draw) -> FOLD to any bet. MDF does NOT apply to unplayable hands

  6. [8]

    - When it says CHECK -> you MUST check, regardless of board texture, position, or scenario suggestions

    ATTACK BUDGET IS BINDING: The >>> ATTACK BUDGET <<< line in SITUATION ANALYSIS is the FINAL authority on whether to bet. - When it says CHECK -> you MUST check, regardless of board texture, position, or scenario suggestions. - When it says BET -> betting is allowed (choose sizing from options). Not mandatory. - ATTACK BUDGET already integrates hand streng...

  7. [13]

    action":

    [Attention] READ the [YOUR HAND STRENGTH] section carefully -- it already classified your hand. Trust it over your own quick judgment. [POT TYPE ADJUSTMENT] - 3-bet pot: Both players have NARROWER, STRONGER ranges. Recalibrate hand strength: top pair weak kicker ~ SRP second pair. Overpair below board top card is vulnerable. Two pair+ gains MORE value -- ...

  8. [16]

    - Pure trash (no pair, no draw) -> FOLD to any bet

    HAND STRENGTH FIRST: Check [YOUR HAND STRENGTH] below BEFORE applying MDF/pot odds. - Pure trash (no pair, no draw) -> FOLD to any bet. MDF does NOT apply to unplayable hands. 37

Show all 23 references
  1. [18]

    - When it says CHECK -> you MUST check, regardless of board texture, position, or scenario suggestions

    ATTACK BUDGET IS BINDING: The >>> ATTACK BUDGET <<< line in SITUATION ANALYSIS is the FINAL authority on whether to bet. - When it says CHECK -> you MUST check, regardless of board texture, position, or scenario suggestions. - When it says BET -> betting is allowed (choose siz...

  2. [19]

    What is your absolute hand strength? (e.g., top pair good kicker, bottom pair, flush draw)

  3. [20]

    [Flop/Turn Only] What draws do you have? (flush draw, straight draw, backdoor draws, combo draws)

  4. [21]

    What is your relative hand strength? Relative strength depends on: absolute strength, board wetness (wet boards devalue non-nut hands), and opponent's action history (aggressive actions = stronger ranges)

  5. [22]

    What is your plan for future streets/opponent actions? (bet 3 streets, check one street, give up)

  6. [23]

    action":

    [Attention] READ the [YOUR HAND STRENGTH] section carefully -- it already classified your hand. Trust it over your own quick judgment. [POT TYPE ADJUSTMENT] - 3-bet pot: Both players have NARROWER, STRONGER ranges. Recalibrate hand strength: top pair weak kicker ~ SRP second p...

  7. [24]

    - DONK BET (15% pot) is viable in specific spots: flop low board (high card <= 8, no board pair), or turn/river when a non-top card pairs the board or board becomes double-paired

    DEFENDER OOP -- CHECK OR DONK: If you are the DEFENDER and OOP, your DEFAULT action is CHECK to the aggressor (let them bet, then check-call or check-raise). - DONK BET (15% pot) is viable in specific spots: flop low board (high card <= 8, no board pair), or turn/river when a ...

  8. [25]

    - High-card / paired boards: c-bet ~25% pot at high frequency

    Flop C-BET: As AGGRESSOR, c-bet is default on most boards. - High-card / paired boards: c-bet ~25% pot at high frequency. - Default dry boards: c-bet ~65% pot at moderate frequency. - Wet / low connected boards (e.g. T98, 753): OOP aggressor -> range check in SRP/3BP. IP aggre...

  9. [26]

    - Pure trash (no pair, no draw) -> FOLD to any bet

    HAND STRENGTH FIRST: Check [YOUR HAND STRENGTH] below BEFORE applying MDF/pot odds. - Pure trash (no pair, no draw) -> FOLD to any bet. MDF does NOT apply to unplayable hands. 41

  10. [27]

    - IP: check back for pot control (don't bloat pot with marginal hands)

    POT CONTROL: Medium-strength hands control pot size. - IP: check back for pot control (don't bloat pot with marginal hands). If the oop aggressor check to you, you can stab ~25% pot and then check to the river. - OOP: check is always the mainly option. Sometimes if the IP aggr...

  11. [28]

    - When it says CHECK -> you MUST check, regardless of board texture, position, or scenario suggestions

    ATTACK BUDGET IS BINDING: The >>> ATTACK BUDGET <<< line in SITUATION ANALYSIS is the FINAL authority on whether to bet. - When it says CHECK -> you MUST check, regardless of board texture, position, or scenario suggestions. - When it says BET -> betting is allowed (choose siz...

  12. [29]

    What is your absolute hand strength? (check [YOUR HAND STRENGTH] below -- trust its classification)

  13. [30]

    There are NO draws on the river -- all draws have either completed or missed

  14. [31]

    What is your relative hand strength? Consider: board texture, opponent's action history across all streets, and what hands opponent is representing

  15. [32]

    action":

    Is this a VALUE BET, BLUFF, or CHECK/CALL situation? Value bet strong hands, bluff with no showdown value + good blockers, check back medium hands. [POT TYPE ADJUSTMENT] - 3-bet pot: Both players have NARROWER, STRONGER ranges. Recalibrate hand strength: top pair weak kicker ~...

Pith tools

Reviewed June 29, 2026 · model on record in the stance chip above.