{"id":"3156bbb7-d2ba-4087-bfea-200807f8b22a","arxiv_id":"2607.06854","paper_version":1,"verdict":"ACCEPT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":4,"one_line_summary":"A controlled study using a fixed Gin Rummy expert as a yardstick isolates which lightweight RL training choices help (TRPO, knock-first reward, curriculum, warm-start, best-checkpoint) and which fail (imitation, dense rewards, embeddings, LLM opponents), finding the performance ceiling is set by the","lead":"This paper builds a fixed rule-based expert for Gin Rummy to systematically test which lightweight RL training choices actually improve agent strength. It finds that trust-region updates, knock-first rewards, opponent curricula, warm-starting, and checkpoint selection each help, while imitation learning, dense rewards, learned embeddings, and live LLM opponents fail for identifiable reasons.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"The 'information-bound ceiling' claim rests on an oracle-vs-fair-search gap measured on a search method, not on the trained agents themselves; the paper never tests whether giving the trained agent opponent-hand observations would raise its ceiling.","rationale":"The reader's weakest_assumption—that the expert might be an exploitable yardstick—is valid but somewhat addressed by the paper's explicit framing (strong, reproducible, cheap, not claimed optimal). My concern is adjacent but more load-bearing for the interpretive claim: the evidence for 'information-bound' comes from a search-method comparison rather than a direct test on the trained agents. However, this concern does not undermine the paper's primary empirical contributions—the controlled study of training choices, the negative results, or the recipe itself. The architecture sweep does directly support 'not capacity-bound.' The information-bound claim is an interpretation that would be strengthened by the proposed test but is not refuted by any evidence in the paper. The paper is honest about limitations and frames the ceiling as a 'measurement, not a disappointment.' The verdict remains ACCEPT because the empirical methodology is sound, the contributions are clear, and the interpretive claim—while not fully proven—is reasonably argued and explicitly hedged. The concern lowers confidence slightly but does not move the verdict.","tokens_in":14850,"tokens_out":3970,"duration_ms":237420,"concrete_test":"Retrain the stacked-recipe TRPO agent with the observation augmented to include the opponent's current hand (e.g., a 5×52 tensor adding an opponent-hand plane). Run 2000 evaluation games against the fixed expert with the same protocol. If win-rate rises substantially above 34% (toward the oracle's 85%), the information-bound claim is confirmed. If it stays in the 30–38% band, the ceiling is set by something other than hidden information (e.g., optimization, reward design, or the expert's specific exploitable strategy).","verdict_should_be":"ACCEPT","load_bearing_attack":"The paper's central interpretive claim is that the ~34% ceiling is 'information-bound, not capacity-bound.' Three lines of evidence are offered: (1) architecture sweep shows overlapping CIs, (2) fair ISMCTS achieves only 26% while oracle ISMCTS reaches 85%, (3) the reactive policy lacks opponent-hand estimation. Evidence (1) directly tests capacity on the trained agents and is solid. But evidence (2)—the strongest support for the 'information' part—compares two variants of a *search method*, not two variants of the *trained agent*. The trained agents at 34% already exceed the fair search at 26%, meaning they implicitly extract something the fair search lacks. The 26→85 gap therefore measures the value of perfect information *for search*, not the value of information *for the trained policy*. The paper never runs the most direct test: retrain the same agent with the opponent's hand added to the 4×52 observation. If that agent stays near 34%, the ceiling is not information-bound; if it jumps toward 85%, the claim is confirmed. Without this test, 'information-bound' is an inference from a non-comparable system. The paper acknowledges opponent-hand inference as future work but still states the ceiling as an established finding in the abstract and conclusion.","agreement_with_reader":"partial"},"referee_report":{"model":"glm-5.2","summary":"This paper presents a controlled study of training choices for lightweight reinforcement-learning agents in imperfect-information card games, using Gin Rummy as the primary testbed. The authors build a fixed, deterministic, rule-based expert (used only for evaluation, never for training) and run over one hundred controlled experiments, changing one factor at a time, to isolate which ingredients help: TRPO over PPO, a knock-first reward, a rising opponent curriculum, warm-starting, and keeping the best checkpoint. Stacking these lifts a self-play champion from ~30% to ~34.2% win-rate against the expert. Several popular approaches fail (learned embeddings, imitation/DAgger, dense step rewards, live LLM opponent), each with an identified mechanism. An architecture sweep (MLP, convolutional, Deep Sets, recurrent, attention) finds no encoder breaks the ceiling. A fair ISMCTS baseline reaches only 26% while an oracle variant reaches 85%, which the authors interpret as evidence that the ceiling is information-bound rather than capacity-bound. The method is replicated on Leduc Hold'em against a CFR-computed optimum.","tokens_in":15035,"tokens_out":4097,"duration_ms":232639,"significance":"The paper makes a solid methodological contribution by demonstrating the value of a fixed, strong, reproducible reference opponent for grading RL training choices in imperfect-information games—a setting where evaluation is notoriously difficult. The controlled, one-factor-at-a-time experimental design, the use of IQM with stratified bootstrap CIs for the architecture sweep, the honest reporting of negative results with mechanistic explanations, and the release of a reusable pipeline are all commendable. The finding that reward shaping cannot bribe an agent into a losing habit (chasing gin) is a clean and memorable result. The Leduc Hold'em replication against a computable optimum adds external validation. The code release is a genuine strength.","major_comments":[{"comment":"The central interpretive claim—that the ~34% ceiling is 'information-bound, not capacity-bound' (Abstract; Discussion, 'An information-bound ceiling'; Conclusion)—is supported by three lines of evidence, but the strongest of these rests on a comparison between two variants of a search method (ISMCTS), not of the trained agent. Specifically, the 26% (fair ISMCTS) vs. 85% (oracle ISMCTS) gap in Figure 5b measures the value of perfect information for search, not for the trained policy. The trained agents already exceed the fair search (34% vs. 26%), meaning they implicitly extract something the fair ISMCTS lacks, so the two systems are not directly comparable. The most direct test—retraining the same agent architecture with the opponent's hand added to the 4×52 observation—is not performed. The paper acknowledges opponent-hand inference as future work (Discussion, 'Future work'), yet states","section":null},{"comment":"The headline result of 34.2±2.1% (Table 1) appears to be from a single training run evaluated over 2000 games, where the confidence interval reflects binomial evaluation noise rather than training-seed variability. In contrast, the architecture sweep (Figure 5a) properly uses IQM with stratified bootstrap CIs over multiple seeds, following Agarwal et al. (2021). The headline result should be reported with the same multi-seed protocol to establish that the stacked recipe reliably achieves ~34% rather than reflecting a single favorable run. The two PFSP sibling agents at 34.0% provide partial evidence of reproducibility, but they use a different opponent-sampling scheme, so they do not directly replicate the headline recipe.","section":null},{"comment":"The Leduc Hold'em result (Table 3) creates a tension with the information-bound claim that is not addressed. The tabular self-play learner reaches near-parity with the CFR optimum (mean return −0.085), yet Leduc is also an imperfect-information game with hidden cards. If the ceiling is fundamentally set by hidden information, one would expect a gap in Leduc as well. The likely explanation is that Leduc is small enough for tabular methods to effectively reason over the hidden information, but this means the ceiling is a function of information complexity relative to model expressiveness and game size—not purely 'information-bound' as a general principle. The paper should either scope the claim to Gin Rummy specifically or discuss why the Leduc near-parity does not contradict the general claim.","section":null}],"minor_comments":[{"comment":"The abstract states the stacked recipe lifts the agent 'from about 30 to 36 percent against the expert,' but Table 1 reports 34.2±2.1%. The value 36 appears to correspond to the upper bound of the 95% CI (36.3), not the point estimate. The abstract should report the point estimate.","section":null},{"comment":"The architecture sweep (Figure 5a) retrains from scratch without warm-starting, since a different network shape cannot warm-start from the MLP champion. The best IQM in the sweep is ~31% (convolutional), below the headline 34.2% which uses warm-start. The paper should explicitly note that the architecture sweep tests a different (and harder) configuration than the headline recipe, so the 'architecture does not break the ceiling' claim is relative to the from-scratch baseline, not the full stacked recipe.","section":null},{"comment":"In the 'Reward: you cannot pay the agent into ginning' section, the text states 'paying three times more for a gin than a knock leaves the gin rate under one percent.' It would help to specify the exact reward coefficients used for each condition in the sweep, perhaps in a table or appendix, for reproducibility.","section":null},{"comment":"The self-attention encoder result is reported as unable to train under TRPO due to double-differentiation cost, and under PPO it does not beat a plain MLP (17% vs. 21%). Since the attention encoder is the most natural fit for an unordered card set, a brief discussion of whether this is a fundamental limitation or an implementation artifact would strengthen the architecture conclusion.","section":null},{"comment":"The paper mentions a 'web game (human vs. agent)' component in Figure 1 but does not report any human-play results or evaluation. If this is only a tool release, that should be clarified; if human data was collected, it should be reported or removed from the figure.","section":null},{"comment":"Table 2 lists 'Pay three times more for gin' as 'no effect,' but the text also notes that against weak opponents, a gin-first reward does increase the gin rate (22% vs. 97% knocks). The verdict 'no effect' should be qualified as 'no effect against the expert' to avoid confusion.","section":null}],"recommendation":"major_revision","confidential_remarks":"The paper is well-executed and the experimental work is careful, but the central interpretive claim ('information-bound ceiling') is overstated relative to the evidence. The missing experiment—retraining with opponent-hand observations—is inexpensive and would either confirm or refute the claim decisively. Without it, the claim should be substantially hedged or the experiment added. The single-seed headline result is also a concern given that the authors themselves advocate for robust evaluation elsewhere in the paper. I would be inclined toward accept if these two issues are addressed, either by running the missing experiment/multi-seed evaluation or by appropriately scoping the claims."},"author_rebuttal":null,"desk_editor":{"model":"glm-5.2","letter":"The paper builds a fixed rule-based expert for Gin Rummy, never trains against it, and uses it purely as a yardstick to grade 100+ controlled experiments on lightweight RL agents. That methodological pattern—build a strong fixed reference, change one factor at a time, measure against it—is the real contribution here, and it's executed cleanly. The finding that strong play almost never gins (under 2%) and that no reward scheme can bribe the agent into chasing gin is a genuinely useful result. The negative results (DAgger fails from causal confusion, dense rewards make the agent short-sighted, LLM opponent is too slow) are each diagnosed with a reason, not just reported. Code is released. The Leduc Hold'em replication against a CFR optimum is a nice generality check. The statistics are above average for the subfield: IQM with stratified bootstrap CIs, seat rotation, 2000 games for headline numbers. The architecture sweep showing overlapping CIs across MLP, conv, Deep Sets, and LSTM is solid evidence that capacity is not the bottleneck. Where the paper overreaches is the claim that the ceiling is 'information-bound.' The strongest piece of evidence is the gap between fair ISMCTS (26%) and oracle ISMCTS (85%). But that gap measures the value of hidden information for a search method, not for the trained policy. The trained agents at 34% already beat the fair search at 26%, so they're extracting something the search lacks. The most direct test—retrain the same agent with the opponent's hand added to the observation—never appears. If that agent stays near 34%, the claim is confirmed; if it jumps, the ceiling is something else. The paper acknowledges opponent-hand inference as future work but still states the information-bound conclusion as established in the abstract and conclusion. This is a real gap but not a fatal one; the architecture evidence and the oracle comparison are suggestive even if not conclusive. The reader's take (accept, moderate confidence) is about right. The weakest assumption flagged by the reader—that the expert is a valid yardstick—is explicitly acknowledged and reasonable. The stress-test concern lands: the information-bound claim is an inference from a non-comparable system. But the paper's empirical work is careful and the methodological pattern is worth disseminating. Recommend accept for peer review; ask the authors to soften the information-bound framing from established fact to supported hypothesis.","headline":"Solid empirical study with a clean methodology; the central interpretive claim is slightly oversold but the paper deserves a serious referee.","tokens_in":15581,"tokens_out":560,"would_cite":false,"duration_ms":114104,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"Hidden information, not network size, caps lightweight game agents","keywords":[],"falsifier":"Construct a training method that does not practice against the expert but achieves a win-rate against it significantly above 34 percent—say, above 40—while using a network of comparable size. Alternatively, show that a different fixed expert (still strong, cheap, and deterministic) produces a different ranking of the same training choices, which would undermine the claim that the yardstick cleanly isolates which ingredients matter.","tokens_in":15143,"feed_emoji":"🃏","tokens_out":1203,"duration_ms":211751,"temperature":0.7,"pith_summary":"The paper builds a fixed, strong, rule-based expert for Gin Rummy and uses it purely as a measuring stick—never as a training partner—to run over a hundred controlled experiments that isolate which training choices actually make a lightweight reinforcement-learning agent stronger. The central finding is twofold. First, a specific recipe works: trust-region updates (TRPO over PPO), a reward that favors early knocking over chasing gin, a curriculum of progressively tougher opponents, warm-starting from a prior champion, and shipping the best checkpoint rather than the last one. Stacking these lifts a self-play agent from roughly 30 to about 34 percent against the expert. Second, and more fundamentally, the remaining performance ceiling is set by hidden information, not by model capacity or training technique. Every architecture tested—MLP, convolutional, permutation-invariant set encoders, attention, and recurrent networks—lands in a statistically indistinguishable band. A fair search that cannot see the opponent's cards tops out at 26 percent, below the trained agents, while the same search given oracle access to hidden cards reaches 85 percent. That 26-versus-85 gap is the paper's key piece of evidence: the bottleneck is what the agent does not know, not how big or clever its network is.","feed_headline":"Hidden info, not model size, caps card-game AI","feed_subtitle":"A fixed expert yardstick reveals which training tricks help and why the ceiling is what you don't know, not how big your network is","key_machinery":"The fixed rule-based expert serves as the central measuring instrument. It solves one subproblem exactly—finding the lowest-deadwood meld decomposition of a hand—and otherwise plays deterministic, principled endgame heuristics: draw only when it strictly lowers deadwood, discard the card that minimizes resulting deadwood, and knock as soon as legally possible. This expert never enters the training loop; it only grades agents. The training pipeline itself centers on a masked actor-critic (illegal actions get a large finite negative logit before softmax, so both PPO and TRPO can use the same network), a three-stage opponent curriculum (random, then a pool of past checkpoints, then self-play),","core_discovery":"The performance ceiling for a lightweight agent in an imperfect-information card game is information-bound rather than capacity-bound. The paper establishes this through three converging lines of evidence: (1) varying network architecture across MLP, convolutional, set-based, attention, and recurrent encoders does not meaningfully change win-rate against a fixed expert, with all confidence intervals overlapping; (2) a determinized search graded fairly—re-dealing hidden cards before each rollout—tops out at 26 percent, below the trained agents, while the same search given oracle access to the true hidden state reaches 85 percent, directly quantifying the value of the concealed information; (3","pith_inferences":["The information-bound ceiling argument implicitly assumes the fixed expert is not itself exploitable in ways a different training method could discover. If the expert has systematic weaknesses that self-play cannot find but, say, curriculum-based exploitation could, then the 34-percent ceiling is partly an artifact of the yardstick rather than a fundamental information limit.","The failure of learned state embeddings to beat the raw 4×52 sparse observation may be specific to Gin Rummy's small observation space. Games with larger or more structured hidden-state spaces (e.g., Mahjong, DouDizhu) might still benefit from learned embeddings, so the negative result may not generalize as broadly as the paper's game-agnostic framing suggests.","The LLM opponent's competence but impractical speed (9–27 seconds per move) hints that offline distillation—generating expert games with an LLM and training on that dataset—could be a productive path the paper identifies but does not pursue, which if successful would complicate the clean information-bound narrative by injecting external strategic knowledge."],"forward_implications":["For any imperfect-information game where a strong, cheap, fixed reference opponent can be constructed, the controlled one-factor-at-a-time methodology demonstrated here can be applied to separate training choices that genuinely help from those that merely sound plausible.","The finding that reward shaping cannot bribe an agent into a losing habit—chasing gin—suggests that in adversarial settings, agents may discover strategic truths that override designer intentions, which has implications for how reward design is taught and practiced.","The 26-versus-85-percent gap between fair and oracle search provides a reusable diagnostic: for any hidden-information game, comparing fair-search performance to oracle-search performance quantifies how much of the remaining difficulty is informational versus computational.","If the information-bound ceiling is real, then the next gains in lightweight card-game agents should come from opponent-hand inference or belief-state reasoning, not from larger networks or more elaborate reward shaping."],"fun_headline_variants":["Bigger networks won't fix what hidden cards hide","Card-game AI ceiling set by hidden info, not model size","Network size doesn't break the card-game AI ceiling","A fixed expert exposes the real ceiling on card-game AI","Stronger opponents and trust regions help; bigger models don't"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The fixed rule-based expert is assumed to be a meaningful and stable yardstick for measuring agent strength. The paper acknowledges it is not a game-theoretic optimum, just a strong, cheap, reproducible heuristic. If the expert's specific endgame strategy is exploitable in a way that self-play agents systematically cannot discover but a different training method could, then the measured ceiling of roughly 34 percent would be an artifact of this particular yardstick rather a","fun_headline_variants_meta":{"raw":{"variants":["Bigger networks won't fix what hidden cards hide","Card-game AI ceiling set by hidden info, not model size","Network size doesn't break the card-game AI ceiling","A fixed expert exposes the real ceiling on card-game AI","Stronger opponents and trust regions help; bigger models don't","Oracle access lifts card-game search from 26 to 85 percent","Known tricks help card-game AI; reward shaping and LLMs don't","Information, not capacity, bounds lightweight card-game AI","A rule-based expert grades what makes card-game agents strong","Stacking known tricks lifts self-play from 30 to 36 percent"]},"model":"glm-5.2","effort":"low","cost_usd":0.0,"raw_usage":{"total_tokens":2018,"prompt_tokens":616,"completion_tokens":1402,"prompt_tokens_details":null},"tokens_in":616,"tokens_out":1402,"duration_ms":59912,"temperature":1.0,"reasoning_tokens":1287,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-10T00:09:25.809070+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"Construct a training method that does not practice against the expert but achieves a win-rate against it significantly above 34 percent—say, above 40—while using a network of comparable size. Alternatively, show that a different fixed expert (still strong, cheap, and deterministic) produces a different ranking of the same training choices, which would undermine the claim that the yardstick cleanly isolates which ingredients matter.","supporting_citations":[],"review_version":1}