{"id":"b82e53ea-7526-494e-adab-9f0c054873a4","arxiv_id":"2412.12119","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A game-only language model, MAV, can act as world model, value function, and policy for four board games, and external or internal search over its own predictions reaches grandmaster-level chess.","lead":"This paper trains a language model to play chess, Chess960, Connect Four, and Hex by predicting moves, values, and board-state changes all in text. It shows that adding search, either as an external Monte Carlo loop or as a distilled in-text search procedure, pushes the chess-playing model to roughly grandmaster strength while considering far fewer moves per decision than traditional engines.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Engine-free MCTS assumes transition errors do not prune winning lines; Table 2 measures only one-step FEN accuracy, not search-depth error, so the +340-Elo improvement may partly come from a silently biased search tree.","rationale":"The reader's weakest assumption identifies the same point: external search without an engine assumes MAV's learned transition is accurate enough that the MCTS tree contains the true game states. I agree this is the most load-bearing concern for the central claim because the engine-free external MCTS is the paper's principal novelty, and the -infinity fallback in Section 3.1 makes a transition error more consequential than simple noise: it silently discards an action and its entire subtree. The provided evidence (Table 2) supports one-step FEN accuracy on test distributions, but does not bound error accumulation over search depth or conditional on position complexity. The proposed oracle-transition ablation would directly settle whether the search improvement survives when the transition function is correct, which is the heart of the engine-free claim. I am not treating the Elo-calibration caveats or the internal-search early-stopping rule as the primary concern because those mainly affect the magnitude of the headline external numbers; the transition-fidelity issue affects whether the external-search mechanism itself is sound. The reader's CONDITIONAL verdict remains appropriate, so no change is recommended.","tokens_in":92411,"tokens_out":6784,"duration_ms":80336,"concrete_test":"Re-run the MAV-MCTS (M=2000) chess league twice: once with MAV state transitions and the -infinity parsing fallback as in Section 3.1, and once with an oracle transition function (e.g., OpenSpiel) using identical MAV priors/values, simulation counts, and final move selection. Also record per-depth parse-failure rates in actual searches and the frequency with which a failed node is the true best move according to Stockfish. If oracle-transition Elo is statistically indistinguishable from MAV-transition Elo, the transition assumption is not load-bearing; if it is materially higher, the engine-free results are unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central value of Contribution 2 is that MCTS can run without an external game engine, with MAV serving as the transition function. Section 3.1 handles hallucinations by parsing the model output and assigning -infinity to any node whose state prediction fails, \"to avoid future consideration.\" This only preserves correctness if parse failures are rare in exactly the lines that matter and do not correlate with move quality. Table 2 reports 99.6%-100% FEN accuracy on Lichess puzzles and OOD positions, but these are single-step accuracy numbers on test distributions, not cumulative or search-tree-conditioned rates. A 0.4% per-node error is not negligible over a 2000-simulation tree (roughly 2000 node evaluations per search, replayed across a game), and an error at a high-value child is not neutral: it prunes that child permanently and can remove the true best move from Final Move Selection. The paper reports no parse-failure statistics during actual MCTS, no oracle-transition ablation, and no analysis of whether failure rate increases in sharp tactical positions, which is precisely where search should add value. Without this, the +340 Elo gain of MAV-MCTS over searchless MAV could be partly an artifact of search over a silently biased subset of lines. This is the load-bearing assumption behind the engine-free external-search claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces MAV, a 2.7B-parameter decoder-only Transformer trained on textual game data to act as a policy, value function, and world model for chess, Chess960, Connect Four, and Hex. It then presents two planning methods: external search, where MAV guides an AlphaZero-style MCTS without an external game engine, and internal search, where a fine-tuned model (MAV-IS) generates a linearized minimax tree in a single call. The paper reports internal Elo gains from MCTS and maps them to external Elo, claiming Grandmaster-level chess performance at roughly 100-1000 considered moves per decision, and it reports that internal search strength increases smoothly with search budget. The appendices contain dataset statistics, tournament details, hyperparameters, algorithm pseudocode, and annotated example games.","tokens_in":92721,"tokens_out":5743,"duration_ms":65759,"significance":"If the central claims hold, the paper would be a meaningful contribution: it demonstrates that a language model can serve as a learned transition and value function inside MCTS in deterministic board games, and that a distilled search procedure can be executed in a single model call with scaling behavior in the search budget. The paper is strong in its breadth: four games, large tournament sizes (about 15k chess games, 6k Hex games, and 2k+ Connect Four games), detailed pseudocode, and an explicit discussion of limitations. It also ships a playable artifact (MAV-small) and openly discloses the dependence on engine annotation for training data. The main weaknesses are the reliability of the external Elo calibration and the lack of evidence that learned transition errors do not silently corrupt the search tree.","major_comments":[{"comment":"The headline external Elo values are derived from a linear fit of internal Elo to externally reported Stockfish ratings, but the paper gives no details of the fit, no anchor points, and no confidence intervals. Table 1's own footnote states that the external Elo estimates for MAV-MCTS agents are 'not accurate' because those agents would not be able to play at blitz time controls, which is precisely the rating scale used for the anchors. Since the abstract and introduction rely on 'Grandmaster-level performance' in chess, the external Elo values (e.g., 3209 for MAV-MCTS with 2000 simulations) should either be replaced by relative/internal Elo comparisons, or accompanied by a defensible calibration with uncertainty quantification and a clear statement of the time-control mismatch.","section":"§5.1, Table 1"},{"comment":"The engine-free external search claim rests on MAV's reliability as a transition function. Section 3.1 handles hallucinated outputs by assigning -infinity to the affected node and removing it from future consideration. Table 2 reports one-step FEN accuracy of 99.6-100%, but this is not sufficient evidence for the search setting: a per-node failure rate of 0.4% can accumulate over 2000 simulations, and a failure at a high-value child permanently prunes that line, potentially removing the true best move before Final Move Selection. The paper does not report parse-failure rates during actual MCTS searches, does not measure how transition errors grow with search depth, does not analyze whether failures correlate with sharp tactical positions, and does not provide an oracle-transition ablation. Without these analyses, the +340 internal Elo improvement of MAV-MCTS(2000) over searchless MAV could be partly an artifact of searching over a silently biased subset of the game tree. I would like to see these measurements, or, failing that, a substantially weakened claim about the engine-free variant.","section":"§3.1, Table 2"},{"comment":"The internal search results are evaluated under a different protocol from the other agents: games are stopped once either side reaches a decisive 1200+ centipawn advantage. The external Elo axis in Figure 5 (right) is therefore built from truncated games, and the magnitudes such as roughly 2930 Elo for MAV-IS(b=4,d=2) are not directly comparable to the completed-game Elo of the other agents. The qualitative claim that internal search strength scales with the search budget may survive, but the quantitative Elo levels are not supported under a common protocol. Please either report a consistent evaluation protocol for all agents or analyze how the truncation threshold changes the resulting ratings.","section":"§5.4 and footnote 3"},{"comment":"The paper states that for Hex, 'only a game engine is used to determine legal actions, state transitions, and terminal states' because the final model was not adequately trained for state tracking in Hex. Consequently, the Hex entries in Table 1 for MAV-MCTS do not demonstrate Contribution 2, the engine-free external search method. The abstract and Contribution 1 say the model reliably captures transition and value functions in 'the respective environments,' which is misleading for Hex. The claim should be qualified to the games where state-tracking MAV was actually used as the transition model.","section":"§5.3"}],"minor_comments":[{"comment":"The sentence 'The average amount of time taken per search is shown in Table 1' appears to refer to Table 3 in the same appendix; please correct the cross-reference.","section":"Appendix A.2"},{"comment":"The phrase 'As an additional, we include the Ext-BoN model' should be 'As an additional baseline, we include the Ext-BoN model.'","section":"§5.1"},{"comment":"The table is titled 'Error rate analysis' but the reported values are accuracies (e.g., 0.999, 0.996). Please rename the table or invert the metrics to avoid confusion.","section":"Table 2"},{"comment":"There is a typo in 'Language Agent Tree Seatch (LATS)' in the related work; it should be 'Search.'","section":"Appendix E"},{"comment":"The external MCTS hyperparameters are numerous (prior temperature, top-k, epsilon, batch size, timeout, dynamic virtual counts), and the paper reports only final values without a sensitivity analysis. At minimum, please state whether the simulation-count trend in Figure 5 (left) is stable across reasonable variations of these hyperparameters.","section":"§5.3, Appendix A.3"},{"comment":"The generalization analysis in Appendix D is useful, but the claim that '10% of the positions played by MAV in evaluation appear in its training data' would be easier to interpret if the paper reported how the overlap rate varies across game phases, which the appendix partially does; please make that connection explicit in the main text.","section":"§5.2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is technically solid in its breadth and transparency, and I found no evidence of inappropriate citation or undisclosed prior work. My recommendation of major revision is driven by two load-bearing gaps: the external Elo calibration lacks uncertainty and is explicitly caveated in Table 1, and the engine-free MCTS claim lacks an analysis of transition-error accumulation during search. Both are addressable within the scope of the paper, so I do not recommend rejection. The early-termination protocol for internal search is also a comparability issue that should be fixed or analyzed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Let me skip the preamble. The one thing to know: this is the strongest version I've seen of the \"LLM as world model + policy + value in one transformer\" idea, and the qualitative claim—search improves the base model, more simulations help, internal search can be distilled into a single call—is supported by the tournament data. That is real and it is new. The integrated MAV format, the multi-game scope, and the internal-search distillation over linearized minimax traces go beyond Ext-BoN (chess only, no search) and Stream of Search (simpler domains).\n\nWhat the paper does well: large-scale training data, sensible use of value-bucket classification, a genuinely engine-free MCTS for chess, Chess960, and Connect Four, and an honest limitations section. The example games (e.g., MAV-MCTS sacrificing for long-term attacks, the underpromotion finish) are striking and not cherry-picked BS—they are consistent with strong positional play. The one-step FEN accuracy of 99.6-100% on OOD positions is good evidence that hallucination is not rampant.\n\nSoft spots, in proportion. The headline \"3209 external Elo\" is built on an internal-to-external linear fit with no confidence intervals, and the paper itself warns the estimate is not valid at blitz time controls. That weakens the \"Grandmaster-level\" framing; the honest version is \"beats Stockfish-L19/L20 at their own time control.\" Second, the engine-free MCTS claim does not hold for Hex—they say so in the text, but the abstract implies full coverage. Third, the stress-test concern about transition errors is legitimate but not fatal: the -infinity pruning rule could in principle cut the true best line, and Table 2 measures one-step accuracy, not search-depth or parse-failure rates during MCTS. However, given the strong per-step accuracy and the fact that search improves play across all games and simulation counts, I'd be surprised if this artifact explains the +340 Elo gain. It is a fixable reporting gap—an oracle-transition ablation and parse-failure stats would settle it. Also, internal search uses a post-hoc game stopping rule (1200+ centipawn advantage); that is disclosed and reasonable, but it makes the internal-search Elo comparisons softer than they look. No code or data release is a real reproducibility cost.\n\nWho this is for: anyone working on LLM planning, world models, or transformer-based game play. It deserves a serious referee, but an editor should push for evaluation transparency before publication. My recommendation: do not desk-reject; send to review with a request for confidence intervals on the Elo mapping, an oracle-transition ablation for MCTS, and parse-failure statistics from actual searches. The qualitative core will survive that scrutiny.","headline":"A genuinely useful empirical study of LLM planning on board games, with a real (if noisy) result: search over a learned world model improves play, and the internal-search distillation works; the headline Elo numbers need more caveats than the abstract gives them.","tokens_in":93298,"tokens_out":1517,"would_cite":true,"duration_ms":22402,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Language models can reach grandmaster-level chess by planning with search, without a game engine at play time.","keywords":["language models","planning","Monte Carlo Tree Search","world model","chess","internal search","external search","board games"],"falsifier":"Take a set of chess positions whose best move wins only through a tactic of depth five or more, run MAV-MCTS with 2000 simulations on each, and record how often the parser's $-\\infty$ fallback removes the true best move from the tree; if that frequency grows with tactical depth, the learned-transition assumption fails precisely where search is supposed to add the most value.","tokens_in":92246,"feed_emoji":"♟","tokens_out":9394,"duration_ms":88208,"temperature":0.7,"pith_summary":"This paper argues that search-based planning can turn language models into strong game players, provided the model is first trained to act as a reliable world model. The authors introduce the Multi-Action-Value (MAV) model, a transformer pretrained on textual game data to track board states, predict legal moves, and estimate action values in a single call. Embedding MAV inside a Monte Carlo Tree Search controller raises its chess strength from roughly 2923 external Elo to about 3209 at 2000 simulations, and a variant fine-tuned to emit an internal search trace reaches about 2930 external Elo in a single call. The central message is that planning, not just next-token prediction, is a direct lever on LLM competence in sequential decision tasks.","feed_headline":"Search makes language models grandmaster chess players","feed_subtitle":"With 2000 simulations a single model reaches ~3200 Elo; a one-call internal search reaches ~2930.","key_machinery":"The load-bearing object is the Multi-Action-Value (MAV) model, a decoder-only transformer trained to answer a command header such as `%state %top_5 %best_action` by emitting legal moves, bucket-encoded win probabilities, and the engine's preferred move in one token stream. The external-search mechanism is an AlphaZero-style MCTS whose transition, legal-move, value, and terminal checks are all taken from MAV's output rather than from a game engine, with a parser that discards malformed responses by assigning them $-\\infty$. The internal-search mechanism is a fine-tuned MAV (MAV-IS) trained on linearized depth-first minimax trees, so that a single model call alternates between evaluating a node, expanding it, and selecting a move, effectively executing a search algorithm in language. Two auxiliary mechanisms carry the performance: mean scoring over the 64 value buckets, which distinguishes near-tied moves better than taking the mode, and an asynchronous MCTS with dynamic virtual counts, which lets batches of LLM evaluations proceed in parallel without collapsing exploration.","core_discovery":"Stated on the paper's own terms, the discovery is that one language model can simultaneously serve as a world model, a value function, and a policy for several perfect-information games, and that this model supports two modes of planning. In external search, MAV replaces the game engine inside an AlphaZero-style MCTS: it predicts the next state from a state-action pair, returns legal moves with their values, and detects terminal states, with any unparsable output assigned $-\\infty$ to keep the search tree clean. In internal search, MAV is fine-tuned on linearized minimax trees and learns to produce a search trace and a final choice within one model call, occasionally correcting its own first impulse (in the paper's example, finding a winning move that the initial value ranking missed). The result is grandmaster-level chess with a search budget closer to human thinking than to engine search: MAV-MCTS with 2000 simulations reaches internal Elo 1707, mapped to roughly 3209 external Elo, and MAV-IS at breadth 4 and depth 2 reaches roughly 2930 external Elo. The same framework, with game-specific caveats, lifts performance across Chess960, Connect Four, and Hex.","pith_inferences":["My inference: if the learned-transition assumption holds outside board games, the same \"one model as world model plus parser-enforced search\" recipe should transfer to any domain whose state transitions can be written as text and checked for well-formedness, such as code execution or formal proof search; the paper does not test this.","My inference: the external/internal distinction is essentially a choice of where to spend the compute budget, many short model calls versus one long call, and the paper leaves open whether the two budgets compose additively at a fixed total token cost.","My inference: the parser's $-\\infty$ fallback is where hallucination risk is concentrated; directly measuring how often the true best move is dropped as search depth grows would quantify the ceiling of engine-free search, which the paper does not report.","My inference: internal-search training data could plausibly be regenerated from the external search's own visit statistics, letting the model learn from its own search rather than only from engine annotations; this bootstrapping loop is an extension the paper leaves implicit. "],"forward_implications":["Search-based planning adds roughly 300 Elo to the searchless MAV model in chess, with strength growing logarithmically in the number of MCTS simulations.","MAV-IS shows that a model can be trained to emit a structured search trace and a final decision in a single call, with playing strength scaling with the search budget measured in tokens.","Because `%best_action` training teaches the model to convert decisive advantages, MAV and its search variants can play complete games without any external game engine at inference time.","The same architecture and training format transfer across four different games, indicating the recipe is not tied to chess-specific features.","Depth and breadth of internal search provide a test-time compute knob: more tokens per call buy more playing strength, analogous to adding simulations in external search. "],"supporting_citations":[{"why":"Supplies the AlphaZero-style MCTS that external search adapts, replacing the engine's world model with MAV.","marker":"[Silver et al., 2017]"},{"why":"Provides the Ext-BoN baseline and the win-probability bucket mapping and Stockfish-calibrated Elo estimation used for chess.","marker":"[Ruoss et al., 2024a]"},{"why":"Motivates removing the explicit game engine by learning a model of environment dynamics inside the search.","marker":"[Schrittwieser et al., 2020]"},{"why":"Stream of Search is the approach that internal search generalizes to the more complex board-game domain.","marker":"[Gandhi et al., 2024b]"},{"why":"Justifies treating value prediction as classification over discrete buckets, which MAV uses for action values.","marker":"[Farebrother et al., 2024]"},{"why":"Frames the linearized search traces as an algorithmic execution task in language, guiding the MAV-IS training format.","marker":"[Markeeva et al., 2024]"},{"why":"OpenSpiel provides the game engine used for evaluation and supplies legal actions for the Hex experiments.","marker":"[Lanctot et al., 2019]"},{"why":"The TCEC opening position set anchors the chess match-ups and the Elo comparisons behind the reported strengths.","marker":"(tce)"}],"fun_headline_variants":["Search planning lifts LLM to grandmaster chess","LLM hits 3200 Elo chess via 2000 search simulations","One LLM masters chess and more with internal search","External and internal search boost LLM to chess grandmaster","In-context search tree lets LLM plan chess in one call"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"External search without a game engine relies on MAV's learned transition function being accurate on the states that search visits, because a single hallucinated transition silently drops that node from consideration with value $-\\infty$ and the paper does not measure how such errors accumulate with depth.","fun_headline_variants_meta":{"raw":{"variants":["Search planning lifts LLM to grandmaster chess","LLM hits 3200 Elo chess via 2000 search simulations","One LLM masters chess and more with internal search","External and internal search boost LLM to chess grandmaster","In-context search tree lets LLM plan chess in one call"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000534,"raw_usage":{"total_tokens":2596,"prompt_tokens":1004,"completion_tokens":1592,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":620,"completion_tokens_details":{"reasoning_tokens":1508}},"tokens_in":620,"tokens_out":1592,"duration_ms":16780,"temperature":1.0,"reasoning_tokens":1508,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T00:53:47.578094+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of chess positions whose best move wins only through a tactic of depth five or more, run MAV-MCTS with 2000 simulations on each, and record how often the parser's $-\\infty$ fallback removes the true best move from the tree; if that frequency grows with tactical depth, the learned-transition assumption fails precisely where search is supposed to add the most value.","supporting_citations":[],"review_version":1}