Understanding the learned look-ahead behavior of chess neural networks

Diogo Cruz

REVIEW 3 major objections 5 minor 15 references

Understanding the learned look-ahead behavior of chess neural networks

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that the Leela Chess Zero policy network, a transformer that reads a chessboard as a sequence of 64 squares, represents board states up to seven moves ahead when picking its next move.

desk verdict Credible multi-method extension of Jenner et al. to 5th/7th moves and alternative branches, but the 7th-move claim hinges on an untested identification of Lichess PVs with the model's internal line. read the letter →

arxiv 2505.21552 v1 pith:WTU6GIEN submitted 2025-05-26 cs.AI cs.LG

Diogo Cruz This is my paper

classification cs.AIcs.LG

keywords mechanisticinterpretabilitychessneuralnetworkslook-aheadLeelaZeroattentionheadsactivationpatchinglinearprobingplanning

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that the Leela Chess Zero policy network, a transformer that reads a chessboard like a sequence of 64 tokens, represents information about board states up to seven moves ahead when it chooses its next move. Using activation patching, linear probing, and attention-head ablation on chess puzzles, the author shows that destination squares of the third, fifth, and seventh future moves carry causally relevant information, with salience weakening with distance. The look-ahead is context-dependent: different square-repetition patterns, such as AAC versus ACC, and checkmate versus non-checkmate positions recruit different attention heads. The paper also argues the model weighs multiple alternative move sequences at once, since corrupting one branch's future square shifts probability to the other branch.

What carries the argument

The load-bearing objects are the puzzle-set notation and the trio of interpretability interventions. Puzzle-set notation labels each puzzle by the sequence of destination squares of its principal variation, so 112 means the first two moves go to the same square and the third to a new one, while letters such as A, B, and C abstract distinct squares in patterns like AAC, ABC, and ACC. Activation patching measures the causal importance of a board square by replacing its activations with those from a minimally corrupted position and recording the log-odds reduction of the correct move; linear probing decodes whether a future move's destination square is linearly readable from the residual stream; and zero ablation removes attention-head weights to identify which heads move which information. Together they show that future-move destination squares are causally relevant, that their information is encoded, and that specific heads, notably L12H12, implement the backward-in-time copying of future-square information to the first-move square, with the square-repetition pattern determining which head engages.

What would settle it

Build a test set of positions where the same AAC destination-square pattern is preserved but the future line no longer leads to the advertised outcome, such as a seventh move square that becomes losing, then patch that square: if the log-odds reduction persists unchanged, the effect is pattern-triggered rather than line-tracking; if it collapses, the model encodes the actual future sequence.

Watch

Extended reading notes

Core claim

The central discovery is that a single forward pass of the Leela policy network encodes and uses the destination squares of moves up to seven plies ahead, not just the immediate next move. The evidence is that patching the residual-stream activation at a future move square, say the fifth or seventh, reduces the log odds of the correct next move, and linear probes decode the future square's identity from intermediate layers, with accuracy declining toward the seventh move. The strongest single mechanism is attention head L12H12, which copies information backward in time from a future square to an earlier one, third to first, fifth to first, or seventh to first, depending on the pattern of square repetitions; this pattern sensitivity is time-insensitive, so the same AAC, ABC, and ACC templates apply at different move offsets. The model also tracks alternative branches: when two first moves are nearly tied, patching the alternative branch's third-move square increases the probability of the main first move, indicating simultaneous evaluation of multiple lines. The author interprets this as learned look-ahead behavior with specialized, context-dependent circuitry, while conceding that the experiments cannot fully rule out pattern matching over memorized tactical motifs.

Load-bearing premise

The central assumption is that the fine-tuned, history-free version of Leela, judged against the Lichess principal variation, is still doing the same look-ahead computation the original engine does, so that the patched attention to fifth- and seventh-move squares reflects forward planning rather than memorized tactical motifs.

Editorial extensions

If this is right

The policy network itself performs a form of multi-step planning: information about the fifth and seventh future moves is present and causally relevant in the same forward pass, so search-based engines are not the only way to achieve look-ahead.
Look-ahead is branch-aware: the model assigns probability across at least two alternative sequences and shifts its choice when one branch's future square is corrupted, so a single-line description of its planning is inadequate.
The internal machinery generalizes across time: the same attention head that copies third-move information to the first-move square also copies fifth- and seventh-move information under the same square-pattern rules, implying a time-insensitive planning mechanism.
Any mechanistic account of chess transformers must be position-type-specific, because checkmate positions recruit L12H12 while non-checkmate positions rely more on L12H17, and different square patterns produce different effect sizes.
The combined patching, probing, and ablation protocol, with puzzle-set notation, is a transferable recipe for detecting look-ahead in other transformer policies trained on sequential decision tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the seven-move horizon reflects this model's depth and training distribution rather than a universal ceiling, one direct extension would be to measure whether deeper or longer-trained chess transformers push the horizon further; the paper does not test this.
The AAC, ABC, and ACC sensitivity raises the possibility that the network has learned tactical geometry templates, and a decisive test would be to see whether patching a future square still matters when the pattern is preserved but the line stops being winning.
The backward-in-time copying performed by L12H12 resembles a broadcasting operation that other planning networks might share, so looking for analogous future-square heads in transformer policies for Sokoban or maze-solving is a natural next test.
The alternative-branch result suggests the model's probability mass itself encodes a small search tree, so one could try to decode the full tree from the residual stream rather than only pairwise branch comparisons.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a credible extension of Jenner et al. (2024) to longer horizons and multiple branches, and it is honest about its own limits. The headline result — look-ahead to the 7th move — is real but softer than the abstract suggests, and the load-bearing assumption that Lichess principal variations are the model's internal future line is never tested.

What's actually new: the 5th- and 7th-move probing, patching, and ablation results; the puzzle-set notation for separating move-pattern contexts; and the two-branch corruption analysis showing that corrupting one branch's future square shifts probability to the other branch. The paper uses three complementary methods, reports confidence intervals, and compares probes against a random-model baseline. That is more than most interpretability papers do. The L12H12 result — that the same head copies future-square information backward in time for AAC/ABC/ACC patterns, largely round-insensitive — is a concrete, falsifiable claim, and the ablation evidence supports it. The author also deserves credit for stating in Section 5 that the work cannot definitively distinguish planning from pattern matching.

Soft spots, in rough order of seriousness. First, the stress-test concern is accurate: the future squares are defined by Lichess PVs, but the model sometimes does not choose the PV first move (Appendix F, Fig. 29). If the model's internal line differs, salience of those squares could reflect tactical geometry rather than forward simulation. The paper explicitly concedes this, but the concession sits in the conclusion rather than driving the design; a model-derived future-line control would address it. Second, the 7th-move signal is weak, as the paper says: probe accuracy barely exceeds the random baseline, and patching effects are small. The claim should be phrased with that weight. Third, no code or checkpoint identifiers are provided (the link is omitted for review), and the history-free finetuned model's "equivalent performance" is unquantified, so the patch analysis cannot yet be tied to the shipped engine. That is a reproducibility gap, not a fatal flaw. Fourth, Hypothesis 1 is post hoc: the AAC/ABC/ACC ordering is derived after looking at the data and then tested on related sets. The paper is transparent about this, and the consistency across sets helps, but it is not a pre-registered prediction.

Who this is for: mechanistic interpretability researchers working on game-playing transformers and planning, especially people building on Jenner et al. It deserves a serious referee. My recommendation: send it to peer review with a request for code, data, and checkpoint release, and for the PV-control analysis. The central qualitative claim — that the model uses future-square information beyond the next move — holds up reasonably well; the 7th-move strength and the planning-versus-pattern interpretation need tightening, not redoing.

Referee Report

3 major / 5 minor

Summary. The paper studies the Leela Chess Zero policy network and asks whether its internal representations encode and causally use information about board states several moves ahead. Extending the framework of Jenner et al. (2024), the author introduces a puzzle-set notation based on destination-square patterns, curates 3-, 5-, and 7-move Lichess puzzle datasets, and applies activation patching, linear probing, and attention-head ablation to measure the importance of future-move squares. The central claims are that the model processes information about the 5th and 7th future moves (with decreasing strength), that this look-ahead behavior is highly context-dependent across puzzle sets, that specific attention heads such as L12H12 move information backward in time from future to current move squares in a pattern-sensitive, time-insensitive way, and that the model evaluates multiple alternative move branches simultaneously.

Significance. If the central claims hold, the paper would be a valuable mechanistic-interpretability contribution: it extends the known 3-move look-ahead result to longer horizons, identifies candidate attention-head circuits, provides evidence for multiple-branch consideration, and combines three complementary analysis techniques with confidence intervals and random-model baselines. The paper is also reasonably candid about its limitations, explicitly conceding in Section 5 that it cannot distinguish true planning from sophisticated pattern matching. However, the significance is currently conditional because the headline claim depends on identifying the Lichess principal variation with the model's internal future line, an identification that is asserted rather than tested.

major comments (3)

[Section 3 (Figs. 2, 3, 13, 14)] The claim that the model 'can process information about board states up to seven moves ahead' equates the Lichess-annotated principal-variation destination squares with the model's own internally planned future line. This identification is load-bearing for the headline result, but the paper provides no control using model-derived future lines: Appendix F (Fig. 29) shows cases where Leela does not choose the PV first move, and Section 5 concedes that the observed behavior could be sophisticated pattern matching rather than forward simulation. Because the 5th and 7th PV destination squares are often the squares on which the tactical motif lands (mating square, capture square, escape square), the observed probing and patching salience may be confounded with tactically salient geometry. The authors should add a control in which the target future squares are produced by the model's own policy (e.g., high-probability rollouts or MCTS continuations) rather than by the Lichess annotation, and show that the same salience pattern appears for those squares; alternatively, they should weaken the claim to 'squares of the annotated PV are represented' and discuss the confound explicitly.
[Section 2.1 and Appendix H] All activation-patching, probing, and ablation experiments use a fine-tuned, history-free variant of Leela that is asserted to maintain 'equivalent performance' to the original model, but no quantitative comparison is provided: there is no reported policy accuracy, puzzle solve rate, or game-play strength for the fine-tuned variant versus the original. The appendix also omits code and checkpoint identifiers (the link is 'omitted for review'). Without this information, the reader cannot verify that the circuit-level findings transfer to the actual Leela engine. The authors should report concrete performance metrics for both models on the puzzle datasets and release the exact fine-tuned checkpoint and analysis code.
[Section 3, Fig. 3, and Appendix H] The evidence for the 7th-move claim is weak by the paper's own reporting: the probe accuracy for the 7th move is described as 'considerably low' and only slightly above the random-model baseline in Fig. 3, and the patching effect sizes in Fig. 14 are small and vary across sets. The 7-move dataset also uses relaxed filtering criteria (Appendix H), and it is selected for puzzles where Leela already assigns at least 50% probability to the first three PV moves. Given these conditions, the marginal signal at move 7 is thin, yet the abstract and Section 3 state the finding as a central result. The authors should provide a statistical comparison of the 7th-move probing/patching signal against a proper null (e.g., a matched non-PV future-square control) and quantify effect sizes across all 7-move pattern sets, or explicitly reframe the conclusion as preliminary evidence of a near-limit signal.

minor comments (5)

[Fig. 2 caption] The word 'accross' should be 'across'.
[Fig. 25 caption] The phrase 'In the bottow row' contains a typo: it should be 'bottom row'.
[Section 3, 'The model processes 3rd, 5th, and 7th moves similarly'] The sentence 'In we particular, we note that patching shows...' is grammatically broken and should be rewritten, for example as 'In particular, we note that patching shows...'.
[Appendix H, Eq. (1)] Equation (1) defines the log odds change with a sign convention in which negative values indicate reduced preference for the correct move, while the main text refers to 'log odds reduction' and presents positive values as harmful in the figures; the sign convention should be stated consistently in the main text.
[Section 2.4 and Appendix B, Hypothesis 1] Hypothesis 1 is introduced after observing the data and is then confirmed on related puzzle sets; the paper should state more explicitly that this ordering is a post-hoc pattern with a pre-registered prediction only for future work, and should note that no correction for multiple comparisons is applied across the many pattern sets analyzed.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found: the 7-move look-ahead evidence is measured against externally defined Lichess PV squares, and the principal caveats are validity/generality rather than definitional circularity.

full rationale

The paper's central claim is that the Leela policy network internally processes information about board squares of future moves up to the 7th move. The supporting evidence (activation patching, probing, and zero-ablation of attention heads) is measured on destination squares taken from Lichess principal variations, which are external labels and not functions of the model's own outputs. The dataset in Appendix H is filtered so that the Leela model already assigns at least 50% probability to the 1st, 2nd, and 3rd PV moves, but this selection does not force the observed salience of the 5th or 7th move squares; those distant squares are not the corrupted squares in the patching setup, and their probe accuracy is compared against a random-model baseline. The 'alternative move' analysis similarly constructs puzzles where the model is roughly 50/50 between two branches, yet the mechanistic finding (patching the alternative branch's 3rd-move square shifts the model's odds) is not entailed by that selection. The finetuned history-free variant and the identification of the Lichess PV with the model's internal future line are unvalidated assumptions, and Section 5 explicitly concedes that true planning cannot be distinguished from sophisticated pattern matching; these are correctness and generalization risks, not cases where a conclusion is equivalent to its input by construction. The paper builds on Jenner et al. (2024), but that is prior work by different authors, and no uniqueness theorem, ansatz, or fitted parameter is imported to force the result. No circular step meeting the quote-and-reduce bar was found.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The central evidence rests on five unproved modeling assumptions: localized intervention semantics, Lichess principal variations as ground truth, history-free variant equivalence, ignoring starting squares, and linear-probe/random-baseline validity. These are standard for the mechanistic interpretability subfield but are not independently verified here. No fitted constants enter the core claim; the hand-chosen dataset thresholds shape which puzzle patterns are visible and how strong the reported effects appear.

free parameters (3)

Puzzle-set minimum sample threshold = 50 puzzles
Only puzzle sets with at least 50 examples are analyzed in the patching and ablation summaries. This hand-chosen cutoff excludes rare patterns and defines which orderings can be checked.
Alternative-branch selection thresholds = p >= 0.3 for each first move; p >= 0.7 for second and third moves; principal variation must be one branch…
These hand-picked thresholds reduce 4,062,423 Lichess puzzles to roughly 600, so all multi-branch evidence comes from a narrow, model-selected subset.
7-move dataset relaxed criteria = weaker-model first-move probability up to 0.20; no weaker-model second-move constraint; Leela probability >= 0.5 on…
Relaxation was needed to obtain a 2.2k dataset; it changes the difficulty distribution and may limit the generality of the seventh-move conclusion.

assumptions (6)

domain assumption Activation patching traces causal contribution by replacing a single component's clean activation with a corrupted one, assuming effects are sufficiently localized and additive.
Section 2.3 acknowledges that patching "may miss indirect effects distributed across multiple components"; all square-saliency plots depend on this assumption.
domain assumption The Lichess puzzle principal variation is the true optimal future sequence and the network's look-ahead targets are defined by these annotated moves.
Introduced in Section 2.2; every future-move label used in probing and patching comes directly from these annotations.
domain assumption The fine-tuned history-free Leela variant preserves the look-ahead behavior of the original model.
Section 2.1 relies on Jenner et al. (2024) for this claim; no independent evaluation of look-ahead equivalence is provided here.
domain assumption Starting squares can be ignored when categorizing moves, based on the null direct-patching results in Appendix A.
Appendix A tests only direct residual effects; interaction effects between starting and destination squares are not tested, so the simplification could hide context dependence.
domain assumption Linear probes measure whether information is explicitly encoded in the residual stream.
Section 2.3 interprets probe accuracy as evidence of encoding; probe success also depends on the probe's capacity and the linear separability of the feature.
domain assumption The random chess model provides an appropriate baseline for probe accuracy.
Appendix C compares trained probes against a random-model baseline; if the random model is not matched to the same position distribution, the trained-minus-random gap could be biased.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Understanding the learned look-ahead behavior of chess neural networks." pith.science (2026). https://pith.science/paper/WTU6GIEN

@misc{pith2026250521552,
  author       = {Pith},
  title        = {Pith review of: Understanding the learned look-ahead behavior of chess neural networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WTU6GIEN}},
  note         = {Machine review of arXiv:2505.21552}
}

read the original abstract

We investigate the look-ahead capabilities of chess-playing neural networks, specifically focusing on the Leela Chess Zero policy network. We build on the work of Jenner et al. (2024) by analyzing the model's ability to consider future moves and alternative sequences beyond the immediate next move. Our findings reveal that the network's look-ahead behavior is highly context-dependent, varying significantly based on the specific chess position. We demonstrate that the model can process information about board states up to seven moves ahead, utilizing similar internal mechanisms across different future time steps. Additionally, we provide evidence that the network considers multiple possible move sequences rather than focusing on a single line of play. These results offer new insights into the emergence of sophisticated look-ahead capabilities in neural networks trained on strategic tasks, contributing to our understanding of AI reasoning in complex domains. Our work also showcases the effectiveness of interpretability techniques in uncovering cognitive-like processes in artificial intelligence systems.

Figures

Figures reproduced from arXiv: 2505.21552 by the authors.

**Figure 2.** Log odds reduction of the correct move as a result of activation patching, for 5-move puzzle [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Probing the model’s residual stream for the puzzle set 1123456. The probe’s accuracy [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (31 more)

**Figure 4.** Figure 4: Attention head patching results for puzzles with 3 moves. Darker tones indicate higher [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

**Figure 5.** Figure 5: Ablation results of the L12H12 head for checkmate (M112, left) and non-checkmate (N112, [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

**Figure 6.** Figure 6: Patching results of the alternative move analysis, for puzzle set 123 [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]

**Figure 7.** Figure 7: Residual effects of the starting squares for the puzzle set 112. The baseline analysis is [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]

**Figure 8.** Figure 8: Log odds reduction of the correct move as a result of residual stream patching, for puzzles [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]

**Figure 9.** Figure 9: Log odds reduction as a result of residual stream patching, for puzzle sets of the form [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]

**Figure 10.** Figure 10: Residual effects the puzzle set 122 and 12223. [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]

**Figure 11.** Figure 11: Residual effects of puzzle sets of the form 123XY. The 3-move puzzle set 123 is shown on [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]

**Figure 12.** Figure 12: Residual effects for the remaining puzzle sets of the form 112XY, not shown in Fig. 2. [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]

**Figure 13.** Figure 13: Residual effects for the puzzle sets with 7 moves. The effect of patching the seventh move [PITH_FULL_IMAGE:figures/full_fig_p017_13.png]

**Figure 14.** Figure 14: Log odds reduction of the correct move as a result of residual stream patching, for puzzles [PITH_FULL_IMAGE:figures/full_fig_p017_14.png]

**Figure 15.** Figure 15: Probing the model for the puzzle set 12345. While activation patching does not seem to [PITH_FULL_IMAGE:figures/full_fig_p019_15.png]

**Figure 16.** Figure 16: Ablation results of the L12H12 head for the 112 and 123 move analysis. [PITH_FULL_IMAGE:figures/full_fig_p020_16.png]

**Figure 17.** Figure 17: Ablation results of the L12H12 head for sets with 5 moves. This head’s role varies [PITH_FULL_IMAGE:figures/full_fig_p021_17.png]

**Figure 18.** Figure 18: Ablation results for some puzzle sets with 7 moves, for head L12H12. The results are [PITH_FULL_IMAGE:figures/full_fig_p022_18.png]

**Figure 19.** Figure 19: Ablation results for head L12H17, for the puzzle sets with 5 moves that seem to respond [PITH_FULL_IMAGE:figures/full_fig_p022_19.png]

**Figure 20.** Figure 20: Ablation results for head L13H3, for the puzzle sets with 5 moves that seem to respond [PITH_FULL_IMAGE:figures/full_fig_p023_20.png]

**Figure 21.** Figure 21: Ablation results for heads L11H10 and L11H13, for the puzzle sets with 5 moves that [PITH_FULL_IMAGE:figures/full_fig_p023_21.png]

**Figure 22.** Figure 22: Ablation results for L12H12, for the puzzle sets with 5 moves that seem to respond [PITH_FULL_IMAGE:figures/full_fig_p024_22.png]

**Figure 23.** Figure 23: Ablation results for L12H17, for the puzzle sets with 5 moves that seem to respond [PITH_FULL_IMAGE:figures/full_fig_p025_23.png]

**Figure 24.** Figure 24: Attention head patching for some puzzle sets with 5 moves. In the top row, the model [PITH_FULL_IMAGE:figures/full_fig_p027_24.png]

**Figure 25.** Figure 25: Attention head patching for the remaining puzzle sets with 5 moves. In the top row, some [PITH_FULL_IMAGE:figures/full_fig_p028_25.png]

**Figure 26.** Figure 26: Attention head patching for some puzzle sets with 7 moves. [PITH_FULL_IMAGE:figures/full_fig_p029_26.png]

**Figure 27.** Figure 27: Attention head patching for puzzle sets with 3 moves, for both checkmate and non [PITH_FULL_IMAGE:figures/full_fig_p030_27.png]

**Figure 28.** Figure 28: Attention head patching for some puzzle sets with 5 moves, for both checkmate and [PITH_FULL_IMAGE:figures/full_fig_p031_28.png]

**Figure 29.** Figure 29: Instead of considering an arbitrary puzzle (top), where the number of branching moves [PITH_FULL_IMAGE:figures/full_fig_p032_29.png]

**Figure 30.** Figure 30: Patching results of the alternative move analysis. The log odds reduction for the next [PITH_FULL_IMAGE:figures/full_fig_p034_30.png]

**Figure 31.** Figure 31: Ablation results for the alternative move analysis. The log odds reductions are shown for [PITH_FULL_IMAGE:figures/full_fig_p035_31.png]

**Figure 32.** Figure 32: Simple puzzle where Leela fails to pick any of the checkmates in 2 moves. Nonetheless, [PITH_FULL_IMAGE:figures/full_fig_p036_32.png]

**Figure 33.** Figure 33: Modification of the puzzle in Fig. 32 where the rook moves are discouraged, leading Leela [PITH_FULL_IMAGE:figures/full_fig_p037_33.png]

**Figure 34.** Figure 34: Puzzle of the set 112 where there are 2 third move options for a checkmate in 2. Both [PITH_FULL_IMAGE:figures/full_fig_p038_34.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

15 extracted references · 5 canonical work pages

[2]

Nonetheless, the contribution of L12H12 is relatively limited

Both options reinforce the choice of the first move square. Nonetheless, the contribution of L12H12 is relatively limited. 38 H Implementation details Our implementation is heavily based on the implementation described in Jenner et al. [2024], and previously made available at https://github.com/HumanCompatibleAI/leela-interp. For the activation patching, ...

work page 2024
[3]

• The puzzle’s PV must not be a checkmate

This is in order to simplify the analysis, and remove higher future move squares from consideration. • The puzzle’s PV must not be a checkmate. In practice, we observe that not only does the model treat different puzzle sets differently, but it also seems to have a different behavior when a checkmate in 2 is a likely option, even if not the most likely. W...

work page 2024
[4]

Reasoning with language model is planning with world model

Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhiting Hu. Reasoning with language model is planning with world model. arXiv preprint arXiv:2305.14992,

arXiv
[5]

Designing and interpreting probes with control tasks

John Hewitt and Percy Liang. Designing and interpreting probes with control tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP),

work page 2019
[7]

Emergent linear representations in world models of self-supervised sequence models

Neel Nanda, Andrew Lee, and Martin Wattenberg. Emergent linear representations in world models of self-supervised sequence models. arXiv preprint arXiv:2309.00941,

arXiv
[8]

Future lens: Anticipating subsequent tokens from a single hidden state

Koyena Pal, Jiuding Sun, Andrew Yuan, Byron C Wallace, and David Bau. Future lens: Anticipating subsequent tokens from a single hidden state. arXiv preprint arXiv:2311.04897,

arXiv
[9]

David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, et al

URL https://arxiv.org/abs/2402.04494. David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, et al. A general reinforcement learning algorithm that masters chess, shogi, and go through self-play. Science, 362(6419): 1140–1144,

arXiv
[11]

Alpine: Un- veiling the planning capability of autoregressive learning in language models

Siwei Wang, Yifei Shen, Shi Feng, Haoran Sun, Shang-Hua Teng, and Wei Chen. Alpine: Un- veiling the planning capability of autoregressive learning in language models. arXiv preprint arXiv:2405.09220,

arXiv

Show all 15 references

[12]

Do language models plan ahead for future tokens? arXiv preprint arXiv:2404.00859,

Wilson Wu, John X Morris, and Lionel Levine. Do language models plan ahead for future tokens? arXiv preprint arXiv:2404.00859,

arXiv
[112]

Corrupted

This consistency across different starting square configurations suggests that the starting squares do not play a critical direct role in the model’s decision-making process for these puzzles. Based on these findings, we concluded that it was unnecessary to disentangle the eff...

2024
[2020]

Label words are anchors: An information flow perspective for understanding in-context learning

Lean Wang, Lei Li, Damai Dai, Deli Chen, Hao Zhou, Fandong Meng, Jie Zhou, and Xu Sun. Label words are anchors: An information flow perspective for understanding in-context learning. arXiv preprint arXiv:2305.14160,

arXiv
[2021]

Dissecting recall of factual associations in auto-regressive language models

Mor Geva, Jasmijn Bastings, Katja Filippova, and Amir Globerson. Dissecting recall of factual associations in auto-regressive language models. arXiv preprint arXiv:2304.14767,

arXiv
[2022]

URL https://www.pnas.org/doi/abs/10.1073/ pnas.2206625119

doi: 10.1073/pnas.2206625119. URL https://www.pnas.org/doi/abs/10.1073/ pnas.2206625119. Thomas McGrath, Matthew Rahtz, Janos Kramar, Vladimir Mikulik, and Shane Legg. The hydra effect: Emergent self-repair in language model computations. arXiv preprint arXiv:2307.15771,

arXiv
[2023]

Finding neurons in a haystack: Case studies with sparse probing

Wes Gurnee, Neel Nanda, Matthew Pauly, Katherine Harvey, Dmitrii Troitskii, and Dimitris Bertsimas. Finding neurons in a haystack: Case studies with sparse probing. arXiv preprint arXiv:2305.01610,

arXiv
[2024]

Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei

URL https://arxiv.org/abs/2304.14918. Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. Knowledge neurons in pretrained transformers. In Annual Meeting of the Association for Computational Linguistics (ACL),

arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [2]

Nonetheless, the contribution of L12H12 is relatively limited

Both options reinforce the choice of the first move square. Nonetheless, the contribution of L12H12 is relatively limited. 38 H Implementation details Our implementation is heavily based on the implementation described in Jenner et al. [2024], and previously made available at https://github.com/HumanCompatibleAI/leela-interp. For the activation patching, ...

work page 2024

[2] [3]

• The puzzle’s PV must not be a checkmate

This is in order to simplify the analysis, and remove higher future move squares from consideration. • The puzzle’s PV must not be a checkmate. In practice, we observe that not only does the model treat different puzzle sets differently, but it also seems to have a different behavior when a checkmate in 2 is a likely option, even if not the most likely. W...

work page 2024

[3] [4]

Reasoning with language model is planning with world model

Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhiting Hu. Reasoning with language model is planning with world model. arXiv preprint arXiv:2305.14992,

arXiv

[4] [5]

Designing and interpreting probes with control tasks

John Hewitt and Percy Liang. Designing and interpreting probes with control tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP),

work page 2019

[5] [7]

Emergent linear representations in world models of self-supervised sequence models

Neel Nanda, Andrew Lee, and Martin Wattenberg. Emergent linear representations in world models of self-supervised sequence models. arXiv preprint arXiv:2309.00941,

arXiv

[6] [8]

Future lens: Anticipating subsequent tokens from a single hidden state

Koyena Pal, Jiuding Sun, Andrew Yuan, Byron C Wallace, and David Bau. Future lens: Anticipating subsequent tokens from a single hidden state. arXiv preprint arXiv:2311.04897,

arXiv

[7] [9]

David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, et al

URL https://arxiv.org/abs/2402.04494. David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, et al. A general reinforcement learning algorithm that masters chess, shogi, and go through self-play. Science, 362(6419): 1140–1144,

arXiv

[8] [11]

Alpine: Un- veiling the planning capability of autoregressive learning in language models

Siwei Wang, Yifei Shen, Shi Feng, Haoran Sun, Shang-Hua Teng, and Wei Chen. Alpine: Un- veiling the planning capability of autoregressive learning in language models. arXiv preprint arXiv:2405.09220,

arXiv