Pith. sign in

REVIEW 4 major objections 5 minor 13 references

Path Channels and Plan Extension Kernels: a Mechanistic Description of Planning in a Sokoban RNN

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A model-free RL-trained Sokoban agent stores its plan directly in individual hidden-state channels, and constructs it with convolutional kernels that extend paths forward from boxes and backward from goals, pruning dead ends by…

desk verdict Real progress on direct plan representation in a non-toy agent, but the plan-extension kernel evidence needs a held-out offset control before it fully convinces. read the letter →

arxiv 2506.10138 v3 pith:V5D3YYHL submitted 2025-06-11 cs.LG cs.AI

Mohammad Taufeeque , Aaron David Tucker , Adam Gleave , Adrià Garriga-Alonso This is my paper · ORCID
classification cs.LGcs.AI
keywords Sokobanmechanisticinterpretabilitypathchannelsplanextensionkernelsbidirectionalsearchwinner-takes-allConvLSTMmodel-freereinforcementlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DRC(3,3), a convolutional recurrent network trained only with model-free reinforcement learning, represents its entire Sokoban plan directly in individual hidden-state channels: each path channel corresponds to a push or move direction, and its activation at a grid square says that the box or agent will move that way from that square at a future time. The paper shows how this plan is constructed by convolutional machinery: encoder kernels seed activations near boxes, the agent, and targets; plan-extension kernels propagate those activations one square forward from boxes and backward from goals; negative activations at obstacles are propagated backward to prune failed path segments, implementing backtracking; and a winner-takes-all kernel inhibits competing directions to select a single path. The causal role of the path channels is supported by ablations (zeroing the 59 path channels drops solve rate by 57.6%, versus 10.5% for non-path channels), by intervention scores above 98% for the next-action channels, and by replication across four additional training seeds. If the claim holds, a policy network is shown to have self-organized a readable bidirectional planner from reward alone, and the agent's plans can be read — and edited — by inspecting channel activations rather than fitting probes.

What carries the argument

The load-bearing objects are the path channels — hidden-state channels, one per cardinal movement direction (for box and agent separately), whose activation at a grid location encodes a future move from that square — and the plan-extension kernels, the averaged 3×3 recurrent convolution weights from previous hidden states into the i, j, f, o gates of the ConvLSTM. Linear plan-extension kernels propagate an activation one square along the channel's direction (forward chaining from boxes, backward chaining from targets), turn kernels move it into another direction channel, the encoder's combined 9×9 kernels seed the initial activations near agent, boxes, and targets, and the winner-takes-all kernel subtracts each direction channel from the others to select a single path. Negative activations placed on obstacles are what let the extension kernels perform backtracking by propagating the negative value in reverse along a path segment.

What would settle it

Re-run the average plan-extension-kernel analysis after randomly permuting the channel labels: if a random permutation produces kernels with the same forward/backward one-square structure as the claimed ones, the structure is an artifact of the averaging procedure rather than of the labels. On a held-out seed, take a channel the automatic method labels box-right, force its activation positive on a wall square, and measure whether the agent's next actions become more likely to push boxes right; a success rate near chance would contradict the claim that these channels are causal plan components.

Watch

Extended reading notes

Core claim

The central discovery is that the DRC(3,3) network's hidden state contains direction-indexed path channels whose activation pattern is the plan: a high activation of, say, the box-right channel at a square means the box will be pushed right from that square when it gets there, with separate long-term and short-term channels letting the same square be reserved for moves at different future times. The recurrent weights between these channels form linear plan-extension kernels that translate an activation one square along the channel's labeled direction, and turn kernels that translate it into a different direction channel, while the encoder's combined kernels initialize activations adjacent to boxes, the agent, and targets. Because these kernels can pass activation in both directions, negative values written onto obstacle squares at the far end of a path segment are propagated back along the segment, pruning it and letting an alternative route emerge — a mechanism demonstrated causally on a custom level by forcing the obstacle activation positive, which blocks backtracking and makes the agent push into a wall. The same negative-activation logic, together with the winner-takes-all kernel that subtracts competing direction channels, also stops plan extension at targets and obstacles. The authors verify the interpretation with single-step ablations, causal interventions that force the agent to take a target action, weight-steering that extends planning to 40×40 boards, and automatic relabeling on four further seeds.

Load-bearing premise

The whole analysis stands on the manual per-channel labels and fitted spatial offsets (which channel means which direction, shifted by which square); if those labels or offsets are wrong for even a cluster of channels, the averaged kernels would look path-like purely from the labeling choice, and the causal interventions would be steering random channels rather than the plan.

Editorial extensions

If this is right

  • The agent's plan can be extracted by simply reading off path channel activations, with no probes needed, and the same reading works on four independently trained networks.
  • Ablating the 59 path channels costs 57.6% of solve rate versus 10.5% for non-path channels, so long-term planning is carried by exactly the channels the paper labels.
  • Editing the PNA/GNA channels lets an external controller set the agent's next action with 98–99% success, opening channel-level control over behavior.
  • Scaling the plan-extension kernel weights by a factor of 1.4 lets the same network plan on levels four times larger (40×40) than the 10×10 training grids, because plan activations no longer decay mid-extension.
  • The authors argue the path-channel activations act as a learned Q-function that does not encode the training reward's per-step cost; plan-length preference instead emerges from the extension-kernel dynamics.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editor's inference: If the plan is truly resident in single channels, then a fully testable application is writing an arbitrary user-chosen plan into the path channels (at the correct offsets) and verifying the agent executes it exactly; the paper's Section J is an initial demonstration using initial hidden state, not a general injection procedure.
  • Editor's inference: The one-square-per-tick propagation of the extension kernels predicts that the minimum resolution time of a plan scales linearly with corridor length; measuring the number of ticks between plan initiation and a stable WTA selection on straight corridors of increasing length would confirm the mechanism's dynamics.
  • Editor's inference: The finding that probe-based analyses were predictive-but-not-causal suggests re-examining other 'emergent planning' claims (including in transformer agents) with direct channel-level causal identification, which could simplify or overturn prior probe-based mechanism attributions.
  • Editor's inference: The negative-activation backtracking mechanism implies the network maintains a signed spatial 'cost field' rather than a tree of alternatives; a natural stress test is whether the network can recover from a locally impossible push that requires pushing the box back, a situation where a sign-propagating field would need revisiting several squares at once.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper analyzes a pre-trained DRC(3,3) ConvLSTM agent for the game Sokoban and claims to identify 'path channels' in the hidden state that directly encode future box and agent movements. It then proposes that the planning algorithm is implemented by convolutional kernels that extend these path activations forward from boxes and the agent and backward from goals, with negative activations at obstacles propagating backward to prune failed path segments (backtracking), and a winner-takes-all mechanism which selects among conflicting path segments. The evidence includes single-step cache ablations, causal interventions, AUC-based predictive analyses, weight-steering experiments, and replication across four additional training seeds.

Significance. If the central claims hold, this is a substantial advance in the mechanistic interpretability of reinforcement-learned agents: it offers a weight-level description of a learned planning algorithm in a non-toy network, going beyond linear probes to read the plan directly from channel activations. The paper has notable strengths: the cache ablation experiment (57.6% solve-rate drop for path channels), the causal intervention scores (PNA 99.7%, box-movement 86.3%), and the cross-seed replication attempt all provide converging evidence that the identified channels carry causally relevant plan information. However, the central kernel-level claim ('plan extension kernels encode the change in position resulting from each possible action') rests on a spatial-offset convention that is fitted on the same data used for the kernel visualization, creating a circularity that is not broken by the cross-seed replication. The paper's contribution would be significantly strengthened by a held-out validation of the offset-fitting procedure and by quantitative sensitivity analyses with respect to label noise.

major comments (4)
  1. [Section E, Table 5, Figures 7b/25/26] The per-channel spatial offsets in Table 5 are computed by performing linear regression on future-movement features and selecting the shift with lowest loss; the same future-movement features are used to define the path-channel labels. The plan-extension kernel averages in Figures 7b, 25, and 26 are then centered using these fitted offsets. For two channels labeled with the same movement direction, the offset difference is a free parameter chosen to align each channel's activation with the same future-movement event, so a recurrent kernel between them will exhibit a peak at approximately that offset difference once centered. The 'forward/backward extension' pattern could therefore be a bookkeeping artifact of the offset convention rather than evidence that the ConvLSTM weights implement a directional path-extension operator. This is load-bearing for the paper's central claim. I recommend a held-out validation: fit the offsets on a training partition of levels, then compute the kernel-centered averages on a held-out partition using those fixed offsets; if the directional pattern persists, it cannot be attributed to overfitting the offset parameters. The cross-seed replication in Section N uses the same AUC-based search over offsets on the same data used for the kernel computation, so it does not break this circularity.
  2. [Section 2.2, Table 8, Section N] The channel labels in Table 8 are predominantly derived from manual inspection, and the automatic labeling method in Section N reports an F1 score of only 73.7% for box/agent channels. All subsequent analyses—causal interventions, kernel visualizations, the backtracking case study, and the winner-takes-all matrices—select channels and spatial positions on the basis of these labels. A mislabeled or misaligned channel would cause the proposed kernel mechanisms to appear where none exist. The paper should quantify the sensitivity of the main results to label noise, for example by repeating the kernel analysis and causal interventions using only channels with high-confidence automatic labels, or by reporting how the results change when channels are randomly assigned to groups. This is load-bearing because the central claims about the planning algorithm depend on the correctness of the channel taxonomy.
  3. [Section 3.2, Figures 25/26] The paper acknowledges in the caption text that 'in many cases we see the idealized weight pattern, but in most we do not' when individual kernels are examined. The prominent pattern in Figure 7b is an average over many kernels, and it is unclear how many individual kernels contribute to that average and whether the pattern is statistically significant. I recommend reporting the fraction of individual kernels with a clear directional peak at the expected offset (and at no other offset), along with a null-hypothesis comparison—for instance, using random spatial offsets or shuffled channel labels. Without such quantification, the claim that the recurrent weights 'encode the change in position resulting from each possible action' is under-supported, because a few strong kernels could dominate the average while most kernels behave differently.
  4. [Section I] The backtracking mechanism is supported primarily by a single case study. The quantitative intervention test reports a success rate of 85.1%±5.0% for long-term channels but only 48.9%±3.3% for short-term channels, the latter being essentially at chance. The paper explains that short-term negative activations also serve winner-takes-all and deadlock-prevention functions, but this explanation is post hoc and not directly tested. The claim that 'negative activations generated at obstacles is the key driver for backtracking' is therefore only directly supported for long-term channels. I recommend either filtering the short-term intervention dataset to remove WTA-related activations or explicitly weakening the claim for short-term channels.
minor comments (5)
  1. [Section 2.2] The text refers to 'hidden states h 0, h1, and h 3' but the network has layers d=0,1,2; this appears to be a typo and should read h0, h1, and h2.
  2. [Figures 25/26] The captions and the referencing text describe 'turn plan extension kernels' and 'box-down to box-right' mappings, but the captions in the appendix appear to describe 'Box-down to box-down' kernels; please ensure the captions match the content shown.
  3. [Abstract and Section 3] The phrase 'This causes the extension kernels to propagate the negative value in reverse' is slightly ambiguous; it would be clearer to state explicitly that the same linear and turn kernels propagate both positive and negative activations along the path, and that negative activations therefore reverse-propagate along previously extended segments.
  4. [Section 4] The claim of advancing 'the Pareto frontier between complexity of a network and the detail of its characterization' is subjective and not quantified; consider providing a concrete comparison metric or softening the statement.
  5. [Section N] The F1 score of 73.7% for the automatic labeler is reported without a confusion matrix or per-direction breakdown; reporting these would help the reader assess how much label noise is present in the cross-seed replication.

Circularity Check

1 steps flagged · score 4.0 of 10

AUC-based label verification is partly circular because channel offsets are fit to the same future-movement features later used as the prediction target; kernel and intervention results remain independently grounded.

  1. fitted input called prediction [Section 2.3 "Area under the curve analysis" and Section E "Label Verification and Offset Computation"]
    "To automatically compute the offsets, we perform linear regression on the base and future features to predict the channel activation by shifting the features along x, y∈ {−2,−1,0,1,2}and selecting the offset regression model with the lowest loss. ... Second, we find that each path channel is predictive of the box or agent’s future movements."

    The same future box/agent movement features serve as both the regression targets used to fit each channel's spatial offset (choosing the best of 25 shifts) and the labels whose predictability from the offset-corrected channel is then reported as AUC evidence. A channel shifted to minimize in-sample regression loss against a feature will, by construction, show elevated alignment with that feature; therefore the AUC 'finding' is partially baked into the offset-fitting procedure rather than being an independent confirmation. The manual labels and the causal interventions provide independent content, so this is a supporting circularity, not the central derivation.

full rationale

The paper's central planning-algorithm claims are not simply renamed inputs. The path-channel labels are initially assigned by manual inspection, and the causal interventions (Table 2), zero-ablations of WTA kernels (Figure 9), obstacle-positivity interventions in the backtracking case study (Figure 16), and successful weight-steering to solve larger levels (Section L) each test the proposed mechanisms independently of the regression-fitted offsets. The main circularity is confined to the quantitative label verification: the spatial offsets are fit by regression on the future-movement features, and the AUC/correlation tables then measure how well offset-corrected channels predict those same features, so those numbers are partly assured by the fitting procedure. A held-out refit of offsets, or an intervention-based determination of offsets, would remove this residual circularity. Because the central kernel-extension and backtracking evidence does not reduce to the fitted offsets, the overall circularity is moderate rather than severe.

Assumptions & free parameters 4 free parameters · 4 assumptions · 2 invented entities

The paper introduces no new physical entities. The main contributions, path channels, plan extension kernels, and winner-takes-all inhibition, are interpretations of existing network components, with causal interventions providing independent evidence for the channel roles. The central quantitative machinery includes fitted per-channel offsets, hand-selected horizon thresholds, and hand-tuned steering factors.

free parameters (4)
  • Per-channel spatial offsets = Table 5 entries, e.g., L0H0 (1,0), L1H0 (0,0), L2H0 (-1,0)
    Selected by lowest-loss linear regression on future movement features; used to align every channel before all kernel visualizations and analyses.
  • Short-term vs long-term horizon thresholds = 0-10 steps vs 10-50 steps
    Hand-chosen split used to label channels; the automatic labeling thresholds (AUC > 0.95, AUC at 50 > AUC at 1) are also hand tuned.
  • Weight steering factor = 1.2 for zig-zag up to 25x25; 1.4 for 40x40 levels
    Manual scale applied to W^d_ch1 and W^d_ch2 to stabilize longer path extension; no principled rule is given.
  • AUC label threshold = 0.95
    Threshold for automatic channel discovery in Section N; not justified beyond convenience.
assumptions (4)
  • domain assumption The DRC(3,3) architecture and learned weights from Taufeeque et al. (2024) are faithfully reproduced and used as the object of study.
    A single seed is analyzed in depth; other seeds are used for replication but with automated labeling.
  • domain assumption Hidden channel activations are the relevant carrier of planning information; encoder and MLP heads are treated as peripheral.
    Section D shows ablating i, j, o gates or h breaks the network (100% drop) while cell state and forget gate matter less; this is evidence, but the assumption that planning is 'in the channels' is prior to the analysis.
  • standard math The encoder is linear, so combined kernels exactly represent input-to-gate maps up to edge effects.
    Equation 2 uses associativity of the linear encoder; boundary effects are neglected.
  • domain assumption The Boxoban medium validation set is representative for evaluating planning.
    Used throughout for AUC, ablation, and intervention measurements.
invented entities (2)
  • Path channels (box-movement, agent-movement, combined path channels) independent evidence
    purpose: Posited as the direct representation of the plan in hidden state activations.
    Supported by cache ablation (57.6% solve rate drop) and causal interventions (up to 99.7% success), which provide handles outside the labeling procedure.
  • Winner-takes-all mechanism independent evidence
    purpose: Inhibition between direction channels to select a single path.
    Supported by zero-ablation experiment on a constructed level and cross-seed WTA matrices.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Path Channels and Plan Extension Kernels: a Mechanistic Description of Planning in a Sokoban RNN." pith.science (2026). https://pith.science/paper/V5D3YYHL

@misc{pith2026250610138,
  author       = {Pith},
  title        = {Pith review of: Path Channels and Plan Extension Kernels: a Mechanistic Description of Planning in a Sokoban RNN},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/V5D3YYHL}},
  note         = {Machine review of arXiv:2506.10138}
}
read the original abstract

We partially reverse-engineer a convolutional recurrent neural network (RNN) trained with model-free reinforcement learning to play the box-pushing game Sokoban. We find that the RNN stores future moves (plans) as activations in particular channels of the hidden state, which we call path channels. A high activation in a particular location means that, when a box is in that location, it will get pushed in the channel's assigned direction. We examine the convolutional kernels between path channels and find that they encode the change in position resulting from each possible action, thus representing part of a learned transition model. The RNN constructs plans by starting at the boxes and goals. These kernels extend activations in path channels forwards from boxes and backwards from the goal. Negative values are placed in channels at obstacles. This causes the extension kernels to propagate the negative value in reverse, thus pruning the last few steps and letting an alternative plan emerge; a form of backtracking. Our work shows that, a precise understanding of the plan representation allows us to directly understand the bidirectional planning-like algorithm learned by model-free training in more familiar terms.

Figures

Figures reproduced from arXiv: 2506.10138 by the authors.

Figure 1
Figure 1. A level with channel activations for a single channel from every group in Table 1. Note the [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Left: The DRC(3, 3) architecture. There are three layers of ConvLSTM modules with all the layers repeatedly applied three times before predicting the next action. Right: The ConvLSTM block in the DRC. Note the use of convolutions instead of linear layers, and the last layer of the previous tick (h n−1 D ) as input to the first layer. “Pool” refers to a weighted combination of mean- and max-pooling. and θ3 in sequenc… view at source ↗
Figure 3
Figure 3. An idealized path channels diagram. At each layer, the DRC has C channels, each of which is a H ×W array. The DRC repeats the same compu￾tations convolutionally over each square. This results in a subset of hidden state channels hd representing the plan where each channel corresponds to a move￾ment direction. If the agent or a box is at a position where the channel is activated, this causes the DRC to choose that ch… view at source ↗
Figures from the paper (24 more)
Figure 4
Figure 4. Figure 4: AUC scores for predicting box and agent movements from the path channels at different #s [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Activations of the long- and short-term channels for all directions when a different direction [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Visualizations of combined kernels that map from the RGB input to the [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Empirical plan extension kernels are formed by averaging over all kernels [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Plan stopping mechanism demonstration [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: After zero-ablating the kernels connect [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: A situation with two equally good paths from the box [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]
Figure 11
Figure 11. Figure 11: The planning algorithm circuit learned by [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 2
Figure 2. Figure 2: C NETWORK TRAINING DETAILS The network was trained using the IMPALA V-trace actor-critic (Espeholt et al., 2018) reinforcement learning (RL) algorithm for 2 · 109 environment steps with Guez et al.’s Deep Repeating ConvLSTM (DRC) recurrent architecture consisting of th…
Figure 12
Figure 12. Figure 12: High resolution visualization of a Sokoban level along with the corresponding symbolic [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: 16 × 16 zig-zag level that the original DRC(3, 3) network fails to solve. Steering Wd ch1 and Wd ch2 by a factor of 1.2 solves this level and similar zig-zag levels for sizes upto 25 × 25. of H = W = 10. The H × W observations are normalized by dividing each pixel com…
Figure 14
Figure 14. Figure 14: Left: Observation at step 3 where the agent moves down.Right: The GNA channels, which represent the direction that the agent will move in at the next step, predict the agent mov￾ing down primarily through f-gate. The box- and agent-down channels are offset and subtrac…
Figure 15
Figure 15. Figure 15: Transfer mechanism from long to short-term channel shown through contributions into [PITH_FULL_IMAGE:figures/full_fig_p020_15.png]
Figure 16
Figure 16. Figure 16: (a) 20 × 20 level we term as the “backtrack level” with key decision nodes D1-D3 for backward chaining. (b) The sum of box-movement channels at step 1 tick 0 indicates forward (from box) and backward (from target) chaining. (c-e) Activation of the box-right channel L2…
Figure 17
Figure 17. Figure 17: A level with two paths, one longer than the other. We initialize the starting hidden state [PITH_FULL_IMAGE:figures/full_fig_p022_17.png]
Figure 18
Figure 18. Figure 18: Sum of activations of box-movement channels on the [PITH_FULL_IMAGE:figures/full_fig_p023_18.png]
Figure 19
Figure 19. Figure 19: Emergence of the Winner-Takes-All mechanism during training. [PITH_FULL_IMAGE:figures/full_fig_p024_19.png]
Figure 20
Figure 20. Figure 20: Plan extension kernels across four networks trained independently with different seeds. [PITH_FULL_IMAGE:figures/full_fig_p026_20.png]
Figure 21
Figure 21. Figure 21: Winner-takes-all connectivity across four networks trained independently with different [PITH_FULL_IMAGE:figures/full_fig_p026_21.png]
Figure 22
Figure 22. Figure 22: The sum of activations of the box-movement channels on a [PITH_FULL_IMAGE:figures/full_fig_p027_22.png]
Figure 23
Figure 23. Figure 23: Plan extension and box path to agent path kernels. [PITH_FULL_IMAGE:figures/full_fig_p031_23.png]
Figure 24
Figure 24. Figure 24: 9 × 9 combined convolutional filters W0 oe that map the RGB observation image to the O gate in layer 0. The positive and negative components of each channel filters are separated visualized by computing max(0, W0 oe) and max(0, −W0 oe) respectively. The green, red, an…
Figure 25
Figure 25. Figure 25: Each of the Box-down to box-down plan-extension kernels, centered by their channels’ [PITH_FULL_IMAGE:figures/full_fig_p034_25.png]
Figure 26
Figure 26. Figure 26: Each of the Box-down to box-down plan-extension kernels, centered by their channels’ [PITH_FULL_IMAGE:figures/full_fig_p034_26.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

13 extracted references · 10 canonical work pages

  1. [1]

    A representation of states

    work page

  2. [2]

    A transition model that defines which nodes (states) are reachable from a currently expanded node when taking a certain action

    work page

  3. [3]

    A heuristic function that determines which nodes to expand

    work page

  4. [4]

    The heuristic varies by algorithm: • For A*, it is distance(n) +heuristic(n)

    A value function that determines which plan to choose once the search ends (in online search algorithms, only the first action from the chosen plan is taken). The heuristic varies by algorithm: • For A*, it is distance(n) +heuristic(n). (Russell & Norvig, 2009) • For iterative-deepening alpha-beta search (as used in Stockfish), the heuristic comprises mov...

    work page 2019

  5. [9]

    Positive values indicate self- reinforcement

    Self-connection strength: The average weight magnitude of kernels connecting each direction channel to itself (e.g., box-down → box-down). Positive values indicate self- reinforcement. 2.Cross-inhibition strength: The average weight magnitude of kernels connecting different direction channels (e.g., box-down → box-right). Negative values indicate mutual i...

    work page

  6. [10]

    The normalized difference is approximately −1, indicating no WTA structure—if anything, the opposite pattern (cross-excitation)

    Early training (0–10M steps):Both self-connection and cross-inhibition weights remain near zero. The normalized difference is approximately −1, indicating no WTA structure—if anything, the opposite pattern (cross-excitation)

    work page

  7. [11]

    Transition phase (10–19M steps):A rapid phase transition occurs where self-connection becomes positive while cross-inhibition becomes negative

    work page

  8. [12]

    This occurs at less than 1% of total training (19M of 2000M steps)

    WTA emergence (19M steps):The normalized self-cross difference reaches +1, indicating that the full WTA structure has emerged. This occurs at less than 1% of total training (19M of 2000M steps)

    work page

Show all 13 references
  1. [13]

    Box down

    Continued refinement (19M–2000M steps):After the WTA structure emerges, the absolute magnitudes continue to increase (self-connection reaches ∼0.5, cross-inhibition reaches ∼−0.2), but the qualitative structure remains stable. This analysis complements our weight-level mechani...

    2026

  2. [255]

    backtrack level

    The edge tiles in the levels from the dataset are always walls, so the playable area is 8×8 . The player has four actions available to move in cardinal directions (Up, Down, Left, Right). The reward is -0.1 per step, +1 for placing a box on a target, -1 for removing it, and +1...

    2024

  3. [2017]

    URL https://proceedings.neurips.cc/paper_files/paper/2017/ file/9e82757e9a1c12cb710ad680db11f6f1-Paper.pdf. S. Russell and P. Norvig.Artificial Intelligence: A Modern Approach. Prentice Hall Press, Upper Saddle River, NJ, USA, 3rd edition, 2009. ISBN 9780136042594. Max-Philipp...

    2017 arXiv

  4. [2019]

    https://distill.pub/2019/activation-atlas

    doi: 10.23915/distill.00015. https://distill.pub/2019/activation-atlas. 10 Published as a conference paper at ICLR 2026 Lawrence Chan, Adrià Garriga-Alonso, Nicholas Goldowsky-Dill, Ryan Greenblatt, Jenny Nitishin- skaya, Ansh Radhakrishnan, Buck Shlegeris, and Nate Thomas. Ca...

    2019 arXiv

  5. [2023]

    Philip Quirke and Fazl Barez

    Accessed: 2025-05-15. Philip Quirke and Fazl Barez. Understanding addition in transformers.arXiv preprint arXiv:2310.13121, 2023. URLhttp://arxiv.org/abs/2310.13121v9. Sébastien Racanière, Theophane Weber, David Reichert, Lars Buesing, Arthur Guez, Danilo Jimenez Rezende, Adri...

    2025 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.