Pith. sign in

REVIEW 4 major objections 6 minor 42 references

Laguerre Geometry for Interpreting Large Language Models

T0 review · 4 major / 6 minor · reviewed 2026-07-14 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A concept in an LLM is a Laguerre–Voronoi cell—not a point or direction—and that region geometry yields a training-free readout of what any hidden state encodes.

desk verdict Solid packaging of power diagrams into LLM concepts plus a usable training-free lens; the multi-token story and hierarchy results are real, but the interference claim and the GELU linearization are softer than the abstract sells. read the letter →

arxiv 2607.10578 v1 pith:RC5UNS3V submitted 2026-07-12 cs.AI

Chunwei Ma , Russell Wolfinger This is my paper · ORCID
classification cs.AI
keywords LaguerregeometryLaguerre-VoronoidiagramLLMinterpretabilityGeometricLensconcepthierarchypiecewise-linearregionslinearrepresentationhypothesisin-contextinterference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the usual pictures of concepts in large language models—single points, linear directions, or Gaussian clusters—are incomplete. The final unembedding layer of a decoder-only transformer already carves representation space into a Laguerre–Voronoi diagram: each vocabulary token owns a full cell defined by a center and a weight. A concept is that cell (or a union of cells), so notions like linear separability, inclusion, and hierarchy become precise geometric statements read off the model weights. The same subdivision idea is pushed inside every transformer layer by treating residual flow as piecewise-linear trees that can be hopped between when cross-token attention fires. From that decomposition the authors obtain Geometric Lens, a method that labels any intermediate hidden vector by the final cell its residual path would reach if later context is silenced, plus a 2-D Laguerre Autoencoder that draws both the decision cells and full reasoning trajectories. Empirically, the lens recovers the factually correct token under in-context interference better than prior heuristic lenses.

What carries the argument

Laguerre–Voronoi cells of the unembedding layer (Theorem 4.1) together with Geometric Lens: force later self-attention weights to 1 so the residual stream stays inside one piecewise-linear tree and read the final-cell label of that tree.

What would settle it

On a held-out set of factual prompts with deliberate in-context interference, check whether the Geometric Lens top-1 token at a fixed middle layer systematically fails to match the factually correct city while Logit Lens or Patchscopes succeed, or whether ablating Laguerre weights collapses hierarchy recovery to chance on WordNet hyponym sets.

Watch

Extended reading notes

Core claim

Concept geometry in an LLM is exactly the Laguerre–Voronoi diagram induced by the unembedding parameters: centers are half the unembedding rows and weights combine bias with squared row norms. Every intermediate residual state lives in a piecewise-linear region whose semantic label is the final Laguerre cell reached by residual flow with later cross-token attention zeroed. That label is therefore a grounded, hyperparameter-free decoding of what the hidden vector encodes.

Load-bearing premise

The argument needs smooth activations such as GELU or SwiGLU to be replaceable by ReLU networks that keep the same final-cell labels on real data, and it treats the label obtained after silencing later cross-token attention as the true concept of the hidden state rather than an artifact of that intervention.

Editorial extensions

If this is right

  • Linear separability of concept categories can be checked by linear programming over cell centers and weights rather than by training probes on ad-hoc vectors.
  • Hyponymy and hypernymy become readable from domination scores built only from unembedding centers and weights, without external attribute matrices.
  • Any residual stream at any layer can be assigned a unique vocabulary concept by Geometric Lens without training or hyperparameters.
  • When a model is misled by fictional prefixes, the middle-layer Geometric Lens token can recover the fact the final greedy token has abandoned.
  • A single 2-D Laguerre Autoencoder can display both decision cells and multi-token reasoning trajectories for visual diagnosis.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If cells are the primitive units of meaning, sparse autoencoders and activation-steering directions may be approximating cell geometry rather than discovering independent features.
  • Steering or editing that moves a residual vector across a Laguerre cell boundary should flip the decoded concept more reliably than equal-length moves inside a cell.
  • The same region-labeling logic could be applied layer-wise during training to regularize hierarchical concept structure explicitly.
  • Failures of domination scores on abstract domains may indicate that those concepts are not yet cell-organized in smaller models.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Laguerre geometry as a unifying account of concept structure in LLMs. It shows that the unembedding layer induces a Laguerre–Voronoi diagram (LVD) with centers c_j = W_j/2 and weights ν_j = b_j + ||W_j||²/4 (Theorem 4.1), and defines a concept as an LVD cell (or union of cells). Hierarchy is scored by a domination quantity involving centers and weights (Def. 4.3, Thm. 4.4). Extending piecewise-linear subdivision to transformers, the authors decompose multi-token computation into residual piecewise-linear flow plus attention-driven inter-region transport, and introduce Geometric Lens: force later self-attention to the current token and read the final-cell label of the remaining network. A Laguerre Autoencoder visualizes trajectories and cells in 2D. Empirically, region-based concepts improve linear probing slightly over categorical/lattice baselines (Tables 1, 4); domination recovers WordNet hyponyms better than lattice inclusion (Tables 2, 5); and Geometric Lens at a middle layer improves recovery of the factual token under in-context interference relative to Logit Lens and Patchscopes (Table 3).

Significance. If the framework holds, it would give a parameter-grounded, training-free definition of concepts and a principled alternative to heuristic lenses, with hierarchy readable from unembedding weights alone. Strengths that should be credited: Theorem 4.1 is a clean, correct specialization of the known power-diagram structure of linear heads; the single-token piecewise-affine partition (Theorem 4.6) is carefully stated; Geometric Lens and the LAE are fully specified and code is released; and the interference evaluation spans Phi-2, Gemma-2 (2B/9B), and Llama-3.1-8B. The multi-token “static tree + dynamic transport” cut is a useful conceptual contribution even if some downstream claims need tightening. These are real assets for mechanistic interpretability if the load-bearing gaps below are closed.

major comments (4)
  1. [§4.1.1, Theorem 4.4, Definition 3.1] Theorem 4.4 and Def. 4.1/3.1 are inconsistent. By Def. 3.1 the LVD cells form a partition: ω_r ∩ ω_r' = ∅ for r ≠ r'. Theorem 4.4 asserts that Domination(j→j')>0 implies ω_j' ⊆ ω_j, which is impossible for distinct cells. The proof only checks that the center c_j' prefers site j over j' and then incorrectly concludes full cell inclusion. At best the score detects that c_j' lies in ω_j (or that one power ball contains another); it does not establish geometric containment of cells. The hierarchy experiments (Fig. 1B, Fig. 9, Tables 2 and 5) may still be empirically useful, but the geometric claim as stated is false and must be restated (e.g., as a center-in-cell or sphere-containment criterion) with a corrected proof.
  2. [§4.2.2, Geometric Lens definition; Table 3] The central actionable claim—that Geometric Lens reads the “exact concept” of a hidden vector and recovers the factual token under interference (abstract; §4.2.2; Table 3)—rests on an intervention that changes the residual path: later self-attention is forced to π_s=1 only for the current token so Δ=0. Nothing shows that the final-cell label of this modified trajectory equals the semantic content of the original multi-token residual stream. The method is therefore a well-defined probe of the remaining piecewise-linear map, not a proven readout of the unintervened state. The paper should either (i) justify the intervention with a formal invariance argument or ablations (e.g., partial attention zeroing, comparison to residual patching), or (ii) weaken the claim from “exact concept” to “label of the self-contained residual flow.”
  3. [§5.3, Table 3; abstract claim of hyperparameter-free] Table 3 reports Geometric Lens accuracy using the token at layer 20 rather than the final layer, motivated by the observation that layers 20–25 are factually correct on one capital prompt (§5.3, Fig. 6). That layer index is a free hyperparameter chosen after inspecting trajectories, which undercuts the repeated claim that Geometric Lens is “training-free, hyperparameter-free” (abstract; §4.2.2; §5.3). Please either fix the layer a priori (or average over a pre-specified band), report a full layer-wise accuracy curve for all three lenses on the interference sets, and/or show that a single rule (e.g., first layer where top-1 stabilizes) works across models without per-dataset retuning.
  4. [§4.1.2, Proposition 4.1; Def. 4.5] Proposition 4.1 asserts that GELU/SwiGLU networks can be replaced by multi-layer ReLU nets so that ε-effective linear regions preserve classification labels on natural-language data D. No approximation error, label-agreement rate, or sensitivity analysis is reported for any model used in the experiments (Phi-2, Gemma, Llama, Pythia). Because Geometric Lens and the region-label definition (Def. 4.5) inherit this linearization for modern activations, the paper needs at least a small empirical check (e.g., agreement of top-1 under the true activation vs. a fitted ReLU surrogate on a held-out prompt set) or an explicit restriction of the piecewise-linear theory to models where the activation is already piecewise linear.
minor comments (6)
  1. [Abstract; Eq. (17); §5.3] Abstract and §4.2.2 call Geometric Lens hyperparameter-free, while the LAE loss (Eq. 17) uses φ1, φ2 and the interference protocol uses a chosen layer. Please scope the claim to the lens definition alone and list free choices in the experimental setup.
  2. [Theorem 4.3; Appendix A.1.2] In Theorem 4.3 the β terms are written with ||c||²-style norms inconsistently typeset (sometimes ||c_j||², sometimes ||c_j||_2 without square in nearby displays). Align notation with the expanded cell inequality in the proof (A.1.2).
  3. [§5.3; Appendix A.2] Patchscopes is evaluated with the official default target prompt only (§A.2). A short sensitivity note (one alternative prompt) would strengthen the comparison, even if full tuning is out of scope.
  4. [Figures 1, 6, 8] Figure 1(C) and several trajectory figures are hard to read in grayscale; consider markers or direct cell labels for the key tokens (_Paris, _Lyon, etc.).
  5. [§2.2; Theorem 4.1] Related work on power diagrams of linear layers (Balestriero et al., 2019) is cited; a one-sentence clarification that Theorem 4.1 is an application of that identity to LLM unembeddings (not a new existence proof) would set expectations correctly.
  6. [Abstract; throughout] Typos / polish: “Laguerre-V oronoi” spacing artifacts appear throughout; “adynamic transport” / “astatic tree” missing spaces in the abstract; arXiv id and July 2026 date look like placeholders—verify before camera-ready.

Circularity Check

2 steps flagged · score 3.0 of 10

LVD identity and region labels are definitional; linear separability of concepts and Geometric Lens 'exact concept' readout follow by construction, while hierarchy and interference results are external tests.

  1. self definitional [Def. 4.1 + Theorem 4.2 (§4.1.1)]
    "A concept C_j is defined as a Laguerre-Voronoi cell, characterized by the tuple (ω_j, c_j, ν_j)... Theorem 4.2 (Linear Separability of Concepts). By the Hyperplane Separation Theorem, any two concepts C_j, C_j′, for all j, j′ ∈ {1,...,V}, are linearly separable in R^d."

    Concepts are defined to be the LVD cells of the unembedding layer. Those cells are convex polyhedra separated by the power bisectors of the diagram, so pairwise linear separability is true by construction of the definition plus the Hyperplane Separation Theorem. The theorem does not discover a new geometric fact about LLM concepts; it restates the definition.

  2. self definitional [Def. 4.5 + Geometric Lens §4.2.2; Abstract]
    "C(R^(ℓ)(h)) = arg max_j G^(ℓ)_j(h)... To decode h^(ℓ)_i, we directly apply Eq. 15 to obtain the label of its response region... so that the contextual incremental vector Δ=0 at every layer after ℓ. We then run a forward pass to obtain y_GeometricLens(h^(ℓ)_i)=G^(ℓ)(h^(ℓ)_i). ... Geometric Lens, a training-free, hyperparameter-free method for reading out the exact concept a hidden vector encodes at any layer."

    The paper defines the concept/label of a hidden vector as the final LVD cell reached by the remaining piecewise-linear path with later cross-token attention forced off. Geometric Lens is exactly that procedure. Calling the output the 'exact concept the hidden vector encodes' is therefore true by definition of concept-for-hidden-states, not an independent readout of a separately specified ground truth. Empirical usefulness (Table 3, hierarchy) is separate and non-circular; the 'exact concept' slogan is not.

full rationale

The paper's core geometry is not a fitted prediction: Theorem 4.1 reparameterizes the unembedding affine map as an LVD (centers W_j/2, weights b_j + ||W_j||²/4), a standard Power-diagram identity for linear classifiers, not a circular fit. Hierarchy (domination scores vs WordNet) and in-context interference (Table 3) are checked against external labels and prompts, so those claims are not forced by construction. Circularity is limited and local: (1) a concept is defined as an LVD cell, after which Theorem 4.2's linear separability of any two concepts is immediate from the Hyperplane Separation Theorem and the cell definition; (2) Geometric Lens is defined to return the final-cell label of the residual path with later cross-token attention zeroed (Eq. 15 / §4.2.2), so the slogan that it reads the 'exact concept' a hidden vector encodes is tautological with that definition—usefulness is then argued empirically, not derived. Self-citations to Ma et al. on Voronoi DNNs are background, not a uniqueness chain that forces the main results. No fitted scalar is renamed as a first-principles prediction. Score 3 reflects mild self-definitional framing around 'concept' and 'exact readout,' not a load-bearing circular derivation of the empirical claims.

Assumptions & free parameters 4 free parameters · 6 assumptions · 4 invented entities

The framework rests on classical power-diagram geometry of linear classifiers, piecewise-linear region theory for ReLU nets, and a Universal-Approximation relaxation for smooth activations. Free choices include LAE loss weights, the middle-layer index used for interference accuracy, and the random-token baseline for domination. Invented operational entities are Geometric Lens, Laguerre Autoencoder, and the domination score; they are defined from model parameters but lack independent external measurement beyond the paper’s own tasks.

free parameters (4)
  • LAE loss weights φ1, φ2 = φ1=1.0, φ2=20.0
    Set to 1.0 and 20.0 by hand to balance reconstruction vs. cell-classification loss when training the 2D visualizer; not derived.
  • Interference readout layer index = layer 20
    Layer 20 is chosen after observing factual peaks around layers 20–25 on the France/Paris trajectory, then used as the reported output layer for all interference accuracy numbers.
  • ϵ in ϵ-effective linearization
    Proposition 4.1 asserts existence of ReLU approximators with sup error < ϵ on dataset D; no numerical ϵ or trained approximators are reported for GELU/SwiGLU models used in experiments.
  • Domination random baseline set J
    Hierarchy accuracy subtracts average Domination toward a random background token set; size and sampling of J affect the discriminant threshold.
assumptions (6)
  • standard math A linear multiclass head induces a Laguerre–Voronoi (power) diagram with centers W_j/2 and weights b_j + ||W_j||²/4.
    Theorem 4.1 / Appendix A.1.1; classical (Aurenhammer 1987; Balestriero et al. 2019).
  • standard math Feedforward nets with piecewise-linear activations partition input and intermediate spaces into linear regions; residual shortcuts preserve piecewise affinity.
    Theorems 4.5–4.6, Lemma A.2; Montúfar/Pascanu/Arora line of work.
  • ad hoc to paper Smooth activations (GELU, SwiGLU) may be replaced by ReLU networks so that region labels on natural-language data are preserved up to ϵ (Proposition 4.1).
    Invoked to extend the piecewise-linear story to modern LLMs without empirical verification of ϵ or label preservation.
  • domain assumption Forcing later multi-head attention weights to self-only (π_s=1 iff s=i) isolates the semantic concept of a residual vector.
    Definition of Geometric Lens §4.2.2; load-bearing interpretive assumption, not a theorem.
  • domain assumption RMSNorm and RoPE do not change the induced piecewise-affine partition for single-token attention (remark after Theorem 4.6).
    Stated without full multi-token analysis; used to claim partitions exist in practical transformers.
  • domain assumption WordNet-filtered single-token vocabulary items are pure enough concept proxies for linear-separability and hierarchy benchmarks.
    §5.1–5.2 and Appendix A.2; authors note residual polysemy as a limitation.
invented entities (4)
  • Geometric Lens
    purpose: Training-free, hyperparameter-free decoding of the concept (final LVD cell) encoded by any residual stream vector.
    Defined by zeroing cross-token attention after layer ℓ and reading G^(ℓ)(h). Validated only on the paper’s own trajectory and interference tasks.
  • Laguerre Autoencoder (LAE)
    purpose: 2D visualization that keeps reconstructed points inside the same final LVD cell.
    Custom autoencoder with CE-to-cell loss; visualization aid, not an external measurement device.
  • Domination score Domination(j→j') = ν_j − ν_j' − ||c_j − c_j'||²
    purpose: Attribute-free detector of concept inclusion/hierarchy from unembedding parameters alone.
    New scalar derived from Laguerre weights; empirical hierarchy accuracy reported, but Theorem 4.4’s full-inclusion claim is under-proved.
  • Concept as Laguerre–Voronoi cell (Definition 4.1)
    purpose: Replace point/direction/Gaussian concept hypotheses with a region that is linearly separable by construction.
    Redefinition that makes LRH questions (I)–(III) answerable; success measured by probing and hierarchy tasks inside the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Laguerre Geometry for Interpreting Large Language Models." pith.science (2026). https://pith.science/paper/RC5UNS3V

@misc{pith2026260710578,
  author       = {Pith},
  title        = {Pith review of: Laguerre Geometry for Interpreting Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RC5UNS3V}},
  note         = {Machine review of arXiv:2607.10578}
}
read the original abstract

Existing hypotheses represent a concept in an LLM as a single point, a linear direction, or a Gaussian cluster, yet it remains unclear how and why such structures emerge. Here, we show that concept geometry can be precisely characterized via Laguerre Geometry, in which a concept is defined as a region--a Laguerre-Voronoi cell or a union of cells--allowing us to strictly define, measure, and separate concepts. Building on this formulation, we show that finer-grained concept structures, such as inclusion and hierarchy, are naturally revealed by the Laguerre weights. We then push this geometry inside the transformer. Decomposing each layer into piecewise-linear operators, we show that a token's hidden trajectory is governed by two coupled mechanisms: a static tree of self-contained piecewise-linear flow, and a dynamic transport that hops the trajectory across trees when cross-token attention fires. This decomposition yields Geometric Lens, a training-free, hyperparameter-free method for reading out the exact concept a hidden vector encodes at any layer. We also develop Laguerre Autoencoder, a 2D visualizer that renders both the decision geometry and a model's full reasoning trajectory in one view. Finally, we move beyond explanatory geometry toward actionable interpretability, showing that Geometric Lens recovers the correct factual token when a model is prompted with in-context interference. The code is available on GitHub.

Figures

Figures reproduced from arXiv: 2607.10578 by the authors.

Figure 1
Figure 1. (A) A demonstration of all possible conditions of two oriented hyperspheres in Laguerre Geometry. (B) The [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. An example piecewise-linear space subdivision induced by a one-layer ReLU network with three classes •, •, and •. Each region in the 2D input space is as￾signed a label, e.g., C(R) =•. See §A.2 for more details. Space Partition for LLMs with Piecewise-Linear Activations. It is well known that a deep neural network (DNN) with piecewise-linear ac￾tivation functions (e.g., ReLU, leaky-ReLU, MaxOut) and affine preacti￾v… view at source ↗
Figure 4
Figure 4. The mean trajectories along the layers of the Phi-2 model on the country dataset. The [25%, 75%] and [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figures from the paper (13 more)
Figure 3
Figure 3. Figure 3: Schematic illustration of inter-region transport within an LLM. The orange arrows in￾dicate the flow from a single token "is" and the blue arrows denote the entire prompt "The capital of France is". This lets us decompose the full transformer computation into a flow "i…
Figure 5
Figure 5. Figure 5: Schematic illustration of the Laguerre Autoencoder. To visually inspect the Laguerre Geometry of LLMs proposed above, we need a tool that can jointly visualize (1) final-layer representations, (2) intermediary￾layer representations, (3) representation trajectories, (4)…
Figure 6
Figure 6. Figure 6: Visualization of trajectories for the prompt [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Monitored log ranks of the top-5 final tokens for the prompt " [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 11
Figure 11. Figure 11: The first path, generated by the token "The", collapses to a single point, confirming that all its intermediary [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]
Figure 8
Figure 8. Figure 8: Laguerre Autoencoder visualization of the [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: (A) Use distance to the embedding vector of "_Dogs" to retrieve its hyponyms/hypernyms. (B) Use the [PITH_FULL_IMAGE:figures/full_fig_p021_9.png]
Figure 10
Figure 10. Figure 10: The token revealed by different lenses at each layer for the prompt: "The capital of France is". Three [PITH_FULL_IMAGE:figures/full_fig_p021_10.png]
Figure 11
Figure 11. Figure 11: The full trajectories from the Geometric Lens for all token positions for the prompt: " [PITH_FULL_IMAGE:figures/full_fig_p022_11.png]
Figure 12
Figure 12. Figure 12: The monitored log ranks of the top-5 final tokens for the prompt " [PITH_FULL_IMAGE:figures/full_fig_p022_12.png]
Figure 13
Figure 13. Figure 13: The monitored log ranks of the top-5 final tokens for the prompt " [PITH_FULL_IMAGE:figures/full_fig_p023_13.png]
Figure 14
Figure 14. Figure 14: The Laguerre Autoencoder visualization of the prompt: " [PITH_FULL_IMAGE:figures/full_fig_p023_14.png]
Figure 15
Figure 15. Figure 15: The Laguerre Autoencoder visualization of the prompt: " [PITH_FULL_IMAGE:figures/full_fig_p024_15.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 29 linked inside Pith

  1. [1]

    The geometry of thought: Disclosing the transformer as a tropical polynomial circuit

    Faruk Alpay and Bilge Senturk. The geometry of thought: Disclosing the transformer as a tropical polynomial circuit. arXiv preprint arXiv:2601.09775,

    arXiv

  2. [2]

    Understanding deep neural networks with rectified linear units.arXiv preprint arXiv:1611.01491,

    Raman Arora, Amitabh Basu, Poorya Mianjy, and Anirbit Mukherjee. Understanding deep neural networks with rectified linear units.arXiv preprint arXiv:1611.01491,

    arXiv

  3. [3]

    Layer normalization.arXiv preprint arXiv:1607.06450,

    Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization.arXiv preprint arXiv:1607.06450,

    arXiv

  4. [4]

    Representation alignment rests on linear structure.arXiv preprint arXiv:2605.28870,

    Kiril Bangachev, Guy Bresler, and Yury Polyanskiy. Representation alignment rests on linear structure.arXiv preprint arXiv:2605.28870,

    arXiv

  5. [5]

    Eliciting latent predictions from transformers with the tuned lens.arXiv preprint arXiv:2303.08112,

    Nora Belrose, Igor Ostrovsky, Lev McKinney, Zach Furman, Logan Smith, Danny Halawi, Stella Biderman, and Jacob Steinhardt. Eliciting latent predictions from transformers with the tuned lens.arXiv preprint arXiv:2303.08112,

    arXiv

  6. [6]

    Temporal sparse autoencoders: Leveraging the sequential nature of language for interpretability.arXiv preprint arXiv:2511.05541,

    Usha Bhalla, Alex Oesterling, Claudio Mayrink Verdun, Himabindu Lakkaraju, and Flavio P Calmon. Temporal sparse autoencoders: Leveraging the sequential nature of language for interpretability.arXiv preprint arXiv:2511.05541,

    arXiv

  7. [7]

    Do sparse autoencoders capture concept manifolds?arXiv preprint arXiv:2604.28119,

    Usha Bhalla, Thomas Fel, Can Rager, Sheridan Feucht, Tal Haklay, Daniel Wurgaft, Siddharth Boppana, Matthew Kowal, Vasudev Shyam, Jack Merullo, et al. Do sparse autoencoders capture concept manifolds?arXiv preprint arXiv:2604.28119,

    arXiv

  8. [8]

    Belief dynamics reveal the dual nature of in-context learning and activation steering.arXiv preprint arXiv:2511.00617,

    Eric Bigelow, Daniel Wurgaft, YingQiao Wang, Noah Goodman, Tomer Ullman, Hidenori Tanaka, and Ekdeep Singh Lubana. Belief dynamics reveal the dual nature of in-context learning and activation steering.arXiv preprint arXiv:2511.00617,

    arXiv

Show all 42 references
  1. [9]

    Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374,

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374,

    arXiv

  2. [10]

    Toy models of superposition.arXiv preprint arXiv:2209.10652,

    Elhage, N, Hume, T, Olsson, C, Schiefer, N, Henighan, T, Kravec, S, Hatfield-Dodds, Z, Lasenby, R, Drain, D, Chen, C, and et al. Toy models of superposition.arXiv preprint arXiv:2209.10652,

    arXiv

  3. [11]

    Not all language model features are one- dimensionally linear

    Josh Engels, Eric Michaud, Isaac Liao, Wes Gurnee, and Max Tegmark. Not all language model features are one- dimensionally linear. InInternational Conference on Learning Representations, volume 2025, pp. 84591–84622,

    2025

  4. [12]

    Characterizing the discrete geometry of relu networks.arXiv preprint arXiv:2606.07728,

    Blake B Gaines and Jinbo Bi. Characterizing the discrete geometry of relu networks.arXiv preprint arXiv:2606.07728,

    arXiv

  5. [13]

    The pile: An 800gb dataset of diverse text for language modeling.arXiv preprint arXiv:2101.00027,

    Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al. The pile: An 800gb dataset of diverse text for language modeling.arXiv preprint arXiv:2101.00027,

    arXiv

  6. [14]

    Gemma 2: Improving open language models at a practical size.arXiv preprint arXiv:2408.00118,

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. Gemma 2: Improving open language models at a practical size.arXiv preprint arXiv:2408.00118,

    arXiv

  7. [15]

    Patchscopes: A unifying framework for inspecting hidden representations of language models.arXiv preprint arXiv:2401.06102,

    Asma Ghandeharioun, Avi Caciularu, Adam Pearce, Lucas Dixon, and Mor Geva. Patchscopes: A unifying framework for inspecting hidden representations of language models.arXiv preprint arXiv:2401.06102,

    arXiv

  8. [16]

    Intricacies of feature geometry in large language models

    Satvik Golechha, Lucius Bushnaq, Euan Ong, Neeraj Kayal, and Nandi Schoots. Intricacies of feature geometry in large language models. InThe Fourth Blogpost Track at ICLR 2025,

    2025

  9. [17]

    When models manipulate manifolds: The geometry of a counting task.arXiv preprint arXiv:2601.04480, 2026a

    Wes Gurnee, Emmanuel Ameisen, Isaac Kauvar, Julius Tarng, Adam Pearce, Chris Olah, and Joshua Batson. When models manipulate manifolds: The geometry of a counting task.arXiv preprint arXiv:2601.04480, 2026a. Wes Gurnee, Nicholas Sofroniew, Adam Pearce, Mateusz Piotrowski, Isaa...

    2026

  10. [18]

    A structural probe for finding syntax in word representations

    John Hewitt and Christopher D Manning. A structural probe for finding syntax in word representations. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Pape...

    2019

  11. [19]

    On the origins of linear representations in large language models.arXiv preprint arXiv:2403.03867,

    Yibo Jiang, Goutham Rajendran, Pradeep Ravikumar, Bryon Aragam, and Victor Veitch. On the origins of linear representations in large language models.arXiv preprint arXiv:2403.03867,

    arXiv

  12. [20]

    Are sparse autoencoders useful? a case study in sparse probing.arXiv preprint arXiv:2502.16681,

    Subhash Kantamneni, Joshua Engels, Senthooran Rajamanoharan, Max Tegmark, and Neel Nanda. Are sparse autoencoders useful? a case study in sparse probing.arXiv preprint arXiv:2502.16681,

    arXiv

  13. [21]

    S, Dean, and J

    Mikolov, T, Sutskever, I, Chen, K, Corrado, G. S, Dean, and J. Distributed representations of words and phrases and their compositionality.Advances in neural information processing systems, 2013a. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of...

    arXiv

  14. [22]

    Incorporating hierarchical semantics in sparse autoencoder architectures.arXiv preprint arXiv:2506.01197,

    Mark Muchane, Sean Richardson, Kiho Park, and Victor Veitch. Incorporating hierarchical semantics in sparse autoencoder architectures.arXiv preprint arXiv:2506.01197,

    arXiv

  15. [23]

    Steering llama 2 via contrastive activation addition.arXiv preprint arXiv:2312.06681,

    Nina Panickssery, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner. Steering llama 2 via contrastive activation addition.arXiv preprint arXiv:2312.06681,

    arXiv

  16. [24]

    The linear representation hypothesis and the geometry of large language models.arXiv preprint arXiv:2311.03658,

    Kiho Park, Yo Joong Choe, and Victor Veitch. The linear representation hypothesis and the geometry of large language models.arXiv preprint arXiv:2311.03658,

    arXiv

  17. [25]

    The geometry of categorical and hierarchical concepts in large language models

    Kiho Park, Yo Joong Choe, Yibo Jiang, and Victor Veitch. The geometry of categorical and hierarchical concepts in large language models. InInternational Conference on Learning Representations, volume 2025, pp. 76441–76463,

    2025

  18. [26]

    On the number of response regions of deep feed forward networks with piecewise linear activations.arXiv preprint arXiv:1312.6098,

    Razvan Pascanu, Guido Montufar, and Yoshua Bengio. On the number of response regions of deep feed forward networks with piecewise linear activations.arXiv preprint arXiv:1312.6098,

    arXiv

  19. [27]

    Linear representations of hierarchical concepts in language models.arXiv preprint arXiv:2604.07886,

    Masaki Sakata, Benjamin Heinzerling, Takumi Ito, Sho Yokoi, and Kentaro Inui. Linear representations of hierarchical concepts in language models.arXiv preprint arXiv:2604.07886,

    arXiv

  20. [28]

    From directions to regions: Decomposing activations in language models via local geometry.arXiv preprint arXiv:2602.02464,

    16 Laguerre Geometry for Interpreting Large Language Models Or Shafran, Shaked Ronen, Omri Fahn, Shauli Ravfogel, Atticus Geiger, and Mor Geva. From directions to regions: Decomposing activations in language models via local geometry.arXiv preprint arXiv:2602.02464,

  21. [29]

    Open problems in mechanistic interpretability.arXiv preprint arXiv:2501.16496,

    Lee Sharkey, Bilal Chughtai, Joshua Batson, Jack Lindsey, Jeff Wu, Lucius Bushnaq, Nicholas Goldowsky-Dill, Stefan Heimersheim, Alejandro Ortega, Joseph Bloom, et al. Open problems in mechanistic interpretability.arXiv preprint arXiv:2501.16496,

    arXiv

  22. [30]

    Emotion concepts and their function in a large language model.arXiv preprint arXiv:2604.07729,

    Nicholas Sofroniew, Isaac Kauvar, William Saunders, Runjin Chen, Tom Henighan, Sasha Hydrie, Craig Citro, Adam Pearce, Julius Tarng, Wes Gurnee, et al. Emotion concepts and their function in a large language model.arXiv preprint arXiv:2604.07729,

    arXiv

  23. [31]

    Expressivity of transformers: A tropical geometry perspective.arXiv preprint arXiv:2604.14727,

    Ye Su and Yong Liu. Expressivity of transformers: A tropical geometry perspective.arXiv preprint arXiv:2604.14727,

    arXiv

  24. [32]

    Sparsity is combinatorial depth: Quantifying moe expressivity via tropical geometry.arXiv preprint arXiv:2602.03204,

    Ye Su, Huayi Tang, Zixuan Gong, and Yong Liu. Sparsity is combinatorial depth: Quantifying moe expressivity via tropical geometry.arXiv preprint arXiv:2602.03204,

    arXiv

  25. [33]

    Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet.arXiv preprint arXiv:2605.29358,

    Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, et al. Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet.arXiv preprint arXiv:2605.29358,

    arXiv

  26. [34]

    What do you learn from context? probing for sentence structure in contextualized word representations.arXiv preprint arXiv:1905.06316,

    Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R Thomas McCoy, Najoung Kim, Benjamin Van Durme, Samuel R Bowman, Dipanjan Das, et al. What do you learn from context? probing for sentence structure in contextualized word representations.arXiv preprint arXiv:1905.06316,

    1905 arXiv

  27. [35]

    Steering language models with activation engineering.arXiv preprint arXiv:2308.10248,

    Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J Vazquez, Ulisse Mini, and Monte MacDiarmid. Steering language models with activation engineering.arXiv preprint arXiv:2308.10248,

    arXiv

  28. [36]

    Next-gpt: Any-to-any multimodal llm.arXiv preprint arXiv:2309.05519,

    Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji, and Tat-Seng Chua. Next-gpt: Any-to-any multimodal llm.arXiv preprint arXiv:2309.05519,

    arXiv

  29. [37]

    The lattice representation hypothesis of large language models.arXiv preprint arXiv:2603.01227,

    Bo Xiong. The lattice representation hypothesis of large language models.arXiv preprint arXiv:2603.01227,

    arXiv

  30. [38]

    Beyond single concept vector: Modeling concept subspace in llms with gaussian distribution

    Haiyan Zhao, Heng Zhao, Bo Shen, Ali Payani, Fan Yang, and Mengnan Du. Beyond single concept vector: Modeling concept subspace in llms with gaussian distribution. InInternational Conference on Learning Representations, volume 2025, pp. 82998–83026,

    2025

  31. [39]

    Representation engineering: A top-down approach to ai transparency

    Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, et al. Representation engineering: A top-down approach to ai transparency. arXiv preprint arXiv:2310.01405,

    arXiv

  32. [40]

    Let z′ denote the vertical projection of z onto U, and let z′′ denote its vertical projection onto Π(S)

    Next, let U denote the paraboloid y=∥z∥ 2 2, and let Π(S) denote the transformation that maps a sphere S with center c and radiusrto the hyperplane Π(S) :y= 2z ⊤c−c ⊤c+r 2.(21) Aurenhammer [Aurenhammer, 1987] showed that Π is a bijection between arbitrary spheres in Rn and non...

    1987

  33. [41]

    _Dogs" to retrieve its hyponyms/hypernyms. (B) Use the domination score of

    (23) Although ν admits the geometric interpretation of a squared radius, i.e., ν=r 2, it is not necessarily nonnegative for an arbitrary unembedding layer (or, more generally, an arbitrary multiclass linear classifier), since bk may take sufficiently negative values. A.1.2 Pro...

    2016

  34. [42]

    You are in a fictional world where Hamburg and Frankfurt have swapped their names. The Bode Museum is located in the city of

    Each hidden neuron partitions the input space into two half-spaces (gray lines in the figure), while the final decision boundary between two classes is projected onto the input space as a piecewise-linear curve composed of linear segments (red segments). Together, the hidden-l...

    2014

Pith tools

Reviewed July 14, 2026 · model on record in the stance chip above.