Pith. sign in

REVIEW 3 major objections 4 minor 31 references

Language models only form clean 1D ordinal manifolds when the value can be read off local token identity; otherwise geometry is higher-dimensional or incoherent, and twisting is architecture-dependent.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 21:12 UTC pith:ICLZEIVL

load-bearing objection Solid multi-task, multi-model extension of Gurnee-style ordinal geometry; the local-computability taxonomy is useful but rests on centroid PCA that the authors themselves flag as incomplete. the 3 major comments →

arxiv 2607.04167 v1 pith:ICLZEIVL submitted 2026-07-05 cs.LG

Geometry of Ordinal Representations in Language Models

Saksham Bassi , Sharvi Tomar This is my paper
classification cs.LG
keywords language modelsrepresentation geometryordinal manifoldsplace-cell featuresattention twistingactivation patchingmechanistic interpretability
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Earlier work found that models keep character counts on curved one-dimensional manifolds and use attention heads to twist those manifolds so they can compute distances. This paper asks whether the same geometry appears for other ordinal quantities—bracket nesting depth, Python indentation, table row/column indices, and numeric magnitude—across three open models. The answer is partial: clean 1D manifolds with place-cell-like feature tiling appear only when the ordinal value is already available from the current token’s identity. Tasks that require integrating information across positions or extracting semantic magnitude produce higher-dimensional or messy representations even when linear probes still decode the value well. How much geometric twisting attention heads perform also depends on the model family: one architecture twists far more strongly than the others, and only some of those twists preserve ordinal order. Ablating the discovered manifold directions collapses probe accuracy far more than ablating random directions, showing that the subspaces really do concentrate the task information.

Core claim

Across four ordinal tasks and three models, one-dimensional manifolds with place-cell feature tiling emerge precisely when the ordinal variable is locally computable from token identity; tasks that need cross-position integration or semantic extraction yield higher-dimensional or incoherent representations, geometric twisting strength is architecture-dependent, and only some high-twist heads preserve ordinal order, while the identified subspaces concentrate task-relevant information under activation patching.

What carries the argument

The centroid manifold obtained by PCA on per-value mean residual-stream activations, together with three head-level geometric scores (twist, alignment, ordinality) measured after projection into each head’s query space, and top-k manifold-direction activation patching as a necessity test.

Load-bearing premise

That the geometry of class-average activation vectors is a faithful enough picture of the geometry that individual tokens actually live on, so that high variance explained by the first principal component can be treated as evidence of a usable one-dimensional manifold.

What would settle it

Measure the residual-stream activations of individual tokens (not just class means) for a locally-computable task such as bracket depth; if those tokens do not concentrate near a one-dimensional curve whose principal axis still predicts the ordinal label after the same subspace ablation, the claimed manifold geometry fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper tests whether the curved 1D manifold geometry and attention-head geometric transformations reported by Gurnee et al. for character counting generalize to four ordinal tasks (bracket nesting depth, Python indentation, markdown table row/column, numeric magnitude) in Gemma-2-2B, Gemma-2-9B, and Qwen3-4B. Using linear probes, PCA on per-value mean residual activations, Gemma Scope SAE tuning curves, Q-space twist/alignment/ordinality scores, and top-k vs random subspace activation patching, the authors report that clean 1D manifolds with place-cell-like feature tiling appear for tasks whose ordinal variable is locally computable from token identity (brackets, table columns), while cross-position integration or semantic extraction yields higher-dimensional or incoherent structure. Geometric twisting is architecture-dependent (stronger and sometimes order-preserving in Qwen3-4B), and manifold-direction ablation produces far larger probe drops than random controls.

Significance. If the local-computability taxonomy and architecture-dependent geometric computation hold, the work supplies a concrete, multi-task, multi-model extension of the Gurnee et al. motif and a useful organizing principle for when ordinal structure is expected to be low-dimensional and place-cell tiled versus diffuse. Strengths include cleanly defined diagnostics (twist, alignment, ordinality), systematic cross-model comparison, random-direction patching controls, and explicit caveats about probe overfitting and centroid geometry. The empirical package is reproducible in principle and advances mechanistic interpretability of structured representations beyond single-task case studies.

major comments (3)
  1. [Methods §3.2; Fig. 4; Table 3; Discussion] Methods §3.2 and the subsequent taxonomy rest on PCA of per-value mean activations (class centroids). The authors correctly note that high PC1 variance among centroids is necessary but not sufficient for individual tokens to concentrate near a 1D manifold. Yet every load-bearing claim—PC1 curves (Fig. 4), the local-computability taxonomy, place-cell tiling interpretation, and the twist/alignment/ordinality analysis that uses those same centroids (Eq. 1, Table 3)—treats centroid geometry as the manifold the model uses. Subspace ablation (Table 4) only establishes informational concentration of the top-k centroid directions for linear probes; it does not show token-level concentration. A token-level scatter or residual-variance check (or an explicit statement that the taxonomy is only about centroid geometry) is needed before the strongest claim is secured.
  2. [§4.1; Table 1; Limitations] Bracket-depth probes show severe train/test gaps (≈99% train vs 53–57% test; Table 1 and Limitations). The paper attributes this to token-identity shortcuts and still reports clean 1D manifolds and place-cell features for brackets. Because the local-computability story leans heavily on brackets as the paradigmatic “pre-organized 1D” case, the manifold findings for this task should be re-evaluated on a held-out distribution that breaks the token–depth correlation, or the claim should be explicitly restricted to the training distribution.
  3. [Abstract; §4.2; Limitations; Future directions] Place-cell feature tiling is demonstrated only on Gemma (Gemma Scope SAEs); the strongest geometric twisting is demonstrated only on Qwen3-4B (no pretrained SAEs). The abstract and Discussion present both as joint support for the same geometric-computation story. Either train an SAE on Qwen3-4B or clearly separate the two findings so that the architecture-dependent twisting claim is not over-read as also confirming cleaner feature tiling.
minor comments (4)
  1. [Table 1; Table 4] Numeric magnitude on Gemma-2-9B yields negative test R² (Table 1, Table 4 footnote); the large manifold drop should be interpreted with an explicit warning that the baseline is broken, not merely footnoted.
  2. [§3.1; Limitations] The synthetic bracket labeling inconsistency (always closing with ‘)’) is mentioned only in Limitations; it should be flagged when the bracket results are first presented.
  3. [Figure 3] Figure 3 heatmaps use very different color scales (0.03 vs 0.8); a shared or clearly annotated scale would make the 25× difference easier to read without relying on the caption alone.
  4. [§3.2; §4.1] Clarify early that “manifold dimensionality” always means centroid PCA dimensionality, not token-level intrinsic dimension, so readers do not over-interpret Fig. 4.

Circularity Check

0 steps flagged

Empirical measurement paper with no circular derivation: scores and taxonomy are computed from data, not forced by definition or self-citation.

full rationale

The paper is a multi-task, multi-model empirical extension of Gurnee et al. (2026). Manifold dimensionality is obtained by PCA on per-value mean residual activations; twist/alignment/ordinality scores are defined directly from pairwise distances and Spearman correlations of those centroids after Q-projection (Eq. 1 and surrounding definitions); SAE selectivity/monotonicity are thresholded statistics on tuning curves; and subspace necessity is tested by activation patching of the top-k PCA directions against random-direction controls. None of these quantities is fitted to the target claim and then re-presented as a prediction, nor is any result obtained by renaming a known pattern or by invoking a uniqueness theorem from overlapping authors. The local-computability taxonomy is an after-the-fact observational grouping of the measured dimensionalities, not a derivation that reduces to its inputs by construction. The sole methodological caveat (centroid PCA is necessary but not sufficient for token-level manifold concentration) is a limitation of evidence strength, not circularity. No self-citation is load-bearing. Score 0 is therefore appropriate.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 2 invented entities

The central taxonomy rests on standard residual-stream and linear-probe assumptions plus a handful of analysis choices (centroid PCA, 90% variance cutoff, selectivity 3× mean, k=3 patching, Spearman ordinality). No new physical entities are postulated; the ‘manifold’ and ‘twister’ language is descriptive of observed geometry rather than a new ontological claim. Free parameters are analysis thresholds, not fitted physical constants.

free parameters (4)
  • PCA variance threshold for manifold dimensionality = 90%
    Number of PCs needed to explain 90% of variance among per-value means is used to declare 1D vs higher-D manifolds; the 90% cutoff is conventional but free.
  • SAE feature selectivity threshold = 3× mean
    A feature is selective if peak > 3× mean activation; used to count place-cell-like features.
  • Top-k manifold directions for patching = k=3
    k=3 directions are ablated; reported as the layer with largest manifold drop.
  • Feature activity filter = >5%
    Features kept only if active on >5% of tokens before ranking by between-value variance.
axioms (5)
  • domain assumption Residual-stream activations at labeled token positions linearly encode the ordinal variable sufficiently for logistic/Ridge probes to recover it.
    Standard linear-representation / probing assumption used throughout §3.2 and Table 1.
  • ad hoc to paper PCA on per-value mean activations (class centroids) adequately indexes the geometry relevant to ordinal computation.
    Explicitly adopted in §3.2; authors note it is necessary but not sufficient for token-level manifold concentration.
  • domain assumption Projecting residual means through an attention head’s query matrix W_Q yields a meaningful Q-space geometry for measuring twist, alignment, and ordinality.
    Adopted from Gurnee et al. (2026) and formalized in Eq. (1) and the three diagnostic scores in §3.3.
  • domain assumption Gemma Scope residual-stream SAEs (16k dictionary) recover features whose tuning curves can be interpreted as place-cell tiling.
    Used for feature-family analysis on Gemma models only (§3.2, Table 2, Figure 2).
  • domain assumption Subspace activation patching of top-k PCA directions, compared with random directions, tests informational concentration of the ordinal variable.
    §3.3; authors cite Zhang & Nanda (2024) best practices and Makelov et al. (2024) caveats.
invented entities (2)
  • Ordinality score (|ρ_Spearman(v, PC1_Q)|) independent evidence
    purpose: Distinguishes coherent order-preserving twisting from incoherent rearrangement of the manifold in Q-space.
    New diagnostic introduced in §3.3; defined from data, not postulated as a physical object. Independent evidence is the score itself on held-out heads/tasks.
  • Local-computability taxonomy of ordinal representations independent evidence
    purpose: Organizes when 1D place-cell manifolds appear versus higher-D or incoherent geometry.
    Descriptive organizing claim derived from the four tasks; not an ontological invention. Falsifiable by additional ordinal tasks.

pith-pipeline@v1.1.0-grok45 · 15790 in / 3682 out tokens · 30672 ms · 2026-07-11T21:12:59.410671+00:00 · methodology

0 comments
read the original abstract

Recent work showed that language models represent character counts on curved 1D manifolds, with attention heads performing geometric transformations to enable computation. We test whether this generalizes across four ordinal tasks (bracket depth, indentation, table position, numeric magnitude) in Gemma-2-2B, Gemma-2-9B, and Qwen3-4B. We find that 1D manifolds with place-cell feature tiling emerge for tasks where the ordinal variable is locally computable from token identity, while tasks requiring cross-position integration or semantic extraction produce higher-dimensional or incoherent representations. Geometric computation is architecture-dependent: Qwen3-4B shows substantially stronger twisting than Gemma models for indentation, and its twisters preserve ordinal order, unlike its numeric twisters. Activation patching confirms that the identified manifold subspaces concentrate task-relevant information, with manifold-direction ablation causing dramatically larger probe accuracy drops than random-direction controls.

Figures

Figures reproduced from arXiv: 2607.04167 by Saksham Bassi, Sharvi Tomar.

Figure 1
Figure 1. Figure 1: Pairwise cosine similarity of bracket depth mean acti￾vations (Gemma-2-2B, L8). Depth 0 occupies a distinct direction; depths 2–10 are nearly identical, confirming a 1D manifold with one outlier at depth 0. 6–12 out of 20 features being selective, zero are monotonic at any layer in either model. Tuning curves are erratic, and the SAE cannot decompose numeric magnitude into an or￾derly feature family. This … view at source ↗
Figure 2
Figure 2. Figure 2: SAE feature tuning curves - brackets L8 (Gemma-2-2B). Three archetypes of place-cell tiling: a sharp depth=0 detector (F9213), a monotonically increasing feature (F302), and a feature peaking at maximum depth (F13542). Activations normalized to [0, 1] per feature [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Twist score heatmaps (layer × head). Left: Brackets in Gemma-2-2B (max twist = 0.033) – uniformly low, confirming pre￾organized manifold. Right: Indentation in Qwen3-4B (max twist = 0.803) – concentrated in layers 5–20, with multiple ordinal-preserving twists. Note the 25× difference in scale. 0 10 20 30 40 Layer 65 70 75 80 85 90 95 100 PC1 variance explained (%) Brackets 0 10 20 30 40 Layer 65 70 75 80 8… view at source ↗
Figure 4
Figure 4. Figure 4: PC1 variance explained across layers. For brackets and table columns, PC1 captures > 90% of variance (dashed line) in all three models, consistent with 1D centroid manifold structure. For indentation and table rows, Qwen3-4B collapses to a 1D manifold at mid-layers (step-jump to ∼100%) while Gemma models remain at 45–65%, revealing architecture-dependent manifold organization. able is locally computable fr… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

31 extracted references · 4 linked inside Pith

  1. [1]

    arXiv preprint arXiv:2601.04480 , year=

    When Models Manipulate Manifolds: The Geometry of a Counting Task , author=. arXiv preprint arXiv:2601.04480 , year=

    arXiv

  2. [2]

    2026 , eprint=

    Transformers Learn Factored Representations , author=. 2026 , eprint=

    2026

  3. [3]

    2026 , eprint=

    The Information Geometry of Softmax: Probing and Steering , author=. 2026 , eprint=

    2026

  4. [4]

    The Fourteenth International Conference on Learning Representations , year=

    Priors in Time: Missing Inductive Biases for Language Model Interpretability , author=. The Fourteenth International Conference on Learning Representations , year=

  5. [5]

    Philosophical Studies , year=

    Mechanistic Indicators of Understanding in Large Language Models , author=. Philosophical Studies , year=

  6. [6]

    arXiv preprint arXiv:2602.07488 , year=

    Deriving Neural Scaling Laws from the Statistics of Natural Language , author=. arXiv preprint arXiv:2602.07488 , year=

    Pith/arXiv arXiv

  7. [7]

    Transformer Circuits Thread , year=

    Toy Models of Superposition , author=. Transformer Circuits Thread , year=

  8. [8]

    Distill , year=

    Zoom In: An Introduction to Circuits , author=. Distill , year=

  9. [9]

    Transformer Circuits Thread , year=

    Towards Monosemanticity: Decomposing Language Models With Dictionary Learning , author=. Transformer Circuits Thread , year=

  10. [10]

    Daniel and Sumers, Theodore R

    Templeton, Adly and Conerly, Tom and Marcus, Jonathan and Lindsey, Jack and Bricken, Trenton and Chen, Brian and Pearce, Adam and Citro, Craig and Ameisen, Emmanuel and Jones, Andy and Cunningham, Hoagy and Turner, Nicholas L and McDougall, Callum and MacDiarmid, Monte and Freeman, C. Daniel and Sumers, Theodore R. and Rees, Edward and Batson, Joshua and ...

    2024

  11. [11]

    The Eleventh International Conference on Learning Representations , year=

    Progress Measures for Grokking via Mechanistic Interpretability , author=. The Eleventh International Conference on Learning Representations , year=

  12. [12]

    Advances in Neural Information Processing Systems , volume=

    Towards Automated Circuit Discovery for Mechanistic Interpretability , author=. Advances in Neural Information Processing Systems , volume=

  13. [13]

    The Twelfth International Conference on Learning Representations , year=

    Sparse Autoencoders Find Highly Interpretable Features in Language Models , author=. The Twelfth International Conference on Learning Representations , year=

  14. [14]

    Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on

    Lieberum, Tom and others , journal=. Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on

  15. [15]

    Interpreting

    nostalgebraist , journal=. Interpreting

  16. [16]

    arXiv preprint arXiv:2303.08112 , year=

    Eliciting Latent Predictions from Transformers with the Tuned Lens , author=. arXiv preprint arXiv:2303.08112 , year=

    Pith/arXiv arXiv

  17. [17]

    Annual Review of Neuroscience , volume=

    Place Cells, Grid Cells, and the Brain's Spatial Representation System , author=. Annual Review of Neuroscience , volume=

  18. [18]

    The Thirty-eighth Annual Conference on Neural Information Processing Systems , year=

    Transformers Represent Belief State Geometry in their Residual Stream , author=. The Thirty-eighth Annual Conference on Neural Information Processing Systems , year=

  19. [19]

    Causal Representation Learning Workshop at NeurIPS 2023 , year=

    The Linear Representation Hypothesis and the Geometry of Large Language Models , author=. Causal Representation Learning Workshop at NeurIPS 2023 , year=

    2023

  20. [20]

    Is This the Subspace You Are Looking for?

    Makelov, Aleksandar and Lange, Georg and Geiger, Atticus and Nanda, Neel , booktitle=. Is This the Subspace You Are Looking for?. 2024 , url=

    2024

  21. [21]

    The Twelfth International Conference on Learning Representations , year=

    Towards Best Practices of Activation Patching in Language Models: Metrics and Methods , author=. The Twelfth International Conference on Learning Representations , year=

  22. [22]

    2026 , url=

    The Origins of Representation Manifolds in Large Language Models , author=. 2026 , url=

    2026

  23. [23]

    The Thirteenth International Conference on Learning Representations , year=

    Not All Language Model Features Are One-Dimensionally Linear , author=. The Thirteenth International Conference on Learning Representations , year=

  24. [24]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages=

    Monotonic Representation of Numeric Attributes in Language Models , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages=. 2024 , address=

    2024

  25. [25]

    The Twelfth International Conference on Learning Representations , year=

    Language Models Represent Space and Time , author=. The Twelfth International Conference on Learning Representations , year=

  26. [26]

    First Conference on Language Modeling , year=

    The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets , author=. First Conference on Language Modeling , year=

  27. [27]

    Proceedings of the 2019 Conference of the North

    A Structural Probe for Finding Syntax in Word Representations , author=. Proceedings of the 2019 Conference of the North. 2019 , address=

    2019

  28. [28]

    NeurIPS 2025 Workshop on Mechanistic Interpretability , year=

    Understanding Sparse Autoencoder Scaling in the Presence of Feature Manifolds , author=. NeurIPS 2025 Workshop on Mechanistic Interpretability , year=

    2025

  29. [29]

    The Neural Basis of the

    Dehaene, Stanislas , journal=. The Neural Basis of the

  30. [30]

    arXiv preprint arXiv:2408.00118 , year=

    Gemma 2: Improving Open Language Models at a Practical Size , author=. arXiv preprint arXiv:2408.00118 , year=

    Pith/arXiv arXiv

  31. [31]

    arXiv preprint arXiv:2505.09388 , year=

    Qwen3 Technical Report , author=. arXiv preprint arXiv:2505.09388 , year=

    Pith/arXiv arXiv