Pith. sign in

REVIEW 2 major objections 3 minor 56 references

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

Curved pooling beats flat averaging on BERT embeddings

2026-07-09 21:00 UTC pith:6CERFW2N

load-bearing objection Riemannian SPD aggregation of BERT token embeddings beats Euclidean pooling, but the gain may be a dimensionality artifact rather than a geometric one. the 2 major comments →

arxiv 2607.07047 v1 pith:6CERFW2N submitted 2026-07-08 cs.CL cs.AI

Riemannian Geometry for Pre-trained Language Model Embeddings

Szczepan Konior , Alexandre Quemy , Przemys{\l}aw Klocek , Gr\'egoire Cattan , Bart{\l}omiej Sobieski This is my paper
classification cs.CL cs.AI
keywords datasetsembeddingsencodermeanpoolingriemannianstructurethree
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper asks whether sentence-level classification signal in pre-trained language model embeddings lives in their local Riemannian geometry rather than in their flat Euclidean coordinates. To test this, the authors extract a per-token pullback metric—a symmetric positive definite (SPD) matrix capturing how a learned encoder locally stretches and compresses the embedding space at each token—by computing the analytical Jacobian of the encoder at that token's embedding. They aggregate these per-token SPD matrices into a single sentence-level descriptor using the Fréchet mean on the SPD manifold, a procedure they call Riemannian Mean Pooling (RMP), then classify in the tangent space at the population mean. On three signal-bearing datasets (CoLA, CREAK, RTE), RMP outperforms standard Euclidean mean pooling under a controlled comparison on identical BERT embeddings, while on FEVER-Symmetric—a dataset constructed to remove annotation artifacts—RMP correctly stays at chance. The central discovery comes from the ablations: a randomly initialised, untrained encoder combined with Fréchet aggregation already beats Euclidean pooling on two of three signal-bearing datasets. This localises most of the gain to the geometric aggregation step itself—the Fréchet mean on the SPD manifold—rather than to any learned manifold structure in the encoder. The trained encoder contributes additional signal only on CREAK, the most knowledge-heavy dataset, suggesting that learned geometric structure matters specifically where classification depends on world knowledge rather than local linguistic form.

Core claim

The Fréchet mean of per-token pullback metrics on the SPD manifold extracts classification signal from BERT embeddings that Euclidean mean pooling discards, and this gain is attributable primarily to the geometric aggregation operation itself rather than to learned manifold structure in the encoder. A randomly initialised encoder's Jacobian already produces metrics that, when aggregated via the Fréchet mean, outperform Euclidean pooling on two of three signal-bearing datasets, because the pullback metric captures local anisotropy of the token's position under any smooth nonlinear transformation—even a random one—and the SPD aggregation preserves this position-dependent structure that Euclide

What carries the argument

The pipeline has two stages. In the training stage, a small MLP encoder maps 768-dimensional BERT token embeddings to a 64-dimensional latent code, trained via Intrinsic Green's Learning (IGL)—an inverse-PDE formulation where a closed-form Green's-function kernel readout replaces the decoder, forcing the encoder to find coordinates aligned with the data's smooth structure. In the inference stage, the trained encoder's analytical Jacobian J(v) = ∂Ψ/∂v at each token embedding v yields a pullback metric g(v) = (JJ^T + εI)^{-1}, an SPD matrix capturing local stretching/compression of the embedding space. Per-token SPD matrices are aggregated via the Fréchet mean—the point on the SPD manifold最小化总

Load-bearing premise

The pullback metric g(v) = (JJ^T + εI)^{-1} is assumed to encode meaningful local geometric structure of the embedding space that carries classification signal. But the random-encoder ablation shows that even a random MLP's Jacobian produces metrics that beat Euclidean pooling, which means the 'geometric signal' may simply be a deterministic nonlinear transformation of token position whose SPD aggregation happens to be a useful feature—raising the question of whether this isR

What would settle it

If a non-learned, fixed nonlinear transformation (e.g., a fixed random matrix followed by a fixed nonlinearity) combined with Fréchet aggregation produces the same gains as the trained-encoder RMP on CoLA, CREAK, and RTE, then the contribution of learned manifold structure is zero and the method reduces to a deterministic feature-engineering step with no dependence on the data manifold's actual geometry. Conversely, if RMP fails to outperform Euclidean pooling on datasets beyond these three, the geometric advantage may be an artifact of specific dataset properties rather than a general signal.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If geometric aggregation rather than learned manifold structure drives the gain, then any differentiable encoder—not just IGL—could serve as a metric source, and the specific training objective may be largely irrelevant for most tasks.
  • The finding that random encoders suffice suggests the signal lies in the nonlinear transformation of token position itself, opening the question of whether simpler deterministic nonlinear maps (fixed random features, kernel functions) could replace the encoder entirely.
  • The trained encoder's contribution specifically on CREAK (knowledge-heavy) versus CoLA/RTE (linguistic structure) suggests a division of labor: geometric aggregation captures local linguistic form, while learned coordinates capture world-knowledge content.
  • If the pullback metric filters content features by construction (as argued for FEVER-Symmetric), then RMP could serve as a debiasing tool: a representation that preserves linguistic structure while discarding specific knowledge content.
  • Extending to other architectures (decoder-only LLMs, different layers) would test whether the geometric-aggregation gain is specific to BERT layer 9 or reflects a general property of transformer token spaces.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The random-encoder result raises the possibility that RMP is not probing 'the Riemannian geometry of the data manifold' but rather exploiting the fact that any smooth nonlinear transformation of a high-dimensional vector produces a position-dependent SPD matrix whose Fréchet mean is a more informative summary than the arithmetic mean. The 'geometric signal' may be a deterministic feature-engineeri
  • If the pullback metric of a random encoder already encodes token position through local Jacobian anisotropy, then the method may be closely related to random-feature methods (random kitchen sinks, extreme learning machines) where a nonlinear projection into a richer representation space improves separability—reinterpreted through the lens of SPD-manifold aggregation.
  • The FEVER-Symmetric result where the trained encoder + Euclidean aggregation detects residual signal (0.553 AUC) but the full geometric pipeline stays at chance suggests the pullback metric acts as a low-pass filter on content features, which could be either a bug (information loss) or a feature (built-in debiasing) depending on the application.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 3 minor

Summary. The paper proposes Riemannian Mean Pooling (RMP), which aggregates per-token pullback metrics extracted from a learned encoder's Jacobian via the Fréchet mean on the SPD manifold. The method is evaluated on four binary classification datasets (CoLA, CREAK, RTE, and FEVER-Symmetric as a negative control) using BERT-base layer 9 embeddings. The authors find that RMP outperforms Euclidean mean pooling on the three signal-bearing datasets while remaining at chance on the negative control. Ablations using randomly initialized encoders suggest that the geometric aggregation, rather than learned manifold structure, accounts for most of the gain. The experimental design is methodologically careful, featuring controlled comparisons on identical embeddings, multi-seed cross-validation, permutation testing, and a negative control.

Significance. The paper addresses a well-motivated question about the geometric structure of PLM embeddings. Its primary strength is the rigorous experimental design: the controlled comparison isolates the aggregation step, the FEVER-Symmetric negative control tests for artifact exploitation, and the random-encoder ablation is a genuinely informative test of whether learned manifold structure is necessary. The finding that random encoders suffice for the geometric gain on two of three datasets is a valuable and honest negative result that refines the claim from 'embeddings lie on a learned manifold' to 'embeddings exhibit locally manifold-like structure.' However, the central claim is currently confounded by a dimensionality mismatch between the RMP pipeline and the baselines, which must be addressed before the contribution can be considered fully sound.

major comments (2)
  1. §3.2, §3.5, Table 1: The comparison between RMP and the Linear Probe baseline is confounded by a large difference in feature dimensionality. The Linear Probe classifies on 768-dimensional mean-pooled vectors, while RMP classifies on tangent-space projections of 64×64 SPD matrices, yielding d(d+1)/2 = 2080 features. With logistic regression and no regularization (as stated for the Linear Probe in §3.2), higher dimensionality alone can improve classification on these small datasets (CoLA: 3000, CREAK: 3000, RTE: 2490). The random-encoder ablation results (§5.3) are also consistent with this confound: any random nonlinear projection to 64-dim, converted to 2080 SPD features, could outperform 768-dim Euclidean averaging purely from feature expansion. The paper frames the random-encoder result as evidence that 'geometric aggregation' carries the signal, but it does not distinguish between (a)
  2. §5.3, §6.1: To support the claim that the gain comes from Riemannian/Fréchet structure rather than second-order statistics in a higher-dimensional space, a dimensionality-controlled baseline is needed. Specifically, a Euclidean baseline that computes the same 64×64 outer-product matrices (e.g., v_i v_i^T after random projection), averages them arithmetically (not Fréchet), vectorizes the upper triangle to 2080 features, and classifies with the same logistic regression would isolate the contribution of the Riemannian operations from the contribution of dimensionality expansion. Without this control, the central claim that 'geometric aggregation extracts signal that flat-Euclidean pooling discards' is not fully established.
minor comments (3)
  1. §3.2: The Linear Probe is stated to use 'no regularisation' with the LBFGS solver. If the dimensionality-controlled baseline suggested above is added, regularization should be matched carefully across all compared methods to ensure a fair comparison.
  2. Figure 1: The schematic is helpful, but the distinction between the 'Product Green's kernel' and 'Variable projection' boxes in the training path could be clearer for readers unfamiliar with IGL.
  3. §3.4, Eq. (6): The two-stage regularization (epsilon=1e-6 and lambda=1e-2) is mentioned, but the sensitivity to lambda is noted as unverified on signal-bearing datasets (§6.2). A brief note on the stability of the results to this choice would strengthen the manuscript.

Circularity Check

0 steps flagged

No significant circularity: the derivation chain is self-contained, and the one self-citation (IGL) is explicitly ablated away as non-load-bearing.

full rationale

The paper's central claim—that Riemannian Mean Pooling (Fréchet aggregation of per-token pullback metrics on the SPD manifold) outperforms Euclidean mean pooling—is tested empirically against external baselines (Linear Probe, CLS Token Aggregation) on four standard NLP benchmarks. The mathematical construction of the pullback metric (Eqs. 4–6: g(v) = (JJ^T + εI)^{-1}) is a standard differential-geometric operation independent of any author's prior work. The one self-citation is Intrinsic Green's Learning (Quemy, 2026), authored by co-author Quemy, used as the encoder framework. However, this citation is not load-bearing for the central claim: the paper explicitly ablates whether the trained IGL encoder matters by comparing against randomly initialized encoders (Frozen Random IGL, Random Projection + SiLU) and finds that 'a randomly initialised encoder combined with Fréchet aggregation already beats Euclidean pooling on two of the three signal-bearing datasets' (§5.3). The pullback metric extraction is defined purely in terms of the Jacobian of any differentiable encoder (§3.4: 'the geometric pipeline downstream of g(v) is well-defined for any differentiable encoder'), so the IGL self-citation serves as one instantiation rather than a logical dependency. No 'prediction' or 'first-principles result' reduces to its inputs by construction. The results are externally falsifiable against the reported baselines. The reader's concern about dimensionality confounding (2080 SPD features vs. 768 Euclidean) is a correctness risk, not a circularity issue—it does not involve any step where an output is defined in terms of itself or a fitted parameter is renamed as a prediction.

Axiom & Free-Parameter Ledger

7 free parameters · 4 axioms · 0 invented entities

The paper introduces no new mathematical entities or physical objects. It composes existing components (IGL encoder, pullback metric, Fréchet mean, tangent-space classifier) into a new pipeline. The free parameters are mostly standard regularization and architectural choices. The key domain assumptions are that BERT embeddings carry geometric structure and that the pullback metric captures it; the random-encoder ablation partially undermines the second assumption by showing the structure may be generic rather than learned.

free parameters (7)
  • ε (regularization for SPD inversion) = 1e-6
    Chosen to ensure strict positive-definiteness of the pullback metric; stated in Eq. 6.
  • λ (additive regularization for numerical stability) = 1e-2
    Added to guarantee numerical stability across input tokens; stated in Eq. 6.
  • d_max (encoder latent dimension) = 64
    Encoder output dimensionality; stated in Section 3.3.
  • K (number of anchors) = 128
    Number of learned anchor points for Green's function kernel; stated in Section 3.3.
  • Number of Gaussian scales per factor = 4
    Number of Gaussian basis functions per kernel factor; stated in Section 3.3.
  • BERT layer choice = 9
    Ninth hidden layer of BERT-base-uncased chosen for token embedding extraction; motivated by prior work on layer semantics.
  • Ridge regression regularization λ (VP) = 1e-3
    Regularization for variable-projection ridge regression; stated in Eq. 11.
axioms (4)
  • domain assumption Token embeddings from BERT layer 9 carry sentence-level classification signal accessible via local geometric structure.
    The entire pipeline depends on this; the layer choice is motivated by Rogers et al. 2020 but the geometric-signal assumption is unstated.
  • domain assumption The pullback metric g(v) = (JJ^T + εI)^{-1} captures meaningful local geometric structure of the embedding manifold.
    Invoked in Section 3.4; the paper acknowledges this is a local, not global, manifold assumption following Robinson et al. 2025.
  • standard math The affine-invariant Riemannian distance on the SPD manifold is the appropriate metric for aggregating per-token pullback metrics.
    Standard choice from BCI literature (Förstner et al. 2000); used in Eq. 8.
  • domain assumption IGL's closed-form kernel readout constrains the encoder to coordinates aligned with the data manifold's smooth structure.
    Stated in Section 3.3; this motivates IGL over other encoders, though the ablation shows the specific objective is largely transparent.

pith-pipeline@v1.1.0-glm · 20420 in / 3220 out tokens · 295894 ms · 2026-07-09T21:00:07.914158+00:00 · methodology

0 comments
read the original abstract

Understanding the geometric structure of pre-trained language model embeddings matters for interpretability and safety. We ask whether sentence-level classification signal lives in the Riemannian geometry of contextual token embeddings, and probe it by extracting per-token pullback metrics from a learned encoder's analytical Jacobian and aggregating them with the Fr\'echet mean on the symmetric positive definite (SPD) manifold; we call this procedure Riemannian Mean Pooling (RMP). Across three datasets with non-trivial linguistic structure (CoLA, CREAK, RTE), RMP outperforms Euclidean mean pooling, while on FEVER-Symmetric, a benchmark constructed to remove annotation-driven lexical artifacts, the method correctly stays at chance. Ablations show that a randomly initialised encoder combined with Fr\'echet aggregation already beats Euclidean pooling on two of the three signal-bearing datasets, localising the source of the gain to the geometric aggregation rather than to learned manifold structure; the trained encoder contributes additional signal specifically on CREAK, the most knowledge-heavy of the three signal-bearing datasets.

Figures

Figures reproduced from arXiv: 2607.07047 by Alexandre Quemy, Bart{\l}omiej Sobieski, Gr\'egoire Cattan, Przemys{\l}aw Klocek, Szczepan Konior.

Figure 1
Figure 1. Figure 1: Overview of the pipeline. Top row (training): the encoder Ψθ is trained to reconstruct token embeddings (early stopping with patience=50, δ = 2 × 10−5 ), mapping them to a low-dimensional latent code from which a product Green’s kernel with variable projection recovers the input; the reconstruction loss flows back to the encoder (dashed arrow). Bottom row (inference): once trained, the encoder’s Jacobian i… view at source ↗
Figure 2
Figure 2. Figure 2: Statistical comparison between Linear Probe and [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Riemannian Mean Pooling vs Encoder-Only: forest [PITH_FULL_IMAGE:figures/full_fig_p012_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: AUC bar comparison: Riemannian Mean Pooling [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Riemannian Mean Pooling vs Frozen Random IGL: [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 3
Figure 3. Figure 3: Detailed architecture of token-level IGL for sentence classification via Riemannian mean pooling. [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figure 7
Figure 7. Figure 7: AUC bar comparison: Riemannian Mean Pooling [PITH_FULL_IMAGE:figures/full_fig_p013_7.png] view at source ↗
Figure 9
Figure 9. Figure 9: AUC bar comparison: Riemannian Mean Pooling [PITH_FULL_IMAGE:figures/full_fig_p014_9.png] view at source ↗
Figure 11
Figure 11. Figure 11: Multi-seed accuracy comparison across datasets. [PITH_FULL_IMAGE:figures/full_fig_p014_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Multi-seed F1-score comparison across datasets. [PITH_FULL_IMAGE:figures/full_fig_p015_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Statistical comparison between CLS Token Ag [PITH_FULL_IMAGE:figures/full_fig_p015_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Training (blue) and validation (red) reconstruction [PITH_FULL_IMAGE:figures/full_fig_p015_14.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

56 extracted references · 56 canonical work pages · 31 internal anchors

  1. [1]

    Guillaume Alain and Yoshua Bengio. 2017. https://arxiv.org/abs/1610.01644 Understanding intermediate layers using linear classifier probes . In International Conference on Learning Representations (ICLR) Workshop Track

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  2. [2]

    Anton Andreev, Gregoire Cattan, and Marco Congedo. 2025. https://www.mdpi.com/1424-8220/25/7/2305 The Riemannian Means Field Classifier for EEG - Based BCI Data . Sensors, 25(7):2305

    work page 2025

  3. [3]

    Bruno Aristimunha, Igor Carrara, Pierre Guetschel, Sara Sedlar, Pedro Rodrigues, Jan Sosulski, Divyesh Narayanan, Erik Bjareholt, Barth\'el\'emy Quentin, Robin Tibor Schirrmeister, Emmanuel Kalunga, Ludovic Darmet, Cattan Gregoire, Ali Abdul Hussain, Ramiro Gatti, Vladislav Goncharenko, Jordy Thielen, Thomas Moreau, Yannick Roy, and 3 others. 2023. https:...

    work page doi:10.5281/zenodo.10034224 2023

  4. [4]

    Georgios Arvanitidis, Lars Kai Hansen, and S ren Hauberg. 2018. https://arxiv.org/abs/1710.11379 Latent space oddity: On the curvature of deep generative models . In International Conference on Learning Representations (ICLR)

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  5. [5]

    Alexandre Barachant, Quentin Barthélemy, Jean-Rémi King, Alexandre Gramfort, Sylvain Chevallier, Pedro L. C. Rodrigues, Emanuele Olivetti, Vladislav Goncharenko, Gabriel Wagner vom Berg, Ghiles Reguig, Arthur Lebeurrier, Erik Bjäreholt, Maria Sayu Yamamoto, Pierre Clisson, Marie-Constance Corsi, Igor Carrara, Apolline Mellot, Bruna Junqueira Lopes, Brent ...

    work page doi:10.5281/zenodo.593816 2026

  6. [6]

    Alexandre Barachant, Stéphane Bonnet, Marco Congedo, and Christian Jutten. 2012. https://doi.org/10.1109/TBME.2011.2172210 Multiclass brain-computer interface classification by Riemannian geometry . IEEE transactions on bio-medical engineering, 59(4):920--928

    work page doi:10.1109/tbme.2011.2172210 2012

  7. [7]

    Matthew Brand. 2003. Charting a manifold. In Advances in Neural Information Processing Systems (NeurIPS)

    work page 2003

  8. [8]

    Julian Brandon, Angus Chadwick, and Arthur Pellegrino. 2025. https://arxiv.org/abs/2512.00196 Emergent R iemannian geometry over learning discrete computations on continuous manifolds . arXiv preprint arXiv:2512.00196

    work page arXiv 2025

  9. [9]

    Daniel Brooks, Olivier Schwander, Fr\'ed\'eric Barbaresco, Jean-Yves Schneider, and Matthieu Cord. 2019. https://arxiv.org/abs/1909.02414 Riemannian batch normalization for SPD neural networks . In Advances in Neural Information Processing Systems (NeurIPS)

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  10. [10]

    Ziheng Chen, Yue Song, Rui Wang, Xiaojun Wu, and Nicu Sebe. 2024. https://arxiv.org/abs/2409.19433 RMLR : Extending multinomial logistic regression into general geometries . In Advances in Neural Information Processing Systems (NeurIPS)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  11. [11]

    Emily Cheng, Diego Doimo, Corentin Kervadec, Iuri Macocco, Jade Yu, Alessandro Laio, and Marco Baroni. 2025. https://arxiv.org/abs/2405.15471 Emergence of a high-dimensional abstraction phase in language transformers . In International Conference on Learning Representations (ICLR)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  12. [12]

    Sylvain Chevallier, Igor Carrara, Bruno Aristimunha, Pierre Guetschel, Sara Sedlar, Bruna Lopes, Sebastien Velut, Salim Khazem, and Thomas Moreau. 2024. https://doi.org/10.48550/arXiv.2404.15319 The largest EEG -based BCI reproducibility study for open science: the MOABB benchmark . arXiv preprint. ArXiv:2404.15319

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2404.15319 2024

  13. [13]

    Sungjun Cho, Seunghyuk Cho, Sungwoo Park, Hankook Lee, Honglak Lee, and Moontae Lee. 2023. https://arxiv.org/abs/2309.04082 Curve your attention: Mixed-curvature transformers for graph representation learning . In Proceedings of the AAAI Conference on Artificial Intelligence

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  14. [14]

    Marco Congedo, Alexandre Barachant, and Rajendra Bhatia. 2017. https://doi.org/10.1080/2326263X.2017.1297192 Riemannian geometry for EEG -based brain-computer interfaces; a primer and a review . Brain-Computer Interfaces, 4(3):155--174

    work page doi:10.1080/2326263x.2017.1297192 2017

  15. [15]

    Hamed Damirchi, Ignacio Meza De la Jara, Ehsan Abbasnejad, Afshar Shamsi, Zhen Zhang, and Javen Shi. 2026. https://arxiv.org/abs/2603.01326 Truth as a trajectory: What internal representations reveal about large language model reasoning . Preprint, arXiv:2603.01326

    work page arXiv 2026

  16. [16]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://arxiv.org/abs/1810.04805 BERT : Pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT)

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  17. [17]

    Riccardo Di Sipio, Jairo Diaz-Rodriguez, and Luis Serrano. 2025. https://arxiv.org/abs/2511.03060 The curved spacetime of transformer architectures . arXiv preprint arXiv:2511.03060

    work page arXiv 2025

  18. [18]

    Willem Diepeveen, Georgios Batzolis, Zakhar Shumaylov, and Carola-Bibiane Sch \"o nlieb. 2024. https://arxiv.org/abs/2410.01950 Score-based pullback R iemannian geometry: Extracting the data manifold geometry using anisotropic flows . arXiv preprint arXiv:2410.01950

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  19. [19]

    Kawin Ethayarajh. 2019. https://arxiv.org/abs/1909.00512 How contextual are contextualized word representations? C omparing the geometry of BERT , ELM o, and GPT-2 embeddings . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP)

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  20. [20]

    P Fletcher, Conglin Lu, Stephen Pizer, and Sarang Joshi. 2004. https://doi.org/10.1109/TMI.2004.831793 Principal geodesic analysis for the study of nonlinear statistics of shape . Medical Imaging, IEEE Transactions on, 23:995 -- 1005

    work page doi:10.1109/tmi.2004.831793 2004

  21. [21]

    Wolfgang F \"o rstner, Boudewijn Moonen, and Carl Gauss. 2000. https://doi.org/10.1007/978-3-662-05296-9_31 A metric for covariance matrices

    work page doi:10.1007/978-3-662-05296-9_31 2000

  22. [22]

    Xiaofang Gao and Jiye Liang. 2011. https://doi.org/10.1016/j.patrec.2010.08.005 The dynamical neighborhood selection based on the sampling density and manifold curvature for isometric data embedding . Pattern Recognition Letters, 32(2):202--209

    work page doi:10.1016/j.patrec.2010.08.005 2011

  23. [23]

    Wes Gurnee and Max Tegmark. 2024. https://arxiv.org/abs/2310.02207 Language models represent space and time . In International Conference on Learning Representations (ICLR)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  24. [24]

    Michael Hauser and Asok Ray. 2017. https://proceedings.neurips.cc/paper_files/paper/2017/file/0ebcc77dc72360d0eb8e9504c78d38bd-Paper.pdf Principles of R iemannian geometry in neural networks . In Advances in Neural Information Processing Systems (NeurIPS)

    work page 2017

  25. [25]

    Saahith Janapati and Yangfeng Ji. 2025. https://doi.org/10.18653/v1/2025.repl4nlp-1.5 A comparative study of learning paradigms in large language models via intrinsic dimension . In Proceedings of the 10th Workshop on Representation Learning for NLP (RepL4NLP-2025), pages 59--86, Albuquerque, NM. Association for Computational Linguistics

    work page doi:10.18653/v1/2025.repl4nlp-1.5 2025

  26. [26]

    Zhongping Ji. 2025. https://arxiv.org/abs/2506.07405 RiemannFormer : A framework for attention in curved spaces . arXiv preprint arXiv:2506.07405

    work page arXiv 2025

  27. [27]

    Ce Ju, Reinmar Kobler, Antoine Collas, Motoaki Kawanabe, Cuntai Guan, and Bertrand Thirion. 2024. https://arxiv.org/abs/2504.18882 SPD matrix learning for neuroimaging analysis: Perspectives, methods, and challenges . arXiv preprint arXiv:2504.18882

    work page arXiv 2024

  28. [28]

    Takuya Kataiwa, Cho Hakaze, and Tetsushi Ohki. 2025. https://arxiv.org/abs/2503.02142 Measuring intrinsic dimension of token embeddings . arXiv preprint arXiv:2503.02142

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  29. [29]

    Aditya Kusupati, Gantavya Bhatt, Aniket Rege, Matthew Wallingford, Aditya Sinha, Vivek Ramanujan, William Howard-Snyder, Kaifeng Chen, Sham Kakade, Prateek Jain, and Ali Farhadi. 2022. https://proceedings.neurips.cc/paper_files/paper/2022/file/c32319f4868da7613d78af9993100e42-Paper-Conference.pdf Matryoshka representation learning . In Advances in Neural ...

    work page 2022

  30. [30]

    Yangyang Li. 2017. https://doi.org/10.48550/arXiv.1706.07167 Curvature-aware Manifold Learning . arXiv preprint. ArXiv:1706.07167 [cs.LG]

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.1706.07167 2017

  31. [31]

    Aaron Lou, Isay Katsman, Qingxuan Jiang, Serge Belongie, Ser-Nam Lim, and Christopher De Sa. 2020. https://arxiv.org/abs/2003.00335 Differentiating through the F r\'echet mean . In Proceedings of the 37th International Conference on Machine Learning (ICML)

    work page internal anchor Pith review Pith/arXiv arXiv 2020

  32. [32]

    Rob Manson. 2025. https://arxiv.org/abs/2507.21107 Curved inference: Concern-sensitive geometry in large language model residual streams . Preprint, arXiv:2507.21107

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  33. [33]

    Eric Maris and Robert Oostenveld. 2007. https://doi.org/10.1016/j.jneumeth.2007.03.024 Nonparametric statistical testing of EEG - and MEG -data . Journal of Neuroscience Methods, 164(1):177--190

    work page doi:10.1016/j.jneumeth.2007.03.024 2007

  34. [34]

    Samuel Marks and Max Tegmark. 2024. https://arxiv.org/abs/2310.06824 The geometry of truth: Emergent linear structure in large language model representations of true/false datasets . In Conference on Language Modeling (COLM)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  35. [35]

    Leland McInnes, John Healy, and James Melville. 2018. https://arxiv.org/abs/1802.03426 UMAP : Uniform manifold approximation and projection for dimension reduction . arXiv preprint arXiv:1802.03426

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  36. [36]

    Amir Hameed Mir. 2025. https://arxiv.org/abs/2510.04933 The geometry of truth: Layer-wise semantic dynamics for hallucination detection . arXiv preprint arXiv:2510.04933

    work page arXiv 2025

  37. [37]

    Gernot Müller-Putz, Reinhold Scherer, Clemens Brunner, Robert Leeb, and Gert Pfurtscheller. 2008. Better than Random ? A closer look on BCI results. International Journal of Bioelektromagnetism, 10:52--55

    work page 2008

  38. [38]

    Maximilian Nickel and Douwe Kiela. 2017. https://arxiv.org/abs/1705.08039 P oincar\'e embeddings for learning hierarchical representations . In Advances in Neural Information Processing Systems (NeurIPS)

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  39. [39]

    Maximilian Nickel and Douwe Kiela. 2018. https://arxiv.org/abs/1806.03417 Learning continuous hierarchies in the L orentz model of hyperbolic geometry . In Proceedings of the 35th International Conference on Machine Learning (ICML)

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  40. [40]

    CREAK: A Dataset for Commonsense Reasoning over Entity Knowledge

    Yasumasa Onoe, Michael J.Q. Zhang, Eunsol Choi, and Greg Durrett. 2021. https://arxiv.org/abs/2109.01653 Creak: A dataset for commonsense reasoning over entity knowledge . OpenReview

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  41. [41]

    Kiho Park, Yo Joong Choe, Yibo Jiang, and Victor Veitch. 2025. https://arxiv.org/abs/2406.01506 The geometry of categorical and hierarchical concepts in large language models . In International Conference on Learning Representations (ICLR)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  42. [42]

    Kiho Park, Yo Joong Choe, and Victor Veitch. 2024. https://arxiv.org/abs/2311.03658 The linear representation hypothesis and the geometry of large language models . In Proceedings of the 41st International Conference on Machine Learning (ICML)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  43. [43]

    Pedregosa, G

    F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. https://jmlr.org/papers/v12/pedregosa11a.html Scikit-learn: Machine learning in P ython . Journal of Machine Learning Research, 12:2825--2830

    work page 2011

  44. [44]

    Alexandre Quemy. 2026. https://openreview.net/forum?id=Y6RpdS98l8 Intrinsic Green 's Learning : Supervised Learning on Manifolds via Inverse PDE . In AI & PDE Workshop at ICLR

    work page 2026

  45. [45]

    Nils Reimers and Iryna Gurevych. 2019. https://arxiv.org/abs/1908.10084 Sentence-BERT : Sentence embeddings using siamese BERT -networks . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP)

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  46. [46]

    Michael Robinson, Sourya Dey, and Tony Chiang. 2025. https://arxiv.org/abs/2504.01002 Token embeddings violate the manifold hypothesis . In Advances in Neural Information Processing Systems (NeurIPS)

    work page arXiv 2025

  47. [47]

    Anna Rogers, Olga Kovaleva, and Anna Rumshisky. 2020. https://arxiv.org/abs/2002.12327 A primer in BERT ology: What we know about how BERT works . Transactions of the Association for Computational Linguistics (TACL), 8:842--866

    work page internal anchor Pith review Pith/arXiv arXiv 2020

  48. [48]

    Frederic Sala, Christopher De Sa, Albert Gu, and Christopher R\'e. 2018. https://arxiv.org/abs/1804.03329 Representation tradeoffs for hyperbolic embeddings . In Proceedings of the 35th International Conference on Machine Learning (ICML)

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  49. [49]

    Tal Schuster, Darsh J Shah, Yun Jie Serene Yeo, Daniel Filizzola, Enrico Santus, and Regina Barzilay. 2019. https://arxiv.org/abs/1908.05267 Towards debiasing fact verification models . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  50. [50]

    A. Singer. 2006. https://doi.org/10.1016/j.acha.2006.03.004 From graph to manifold laplacian: The convergence rate . Applied and Computational Harmonic Analysis, 21(1):128--134. Special Issue: Diffusion Maps and Wavelets

    work page doi:10.1016/j.acha.2006.03.004 2006

  51. [51]

    Bartlomiej Sobieski, Matthew Tivnan, Dawid Płudowski, Michał Jan Włodarczyk, Pengfei Jin, Przemyslaw Biecek, and Quanzheng Li. 2026. https://arxiv.org/abs/2605.05026 Local intrinsic dimension unveils hallucinations in diffusion models . Preprint, arXiv:2605.05026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  52. [52]

    Matthieu Tehenan, Vikram Natarajan, Jonathan Michala, Milton Lin, and Juri Opitz. 2025. https://arxiv.org/abs/2506.04373 Mechanistic decomposition of sentence representations . arXiv preprint arXiv:2506.04373

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  53. [53]

    James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. https://arxiv.org/abs/1803.05355 FEVER : a large-scale dataset for fact extraction and VERification . In NAACL-HLT

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  54. [54]

    Lucrezia Valeriani, Diego Doimo, Francesca Cuturello, Alessandro Laio, Alessio Ansuini, and Alberto Cazzaniga. 2023. https://arxiv.org/abs/2302.00294 The geometry of hidden representations of large transformer models . In Advances in Neural Information Processing Systems (NeurIPS)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  55. [55]

    Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2018. https://doi.org/10.18653/v1/W18-5446 GLUE : A multi-task benchmark and analysis platform for natural language understanding . In Proceedings of the 2018 EMNLP Workshop B lackbox NLP : Analyzing and Interpreting Neural Networks for NLP , pages 353--355, Brussels,...

    work page doi:10.18653/v1/w18-5446 2018

  56. [56]

    Alex Warstadt, Amanpreet Singh, and Samuel R Bowman. 2018. https://arxiv.org/abs/1805.12471 Neural network acceptability judgments . arXiv preprint arXiv:1805.12471

    work page internal anchor Pith review Pith/arXiv arXiv 2018