Pith. sign in

REVIEW 3 major objections 6 minor 18 references

Space filling positionality and the Spiroformer

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A spiral space-filling curve supplies the global order transformers lack on the sphere, and a small transformer trained along it reaches roughly 90% training accuracy at predicting spherical Hamiltonian vector fields.

desk verdict A modestly novel spiral-ordering positionality idea whose only experimental support is a single training curve, so the geometric-reconstruction claim is untested as written. read the letter →

arxiv 2507.08456 v1 pith:OFXCOTI2 submitted 2025-07-11 cs.LG cs.AImath.DGmath.DSmath.SG

M. Maurin , M.Á. Evangelista-Alvarado , P. Suárez-Serrato This is my paper · ORCID
classification cs.LGcs.AImath.DGmath.DSmath.SG MSC 68T0753D0553D17
keywords transformersspace-fillingcurvesphericalspiralpositionalencodingHamiltonianvectorfieldsharmonicsgeometricdeeplearningmanifolds
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Transformers rely on a fixed global order—text, time, or grid position—but manifolds like the sphere have no canonical ordering. This paper proposes that a space-filling curve, specifically a polar spherical spiral, can supply that order, converting geometric data into a sequence a transformer can process. The authors build the Spiroformer, which reads samples of Hamiltonian vector fields along the spiral and predicts the next sample, and report about 90% training accuracy, with validation still lagging due to overfitting. The point they are trying to establish is that this spiral-based positionality is a viable way to import the full transformer toolkit into geometric domains.

What carries the argument

The central object is the polar spherical spiral, the curve $(x,y,z)=(\sin t \cos ct, \sin t \sin ct, \cos t)$ for $t \in [0,\pi]$, used as a space-filling curve that assigns a total order to points of $S^2$. The transformer's positional encoding is tied to position along this curve, so attention operates along the spiral rather than on the sphere directly. The training signal comes from Hamiltonian vector fields $X_H$ built with the Poisson bivector $\pi = \sin\theta\,\partial_\theta \wedge \partial_\phi$ applied to spherical harmonics, and the task is next-sample prediction with causal masking. Together these convert the geometric problem into a sequence-to-sequence one.

What would settle it

Evaluate the trained model on vectors at points randomly sampled across the whole sphere rather than on the spiral; if prediction error at those points is no better than guessing the mean field, the model has learned curve-following, not spherical geometry. A complementary test is to retrain on the same vector samples in a randomly permuted order—if training accuracy stays near 90%, the spiral's geometric ordering is not the cause of the result.

Watch

Extended reading notes

Core claim

The paper's central claim is that attention heads placed along a space-filling curve give a transformer the global ordering it needs to handle manifold-valued data, and that this works concretely for reconstructing Hamiltonian vector fields on the 2-sphere. The Spiroformer takes sequences of vector samples generated from spherical-harmonic Hamiltonians, uses the spiral's parameter to define positional encodings, and learns to predict the next vector in the sequence, achieving roughly 90% training accuracy (Fig. 5). On the authors' framing, the model 'effectively learns to reconstruct Hamiltonian vector fields on the sphere,' so the discovery is a new positional-ordering mechanism, with the overfitting gap acknowledged as an open technical issue.

Load-bearing premise

The claim rests on the assumption that training the transformer to predict the next vector sample along the spiral measures understanding of the sphere's geometry, even though the model never sees off-spiral data and consecutive samples are strongly correlated by the smoothness of the field.

Editorial extensions

If this is right

  • If the spiral ordering works, any manifold admitting a space-filling curve can be fed into a standard transformer by using the curve as the positional encoding, bypassing custom spherical attention layers.
  • The ~90% training accuracy shows that an autoregressive transformer can track the local evolution of spherical Hamiltonian vector fields along the curve, which is the first concrete evidence for the approach.
  • The validation gap implies a concrete next step: enlarge the dataset (e.g., more than 100 samples per field) and regularize, which the authors expect could bring validation close to training accuracy.
  • A structure-preserving variant that recovers the symbolic Hamiltonian (via symbolic regression) would guarantee the model outputs are genuinely Hamiltonian, a direction the paper identifies as future work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial: the ~90% training accuracy admits a mundane alternative explanation—along a smooth spiral, consecutive points are heavily correlated, so a model that learns local interpolation could succeed without any global geometric representation; a shuffled-order control would distinguish these.
  • Editorial: the mechanism should be tested for curve-specificity: swapping the polar spiral for another space-filling curve (e.g., a Hilbert-like curve on a cubed sphere or a Peano curve) on the same task would show whether the benefit comes from the particular spiral or from the general property of having a locality-preserving total order.
  • Editorial: the Hamiltonian structure is not essential to the ordering idea; rerunning the same setup with generic smooth functions would clarify whether the spiral positionality, rather than the symplectic structure, is what the transformer exploits.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes to extend transformers to geometric domains by ordering tokens along a spherical spiral, which it calls a 'space-filling curve.' It introduces the Spiroformer, a transformer trained to predict the next sample of a Hamiltonian vector field on the 2-sphere given a sequence of 100 vector samples along a polar spiral. The authors report roughly 90% training accuracy (Fig. 5), acknowledge lower validation performance and overfitting, and suggest future improvements including regularization and a SymFlux-like symbolic Hamiltonian recovery. The central claim, stated in Sections 1.3 and 1.4, is that the model 'effectively learns to reconstruct Hamiltonian vector fields on the sphere.'

Significance. The idea of using a curve that imposes a global order on a manifold to adapt sequence models to geometric data is potentially interesting, and the paper makes its data-generation pipeline concrete by using spherical harmonics and the Poisson bivector on S^2. However, the significance as presented is limited by the weakness of the experimental evidence and by a mathematically inaccurate core premise. The paper does not provide machine-checked proofs, reproducible code, or a parameter-free derivation; its only quantitative result is a single training/validation curve. If the evaluation were properly controlled and the space-filling claim corrected, the approach could be a modest contribution to geometric deep learning, but as it stands the supporting evidence does not establish the stated reconstruction claim.

major comments (3)
  1. [Section 1.3 and 1.4, Fig. 5] The claim that the Spiroformer 'effectively learns to reconstruct Hamiltonian vector fields on the sphere' is not supported by the experimental protocol. The only reported measure is next-sample prediction accuracy along one polar spiral. Since the data are 100 samples of a smooth vector field along a 1D curve, the target v_{t+1} is essentially a function of the curve parameter t; a transformer with enough capacity can fit such a 1D sequence without learning the geometry of S^2. There is no evaluation on points off the spiral, no comparison with a random or reversed ordering of the same 100 points, and no quantitative error metric (e.g., mean squared error against true off-spiral vectors). The lower validation accuracy in Fig. 5 is consistent with interpolation along the curve rather than geometric reconstruction.
  2. [Section 1.1, 'Spiral Over the Sphere'] The polar spiral is not a space-filling curve. Its image is a 1D submanifold of S^2 with measure zero; it is not surjective, so it does not 'cover the sphere' or 'provide a continuous traversal of the sphere's surface' in the sense of a space-filling curve. This error is load-bearing because the title, abstract, and Section 1.1 motivate the entire method through the space-filling property. Either the terminology should be corrected to 'spiral ordering' or a genuinely space-filling construction (e.g., an approximation of a Hilbert curve on the sphere) should be used.
  3. [Section 1.4 and Fig. 5] The sentence 'With larger sample sizes on machines with more local memory resources, validation could match training performance close to 90%' is speculation, not a result. Moreover, the reported 'high accuracy during training, ~90%' is based on a single curve with no error bars, no definition of accuracy (e.g., thresholded cosine similarity, classification, or regression), and no table of exact values. Without a precise definition and repeated runs, the number is not reproducible and does not support the reconstruction claim.
minor comments (6)
  1. [Abstract and Title] The term 'space-filling curve' is used inaccurately; a polar spiral on S^2 is not space-filling. Consider using 'spiral positionality' or 'spiral ordering' throughout.
  2. [Section 1.2, Dataset Generation] The dataset consists of Hamiltonian vector fields generated from spherical harmonics through the authors' own poissongeometry modules and sampled on a geodesic sphere from geomstats. This is not inherently circular, but the absence of any independent data source or held-out field family makes the evaluation self-referential to the generation pipeline.
  3. [Section 1.3, Positional Encodings] The positional encoding is not specified; the reader does not know whether it is a learned embedding, a sinusoidal encoding, or a function of the spiral parameter t. The manuscript should state this explicitly.
  4. [Fig. 5] The axes and legend of Fig. 5 are not fully described; in particular, the y-axis is not labeled, and it is unclear whether the plotted quantity is accuracy, loss, or something else.
  5. [References] Reference [7] contains a typo ('an others' should be 'and others').
  6. [Section 2, Conclusions] The mention of SymFlux as a future direction is not elaborated; since it is used to motivate structure preservation, a brief explanation of how it would be adapted to spherical data would be helpful.

Circularity Check

1 steps flagged · score 6.0 of 10

Training accuracy on next-token prediction is presented as evidence of spherical reconstruction, reducing the central claim to the fitted training objective.

  1. fitted input called prediction [Section 1.3 'Our Model: the Spiroformer' and Section 1.4 'Results']
    "Given a sequence of vector field samples v1, v2, ..., vt along the spiral, our model is trained to predict the next sample vt+1. By learning such ordered sequences, our model effectively learns to reconstruct Hamiltonian vector fields on the sphere. ... Our Spiroformer model achieves high accuracy during training, ∼ 90%, as can be seen in fig. 5."

    The training objective is exactly next-token prediction vt -> vt+1 on samples taken along the spiral, and the claimed capability is 'reconstruct Hamiltonian vector fields on the sphere.' The only reported evidence is training accuracy on that same objective: the model is fit to predict these exact next samples, and then that fit quality is cited as confirmation of the geometric reconstruction claim. No held-out off-spiral evaluation and no alternative ordering are used, so the ~90% figure measures memorization of the training sequences rather than an independent prediction. The conclusion that the model reconstructs the field on S^2 therefore reduces, by construction, to reporting the fit accuracy of the sequence model.

full rationale

There is no parametric derivation whose output is fed back into its assumptions, and the authors' use of their own poissongeometry and numericalpoissongeometry packages to generate Hamiltonian vector fields is standard tooling with explicit geometric equations, so those self-citations are not load-bearing circularity. The circular component is the evaluation logic: the paper's central claim, 'our model effectively learns to reconstruct Hamiltonian vector fields on the sphere,' is operationally defined as the ability to predict the next sample along the spiral, and the reported ~90% figure is the training accuracy on that very prediction task. Because all data lie on a one-dimensional spiral, the model is never asked to reconstruct the field at off-spiral points, and no comparison with random or other orderings isolates the geometric contribution of the spiral. The paper honestly acknowledges the lower validation scores and overfitting, which weakens the claim further, but the specific circular step is the presentation of fitted in-sample next-token accuracy as evidence for a geometric reconstruction result. This warrants a score of 6: the central claim reduces to the fitted training objective, while the paper's caveats and the absence of a closed self-citation loop prevent a higher score. The additional assertion that the polar spiral is a 'space-filling curve' that 'covers the sphere' is mathematically false, since the curve has measure zero on S^2, but that is a correctness issue rather than a circularity issue and is not scored as such here.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The main hidden choices are the spiral constant c, harmonic degree n=32, 100-point sampling, and transformer hyperparameters; none are ablated. The pipeline also assumes a polar spiral on S2 is a valid 'space-filling' ordering for geometric data, which is mathematically questionable. No new physical or conceptual entities are postulated.

free parameters (4)
  • Spiral turn parameter c
    Controls the number of spiral turns and the density and ordering of sampled points. No value is reported, and the result likely depends on it.
  • Maximum spherical harmonic degree n = 32
    Chosen by hand; generates 1024 Hamiltonian vector fields and fixes the difficulty of the learning task.
  • Number of spiral sample points per field = 100
    Sequence length for the transformer. Dense sampling makes adjacent-token prediction easier; no ablation is given.
  • Transformer hyperparameters = 2 layers, 4 heads, dropout 0.2, 2000 epochs
    Selected via Optuna with the note 'always obtaining similar results', but no search details, seeds, or architecture dimensions are given.
assumptions (4)
  • domain assumption Spherical harmonics of degree n=32 form a basis and their induced Hamiltonian vector fields cover a representative target class.
    The method is trained only on these generated fields; generalization to arbitrary spherical vector fields is assumed without evidence.
  • ad hoc to paper The polar spiral is a space-filling curve and its 100 sampled points cover S2 uniformly enough for ordered representation.
    A one-dimensional curve has measure zero in S2 and is not space-filling; the uniformity claim is unquantified, so the ordering could dominate or distort the learning signal.
  • domain assumption Next-sample prediction along one fixed spiral measures geometric reconstruction rather than interpolation between neighboring samples.
    Adjacent samples on a smooth Hamiltonian vector field are highly correlated; no test on unseen fields or shifted spirals distinguishes memorization from geometric learning.
  • standard math Standard symplectic and Poisson geometry equations, including X_H = #omega(-dH).
    Used as background to generate ground-truth Hamiltonian vector fields; not in question but is assumed without proof.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Space filling positionality and the Spiroformer." pith.science (2026). https://pith.science/paper/OFXCOTI2

@misc{pith2026250708456,
  author       = {Pith},
  title        = {Pith review of: Space filling positionality and the Spiroformer},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OFXCOTI2}},
  note         = {Machine review of arXiv:2507.08456}
}
abstract

Transformers excel when dealing with sequential data. Generalizing transformer models to geometric domains, such as manifolds, we encounter the problem of not having a well-defined global order. We propose a solution with attention heads following a space-filling curve. As a first experimental example, we present the Spiroformer, a transformer that follows a polar spiral on the $2$-sphere.

Figures

Figures reproduced from arXiv: 2507.08456 by the authors.

Figure 1
Figure 1. Spherical data may come from geometric domains, such as global environmental sensors (im￾age from ClimateReanalyzer.org). This focus has led to significant advance￾ments in domains where data inherently pos￾sesses a linear or grid-like structure, first on text and now even in images [ 7]. However, the inherent geometry of many real-world datasets, particularly in biological and social networks, often deviates signif… view at source ↗
Figure 2
Figure 2. A selection of spherical Hamiltonian vector fields, showing samples of vectors with base points on a spherical spiral. are constrained to spherical geometries, such as fluid flow on a spherical surface, or the dynamics of rotating bodies. In general, let (M, ω) be a symplectic manifold, and H : M → R be a smooth function. There exists a unique vector field on M, denoted XH, which is determined by the following equat… view at source ↗
Figure 3
Figure 3. Visualizations of spherical harmonics. In many applications, we encounter vector fields that are not inherently Hamiltonian. We may seek to approximate such a dynamical system with a Hamiltonian one. This is known as Hamiltonization. Using recent kernel-based methods the underlying Hamiltonian function may be recovered from data to guarantee that the learned model is Hamiltonian [13]. Spherical Harmonics Spherical h… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: A spherical spiral (left), and a collection of points sampled on it (right). Using this spherical spiral (see fig. 4), we transform the problem of processing vector fields on a manifold into a sequence-based task. With this setup, we can now apply a Transformer model o…
Figure 5
Figure 5. Figure 5: The graph shows our Spiroformer training and validation performances. We used the Optuna optimizer for parameter search, always obtaining similar results. For this particular experiment we used 2 layers, 4 attention heads, a dropout rate of 0.2, and trained for 2000 ep…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 14 canonical work pages

  1. [1]

    Arnold, A

    V.I. Arnold, A. Weinstein, K. Vogtmann: Mathematical Methods of Classical Me- chanics. 2nd edn. Springer, New York (1989)

    work page 1989

  2. [2]

    Ballerin, N

    F. Ballerin, N. Blaser, E. Grong:SO(3)-Equivariant Neural Networks for Learning Vector Fields on Spheres. arXiv preprint arXiv:2503.09456 (2025)

    arXiv 2025

  3. [3]

    Bogatskiy, S

    A. Bogatskiy, S. Ganguly, T. Kipf, R. Kondor, D. W. Miller, D. Murnane, J. T. Offermann, M. Pettee, P. Shanahan, C. Shimmin, and S. Thais: Symmetry Group Equivariant Architectures for Physics, arXiv preprint arXiv:2203.06153, (2022)

    arXiv 2022

  4. [4]

    S. Cho, R. Jung, and J. Kwon: Sampling based spherical transformer for 360 degree image classification, Expert Systems with Applications, vol. 238, 121853, (2024). doi: 10.1016/j.eswa.2023.121853

    work page arXiv 2024

  5. [5]

    T. S. Cohen, M. Geiger, J. Köhler, and M. Welling: Spherical CNNs. Proceedings of the 6th International Conference on Learning Representations (ICLR), (2018). arXiv: 1801.10130

    arXiv 2018

  6. [6]

    Crainic, R

    M. Crainic, R. L. Fernandes, I. Mărcuţ: Lectures on Poisson Geometry. Graduate Studies in Mathematics, vol. 217. American Mathematical Society (2021)

    work page 2021

  7. [7]

    Dosovitskiy, an others: An image is worth 16x16 words: Transformers for image recognition at scale

    A. Dosovitskiy, an others: An image is worth 16x16 words: Transformers for image recognition at scale. Proceedings of the 9th International Conference on Learning Representations (ICLR), (2021). arXiv: 2010.11929

    arXiv 2021

  8. [8]

    M. Á. Evangelista-Alvarado, J. C. Ruíz-Pantaleón, and P. Suárez-Serrato: On com- putational Poisson geometry I: Symbolic foundations, Journal of Geometric Me- chanics, vol. 13, no. 4, pp. 607–628, (2021). doi: 10.3934/jgm.2021018. Space filling positionality and the Spiroformer

    work page doi:10.3934/jgm.2021018 2021

Show all 18 references
  1. [9]

    M. Á. Evangelista-Alvarado, J. C. Ruíz-Pantaleón, and P. Suárez-Serrato: On com- putational Poisson geometry II: Numerical methods, Journal of Computational Dy- namics, vol. 8, no. 3, pp. 273–307, (2021). doi: 10.3934/jcd.2021012

    2021 doi

  2. [10]

    M. Á. Evangelista-Alvarado, J. C. Ruíz-Pantaleón, and P. Suárez-Serrato: Exam- ples of Symbolic and Numerical Computation in Poisson Geometry , Geometric Science of information GSI (2021), Lecture Notes in Computer Science , vol. 12829, 200–208

    2021

  3. [11]

    Evangelista-Alvarado, P

    M. Evangelista-Alvarado, P. Suárez-Serrato: SymFlux: deep symbolic regression of Hamiltonian vector fields. arXiv preprint arXiv:2507.06342 [cs.LG] (2025)

    2025 arXiv

  4. [12]

    Y. Guo, K. Mokany, S.R. Levick, J. Yang, P. Moghadam: Spatioformer: A Geo- encoded Transformer for Large-Scale Plant Species Richness Prediction. IEEE Transactions on Geoscience and Remote Sensing (2025)

    2025

  5. [13]

    Hu, J.-P

    J. Hu, J.-P. Ortega, D. Yin: A structure-preserving kernel method for learning Hamiltonian systems. arXiv preprint arXiv:2403.10070 (2025)

    2025 arXiv

  6. [14]

    Koszul, Y.M

    J.L. Koszul, Y.M. Zou: Introduction to Symplectic Geometry. Springer, Singapore (2019)

    2019

  7. [15]

    X. Lai, Y. Chen, F. Lu, J. Liu, J. Jia: Spherical transformer for lidar-based 3d recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17545–17555 (2023)

    2023

  8. [16]

    Miolane, N

    N. Miolane, N. Guigui, A. Le Brigant, J. Mathe, B. Hou, Y. Thanwerdas, S. Hey- der, O. Peltre, N. Koep, H. Zaatiti, H. Hajri, Y. Cabanes, T. Gerald, P. Chauchat, C. Shewmake, D. Brooks, B. Kainz, C. Donnat, S. Holmes, and X. Pennec: Geom- stats: A Python Package for Riemannian...

    2020

  9. [17]

    Rußwurm, K

    M. Rußwurm, K. Klemmer, E. Rolf, R. Zbinden, and D. Tuia: Geographic loca- tion encoding with spherical harmonics and sinusoidal representation networks, in Proceedings of the Twelfth International Conference on Learning Representations, (2024)

    2024

  10. [18]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, Ł Kaiser, I. Polosukhin: Attention is all you need. Advances in neural information processing systems, 30 (2017)

    2017

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.