Pith. sign in

REVIEW 5 major objections 4 minor 38 references

Geometric Generality of Transformer-Based Gr\"obner Basis Computation

T0 review · 5 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Pith's one-line read Under an unproved irreducibility heuristic, the outputs of the proposed generator-sampling algorithm are Zariski dense in the space of generator sets of a fixed ideal, supplying a conditional geometric notion of dataset generality for transformer training.

desk verdict A solid conditional density theorem for a generalized generator construction, but the abstract overclaims: the result does not cover the restricted output set of the KIK+24 dataset, and the key heuristic is unproven. read the letter →

arxiv 2504.12465 v1 pith:KWPJGK4J submitted 2025-04-16 cs.LG cs.SCmath.AGstat.ML

classification cs.LGcs.SCmath.AGstat.ML
keywords datasetsobnerbasiscomputationtrainingtransformersaddressalgorithm
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

To train a transformer to compute Gröbner bases, you need many pairs: a list of polynomials as input, and the Gröbner basis of the ideal they generate. Since computing Gröbner bases is expensive, a common trick is to start from a known basis and randomly build polynomial systems that generate the same ideal. Whether the random systems cover the full space of possible inputs has remained heuristic.

This paper formalizes 'covers the full space' using Zariski density, a topology from algebraic geometry where a set is dense if every nonzero polynomial condition that vanishes on the set vanishes everywhere. The authors consider the algorithm that applies a product of random elementary row operations to a fixed basis. They prove that, provided a certain algebraic set X≤D is irreducible (their Heuristic 2.2), and the field is Hilbertian, the possible outputs are Zariski dense. The proof uses classical theorems: Quillen-Suslin on free modules, Suslin's stability theorem, and Hilbert's irreducibility theorem.

The gap between this conditional theorem and the abstract's learning claim is real. Zariski density says the support of the generated examples is generic; it does not say a finite sample trains a transformer, and the theorem's range m≥2n lies mostly outside the m≤n+2 setting of the earlier experiments. The abstract says the result covers the previously proposed algorithm, but the theorem is proved for a more general class; density of a superset does not imply density of the original subset.

Extended reading notes

Core claim

Theorem 2.3 (Corollary 4.12): for a Hilbertian field K, m≥2n≥3, and under Heuristic 2.2 for D and G, the set F0 ∩ F~≤D is dense in F~≤D for the relative Zariski topology; consequently, if the heuristic holds for all sufficiently large D, F0 is Zariski dense in F = {F ∈ R^m | ⟨F⟩ = ⟨G⟩}. If the paper is correct, the possible outputs of the generalized generator algorithm form a generic set in the coefficient space, not a special subvariety.

Load-bearing premise

Heuristic 2.2 (Section 2.2): for given D and G, the algebraic set X≤D = {(B,A) ∈ R^{n×m}_{≤D} × R^{m×n}_{≤D} | (BA−En)G=0, B1A1∈R^{n×n}_{≤D}} is irreducible. The proof of Theorem 2.3 needs X≤D irreducible to apply Lemma 4.10 and Theorem 4.11; if the heuristic fails for the relevant D and G, the density conclusion is not established. The paper gives no proof or evidence for this irreducibility. Independently, the application to the earlier KIK+24 algorithm assumes its specific Bruhat-like output set is dense, a statement not proven in the paper.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. This paper studies the geometric generality of training data for Transformer-based Gröbner basis computation. It formulates Problem 2.1: an algorithm that, for a fixed tuple G of polynomials, randomly outputs generators F of the ideal ⟨G⟩, such that the set F0 of all possible outputs is Zariski dense in the set F of all generator tuples of ⟨G⟩. The authors propose Algorithm 2, which outputs F = A G where A = U (En;0)^T and U is a finite product of elementary matrices over the polynomial ring R. They prove (Theorem 2.3/Corollary 4.12) that, over a Hilbertian field, for m ≥ 2n ≥ 3 and under Heuristic 2.2 (irreducibility of X≤D), the set F0 ∩ ilde F≤D is dense in ilde F≤D; Corollary 2.4 upgrades this to density of F0 in F when the heuristic holds for all sufficiently large D. The proof uses Quillen–Suslin, Suslin's stability theorem, and Hilbert's irreducibility theorem. The introduction and abstract further claim that this justifies the earlier dataset generation method of [KIK+24].

Significance. If the conditional theorem is correct, it provides a genuine geometric statement about a class of generator-construction algorithms: the possible outputs are Zariski dense rather than confined to a special subvariety. The use of Quillen–Suslin to replace left-regular matrices by an elementary-matrix action is elegant, and the section construction in Corollary 4.12 is a good idea. The paper is also honest in labelling Heuristic 2.2 as an assumption, and the proof is mostly self-contained. However, the significance as stated in the abstract is not achieved: the theorem concerns the new generalized Algorithm 2, not the specific Bruhat-like construction of [KIK+24], and it applies in a parameter regime (m≥2n) that is disjoint from the experiments except for n=2,m=4. The unproven Heuristic 2.2 and the flat-locus proof gap further limit the current version.

major comments (5)
  1. [Abstract, §2.2, §3.1] The advertised claim that the dataset generation algorithm of [KIK+24] is shown to be sufficiently general is not supported by the theorems. Theorem 2.3 and Corollary 4.12 prove that F0 = {U(En;0)^T G | U ∈ E(m)} is dense in F, and Proposition 4.2 identifies this set with all outputs F=AG for left-regular A over R. The construction of [KIK+24] recalled in §3.1 uses the restricted Bruhat-like family A = U1 S (U2;0)^T. Since F0 is a superset of the KIK output set, density of F0 says nothing about density of the restricted family. No statement or proof in the paper establishes that the Bruhat-like family is itself Zariski dense in F, and the missing logical bridge is not repaired by Proposition 4.2, which only characterizes the larger class.
  2. [Remark 2.6 / Theorem 2.3] There is a parameter-regime mismatch: the main theorem assumes m ≥ 2n ≥ 3, whereas Remark 2.6 states that the experiments in [KIK+24] impose m ≤ n+2. For n ≥ 3 these conditions are incompatible; the only overlapping case is (m,n) = (4,2). Thus even if Heuristic 2.2 were fully justified, the theorem would not cover the experimental regime used in the prior work. The authors should prove a version for m < 2n or explicitly restrict their claims to the new setting.
  3. [Heuristic 2.2] The core density theorem is conditional on Heuristic 2.2, the irreducibility of X≤D, for which the paper offers no proof, no evidence, and no discussion of regimes where it might fail. Since this assumption is load-bearing for Lemma 4.10 and Theorem 4.11, and hence for Theorem 2.3, the paper's central conclusion is only an implication from an unverified algebraic-geometric statement. At minimum, the authors should prove or test irreducibility in nontrivial cases such as n=2, m=4, or give a structural argument based on the equations (BA−En)G=0.
  4. [Corollary 4.12 / Theorem 4.11] The proof that the flat locus Y is nonempty contains a gap. The sentence 'Let L be the field of fractions ... Then Spec L is isomorphic to a non-empty open subscheme V in R^{n×n}_{≤D}' is not correct: Spec L is the generic point and is not an open subscheme of a positive-dimensional affine space. Consequently the claim Y ≠ ∅ is not established as written. The gap appears repairable by invoking generic flatness for finite-type morphisms to a reduced Noetherian scheme, which gives a nonempty open V over which p is flat, but the current text does not supply that argument.
  5. [Problem 2.1 / Abstract] Problem 2.1 and the abstract formulate the result in terms of datasets being dense, but a finite training dataset is never Zariski dense in an infinite variety. What the proof actually establishes is that the set of all possible outputs of Algorithm 2 is dense in the set of all generators of ⟨G⟩. This distinction matters for the claimed learning guarantee: density of the generator's support is only a necessary condition for a finite sample to be representative, not a sufficient one. The paper should state this limitation explicitly.
minor comments (4)
  1. [§4.2] In the displayed definition of ilde F≤D before Lemma 4.10, the notation X_{≤D}^{m×n} is inconsistent with the definition of X_{≤D} in Heuristic 2.2; this appears to be a typo.
  2. [Lemma 4.8] In the proof, the displayed factorization uses the symbol u3 twice in the second factor; this is presumably a typo and should involve two distinct coefficients.
  3. [§3.1] There is a typo: 'basss' should be 'bases'.
  4. [§2.1] The notation F = {F ∈ R^m | ⟨F⟩ = ⟨G⟩} treats G as fixed, while Problem 2.1 presents G as an input; please make the dependence on G explicit in the notation throughout.
Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

All parameters (m, n, r, D, G) are inputs, not fitted values. The proof relies on standard theorems from algebraic geometry and commutative algebra, plus one unproved ad hoc heuristic, Heuristic 2.2, on the irreducibility of X≤D. No new entities are postulated.

assumptions (6)
  • standard math Quillen-Suslin theorem (Serre's conjecture): finitely generated projective modules over K[x1,...,xr] are free.
    Invoked in Proposition 4.2 to identify the kernel of phi_B as free and to obtain the equivalence of split exact sequences.
  • standard math Suslin's stability theorem: for m>=3, E(m) = <GL(K^m), SL(R^m)>.
    Used in Proposition 4.2 to express the isomorphisms U and V as products of elementary matrices.
  • domain assumption Hilbert's irreducibility theorem holds for Hilbertian fields K.
    Used in Lemma 4.9 to show matrices whose determinants are non-zero irreducible polynomials form a Zariski-dense set; Theorem 2.3 assumes K is Hilbertian, e.g., number fields.
  • ad hoc to paper Heuristic 2.2: X≤D is irreducible for the given D and G.
    This is the unproved assumption on which the density theorem is conditional; the flat-locus argument needs an irreducible domain.
  • standard math Flat morphisms of finite type between Noetherian schemes are open (Hartshorne III.9.1).
    Used in Lemma 4.10 to show that the preimage of a dense set under a flat morphism is dense.
  • domain assumption Parameter constraints m>=2n>=3 and R=K[x1,...,xr] with K Hilbertian.
    The theorem and proof require these assumptions; they exclude most experimental settings from KIK+24 where m<=n+2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Geometric Generality of Transformer-Based Gr\"obner Basis Computation." pith.science (2026). https://pith.science/paper/KWPJGK4J

@misc{pith2026250412465,
  author       = {Pith},
  title        = {Pith review of: Geometric Generality of Transformer-Based Gr\"obner Basis Computation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KWPJGK4J}},
  note         = {Machine review of arXiv:2504.12465}
}
read the original abstract

The intersection of deep learning and symbolic mathematics has seen rapid progress in recent years, exemplified by the work of Lample and Charton. They demonstrated that effective training of machine learning models for solving mathematical problems critically depends on high-quality, domain-specific datasets. In this paper, we address the computation of Gr\"obner basis using Transformers. While a dataset generation method tailored to Transformer-based Gr\"obner basis computation has previously been proposed, it lacked theoretical guarantees regarding the generality or quality of the generated datasets. In this work, we prove that datasets generated by the previously proposed algorithm are sufficiently general, enabling one to ensure that Transformers can learn a sufficiently diverse range of Gr\"obner bases. Moreover, we propose an extended and generalized algorithm to systematically construct datasets of ideal generators, further enhancing the training effectiveness of Transformer. Our results provide a rigorous geometric foundation for Transformers to address a mathematical problem, which is an answer to Lample and Charton's idea of training on diverse or representative inputs.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 34 canonical work pages

  1. [1]

    Roberto Avanzi, Joppe Bos, L \'e o Ducas, Eike Kiltz, Tancr \`e de Lepoint, Vadim Lyubashevsky, John M Schanck, Peter Schwabe, Gregor Seiler, and Damien Stehl \'e , CRYSTALS-Kyber (version 3.02) , Submission to round 3 of the NIST post-quantum project, Available at https://pq-crystals.org/

  2. [2]

    Alberto Alfarano, Fran c ois Charton, and Amaury Hayat, Global lyapunov functions: a long-standing open problem in mathematics, with symbolic transformers, Advances in Neural Information Processing Systems 37 (2024), 93643--93670

  3. [3]

    ACC (IEEE Cat

    Hirokazu Anai and Shinji Hara, Fixed-structure robust controller synthesis based on sign definite condition by a special quantifier elimination , Proceedings of the 2000 American Control Conference. ACC (IEEE Cat. No. 00CH36334), vol. 2, IEEE, 2000, pp. 1312--1316

  4. [4]

    B \^o cher, Introduction to Higher Algebra , Macmillan, 1907

    M. B \^o cher, Introduction to Higher Algebra , Macmillan, 1907

  5. [5]

    Proceedings, Springer, 2006, pp

    Johannes Buchmann, Andrei Pyshkin, and Ralf-Philipp Weinmann, Block ciphers sensitive to G r \"o bner Basis Attacks , Topics in Cryptology--CT-RSA 2006: The Cryptographers’ Track at the RSA Conference 2006, San Jose, CA, USA, February 13-17, 2005. Proceedings, Springer, 2006, pp. 313--331

  6. [6]

    Buchberger, A theoretical basis for the reduction of polynomials to canonical forms, SIGSAM Bull

    Bruno. Buchberger, A theoretical basis for the reduction of polynomials to canonical forms, SIGSAM Bull. 10 (1976), no. 3, 19–29

  7. [7]

    Leandro Caniglia, Guillermo Corti \ n as, Silvia Dan \'o n, Joos Heintz, Teresa Krick, and Pablo Solern \'o , Algorithmic aspects of suslin's proof of serre's conjecture, computational complexity 3 (1993), 31--55

  8. [8]

    David Cox, John Little, and Donal O'Shea, Ideals, Varieties, and Algorithms , second ed., Undergraduate Texts in Mathematics, Springer-Verlag, New York, 1997, An introduction to computational algebraic geometry and commutative algebra. 1417938

Show all 38 references
  1. [9]

    Cox, John B

    David A. Cox, John B. Little, and Donal O'Shea, Using Algebraic Geometry , first ed., Graduate Texts in Mathematics, vol. 185, Springer, 1998

  2. [10]

    345--359

    Nicolas T Courtois and Willi Meier, Algebraic Attacks on Stream Ciphers with Linear Feedback , Advances in Cryptology—EUROCRYPT 2003: International Conference on the Theory and Applications of Cryptographic Techniques, Warsaw, Poland, May 4--8, 2003 Proceedings 22, Springer, 2...

  3. [11]

    150, Springer-Verlag, New York, 1995

    David Eisenbud, Commutative Algebra with a View Toward Algebraic Geometry , Graduate Texts in Mathematics, vol. 150, Springer-Verlag, New York, 1995. 1322960

  4. [12]

    1, 61--88

    Jean-Charles Faug \`e re, A new efficient algorithm for computing G r\" o bner bases ( F 4) , Journal of Pure and Applied Algebra 139 (1999), no. 1, 61--88

  5. [13]

    Jean-Charles Faugere and Antoine Joux, Algebraic Cryptanalysis of Hidden Field Equation (HFE) Cryptosystems Using Gr \"o bner Bases , Annual International Cryptology Conference, Springer, 2003, pp. 44--60

  6. [14]

    Fried and Moshe Jarden, Field Arithmetic , 4th corrected edition ed., Ergeb

    Michael D. Fried and Moshe Jarden, Field Arithmetic , 4th corrected edition ed., Ergeb. Math. Grenzgeb., 3. Folge, vol. 11, Cham: Springer, 2023 (English)

  7. [15]

    Hiroki Furuta, Gouki Minegishi, Yusuke Iwasawa, and Yutaka Matsuo, Towards empirical interpretation of internal circuits and properties in grokked transformers on modular polynomials, 2024, arXiv preprint arXiv:2402.16726

  8. [16]

    247--257

    Patrizia Gianni and Teo Mora, Algebraic solution of systems of polynomial equations using G r \"o bner bases , Applied Algebra, Algebraic Algorithms and Error-Correcting Codes (Berlin, Heidelberg), Springer Berlin Heidelberg, 1989, pp. 247--257

  9. [17]

    Andrey Gromov, Grokking modular arithmetic, 2023, arXiv preprint arXiv:2301.02679

  10. [18]

    2-3, 149--167

    Patrizia Gianni, Barry Trager, and Gail Zacharias, Gr \"o bner Bases and Primary Decomposition of Polynomial Ideals , Journal of Symbolic Computation 6 (1988), no. 2-3, 149--167

  11. [19]

    Robin Hartshorne, Algebraic Geometry , Springer-Verlag, New York-Heidelberg, 1977, Graduate Texts in Mathematics, No. 52. 0463157

  12. [20]

    Michael Hahn and Mark Rofin, Why are sensitive functions hard for transformers?, 2024, arXiv preprint arXiv:2402.09963

  13. [21]

    Hiroshi Kera, Yuki Ishihara, Yuta Kambe, Tristan Vaccon, and Kazuhiro Yokoyama, Learning to compute G r \"o bner bases , Advances in Neural Information Processing Systems 37 (2024), 33141--33187

  14. [22]

    206--222

    Aviad Kipnis, Jacques Patarin, and Louis Goubin, Unbalanced Oil and Vinegar signature Schemes , International Conference on the Theory and Applications of Cryptographic Techniques, Springer, 1999, pp. 206--222

  15. [23]

    ed., Grad

    Serge Lang, Algebra , 3rd revised ed. ed., Grad. Texts Math., vol. 211, New York, NY: Springer, 2002 (English)

  16. [24]

    Guillaume Lample and Fran c ois Charton, Deep Learning for Symbolic Mathematics , 2019, arXiv preprint arXiv:1912.01412

  17. [25]

    Available at https://pq-crystals.org/

    Vadim Lyubashevsky, L \'e o Ducas, Eike Kiltz, Tancr \`e de Lepoint, Peter Schwabe, Gregor Seiler, Damien Stehl \'e , and Shi Bai, CRYSTALS-Dilithium , Algorithm Specifications and Supporting Documentation (Version 3.1). Available at https://pq-crystals.org/

  18. [26]

    10, 882--891

    Reinhard Laubenbacher and Bernd Sturmfels, Computer Algebra in Systems Biology , The American Mathematical Monthly 116 (2009), no. 10, 882--891

  19. [27]

    Cathy Li, Jana Sotáková, Emily Wenger, Mohamed Malhou, Evrard Garcelon, Francois Charton, and Kristin Lauter, SALSA PICANTE: a machine learning attack on LWE with binary secrets , Cryptology ePrint Archive, Paper 2023/340, 2023, https://eprint.iacr.org/2023/340

  20. [28]

    Cathy Li, Emily Wenger, Zeyuan Allen-Zhu, Francois Charton, and Kristin E Lauter, SALSA VERDE: a machine learning attack on Learning with Errors with sparse small secrets , Advances in Neural Information Processing Systems 36 (2023), 53343--53361

  21. [29]

    3, 305--329

    Ernst W Mayr and Albert R Meyer, The Complexity of the Word Problems for Commutative Semigroups and Polynomial Ideals , Advances in mathematics 46 (1982), no. 3, 305--329

  22. [30]

    NIST, Post-Quantum Cryptography Standardization Selected Algorithms 2022 , https://csrc.nist.gov/Projects/post-quantum-cryptography/selected-algorithms-2022

  23. [31]

    4, 579--590

    Keith Pardue, Generic sequences of polynomials, Journal of Algebra 324 (2010), no. 4, 579--590

  24. [32]

    6, 1--40

    Oded Regev, On Lattices, Learning with Errors, Random Linear Codes, and Cryptography , Journal of the ACM (JACM) 56 (2009), no. 6, 1--40

  25. [33]

    o bner bases using gr \

    Akira Suzuki and Yosuke Sato, A simple algorithm to compute comprehensive gr \"o bner bases using gr \"o bner bases , Proceedings of the 2006 international symposium on Symbolic and Algebraic Computation, 2006, pp. 326--331

  26. [34]

    2, 221--238

    Andrey Aleksandrovich Suslin, On the structure of the special linear group over polynomial rings , Mathematics of the USSR - Izvestija 11 (1977), no. 2, 221--238

  27. [35]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin, Attention Is All You Need , Advances in neural information processing systems 30 (2017)

  28. [36]

    Emily Wenger, Mingjie Chen, Francois Charton, and Kristin E Lauter, SALSA: Attacking Lattice Cryptography with Transformers , Advances in Neural Information Processing Systems 35 (2022), 34981--34994

  29. [37]

    1, 1--29

    Volker Weispfenning, Comprehensive gr \"o bner bases , Journal of symbolic computation 14 (1992), no. 1, 1--29

  30. [38]

    393--419

    Mizuki Yoshizawa, Akira Terui, and Masahiko Mikawa, Inverse Kinematics and Path Planning of Manipulator Using Real Quantifier Elimination Based on Comprehensive Gr \"o bner Systems , International Workshop on Computer Algebra in Scientific Computing, Springer, 2023, pp. 393--419

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.