REVIEW 5 major objections 4 minor 38 references
Geometric Generality of Transformer-Based Gr\"obner Basis Computation
T0 review · 5 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read Under an unproved irreducibility heuristic, the outputs of the proposed generator-sampling algorithm are Zariski dense in the space of generator sets of a fixed ideal, supplying a conditional geometric notion of dataset generality for transformer training.
desk verdict A solid conditional density theorem for a generalized generator construction, but the abstract overclaims: the result does not cover the restricted output set of the KIK+24 dataset, and the key heuristic is unproven. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
This paper formalizes 'covers the full space' using Zariski density, a topology from algebraic geometry where a set is dense if every nonzero polynomial condition that vanishes on the set vanishes everywhere. The authors consider the algorithm that applies a product of random elementary row operations to a fixed basis. They prove that, provided a certain algebraic set X≤D is irreducible (their Heuristic 2.2), and the field is Hilbertian, the possible outputs are Zariski dense. The proof uses classical theorems: Quillen-Suslin on free modules, Suslin's stability theorem, and Hilbert's irreducibility theorem.
The gap between this conditional theorem and the abstract's learning claim is real. Zariski density says the support of the generated examples is generic; it does not say a finite sample trains a transformer, and the theorem's range m≥2n lies mostly outside the m≤n+2 setting of the earlier experiments. The abstract says the result covers the previously proposed algorithm, but the theorem is proved for a more general class; density of a superset does not imply density of the original subset.
Extended reading notes
Core claim
Theorem 2.3 (Corollary 4.12): for a Hilbertian field K, m≥2n≥3, and under Heuristic 2.2 for D and G, the set F0 ∩ F~≤D is dense in F~≤D for the relative Zariski topology; consequently, if the heuristic holds for all sufficiently large D, F0 is Zariski dense in F = {F ∈ R^m | ⟨F⟩ = ⟨G⟩}. If the paper is correct, the possible outputs of the generalized generator algorithm form a generic set in the coefficient space, not a special subvariety.
Load-bearing premise
Heuristic 2.2 (Section 2.2): for given D and G, the algebraic set X≤D = {(B,A) ∈ R^{n×m}_{≤D} × R^{m×n}_{≤D} | (BA−En)G=0, B1A1∈R^{n×n}_{≤D}} is irreducible. The proof of Theorem 2.3 needs X≤D irreducible to apply Lemma 4.10 and Theorem 4.11; if the heuristic fails for the relevant D and G, the density conclusion is not established. The paper gives no proof or evidence for this irreducibility. Independently, the application to the earlier KIK+24 algorithm assumes its specific Bruhat-like output set is dense, a statement not proven in the paper.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper studies the geometric generality of training data for Transformer-based Gröbner basis computation. It formulates Problem 2.1: an algorithm that, for a fixed tuple G of polynomials, randomly outputs generators F of the ideal ⟨G⟩, such that the set F0 of all possible outputs is Zariski dense in the set F of all generator tuples of ⟨G⟩. The authors propose Algorithm 2, which outputs F = A G where A = U (En;0)^T and U is a finite product of elementary matrices over the polynomial ring R. They prove (Theorem 2.3/Corollary 4.12) that, over a Hilbertian field, for m ≥ 2n ≥ 3 and under Heuristic 2.2 (irreducibility of X≤D), the set F0 ∩ ilde F≤D is dense in ilde F≤D; Corollary 2.4 upgrades this to density of F0 in F when the heuristic holds for all sufficiently large D. The proof uses Quillen–Suslin, Suslin's stability theorem, and Hilbert's irreducibility theorem. The introduction and abstract further claim that this justifies the earlier dataset generation method of [KIK+24].
Significance. If the conditional theorem is correct, it provides a genuine geometric statement about a class of generator-construction algorithms: the possible outputs are Zariski dense rather than confined to a special subvariety. The use of Quillen–Suslin to replace left-regular matrices by an elementary-matrix action is elegant, and the section construction in Corollary 4.12 is a good idea. The paper is also honest in labelling Heuristic 2.2 as an assumption, and the proof is mostly self-contained. However, the significance as stated in the abstract is not achieved: the theorem concerns the new generalized Algorithm 2, not the specific Bruhat-like construction of [KIK+24], and it applies in a parameter regime (m≥2n) that is disjoint from the experiments except for n=2,m=4. The unproven Heuristic 2.2 and the flat-locus proof gap further limit the current version.
major comments (5)
- [Abstract, §2.2, §3.1] The advertised claim that the dataset generation algorithm of [KIK+24] is shown to be sufficiently general is not supported by the theorems. Theorem 2.3 and Corollary 4.12 prove that F0 = {U(En;0)^T G | U ∈ E(m)} is dense in F, and Proposition 4.2 identifies this set with all outputs F=AG for left-regular A over R. The construction of [KIK+24] recalled in §3.1 uses the restricted Bruhat-like family A = U1 S (U2;0)^T. Since F0 is a superset of the KIK output set, density of F0 says nothing about density of the restricted family. No statement or proof in the paper establishes that the Bruhat-like family is itself Zariski dense in F, and the missing logical bridge is not repaired by Proposition 4.2, which only characterizes the larger class.
- [Remark 2.6 / Theorem 2.3] There is a parameter-regime mismatch: the main theorem assumes m ≥ 2n ≥ 3, whereas Remark 2.6 states that the experiments in [KIK+24] impose m ≤ n+2. For n ≥ 3 these conditions are incompatible; the only overlapping case is (m,n) = (4,2). Thus even if Heuristic 2.2 were fully justified, the theorem would not cover the experimental regime used in the prior work. The authors should prove a version for m < 2n or explicitly restrict their claims to the new setting.
- [Heuristic 2.2] The core density theorem is conditional on Heuristic 2.2, the irreducibility of X≤D, for which the paper offers no proof, no evidence, and no discussion of regimes where it might fail. Since this assumption is load-bearing for Lemma 4.10 and Theorem 4.11, and hence for Theorem 2.3, the paper's central conclusion is only an implication from an unverified algebraic-geometric statement. At minimum, the authors should prove or test irreducibility in nontrivial cases such as n=2, m=4, or give a structural argument based on the equations (BA−En)G=0.
- [Corollary 4.12 / Theorem 4.11] The proof that the flat locus Y is nonempty contains a gap. The sentence 'Let L be the field of fractions ... Then Spec L is isomorphic to a non-empty open subscheme V in R^{n×n}_{≤D}' is not correct: Spec L is the generic point and is not an open subscheme of a positive-dimensional affine space. Consequently the claim Y ≠ ∅ is not established as written. The gap appears repairable by invoking generic flatness for finite-type morphisms to a reduced Noetherian scheme, which gives a nonempty open V over which p is flat, but the current text does not supply that argument.
- [Problem 2.1 / Abstract] Problem 2.1 and the abstract formulate the result in terms of datasets being dense, but a finite training dataset is never Zariski dense in an infinite variety. What the proof actually establishes is that the set of all possible outputs of Algorithm 2 is dense in the set of all generators of ⟨G⟩. This distinction matters for the claimed learning guarantee: density of the generator's support is only a necessary condition for a finite sample to be representative, not a sufficient one. The paper should state this limitation explicitly.
minor comments (4)
- [§4.2] In the displayed definition of ilde F≤D before Lemma 4.10, the notation X_{≤D}^{m×n} is inconsistent with the definition of X_{≤D} in Heuristic 2.2; this appears to be a typo.
- [Lemma 4.8] In the proof, the displayed factorization uses the symbol u3 twice in the second factor; this is presumably a typo and should involve two distinct coefficients.
- [§3.1] There is a typo: 'basss' should be 'bases'.
- [§2.1] The notation F = {F ∈ R^m | ⟨F⟩ = ⟨G⟩} treats G as fixed, while Problem 2.1 presents G as an input; please make the dependence on G explicit in the notation throughout.
Assumptions & free parameters
assumptions (6)
- standard math Quillen-Suslin theorem (Serre's conjecture): finitely generated projective modules over K[x1,...,xr] are free.
- standard math Suslin's stability theorem: for m>=3, E(m) = <GL(K^m), SL(R^m)>.
- domain assumption Hilbert's irreducibility theorem holds for Hilbertian fields K.
- ad hoc to paper Heuristic 2.2: X≤D is irreducible for the given D and G.
- standard math Flat morphisms of finite type between Noetherian schemes are open (Hartshorne III.9.1).
- domain assumption Parameter constraints m>=2n>=3 and R=K[x1,...,xr] with K Hilbertian.
Cite this review
Pith. "Pith review of Geometric Generality of Transformer-Based Gr\"obner Basis Computation." pith.science (2026). https://pith.science/paper/KWPJGK4J
@misc{pith2026250412465,
author = {Pith},
title = {Pith review of: Geometric Generality of Transformer-Based Gr\"obner Basis Computation},
year = {2026},
howpublished = {\url{https://pith.science/paper/KWPJGK4J}},
note = {Machine review of arXiv:2504.12465}
}
read the original abstract
The intersection of deep learning and symbolic mathematics has seen rapid progress in recent years, exemplified by the work of Lample and Charton. They demonstrated that effective training of machine learning models for solving mathematical problems critically depends on high-quality, domain-specific datasets. In this paper, we address the computation of Gr\"obner basis using Transformers. While a dataset generation method tailored to Transformer-based Gr\"obner basis computation has previously been proposed, it lacked theoretical guarantees regarding the generality or quality of the generated datasets. In this work, we prove that datasets generated by the previously proposed algorithm are sufficiently general, enabling one to ensure that Transformers can learn a sufficiently diverse range of Gr\"obner bases. Moreover, we propose an extended and generalized algorithm to systematically construct datasets of ideal generators, further enhancing the training effectiveness of Transformer. Our results provide a rigorous geometric foundation for Transformers to address a mathematical problem, which is an answer to Lample and Charton's idea of training on diverse or representative inputs.
Reference graph
Works this paper leans on
-
[1]
Roberto Avanzi, Joppe Bos, L \'e o Ducas, Eike Kiltz, Tancr \`e de Lepoint, Vadim Lyubashevsky, John M Schanck, Peter Schwabe, Gregor Seiler, and Damien Stehl \'e , CRYSTALS-Kyber (version 3.02) , Submission to round 3 of the NIST post-quantum project, Available at https://pq-crystals.org/
-
[2]
Alberto Alfarano, Fran c ois Charton, and Amaury Hayat, Global lyapunov functions: a long-standing open problem in mathematics, with symbolic transformers, Advances in Neural Information Processing Systems 37 (2024), 93643--93670
work page 2024
-
[3]
Hirokazu Anai and Shinji Hara, Fixed-structure robust controller synthesis based on sign definite condition by a special quantifier elimination , Proceedings of the 2000 American Control Conference. ACC (IEEE Cat. No. 00CH36334), vol. 2, IEEE, 2000, pp. 1312--1316
work page 2000
-
[4]
B \^o cher, Introduction to Higher Algebra , Macmillan, 1907
M. B \^o cher, Introduction to Higher Algebra , Macmillan, 1907
work page 1907
-
[5]
Proceedings, Springer, 2006, pp
Johannes Buchmann, Andrei Pyshkin, and Ralf-Philipp Weinmann, Block ciphers sensitive to G r \"o bner Basis Attacks , Topics in Cryptology--CT-RSA 2006: The Cryptographers’ Track at the RSA Conference 2006, San Jose, CA, USA, February 13-17, 2005. Proceedings, Springer, 2006, pp. 313--331
-
[6]
Buchberger, A theoretical basis for the reduction of polynomials to canonical forms, SIGSAM Bull
Bruno. Buchberger, A theoretical basis for the reduction of polynomials to canonical forms, SIGSAM Bull. 10 (1976), no. 3, 19–29
work page 1976
-
[7]
Leandro Caniglia, Guillermo Corti \ n as, Silvia Dan \'o n, Joos Heintz, Teresa Krick, and Pablo Solern \'o , Algorithmic aspects of suslin's proof of serre's conjecture, computational complexity 3 (1993), 31--55
work page 1993
-
[8]
David Cox, John Little, and Donal O'Shea, Ideals, Varieties, and Algorithms , second ed., Undergraduate Texts in Mathematics, Springer-Verlag, New York, 1997, An introduction to computational algebraic geometry and commutative algebra. 1417938
work page 1997
Show all 38 references
-
[9]
Cox, John B
David A. Cox, John B. Little, and Donal O'Shea, Using Algebraic Geometry , first ed., Graduate Texts in Mathematics, vol. 185, Springer, 1998
1998
-
[10]
345--359
Nicolas T Courtois and Willi Meier, Algebraic Attacks on Stream Ciphers with Linear Feedback , Advances in Cryptology—EUROCRYPT 2003: International Conference on the Theory and Applications of Cryptographic Techniques, Warsaw, Poland, May 4--8, 2003 Proceedings 22, Springer, 2...
2003
-
[11]
150, Springer-Verlag, New York, 1995
David Eisenbud, Commutative Algebra with a View Toward Algebraic Geometry , Graduate Texts in Mathematics, vol. 150, Springer-Verlag, New York, 1995. 1322960
1995
-
[12]
1, 61--88
Jean-Charles Faug \`e re, A new efficient algorithm for computing G r\" o bner bases ( F 4) , Journal of Pure and Applied Algebra 139 (1999), no. 1, 61--88
1999
-
[13]
Jean-Charles Faugere and Antoine Joux, Algebraic Cryptanalysis of Hidden Field Equation (HFE) Cryptosystems Using Gr \"o bner Bases , Annual International Cryptology Conference, Springer, 2003, pp. 44--60
2003
-
[14]
Fried and Moshe Jarden, Field Arithmetic , 4th corrected edition ed., Ergeb
Michael D. Fried and Moshe Jarden, Field Arithmetic , 4th corrected edition ed., Ergeb. Math. Grenzgeb., 3. Folge, vol. 11, Cham: Springer, 2023 (English)
2023
-
[15]
Hiroki Furuta, Gouki Minegishi, Yusuke Iwasawa, and Yutaka Matsuo, Towards empirical interpretation of internal circuits and properties in grokked transformers on modular polynomials, 2024, arXiv preprint arXiv:2402.16726
2024 arXiv
-
[16]
247--257
Patrizia Gianni and Teo Mora, Algebraic solution of systems of polynomial equations using G r \"o bner bases , Applied Algebra, Algebraic Algorithms and Error-Correcting Codes (Berlin, Heidelberg), Springer Berlin Heidelberg, 1989, pp. 247--257
1989
-
[17]
Andrey Gromov, Grokking modular arithmetic, 2023, arXiv preprint arXiv:2301.02679
2023 arXiv
-
[18]
2-3, 149--167
Patrizia Gianni, Barry Trager, and Gail Zacharias, Gr \"o bner Bases and Primary Decomposition of Polynomial Ideals , Journal of Symbolic Computation 6 (1988), no. 2-3, 149--167
1988
-
[19]
Robin Hartshorne, Algebraic Geometry , Springer-Verlag, New York-Heidelberg, 1977, Graduate Texts in Mathematics, No. 52. 0463157
1977
-
[20]
Michael Hahn and Mark Rofin, Why are sensitive functions hard for transformers?, 2024, arXiv preprint arXiv:2402.09963
2024 arXiv
-
[21]
Hiroshi Kera, Yuki Ishihara, Yuta Kambe, Tristan Vaccon, and Kazuhiro Yokoyama, Learning to compute G r \"o bner bases , Advances in Neural Information Processing Systems 37 (2024), 33141--33187
2024
-
[22]
206--222
Aviad Kipnis, Jacques Patarin, and Louis Goubin, Unbalanced Oil and Vinegar signature Schemes , International Conference on the Theory and Applications of Cryptographic Techniques, Springer, 1999, pp. 206--222
1999
-
[23]
ed., Grad
Serge Lang, Algebra , 3rd revised ed. ed., Grad. Texts Math., vol. 211, New York, NY: Springer, 2002 (English)
2002
-
[24]
Guillaume Lample and Fran c ois Charton, Deep Learning for Symbolic Mathematics , 2019, arXiv preprint arXiv:1912.01412
2019 arXiv
-
[25]
Available at https://pq-crystals.org/
Vadim Lyubashevsky, L \'e o Ducas, Eike Kiltz, Tancr \`e de Lepoint, Peter Schwabe, Gregor Seiler, Damien Stehl \'e , and Shi Bai, CRYSTALS-Dilithium , Algorithm Specifications and Supporting Documentation (Version 3.1). Available at https://pq-crystals.org/
-
[26]
10, 882--891
Reinhard Laubenbacher and Bernd Sturmfels, Computer Algebra in Systems Biology , The American Mathematical Monthly 116 (2009), no. 10, 882--891
2009
-
[27]
Cathy Li, Jana Sotáková, Emily Wenger, Mohamed Malhou, Evrard Garcelon, Francois Charton, and Kristin Lauter, SALSA PICANTE: a machine learning attack on LWE with binary secrets , Cryptology ePrint Archive, Paper 2023/340, 2023, https://eprint.iacr.org/2023/340
2023
-
[28]
Cathy Li, Emily Wenger, Zeyuan Allen-Zhu, Francois Charton, and Kristin E Lauter, SALSA VERDE: a machine learning attack on Learning with Errors with sparse small secrets , Advances in Neural Information Processing Systems 36 (2023), 53343--53361
2023
-
[29]
3, 305--329
Ernst W Mayr and Albert R Meyer, The Complexity of the Word Problems for Commutative Semigroups and Polynomial Ideals , Advances in mathematics 46 (1982), no. 3, 305--329
1982
-
[30]
NIST, Post-Quantum Cryptography Standardization Selected Algorithms 2022 , https://csrc.nist.gov/Projects/post-quantum-cryptography/selected-algorithms-2022
2022
-
[31]
4, 579--590
Keith Pardue, Generic sequences of polynomials, Journal of Algebra 324 (2010), no. 4, 579--590
2010
-
[32]
6, 1--40
Oded Regev, On Lattices, Learning with Errors, Random Linear Codes, and Cryptography , Journal of the ACM (JACM) 56 (2009), no. 6, 1--40
2009
-
[33]
o bner bases using gr \
Akira Suzuki and Yosuke Sato, A simple algorithm to compute comprehensive gr \"o bner bases using gr \"o bner bases , Proceedings of the 2006 international symposium on Symbolic and Algebraic Computation, 2006, pp. 326--331
2006
-
[34]
2, 221--238
Andrey Aleksandrovich Suslin, On the structure of the special linear group over polynomial rings , Mathematics of the USSR - Izvestija 11 (1977), no. 2, 221--238
1977
-
[35]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin, Attention Is All You Need , Advances in neural information processing systems 30 (2017)
2017
-
[36]
Emily Wenger, Mingjie Chen, Francois Charton, and Kristin E Lauter, SALSA: Attacking Lattice Cryptography with Transformers , Advances in Neural Information Processing Systems 35 (2022), 34981--34994
2022
-
[37]
1, 1--29
Volker Weispfenning, Comprehensive gr \"o bner bases , Journal of symbolic computation 14 (1992), no. 1, 1--29
1992
-
[38]
393--419
Mizuki Yoshizawa, Akira Terui, and Masahiko Mikawa, Inverse Kinematics and Path Planning of Manipulator Using Real Quantifier Elimination Based on Comprehensive Gr \"o bner Systems , International Workshop on Computer Algebra in Scientific Computing, Springer, 2023, pp. 393--419
2023
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.