Studying number theory with deep learning: a case study with the M\"obius and squarefree indicator functions

David Lowry-Duda

REVIEW 2 major objections 5 minor 19 references

Studying number theory with deep learning: a case study with the M\"obius and squarefree indicator functions

T0 review · 2 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Small transformers trained on integers encoded as remainders modulo the first 100 primes predict squarefree status with about 70.6% accuracy, and their behavior reduces to checking divisibility by 2 and 3.

desk verdict A careful, honest case study: small transformers on CRT representations learn elementary mod-2/mod-3 divisibility statistics, not deep number-theoretic structure, and the paper's evidence supports that conclusion well. read the letter →

arxiv 2502.10335 v2 pith:LG5YODCV submitted 2025-02-14 math.NT cs.LG

David Lowry-Duda This is my paper · ORCID

classification math.NTcs.LG MSC 11A2568T0711N37

keywords MöbiusfunctionsquarefreeindicatorChineseremaindertheoremtransformernaturaldensityDirichletseriesfeaturecorruptionconditionalprobability

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether small transformers can learn to predict the Möbius function $\mu(n)$ and the squarefree indicator $\mu^2(n)$ when each integer is encoded not by its digits but by its remainders modulo the first 100 primes, a Chinese Remainder Theorem (CRT) representation chosen to hide easy divisibility clues. The paper shows that the models beat trivial baselines—about 70.6% accuracy on $\mu^2(n)$ versus 60.8% for always guessing 1—and then explains what they actually learned. They have effectively learned conditional densities: even numbers are more often divisible by a nontrivial square, while odd numbers are more often squarefree, and the rule 'even means squarefull, odd means squarefree' already achieves about 70.3% accuracy. The models gain no ability to distinguish $\mu(n)=1$ from $\mu(n)=-1$, so all predictive power reduces to detecting squarefullness. The paper's point is that a deliberately obfuscating encoding changes which trivial correlation a transformer finds, and that the resulting behavior can be explained exactly by natural densities.

What carries the argument

The central objects are the CRT representation—the length-200 sequence of pairs $(n \bmod p_j, p_j)$ for the first 100 primes—and the Dirichlet series $\prod_p(1+p^{-s}) = \zeta(s)/\zeta(2s)$ that encodes squarefree numbers. Perron's formula turns the pole at $s=1$ into the natural densities used throughout, and the parity-conditional probabilities derived from this series form the null model the transformers are compared against. The second mechanism is feature corruption: randomly replacing residues in a trained model's input and measuring accuracy isolates which residues carry the decision, showing that $n \bmod 2$ and $n \bmod 3$ explain almost all of its behavior.

What would settle it

Train a transformer on the same CRT representation with the residues modulo 2 and 3 deleted from the input. The paper's explanation predicts accuracy near the trivial 60.8% baseline for $\mu^2(n)$; if accuracy stays near the 70% level, the model is extracting signal from primes beyond 2 and 3 and the small-prime-divisibility story is incomplete.

Watch

Extended reading notes

Core claim

The central claim is that transformer models trained on CRT representations learn a conditional-probability approximation of $\mu^2(n)$, not an algorithm for $\mu(n)$. The full model reaches about 70.64% accuracy on $\mu^2(n)$ and about 50.98% on $\mu(n)$; feature-corruption experiments show that predictions depend almost entirely on $n \bmod 2$ and $n \bmod 3$. This is explained by Euler products and Perron's formula: the Dirichlet series for squarefree numbers is $\zeta(s)/\zeta(2s)$, giving squarefree density $6/\pi^2$, and restricting by parity gives $P(\text{squarefree}\mid\text{even}) = \frac{1}{3}\cdot \frac{6}{\pi^2} \big/ \frac12 \approx 0.4052$ and $P(\text{squarefree}\mid\text{odd}) = \frac{2}{3}\cdot \frac{6}{\pi^2} \big/ \frac12 \approx 0.8105$. The resulting rule—predict squarefull for even $n$, squarefree for odd $n$—yields about 70.26% for $\mu^2(n)$ and about 50% for $\mu(n)$, matching the models. A combinatorial computation using divisibility by the first 25 primes gives 70.34%, and training on squarefree $n$ alone drops $\mu$-prediction to chance, confirming that the models never learn the sign of $\mu(n)$.

Load-bearing premise

The claim that the model relies almost entirely on $n \bmod 2$ and $n \bmod 3$ rests on the assumption that randomly scrambling every other residue in the input isolates the causal contribution of those two remainders; if the transformer uses interactions between scrambled and preserved residues, the experiment could overstate how exclusively 2 and 3 matter.

Editorial extensions

If this is right

The squarefree indicator is, for this encoding and scale, essentially solved by the two residues n mod 2 and n mod 3: the parity rule already gives about 70.3% accuracy and the best full model gets 70.6%.
The sign of the Möbius function is not learned: when evaluation is restricted to squarefree n, accuracy on μ(n) falls to chance, so all predictive power lies in distinguishing squarefree from squarefull.
A combinatorial density computation using divisibility by the first 25 primes gives 70.34% accuracy, so the transformer's performance is explained by ordinary conditional probabilities rather than by reconstructing n or factoring.
Changing the input representation changes which trivial statistical regularity the model finds: the CRT encoding hides square divisibility, but squarefullness still leaks through small-prime divisibility.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct test of the paper's explanation would train on CRT inputs with n mod 2 and n mod 3 removed; the density analysis predicts accuracy near the trivial 60.8% baseline, and any sustained gain would indicate learning beyond those two primes.
The parity-only result suggests that researchers using transformers on number-theoretic functions should always compare against small-prime conditional-density baselines before claiming the model has discovered a nontrivial structure.
The paper's closing challenge—distinguishing $\mu(n)=1$ from $\mu(n)=-1$ using inputs much cheaper than factoring—seems to require information not contained in residue classes modulo small primes, since those classes carry essentially no signal for the sign of $\mu$.
The same CRT-plus-density analysis could be applied to other multiplicative functions whose values depend on divisibility by small primes; the Euler-product computation would provide the null model.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know two things about this paper. First, it delivers a genuinely clean negative-ish result: small transformers trained on CRT representations of integers learn to predict squarefreeness with about 70.6% accuracy, and that accuracy is essentially explained by a simple mod-6 rule—guess squarefull for even n, squarefree for odd n—which gets 70.26% from natural densities alone. Second, it is honest about the limits: the models cannot distinguish mu(n)=1 from mu(n)=-1, and all predictive power collapses to detecting divisibility by small primes. This is a useful cautionary tale for the ML-for-mathematics crowd, not a breakthrough that opens new number theory.

What is actually new: applying Charton's Int2Int architecture to mu and mu^2 with a CRT encoding, and then explaining the behavior with classical analytic number theory. The explanation has no fitted parameters: the conditional probabilities P(squarefree | even) and P(squarefree | odd) come straight from Euler products and Perron's formula. The paper also does several converging experiments—feature corruption, training only on (n mod 2, n mod 3), restricting to squarefree inputs—and they all point the same direction. That is good experimental hygiene.

Soft spots are minor and do not threaten the central claim. There are no error bars reported, which is annoying for a paper whose quantitative claims are so specific. The combinatorial calculation showing that using the first 25 primes gives 70.34% accuracy is asserted but not shown; I would want that spelled out or the code pointed to. The feature-corruption experiment is behavioral rather than mechanistic, but the dedicated (n mod 2, n mod 3) model directly validates that two residues suffice, so the concern does not bite. The Mobius Challenge at the end is a nice touch, though the time bound "≪ log^A(n)" is vague enough to be mostly rhetorical.

Who should read this: anyone training transformers on arithmetic functions, and number theorists who want a concrete check on what "Möbius randomness" does and does not imply for learnability. It is a solid benchmark paper. I would bring it to a reading group, and I would cite it if I were writing about ML and number-theoretic functions.

My recommendation: send it to serious peer review. It deserves a referee's time, and with minor revisions (error bars, the 25-prime calculation) it should be accepted. No desk reject.

Referee Report

2 major / 5 minor

Summary. The paper trains small transformer models on CRT representations of integers to predict the Möbius function μ(n) and the squarefree indicator μ²(n). The models achieve about 70.64% accuracy on μ²(n), well above the trivial baseline of 60.79%, and about 50.98% accuracy on μ(n). Through feature corruption experiments, training on only the residues modulo 2 and 3, and restriction to squarefree inputs, the paper argues that the models' predictive power comes almost entirely from detecting divisibility by small primes, chiefly 2 and 3. The theoretical explanation is a simple analytic strategy: for even n guess squarefull, for odd n guess squarefree, whose accuracy is about 70.26% by natural density. The paper also shows that the models cannot distinguish μ(n)=1 from μ(n)=−1 beyond this divisibility signal.

Significance. If the results hold, this is a valuable case study in interpretable machine learning for number theory. The paper demonstrates that a small transformer can learn a nontrivial statistical property of arithmetic functions, and that the learned behavior can be explained by classical density calculations from Euler products and Perron's formula. The theoretical explanation is parameter-free and is supported by multiple converging experiments: feature corruption, a dedicated (n mod 2, n mod 3) model, and a squarefree-only training regime. The associated code is available, which aids reproducibility. The 'Möbius Challenge' is a useful concrete open problem for the community. The main conclusion—that the models are effectively learning a parity-based heuristic—is robust, though one formula in the general theoretical derivation needs correction.

major comments (2)

[§3.3, general conditional probability formula] The displayed formula for the probability that n is squarefree given divisibility constraints is missing a factor. The density of squarefree numbers satisfying the constraints is correctly given as ∏_{p_i} 1/(p_i+1) ∏_{q_j} q_j/(q_j+1) · 6/π². However, dividing by the unconditional density of the constraints, ∏ 1/p_i ∏ (1−1/q_j), yields ∏ p_i/(p_i+1) ∏ q_j²/(q_j²−1) · 6/π², not the printed ∏ 1/(p_i+1) ∏ q_j²/(q_j²−1) · 6/π². For example, with p_1=2 and no q_j, the printed formula gives 2/π² ≈ 0.2027 instead of the correct 4/π² ≈ 0.4053, which contradicts the earlier correct computation of P(squarefree | even) in the same section. Please correct this formula and re-examine the subsequent claim about 70.34% accuracy for the first 25 primes.
[§3.3, 25-prime accuracy claim] The sentence 'Straightforward but tedious combinatorial analysis shows that using only divisibility by the first 25 primes leads to a strategy to approximate μ²(n) with accuracy 70.34%' is not backed by a derivation, a reference, or the code. Given that the general formula just preceding it appears to have a factor error, this quantitative claim is unverified as stated. The authors should supply the calculation (or a script) so that the reader can confirm the 70.34% figure and the consistency with the models' observed 70.64% accuracy.

minor comments (5)

[§2, results] The paper reports accuracies such as 70.64%, 70.1%, and 50.98% without error bars or multiple seeds. Since the central comparison with the 70.26% analytic baseline involves differences of a few tenths of a percent, the authors should state the standard deviation across independent runs or at least report the number of runs and the observed spread.
[§3.2, feature corruption] The text says 'there is almost no change when restricting to only use n mod 2 and n mod 3', but Figure 3 shows a drop from 69.72% to 68.15% for μ²(n) when only residues 2 and 3 are preserved. This is a small but visible decrease; please describe the magnitude more precisely.
[Table 1] Table 1 shows that for μ(n) the model correctly recognizes 2,305 of the +1 cases but only 1,203 of the −1 cases, which is not obviously 'random' behavior. The text in §2 and §3.2 says the models 'pick randomly' among μ(n)=±1; please clarify whether the model has a bias toward predicting +1, and reconcile this with the table.
[§4, Möbius Challenge] In the displayed challenge, 'inputs computable in time ≪ log A(n)' appears to contain a typo; it should likely read 'log^A n' for some finite A. Please correct this.
[§2, training details] The phrase 'within 2 epochs' or 'after each epoch (100,000 examples)' is slightly ambiguous. Please specify exactly how many epochs were trained and at which epoch the reported accuracies are measured.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the parity-based theoretical explanation is derived independently from classical densities, with no fitted parameters renamed as predictions.

full rationale

No circular steps are present. The paper's central empirical claim—small transformers on CRT encodings achieve about 70.64% accuracy on mu^2—is measured directly, and its theoretical explanation is computed independently from natural densities via Euler products and Perron's formula. The conditional probabilities P(sqfree|even) ≈ 0.4052 and P(sqfree|odd) ≈ 0.8105 yield the parity-only strategy accuracy of about 70.26%, and the first-25-primes calculation gives 70.34%; these numbers contain no constants fitted to model outputs and do not use the transformer's predictions as inputs. The feature-corruption experiment merely suggests which residues are salient, and the dedicated (n mod 2, n mod 3) model provides direct in-distribution confirmation without relying on the full model's internal parameters. Citations to Charton's architecture and code concern implementation, not the load-bearing number-theoretic explanation. The conclusion that the model cannot distinguish mu(n)=1 from mu(n)=-1 is supported by an independent restricted experiment on squarefree inputs. No equation reduces to another by construction, and no fitted parameter is renamed as a prediction.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central explanation uses no fitted parameters and no new entities. It relies on standard analytic number theory and on the domain assumption that the transformer's behavior reflects conditional probabilities from natural densities.

assumptions (4)

domain assumption Natural density defines the probability over random integers (Section 1.1, footnote 1).
The paper interprets 'probability' as limiting natural density, which underlies all computed accuracies and the theoretical strategy.
standard math Dirichlet series and Perron's formula relate poles to natural densities (Section 3.3).
Used to compute densities of squarefree numbers with divisibility constraints, following Montgomery-Vaughan.
standard math Euler product identity ∏_p (1 + p^{-s}) = ζ(s)/ζ(2s) (Section 3.3).
Yields the squarefree density 6/π² and the conditional densities for odd and even squarefree numbers.
domain assumption The model's decision rule is equivalent to thresholding conditional probabilities derived from these densities (Section 3.3).
The theoretical explanation assumes the model learns these conditional probabilities; supported by the match between predicted and observed accuracy, but not proven mechanistically.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Studying number theory with deep learning: a case study with the M\"obius and squarefree indicator functions." pith.science (2026). https://pith.science/paper/LG5YODCV

@misc{pith2026250210335,
  author       = {Pith},
  title        = {Pith review of: Studying number theory with deep learning: a case study with the M\"obius and squarefree indicator functions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LG5YODCV}},
  note         = {Machine review of arXiv:2502.10335}
}

abstract

Building on work of Charton, we train small transformer models to calculate the M\"{o}bius function $\mu(n)$ and the squarefree indicator function $\mu^2(n)$. The models attain nontrivial predictive power. We apply a mixture of additional models and feature scoring to give a theoretical explanation.

Figures

Figures reproduced from arXiv: 2502.10335 by the authors.

**Figure 1.** Accuracy in predicting µ(n) and µ 2 (n) from CRT representation. The left is for all n. The right is when restricted to squarefree n. µ(n) µ 2 (n) Output # in Eval Set # Recognized # in Eval Set # Recognized −1 6,056 1,203 - - 0 7,852 6,688 7,836 4,654 1 6,092 2,305 12,164 9,473 [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗

**Figure 2.** Accuracy in predicting µ 2 (n) using only (n mod 2, n mod 3) as input. Feature Corruption. One method to determine whether the best-performing models really care only about (n mod 2, n mod 3) is through feature corruption: take the models trained with the full CRT representation described in §2 and evaluate it on deliberately corrupted inputs. We compare the performance across five different inputs: (1) The true in… view at source ↗

**Figure 3.** Accuracy in predicting µ(n) and µ 2 (n) from corrupted inputs. Random n mod p means that in the CRT representation of n, the residue mod p is replaced with a uniform random integer mod p. And “only n mod 2, 3 correct” means that the residues mod each prime p ̸= 2, 3 were all replaced with uniform random integers mod p. The plot on the right visualizes the data in the table on the left. An application of Perron’s f… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 16 canonical work pages

[1]

Adleman and Kevin S

Leonard M. Adleman and Kevin S. McCurley. Open problems in number-theoretic complexity. II . In Algorithmic number theory ( I thaca, NY , 1994) , volume 877 of Lecture Notes in Comput. Sci. , pages 291--322. Springer, Berlin, 1994

work page 1994
[2]

Booker, Ghaith A

Andrew R. Booker, Ghaith A. Hiary, and Jon P. Keating. Detecting squarefree numbers. Duke Math. J. , 164(2):235--275, 2015

work page 2015
[3]

Transformers to Predict the Applicability of Symbolic Integration Routines

Rashid Barket, Uzma Shafiq, Matthew England, and Juergen Gerhard. Transformers to predict the applicability of symbolic integration routines. http://arxiv.org/abs/2410.23948v1, 2024. arXiv:cs.LG:2410.23948v1

work page Pith review arXiv 2024
[4]

Computing the roots of polynomials

Fran c ois Charton. Computing the roots of polynomials. https://f-charton.github.io/polynomial-roots/, 2022

work page 2022
[5]

Linear algebra with transformers

Fran c ois Charton. Linear algebra with transformers. Transactions on Machine Learning Research , 2022

work page 2022
[6]

Int2int, 2024

Fran c ois Charton. Int2int, 2024. https://github.com/f-charton/Int2Int/

work page 2024
[7]

Learning the greatest common divisor: explaining transformer predictions

Fran c ois Charton. Learning the greatest common divisor: explaining transformer predictions. In The Twelfth International Conference on Learning Representations , 2024

work page 2024
[8]

What does machine learning have to offer number theory? Available at https://youtu.be/jjA2CqY5Ang, 2024

Jordan Ellenberg. What does machine learning have to offer number theory? Available at https://youtu.be/jjA2CqY5Ang, 2024. Talk at the Number Theory Web Seminar

work page 2024

Show all 19 references

[9]

The M \"obius function is strongly orthogonal to nilsequences

Ben Green and Terence Tao. The M \"obius function is strongly orthogonal to nilsequences. Ann. of Math. (2) , 175(2):541--566, 2012

2012
[10]

Kreber, Markus Norman Rabe, and Bernd Finkbeiner

Christopher Hahn, Frederik Schmitt, Jens U. Kreber, Markus Norman Rabe, and Bernd Finkbeiner. Teaching temporal logics to neural networks. In International Conference on Learning Representations , 2021

2021
[11]

Adam: A method for stochastic optimization

Diederik P Kingma and Jimma Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014

2014 arXiv
[12]

Neural GPUs learn algorithms

Lukasz Kaiser and Ilya Sutskever. Neural GPUs learn algorithms. In International Conference on Learning Representations , 2016

2016
[13]

Code accompanying ``Studying number theory with deep learning: a case study with the M\" o bius and squarefree indicator functions'' , 2025

David Lowry-Duda. Code accompanying ``Studying number theory with deep learning: a case study with the M\" o bius and squarefree indicator functions'' , 2025. https://github.com/davidlowryduda/mobius_case_study

2025
[14]

Montgomery and Robert C

Hugh L. Montgomery and Robert C. Vaughan. Multiplicative number theory. I . C lassical theory , volume 97 of Cambridge Studies in Advanced Mathematics . Cambridge University Press, Cambridge, 2007

2007
[15]

Investigating the ability of neural networks to learn simple modular arithmetic

Theodoros Palamas. Investigating the ability of neural networks to learn simple modular arithmetic . MSc T hesis, University of Edinburgh, 2017

2017
[16]

Generative language modeling for automated theorem proving

Stanislas Polu and Ilya Sutskever. Generative language modeling for automated theorem proving. http://arxiv.org/abs/2009.03393v1, 2020. arXiv:cs.LG:2009.03393v1

2009 arXiv
[17]

Three lectures on M öbius randomness

Peter Sarnak. Three lectures on M öbius randomness. https://publications.ias.edu/sites/default/files/MobiusFunctionsLectures(2).pdf, 2011

2011
[18]

Transformer-based machine learning for fast SAT solvers and logic synthesis

Feng Shi, Chonghan Lee, Mohammad Khairul Bashar, Nikhil Shukla, Song-Chun Zhu, and Vijaykrishnan Narayanan. Transformer-based machine learning for fast SAT solvers and logic synthesis. http://arxiv.org/abs/2107.07116v1, 2021. arXiv:cs.NE:2107.07116v1

2021 arXiv
[19]

Number-theoretic functions which are equivalent to number of divisors

Jeffrey Shallit and Adi Shamir. Number-theoretic functions which are equivalent to number of divisors. Information Processing Letters , 20(3):151--153, 1985

1985

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Adleman and Kevin S

Leonard M. Adleman and Kevin S. McCurley. Open problems in number-theoretic complexity. II . In Algorithmic number theory ( I thaca, NY , 1994) , volume 877 of Lecture Notes in Comput. Sci. , pages 291--322. Springer, Berlin, 1994

work page 1994

[2] [2]

Booker, Ghaith A

Andrew R. Booker, Ghaith A. Hiary, and Jon P. Keating. Detecting squarefree numbers. Duke Math. J. , 164(2):235--275, 2015

work page 2015

[3] [3]

Transformers to Predict the Applicability of Symbolic Integration Routines

Rashid Barket, Uzma Shafiq, Matthew England, and Juergen Gerhard. Transformers to predict the applicability of symbolic integration routines. http://arxiv.org/abs/2410.23948v1, 2024. arXiv:cs.LG:2410.23948v1

work page Pith review arXiv 2024

[4] [4]

Computing the roots of polynomials

Fran c ois Charton. Computing the roots of polynomials. https://f-charton.github.io/polynomial-roots/, 2022

work page 2022

[5] [5]

Linear algebra with transformers

Fran c ois Charton. Linear algebra with transformers. Transactions on Machine Learning Research , 2022

work page 2022

[6] [6]

Int2int, 2024

Fran c ois Charton. Int2int, 2024. https://github.com/f-charton/Int2Int/

work page 2024

[7] [7]

Learning the greatest common divisor: explaining transformer predictions

Fran c ois Charton. Learning the greatest common divisor: explaining transformer predictions. In The Twelfth International Conference on Learning Representations , 2024

work page 2024

[8] [8]

What does machine learning have to offer number theory? Available at https://youtu.be/jjA2CqY5Ang, 2024

Jordan Ellenberg. What does machine learning have to offer number theory? Available at https://youtu.be/jjA2CqY5Ang, 2024. Talk at the Number Theory Web Seminar

work page 2024