{"id":"17c02373-bb8f-4d36-801c-2abe6d1afc5a","arxiv_id":"2502.10335","paper_version":2,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"Transformers can predict squarefree numbers with around 70% accuracy from CRT encodings, but only by exploiting divisibility by 2 and 3, and they cannot separate the two signs of the Möbius function.","lead":"Researchers trained small AI models to guess two number-theory properties, the Möbius function and whether a number is squarefree, using a Chinese-remainder encoding of the input. The models beat random guessing, but their success comes from a simple parity rule, not from deep number theory.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No significant objection identified: the parity-based explanation is robust, and the feature-corruption concern is mitigated by the dedicated (n mod 2, n mod 3) model.","rationale":"The reader's verdict of ACCEPT with high confidence is appropriate. The paper's central claim is modest and well supported: a small transformer trained on CRT encodings reaches about 70.6% accuracy on mu^2, and a parameter-free density calculation shows that the simple rule 'even means squarefull, odd means squarefree' already yields about 70.26%. The dedicated model trained only on (n mod 2, n mod 3) reaches 70.1%, matching the theoretical baseline and the full model within a percentage point. This direct experiment is stronger evidence for the 'chiefly 2 and 3' explanation than the corruption experiment alone, so the reader's flagged weakest assumption is mitigated. The unshown '25 primes' calculation and the missing error bars are minor and do not affect the main conclusion. The inability to distinguish mu=1 from mu=-1 is honestly presented as a null result and is not load-bearing for the paper's contribution. A parity-only training run would sharpen the quantitative claim but is not required to sustain the verdict.","tokens_in":7835,"tokens_out":18749,"duration_ms":175585,"concrete_test":"Run the released evaluation set through a model trained on the single residue n mod 2 and compare its accuracy to the theoretical 70.26% and to the reported 70.64%; if the parity-only model matches within sampling error, the 'chiefly 2' explanation is quantitatively confirmed, and any residual gap can be attributed to higher-order use of other residues.","verdict_should_be":"UNCHANGED","load_bearing_attack":"No significant objection identified. The central claim—small transformers on CRT encodings learn to predict mu^2 at about 70.6%, explained by detecting divisibility by small primes, chiefly 2 and 3—is well supported. The parity-only analytic baseline (about 70.26%) nearly matches the best model (70.64%) and the dedicated (n mod 2, n mod 3) model (70.1%). The feature-corruption experiment is the most interpretative step, but the dedicated model provides direct in-distribution evidence that residues 2 and 3 suffice, so the causal-attribution concern does not transfer to the central claim. The remaining gap between the full and corruption-preserved models (about 1.5 points) and the absence of error bars do not threaten the main conclusion, which is the existence and explanation of nontrivial predictive power over the 60.79% trivial baseline.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper trains small transformer models on CRT representations of integers to predict the Möbius function μ(n) and the squarefree indicator μ²(n). The models achieve about 70.64% accuracy on μ²(n), well above the trivial baseline of 60.79%, and about 50.98% accuracy on μ(n). Through feature corruption experiments, training on only the residues modulo 2 and 3, and restriction to squarefree inputs, the paper argues that the models' predictive power comes almost entirely from detecting divisibility by small primes, chiefly 2 and 3. The theoretical explanation is a simple analytic strategy: for even n guess squarefull, for odd n guess squarefree, whose accuracy is about 70.26% by natural density. The paper also shows that the models cannot distinguish μ(n)=1 from μ(n)=−1 beyond this divisibility signal.","tokens_in":7930,"tokens_out":9580,"duration_ms":83620,"significance":"If the results hold, this is a valuable case study in interpretable machine learning for number theory. The paper demonstrates that a small transformer can learn a nontrivial statistical property of arithmetic functions, and that the learned behavior can be explained by classical density calculations from Euler products and Perron's formula. The theoretical explanation is parameter-free and is supported by multiple converging experiments: feature corruption, a dedicated (n mod 2, n mod 3) model, and a squarefree-only training regime. The associated code is available, which aids reproducibility. The 'Möbius Challenge' is a useful concrete open problem for the community. The main conclusion—that the models are effectively learning a parity-based heuristic—is robust, though one formula in the general theoretical derivation needs correction.","major_comments":[{"comment":"The displayed formula for the probability that n is squarefree given divisibility constraints is missing a factor. The density of squarefree numbers satisfying the constraints is correctly given as ∏_{p_i} 1/(p_i+1) ∏_{q_j} q_j/(q_j+1) · 6/π². However, dividing by the unconditional density of the constraints, ∏ 1/p_i ∏ (1−1/q_j), yields ∏ p_i/(p_i+1) ∏ q_j²/(q_j²−1) · 6/π², not the printed ∏ 1/(p_i+1) ∏ q_j²/(q_j²−1) · 6/π². For example, with p_1=2 and no q_j, the printed formula gives 2/π² ≈ 0.2027 instead of the correct 4/π² ≈ 0.4053, which contradicts the earlier correct computation of P(squarefree | even) in the same section. Please correct this formula and re-examine the subsequent claim about 70.34% accuracy for the first 25 primes.","section":"§3.3, general conditional probability formula"},{"comment":"The sentence 'Straightforward but tedious combinatorial analysis shows that using only divisibility by the first 25 primes leads to a strategy to approximate μ²(n) with accuracy 70.34%' is not backed by a derivation, a reference, or the code. Given that the general formula just preceding it appears to have a factor error, this quantitative claim is unverified as stated. The authors should supply the calculation (or a script) so that the reader can confirm the 70.34% figure and the consistency with the models' observed 70.64% accuracy.","section":"§3.3, 25-prime accuracy claim"}],"minor_comments":[{"comment":"The paper reports accuracies such as 70.64%, 70.1%, and 50.98% without error bars or multiple seeds. Since the central comparison with the 70.26% analytic baseline involves differences of a few tenths of a percent, the authors should state the standard deviation across independent runs or at least report the number of runs and the observed spread.","section":"§2, results"},{"comment":"The text says 'there is almost no change when restricting to only use n mod 2 and n mod 3', but Figure 3 shows a drop from 69.72% to 68.15% for μ²(n) when only residues 2 and 3 are preserved. This is a small but visible decrease; please describe the magnitude more precisely.","section":"§3.2, feature corruption"},{"comment":"Table 1 shows that for μ(n) the model correctly recognizes 2,305 of the +1 cases but only 1,203 of the −1 cases, which is not obviously 'random' behavior. The text in §2 and §3.2 says the models 'pick randomly' among μ(n)=±1; please clarify whether the model has a bias toward predicting +1, and reconcile this with the table.","section":"Table 1"},{"comment":"In the displayed challenge, 'inputs computable in time ≪ log A(n)' appears to contain a typo; it should likely read 'log^A n' for some finite A. Please correct this.","section":"§4, Möbius Challenge"},{"comment":"The phrase 'within 2 epochs' or 'after each epoch (100,000 examples)' is slightly ambiguous. Please specify exactly how many epochs were trained and at which epoch the reported accuracies are measured.","section":"§2, training details"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid and interesting contribution, and the central explanation via parity is convincing. The main issue is the incorrect conditional-probability formula in §3.3, which appears to be a genuine mathematical error. It is easily fixable, but it is load-bearing for the secondary claim of matching the model's accuracy with the first 25 primes. I recommend major revision rather than rejection because the correction can be made within the manuscript's scope. The other comments are minor presentation issues."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper. First, it delivers a genuinely clean negative-ish result: small transformers trained on CRT representations of integers learn to predict squarefreeness with about 70.6% accuracy, and that accuracy is essentially explained by a simple mod-6 rule—guess squarefull for even n, squarefree for odd n—which gets 70.26% from natural densities alone. Second, it is honest about the limits: the models cannot distinguish mu(n)=1 from mu(n)=-1, and all predictive power collapses to detecting divisibility by small primes. This is a useful cautionary tale for the ML-for-mathematics crowd, not a breakthrough that opens new number theory.\n\nWhat is actually new: applying Charton's Int2Int architecture to mu and mu^2 with a CRT encoding, and then explaining the behavior with classical analytic number theory. The explanation has no fitted parameters: the conditional probabilities P(squarefree | even) and P(squarefree | odd) come straight from Euler products and Perron's formula. The paper also does several converging experiments—feature corruption, training only on (n mod 2, n mod 3), restricting to squarefree inputs—and they all point the same direction. That is good experimental hygiene.\n\nSoft spots are minor and do not threaten the central claim. There are no error bars reported, which is annoying for a paper whose quantitative claims are so specific. The combinatorial calculation showing that using the first 25 primes gives 70.34% accuracy is asserted but not shown; I would want that spelled out or the code pointed to. The feature-corruption experiment is behavioral rather than mechanistic, but the dedicated (n mod 2, n mod 3) model directly validates that two residues suffice, so the concern does not bite. The Mobius Challenge at the end is a nice touch, though the time bound \"≪ log^A(n)\" is vague enough to be mostly rhetorical.\n\nWho should read this: anyone training transformers on arithmetic functions, and number theorists who want a concrete check on what \"Möbius randomness\" does and does not imply for learnability. It is a solid benchmark paper. I would bring it to a reading group, and I would cite it if I were writing about ML and number-theoretic functions.\n\nMy recommendation: send it to serious peer review. It deserves a referee's time, and with minor revisions (error bars, the 25-prime calculation) it should be accepted. No desk reject.","headline":"A careful, honest case study: small transformers on CRT representations learn elementary mod-2/mod-3 divisibility statistics, not deep number-theoretic structure, and the paper's evidence supports that conclusion well.","tokens_in":8488,"tokens_out":1282,"would_cite":true,"duration_ms":14479,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["11A25","68T07","11N37"],"pacs":[],"model":"deepseek-v4-flash","headline":"Small transformers trained on integers encoded as remainders modulo the first 100 primes predict squarefree status with about 70.6% accuracy, and their behavior reduces to checking divisibility by 2 and 3.","keywords":["Möbius function","squarefree indicator","Chinese remainder theorem","transformer","natural density","Dirichlet series","feature corruption","conditional probability"],"falsifier":"Train a transformer on the same CRT representation with the residues modulo 2 and 3 deleted from the input. The paper's explanation predicts accuracy near the trivial 60.8% baseline for $\\mu^2(n)$; if accuracy stays near the 70% level, the model is extracting signal from primes beyond 2 and 3 and the small-prime-divisibility story is incomplete.","tokens_in":7594,"feed_emoji":"🧮","tokens_out":20137,"duration_ms":174809,"temperature":0.7,"pith_summary":"This paper asks whether small transformers can learn to predict the Möbius function $\\mu(n)$ and the squarefree indicator $\\mu^2(n)$ when each integer is encoded not by its digits but by its remainders modulo the first 100 primes, a Chinese Remainder Theorem (CRT) representation chosen to hide easy divisibility clues. The paper shows that the models beat trivial baselines—about 70.6% accuracy on $\\mu^2(n)$ versus 60.8% for always guessing 1—and then explains what they actually learned. They have effectively learned conditional densities: even numbers are more often divisible by a nontrivial square, while odd numbers are more often squarefree, and the rule 'even means squarefull, odd means squarefree' already achieves about 70.3% accuracy. The models gain no ability to distinguish $\\mu(n)=1$ from $\\mu(n)=-1$, so all predictive power reduces to detecting squarefullness. The paper's point is that a deliberately obfuscating encoding changes which trivial correlation a transformer finds, and that the resulting behavior can be explained exactly by natural densities.","feed_headline":"Transformers predict squarefree numbers at 70.6%","feed_subtitle":"A small transformer on modular remainders matches a simple even/odd rule, and never learns the sign of μ.","key_machinery":"The central objects are the CRT representation—the length-200 sequence of pairs $(n \\bmod p_j, p_j)$ for the first 100 primes—and the Dirichlet series $\\prod_p(1+p^{-s}) = \\zeta(s)/\\zeta(2s)$ that encodes squarefree numbers. Perron's formula turns the pole at $s=1$ into the natural densities used throughout, and the parity-conditional probabilities derived from this series form the null model the transformers are compared against. The second mechanism is feature corruption: randomly replacing residues in a trained model's input and measuring accuracy isolates which residues carry the decision, showing that $n \\bmod 2$ and $n \\bmod 3$ explain almost all of its behavior.","core_discovery":"The central claim is that transformer models trained on CRT representations learn a conditional-probability approximation of $\\mu^2(n)$, not an algorithm for $\\mu(n)$. The full model reaches about 70.64% accuracy on $\\mu^2(n)$ and about 50.98% on $\\mu(n)$; feature-corruption experiments show that predictions depend almost entirely on $n \\bmod 2$ and $n \\bmod 3$. This is explained by Euler products and Perron's formula: the Dirichlet series for squarefree numbers is $\\zeta(s)/\\zeta(2s)$, giving squarefree density $6/\\pi^2$, and restricting by parity gives $P(\\text{squarefree}\\mid\\text{even}) = \\frac{1}{3}\\cdot \\frac{6}{\\pi^2} \\big/ \\frac12 \\approx 0.4052$ and $P(\\text{squarefree}\\mid\\text{odd}) = \\frac{2}{3}\\cdot \\frac{6}{\\pi^2} \\big/ \\frac12 \\approx 0.8105$. The resulting rule—predict squarefull for even $n$, squarefree for odd $n$—yields about 70.26% for $\\mu^2(n)$ and about 50% for $\\mu(n)$, matching the models. A combinatorial computation using divisibility by the first 25 primes gives 70.34%, and training on squarefree $n$ alone drops $\\mu$-prediction to chance, confirming that the models never learn the sign of $\\mu(n)$.","pith_inferences":["A direct test of the paper's explanation would train on CRT inputs with n mod 2 and n mod 3 removed; the density analysis predicts accuracy near the trivial 60.8% baseline, and any sustained gain would indicate learning beyond those two primes.","The parity-only result suggests that researchers using transformers on number-theoretic functions should always compare against small-prime conditional-density baselines before claiming the model has discovered a nontrivial structure.","The paper's closing challenge—distinguishing $\\mu(n)=1$ from $\\mu(n)=-1$ using inputs much cheaper than factoring—seems to require information not contained in residue classes modulo small primes, since those classes carry essentially no signal for the sign of $\\mu$.","The same CRT-plus-density analysis could be applied to other multiplicative functions whose values depend on divisibility by small primes; the Euler-product computation would provide the null model."],"forward_implications":["The squarefree indicator is, for this encoding and scale, essentially solved by the two residues n mod 2 and n mod 3: the parity rule already gives about 70.3% accuracy and the best full model gets 70.6%.","The sign of the Möbius function is not learned: when evaluation is restricted to squarefree n, accuracy on μ(n) falls to chance, so all predictive power lies in distinguishing squarefree from squarefull.","A combinatorial density computation using divisibility by the first 25 primes gives 70.34% accuracy, so the transformer's performance is explained by ordinary conditional probabilities rather than by reconstructing n or factoring.","Changing the input representation changes which trivial statistical regularity the model finds: the CRT encoding hides square divisibility, but squarefullness still leaks through small-prime divisibility."],"supporting_citations":[{"why":"It supplies the transformer architecture, training setup, and the base-distinguishing behavior that this paper adapts and compares against.","marker":"[Cha24b]"},{"why":"It provides Perron's formula, the tool used to convert the Dirichlet series for squarefree sets into the natural densities at the heart of the explanation.","marker":"[MV07]"},{"why":"It documents the trivial 'check divisibility by small prime squares' strategy that motivates the CRT encoding as a deliberate obfuscation.","marker":"[Ell24]"},{"why":"It makes the paper's primary experiments reproducible by releasing the code that produced the accuracy figures.","marker":"[LD25]"}],"fun_headline_variants":["Transformers learn parity shortcut for squarefree","AI learns parity, not Möbius sign","Squarefree transformer: parity is the whole story","AI misses Möbius sign, nails squarefree parity"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that the model relies almost entirely on $n \\bmod 2$ and $n \\bmod 3$ rests on the assumption that randomly scrambling every other residue in the input isolates the causal contribution of those two remainders; if the transformer uses interactions between scrambled and preserved residues, the experiment could overstate how exclusively 2 and 3 matter.","fun_headline_variants_meta":{"raw":{"variants":["Transformers learn parity shortcut for squarefree","AI learns parity, not Möbius sign","Squarefree transformer: parity is the whole story","AI misses Möbius sign, nails squarefree parity"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001695,"raw_usage":{"total_tokens":6697,"prompt_tokens":912,"completion_tokens":5785,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":528,"completion_tokens_details":{"reasoning_tokens":5732}},"tokens_in":528,"tokens_out":5785,"duration_ms":39117,"temperature":1.0,"reasoning_tokens":5732,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T18:29:03.912996+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a transformer on the same CRT representation with the residues modulo 2 and 3 deleted from the input. The paper's explanation predicts accuracy near the trivial 60.8% baseline for $\\mu^2(n)$; if accuracy stays near the 70% level, the model is extracting signal from primes beyond 2 and 3 and the small-prime-divisibility story is incomplete.","supporting_citations":[],"review_version":1}