{"id":"c6562b6a-d3c5-48e1-8d56-8775d429436f","arxiv_id":"2502.10357","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Neural networks predict the parity of Frobenius traces of elliptic curves from other traces with high accuracy, and their embeddings reveal learned mod-2 and mod-4 structure.","lead":"This paper tests whether transformer and feedforward neural networks can predict a number attached to elliptic curves, the Frobenius trace, from other nearby traces. The models predict whether the trace is even or odd with about 94 percent accuracy, and their internal embeddings visibly separate curves by parity.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported accuracies may be inflated by train/test overlap on identical input tuples; the paper quantifies this leakage only for mod-2 inputs (Section 4.2) but not for the full-trace experiments in Sections 3 and 4.1.","rationale":"The reader's weakest_assumption identifies exactly the same load-bearing concern: the train/test split is by isogeny class rather than by input tuple, so duplicate input sequences can leak labels from training into test. This concern is the most direct threat to the paper's central empirical claim, because if the reported accuracies are substantially inflated by such leakage, the claim that transformers have learned the finite modular/Galois structure would not be supported. The paper's internal admission in Section 4.2 — that over 80% of test data is seen in training for mod-2 inputs — makes the concern concrete and empirically grounded. The proposed test is a straightforward computational check that would settle whether the concern lands. I agree with the reader's conditional verdict: the paper is promising and clearly written, but the missing overlap analysis, together with absent error bars and classical baselines, prevents full acceptance. No change to the reader's verdict is needed; the conditional status already captures the need for this check.","tokens_in":11592,"tokens_out":4095,"duration_ms":43133,"concrete_test":"For each p in Section 4.1 (and for p = 2, 3, 97 in Section 3), reconstruct the exact training/test split from the published description, count how many test input tuples (a_q)_{q<100, q != p} also occur in the training set, and recompute accuracy/MCC on the subset of test tuples that are absent from the training set. If accuracy on these novel tuples drops materially (e.g., parity accuracy from ~0.94 toward 0.5, or exact a_97 accuracy toward baseline), the reported numbers reflect memorization of duplicate tuples rather than generalization. Also report the fraction of test tuples whose label conflicts with the training label, which would reveal indeterminacy of the target mapping.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that transformers achieve high accuracy in predicting Frobenius traces and their parity, which requires that test accuracy measure generalization to new input tuples. The data are split by isogeny class (Section 3.1), not by the input tuple (a_q)_{q<100}, yet distinct isogeny classes can share the same tuple. The paper itself demonstrates the severity of this issue for mod-2 inputs: Section 4.2 states that after reduction modulo 2, more than 80% of the test data appears in the training set, and that duplicate removal was necessary. For the full-trace experiments in Sections 3 and 4.1, no duplicate removal or overlap quantification is reported. Consequently, the reported results — e.g., a_p mod 2 accuracy around 0.94 (Figure 5) and a_97 exact accuracy 0.4923 against a 0.082 baseline (Table 1) — could be inflated by the model memorizing tuple-to-label mappings that recur in the test set. Because Section 4.2 explicitly acknowledges the identical mechanism for mod-2 inputs, the same mechanism must be presumed active, to an unknown degree, for full-trace inputs. The headline claim of learning the underlying modular/Galois structure is therefore conditional on an unmeasured overlap statistic.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper trains encoder-only transformers and feedforward neural networks on datasets of isogeny classes of elliptic curves over Q (conductors up to 10^7) to predict Frobenius traces a_p, or their residues mod 2, from traces at other small primes q < 100. The main empirical claims are that exact a_p prediction for p = 97 reaches accuracy 0.4923 against a baseline of 0.082, that a_p mod 2 prediction from a_q mod 2 reaches accuracies near 0.94 and MCC near 0.84, and that PCA and saliency analyses suggest the models implicitly learn mod-2 and mod-4 structure. The paper interprets these results as evidence that neural networks can learn relationships among Euler factors without explicit number-theoretic tools such as functional equations.","tokens_in":11826,"tokens_out":4880,"duration_ms":52770,"significance":"If the quantitative claims are robust, this is a useful experimental data point for the machine-learning/number-theory interface: it would show that standard transformer architectures can pick up congruence structure in Frobenius traces and expose it in learned embeddings. The paper is also explicit about some limitations, such as the overlap problem for mod-2 inputs. However, the strength of the claims depends critically on whether test accuracy measures generalization to genuinely new input tuples, and the current manuscript does not establish this for the exact-trace experiments. The PCA and saliency sections are suggestive but informal; they do not by themselves carry the central quantitative claim.","major_comments":[{"comment":"The train/test split is by isogeny class, not by the input tuple (a_q)_{q<100, q≠p}. Section 4.2 demonstrates that this distinction matters: after reducing inputs modulo 2, more than 80% of the test data appears in the training set, and the authors had to remove duplicates before reporting results. For the exact-trace experiments in Sections 3 and 4.1, no duplicate removal or overlap statistic is reported. If identical input tuples recur in both training and test, then the reported accuracies and MCCs (e.g., a_97 accuracy 0.4923 in Table 1, parity accuracies near 0.94 in Figure 5) may reflect memorization of tuple-to-label mappings rather than generalization to new trace sequences. The authors should quantify, for each experiment, the fraction of test tuples that also occur in the training set, and should report results after deduplication or after explicitly removing overlapping tuples from the test set.","section":"§3.1, §3.2, §4.1, §4.2"},{"comment":"The paper reports 'maximum' test accuracy and 'maximum' test MCC obtained during training, but it does not specify whether the checkpoint used for evaluation was selected on a validation set or on the test set. Selecting the best epoch by test performance inflates the reported numbers and makes the results non-reproducible without additional protocol details. The authors should report early stopping on a validation split, or else provide the full training curve and state explicitly that the maximum is over validation epochs. In addition, no repeated-seed experiments or confidence intervals are reported, so it is unclear how stable the numbers in Table 1 and Figure 5 are.","section":"Table 1, Table 3, Figure 5, Figure 6"},{"comment":"The handling of indeterminate cases after duplicate removal is ambiguous. The text says that the dataset is balanced and that between 5% and 11.5% of the training set consists of indeterminate cases (same input tuple with different a_p mod 2 labels). It is not stated whether the test set likewise contains indeterminate cases, nor how predictions are scored when the input admits two valid labels. If indeterminate cases are present in the test set, the achievable accuracy is bounded by the majority fraction for those tuples, and reporting accuracy without separating deterministic and indeterminate inputs makes the parity results difficult to interpret. The authors should specify the evaluation rule and report results separately for deterministic and indeterminate test cases.","section":"§4.2"}],"minor_comments":[{"comment":"There is a typo: 'the the Mathews correlation coefficient' should be 'the Matthews correlation coefficient', and the attribution should be to Matthews.","section":"§2.4.2"},{"comment":"The caption contains a stray '5,' in 'The highest MCC, 5, 0.8703'; this should read 'The highest MCC, 0.8703'.","section":"Figure 5 caption"},{"comment":"The introduction says models predict a_97 from a_q with q < 97, whereas Section 3.1 specifies the input as (a_q)_{q≠p, q≤100}; these descriptions should be reconciled.","section":"§1 and §3.1"},{"comment":"The convention 'a ≡ b (mod 0) ⇔ a = b' is nonstandard and should be explained more clearly in the main text rather than in a footnote-like parenthetical.","section":"§5"},{"comment":"The caption 'Primes excluded' is unclear: it should state explicitly that the model's input excludes a_q for the listed primes while predicting a_97 from the remaining a_q values.","section":"Table 3"},{"comment":"The reference [AHL+] is listed as 'In preparation'; if it is used as context, the authors should provide a stable reference or remove it.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The central idea is interesting and the paper is honest about several limitations, but the evaluation protocol needs substantial clarification and additional experiments before the quantitative claims can be accepted. The overlap issue is explicitly demonstrated in Section 4.2 for mod-2 inputs, and the same issue must be addressed for the exact-trace experiments rather than assumed negligible. The checkpoint-selection issue also affects all headline numbers. I would be willing to reconsider after a revision that reports overlap statistics, deduplicated results, and validation-based model selection with repeated runs."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this paper shows transformers can predict a_p mod 2 from neighboring traces at scale, and the PCA/embedding evidence for mod-2 structure is new and plausible. But the reported accuracies are not trustworthy as stated, because the train/test split is by isogeny class, not by input tuple, and the paper only quantifies the resulting overlap for the mod-2-input experiments. In Section 4.2 the authors admit that after reducing modulo 2, more than 80% of test inputs appear in the training set. The same mechanism is unmeasured for the full-trace experiments in Sections 3 and 4.1, so the 0.94 accuracy and 0.84 MCC likely overstate genuine generalization to new trace sequences.\n\nWhat is actually new: nobody has run this exact task—predicting a_p or its parity from the other a_q with transformers—and the embedding visualizations showing mod-2/mod-4 clustering are a nice interpretability result. The number theory is classical and correctly framed (Serre's theorem, finite mod-2 Galois representations), and the authors are refreshingly honest about the indeterminacy introduced by reducing modulo 2.\n\nSoft spots, in proportion: (1) The leakage issue above is the main one. It does not sink the qualitative claim—parity is theoretically determined by finitely many traces, and the model may well be learning that—but it does invalidate the current quantitative numbers. A simple fix: report the fraction of test tuples that also appear in training for the full-trace experiments, or re-run with tuple-level splitting. (2) No repeated seeds or confidence intervals, and the \"maximum MCC across epochs\" is selected without clarifying whether that is validation or test-based selection. Minor but relevant. (3) No classical baseline—comparing to the majority class (0.082 for a_97) is not enough; a k-nearest-neighbor or logistic regression on the same inputs would calibrate how much the transformer actually adds. (4) The exact-trace results are weak (0.49 for a_97), and the authors admit sign confusion; fine, but it tempers the \"learning Euler factors\" framing.\n\nMy verdict: this deserves a serious referee. The writing is clear, the data pipeline is reproducible, and the authors show integrity in flagging the mod-2 overlap problem. But the revision must close the leakage gap and add error bars before the numbers are quotable.\n\nFor your reading group: a maybe. I would not cite it in my own work until the overlap statistic is reported.","headline":"A promising ML-plus-number-theory proof-of-concept whose headline accuracies are compromised by an unmeasured train/test overlap on repeated input tuples.","tokens_in":12408,"tokens_out":1804,"would_cite":false,"duration_ms":19178,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["11G05","11G40","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"An encoder-only transformer trained on Frobenius traces $a_q$ for $q<100$, $q\\neq p$ predicts $a_p \\bmod 2$ with around $0.94$ accuracy and MCC around $0.84$ for most primes $p<100$, without access to L-functions or functional equations.","keywords":["Frobenius traces","elliptic curves","Euler factors","transformers","modular arithmetic","L-functions","isogeny classes","machine learning"],"falsifier":"Build the test set for the $a_p \\bmod 2$ task by discarding every input tuple $(a_q)_{q\\neq p,\\ q<100}$ that also appears in training, then retrain; if accuracy on that tuple-disjoint test set is near $0.5$, the claimed parity learning is an artifact of overlap.","tokens_in":11353,"feed_emoji":"🧮","tokens_out":11822,"duration_ms":105323,"temperature":0.7,"pith_summary":"The paper asks whether a machine-learning model can determine one Euler factor of an elliptic curve from the others. Given the sequence of Frobenius traces $a_q$ for $q<100$, $q\\neq p$, an encoder-only transformer trained on isogeny classes (curves related by isogeny) with conductors below $10^6$ predicts $a_p$ with accuracy far above chance, and its prediction of $a_p \\bmod 2$ reaches about $0.94$ accuracy and a Matthews correlation coefficient near $0.84$ for most $p<100$. Notably, the models receive no L-functions, functional equations, or explicit Galois-representation input, so the authors interpret the performance as evidence that the models learn the congruence structure of Frobenius traces. They support this with confusion matrices and PCA of the learned embeddings, which separate by parity and modulo-4 classes. The result matters because no efficient classical algorithm is known for completing a finite trace sequence, despite a theorem of Serre guaranteeing the sequence determines the isogeny class.","feed_headline":"Neural nets learn elliptic-curve Frobenius parity to 94%","feed_subtitle":"Trained only on neighboring prime traces, the model recovers a_p mod 2 without L-functions or functional equations.","key_machinery":"The load-bearing object is the Frobenius trace $a_p(E) = p + 1 - |E(\\mathbb{F}_p)|$, the coefficient in the local Euler factor $1/(1 - a_p p^{-s} + p^{1 - 2s})$. The argument is carried by an encoder-only transformer whose vocabulary tokenizes signed integers; it reads the sequence $(a_q)_{q\\neq p,\\ q<100}$ and is trained, in the parity experiments, on targets reduced modulo $2$. Two auxiliary mechanisms are essential: the dataset of isogeny classes with conductors below $10^7$, and the modulo-2 duplicate-removal and balancing procedure that separates the training and test sets. The interpretability step uses PCA of token embeddings and of decoder hidden states to show that the model organizes its representations by residues modulo $2$ and $4$.","core_discovery":"On the paper's own terms, the central discovery is that a transformer can learn the arithmetic of Frobenius traces from data alone. For the exact-value task, $a_{97}$ is predicted with test MCC $0.4711$ against a most-common-class baseline of $0.082$, and the model captures $|a_{97}|$ better than the sign (sign-agnostic MCC $0.6266$); $a_2$ and $a_3$ show the same pattern. When the target is reduced modulo $2$, the encoder-only transformer reaches accuracies close to $0.94$ and MCCs around $0.84$ for most primes $p<100$ (accuracy $0.9472$ at $p=83$, MCC $0.8703$ at $p=5$), with the lowest results at $p=2$. The authors read this as evidence that the model implicitly computes $a_p \\bmod 2$ as an intermediate step, and they corroborate it by showing confusion matrices consistent with parity classes and PCA projections of embeddings and decoder hidden states that cluster by residues modulo $2$, $4$, and in one case $6$.","pith_inferences":["Read together with the paper's own Section 4.2, which reports that reducing inputs modulo 2 makes over 80% of test tuples appear in training, the Section 3 exact-value results motivate a tuple-disjoint retest before concluding the model generalizes to genuinely unseen sequences.","If the parity result survives the tuple-disjoint retest, the PCA-visible parity structure could be used to generate explicit conjectural congruences among Frobenius traces for verification by traditional Galois-representation methods.","The parity prediction task may be equivalent, in effect, to learning the mod-2 Galois representation attached to the curve; the paper's embedding evidence suggests the model finds this representation before it finds the full trace."],"forward_implications":["If the central claim is correct, a neural network can recover the parity of a missing Euler factor from its neighbors in a regime where no efficient deterministic completion algorithm is known.","The parity signal is the model's main strength, and exact-value predictions inherit it: models trained on exact $a_p$ already predict $a_p \\bmod 2$ with MCC near the models trained directly on the parity task.","The sign of $a_p$ is the principal obstacle; the sign-agnostic MCC exceeds the full MCC in every exact-value experiment.","The pattern is stable across the dataset range: results on a test set with conductors up to $10^7$ remain well above chance, though the highest conductor band $10^6$--$10^7$ is the least accurate."],"supporting_citations":[{"why":"Shows a finite set of Frobenius traces determines the isogeny class, supplying the mathematical motivation for predicting one trace from the others.","marker":"[Ser81, Théorème 5]"},{"why":"Gives the effective bound on the distinguishing prime that the paper cites to justify finite input tuples.","marker":"[BK16, Corollary 4.8]"},{"why":"Provides the ECQ6 and ECQ7 datasets of isogeny classes used across the experiments.","marker":"[Sut]"},{"why":"Implements the transformer used for the main experiments and the PCA visualization tools.","marker":"[Cha]"},{"why":"Documents the database construction method underlying the extracted datasets.","marker":"[SW02]"},{"why":"Defines the Matthews correlation coefficient used to evaluate binary and multiclass predictions.","marker":"[Mat75]"},{"why":"Supplies the subset of curves used in the a_2 sign experiment with normalized features.","marker":"[LMF24]"}],"fun_headline_variants":["No L-functions needed: nets predict Frobenius parity to 94%","Frobenius trace parity predicted to 94% by transformers","Neighboring prime traces teach nets elliptic-curve parity","Transformers learn Frobenius parity from data alone","94% accuracy on elliptic-curve Frobenius trace parity"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The empirical claim rests on the assumption that test accuracy measures learning of new input sequences rather than recall of sequences already seen during training.","fun_headline_variants_meta":{"raw":{"variants":["No L-functions needed: nets predict Frobenius parity to 94%","Frobenius trace parity predicted to 94% by transformers","Neighboring prime traces teach nets elliptic-curve parity","Transformers learn Frobenius parity from data alone","94% accuracy on elliptic-curve Frobenius trace parity"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000652,"raw_usage":{"total_tokens":2952,"prompt_tokens":870,"completion_tokens":2082,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":486,"completion_tokens_details":{"reasoning_tokens":1994}},"tokens_in":486,"tokens_out":2082,"duration_ms":15824,"temperature":1.0,"reasoning_tokens":1994,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T18:20:14.654211+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build the test set for the $a_p \\bmod 2$ task by discarding every input tuple $(a_q)_{q\\neq p,\\ q<100}$ that also appears in training, then retrain; if accuracy on that tuple-disjoint test set is near $0.5$, the claimed parity learning is an artifact of overlap.","supporting_citations":[],"review_version":1}