{"id":"5eeb418c-0c7a-4f95-b733-4c68eafeb0b9","arxiv_id":"2412.03773","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A trained modular addition transformer's MLP layer is shown to compute its outputs by numerical integration, with a trig integral identity and linear-time error bounds.","lead":"This paper shows that the middle layer of a small transformer that adds numbers modulo 59 is not a black box: it works like a numerical integrator, approximating a known trigonometric integral. If correct, the result demonstrates that nonlinear network layers can be compressed into checkable mathematical explanations, with potential applications to robustness and interpretability tooling.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The promised linear-time non-vacuous bound is proven only for the primary-frequency absolute-value part of the MLP, not for the actual MLP; Appendix L.2 explicitly defers the missing identity, bias, and residual terms, so the central compression claim is not yet delivered.","rationale":"I read the paper in good faith: the integral identity in Eq. (5) is correct and parameter-free, the phase-doubling and single-frequency observations are well visualized (Figures 3–4), and the cross-seed validation in Appendix C is a real strength. The reason I do not move the verdict is that the quadrature interpretation itself is plausible and partially proven; the problem is the mismatch between the headline promise and what the proof covers. The reader's weakest_assumption (single-frequency/phase-doubling) is a necessary condition, but even under that assumption the promised bound does not apply to the actual MLP because the proof bounds a cleaned-up surrogate. This is a load-bearing incompleteness, not a refutation. A CONDITIONAL verdict—accepting the scientific contribution while requiring either a full-model bound or a narrowed claim—is the right outcome, so the reader's verdict stands.","tokens_in":22697,"tokens_out":11923,"duration_ms":117783,"concrete_test":"Reproduce the mainline model's true MLP output (or logits) for all 59² inputs. Build the paper's compressed prediction: per-cluster quadrature approximation of Eq. (5) using the extracted φ_i and widths, plus the linear identity-component matrix from Appendix I, plus the secondary-frequency integral correction from §6.2, plus the residual/bias terms listed in Appendix L.2. Compute the worst-case relative error against the true outputs. If that total error (or a valid linear-time upper bound on it) exceeds 1, the central claim of a non-vacuous bound on the actual MLP fails; if it stays below 1 with the deferred terms included, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing gap is scope, not the empirical phase regularities. The non-vacuous bound in §5 is constructed for the idealized sum in Eq. (4), in which each neuron is replaced by its primary Fourier component and the output phase is set to exactly 2φ_i. The bound in Eq. (13) therefore controls only the quadrature error of that idealized sum. It does not control the projection error between the actual MLP and that idealized sum: the residual stream OV/E terms, the MLP bias, the embed of y, the identity component of ReLU, the secondary frequencies, and the non-primary components of W_L are all excluded. Appendix L.2 states this explicitly: \"To bound the error of our interpretation of the MLP precisely, we'd need to include a bound on the primary frequency contribution of the identity component... and include bounds on the residual components...\" Tables 2 and 3 report errors and bounds for the h-functions only, so the abstract's claim of 'a non-vacuous bound on the behaviour of the ReLU MLP in time linear in the parameter-count' is not demonstrated for the actual trained MLP. Even granting the observed R² values, an R² is not a worst-case bound.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper studies one-layer constant-attention ReLU transformers trained on modular addition (the 'pizza' models of Zhong et al.). The authors propose that the MLP layer implements frequency doubling by numerical quadrature: after sorting neurons by their input phase φ_i, the sum over neurons is interpreted as a Riemann sum for the integral ∫ ReLU[σ_k cos(k(a+b)/2 + φ)] cos(kc + 2φ) dφ = (2/3) cos(k(a+b-c)). They support this with empirical phase-amplitude observations (single-frequency dominance, ψ_i ≈ 2φ_i), an analytic derivation of the integral identity, and computed quadrature error bounds (Tables 1–3) that are non-vacuous (relative error bounds 0.48–0.70) and computable in time linear in the number of neurons. The paper also explains how secondary frequencies at 2k improve robustness when |cos(k(a-b)/2)| is small.","tokens_in":22954,"tokens_out":5913,"duration_ms":58372,"significance":"If the full claimed result were established, this would be an important step in mechanistic interpretability: a rigorous, non-vacuous, linear-time-certifiable compression of a nonlinear MLP feature map, going beyond prior black-box descriptions. The analytic integral identity in Appendix E is clean and parameter-free, and the empirical quadrature errors (0.03–0.05) are convincingly small. The systematic validation across 151 training runs and the released code are strengths. However, the significance is currently undercut by the gap between the bound that is proved and the bound that is advertised; the non-vacuous certificate is for an idealized primary-frequency model, not for the full trained MLP.","major_comments":[{"comment":"The non-vacuous bound advertised in the abstract and Section 5 is not a bound on the full ReLU MLP. Equation (13) and Table 1 bound the quadrature error of the idealized sum in Eq. (4), in which each neuron has been replaced by its primary Fourier component and the output phase is set to exactly 2φ_i. The projection error between the actual MLP and this idealized sum is not bounded: the identity component of ReLU, the residual-stream terms (OVE on x and y, the MLP bias, the embed of y), the secondary frequencies, and the non-primary components of W_L are all excluded. Appendix L.2 states this explicitly ('To bound the error of our interpretation of the MLP precisely, we'd need to include a bound on the primary frequency contribution of the identity component...'). Tables 2 and 3 report errors and bounds for the h-functions, not for the model logits. The abstract's claim of 'a non-vacuous bound on the behaviour of the ReLU MLP' is therefore not supported; at present the paper provides a non-vacuous bound on a component of an idealized version of the MLP.","section":"§5, Appendix L.2"},{"comment":"The empirical 'Normalised ... error' rows are brute-force evaluations of the h-function quadrature error, not worst-case bounds on the model's logits, and the 'Total numerical ... bound' rows are bounds for the same h-functions. Reporting these as evidence that 'the model is indeed performing numerical integration' conflates an empirical fit with a formal guarantee. Moreover, the R² statistics in Figures 3 and 4 are aggregate goodness-of-fit measures; they do not certify the worst-case accuracy of the phase-doubling and single-frequency assumptions on which the bound's validity depends. The paper should either present a bound that covers the full logit expression or explicitly narrow the claim to the idealized primary-frequency model throughout the abstract and introduction.","section":"§5.2, Table 3, Appendix H"},{"comment":"The reduction to Eq. (4) assumes that every neuron can be assigned to a single key frequency cluster I_k and that the output phase satisfies ψ_i = 2φ_i. These are empirical observations, not theorems, and the coverage is incomplete: Appendix C reports that only 100 of 151 models are 'good' (frequency matching for all neurons), and for many of these the phase R² or the uniformity of angle widths falls below the thresholds. The error bound in §5 inherits the projection error from any neuron that does not satisfy these assumptions, and the paper does not quantify that error for the excluded models. The main text should state these coverage limitations and their impact on the validity of the bound.","section":"§4.3, Eq. (4), Appendix C"}],"minor_comments":[{"comment":"The phrase 'can we well-thought of as approximating the integral' should read 'can be thought of as approximating the integral'.","section":"§4.3, Eq. (4)"},{"comment":"The caption says 'The MLP approximately computes the integral', but the plotted h(φ) is not the full MLP output; consider rephrasing to 'the primary-frequency absolute-value component is approximated by the integral'.","section":"Figure 2 caption"},{"comment":"The displayed formula mixes the case distinction with the absolute value; adding a short derivation of the interval integral would improve readability.","section":"§5.1, Eq. (13)"}],"recommendation":"major_revision","confidential_remarks":"The main gap (major comment 1) is well known to the authors, as Appendix L.2 defers the missing bounds to future work. In a revision, the authors should either supply bounds on the excluded terms using the recursive decomposition sketched in L.2, or substantially reframe the paper's title and abstract to claim a non-vacuous bound on the quadrature interpretation of the primary-frequency component. The latter would make the paper honest but would reduce its novelty; the former may be a substantial technical undertaking. I would encourage the editor to ask for the reframing at minimum, and to judge the revised version on whether the full-bound claim is restored or explicitly withdrawn."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis is the first paper I've seen that actually opens up the MLP black box in the modular addition transformers rather than treating it as brute-force enumerated or hand-waved. The core discovery is that the ReLU MLP acts like a quadrature scheme: each neuron is a rectangle in a Riemann sum whose limit is ∫ ReLU[cos(...)] cos(...) = (2/3) cos(...). That identity is clean, derived in Appendix E, parameter-free, and the empirical errors are small (0.03–0.05). The secondary-frequency story in §6 also resolves the clock/pizza discrepancy convincingly: the dominant secondary component at 2k with phase 2φ+π contributes a term of the form −(π/2) cos(k(a−b)) cos(k(a+b−c)), which compensates for the pizza factor |cos(k(a−b)/2)| near its zeros. That's a real mechanistic insight, not just a curve fit.\n\nThe paper is honest about what it doesn't do, and that honesty matters. The headline claim in the abstract says 'a non-vacuous bound on the behaviour of the ReLU MLP in time linear in the parameter-count.' What is actually bounded in §5 is the quadrature error of a projected, idealized sum: each neuron replaced by its primary Fourier component, with output phase exactly 2φ_i, and ReLU split so that only the |x|/2 part is bounded. The identity component is handled separately (it integrates to zero symbolically, but its finite-neuron contribution is non-negligible), and the residual-stream terms, biases, and non-primary components are left out. Appendix L.2 says this explicitly. So the abstract overstates the result by a meaningful margin. The computed bounds 0.48–0.70 are relative errors for that idealized h-function, not for the actual MLP on all inputs.\n\nThe load-bearing empirical regularities — single-frequency dominance and ψ≈2φ — are observed, not derived, but they are extensively validated across 151 models (78 of 100 'good' models have R²>0.9 for the angle fit). An R² is not a worst-case bound, but the paper never claims otherwise. The real gap is scope, not circularity: the integral identity is independent and parameter-free, and the network-specific quantities are extracted from the model, but the extraction is exactly what an interpretation is supposed to provide.\n\nWho is this for? Researchers working on mechanistic interpretability of small training runs, and anyone interested in rigorous compression of nonlinear maps. It won't settle the debate on whether this generalizes to realistic transformers, but it's a solid, reproducible case study with code and a clear method.\n\nMy verdict: worth a serious referee. I would ask for the abstract to be brought in line with the proven claims, and for the missing terms to be either bounded or explicitly kept as a stated weaker theorem. The math and the empirical work are solid; the packaging is ahead of the proof.\n\nRecommendation: send to peer review, not desk reject.","headline":"A genuinely new interpretation of the ReLU MLP in pizza-algorithm transformers, with an honest but incomplete non-vacuous bound; worth refereeing.","tokens_in":23529,"tokens_out":1948,"would_cite":true,"duration_ms":18952,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The ReLU MLP in a modular-addition transformer is best understood as a quadrature scheme, and this interpretation yields non-vacuous error bounds in time linear in the parameter count.","keywords":["mechanistic interpretability","modular addition","ReLU MLP","feature-map compression","numerical quadrature","trigonometric integral identity","frequency doubling","non-vacuous error bounds"],"falsifier":"Train the same constant-attention one-layer transformer on modular addition with a different modulus or width and inspect the phase plot for each key-frequency cluster: if any cluster that still solves the task has the $\\psi_i \\approx 2\\phi_i$ regression with $R^2$ below 0.9, the quadrature interpretation is not the mechanism. Equivalently, compute the paper's relative error bound on a trained model and look for any input triple $(a,b,c)$ where it exceeds 1.","tokens_in":22408,"feed_emoji":"🧮","tokens_out":8472,"duration_ms":76492,"temperature":0.7,"pith_summary":"The paper tries to open the last black box in a classic toy model of mechanistic interpretability: the ReLU MLP inside a one-layer constant-attention transformer trained on modular addition. It claims that the MLP's frequency-doubling step is numerical integration. Each neuron contributes a rectangle whose height is the value of $\\mathrm{ReLU}[\\sigma_k \\cos(k(a+b)/2 + \\phi)] \\cos(kc + 2\\phi)$ and whose width comes from the neuron's weight, and the sum over neurons approximates the identity $\\int_{-\\pi}^{\\pi} \\mathrm{ReLU}[\\sigma_k \\cos(k(a+b)/2 + \\phi)] \\cos(kc + 2\\phi)\\,d\\phi = \\frac{2}{3}\\cos(k(a+b-c))$. If correct, this is the first rigorous compression of a nonlinear feature-map in these models, with non-vacuous relative error bounds, all below 1, computable in time linear in the number of parameters rather than by enumerating all inputs. The paper also explains why 'pizza' models look 'clock-like': a secondary frequency, double the primary, contributes a compensating term when $|\\cos(k(a-b)/2)|$ is near zero.","feed_headline":"Trained transformer MLPs do numerical integration for modular addition","feed_subtitle":"A ReLU layer's weights become rectangle widths under a trigonometric curve; its error is boundable in linear time.","key_machinery":"The central object is the quadrature scheme: the finite neuron sum is read as a Riemann-style integral over the phase variable $\\phi$. The key identity is $\\int_{-\\pi}^{\\pi} \\mathrm{ReLU}[\\sigma_k \\cos(k(a+b)/2 + \\phi)] \\cos(kc + 2\\phi)\\,d\\phi = \\frac{2}{3}\\cos(k(a+b-c))$. The mechanism that makes the reduction work is the amplitude-phase Fourier representation of the model's weight matrices, which reveals that each neuron has a single dominant frequency $k$, that its output phase is approximately double its input phase ($\\psi_i \\approx 2\\phi_i$), and that the phases are roughly uniformly distributed; those facts justify grouping neurons into clusters $I_k$ and replacing each cluster's sum by an integral. To bound the error in linear time, the paper splits ReLU into its identity part $x/2$, whose integral vanishes, and its absolute-value part $|x|/2$, whose Lipschitz constant bounds the rectangle error via $\\sup |h'| \\le 2$.","core_discovery":"On the paper's own terms, the central discovery is that the trained MLP layer computes the pizza algorithm's frequency-doubling step by numerical quadrature, not by an opaque feature interaction. For each key frequency $k$, neurons cluster into a group $I_k$; after sorting their input phases $\\phi_i$, the normalized MLP output $\\sum_{i\\in I_k} w_i\\, \\mathrm{ReLU}[\\sigma_k \\cos(k(a+b)/2 + \\phi_i)] \\cos(kc + 2\\phi_i)$ is a rectangle-rule approximation to the integral in Eq. (5), which equals $\\frac{2}{3}\\cos(k(a+b-c))$. The paper establishes the structure by amplitude-phase Fourier analysis: most neurons are dominated by a single key frequency, and the output phase $\\psi_i$ is almost exactly twice the input phase $\\phi_i$, with $R^2 > 0.99$. It then validates the interpretation by computing relative error bounds between 0.48 and 0.70 for the mainline model, all strictly below 1, in time $O(d_{\\text{mlp}} + p)$, and shows the phenomenon recurs across 100 of 151 randomly seeded models.","pith_inferences":["Because the paper only trains with modulus $p=59$, a direct testable extension is to check whether the single-frequency and phase-doubling regularity persists at larger moduli; if it does, the same $O(d_{\\text{mlp}}+p)$ bound should hold there.","The rectangle-error argument relies only on a bounded derivative and sorted phases, so a similar quadrature compression may apply to other pointwise nonlinearities with bounded Lipschitz constants, not just ReLU.","The paper's split of ReLU into an exactly compressible linear part and a quadrature-compressible nonlinear residual suggests a general recipe for compressing nonlinear layers: isolate the linear component, then integrate the nonlinear residual.","If larger transformers show 'clock-like' logit regressions, the secondary-frequency compensation mechanism implies that the underlying computation may still be pizza-like; a purely logit-level regression cannot distinguish the two algorithms."],"forward_implications":["Verifying the MLP's behavior no longer requires enumerating all $p^2$ input pairs; the paper's bound runs in $O(d_{\\text{mlp}} + p)$ time, which is linear in the parameter count.","Splitting ReLU into identity plus absolute-value parts shows the nonlinear part alone implements the pizza logits, while the identity part is exactly compressible through a matrix $A$ with $\\mathrm{logit} = A[:,a] + A[:,b]$.","Secondary frequencies equal to twice each key frequency produce a $-\\frac{\\pi}{2}\\cos(k(a-b))\\cos(k(a+b-c))$ correction that compensates for the pizza algorithm's weakness when $|\\cos(k(a-b)/2)| \\approx 0$, explaining why overall logits regress better against the clock form.","Across 100 'good' models from 151 random seeds, 82% of (model, frequency) pairs have empirical integration errors below 0.1, and the median normalized bound is 0.40 versus a naive baseline of 0.85.","The same quadrature-error machinery applies to both signs $\\sigma_k = \\pm 1$ because the integral identity holds for both, so the bound covers all inputs without case-splitting."],"supporting_citations":[{"why":"Established the modular addition transformer benchmark and showed the MLP's input and output features are sparse Fourier features while leaving the MLP itself as a black box.","marker":"Nanda et al. (2023)"},{"why":"Defined the constant-attention 'pizza' algorithm and empirically showed by brute-force enumeration that the MLP doubles input frequencies.","marker":"Zhong et al. (2023)"},{"why":"Gave a compressed formula for a cleaner quadratic-activation MLP-only model, the precursor this paper extends to ReLU.","marker":"Gromov (2023)"},{"why":"Supplied the compression metric: a non-vacuous bound checkable in time linear in parameter count is taken as evidence of a compact explanation.","marker":"Gross et al. (2024)"},{"why":"Provided the transformer implementation used to train and analyze the modular addition models.","marker":"Nanda & Bloom (2022)"}],"fun_headline_variants":["MLPs perform modular addition via numerical integration","Neurons compute rectangle areas to add modulo p","ReLU MLP addition decoded as quadrature scheme","Trained MLPs integrate instead of memorizing","MLP neurons: rectangle-rule integrators for addition"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument rests on an empirical regularity rather than a proven theorem: in a trained model, each neuron is dominated by a single key frequency and its output phase is almost exactly twice its input phase; if that pattern breaks, the clustering into $I_k$ and the reduction to Eq. (5) collapse.","fun_headline_variants_meta":{"raw":{"variants":["MLPs perform modular addition via numerical integration","Neurons compute rectangle areas to add modulo p","ReLU MLP addition decoded as quadrature scheme","Trained MLPs integrate instead of memorizing","MLP neurons: rectangle-rule integrators for addition"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000945,"raw_usage":{"total_tokens":4056,"prompt_tokens":983,"completion_tokens":3073,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":599,"completion_tokens_details":{"reasoning_tokens":2999}},"tokens_in":599,"tokens_out":3073,"duration_ms":21569,"temperature":1.0,"reasoning_tokens":2999,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T22:06:24.975288+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same constant-attention one-layer transformer on modular addition with a different modulus or width and inspect the phase plot for each key-frequency cluster: if any cluster that still solves the task has the $\\psi_i \\approx 2\\phi_i$ regression with $R^2$ below 0.9, the quadrature interpretation is not the mechanism. Equivalently, compute the paper's relative error bound on a trained model and look for any input triple $(a,b,c)$ where it exceeds 1.","supporting_citations":[],"review_version":1}