{"id":"4a845fcc-a4af-4d46-8702-e57eea56200a","arxiv_id":"2607.24502","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"Normalized query/key-only RoPE attention on the sphere has reversible consensus kernels with exact Bessel-aliasing spectra, explicit regional contraction rates from a sharp softmax floor, and RoPE-selected twisted equilibria that are generically linearly unstable.","lead":"This paper derives exact local spectra and regional consensus rates for continuous-time spherical self-attention when rotary position embeddings rotate queries and keys but leave values unrotated. It matters because it shows when RoPE slows consensus, when twisted equilibria appear, and why multi-frequency energy allocation need not order rates by frequency magnitude.","discovery_kind":"extension","skeptic_critique":{"model":"moonshotai/kimi-k3","headline":"No significant objection identified. The theorem package is internally sound under its stated controlled specialization; the only load-bearing premise is the idealized model boundary, which the paper itself polices.","rationale":"The reader identified the modeling specialization (identity Q/K/V, query/key-only rotations, continuous normalized-residual limit, no learned maps/masks/heads) as the weakest assumption, and my independent pass lands in the same place: it is the only premise whose failure would void the relevance of the exact spectra and rates, and the paper states it plainly and declines to claim trained-model implications. I found no internal inconsistency to add: the linearization, aliasing, asymptotic, and contraction arguments survive line-by-line inspection, the sharpness examples (bipodal stationarity, same-system energy witnesses) are correctly deployed to bound the theorems' hypotheses, and the paper's non-claims (joint dense-phase limit, nonresonant-chain limit, center-manifold dynamics of twisted states) are exactly the places where a proof is missing — a sign of honest boundary-keeping rather than a soft spot. Per rule 2, I do not manufacture a concern; the recommended verdict remains ACCEPT, with the single independent recomputation of the Bessel-aliasing spectrum as worthwhile hygiene given the absence of a code artifact.","tokens_in":22069,"tokens_out":4351,"duration_ms":144393,"concrete_test":"Independently re-implement the headline spectrum: for (n,m,β) = (16,4,2), build the consensus kernel W*_ij = exp(β cos(2πm(i−j)/n)), row-normalize, and compare its sorted eigenvalues against the congruence-filtered Bessel sum (22); then for fixed L = 8 with several (n,m) realizing n/g = 8, compute γ(β) from (B-3) for β ∈ [5,20] and check the ratio γ/[2Δ_L e^{−βΔ_L}] → 1. If any reachable eigenvalue deviates beyond ~1e−12 or the prefactor ratio fails to converge to 1, Thms 5.1/5.3 have a real defect; this also substitutes for the absent code artifact on the most convention-sensitive formula.","verdict_should_be":"UNCHANGED","load_bearing_attack":"I read the central claims and the appendix proofs in good faith, looking for the least secure condition under which the headline package (Thm 5.1 Bessel-aliasing spectrum, Thm 5.3 fixed-ring gaps, Thms 6.2–6.3 regional contraction, Prop 7.1 twisted instability) could fail. The proofs are short and mostly self-contained, and the key steps check out on inspection: (i) the linearization in Thm 4.3 correctly uses row-stochasticity to kill the δA term at consensus, giving J_C = (A*−I)⊗I_{d−1}; (ii) the Bessel-aliasing derivation in §B.1 correctly handles gcd(m,n)>1 via congruence reachability, and the reachable-mode count L = n/g follows from invertibility of m′ mod L; (iii) the fixed-L asymptotic in §B.3 keeps only the k=±1 nearest-phase terms, with error uniform over the finite mode set — and the paper explicitly refuses the unproved joint β,L→∞ limit rather than overclaiming the continuum 1/(2β) law; (iv) the Dini-calculus contraction proofs (§C) handle the nonsmooth min and the c=0 boundary via negative-part Grönwall rather than a formal tangency argument, and the bipodal counterexample (Prop 6.4) shows the strict-semicircle hypothesis is sharp; (v) the local-vs-uniform rate separation (38)–(39) is derived, not asserted. The one genuinely load-bearing premise is the modeling boundary the reader identified: Q=K=V=I, query/key-only RoPE, unrotated values, first-order normalized-residual limit (Eqs. 3–5). Everything exact in the paper is conditional on it. But this is a stated scoping premise (§3, §9), not a hidden assumption, and the paper makes no claim about trained Transformers. Numerical cross-checks are self-reported without shipped code, but they are not used as proof steps, so that is a reproducibility inconvenience, not a soundness gap.","agreement_with_reader":"agree"},"referee_report":{"model":"moonshotai/kimi-k3","summary":"The paper studies a continuous-time spherical self-attention flow in which RoPE rotations act on queries and keys while values remain unrotated, in the controlled specialization Q=K=V=I (Eqs. 3–5). Main results: (i) global well-posedness, reversibility, and a sharp uniform softmax floor a_{β,n} (Thm 3.1), together with exact same-system witnesses that the natural RoPE interaction energy has no uniform monotonic sign (Prop 3.2); (ii) consensus is always an equilibrium, with transverse linearization J_C=(A*−I)⊗I_{d−1} (Thm 4.3), and on a resonant single-frequency ring the consensus kernel is circulant with an exact congruence-filtered Bessel-aliasing spectrum that handles gcd(m,n)>1 (Thm 5.1), a Gram–Schur positivity argument giving spec(A*)⊂[0,1] (Thm 5.2), and fixed-effective-period large-β gap asymptotics γ∼2∆_L e^{−β∆_L} (Thm 5.3); (iii) regional global results: closed hemispheres forward invariant (Prop 6.1), pairwise non-obtuse and strict open-semicircle contraction with explicit half-angle rates and single-point tail bounds (Thms 6.2–6.3, Cor 6.5), with sharpness shown by a bipodal counterexample (Prop 6.4); (iv) an exact score-flattening twisted branch with full linear spectrum — non-hyperbolic and linearly unstable generically, a hyperbolic saddle after quotienting rotation in the odd antipodal case (Props 7.1–7.2) — and multi-frequency non-monotonicity counterexamples (Props 7.4–7.5). Numerical sections cross-check but are not used as proof steps.","tokens_in":22450,"tokens_out":5988,"duration_ms":194045,"significance":"If the results hold — and in my reading they do — the paper delivers the first exact asymptotic theory of the normalized query/key-only RoPE flow: a parameter-free, closed-form spectral theory on resonant rings (with the gcd reachability structure handled correctly), sharp regional contraction rates with an explicitly proved optimal softmax floor, and an exact classification of the RoPE-locked twisted branch including the odd antipodal saddle. The work is unusually disciplined about its own boundaries: the bipodal counterexample demonstrates sharpness of the strict-semicircle hypothesis rather than leaving it as a conjecture, the energy no-go is established by exact same-system witnesses with closed forms rather than numerics alone, the local-versus-uniform rate separation (38)–(39) is derived rather than asserted, and the authors explicitly decline to claim the dense-phase joint limit that their fixed-L analysis does not prove. The independent matrix/finite-difference/nonlinear-flow cross-checks (Table 2) at near-machine precision add confidence to the convention-sensitive formulas. The significance is bounded by the controlled specialization (Q=K=V=I, unrotated values, first-oder","major_comments":[],"minor_comments":[{"comment":"Eq. (3): as printed, the normalized residual update reads 'x^{ℓ+1}_i = x^ℓ_i + h y^ℓ_i ||x^ℓ_i + h y^ℓ_i||', which appears to be missing a fraction bar and should be (x^ℓ_i + h y^ℓ_i)/||x^ℓ_i + h y^ℓ_i||. Since this is the defining equation of the discrete model whose limit is taken in (4)–(5), the typesetting should be repaired.","section":"§3, Eq. (3)"},{"comment":"The paper leans on 'independent numerical cross-checks' (Table 2, Appendix D) as part of its reliability case, and the parameter grids and tolerances are documented in detail, but no code or repository availability statement appears. Given the emphasis placed on these checks, a public code release (or an explicit statement) would substantially strengthen reproducibility.","section":"§8 / Appendix D"},{"comment":"Numbering mismatch: the proofs in the appendices repeatedly refer to 'theorem 3.2', 'theorem 4.1', 'theorem 4.2', 'theorem 5.1', etc., for statements labeled Proposition 3.2, Proposition 4.1, Proposition 4.2, etc. in the main text. A uniform pass over the cross-references is needed.","section":"Appendices A–B, passim"},{"comment":"Theorem 5.2 and Theorem 7.3 overlap substantially (positive semidefiniteness of W via Gram–Schur, the similarity to D^{-1/2} W D^{-1/2}, and the invisibility equality condition are each proved twice, in B.2 and B.5). A brief consolidation or explicit cross-reference would tighten the presentation.","section":"§5 and §7.2"},{"comment":"Figure 1b: the dashed continuum reference 1 − I_1(β)/I_0(β) is correctly labeled as unproved, but the caption would benefit from a one-line reminder in the main text near the figure that the fixed-L asymptotic (27) and the continuum 1/(2β) law are exponentially different regimes, since a casual reader could conflate them.","section":"Figure 1"}],"recommendation":"minor_revision","confidential_remarks":"The manuscript is framed for a dynamics audience but is motivated by and contextualized in the Transformer literature; the referee should confirm the editor considers this in scope. The citation pattern is appropriate and the contribution-boundary section is unusually careful about not overclaiming novelty relative to [2, 12, 13, 16, 18]."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The thing worth knowing is that this is a careful continuous-time analysis of standard RoPE on the sphere (rotated Q/K, unrotated values, identity maps, normalized residual limit). It is not a Transformer practice paper and does not pretend to be.\n\nWhat is actually new is the combined theorem package for that flow: consensus remains equilibrium with a reversible Markov tangent operator whose kernel depends on plane energies; under single-frequency resonance the attention matrix is circulant with an exact congruence-filtered Bessel-aliasing spectrum (including non-coprime zeros and fixed-L large-β gaps); kernel-generic contraction in non-obtuse and strict-semicircle regions with the sharp softmax floor; and an explicit score-flattening twisted branch that is generically non-hyperbolic/unstable, with an odd-antipodal saddle after quotienting rotation. Multi-plane gaps can be non-monotone in energy allocation, so no universal frequency ordering. The author polices novelty hard against Karagodin/Kuehn, Altafini, Pham, static RoPE logit work, and Kuramoto-attention (Table 1 and the non-claim paragraphs).\n\nThe math looks solid on inspection. Linearization kills the δA term by row-stochasticity; Bessel reachability via gcd is handled correctly; fixed-L asymptotics refuse the unproved joint continuum limit; Dini/negative-part Grönwall closes the nonsmooth boundaries; bipodal states show the semicircle hypothesis is sharp. Energy no-go is same-system closed forms, not a scan. Numerics are independent cross-checks of formulas, not proof steps. No code is a reproducibility inconvenience only.\n\nSoft spots are real but scoped. Everything exact is conditional on Q=K=V=I and the first-order normalized limit; learned maps, multi-head, masks, and finite depth are out. Global results are regional, not arbitrary-data sync. Dense-phase and nonresonant-chain limits are left open honestly. That is proportionate limitation text, not a hole in the theorems.\n\nThis is for people who already care about spherical attention dynamics or exact RoPE spectra as benchmarks. A serious editor should send it to referees. I would bring it to reading group and cite the spectrum/rate statements if I work in this lane.","headline":"Solid math.DS package on query/key-only RoPE spherical attention: exact Bessel-aliasing consensus spectra, sharp regional rates, and twisted-branch linearization under a clearly policed idealized model.","tokens_in":23501,"tokens_out":567,"would_cite":true,"duration_ms":10833,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["37N30","68T07","34D05","37C75"],"pacs":[],"model":"grok-4.5","headline":"RoPE keeps spherical self-attention consensus as an equilibrium, but can slow it exponentially and lock in unstable twisted states.","keywords":["rotary position embeddings","self-attention dynamics","spherical consensus","Bessel aliasing","twisted states","reversible Markov kernel","softmax floor","multi-frequency gap"],"falsifier":"On a resonant ring with gcd(m,n)>1, build the consensus attention matrix and check that unreachable Fourier modes are exactly zero and the remaining eigenvalues match the congruence-filtered Bessel formula to machine precision; separately, integrate the ODE from a strict open semicircle and verify that tan(D/2) stays under the proved exponential envelope set by the sharp softmax floor.","tokens_in":23159,"feed_emoji":"🔁","tokens_out":1096,"duration_ms":36513,"temperature":0.7,"pith_summary":"This paper asks what rotary position embeddings do to token motion when every token is forced to stay on the unit sphere and only queries and keys are rotated. It shows that full agreement among tokens is still always an equilibrium, and the speed of the slowest local correction is exactly the spectral gap of a reversible attention kernel that depends on how the consensus point spreads energy across RoPE frequency planes. On a resonant ring that gap has a closed Bessel-aliasing formula and can become exponentially small as inverse temperature grows, while away from consensus the same kernel still forces contraction inside non-obtuse and open-semicircle regions with explicit half-angle rates. RoPE also selects a score-flattening twisted branch that is generically linearly unstable, and multi-plane energy mixes can reverse any naive ordering by frequency. A sympathetic reader cares because the results turn a widely used positional trick into precise dynamical statements—equilibria, rates, and geometric basins—rather than informal claims about preventing collapse.","feed_headline":"RoPE slows sphere consensus but keeps it locally stable","feed_subtitle":"Exact Bessel gaps, unstable twisted branches, and half-angle rates for query/key-only rotary attention on the sphere.","key_machinery":"The continuous-time spherical flow with attention weights from RoPE-rotated scores and unrotated values, together with its consensus matrix A*: on a resonant ring A* is circulant with Bessel-aliasing eigenvalues, and globally the sharp uniform floor A_ij ≥ a_{β,n} drives kernel-generic Dini contraction estimates.","core_discovery":"For the continuous normalized residual flow with query/key-only RoPE and unrotated values on the sphere, every consensus state is an equilibrium whose transverse linearization is a reversible Markov operator whose kernel depends on consensus only through plane energies. On a resonant single-frequency ring the consensus spectrum is given exactly by congruence-filtered modified Bessel ratios, including non-coprime frequencies and fixed-ring large-β asymptotics; regionally, closed hemispheres are invariant and pairwise non-obtuse or strict open-semicircle data contract with sharp half-angle and single-point tail bounds from the uniform RoPE softmax floor. RoPE further selects an explicit score-","pith_inferences":["Strong resonance plus large β in trained models may leave long-lived near-consensus plateaus even when linear theory still predicts eventual collapse.","Value-rotating or oscillator-style attention variants would break this exact Markov consensus spectrum and need a separate rate theory.","The same sharp floor and positivity argument should apply to any positional score kernel confined to [-1,1], not only RoPE.","Center dynamics of the large neutral subspace on the generic twisted branch may organize finite-depth non-consensus transients that linear instability alone does not explain."],"forward_implications":["Local consensus speed under RoPE is completely determined by the consensus plane-energy vector and the sampled position kernel, without needing the full nonlinear flow.","On fixed resonant contexts the gap can decay like e^{-βΔ_L}, so large inverse temperature can make consensus arbitrarily slow while still locally stable.","Score-flattening twisted configurations selected by RoPE are not linearly attracting under the standard unrotated-value pathway.","No position-independent ordering of frequencies by magnitude controls the consensus gap; aliasing and plane-energy mix can reverse the order.","Inside pairwise non-obtuse or strict open-semicircle regions one obtains explicit geodesic tails to a single consensus point from the uniform floor alone."],"fun_headline_variants":["RoPE keeps sphere consensus locally stable via reversible Markov kernels","Exact Bessel spectrum for RoPE consensus on resonant single-frequency rings","Closed hemispheres stay invariant under query-key RoPE sphere flow","Half-angle bounds contract non-obtuse configs with RoPE softmax floor","Twisted RoPE branches form unstable saddles after rotation quotient"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"Everything is proved for identity query, key, and value maps with RoPE only on queries and keys, in the continuous first-order residual limit, so learned matrices, multi-head structure, masks, and finite depth are left out.","fun_headline_variants_meta":{"raw":{"variants":["RoPE keeps sphere consensus locally stable via reversible Markov kernels","Exact Bessel spectrum for RoPE consensus on resonant single-frequency rings","Closed hemispheres stay invariant under query-key RoPE sphere flow","Half-angle bounds contract non-obtuse configs with RoPE softmax floor","Twisted RoPE branches form unstable saddles after rotation quotient"]},"model":"grok-4.5","effort":"low","cost_usd":0.004788,"raw_usage":{"total_tokens":1435,"prompt_tokens":901,"num_sources_used":0,"completion_tokens":73,"cost_in_usd_ticks":47884000,"prompt_tokens_details":{"text_tokens":901,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":461,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":901,"tokens_out":73,"duration_ms":7607,"temperature":1.0,"reasoning_tokens":461,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-31T13:01:11.546324+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On a resonant ring with gcd(m,n)>1, build the consensus attention matrix and check that unreachable Fourier modes are exactly zero and the remaining eigenvalues match the congruence-filtered Bessel formula to machine precision; separately, integrate the ODE from a strict open semicircle and verify that tan(D/2) stays under the proved exponential envelope set by the sharp softmax floor.","supporting_citations":[],"review_version":1}