{"id":"3b23cbca-3dd1-44b3-ae7b-d93e99913f5d","arxiv_id":"2607.05583","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A physics-inspired damped-resonator mixer replaces attention and, at 6M parameters, improves WikiText accuracy while delivering growing long-context train/prefill/decode speedups versus a matched transformer.","lead":"ResonatorLM replaces transformer attention with damped resonant kernels that mix a sequence as a driven 1D field, using FFT convolution for training and fixed-size recurrence for decode. In a matched 6M-parameter setting it reports better WikiText accuracy and up to 6.47× decode speedup at 32K tokens.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Quality gains may be inflated by a weak transformer baseline rather than the resonant mixer itself.","rationale":"The reader's weakest assumption correctly flags scale/tokenization/production generalization, which the Limitations section already owns. That is a real external risk, but the more load-bearing internal concern for the paper's central joint claim is whether the reported quality lift is robust to stronger non-attention baselines already known to work at this scale. The efficiency story (decode crossover, fixed-size state, kernel-tail numbers) is internally consistent and does not require that comparison. The quality story does: without it, the claim reduces to \"faster than a transformer and better than a possibly under-tuned transformer at 6M char LM.\" Keeping CONDITIONAL is right; the condition should include a same-protocol SSM/linear baseline, not only larger scale. No mathematical inconsistency in the resonator parameterization or causality diagnostics was found.","tokens_in":10049,"tokens_out":524,"duration_ms":4876,"concrete_test":"Train a matched ~6M Mamba (or S4/Hyena) baseline under the exact same WikiText-2 character protocol (same seeds, steps, optimizer, eval) and report test PPL/accuracy. If ResonatorLM's advantage over that baseline falls below ~0.2 PPL / ~2 accuracy points, the quality half of the strongest claim is not mixer-specific.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The strongest claim pairs quality (61.31% vs 55.32% acc; 3.764 vs 4.617 PPL) with long-context speedups against a matched transformer. The speedup side is well-supported by the dual-mode design (FFT prefill + fixed-size recurrent state, Table 4). The quality side is less secure: the baseline is a standard transformer with d_model=248, 6 layers, 8 heads, trained 10k steps on character WikiText-2, with no comparison to stronger linear-time mixers (Mamba/S4/Hyena) that the Related Work cites as already competitive. Ablations (Table 2) only remove coupling/local path inside ResonatorLM; they do not show that a well-tuned SSM or linear-attention baseline would not close the gap. At character level and 6M params, a weak baseline can make any structured recurrence look strong. If the quality edge is baseline-dependent, the joint claim weakens even while efficiency remains real.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes ResonatorLM, a language-model block that replaces self-attention with causal resonant field mixing. Token sequences are treated as a driven 1-D latent field; each head uses a learned damped resonator kernel k_h[t]=exp(-α_h t) cos(ω_h t+φ_h) (Eq. 1). Full-sequence training/prefill is implemented by causal FFT convolution (O(T log T)); autoregressive decode uses a fixed-size complex recurrent state per head (Eqs. 4–5). The block also includes optional head coupling and a local causal depthwise convolution, plus a conventional RMSNorm + SwiGLU stack. In a matched ~6M-parameter character-level WikiText-2 setting (six seeds), ResonatorLM reports lower test perplexity (3.764 vs 4.617) and higher accuracy (61.31% vs 55.32%) than a transformer baseline, with practical block decode speedup rising to 6.47× at 32K tokens (Table 4) and much larger kernel-only speedups (Table 5). Ablations, transfer, longer-context, and short scale sweeps are also reported.","tokens_in":10343,"tokens_out":1308,"duration_ms":9836,"significance":"If the quality and efficiency results hold under stronger baselines and more realistic tokenizers/scales, the work would add a concrete, physics-motivated alternative to attention and to existing linear-time mixers. Strengths that deserve credit include: (i) a clean dual-mode design that keeps the same kernel family for FFT prefill and fixed-size recurrent decode; (ii) multi-seed matched experiments with means/stds, ablations of coupling and local path, transfer and long-context tables, and explicit physics diagnostics (causality prefix error, half-life range); (iii) a clear separation of practical block wall-clock from asymptotic kernel-tail benchmarks. The Limitations section already flags the small scale and limited baselines. The contribution is therefore best read as a solid foundational demonstration rather than a production claim.","major_comments":[{"comment":"§5.1 / Table 1 and Related Work (§2): The joint quality+efficiency claim is load-bearing, but quality is measured only against a matched transformer (d_model=248, 6 layers, 8 heads, 10k steps, character WikiText-2). Related Work cites S4, Hyena, and Mamba as already competitive linear-time alternatives, yet none appear as baselines. At 6M parameters and character level, a weak transformer can inflate the apparent quality edge of any structured recurrence. A head-to-head against at least one strong SSM or linear-attention baseline (same parameter budget, same protocol) is needed to support the claim that the resonant mixer itself, rather than recurrence structure in general, drives the accuracy/PPL gains.","section":null},{"comment":"§4 Experimental Protocol and Limitations: All primary quality numbers use character (or byte) tokenization on WikiText-2/103 and TinyStories at ~6M parameters. The abstract and strongest claim present these as evidence for long-context language modeling. Character-level 6M results are informative for the mixer but are a weak proxy for subword LMs at realistic scale. Either add a subword experiment (even small) or substantially qualify the abstract/conclusion so that the central claim is scoped to the regime actually tested.","section":null},{"comment":"Table 4 vs Table 5 and §5.3: Decode crossover and 6.47× at 32K are credible under the disclosed practical-block protocol, but the kernel-tail numbers (440× / 576×) are easy to over-read. The manuscript already separates the two protocols; the abstract and conclusion should lead exclusively with the practical-block figures and treat kernel-tail as asymptotic only, so that the efficiency claim remains proportionate to what an end-to-end block actually delivers.","section":null}],"minor_comments":[{"comment":"Eq. (1) and §3.1: State explicitly how α_h, ω_h, φ_h are parameterized (softplus/sigmoid ranges, log-spaced half-life init) in the main text, not only in prose; a short parameter table would help reproducibility.","section":null},{"comment":"Figure 1: The block diagram is useful but low-resolution in the manuscript text; ensure labels for FFT path, recurrent state, local path, and head coupling are legible in the camera-ready version.","section":null},{"comment":"Table 1: Training throughput favors the transformer (262k vs 173k tok/s). Mention this tradeoff in the abstract or early results so quality gains are not read as free.","section":null},{"comment":"Reference [1] (Hugging Face forum post) is cited as independent related work; clarify date/access and whether any code or formulation was shared, to avoid novelty ambiguity.","section":null},{"comment":"Minor typos/spacing: e.g. \"Currentalternativestothetransformer\", \"Inaseparateexperiment\", and similar concatenated words appear in the provided text; a full proofread pass is needed.","section":null}],"recommendation":"major_revision","confidential_remarks":"The efficiency story (dual-mode FFT + fixed recurrent state) is the more robust part of the paper; the quality story is currently baseline-limited. I would not reject on novelty grounds—the resonator parameterization and dual-mode execution are a legitimate contribution—but I would not accept without a stronger non-transformer baseline or a clearly scoped claim. Fit for a methods-oriented venue is reasonable after revision; for a top general ML venue the scale and baseline bar would need to be higher."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful takeaway is simple: they replace attention with per-head damped oscillatory kernels (exp(-αt) cos(ωt+φ)), run as causal FFT convolution for train/prefill and fixed-size complex recurrence for decode, and at matched ~6M params on char WikiText-2 they beat a standard transformer on both quality (PPL 3.76 vs 4.62, acc 61.3% vs 55.3%) and long-context decode (6.47× at 32K). That dual-mode design is the real contribution.\n\nWhat is new is the concrete parameterization and the clean split between O(T log T) FFT path and constant-size recurrent state, plus the physics diagnostics (half-lives 2–2048, causality prefix error ~1e-6). Related Work correctly situates it against SSMs, Hyena, linear attention, and Neural ODEs; they even flag the concurrent HF forum post. Experiments are careful for the scale: six seeds on the main match, means±std, ablations of coupling and local path (quality barely moves), transfer to WikiText-103/TinyStories and longer contexts, short scale sweep to 30M. Speed tables separate practical block (projections + state) from pure kernel-tail, which is the right hygiene. Math is elementary and consistent; no circularity.\n\nSoft spots are real but proportional. The quality claim is only vs a vanilla transformer (d=248, 6L, 8H, 10k steps, char-level). No Mamba/S4/Hyena or strong linear-attention baseline, so the accuracy gap could shrink once the comparator is a modern linear-time mixer. Ablations stay inside ResonatorLM. Everything is 6M character/byte; Limitations correctly call out larger scale, subword, and production settings as open. No public code/artifacts. Training throughput is actually lower than the transformer. None of this kills the efficiency story—the fixed-state decode and rising train/prefill speedups with length are well-supported by the architecture itself.\n\nThis is for people working on efficient sequence mixers who want a physics-flavored alternative that is easy to implement and already shows simultaneous quality and wall-clock gains at small scale. It is not yet a production claim. I would send it to peer review: the idea is coherent, the small-scale package is honest, and referees can demand the missing stronger baselines and scale checks. Worth reading and worth engaging; I would not cite the quality numbers yet without those controls, but the mixer design itself is citable.","headline":"Clean dual-mode damped-resonator mixer with real small-scale quality+decode gains vs a matched transformer; efficiency is solid, quality edge is still baseline-limited.","tokens_in":10905,"tokens_out":627,"would_cite":true,"duration_ms":5516,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"A physics-derived resonant mixer can replace attention and deliver both better quality and faster long-context decoding in a matched small language model.","keywords":["long-context language modeling","efficient decoding","sequence modeling","alternative to attention","resonant kernels","causal convolution","state-space models"],"falsifier":"Train matched ResonatorLM and transformer models at substantially larger scale (or on subword WikiText / standard long-context retrieval tasks) and check whether the accuracy lead and the decode speedup at 32K+ tokens both disappear.","tokens_in":10938,"feed_emoji":"📡","tokens_out":888,"duration_ms":6643,"temperature":0.7,"pith_summary":"This paper argues that language models do not need self-attention to model long sequences well. Instead of token-to-token similarity scores, ResonatorLM treats a sequence as one driven latent field and mixes context with learned damped resonators—kernels that decay and oscillate over time. The same kernels run as causal FFT convolution during training and prefill, then as fixed-size recurrent state during generation, so decode memory does not grow with context length. In a carefully matched 6M-parameter character-level WikiText setting, the model improves accuracy and perplexity over a transformer baseline while becoming faster as sequences lengthen, reaching 6.47× decode speed at 32K tokens. A sympathetic reader would care because the work offers a concrete, physics-motivated path to long-context efficiency without abandoning the usual language-model block stack.","feed_headline":"Resonant mixer beats transformer accuracy and 6.47× decode at 32K","feed_subtitle":"Physics-derived damped resonators replace attention in a matched 6M model on WikiText.","key_machinery":"Causal resonant field mixing: each head uses a damped oscillator kernel kh[t] = exp(−αh t) cos(ωh t + ϕh), applied by causal FFT convolution for full sequences and by a fixed-size complex recurrent state for one-step decoding.","core_discovery":"In a matched ~6M-parameter setting, replacing self-attention with causal resonant field mixing improves WikiText-2 character accuracy from 55.32% to 61.31% and perplexity from 4.617 to 3.764, while practical block decode speed reaches 6.47× that of an optimized transformer at 32K tokens, with train and prefill speedups also rising with sequence length.","pith_inferences":["If the recurrent resonator state truly preserves the same dynamics as the FFT path, similar physics kernels could be dropped into other sequence stacks (vision, audio, multimodal) without redesigning the whole model.","The reported crossover where decode becomes faster only after a few thousand tokens suggests the method is most relevant for workloads that routinely exceed that length.","Head coupling and the local lexical path appear secondary; the load-bearing idea is the damped resonant kernel itself, so future work can stress-test minimal resonator-only blocks."],"forward_implications":["Long-context language models can keep a conventional residual + MLP stack while swapping only the mixer for O(T log T) prefill and constant-size decode state.","Decode cost and memory need not grow linearly with context length if the mixer is a fixed-size recurrent resonator state rather than a key-value cache.","Learned half-lives and impulse responses can serve as interpretable diagnostics of which timescales a model actually uses.","Quality need not trade off against long-context efficiency: the same small ResonatorLM improves both perplexity/accuracy and practical block speedups as length grows."],"fun_headline_variants":["ResonatorLM: 61.31% WikiText acc and 6.47× decode at 32K","Causal resonators beat attention: 61.31% acc, 6.47× 32K speed","Damped resonant mixing: higher WikiText scores plus 6.47× decode","ResonatorLM lifts WikiText to 61.31% with 6.47× transformer decode","Physics-derived resonators hit 6.47× decode and 61.31% accuracy"],"cache_read_input_tokens":128,"weakest_assumption_plain":"The paper's central claim rests on the premise that quality and speed gains seen at 6M parameters on character-level WikiText and TinyStories will still matter for larger models, subword tokenizers, and real production long-context workloads.","fun_headline_variants_meta":{"raw":{"variants":["ResonatorLM: 61.31% WikiText acc and 6.47× decode at 32K","Causal resonators beat attention: 61.31% acc, 6.47× 32K speed","Damped resonant mixing: higher WikiText scores plus 6.47× decode","ResonatorLM lifts WikiText to 61.31% with 6.47× transformer decode","Physics-derived resonators hit 6.47× decode and 61.31% accuracy"]},"model":"grok-4.5","effort":"low","cost_usd":0.007484,"raw_usage":{"total_tokens":1824,"prompt_tokens":769,"num_sources_used":0,"completion_tokens":130,"cost_in_usd_ticks":74840000,"prompt_tokens_details":{"text_tokens":769,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":925,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":769,"tokens_out":130,"duration_ms":9020,"temperature":1.0,"reasoning_tokens":925,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-11T05:27:12.943746+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Train matched ResonatorLM and transformer models at substantially larger scale (or on subword WikiText / standard long-context retrieval tasks) and check whether the accuracy lead and the decode speedup at 32K+ tokens both disappear.","supporting_citations":[],"review_version":1}