{"id":"88753c63-8108-4376-955e-6ecaf18aba01","arxiv_id":"2508.18308","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"CoPE, a complex positional encoding with imaginary-position and first-layer phase-aware attention, shows small GLUE gains but its no-decay proof and speed advantage are not fully supported.","lead":"CoPE is a positional encoding that stores token meaning in the real part of a complex embedding and position in the imaginary part, then applies phase-aware attention in the first transformer layer. The paper claims better GLUE scores than RoPE, sinusoidal, and learned encodings, but the evidence is limited to a small model, three datasets, and no released code.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (8)'s proportionality is unproven and dimensionally inconsistent: learned complex projections mix content and position, so the no-decay conclusion does not follow.","rationale":"The paper's headline claims are (a) no long-term decay, (b) compatibility with linear attention, and (c) superior GLUE performance with lower compute. Claim (c) is empirically weak: only three tasks, best-of-five variant selection, single runs, no significance tests, and no code or data. Claim (b) is structurally plausible: the linear-attention derivation in Section 3.3 correctly decomposes the Hermitian kernel into four real inner products and retains O(N) complexity. The load-bearing weakness is claim (a), because it is the main theoretical novelty and is used to motivate the method over RoPE/ALiBi-style encodings. The proof's Eq. (8) is not a minor simplification; it is the step that converts a learned-matrix interaction into a scalar product of sinusoids. Without it, the derived cos(ω(p−q)) − cos(ω(p+q)) expression does not follow, and the no-decay property is unestablished. The reader's weakest_assumption identifies exactly this issue. I agree with the REJECT verdict: the central theoretical claim is unsupported, and the empirical evidence is too thin to carry the paper. This is not a matter of style or missing baselines; it is an internal gap in the argument.","tokens_in":7739,"tokens_out":4517,"duration_ms":60970,"concrete_test":"Set d=2, choose generic 2×2 complex matrices for Qproj and Kproj, and let sin(ωp) be a fixed two-dimensional sinusoidal vector. For many random matrices, compute the residual R(p,q) = (Qproj sin(ωp))·(Kproj sin(ωq)) − c·sin(ωp)·sin(ωq), where c is chosen by least squares. If the residual is nonzero and comparable in magnitude to the positional term, Eq. (8) is false as a general identity. To make it model-specific, repeat using the actual learned Qproj and Kproj from a trained CoPE model and check whether the proportionality holds at the operating point.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.3's proof of no long-term decay rests entirely on Eq. (8), which asserts Qproj sin(ωp) · Kproj sin(ωq) ∝ sin(ωp) sin(ωq). But Qproj and Kproj are arbitrary learned complex matrices, and sin(ωp) is introduced as a scalar (Eq. (1): z = e_x + iγ sin(ωp), with e_x ∈ R^d). A matrix applied to a scalar is ill-defined unless sin(ωp) is implicitly a vector, and even then a generic learned matrix will rotate and mix the sinusoidal components with content directions. The positional term in Eq. (7) therefore is not proportional to the raw sinusoidal product unless Qproj and Kproj are constrained (e.g., diagonal, scalar multiples of identity, or otherwise preserving the sinusoid subspace), and no such constraint is stated or justified. Since Eq. (10) and the claim 'no multiplicative decay factor' follow only from Eq. (8), the paper's central theoretical property is unsupported. This is an internal gap, not a disagreement with consensus. The same issue undermines the claim that phase-aware attention encodes relative position, because the phase of Qcomplex·Kcomplex* includes content-dependent contributions from the cross terms in Eqs. (5)-(6).","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CoPE, a complex-valued positional encoding that separates token content (real part) from sinusoidal position information (imaginary part). A phase-aware attention mechanism is applied only in the first transformer layer, with five variants for reducing complex attention scores to real scores. The authors claim CoPE does not exhibit long-term decay, encodes both relative and absolute positions, is compatible with linear attention, and achieves superior performance with lower computational cost than RoPE, Sinusoidal, and Learned positional encodings. Experiments are reported on three GLUE tasks (SST-2, MRPC, QNLI) using a small 6-layer transformer trained from scratch.","tokens_in":8077,"tokens_out":5733,"duration_ms":67458,"significance":"If the central claims were established, CoPE would be a genuinely useful contribution: it is a simple first-layer adapter that avoids the long-term decay bias and is compatible with linear attention, while the linear-attention derivation in Section 3.3 is explicit and appears internally consistent. However, the proof of the no-decay property rests on an unjustified proportionality, and the experimental evidence consists of single runs with best-of-five variant selection and no significance testing. The empirical superiority and computational advantage claims are therefore not supported. The idea is interesting and clearly presented, but the load-bearing theoretical and empirical support is missing. No code or reproducibility artifacts are provided.","major_comments":[{"comment":"The no-decay theorem rests on the assertion Qproj sin(ωp) · Kproj sin(ωq) ∝ sin(ωp) sin(ωq). This is neither proved nor generally true. Qproj and Kproj are arbitrary learned complex matrices, and sin(ωp) in Eq. (1) is a scalar; a matrix applied to a scalar is undefined unless sin(ωp) is implicitly broadcast to a vector, and even then a generic learned matrix will mix the positional sinusoid with content dimensions. The proportionality holds only under strong constraints (e.g., scalar or diagonal projections, or projections that preserve the sinusoidal subspace), and no such constraint is stated or enforced in Section 3.2. Since Eqs. (9)-(10) and the claimed absence of multiplicative distance decay follow only from Eq. (8), the central theoretical contribution is unsupported. This is an internal gap, not a disagreement with the long-term-decay literature.","section":"Section 3.3, Eq. (8)"},{"comment":"The claim that phase-aware attention encodes both relative and absolute positions relies on the same Eq. (8) proportionality. In the actual computation, Acomplex includes content-position cross terms (Eqs. (5)-(6)) that are not oscillatory functions of p−q alone. Unless Qproj and Kproj are constrained, the phase of Acomplex contains content-dependent contributions, so the phase difference is not purely positional. The relative-position interpretation is therefore not justified by the given derivation.","section":"Section 3.3, Property 2"},{"comment":"The empirical claim of superior performance is not supported. Each configuration appears to be a single run with no seeds, no error bars, and no significance tests, and the 'best' CoPE variant is selected per dataset from five variants, inflating the chance of favorable comparisons. The reported gaps are small (e.g., SST-2: CoPE phase 82.57 ties sinusoidal; MRPC: 81.71 vs. 81.55 for learned; QNLI: CoPE magnitude 61.63 is second-best behind 63.87 sinusoidal). Under these conditions, the results do not establish superiority over RoPE, sinusoidal, or learned encodings. The authors' own Section 5 limitation statement, noting the evaluation is limited to small from-scratch models, further undermines the broad claims in the abstract and conclusion.","section":"Section 4, Table 1"},{"comment":"The claim that CoPE is 'L times faster than RoPE' is an order-of-magnitude argument that omits constant factors and implementation costs. RoPE's rotation is a relatively cheap operation applied to queries/keys in every layer, whereas CoPE replaces the first layer's attention with complex projections and phase-aware score computation; no wall-clock measurements are provided. The asymptotic comparison O(L·N·H·T·dk) vs. O(N·H·T·dk) ignores that the first complex layer is more expensive than a standard attention layer and that the remaining layers still run standard attention. Thus 'less computational complexity' in the abstract is not established.","section":"Section 3.4"}],"minor_comments":[{"comment":"There are numerous typographical errors: 'transfomer', 'propogate', 'postional', 'Sof tmax', 'isclosely', 'i.e,'. These should be corrected.","section":"Throughout"},{"comment":"The formulas for Hybrid and Hybrid-norm are missing explicit division by sqrt(dk) and parentheses are ambiguous. For example, 'Areal = (Amagnitude + α · cos(Aphase))√dk' should presumably be '... /√dk'. Please clarify.","section":"Section 3.2, variants 4-5"},{"comment":"Equation (17) is an empty line; remove or number the subsequent equations correctly.","section":"Section 3.3, Eq. (17)"},{"comment":"The experimental setup does not report random seeds, number of runs, or standard deviations. The choice of α=0.2 and γ=1 is not justified, and no sensitivity analysis is provided.","section":"Section 4"},{"comment":"The figure legends are hard to read in the text (e.g., 'Complex Valued phase Complex Valued magnitude Complex Valued hybrid norm ROPE'). Please ensure the captions and legends are self-contained and legible.","section":"Figures 1-3"}],"recommendation":"reject","confidential_remarks":"The central proof gap in Section 3.3 is disqualifying in the current form: Eq. (8) is an unproven and generally false proportionality, and it underpins the paper's main theoretical claim. The empirical evidence is also insufficient, with single runs and best-of-five selection. If the authors can constrain the projections so that Eq. (8) actually holds, provide an alternative proof, and add multi-seed experiments with significance testing, a resubmission could be reconsidered. As it stands, the paper does not meet the bar for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, what's worth knowing: CoPE (real token embeddings, sinusoidal positions in the imaginary part, phase-aware attention in the first layer) is a sensible recombination of ideas from Wang et al. and Eilers and Jiang. The paper is clearly written and the limitations section is honest. But the headline theoretical claim—no long-term decay—does not survive contact with the math. Section 3.3's Eq. (8) asserts Qproj sin(ωp) · Kproj sin(ωq) ∝ sin(ωp) sin(ωq). That proportionality is not true for generic learned complex projections, and the notation is internally inconsistent: Eq. (1) defines sin(ωp) as a scalar while Eq. (4) treats Qproj sin(ωp) as a vector. Even if you read the sinusoid as a vector, a learned linear map will generally mix it with content directions. So the derivation of Eq. (10) is unsupported—and that is the load-bearing claim of the abstract and the contributions.\n\nThe empirical evaluation is also weaker than the text suggests. Table 1 reports single runs on three GLUE tasks, with no error bars, no significance tests, and selection among five variants per dataset. On SST2, CoPE phase ties with Sinusoidal; on QNLI, only CoPE magnitude beats RoPE, and barely. That does not establish 'superior performance.' The compute comparison is hand-wavy: it ignores the constant cost of complex operations and gives no wall-clock measurements.\n\nWhat is genuinely nice: the linear-attention compatibility section is a clean algebraic derivation. The complex kernel does separate into four real inner products, so the O(N) complexity argument is sound as a statement about separability, not yet tested empirically.\n\nNet: the basic idea is plausible, but the paper's central theoretical property is unproven and the empirical evidence is too thin. I would not cite this work or rely on any of its quantitative claims. For peer review, I'd desk reject: the founding proof is invalid and the experimental support is not there to justify referee time. If the authors fix the proof (or explicitly constrain the projections), run multi-seed experiments with proper baselines (HoPE, Eilers and Jiang), and release code, I'd be willing to look again.","headline":"CoPE's central no-decay claim rests on an unjustified proportionality and the experiments are too thin to support 'superior performance'—the idea is plausible but the current proof and evidence do not hold up.","tokens_in":8491,"tokens_out":6013,"would_cite":false,"duration_ms":68635,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CoPE stores position in the imaginary part of a complex embedding, and its first-layer phase-aware attention removes the long-term decay that standard encodings impose.","keywords":["complex positional encoding","phase-aware attention","long-term decay","linear attention","transformer positional encoding","GLUE benchmark","rotary position embedding","sequence modeling"],"falsifier":"Train the smallest CoPE model on SST-2, freeze it, and for a fixed query token plot the average attention weight assigned to the same token placed at every relative distance from 1 to 256. If the curve falls systematically with distance, for example exponentially, the claimed absence of long-term decay is false. A cheaper calculation-level check is to compute Q_proj·sin(ωp)·K_proj·sin(ωq) with the trained matrices and compare it with sin(ωp)sin(ωq); large deviations break Eq. (8), on which the no-decay conclusion rests.","tokens_in":7649,"feed_emoji":"📐","tokens_out":7262,"duration_ms":77240,"temperature":0.7,"pith_summary":"CoPE claims that a transformer can carry positions in the imaginary part of its input embeddings and keep semantic content in the real part, avoiding the interference of additive positional encodings. The paper introduces a first-layer phase-aware attention that computes complex query-key overlaps, then reduces them to real scores through magnitude, phase, real, or hybrid variants; deeper layers stay standard. It proves that the positional term in these scores is oscillatory rather than exponentially decaying, and that the same mechanism can be rewritten as four real inner products, making it compatible with linear attention. On three GLUE tasks with a small model trained from scratch, the best CoPE variant matches or beats RoPE, sinusoidal, and learned embeddings while doing complex operations in only one layer instead of every layer.","feed_headline":"Complex position trick removes attention's distance penalty","feed_subtitle":"CoPE stores position in the imaginary part of one first-layer embedding, keeping attention oscillatory and linear-attention friendly.","key_machinery":"The complex input embedding E_complex = E_vocab + iγ sin(ω·pos), which stores content and position in orthogonal real and imaginary axes. The load-bearing mechanism is phase-aware first-layer attention: complex-valued query and key projections, a Hermitian inner product, and reduction of the complex scores to real values via one of five scoring rules (magnitude, phase cosine, real part, hybrid, hybrid-norm). The no-decay claim rides on the identity sin(ωp)sin(ωq) = ½[cos(ω(p−q))−cos(ω(p+q))], whose relative-distance term is oscillatory. The linear-attention compatibility claim rides on doubling real and imaginary features and decomposing the complex kernel into four real inner products that","core_discovery":"The paper's central object is the complex embedding z(x,p) = e_x + iγ sin(ωp), with token semantics in the real part and position in the imaginary part. In the first attention layer, queries and keys are obtained by complex-linear projections, and the attention score is the Hermitian inner product Q·K*. The positional part of this score expands to γ²[Q sin(ωp)·K sin(ωq)], which the paper treats as proportional to sin(ωp)sin(ωq) and rewrites as ½[cos(ω(p−q))−cos(ω(p+q))]. Because the relative term cos(ω(p−q)) has no multiplicative decay factor, the paper concludes CoPE does not impose long-term decay. It further shows that by lifting complex queries and keys to doubled real features, the atte","pith_inferences":["A direct test of the no-decay claim would be to train the smallest CoPE model, freeze it, and plot average attention weight versus relative distance for a fixed query token; if that curve falls systematically with distance, the claimed absence of long-term decay does not hold in the final attention distribution even if the raw score term is oscillatory.","Because the imaginary part uses sinusoidal encoding, CoPE may inherit sinusoidal extrapolation failures beyond the training length; the paper lists this as a limitation, but the phase-aware first layer could behave differently and is worth testing head-to-head against ALiBi.","The phase and magnitude variants win different tasks, so the practical value may lie in the architectural separation of content and position rather than in any single scoring rule; comparing CoPE on long-document retrieval would clarify which variant to use.","CoPE's complex first layer could be combined with existing linear-attention long-context models without per-layer changes, a testable route to long-context efficiency that the paper does not run."],"forward_implications":["A transformer built with CoPE can attend to distant positions without the built-in penalty that RoPE and similar encodings impose, so retrieval from arbitrary context positions is not discounted by distance.","CoPE can be plugged into linear-attention models: the same first layer gives relative position signal while retaining O(N) complexity, not the O(N²) of full attention.","Replacing RoPE's per-layer rotations with a single complex first layer cuts the positional-transform cost by roughly a factor of L, the number of layers.","The real/imaginary split gives a clean separation of content and position, so downstream layers receive already-position-aware representations without additional positional bias.","The variant list (magnitude, phase, real, hybrid, hybrid-norm) gives model builders a dial for how much phase difference should control attention."],"supporting_citations":[{"why":"Supplies the Transformer architecture, the sinusoidal positional encoding used in the imaginary part, and the sinusoidal baseline.","marker":"Vaswani et al. [2023]"},{"why":"Defines RoPE, the main comparison baseline and the source of the per-layer rotation cost argument.","marker":"Su et al. [2023]"},{"why":"Supplies the complex waveform positional embedding idea (amplitude, frequency, phase) that CoPE extends by putting content in the real part.","marker":"Wang et al. [2020b]"},{"why":"Provides the complex-valued attention construction and cosine phase-similarity used in phase-aware attention.","marker":"Eilers and Jiang [2023]"},{"why":"Motivates removal of the long-term-decay bias; CoPE's no-decay theorem responds to this work.","marker":"Chen et al. [2024]"},{"why":"Gives the linear attention kernel form used in the compatibility proof.","marker":"Katharopoulos et al. [2020]"},{"why":"Supplies the learned-positional-embedding baseline used in the GLUE comparison.","marker":"Wang et al. [2020a]"}],"fun_headline_variants":["CoPE's imaginary positions cancel attention decay","No distance penalty: complex positions in first layer","Phase-aware attention with no long-term decay","Complex position encoding avoids attention's distance fade"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The no-decay proof assumes that the learned projection matrices scale the sinusoidal position vectors like numbers, so the positional part of an attention score is just a constant times sin(query position) times sin(key position); if the projections mix position with content, the decay-free conclusion does not follow.","fun_headline_variants_meta":{"raw":{"variants":["CoPE's imaginary positions cancel attention decay","No distance penalty: complex positions in first layer","Phase-aware attention with no long-term decay","Complex position encoding avoids attention's distance fade"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001091,"raw_usage":{"total_tokens":4372,"prompt_tokens":697,"completion_tokens":3675,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":441,"completion_tokens_details":{"reasoning_tokens":3619}},"tokens_in":441,"tokens_out":3675,"duration_ms":35432,"temperature":1.0,"reasoning_tokens":3619,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T17:07:24.128447+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the smallest CoPE model on SST-2, freeze it, and for a fixed query token plot the average attention weight assigned to the same token placed at every relative distance from 1 to 256. If the curve falls systematically with distance, for example exponentially, the claimed absence of long-term decay is false. A cheaper calculation-level check is to compute Q_proj·sin(ωp)·K_proj·sin(ωq) with the trained matrices and compare it with sin(ωp)sin(ωq); large deviations break Eq. (8), on which the no-decay conclusion rests.","supporting_citations":[],"review_version":1}