{"id":"a281dfd6-95b4-459a-ba8d-0db44e93b0f9","arxiv_id":"2505.15507","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A D-axis composition algebra on (vector, matrix-power) tuples provides associative per-axis operators and an interchange law when axis matrices commute, recovering RoPE, affine embedding composition, and SSM-style recurrences as special cases.","lead":"This paper defines a multi-axis algebra for composing embeddings, with a distinct associative, non-commutative operator for each axis and an interchange law that links the axes when their transformations commute. The authors argue that this one framework generalizes RoPE, structured state-space recurrences, and transformer attention, but they include no experiments.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Shift-invariance of the §6.2 m-representation is false for finite signals as stated; the claim needs a boundary convention or proof, and the conditional verdict should stand pending correction.","rationale":"The §3 composition operation and interchange law are internally consistent, and the paper honestly disclaims experiments. The reader's commutation concern is real, but it is an explicit design restriction rather than a false theorem. The more decisive issue I find is in §6.2: the headline shift-invariance property is false for finite non-periodic signals, and the paper's own equations make this easy to check. This strengthens rather than replaces the conditional verdict: the framework needs correction or qualification before its advertised multi-dimensional representations can be relied on. I partially agree with the reader's weakest-assumption selection, but I would locate the decisive problem at the concrete false claim rather than at the expressivity trade-off.","tokens_in":9811,"tokens_out":23266,"duration_ms":213592,"concrete_test":"Implement equations (1)-(3) for m=2, K=2 with a block rotation R and compare the summed block-norm representation for a=((1,0,0,0),(0,0,0,0),(0,0,0,0)) and its one-step shift a'=((0,0,0,0),(1,0,0,0),(0,0,0,0)). If the two outputs differ (they do), the shift-invariance claim fails exactly as stated; checking this by hand on the same example localizes where the shifted window set ceases to be a permutation of the original windows.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The advertised property in §6.2, that the m-representation is invariant to global shifts along any axis, fails under the paper's own finite-signal definition. Equations (1)-(3) sum over N-m+1 windows with no cyclic wrap or padding convention. Counterexample: let m=2, K=2, d=4, and let R be any block rotation with two 2×2 blocks. Take N=3 with a_1=(1,0,0,0), a_2=a_3=(0,0,0,0). Then s_1=a_1, s_2=0, so v=(1,0). After a one-step shift, a'=(0,a_1,0); s'_1=R a_1 has first block norm 1, and s'_2=a_1 has first block norm 1, so v'=(2,0). The representation changes under a translation. The same failure occurs in higher dimensions because summing local window norms over a finite, non-cyclic support depends on boundary alignment. A correct invariance proof would need periodic boundary conditions with all N windows, a condition the text neither states nor uses. This does not invalidate the core §3 algebra, which is sound, but it removes a central advertised benefit of the framework and should be corrected or explicitly scoped.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces an algebraic structure for multi-dimensional compositional embeddings. An element is a pair (a, R_1^{n_1}, ..., R_D^{n_D}) with a content vector and one invertible axis transformation per axis, and axis-k composition is defined as (a + R_k^{n_k} b, ..., R_k^{n_k+m_k}, ...). The paper proves associativity along each axis and an interchange law that holds exactly when the axis transformations pairwise commute. It then sketches applications: 1D sequence composition, an m-representation claimed to be shift-invariant, concatenation of structured embeddings, non-commutative self-attention, and special-case recoveries of RoPE, SSM recursions, and transformer attention. No experiments are included; the paper explicitly defers empirical validation to future work.","tokens_in":10029,"tokens_out":8114,"duration_ms":72152,"significance":"The core Section 3 algebra is coherent and checkable by direct calculation: associativity per axis and the interchange-law equivalence are elementary but clean results, and the recovery of RoPE as a special case is a genuine verification of the framework's 1D scope. If the broader claims were fully supported, the framework would offer a useful unifying notation for compositional positional encodings and recurrence. However, the advertised shift-invariance of the m-representation is false as stated, one displayed composition formula contradicts the paper's own definition, and the claimed unification of SSMs and transformers is asserted rather than proved. The contribution is therefore promising but currently overstated, and the manuscript needs substantive corrections before the central benefits can be relied upon.","major_comments":[{"comment":"The assertion that the m-representation is invariant to global shifts is false for the finite-signal definition given, because the sum over N−m+1 windows has no cyclic or padding convention. For m=K=2, d=4, N=3, take a1=(1,0,0,0) and a2=a3=0; then v=(1,0), while after a one-step shift a'=(0,a1,0) one obtains v'=(2,0), so the representation changes. The claim needs periodic boundary conditions, a padding convention, or an explicit restriction to infinite signals; as written it must be corrected or removed.","section":"§6.2, Eqs. (1)–(3)"},{"comment":"The displayed expansion of E is inconsistent with the composition rule in §3.1. Under (a,A)∘(b,B)=(a+Ab, AB), the value of (((e1∘e2)∘e3)⋯∘eT) is v1 + R1v2 + R1R2v3 + ⋯ + R1⋯R_{T−1}vT, not v1 + R1v2 + R2R1v3 + ⋯ + R_{T−1}⋯R2R1vT. Either the product order or the displayed formula must be changed; the current text gives conflicting results.","section":"§6.1"},{"comment":"The block partition in the m-representation construction is not well-defined. The text fixes d=mK and says each block of R is K×K, which gives m blocks, but then asks to partition sk∈R^{mK} into K blocks in R^m. These two specifications agree only when m=K. Please state the intended block size and consistently index the magnitude vector vk.","section":"§6.2"},{"comment":"The claim that the framework unifies or 'reduces to' SSMs and transformers is asserted rather than demonstrated. Equations (7) and (8) show a formal resemblance, but no explicit reduction is given for how uniform attention and a learned recurrent transition structure arise from the compositional operators, nor what constraints on α and R recover Eq. (6). The abstract's unification claim should either be proved as a precise special-case statement or appropriately qualified.","section":"§6.6, Eqs. (6)–(8)"}],"minor_comments":[{"comment":"The phrase 'without imposing global commutativity' is potentially misleading, since the interchange law requires the axis operators to commute pairwise; please clarify that the intended non-commutativity is within each axis, while across-axis commutativity is an explicit design constraint.","section":"Abstract and §3.3"},{"comment":"The notation R_k is called a 'shift operator' and R_k^{n_k}Y is said to 'concatenate Y immediately after X', but no formal definition of applying R_k to a structured embedding is provided; the operation should be defined or the text should make clear that this is only an analogy.","section":"§6.3"},{"comment":"The mentions of capsule networks and graph neural networks are not accompanied by citations at those points; please add the relevant references ([3] and [4]) where they are discussed.","section":"§5.1"},{"comment":"The operation ⊕k is introduced with u_i = max(n_i, m_i) for i≠k, which is not the same as the axis-k composition ◦k that requires all other axes to match; the relationship between ⊕k and ◦k should be stated explicitly.","section":"§6.4"}],"recommendation":"major_revision","confidential_remarks":"The central algebraic construction is sound but modest, and several of the paper's advertised benefits — shift-invariance, recovery of SSMs, and the global consistency claim — come with restrictions or need more precise statements. The false shift-invariance claim in §6.2 and the formula inconsistency in §6.1 are fixable within the manuscript's scope, so I recommend major revision rather than rejection. If the authors scope the claims carefully, the paper could become a reasonable theoretical contribution for a venue that accepts theory without experiments."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the algebra in Section 3 is sound and clean, but the paper overstates both its novelty and one of its flagship properties. The 1D composition rule is the familiar matrix-vector composition (Socher et al. 2012; Rudolph and Giesbrecht 2010), and the D-axis extension with pairwise commuting rotations is a direct algebraic restatement of 2D RoPE (Heo et al. 2024). What is new is the specific tuple-object (vector, R_1^{n_1}, ...) with axis-wise composition and the interchange law; that object is coherent and the proofs in Section 3 are verifiable by direct calculation. I checked the associativity and interchange claims; they hold under the stated commuting condition.\n\nThe paper is honest that it has no experiments and lists limitations. The writing is clear.\n\nThe soft spots are real but localized. The shift-invariance claim in §6.2 is false as stated for finite signals. Equations (1)-(3) define window sums over N-m+1 positions with no padding or cyclic wrap. A one-step shift changes which elements fall in which window, and the norm-pooled vector v changes; the stress-test counterexample (m=2, two nonzero values one step apart) is correct. Fixing it requires either periodic boundary conditions with all N windows or an explicit statement that the property holds only approximately or in the infinite-signal limit. That is not a minor typo; it is the paper's central advertised benefit.\n\nSecond, the 'unifies SSM and transformer' claim is thinner than it looks. Equation (8) writes y_k = sum alpha_ik (prod R_j) V_i, which reduces to SSM when alpha are implicit and to attention when R=I. That is a special-case observation, not a new derivation, and it does not actually show any new architecture. Third, the implementation restriction to commuting block-diagonal rotations (Section 8) is exactly the regime where 2D RoPE already operates; non-commuting learned transforms would break the interchange law, so the claimed generality is conditional on a fairly narrow structure.\n\nOverall: the core algebra is a reasonable stepping stone and may be useful as a notational language for multi-axis embeddings. It deserves peer review because the formal part is coherent and the limitations can be fixed, but the authors should be asked to correct or scope the shift-invariance claim and moderate the 'first' and 'unification' statements.","headline":"A clean but modest algebraic restatement of known matrix-vector composition with a D-axis twist; the central laws check out, but the advertised shift invariance fails as stated and the unification claims outrun the content.","tokens_in":10583,"tokens_out":2385,"would_cite":false,"duration_ms":21621,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A per-axis composition algebra unifies SSMs, attention, and RoPE embeddings.","keywords":["directional non-commutative monoids","compositional embeddings","interchange law","rotary position embeddings","state space models","self-attention","multi-axis representation","shift invariance"],"falsifier":"Take two non-commuting rotation matrices $R_1$ and $R_2$, for instance $3\\times 3$ rotations about different axes in three dimensions, and four one-dimensional embedding elements; substituting them into $(x \\circ_1 y) \\circ_2 (z \\circ_1 w)$ versus $(x \\circ_2 z) \\circ_1 (y \\circ_2 w)$ produces unequal vectors, demonstrating that the interchange law fails exactly when the commutation assumption is dropped.","tokens_in":9575,"feed_emoji":"🧩","tokens_out":6673,"duration_ms":54568,"temperature":0.7,"pith_summary":"This paper proposes a new algebraic structure for composing embeddings along multiple axes: each axis gets its own associative but non-commutative composition operator, and the operators are designed to obey a global interchange law. The central claim is that this structure is a common foundation from which familiar sequence-modeling mechanisms—affine composition, rotary position embeddings (RoPE), structured state-space model (SSM) recurrences, and vanilla transformer self-attention—all emerge as one-dimensional special cases. The paper proves the algebraic properties, gives efficient parameterizations via block-diagonal rotations, and sketches applications such as multi-axis relative attention, directional image embeddings, and shift-invariant representations. It is a theoretical contribution and includes no experiments.","feed_headline":"One algebra unifies SSMs, attention, and RoPE","feed_subtitle":"A per-axis composition operator with an interchange law recovers familiar sequence models as special cases.","key_machinery":"The load-bearing object is the axis-specific composition operator $\\circ_k$ on tuples $(a; R_1^{n_1}, \\ldots, R_D^{n_D})$, where $a$ is a vector and each $R_i$ is an invertible matrix acting as a ``step'' along axis $i$. Composition along axis $k$ adds the second vector after shifting it by $R_k^{n_k}$ and increments the exponent $n_k$ by $m_k$; associativity along each axis follows because matrix powers compose by exponent addition. The interchange law is what couples the axes, and it holds if and only if the per-axis matrices commute, which is exactly what the proposed efficient implementation (block-diagonal $2\\times 2$ rotations, composed by adding angles) guarantees. The machinery also includes a shift-invariant ``$m$-representation'' built by sliding a window, summing rotated contents, and pooling block magnitudes.","core_discovery":"An embedding element is represented as $x = (a; R_1^{n_1}, \\ldots, R_D^{n_D})$, a content vector together with an invertible transformation per axis raised to an integer extent. Composition along axis $k$, written $x \\circ_k y$, is defined when all other axis extents match and gives $(a + R_k^{n_k} b; \\ldots; R_k^{n_k+m_k}, \\ldots)$. This operation is associative along each axis, and the global interchange law $(x \\circ_i y) \\circ_j (z \\circ_i w) = (x \\circ_j z) \\circ_i (y \\circ_j w)$ holds exactly when $R_i R_j = R_j R_i$. The paper shows that in one dimension this recovers the affine composition $(a,A) \\circ (b,B) = (a + Ab, AB)$; with a fixed block-diagonal rotation $R$ the relative transform $T_{p,q} = R^{p-q}$ reproduces RoPE, and in $D$ dimensions $T_{p,q} = \\prod_i R_i^{n_{p,i}-n_{q,i}}$ yields a multi-axis relative attention. With uniform attention and a recurrent transition this same expression reduces to an SSM-style scan, so the framework is presented as unifying transformer attention, SSM recurrence, and rotary positional encoding in one algebraic system.","pith_inferences":["The paper leaves implicit that the commuting-rotation regime is essentially the distance-only regime already explored by RoPE; richer non-commuting per-axis transforms would forfeit the interchange law, so the claimed coherence of cross-axis composition is bought at the cost of expressivity.","One testable extension is to use the $m$-representation as a provably shift-invariant embedding for images or audio and measure whether the invariance survives downstream classification without losing discriminative structure.","The framework could be pushed toward non-commuting axes by replacing the interchange law with a braiding relation, which would allow genuinely directional multi-axis composition at the price of weaker global consistency."],"forward_implications":["In transformer attention, relative position can enter multiplicatively as a journey operator $T_{p,q} = \\prod_i R_i^{n_{p,i}-n_{q,i}}$, making attention depend on the full multi-axis offset rather than a scalar distance.","One-dimensional and two-dimensional rotary position embeddings are recovered as special cases when all $R_i$ are fixed commuting block rotations, so existing RoPE-based architectures fit inside the framework.","With uniform (implicit) attention weights and a learned recurrent transition, the compositional attention expression reduces to an SSM-style scan; with no explicit recurrence and the standard dot-product attention, it reduces to vanilla transformer self-attention.","Whole grids, images, or video volumes can be encoded as a single composed embedding $E = \\sum_{i,j} R_x^{i-1} R_y^{j-1} v_{ij}$, with the interchange law making the order of row and column composition irrelevant.","Because composition is associative, embeddings can be computed with parallel scans and blockwise grouping, and with $2\\times 2$ rotation blocks the updates reduce to angle addition."],"supporting_citations":[{"why":"Supplies the rotary position embedding scheme that the paper recovers as a one-dimensional special case.","marker":"[17]"},{"why":"Provides the 2D RoPE extension that the paper shows matches its two-axis composition with commuting $R_x, R_y$.","marker":"[18]"},{"why":"Gives the structured state-space model recurrence the paper generalizes.","marker":"[21]"},{"why":"Another SSM instance whose selective recurrent scan the paper's associative composition generalizes.","marker":"[23]"},{"why":"Defines vanilla self-attention, which the paper's non-commutative attention expression recovers when the transition factors are trivial and weights are the usual softmax.","marker":"[9]"},{"why":"Supplies the non-commutative matrix-space model of language that motivates directional composition in one dimension.","marker":"[7]"},{"why":"Matrix-vector recursive composition is the immediate ancestor of the one-dimensional affine composition $(a,A) \\circ (b,B) = (a + Ab, AB)$.","marker":"[8]"},{"why":"Categorical compositional distributional semantics is the background example of algebraically enforced composition in meaning.","marker":"[5]"}],"fun_headline_variants":["Non-commutative axes unify SSM, attention, RoPE","Interchange law unifies SSM, attention, RoPE","Per-axis composition unifies SSM, attention, RoPE","Same algebra behind SSM, attention, RoPE","Directional algebra unifies SSM, attention, RoPE"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The global interchange law, and with it the claimed coherence of cross-axis composition, holds only when all per-axis transformation matrices commute, and the practical construction secures this by restricting every axis to independent two-dimensional rotations.","fun_headline_variants_meta":{"raw":{"variants":["Non-commutative axes unify SSM, attention, RoPE","Interchange law unifies SSM, attention, RoPE","Per-axis composition unifies SSM, attention, RoPE","Same algebra behind SSM, attention, RoPE","Directional algebra unifies SSM, attention, RoPE"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001803,"raw_usage":{"total_tokens":7178,"prompt_tokens":1099,"completion_tokens":6079,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":715,"completion_tokens_details":{"reasoning_tokens":5992}},"tokens_in":715,"tokens_out":6079,"duration_ms":40121,"temperature":1.0,"reasoning_tokens":5992,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:16:35.601574+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take two non-commuting rotation matrices $R_1$ and $R_2$, for instance $3\\times 3$ rotations about different axes in three dimensions, and four one-dimensional embedding elements; substituting them into $(x \\circ_1 y) \\circ_2 (z \\circ_1 w)$ versus $(x \\circ_2 z) \\circ_1 (y \\circ_2 w)$ produces unequal vectors, demonstrating that the interchange law fails exactly when the commutation assumption is dropped.","supporting_citations":[{"cited_title":"V aswani, N","cited_arxiv_id":null,"evidence_quote":"Defines vanilla self-attention, which the paper's non-commutative attention expression recovers when the transition factors are trivial and weights are the usual softmax."},{"cited_title":"Rudolph and E","cited_arxiv_id":null,"evidence_quote":"Supplies the non-commutative matrix-space model of language that motivates directional composition in one dimension."},{"cited_title":"Socher, B","cited_arxiv_id":null,"evidence_quote":"Matrix-vector recursive composition is the immediate ancestor of the one-dimensional affine composition $(a,A) \\circ (b,B) = (a + Ab, AB)$."},{"cited_title":"Coecke, M","cited_arxiv_id":null,"evidence_quote":"Categorical compositional distributional semantics is the background example of algebraically enforced composition in meaning."}],"review_version":1}