{"id":"0deb58fc-9e7e-4b17-9eff-a023d8973d01","arxiv_id":"2505.06257","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"The Co4 mechanism adds triadic Q-K-V modulation loops before attention, claiming O(N) complexity and much faster learning than standard Transformers on small benchmarks.","lead":"In an attention mechanism, the author adds biologically inspired 'question, clue, hypothesis' loops that pre-filter information before attending, and reports faster learning on small reinforcement learning, vision, and question-answer tasks. The paper is an early proof-of-concept for a linear-complexity transformer variant, but it ships no code and relies on small, mostly single-run experiments.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central mechanistic claim is confounded: Co4 changes both the modulation loop and the query mechanism, so no experiment isolates the triadic loop; a latent-query-only control is required.","rationale":"The reader's weakest assumption was the scaling of L with input length. I read that as a fair but secondary concern: even with fixed L, the central claim requires that the triadic loop, not the latent-query mechanism, is the source of the gains. The minimal experiment for that attribution is a latent-query-only control. The paper itself notes that a more detailed analysis will follow and calls for further critical testing of Eq. (1), which confirms that the mechanism was not isolated. The lack of released code and error bars strengthens, but does not replace, this argument. I do not see an internal contradiction in the reported numbers; the issue is confounded experimental design. I keep the reader's CONDITIONAL verdict: the required control is cheap and well-defined, so rejection is too strong, but acceptance without it is not justified. Because the reader already lists 'compare against latent-query attention baselines' as a condition, my read does not move the verdict, hence UNCHANGED; agreement is partial because my primary load-bearing concern is the attribution of the effect to the modulation loop, not the L-scaling assumption.","tokens_in":14883,"tokens_out":11713,"duration_ms":128071,"concrete_test":"Run the paper's exact CIFAR-10 and synthetic bAbI pipelines (Tables 3 and 4) with a single change: bypass the Cooperation update by setting Q_m=Q, K_m=K, V_m=V, then compute ordinary scaled dot-product attention between the L learnable latent queries and the N K/V tokens, keeping L, heads, depth, parameter count, augmentation, optimizer, epochs, and seeds identical to the reported Co4 runs. Repeat the same control on the CartPole and PyBullet Ant setups behind Figure 6. If the latent-query-only baseline matches Co4's accuracy and convergence curves, the triadic loop is not responsible for the reported gains; if Co4 remains clearly faster and more accurate, the concern is settled.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Every reported Co4 result varies two ingredients together: (i) the Eq. (1) Cooperation function applied to Q, K, and V, and (ii) a set of L learnable latent query tokens whose attention map is L×N (Section 4; Section 4.1). The baselines in Tables 3 and 4 are standard Transformers whose queries come from the input tokens, not from a latent array; the RL baselines follow [60] with the same limitation. A latent-query bottleneck alone—the Perceiver idea, cited as [2]—already reduces attention cost to O(LN) and can accelerate learning on small tasks. The headline gains are therefore equally consistent with a much weaker explanation: Co4 is essentially 'latent queries plus ordinary attention,' and the triadic modulation contributes little. No experiment sets C=0 or replaces Eq. (1) with the identity while holding L, heads, layers, parameter count, epochs, and seeds fixed. This is an omitted control, not an internal contradiction, but it is load-bearing because it targets the causal claim that triadic modulation loops enable the faster, cheaper learning. The reader's L-scaling concern is real, but it presupposes the loop matters; if the loop does not matter, the 'beyond attention' claim fails even at fixed L.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Co4, a transformer variant that replaces the standard Q-K-V attention computation with triadic modulation loops among latent query tokens (Q), keys (K), and values (V), using a nonlinear \"Cooperation Equation\" (Eq. 1) inspired by context-sensitive pyramidal-cell physiology. The claimed benefits are faster learning, fewer required heads/layers/tokens, and near-linear O(N) complexity. Experiments are reported on three RL tasks (CartPole, PyBullet Ant, CarRacing), CIFAR-10 image classification, and a synthetic bAbI-style QA benchmark, always comparing Co4 against standard Transformer baselines. The paper concludes that Co4 embeds representation-level reasoning and may be a step toward machines with intrinsic higher mental states, while acknowledging that the current evidence is from quick proof-of-concept tests.","tokens_in":15121,"tokens_out":6091,"duration_ms":61115,"significance":"If the central claim were established, Co4 would be a practically relevant contribution: a transformer variant that achieves competitive accuracy with substantially fewer heads, layers, and tokens, at near-linear attention cost. The paper is also valuable for explicitly connecting cellular neuroscience (apical amplification, two-point neurons) to a concrete architectural mechanism, and for testing the idea across RL, vision, and QA settings. Credit is due for the explicit complexity analysis, for comparing against established baselines, and for clearly stating that the experiments are quick-to-run proof-of-concept tests rather than a full evaluation. However, the current evidence does not yet justify the headline claims: the experimental design confounds the latent-query mechanism with the Eq. (1) modulation, the linear-complexity claim depends on an untested assumption, and the empirical results lack repeated trials, error bars, and code. The contribution is therefore promising but preliminary.","major_comments":[{"comment":"The central causal claim that triadic modulation loops 'enable ... faster learning' is confounded, because every Co4 configuration changes two ingredients at once: (i) the latent query mechanism (L learnable query tokens interacting with N input tokens) and (ii) the Eq. (1) nonlinear modulation applied to Q, K, and V. The Transformer baselines use input-derived queries, and no control is run that sets C=0 or replaces Eq. (1) by the identity while holding L, heads, layers, parameter count, epochs, and seeds fixed. Since Perceiver ([2]) already shows that a latent-query bottleneck alone gives O(LN) attention and can accelerate learning on small tasks, the reported gains are equally consistent with 'latent queries plus ordinary attention.' An ablation is required to attribute the results to the triadic modulation.","section":"Section 4, Tables 3-4"},{"comment":"The O(N) complexity claim rests on the assumption that the number of latent query tokens L (or L_q) is a small, fixed fraction of the input length, but no experiment varies N while holding L fixed, and no argument is given that L does not need to grow with task complexity or sequence length. If L grows with N, the attention cost O(L*N) is no longer linear. The MAC comparison also needs clarification: the standard-Transformer formula includes the FF-network term PE^2, yet Section 5.1 states that all results are presented without the second residual (FF) connection; it should be stated explicitly whether the Transformer baselines include or exclude the FF block, and the complexity comparison should be made on the same architectural footing.","section":"Section 4.1"},{"comment":"The empirical support is too thin to substantiate 'orders-of-magnitude faster learning with significantly reduced computational demand.' Tables 3 and 4 report single validation accuracies without error bars or seeds; Figure 6 shows training curves without wall-clock time or FLOPs measurements; and no code or configuration files are released. The paper itself acknowledges that these are 'quick-to-run tests' with a more detailed analysis to follow, so the quantitative claims in the abstract should be either softened or supported by multiple seeds and direct timing/complexity measurements.","section":"Section 5, Tables 3-4, Figure 6"},{"comment":"The statement that Co4 uses 'the same architecture and number of parameters' as the Transformer baseline is not substantiated. In Table 3 the parameter counts are identical for Transformer and Co4 (e.g., 0.215M, 0.61M, 1.20M), but Co4 adds learnable latent query vectors and Eq. (1) operations; if the counts are indeed identical, the baseline must contain compensating parameters somewhere. The paper should specify the exact module composition of both models, including the presence or absence of the feedforward residual block, so that the comparison is interpretable.","section":"Section 5.3, Table 3"}],"minor_comments":[{"comment":"Equation (8) contains a parenthesis error: 'C(1 + |R(Ot,a t−1))|)' should read 'C(1 + |R(Ot,a t−1)|)'.","section":"Section 5.2, Eq. (8)"},{"comment":"Reference [66], 'Cooperation is All You Need', is cited only in the Acknowledgments but appears to be the source of the Cooperation equation; it should be cited at Eq. (1) and the relation to that work should be discussed in the text.","section":"Section 4, Eq. (1)"},{"comment":"The phrase 'representation-level Chain-of-Thought' and 'silent yet deep form of Chain-of-Thought' is presented as a factual property, but the paper provides no analysis of reasoning chains; the claim should be labeled as an analogy or supported by evidence.","section":"Section 4, Figure 3 caption"},{"comment":"The dataset name is spelled inconsistently as 'bAbI' and 'bABi'; please use one consistent spelling throughout.","section":"Section 5.4, Tables and text"},{"comment":"The symbol L is used both for the number of layers and for the number of latent query tokens, which makes the complexity formulas confusing; use one symbol for layers and a distinct symbol (e.g., L_q) for latent tokens consistently.","section":"Section 4.1"},{"comment":"The text refers to 'Figure 6 (bottom right)' for the CarRacing result, but the panel is not labeled in the caption; please label each subplot.","section":"Figure 6"}],"recommendation":"major_revision","confidential_remarks":"The paper is best treated as an early-stage proposal. The main risk is that the headline claims go beyond what the experiments establish; the authors should be required to add a latent-query-only ablation, report repeated runs with seeds, and release code before acceptance. The disclosed provisional patent application is not itself a problem, but it strengthens the case for requiring independent reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nShort version: this is an extension of latent-query attention with a biologically flavored nonlinearity (the Cooperation equation) applied to Q, K, and V before attention. The specific triadic loop is new relative to Perceiver and DeepSeek, and the reported gains are consistent across a few small benchmarks. But the experiments never isolate the loop. Every Co4 result changes two things at once: the latent-query bottleneck and the modulation. The stress-test note is right that this is the load-bearing omission.\n\nWhat the paper does well: the mechanism is concrete enough to reimplement from the text, the complexity analysis in Section 4.1 is honest about the O(NL) cost with L small, and the framing as a proof-of-concept is appropriate. The admission that eq. (1) was chosen after empirical experimentation is a plus, even though it undercuts the “intrinsic” language.\n\nSoft spots are real. No error bars on CIFAR-10 or bAbI, single runs, no released code, no wall-clock time or FLOPs measured, so the “orders-of-magnitude faster learning” is not directly substantiated. The abstract says O(N), but the mechanism is O(NL) with an untested assumption that a fixed L of 4 or 8 suffices for arbitrary N. The baselines are vanilla transformers or the sensory-neuron paper [60], so any latent-query variant might win. I would want four things before believing the strong claim: (1) a latent-query-only control with the Cooperation equation set to identity, (2) repeated seeds with error bars, (3) actual training time comparison, and (4) a scaling experiment varying N and L.\n\nThere is no internal contradiction here. The machinery is plausible, the gains are in the right direction, and the neuro story, while not load-bearing, is not silly. Citation patterns look fine; the self-citations are to the author's prior TPN work, which is the legitimate source of the nonlinearity.\n\nBottom line: this paper deserves a serious referee, but not acceptance as-is. With the missing controls and measurements supplied, it could be a decent contribution. I would not cite it in my own work yet, but I would bring it to a reading group as a case study in confounding by architecture choice.","headline":"A plausible latent-query attention variant whose headline claim about the triadic modulation loop is not actually tested, because every result varies the loop and the latent queries together.","tokens_in":15653,"tokens_out":2101,"would_cite":false,"duration_ms":23413,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Triadic Q-K-V loops before attention let a Transformer learn faster with near-linear cost.","keywords":["Co4","triadic modulation","transformer attention","latent queries","pyramidal neuron model","mental states","linear attention complexity","reinforcement learning"],"falsifier":"Run Co4 with fixed $L=4$ or $L=8$ on sequences of increasing length, for example synthetic bAbI-style stories with 60, 600, and 6000 tokens, and record accuracy and wall-clock cost; if accuracy drops when $L$ is held fixed and restoring accuracy requires $L$ to grow with $N$, the paper's approximate $O(N)$ claim is false.","tokens_in":14628,"feed_emoji":"🧠","tokens_out":5140,"duration_ms":52771,"temperature":0.7,"pith_summary":"The paper claims that a Transformer can learn much faster and with far fewer heads, layers, and tokens if it first runs triadic modulation loops among latent questions, clues, and hypotheses before applying attention. The proposed Co4 mechanism is inspired by two integration sites in neocortical pyramidal neurons, where contextual input amplifies only what is coherent with the current signal. If the claim holds, attention-based models could approach or exceed standard Transformer accuracy while cutting the asymptotic cost from $O(N^2)$ to roughly $O(N)$. The paper reports quick proof-of-concept results on reinforcement learning, CIFAR-10, and a synthetic bAbI question-answering task.","feed_headline":"Triadic Q-K-V loops before attention learn faster at ~O(N) cost","feed_subtitle":"A Transformer variant modulates questions, keys, and values before attending, matching deeper models with far fewer resources.","key_machinery":"The central object is the Cooperation Equation, $Cooperation(R,C)=ReLU6(R^2+2R+C(1+|R|))$, an asynchronous modulatory transfer function used by three populations of two-point-neuron-inspired units for $Q$, $K$, and $V$. In this function the contextual field $C$ acts as the driving force, amplifying coherent responses and attenuating incoherent ones regardless of the raw signal $R$'s strength. Triadic loops apply this function cyclically: $K$ and $V$ provide contextual input to $Q$; $Q$ and $V$ shape $K$; $Q$ and $K$ together drive $V$. This pre-attention co-adaptation is what embeds reasoning into a single lightweight attention step and removes the need for a feedforward residual block.","core_discovery":"The paper's central claim is that pre-attention triadic modulation among latent queries ($Q$), evolving keys ($K$), and evolving values ($V$) enables deep, parallel, representation-level reasoning that standard attention only approximates after many layers. Using a two-point-neuron-inspired Cooperation Equation, Co4 lets the latent question tokens adapt to the clues and hypotheses, lets the keys adapt to who is asking, and lets the values emerge from the coherent interaction of questions and clues. The final attention matrix is only $L \\times N$, where $L$ is a small fixed number of latent tokens (4 or 8 in the experiments), so the dominant cost becomes about $O(N)$. Across the reported tasks, Co4 reaches higher scores or accuracies in fewer epochs, often with one layer and one head, while the standard Transformer needs more depth and width to approach comparable performance.","pith_inferences":["Beyond the paper: a direct scaling test that fixes $L$ and grows $N$ across several orders of magnitude would decide whether the approximate $O(N)$ cost claim holds in practice.","Beyond the paper: the Cooperation Equation could be inserted as a context-gating layer in CNNs or RNNs, testing whether the speed-up comes from modulation in general or from its specific interaction with attention.","Beyond the paper: ablating the triadic loop to a one-shot conditioning step would reveal how much of the gain comes from bidirectional Q-K-V co-adaptation rather than from the extra nonlinearity alone."],"forward_implications":["With a small fixed $L$, the attention cost becomes about $O(N)$ instead of $O(N^2)$, so longer sequences cost far less at the same parameter count.","Co4 removes the feedforward residual block from the Transformer block, eliminating one of the most expensive per-layer components.","Shallow one- or two-layer Co4 models reach accuracies that standard Transformers need more heads and layers to approach, implying the same reasoning can be compressed into fewer resources.","On the synthetic bAbI-style task, Co4 reaches 96-98% accuracy with one or two layers and one or two heads, while the standard Transformer tops out near 77% in the reported setting."],"supporting_citations":[{"why":"Defines the standard Transformer attention mechanism that Co4 replaces and that serves as the main baseline.","marker":"[1]"},{"why":"Supplies the cellular-psychology evidence linking apical dendrite states to perceptual and imaginative mental states that motivate the triadic loop.","marker":"[4]"},{"why":"Introduces an earlier two-point-cell-inspired MOD function in CNNs that eq (1) extends to attention; the paper uses this as a starting point for TPN-style processing.","marker":"[45]"},{"why":"Frames Co4's representation-level modulation as a silent, parallel form of chain-of-thought reasoning without token generation.","marker":"[56]"},{"why":"Provides the permutation-invariant sensory-neuron architecture and baseline code for the reinforcement-learning experiments.","marker":"[60]"},{"why":"Provides the Compact Convolutional Transformer baseline and the CCT-Conv tokenization baseline for CIFAR-10 comparisons.","marker":"[61]"},{"why":"Provides the bAbI synthetic question-answering dataset that the NLP experiments adapt.","marker":"[62]"}],"fun_headline_variants":["Pre-attention Q-K-V loops slash compute to O(N)","Triadic modulation before attention: faster learning, O(N) cost","Co4: Deep reasoning before attention, at linear cost","Neuron-inspired modulation: learn faster with fewer resources","Imagination-like states pre-select relevance before attention"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The speed and near-linear cost claims rest on the assumption that a small fixed number of latent question tokens (4 or 8 in the experiments) remains sufficient as the input gets longer and tasks get harder; the paper reports no scaling experiment showing that $L$ can stay small as $N$ grows.","fun_headline_variants_meta":{"raw":{"variants":["Pre-attention Q-K-V loops slash compute to O(N)","Triadic modulation before attention: faster learning, O(N) cost","Co4: Deep reasoning before attention, at linear cost","Neuron-inspired modulation: learn faster with fewer resources","Imagination-like states pre-select relevance before attention"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001113,"raw_usage":{"total_tokens":4621,"prompt_tokens":916,"completion_tokens":3705,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":532,"completion_tokens_details":{"reasoning_tokens":3624}},"tokens_in":532,"tokens_out":3705,"duration_ms":24821,"temperature":1.0,"reasoning_tokens":3624,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:22:16.390624+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run Co4 with fixed $L=4$ or $L=8$ on sequences of increasing length, for example synthetic bAbI-style stories with 60, 600, and 6000 tokens, and record accuracy and wall-clock cost; if accuracy drops when $L$ is held fixed and restoring accuracy requires $L$ to grow with $N$, the paper's approximate $O(N)$ claim is false.","supporting_citations":[{"cited_title":"Cellular psychology: relating cognition to context-sensitive pyramidal cells,","cited_arxiv_id":null,"evidence_quote":"Supplies the cellular-psychology evidence linking apical dendrite states to perceptual and imaginative mental states that motivate the triadic loop."},{"cited_title":"Unlocking the potential of two-point cells for energy-efficient and resilient train- ing of deep nets,","cited_arxiv_id":null,"evidence_quote":"Introduces an earlier two-point-cell-inspired MOD function in CNNs that eq (1) extends to attention; the paper uses this as a starting point for TPN-style processing."},{"cited_title":"Chain-of-thought prompting elicits reasoning in large language models,","cited_arxiv_id":null,"evidence_quote":"Frames Co4's representation-level modulation as a silent, parallel form of chain-of-thought reasoning without token generation."},{"cited_title":"The sensory neuron as a trans- former: Permutation-invariant neural networks for re- inforcement learning,","cited_arxiv_id":null,"evidence_quote":"Provides the permutation-invariant sensory-neuron architecture and baseline code for the reinforcement-learning experiments."}],"review_version":1}