{"id":"27ad092c-c449-4f15-b13a-cdf79e9d2b41","arxiv_id":"2502.06415","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"The paper identifies activation, weight, and attention outliers as a single phenomenon caused by softmax attention and demonstrates that explicit context-aware scaling eliminates them.","lead":"This paper traces three types of outliers in large language models to the softmax operation inside self-attention, and argues they act as implicit, context-aware scaling factors. It shows that replacing them with an explicit learned scaling factor during training removes the outliers and makes models more compressible.","discovery_kind":"unification","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The causal chain rests on an untested 'near-zero update' premise; if weak tokens do not require (or receive) near-zero MHA updates, the softmax-root-cause claim lacks foundation.","rationale":"The paper's central claim is explanatory: softmax's sum-to-one constraint forces low-update tokens to be handled by attention concentration, and the resulting dynamic range produces the observed outliers. I read Appendix C's derivation as resting entirely on the normative premise that such tokens have 'desired behavior is a near-zero update.' This premise is not derived and not tested; it is also not obviously true. The paper's own Appendix C.4 says initial tokens aggregate global information and have amplified MHA outputs, which suggests nonzero updates. The variant experiments are interesting: explicit scaling and attention bias prevent outlier formation, and sigmoid attention also removes outliers; this is real evidence that sum-to-one softmax is associated with outliers. But preventing outliers with a modified architecture does not prove that the default model's outliers function as context-aware scaling factors, nor that the zero-update requirement drove their formation. The compression results on GPT-2 are practically interesting even if the causal story is incomplete, but they do not rescue the explanatory claim. Thus the conditional verdict is appropriate: the paper should measure actual updates for weak/initial tokens, provide a formal argument for why zero-update requires concentration, or run an ablation demonstrating dependence on the presumed zero-update mechanism. I agree with the reader that this is the weakest load-bearing assumption.","tokens_in":18307,"tokens_out":10847,"duration_ms":103934,"concrete_test":"On LLaMA-2-7B with 100 RedPajama sequences, compute for every query position the L2 norm of the MHA output before the residual connection. Split positions by whether their top-attended key is an outlier/attention-sink token (initial token or punctuation) or a content token. If the Appendix C.1 premise is correct, sink-attending queries should have near-zero output norms; report the medians and the fraction below 0.1x the median content-token norm. Then set the MHA output of those sink-attending positions to zero and measure WikiText-2 perplexity: if PPL degrades substantially, the model depends on nonzero updates for these tokens, contradicting the 'desired near-zero update' premise; if PPL is unchanged, the premise is at least not contradicted descriptively, but its necessity still requires a formal argument.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central causal claim depends on Appendix C.1's premise that initial and weak-semantic tokens 'require minimal contextual updates' and that the 'desired behavior is a near-zero update' of MHA. Every later step—softmax-induced dynamic-range expansion, gradient amplification, propagation into MLP weight outliers—is conditional on this requirement. The paper never tests it: no measurement of MHA output norms for those tokens in LLaMA-2-7B is reported, and no derivation shows that zero-update entails a concentrated attention distribution (a distributed value-weighted mixture can also cancel to zero). There is also internal tension: Appendix C.4 says initial tokens 'aggregate global information' and have amplified MHA values, which is hard to reconcile with a near-zero-update requirement. The Section 5.2 variant experiments show that explicit scaling factors prevent outlier formation, but preventing a phenomenon in an altered architecture does not establish that the original outliers function as scaling factors or that the zero-update requirement drove their emergence. If this premise is false or merely an artifact of softmax training, the causal chain from softmax to systematic outliers loses its foundation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies three types of outliers in LLMs—activation, weight, and attention outliers—and argues that they are systematically interconnected and originate from the softmax operation in self-attention. After documenting the distribution and localization of these outliers across LLaMA-2 and several other models, the authors hypothesize that outliers act as implicit, context-aware scaling factors. They test this hypothesis by training GPT-2 variants with alternative attention formulations (fixed bias, context-aware bias, attention bias, and explicit context-aware scaling factor), finding that only formulations with a scaling factor prevent outlier formation. They further report that an explicit context-aware scaling factor improves robustness to quantization and pruning and accelerates early convergence. Appendix C provides a mathematical narrative connecting softmax-induced dynamic range expansion, zero-update requirements for certain tokens, and gradient amplification to the emergence and localization of outliers.","tokens_in":18528,"tokens_out":2783,"duration_ms":26067,"significance":"If the central claim is correct, the paper would provide a unified mechanistic explanation for three previously separate outlier phenomena and a principled architectural mitigation. The empirical survey across LLaMA-2/3, Mistral, OPT, MPT, Falcon, Phi-2 and fine-tuned variants is a useful systematic documentation, and the GPT-2 variant experiments are an interventionist test that cleanly separates bias effects from scaling effects at a qualitative level. The authors also ship code and give detailed experimental settings, supporting reproducibility. However, the causal derivation in Appendix C rests on an untested assumption about near-zero MHA updates for certain tokens, and the intervention experiments demonstrate that explicit scaling factors prevent outliers without directly establishing that the original outliers function as scaling factors. These gaps are load-bearing for the paper's main claim, but they are addressable with additional measurements and a more carefully scoped interpretation.","major_comments":[{"comment":"The zero-update premise is asserted, not tested. The derivation begins with 'the desired behavior is a near-zero update' for initial and weak-semantic tokens and defines ∆x = MHA(Qx, K, V) ≈ 0, but no measurement of MHA output norms for those tokens in LLaMA-2-7B or any other model is reported. Since every subsequent step (dynamic range expansion, gradient amplification, weight outlier formation) is conditional on this premise, the paper should either measure actual MHA output magnitudes for the claimed outlier tokens or explicitly weaken the claim to a conditional hypothesis.","section":"Appendix C.1"},{"comment":"There is an internal tension between the zero-update requirement in C.1 and the token-level localization story in C.4. C.4 states that initial tokens 'aggregate global information, receiving disproportionately high attention scores, which amplifies their values in the MHA output,' and that weak-semantic tokens receive 'exaggerated updates during training.' Both statements describe non-zero MHA updates, which contradicts the near-zero-update premise required by the derivation. The authors need to reconcile these statements, for example by distinguishing between the attention weights assigned to these tokens and the L2 norm of the resulting contextual update.","section":"Appendix C.4"},{"comment":"The variant experiments show that explicit context-aware scaling factors prevent outlier formation in GPT-2, but they do not establish that the outliers observed in the default model actually function as scaling factors. An intervention that removes a phenomenon is not by itself evidence for the proposed functional role of that phenomenon. Additionally, the comparison between variants (c) and (d) changes two components at once (adding both a bias and a scaling-like term), so the attribution of the effect specifically to the scaling factor is not as clean as claimed. The functional claim would be strengthened by a direct probe, such as comparing the learned Sc(x) against the magnitude of the outliers in the default model, or by showing that removing the attention outlier in a trained model produces effects similar to inserting an explicit scaling factor.","section":"Section 5.2, Table 2"},{"comment":"The paper describes Appendix C as 'theoretical derivations' (Abstract and Section 5.3), but the derivation is qualitative. The step from large dynamic ranges in softmax inputs to steep gradients and thence to concentrated weight outliers is asserted rather than derived; no formal bound or gradient expression is provided. This is acceptable as an intuition, but the wording overclaims. The authors should either present a concrete formal statement (e.g., a bound on gradient norms in terms of softmax logit gaps) or revise the language to describe the appendix as a qualitative mechanism analysis.","section":"Appendix C, overall"}],"minor_comments":[{"comment":"There is a typo: 'avilable' should be 'available'.","section":"Abstract"},{"comment":"The heading reads 'MORE ANALYSIS FO SYSTEMATIC OUTLIERS'; 'FO' should be 'OF'.","section":"Appendix D title"},{"comment":"The figure label says 'GPT-2 with Sigmoid Attenion'; 'Attenion' should be 'Attention'.","section":"Figure 23"},{"comment":"The outlier threshold τ is set to 1000 with no sensitivity analysis. Since all subsequent consistency statistics in Table 1 depend on τ, a brief analysis of how the reported overlaps vary with τ would improve robustness.","section":"Section 3, Definition"},{"comment":"The compression experiments are only reported for GPT-2. Given the paper's emphasis on LLMs, reporting at least one additional model (e.g., TinyLLaMA) for the quantization and pruning results would strengthen the practical claim.","section":"Section 5.3, Table 3"},{"comment":"The GPT-2 variant experiments train for 50,000 iterations, but no training curves or final validation losses for the variants are shown except Figure 12. Reporting the full training curves for all five variants would help rule out that the absence of outliers in variants (d) and (e) is an artifact of underfitting.","section":"Appendix B.3"}],"recommendation":"major_revision","confidential_remarks":"The paper was accepted at ICLR 2025, and the empirical documentation is useful, but the current arXiv version's central causal claim is stronger than the evidence supports. The zero-update premise and the scaling-factor interpretation are the two load-bearing points; both are testable and I believe a revision that adds targeted measurements (MHA output norms for outlier tokens, a comparison of learned Sc(x) with outlier magnitudes, and possibly a gradient-norm analysis) would bring the claims in line with the evidence. I would not reject the paper, as the core observations and the interventionist ablation are solid and the gaps are fixable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should read this paper for the empirical parts, not for the theory. The taxonomy—activation, weight, and attention outliers—is well organized, and the correlations across dimensions (100% alignment on features, 95% on sequence positions) are concrete and reproducible. The cross-model figures (LLaMA-2, Mistral, Phi, OPT, Falcon, etc.) give a genuinely broader picture than prior work that focused on a single outlier type. The architectural intervention is the real contribution: an explicit context-aware scaling factor kills most outliers in GPT-2 and TinyLLaMA, and the compression numbers are striking (W8 PPL drops from 93 to 29, 50% sparsity from 7235 to 39). The sigmoid-attention control strengthens the case that softmax normalization is somehow involved.\n\nNow the soft spots. The causal chain in Appendix C is a qualitative narrative, not a derivation. The linchpin is the claim that certain tokens require near-zero MHA updates, and this is asserted, never tested. There is no measurement of MHA output norms for initial or weak tokens in LLaMA-2-7B, and no ablation that varies the zero-update requirement independently of the architecture. Worse, there is an internal tension: Appendix C.4 says initial tokens 'aggregate global information' and have amplified MHA values, which is hard to square with the near-zero-update premise. If that premise fails, the softmax-driven story loses its foundation. The intervention is still evidence that a scaling factor prevents outlier formation, but it does not show that the original outliers function as scaling factors—a regularizer effect is not ruled out. Also, the variant experiments are on small models with no error bars, and the tau=1000 threshold is arbitrary though stated.\n\nI would not let these flaws stop you from citing the paper for its empirical maps or for the scaling-factor trick, but I would not cite the mechanism as established. The authors are honest that this is a hypothesis, and the paper is coherent on its own terms despite the tension. For a fresh submission, I would send it to review: the question is important, the evidence is substantial, and the weaknesses are correctable with more targeted experiments. If someone asks you what we know about outliers, this is a useful organizing reference, not the final word.","headline":"A useful empirical map of where outliers live and a clever architectural fix, but the 'softmax is the root cause' story is a plausible narrative rather than an established mechanism.","tokens_in":18999,"tokens_out":1832,"would_cite":true,"duration_ms":17419,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"All three LLM outlier types—activation, weight, and attention—stem from one cause, the softmax in self-attention, and act as implicit context-aware scaling factors that an explicit learnable factor can replace.","keywords":["LLM outliers","activation outliers","weight outliers","attention outliers","softmax attention","context-aware scaling factor","model compression","transformer interpretability"],"falsifier":"During training of a transformer, record the L2 norm of the attention output added to the residual stream at start tokens and punctuation tokens and compare it with content tokens: the paper's premise requires these updates to be near zero, so updates comparable in size to those of content tokens would remove the zero-update requirement that drives the derivation. A second check: train the same architecture on text with punctuation removed and no special start token—under the paper's account, the count and magnitude of attention and activation outliers should drop sharply because the tokens identified as needing the largest dynamic range no longer exist.","tokens_in":18085,"feed_emoji":"🤖","tokens_out":23995,"duration_ms":160142,"temperature":0.7,"pith_summary":"Large language models are riddled with outlier values in activations, weights, and attention scores, and prior work has treated them mostly as nuisances to clip or smooth away during compression. This paper argues instead that the three outlier types are one systematic phenomenon: they all emerge from the softmax operation in self-attention, and they function as implicit, context-aware scaling factors that let the model give start tokens and weak-semantic tokens (the initial token, '.', '_') almost no contextual update. The evidence is layered: the three outlier types co-occur at fixed feature channels and sequence positions (100 percent consistency between weight and activation outliers in feature dimensions, 95 percent between activation and attention outliers in sequence positions); a derivation in the appendix traces them to the softmax 'sum to one' constraint; and training five GPT-2 attention variants shows that only explicit context-aware scaling—not fixed or context-aware bias—prevents outliers. When the implicit scaling is replaced by an explicit learnable one, outliers vanish, early convergence speeds up, and tolerance of compression jumps from broken to near-baseline: WikiText2 perplexity under 8-bit weight quantization falls from 93.44 to 29.22, and under 50% pruning from 7235.68 to 39.47.","feed_headline":"One softmax constraint explains all three LLM outlier types","feed_subtitle":"A learnable context-aware factor replaces the implicit one, cutting 8-bit quantization perplexity from 93 to 29.","key_machinery":"The load-bearing object is the softmax operation inside multi-head self-attention, $\\mathrm{softmax}(QK^\\top/\\sqrt{d})V$, whose outputs are strictly positive and sum to one for every query. The derivation turns on a 'zero-update requirement' (Appendix C.1): for tokens that should barely change—the initial token, punctuation, spaces—the desired attention output is close to zero, yet because softmax probabilities sum to one, near-zero output can only be achieved by concentrating probability on a few keys whose values cancel, which in turn forces extreme disparities among the pre-softmax dot products. This dynamic-range demand steepens gradients on the shared $W_K$ and $W_V$ projections and, through residual connections and LayerNorm's compression of the input distribution, drives the MLP's up-, gate-, and down-projections to grow extreme weights at a few fixed channels (weight outliers), yielding activation outliers at those channels and tokens, which then align the corresponding query and key dimensions to produce attention outliers at the same positions. The companion mechanism that validates the account is the explicit context-aware scaling variant, $Sc(x)\\cdot\\mathrm{softmax}(QK^\\top/\\sqrt{d})V$, with a learned input-dependent scalar $Sc(x)$: it supplies the same scaling effect directly, and the training experiments show it prevents outlier formation entirely rather than merely suppressing its effects.","core_discovery":"The paper's central claim is that activation outliers, weight outliers, and attention outliers in LLMs are not independent defects but one interconnected phenomenon with a single root cause: the softmax normalization in self-attention. Because softmax forces attention scores to be nonnegative and sum to one, tokens that need almost no contextual update—start tokens and semantically weak tokens like '.' and '_'—force the model to concentrate attention onto very few keys, which requires the pre-softmax query-key dot products to span a huge dynamic range. That demand propagates outward: steep gradients on the shared key and value projections, amplified through the residual stream and the MLP up-, gate-, and down-projections, produce weight outliers at a few fixed channels, activation outliers at the same channels and tokens, and attention outliers at the same sequence positions. The outliers therefore act as implicit, context-aware scaling factors, dynamically shrinking or amplifying updates instead of adding a fixed bias. The paper's validation is direct: an explicit context-aware scaling factor, $Sc(x)\\cdot\\mathrm{softmax}(QK^\\top/\\sqrt{d})V$ with learned, input-dependent $Sc(x)$, prevents all three outlier types in GPT-2 and TinyLLaMA-120M, while explicit fixed-bias and context-aware-bias variants do not, and sigmoid attention—which drops the sum-to-one requirement—also produces no systematic outliers. Structurally eliminating the outliers this way accelerates early convergence and makes models much more tolerant of 8-bit weight quantization and 50% magnitude pruning.","pith_inferences":["If the zero-update premise is correct, outlier sites should track the training data's low-information tokens: a model trained on text without punctuation or explicit start tokens should produce markedly fewer and weaker attention and activation outliers—a prediction the paper does not run.","Reading outlier channels as implicit scaling suggests post-hoc compression fixes (clipping, smoothing, rotation) may work against the model's actual computation; a cheaper alternative to full retraining would be to emulate the per-token scaling with a small learned correction applied only at the affected channels.","The account extends naturally to streaming LLMs: the attention-sink tokens that current methods inject to stabilize long-context inference are, on this view, an explicit stand-in for the implicit scaling the model already performs, so a properly scaled attention mechanism could reduce or remove the need for sink tokens.","A directly checkable corollary of the derivation is that a token's pre-softmax logit spread should be inversely related to the size of the update it actually needs; probing checkpoints mid-training, rather than only at the end, would test the causal direction the paper asserts."],"forward_implications":["All three outlier types share one removable cause: replacing implicit softmax-driven scaling with an explicit context-aware scaling factor eliminates activation, weight, and attention outliers in both GPT-2 and TinyLLaMA-120M.","Compression becomes practical without post-hoc outlier repair: on WikiText2, the scaled GPT-2 drops 8-bit absmax weight-quantization perplexity from 93.44 to 29.22 and 50% unstructured-magnitude-pruning perplexity from 7235.68 to 39.47, at a parameter overhead below 0.1%.","Training speeds up: explicit context-aware scaling reaches lower validation loss in the earliest steps than default attention, although final loss ends up comparable.","Scaling, not bias, is the operative role: the fixed-bias and context-aware-bias attention variants still produce outliers, which rules out the massive-activations-as-fixed-bias reading of the phenomenon.","Softmax is the root cause, not the data or the tuning: sigmoid attention, which can emit near-zero attention weights without normalization, produces no systematic outliers, while outliers persist across model families and survive instruction fine-tuning."],"supporting_citations":[{"why":"Defines the default scaled dot-product softmax attention used as variant (a) and as the object of the paper's causal derivation.","marker":"(Vaswani, 2017)"},{"why":"Supplies the 'massive activations as fixed bias' hypothesis that the paper tests and rejects, plus the initialization used for the learned bias vectors in the attention variants.","marker":"(Sun et al., 2024)"},{"why":"Documents attention-score outliers tied to specific tokens (the attention-sink phenomenon) that the paper's attention-outlier localization builds on.","marker":"(Xiao et al., 2023b)"},{"why":"Establishes the existence of activation outliers in large models and motivates the quantization context, providing the definition of outliers as values deviating from the distribution mean.","marker":"(Dettmers et al., 2022)"},{"why":"Provides the sigmoid attention formulation, which the paper trains in Appendix D.1 to show that removing the softmax sum-to-one constraint eliminates systematic outliers.","marker":"(Ramapuram et al., 2024)"},{"why":"Defines GPT-2, the architecture on which the five attention variants are trained for the empirical validation and compression experiments.","marker":"(Radford et al., 2019)"},{"why":"Documents weight outliers in transformer layers and their link to language-modeling capability, the prior the paper's weight-outlier analysis and lifecycle explanation extend.","marker":"(Zhang et al., 2024)"},{"why":"Provides LLaMA2-7B, the primary model for the outlier localization and cross-type consistency analysis in the main text.","marker":"(Touvron et al., 2023b)"}],"fun_headline_variants":["Softmax sum-to-one forces all LLM outliers","Single softmax rule creates every LLM outlier type","All LLM outliers trace back to one softmax constraint","One constraint explains three outlier types in LLMs","Softmax's sum rule spawns activation, weight, attention outliers"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire causal chain rests on the premise, asserted in Appendix C.1 and never measured, that start tokens and weak-semantic tokens genuinely require near-zero updates from attention; if that requirement is not real, or is itself an artifact of training under softmax, the derivation from softmax to systematic outliers loses its foundation.","fun_headline_variants_meta":{"raw":{"variants":["Softmax sum-to-one forces all LLM outliers","Single softmax rule creates every LLM outlier type","All LLM outliers trace back to one softmax constraint","One constraint explains three outlier types in LLMs","Softmax's sum rule spawns activation, weight, attention outliers"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000748,"raw_usage":{"total_tokens":3397,"prompt_tokens":1072,"completion_tokens":2325,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":688,"completion_tokens_details":{"reasoning_tokens":2246}},"tokens_in":688,"tokens_out":2325,"duration_ms":14871,"temperature":1.0,"reasoning_tokens":2246,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T15:30:44.309813+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"During training of a transformer, record the L2 norm of the attention output added to the residual stream at start tokens and punctuation tokens and compare it with content tokens: the paper's premise requires these updates to be near zero, so updates comparable in size to those of content tokens would remove the zero-update requirement that drives the derivation. A second check: train the same architecture on text with punctuation removed and no special start token—under the paper's account, the count and magnitude of attention and activation outliers should drop sharply because the tokens identified as needing the largest dynamic range no longer exist.","supporting_citations":[],"review_version":1}