Pith. sign in

REVIEW 4 major objections 4 minor 12 references

Paying Attention to Facts: Quantifying the Knowledge Capacity of Attention Layers

T0 review · 4 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Pith's one-line read A single attention layer's fact-storage capacity is bounded by the tensor rank of its layer tensor, with value-output heads carrying the facts and query-key heads contributing almost nothing to the bound.

desk verdict Genuinely novel tensor-rank framing with clean upper bounds, but the central capacity claim is not empirically established because every reported 'layer rank' is an unproven architectural lower bound. read the letter →

arxiv 2502.05076 v1 pith:YHAWLF2Z submitted 2025-02-07 cs.LG cs.CL

classification cs.LGcs.CL MSC 15A6968T07
keywords tensorrankattention-onlytransformersfactualrecallknowledgecapacityvalue-outputcircuitsquery-keysoftmaxdatabase
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to give a linear-algebraic measure of how many facts a single attention layer can store. It attaches a 3-tensor to a database and a 3-tensor to an attention layer, and uses the comparison of their ranks as a capacity test. The main bound says the layer's rank is at most $d_{\mathrm{model}} + n_{\mathrm{heads}} \cdot d_{\mathrm{head,vo}} \cdot |Q|$, so the value-output heads, not the query-key heads, carry the facts. The paper further claims that reallocating width from query-key to value-output heads raises capacity at no extra parameter cost. The practical interest is that this yields a parameter-free, architecture-level account of factual recall and a concrete design lever for knowledge storage.

What carries the argument

The carrying object is the layer tensor $L$, defined in Eq (8) as $E + \sum_h A^h V^h$, where $E$ is the embed-unembed circuit sliced identically across subjects, $V^h$ are the value-output circuits sliced identically across predicates, and $A^h$ are the softmax attention matrices for in-database key-query pairs. Its rank is bounded by summing the rank of $E$ (at most $d_{\mathrm{model}}$) and the ranks of the products $A^h V^h$ (at most $d_{\mathrm{head,vo}}$ per predicate per head), which is Proposition 3.5. The same construction separates the roles of the weights: the $W_{VO}$ entries do the additive work of storing facts, while $W_{QK}$ acts as a lookup table that populates $A^h$ and contributes almost nothing to the rank bound. The paper also introduces the effective rank of a database, the lowest rank of any tensor that agrees with $D$ on in-database fibers, to explain why models can memorize seemingly high-rank databases by ignoring nonsensical inputs.

What would settle it

Train a set of small attention-only layers ($d_{\mathrm{model}} \le 6$, $n_{\mathrm{heads}} \le 4$) on random databases, compute the actual tensor rank of $L$ via exact CP decomposition or a numerical rank-revealing algorithm, and compare with $d_{\mathrm{model}} + n_{\mathrm{heads}} \cdot d_{\mathrm{head,vo}}$; if layers that successfully memorize a database have $L$-rank below this bound, or if holding parameters fixed while swapping $d_{\mathrm{head,qk}}$ for $d_{\mathrm{head,vo}}$ does not improve softmax$_{\ge 0.95}$ accuracy, the central capacity claim is wrong.

Watch

Extended reading notes

Core claim

The central discovery is a tensor-rank correspondence between databases and single-layer attention-only transformers. Every database is encoded as a 0/1 3-tensor $D$ whose entries mark which (subject, predicate, object) triples are facts, and the database rank is the tensor rank of $D$, bounded above by $\min(\sum_k |V_k|, \sum_q |V_q|)$. Every attention layer is encoded as a 3-tensor $L = E + \sum_h A^h V^h$, built from the embed-unembed circuit, the attention matrices, and the value-output circuits, whose fibers reproduce the logits for length-2 inputs. Proposition 3.5 bounds $\mathrm{rank}(L)$ by $d_{\mathrm{model}} + n_{\mathrm{heads}} \cdot d_{\mathrm{head,vo}} \cdot |Q|$, and Eq (10) turns this into a sufficiency condition for memorization. The empirical section shows that accuracy tracks the bound, that argmax and softmax can inflate rank far above $\mathrm{rank}(L)$, and that moving dimension budget from $d_{\mathrm{head,qk}}$ to $d_{\mathrm{head,vo}}$ increases memorization accuracy along diagonals of constant total parameter count.

Load-bearing premise

The load-bearing premise is Eq (9): the paper assumes that after training, the directions used to read and write tokens (the embed-unembed circuit) and the directions used to store values (the value-output circuits) are essentially disjoint, so their ranks add to $d_{\mathrm{model}} + n_{\mathrm{heads}} \cdot d_{\mathrm{head,vo}}$; if trained representations overlap instead, every experimental 'layer rank' in the paper is an overestimate and the capacity condition could fail.

Editorial extensions

If this is right

  • If Eq (9) holds, a layer with given $d_{\mathrm{model}}$, $n_{\mathrm{heads}}$, and $d_{\mathrm{head,vo}}$ has a capacity bound of $d_{\mathrm{model}} + n_{\mathrm{heads}} \cdot d_{\mathrm{head,vo}}$ regardless of $d_{\mathrm{head,qk}}$, so query-key width can be shrunk without hurting fact storage.
  • Holding parameter count fixed while increasing $d_{\mathrm{head,vo}}$ and decreasing $d_{\mathrm{head,qk}}$ should improve a layer's ability to memorize a database, as observed in Figure 4.
  • Argmax and softmax can map a rank-2 matrix to full-rank output (Propositions 3.6 and 3.7), so next-token argmax accuracy overstates memorization; using $\mathrm{softmax}_{\ge \tau}$ with $\tau$ near 0.95 makes measured accuracy align with the rank bound.
  • Database rank and triple count are linearly related with a scaling factor (about 1.8 in the experiments), so tensor rank offers an alternative database-size measure that accounts for shared subjects and predicates.
  • The tensor construction shows that logits are a linear function of tokens and that facts are stored additively across heads, making the additive motif concrete at the circuit level.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: The rank bound suggests a practical parameter-rebalancing heuristic for transformer training on knowledge-heavy tasks: shrink $d_{\mathrm{head,qk}}$ and grow $d_{\mathrm{head,vo}}$ to fit more facts, but this may degrade tasks that need rich attention patterns, which the toy experiments cannot rule out.
  • Editorial inference: The distinction between true rank and effective rank implies that hallucination on nonsensical inputs is partly a rank defect: a model trained only on in-domain fibers can encode a low-rank approximation that returns confident answers for out-of-domain (subject, predicate) pairs. Evaluating on random non-facts, as the paper suggests in Remark 2.5, would make this measurable.
  • Editorial inference: The softmax threshold $\tau$ acts as a capacity dial; one could derive a scaling law where the required $\tau$ increases with $\mathrm{rank}(L)$, as the paper anticipates, and test whether attention-only models exhibit a softmax bottleneck at larger scale.
  • Editorial inference: The same tensor construction might be used to compare knowledge capacity across architectures, such as models with MLP layers or multiple layers, by computing or bounding the rank of their corresponding output tensors, though the paper's bounds do not directly extend.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a linear-algebraic framework for studying how single-layer attention-only transformers memorize factual databases. A database is represented as a 3-tensor D, whose rank is proposed as a measure of database size, and an attention layer is represented as a 3-tensor L whose fibers are the model logits. The main theoretical results are an upper bound on rank(D) in terms of unique subject/predicate values (Proposition 2.1), an upper bound on rank(L) in terms of d_model, n_heads, d_head,vo, and |Q| (Proposition 3.5), and examples showing that argmax and softmax can inflate rank (Propositions 3.6 and 3.7). The paper then proposes an unproven lower bound (9) on rank(L), derives a capacity condition (10), and reports experiments on toy models and databases. It also suggests that increasing d_head,vo at the expense of d_head,qk can increase capacity without increasing parameter count.

Significance. The construction of L and the proof that its fibers equal the model logits (Proposition 3.1) are clean and provide a useful conceptual bridge between transformer circuits and tensor rank. The rank-inflation results for argmax and softmax are simple but instructive. If the rank-capacity relationship were firmly established, the framework could offer a new perspective on factual recall and knowledge storage. However, the central empirical claims are built on an unproven lower bound, Eq. (9), and on rank bounds rather than actual ranks, so the current evidence is substantially weaker than the paper's language suggests. The paper itself acknowledges several of these gaps in Sections 2.1 and 5.2, but the acknowledgment does not repair the load-bearing role of Eq. (9) in the experiments.

major comments (4)
  1. [Section 3.1, Eq. (9)] The lower bound d_model + n_heads * d_head,vo <= rank(L) is introduced with the phrase 'we feel it is reasonable to assume' and is never proven. This bound is load-bearing: it is used to derive Eq. (10), to define the 'layer rank bound' in Figures 2-4, and to support the paper's capacity claims. Without a proof, or at least a computation of the actual rank of the trained tensor L from Eq. (8), the central capacity measure is not established.
  2. [Section 4, Figures 2-4] The experimental 'layer rank' is the architectural lower bound from Eq. (9), and the 'database rank' is the upper bound from Proposition 2.1, not the rank of the actual database tensor. Figures 3 and 4 therefore plot accuracy against an assumed lower bound and an upper bound, not against measured ranks. The observed monotonic relation is partly a consequence of these choices rather than an empirical test of the rank-capacity hypothesis.
  3. [Section 2.1 and Section 4] The paper concedes in Section 2.1 that evaluating only on fibers (k,q) in D allows a constant-output model to memorize a database whose tensor has high rank, as Example 2.4 shows. This directly undermines the use of rank(D) as the relevant capacity measure for the training objective in Section 4. The experiments do not address this issue, so the empirical demonstration does not establish that database tensor rank is the quantity controlling memorization.
  4. [Section 3.1, Eq. (10)] Even if Eq. (9) were valid, Eq. (10) is described as a sufficient condition for memorization, but the paper immediately acknowledges that it assumes L exactly replicates D. Since argmax and softmax can distort rank, as shown in Propositions 3.6 and 3.7, Eq. (10) is at best a condition for the existence of an exact low-rank representation, not a memorization condition for the trained model. The wording should be corrected to avoid overclaiming.
minor comments (4)
  1. [Footnote 1] There is a typo: 'contenation' should be 'concatenation'.
  2. [Section 4] The notation dhead,kq is used inconsistently; elsewhere the paper uses dhead,qk.
  3. [Figure 2 caption] Calling Eq. (9) a 'rank lower bound estimate' is misleading because it is an unproven assumption, not an estimate computed from the trained model.
  4. [Section 3.2, Eq. (13)-(14)] The definition of softmax≥τ and the τ-accuracy should explicitly state that 'correct' means the entry for the true value v is 1 after thresholding; the current phrasing is a little compressed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's bounds are derived from definitions, Eq (9) is explicitly a heuristic assumption rather than a fitted or definitionally forced prediction, and the experiments transparently use bounds as bounds.

full rationale

The derivation chain is self-contained. Proposition 2.1 derives a database-rank upper bound from the slice decomposition of the tensor D, and Proposition 3.5 derives an attention-layer rank upper bound from the slice ranks of E and A^h V^h; both are proved from the stated definitions. The load-bearing lower bound Eq (9) is introduced not as a theorem but as a heuristic: the paper says 'we feel it is reasonable to assume that the process of training an attention layer will cause the spans of E and V^h to have as little intersection as possible, so that we are likely to have d_model + n_heads · d_head,vo ≤ rank(L)'. The experiments similarly label the plotted quantity as a bound: 'Layer rank bound is the lower bound from (9), while database rank bound is the upper bound from Proposition 2.1'. No parameter is fitted and then renamed a prediction, no claim is justified by a self-citation chain, and no equation reduces to another by construction. The unproven status of Eq (9) and the use of bounds rather than exact tensor ranks are real validity concerns, but they are assumptions and limitations, not circularity under the stated criteria.

Assumptions & free parameters 1 free parameters · 5 assumptions · 2 invented entities

The central claims rest on two self-defined tensors (D and L), a simplified attention architecture, an evaluation protocol restricted to in-domain triples, and the unproven lower bound in Eq (9). The only hand-chosen free parameter is the evaluation threshold tau, which materially affects the empirical conclusions.

free parameters (1)
  • evaluation threshold tau = 0.75, 0.95, 0.99 (0.5 used but omitted)
    Hand-chosen thresholds in the softmax>=tau accuracy metric (Eqs 13-14); the empirical conclusions are sensitive to tau, as the paper itself notes in Section 5.2 (scale tau according to rank of L).
assumptions (5)
  • domain assumption A database is a partial function D : T x T -> T with single-token subjects, predicates and objects.
    Section 2 defines D this way, excluding multi-token predicates and natural-language composition, so results apply only to the toy setting.
  • domain assumption The transformer is a simplified attention-only layer with no layer norm, biases, or positional encodings, and no MLP.
    Section 3 states this simplification following [4,8]; capacity claims are limited to this architecture.
  • domain assumption Evaluation is restricted to pairs (k,q) inside the database domain, not nonsensical inputs.
    Section 2.1 explicitly adopts the conventional practice of evaluating only on (k,q) in D, noting this can drastically reduce required rank. The empirical accuracy figures inherit this restriction.
  • ad hoc to paper The lower bound d_model + n_heads * d_head,vo <= rank(L) holds after training.
    Section 3.1, Eq (9): the paper asserts this is 'reasonable' if the spans of E and V^h have minimal intersection and each head contributes to at least one q; no proof is given, and the bound is used as the layer rank in all experiments.
  • domain assumption Rank-inflation results assume computation with arbitrarily small or large real magnitudes.
    Props 3.6 and 3.7 require points on a unit sphere and a large constant c; the paper itself notes limited machine precision would reduce these effects.
invented entities (2)
  • Database 3-tensor D
    purpose: Encoding each possible (subject, predicate, object) triple as a 0/1 entry so that tensor rank can serve as a measure of database size.
    Defined in Eq (1); its value as a size measure is asserted and tested only within the paper's own framework, with no externally computed rank comparison for large real databases.
  • Attention-layer 3-tensor L
    purpose: Packing the logits of an attention layer on all length-2 sequences into one tensor whose rank is proposed as the layer capacity.
    Defined in Eq (8); despite being constructed from model weights, the paper never computes the actual tensor rank of L in experiments, so its falsifiable prediction about capacity is not tested directly.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Paying Attention to Facts: Quantifying the Knowledge Capacity of Attention Layers." pith.science (2026). https://pith.science/paper/YHAWLF2Z

@misc{pith2026250205076,
  author       = {Pith},
  title        = {Pith review of: Paying Attention to Facts: Quantifying the Knowledge Capacity of Attention Layers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YHAWLF2Z}},
  note         = {Machine review of arXiv:2502.05076}
}
read the original abstract

In this paper, we investigate the ability of single-layer attention-only transformers (i.e. attention layers) to memorize facts contained in databases from a linear-algebraic perspective. We associate with each database a 3-tensor, propose the rank of this tensor as a measure of the size of the database, and provide bounds on the rank in terms of properties of the database. We also define a 3-tensor corresponding to an attention layer, and empirically demonstrate the relationship between its rank and database rank on a dataset of toy models and random databases. By highlighting the roles played by the value-output and query-key weights, and the effects of argmax and softmax on rank, our results shed light on the `additive motif' of factual recall in transformers, while also suggesting a way of increasing layer capacity without increasing the number of parameters.

Figures

Figures reproduced from arXiv: 2502.05076 by the authors.

Figure 1
Figure 1. A database with two tables, and its ‘language’ of RDF triples in [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Relationship of database rank and attention layer ranks to other [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗
Figure 3
Figure 3. Average accuracy, with predictions made using [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Average accuracy of attention layers with [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 3 canonical work pages

  1. [1]

    Physics of language models: Part 3.1, knowledge storage and extraction

    Zeyuan Allen-Zhu and Yuanzhi Li. Physics of language models: Part 3.1, knowledge storage and extraction. arXiv preprint arXiv:2309.14316 , 2023

  2. [2]

    Physics of language models: Part 3.3, knowledge capacity scaling laws

    Zeyuan Allen-Zhu and Yuanzhi Li. Physics of language models: Part 3.3, knowledge capacity scaling laws. arXiv preprint arXiv:2404.05405 , 2024

  3. [3]

    Summing up the facts: Additive mechanisms behind factual recall in LLMs

    Bilal Chughtai, Alan Cooney, and Neel Nanda. Summing up the facts: Additive mechanisms behind factual recall in LLMs. arXiv preprint arXiv:2402.07321, 2024

  4. [4]

    A mathematical framework for transformer circuits

    Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, et al. A mathematical framework for transformer circuits. Transformer Circuits Thread, 1(1):12, 2021

  5. [5]

    Transformer feed-forward layers are key-value memories.arXiv preprint arXiv:2012.14913, 2020

    Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward layers are key-value memories.arXiv preprint arXiv:2012.14913, 2020

  6. [6]

    Tensor rank is NP-complete.Journal of algorithms, 11(4):644– 654, 1990

    Johan H ˚ astad. Tensor rank is NP-complete.Journal of algorithms, 11(4):644– 654, 1990

  7. [7]

    Scaling laws for fact memorization of large language models

    Xingyu Lu, Xiaonan Li, Qinyuan Cheng, Kai Ding, Xuanjing Huang, and Xipeng Qiu. Scaling laws for fact memorization of large language models. arXiv preprint arXiv:2406.15720 , 2024

  8. [8]

    Understanding fac- tual recall in transformers via associative memories

    Eshaan Nichani, Jason D Lee, and Alberto Bietti. Understanding fac- tual recall in transformers via associative memories. arXiv preprint arXiv:2412.06538, 2024. 17

Show all 12 references
  1. [9]

    Tensor rank is hard to approximate

    Joseph Swernofsky. Tensor rank is hard to approximate. In Approximation, Randomization, and Combinatorial Optimization. Algorithms and Tech- niques (APPROX/RANDOM 2018) . Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2018

  2. [10]

    Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N

    Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Neural Information Processing Systems , 2017

  3. [11]

    Breaking the softmax bottleneck: A high-rank RNN language model

    Zhilin Yang, Zihang Dai, Ruslan Salakhutdinov, and William W Cohen. Breaking the softmax bottleneck: A high-rank RNN language model. arXiv preprint arXiv:1711.03953, 2017

  4. [12]

    Knowledge circuits in pretrained transformers

    Yunzhi Yao, Ningyu Zhang, Zekun Xi, Mengru Wang, Ziwen Xu, Shumin Deng, and Huajun Chen. Knowledge circuits in pretrained transformers. arXiv preprint arXiv:2405.17969 , 2024. 18

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.