Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

A transformer can be built so its attention values and feed-forward units are named logical operations, readable without post-hoc tools, at baseline language-model quality.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 20:05 UTC pith:H3HDUZB2

load-bearing objection Clean architectural idea—bound the value, free the write—with real ablations and a working end-to-end 125M model; parity and SELECTIVE rates are single-seed and provisional. the 3 major comments →

arxiv 2607.04319 v1 pith:H3HDUZB2 submitted 2026-07-05 cs.CL cs.LG

Legible-by-Construction: Attention and End-to-End Transformers

Mark Oskin This is my paper
classification cs.CL cs.LG
keywords legible-by-constructionattentiontransformersfuzzy set operationsmechanistic interpretabilitylanguage modelsBoolean attentionselectivity pressures
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Most knowledge of what happens inside a language model is recovered after training with external tools that are partial, contested, and expensive to keep current. This paper takes the other route: choose primitives that are already named operations before anyone interprets them. Building on a companion result that rebuilt feed-forward layers from fuzzy set operations, it makes attention legible by a deliberately small change—pass each head’s value through a sigmoid, or restructure it as an explicit within-token intersection and set-difference—so every value channel becomes a readable detector of whether a feature holds at a token. The load-bearing rule is to bound what a head detects and leave what it writes free; pinning the write to the vocabulary collapses the head into a context-free constant. Two selectivity pressures, one for sparse firing and one for decisive firing at the rails, then shape those bounded values into detectors, and which pressure a design wants is not universal. Across five designs at 125M parameters, 44–62% of value channels become crisp, contextually selective detectors whose legibility rises with depth, quality sits at parity with a conventional baseline, and coupling Boolean attention to the legible feed-forward layer yields an end-to-end model whose named units can be read off a token it generates.

Core claim

Attention becomes legible by construction when a head’s value is bounded into fuzzy memberships—or restructured into an intersection and set-difference—while its output projection is left free, and when two design-dependent selectivity pressures turn those bounded values into crisp, contextually selective detectors. Coupled to a previously legible feed-forward layer, this produces an end-to-end language model at baseline quality whose dominant computations are named operations that can be read in the act of predicting a token.

What carries the argument

The bounded-value / free-readout principle: constrain the pre-projection value as V=σ(W_v x)∈[0,1] (membership head) or as the concatenation of A⊙B and A⊙(1−B) (Boolean head), leave the output projection free, and apply sparsity and/or crispness pressures on the operands so channels fire sparsely and at the rails rather than as diffuse gates or dead constants.

Load-bearing premise

That quality parity and the reported fractions of selective detectors, measured on a single seed at 125 million parameters after one training epoch, will hold under seed variation and at larger scales.

What would settle it

Train the same architectures for multiple seeds at 125M and at least one larger scale; if LAMBADA and BLiMP fall reliably below the conventional baseline, or if the fraction of selective value channels collapses well below the reported 44–62% range, the claim of free-cost by-construction legibility fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes making transformer attention legible by construction by bounding the value path: a membership head applies a sigmoid to the standard value so each channel is a [0,1] detector, and a Boolean head restructures the value into within-token fuzzy intersection and set-difference, both with free (untied) output projection. Two selectivity pressures (sparsity and crispness) are applied to the operands; the right combination is design-dependent. At 125M parameters, five specialized-attention designs yield 44–62% SELECTIVE value channels whose legibility rises with depth, at LAMBADA/BLiMP/ARC-Easy parity with a GELU baseline. Coupling Boolean attention to the companion paper’s legible FFN produces an end-to-end model at baseline quality whose named FFN units can be read off a generated token via direct logit attribution. An appendix characterizes a non-monotonic trainability boundary for mixed bounded/unbounded heads.

Significance. If the result holds under multi-seed and multi-scale checks, the paper would change the object of study in mechanistic interpretability: a language model whose dominant FFN and deep-attention computations are named fuzzy set/quantifier operations by construction, rather than recovered post hoc. The bounded-value / free-readout principle, the design-dependent pressure rule, and the concrete end-to-end reading of a generated token (superlative ∩ negation → “worst”) are concrete contributions. Strengths include parameter-neutral designs, an explicit failed vocabulary-tied control that motivates the free-write choice, and a checkpoint-level mechanism for the trainability boundary. The work is currently limited by single-seed, one-epoch, 125M evidence and author-defined legibility thresholds, so the “free legibility / no quality cost” framing remains provisional until those are tightened.

major comments (3)
  1. §3.8, Tables 3–4, §4.4–5.2: All quality and SELECTIVE claims rest on single-seed, one-epoch 125M runs. The paper itself notes that within-band LAMBADA/BLiMP differences are smaller than seed noise and that ordering inside the parity band is not trustworthy. The central “parity / free legibility” claim therefore needs at least a multi-seed report (or an explicit, quantified seed-variance bound) before it can be treated as established rather than provisional.
  2. §4.1, Table 1: SELECTIVE is defined by fixed thresholds (variance ≥ 0.003, crispness > 0.5, rail 0.1) and fixed λ_s=10^{-3}, λ_c=3×10^{-3}. The 44–62% rates and the design-dependent pressure rule are load-bearing for the main claim; without a short sensitivity sweep over those knobs (or a human naming study of a random sample of SELECTIVE channels), the quantitative legibility numbers remain author-metric-dependent.
  3. §5.4, Figure 3, §6: Once the legible FFN is coupled, shallow attention collapses to CONSTANT (end-to-end SELECTIVE only 30–35%). The end-to-end “legible-by-construction language model” claim is therefore accurate only for FFN throughout and attention at depth. The abstract and conclusion should state this division of labor as a measured limitation, not only as future work, so the scope of the claim matches the evidence.
minor comments (5)
  1. Figure 1 vs Figure 5: The baseline DLA figure is clear; the end-to-end figure would benefit from an explicit side-by-side residual (GELU) vs named-path attribution fraction so the “three-quarters legible” claim is visible without reading the caption.
  2. §3.6: State the exact loss terms (λ_s ∑|v| and λ_c ∑v(1−v)) once in equation form so the pressures are not only described in prose.
  3. Table 2 and Table 6: Max-activating token lists are useful; a short note on how many channels were inspected and what fraction were judged cleanly nameable would make the qualitative sample less anecdotal.
  4. Appendix A: The membership-75 divergence mechanism is well documented; a one-sentence pointer in the main text (§3.5 or §4) would help readers who skip the appendix.
  5. References: Companion paper is cited as arXiv:2606.31845; ensure the final version cross-links the two manuscripts clearly for readers who encounter only one.

Circularity Check

0 steps flagged

No significant circularity: architectural proposals measured by independent statistics and external benchmarks; companion self-citation is prior architecture, not a fitted or definitional force.

full rationale

The paper proposes bounded-value attention (sigmoid membership or Boolean intersection/set-difference), free output projection, and two selectivity penalties, then reports empirical SELECTIVE rates and LM quality. SELECTIVE is a measurement definition (variance ≥ 0.003 and rail-proximity crispness > 0.5) applied to held-out activations; it does not force the reported 44–62% by construction, nor do the free hyperparameters λ_s and λ_c constitute fitted-then-predicted quantities. Quality is scored on external suites (LAMBADA, BLiMP, ARC-Easy) against a conventional baseline. The companion paper (Oskin, 2026) supplies the named FFN operators used only in the end-to-end coupling; specialized-attention results (Section 4) use a conventional FFN and stand alone. That self-citation is ordinary reuse of prior architecture, not a uniqueness theorem, ansatz smuggled as derivation, or a quantity fitted to the same targets later called predictions. No step reduces a claimed prediction or first-principles result to its own inputs by definition or by self-citation chain. Limitations (single-seed, fixed thresholds, shallow-attention collapse) are empirical soft spots, not circularity.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 3 invented entities

The central claims rest on standard transformer equations, product-t-norm fuzzy connectives, and a small set of hand-chosen selectivity coefficients and operator fractions. No new physical entities; the invented constructs are architectural modules whose behavior is measured empirically. Free parameters are the usual training knobs plus the two lambdas and the GELU/operator partition ρ.

free parameters (5)
  • λ_s (sparsity pressure) = 10^{-3}
    Hand-set to 10^{-3}; controls how strongly bounded values are pushed toward zero; central to selectivity results.
  • λ_c (crispness pressure) = 3×10^{-3}
    Hand-set to 3×10^{-3}; drives values to rails; design-dependent interaction with λ_s is a main finding.
  • special-head fraction n_sp/n_h = 50/75/100%
    Swept over {50%,75%,100%}; non-monotonic trainability boundary depends on it.
  • FFN operator fraction ρ (GELU partition) = 0.75 / 0.5
    Set to 0.75 or 0.5 for end-to-end models; controls named vs residual GELU units.
  • SELECTIVE thresholds (variance≥0.003, crispness>0.5, rail 0.1) = 0.003 / 0.5 / 0.1
    Author-chosen cutoffs that define the headline 44–62% legibility numbers.
axioms (4)
  • standard math Standard multi-head attention equations (QK softmax, OV path, residual stream) as in Vaswani et al. 2017 / Elhage et al. 2021
    Background architecture; only the value transform is altered.
  • domain assumption Product t-norm fuzzy intersection and set-difference are valid soft logical connectives
    Imported from fuzzy-logic literature (van Krieken et al., Badreddine et al.); used for Boolean value and FFN operators.
  • ad hoc to paper Language-model quality can be adequately measured by single-seed LAMBADA, BLiMP, ARC-Easy at 125M after one epoch
    Evaluation protocol chosen by the authors; parity claims rest on it (Sections 3.8, 4.4, 5.2).
  • ad hoc to paper A free output projection plus bounded pre-projection value is the correct site of constraint (vocabulary-tied write fails)
    Load-bearing design principle validated by the collapse experiment in §3.2; not derived from prior theory.
invented entities (3)
  • Membership value head (σ on W_v x) independent evidence
    purpose: Turn each value channel into a readable [0,1] feature detector without adding parameters
    Architectural primitive introduced here; independent evidence is the measured SELECTIVE rates and max-activating contexts.
  • Boolean value head (within-token A∩B and A\B) independent evidence
    purpose: Make the OV path an explicit logical operator analogous to the legible FFN
    New restructuring of the value projection; evidence is higher selectivity under crispness-alone and end-to-end composition.
  • Selectivity pressures (sparsity + crispness on operands) independent evidence
    purpose: Carve sparse, decisive detectors from bounded values
    Training terms introduced to shape legibility; design-dependent interaction is a measured result, not an external fact.

pith-pipeline@v1.1.0-grok45 · 30524 in / 3517 out tokens · 31739 ms · 2026-07-11T20:05:58.024793+00:00 · methodology

0 comments
read the original abstract

A companion paper showed that a transformer's feed-forward layer can be rebuilt from explicit fuzzy set operations - intersection, set-difference, and a self-forgetting sequence quantifier - so its hidden units read as named logical operators at no cost to language-model quality. That left the other half of the transformer opaque. Here we carry the same idea into attention and join the two into one model. The mechanism is minimal: a head's value is passed through a sigmoid, so each value channel becomes a readable detector of whether a feature holds at a token. This adds no parameters and leaves the standard head otherwise untouched. A Boolean variant goes further, restructuring the value into an explicit within-token intersection and negation-capable set-difference. In both designs the output projection is left free, not tied to the vocabulary, which is the load-bearing decision: bounding what a head detects while leaving what it writes unconstrained yields selective detectors, whereas constraining the write does not. A bounded value is shaped into a readable detector by two selectivity pressures - one for sparse firing, one for decisive firing at the rails - and which a design wants is not universal. Across five specialized-attention designs at 125M parameters, 44 to 62 percent of value channels become crisp, contextually selective detectors, and their legibility rises with depth rather than crystallizing only on punctuation. Language-model quality is at parity with a conventional baseline. Finally, we couple the Boolean attention to the legible feed-forward layer and train an end-to-end legible-by-construction language model at benchmark parity: its feed-forward units are named set and quantifier operations throughout, and we can take a token it generates and read the named units that compose to produce it.

Figures

Figures reproduced from arXiv: 2607.04319 by Mark Oskin.

Figure 1
Figure 1. Figure 1: Legibility in a standard transformer is real, but post-hoc and one-sided—the situation this paper changes. Direct logit attribution of the token a conventional gelu baseline generates for “It was the best of times, it was the” (again “worst”), across all 12 layers and per attention head / feed-forward block. The feed-forward writes decode, through a logit lens, into coherent vocabulary concepts: a superlat… view at source ↗
Figure 2
Figure 2. Figure 2: (a) Attention legibility rises with depth in every design; the deep half of the network reaches [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: End-to-end legibility by layer, for a crispness-alone model. The feed-forward operands are crisp [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Per-head attention legibility across all 144 heads, measured over held-out corpus text—a description [PITH_FULL_IMAGE:figures/full_fig_p016_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Reading a single prediction on the crispness-alone headline model. [PITH_FULL_IMAGE:figures/full_fig_p018_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Why membership head-75 fails. (a) The run descends cleanly for [PITH_FULL_IMAGE:figures/full_fig_p022_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Training, Reading, and Editing Legible Transformers

    cs.LG 2026-07 conditional novelty 6.5

    A variance-floor objective plus learned operator gates produce an end-to-end legible transformer whose crisp units are 50–184× more local to edit and can be reshaped from fan-out to fan-in circuits without quality loss.

Reference graph

Works this paper leans on

14 extracted references · 12 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Eliciting latent predictions from transformers with the tuned lens.arXiv preprint arXiv:2303.08112,

    Nora Belrose, Zach Furman, Logan Smith, Danny Halawi, Igor Ostrovsky, Lev McKinney, Stella Biderman, and Jacob Steinhardt. Eliciting latent predictions from transformers with the tuned lens.arXiv preprint arXiv:2303.08112,

    Pith/arXiv arXiv

  2. [2]

    Think you have solved question answering? try ARC, the AI2 reasoning challenge.arXiv preprint arXiv:1803.05457,

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try ARC, the AI2 reasoning challenge.arXiv preprint arXiv:1803.05457,

    Pith/arXiv arXiv

  3. [3]

    Softmax linear units

    Nelson Elhage, Tristan Hume, Catherine Olsson, Neel Nanda, Tom Henighan, et al. Softmax linear units. Transformer Circuits Thread, 2022a. Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, et al. Toy models of superposition.Transformer Circuits Thread, 2022b. Leo Gao, Jonathan Tow, Stella Biderman, Sid Black, et al. A framewor...

    doi:10.5281/zenodo.5371628

  4. [4]

    Interpreting attention layer outputs with sparse autoencoders.arXiv preprint arXiv:2406.17759,

    Connor Kissane, Robert Krzyzanowski, Joseph Isaac Bloom, Arthur Conmy, and Neel Nanda. Interpreting attention layer outputs with sparse autoencoders.arXiv preprint arXiv:2406.17759,

    Pith/arXiv arXiv

  5. [5]

    Interpreting GPT: The logit lens

    nostalgebraist. Interpreting GPT: The logit lens. LessWrong, 2020.https://www.lesswrong.com/posts/ AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens. Mark Oskin. Explicit fuzzy logic in the feed-forward layer: Self-forgetting quantifiers discover legible grammatical-licensing detectors.arXiv preprint arXiv:2606.31845,

    Pith/arXiv arXiv 2020

  6. [6]

    Felix Petersen, Christian Borgelt, Hilde Kuehne, and Oliver Deussen

    arXiv:2410.08417. Felix Petersen, Christian Borgelt, Hilde Kuehne, and Oliver Deussen. Deep differentiable logic gate networks. InAdvances in Neural Information Processing Systems (NeurIPS),

    Pith/arXiv arXiv

  7. [7]

    TAB: Transformer attention bottlenecks enable user intervention and debugging in vision-language models.arXiv preprint arXiv:2412.18675,

    Pooyan Rahmanzadehgervi, Hung Huy Nguyen, Rosanne Liu, Long Mai, and Anh Totti Nguyen. TAB: Transformer attention bottlenecks enable user intervention and debugging in vision-language models.arXiv preprint arXiv:2412.18675,

    Pith/arXiv arXiv

  8. [8]

    Theory, analysis, and best practices for sigmoid self-attention.arXiv preprint arXiv:2409.04431,

    Jason Ramapuram, Federico Danieli, Eeshan Dhekane, Floris Weers, Dan Busbridge, Pierre Ablin, Tatiana Likhomanenko, Jagrit Digani, Zijin Gu, Amitis Shidani, and Russ Webb. Theory, analysis, and best practices for sigmoid self-attention.arXiv preprint arXiv:2409.04431,

    Pith/arXiv arXiv

  9. [9]

    Attention lens: A tool for mechanistically interpreting the attention head infor- mation retrieval mechanism.arXiv preprint arXiv:2310.16270,

    Mansi Sakarvadia, Aswathy Ajith, Arham Khan, Daniel Grzenda, Nathaniel Hudson, Andr´ e Bauer, Kyle Chard, and Ian Foster. Attention lens: A tool for mechanistically interpreting the attention head infor- mation retrieval mechanism.arXiv preprint arXiv:2310.16270,

    Pith/arXiv arXiv

  10. [10]

    GLU variants improve transformer.arXiv preprint arXiv:2002.05202,

    Noam Shazeer. GLU variants improve transformer.arXiv preprint arXiv:2002.05202,

    Pith/arXiv arXiv 2002

  11. [11]

    Dense SAE latents are features, not bugs.arXiv preprint arXiv:2506.15679,

    Xiaoqing Sun, Alessandro Stolfo, Joshua Engels, Ben Wu, Senthooran Rajamanoharan, Mrinmaya Sachan, and Max Tegmark. Dense SAE latents are features, not bugs.arXiv preprint arXiv:2506.15679,

    arXiv

  12. [12]

    Alex Tamkin, Mohammad Taufeeque, and Noah D. Goodman. Codebook features: Sparse and discrete interpretability for neural networks.arXiv preprint arXiv:2310.17230,

    Pith/arXiv arXiv

  13. [13]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N

    arXiv:2002.06100. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. InAdvances in Neural Information Processing Systems (NeurIPS),

    Pith/arXiv arXiv 2002

  14. [14]

    Liu, Lechao Xiao, Katie Everett, Alex Alemi, Ben Adlam, John D

    Mitchell Wortsman, Peter J. Liu, Lechao Xiao, Katie Everett, Alex Alemi, Ben Adlam, John D. Co-Reyes, Izzeddin Gur, Abhishek Kumar, Roman Novak, et al. Small-scale proxies for large-scale transformer training instabilities.arXiv preprint arXiv:2309.14322,

    Pith/arXiv arXiv