Pith. sign in

REVIEW 3 major objections 5 minor 61 references

Composing Linear Layers from Irreducibles

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Bivectors rebuild linear layers with O(log^2 d) parameters.

desk verdict Real new algorithm and credible experiments, but the abstract's O(log^2 d) expressivity claim outruns the theorems: the parameter count is for a fixed gadget, not for approximating arbitrary linear layers. read the letter →

arxiv 2507.11688 v4 pith:IZV5ET6T submitted 2025-07-15 cs.LG

Travis Pence , Daisuke Yamada , Vikas Singh This is my paper
classification cs.LG MSC 15A6668T07
keywords CliffordalgebrabivectorsrotorsSpingroupgeometricparameter-efficientlayersLLMattentionlinearlayerapproximation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes that every linear layer can be synthesized from a small set of geometric primitives: bivectors, the oriented-plane elements of a Clifford algebra. It gives a differentiable algorithm that turns an arbitrary bivector into a product of rotors, and stacks c1×c2 such rotor modules to build a linear map with O($log^{2}$ d) learnable parameters instead of O($d^{2}$). The paper tests this construction by replacing key, query, and value projections in LLM attention layers, where the rotor layers match the perplexity and accuracy of low-rank and block-Hadamard baselines while using dramatically fewer parameters. The claim is that this shows how dense-looking linear transformations can be built up from a few composing geometric pieces.

What carries the argument

The central object is the bivector in the Clifford algebra $\mathrm{Cl}(n)$, a grade-2 element $u \wedge v$ that encodes an oriented plane; its exponential $r = \exp(b)$ is a rotor in the group $\mathrm{Spin}(n)$, which acts on multivectors by the sandwich product $x \mapsto r x r^\dagger$. The load-bearing identity is the invariant decomposition: any bivector $b$ can be written as a sum of at most $\lfloor n/2 \rfloor$ mutually commuting simple bivectors, so $\exp(b)$ factors into commuting closed-form exponentials, each computed exactly by $\exp(b_i) = \cos\|b_i\| + \frac{\sin\|b_i\|}{\|b_i\|} b_i$. The paper's differentiable algorithm extracts these simple components by a Clifford-algebra adaptation of power iteration, and the overall gadget maps arbitrary input and output dimensions by running $c_1 c_2$ rotor sandwiches on coordinate subsets and pooling their outputs. The gadget uses the fact that $n \approx \log d$, which turns $O(n^2)$ bivector coefficients into $O(\log^2 d)$ scalar parameters.

What would settle it

Take a dense linear layer whose entries are sampled from a distribution with no near-low-rank or banded structure, fix a target approximation error, and measure how many rotor modules are required as the input dimension $d$ grows. If the required number grows polynomially or exponentially in $d$, or if the rotor layer cannot come within a fixed relative error of such a random target at any budget, then the $O(\log^2 d)$ claim does not describe general linear-layer approximation.

Watch

Extended reading notes

Core claim

The central claim is that a dense linear layer—the kind that stores O($d^{2}$) independent entries—can be represented, up to an approximation error that the paper treats as controllable in practice, as a composition of rotor sandwich maps generated by bivectors. Each bivector encodes an oriented plane, and its exponential is a rotor acting by $x \mapsto r x s^\dagger$ on a subspace of dimension $2^n$ with $n \approx \log d$. Because a bivector in $n$ dimensions has $O(n^2)=O(\log^2 d)$ coefficients and the machinery of Clifford algebra lets one combine rotors by multiplication, the whole layer uses $O(\log^2 d)$ parameters. The paper also supplies a closed-form, differentiable decomposition of a general bivector into mutually commuting simple bivectors so that each rotor can be computed exactly rather than by truncating an infinite exponential series. The authors are explicit that pure rotors are orthogonal maps, so generality is recovered by aggregating many rotor modules and by the parameter-free permutations and nonlinearities in the gadget.

Load-bearing premise

The load-bearing premise is that a fixed small number of rotor modules acting on roughly logarithmic-dimensional subspaces can approximate the target linear layer to the required accuracy, so that $O(\log^2 d)$ is the true cost of approximation rather than a lower bound hiding a growing multiplicative factor; the paper does not bound the number of modules as a function of the target matrix and relies on empirical validation.

Editorial extensions

If this is right

  • Replacing the query, key, and value projections in one to three attention layers of one to 1.5 billion parameter LLMs with rotor layers keeps log-perplexity and accuracy competitive with low-rank and block-Hadamard baselines.
  • The query projection in LLaMa-3.2 1B drops from about 4.19 million dense parameters to at most 896 rotor parameters, a roughly 4700× reduction versus dense and 18× versus rank-4 low-rank.
  • Rotor layers can be trained end-to-end from scratch: an MLP with all dense layers replaced by rotors reaches 88.36% accuracy on FMNIST versus 89.67% for the dense baseline.
  • Because rotors act on full multivectors while using $so(n)$-sized parameters, the same bivector parameters generate transformations on every grade of the Clifford algebra, giving more expressive power per parameter than an $SO(n)$ rotation acting only on plain vectors.
  • If the approximation is accurate, the algebraic decomposition offers a route to synthesize layer weights on-chip from a small parameter set, trading compute for memory bandwidth at inference.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Our inference: the $O(\log^2 d)$ parameter count counts only bivector coefficients; the fixed permutations, normalizations, and nonlinearities that make the gadget expressive are not in the count, so any practical parameter-efficiency comparison needs to account for them or to bound how the module counts $c_1, c_2$ must grow with $d$ to hold fixed error.
  • Our inference: a direct way to test the strength of the claim is to measure how many rotor modules are needed to reach a given Frobenius-norm error on random dense target matrices; if that number grows faster than polylogarithmically in $d$, then $O(\log^2 d)$ does not describe general linear-layer approximation.
  • Our inference: the geometric view suggests decomposing pretrained attention projections into interpretable oriented-plane rotations, allowing one to inspect which bivector planes dominate each Q/K/V projection and to ask whether they align with meaningful directions in activation space.
  • Our inference: the same rotor machinery could be applied to other transformer weight matrices and to statistical interaction modeling, where a rotor acting on a multivector built from $k$ predictors induces pairwise interaction structure with $O(k^2)$ parameters.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes to represent linear layers by composing Clifford-algebra rotors parametrized by bivectors. It introduces a rotor gadget built from c1 by c2 rotor-sandwich modules acting on subspaces of dimension 2^n, a differentiable invariant-decomposition algorithm that expresses a bivector as a sum of simple bivectors, and empirical evaluations replacing query/key/value projections in 1B-1.5B LLMs as well as an end-to-end FMNIST MLP experiment. The headline claim is that a linear layer can be expressed with O(log^2 d) parameters instead of O(d^2). The parameter-count theorem for the fixed gadget is correct, and the decomposition algorithms are plausible and accompanied by proofs, but the paper does not establish that such gadgets can express or approximate arbitrary linear layers, and the implemented architecture includes nonlinearities that are not counted in the parameter budget.

Significance. If the abstract's claim were established, the paper would offer a striking parameter-efficiency result and a new algebraic perspective on layer structure. The bivector-to-rotor decomposition algorithm and the fixed-gadget parameter count are useful technical contributions, and the authors provide code and detailed experimental protocols. However, the central theoretical claim is not derived: no expressivity or approximation bound ties the number of rotor modules to the target matrix and error, and the empirical rotor layer's expressivity partly comes from learnable PReLU slopes, pooling, and nonlinearities outside the counted parameters. These are load-bearing gaps rather than presentation issues, so the significance of the paper as it stands is substantially lower than the abstract suggests.

major comments (3)
  1. [§3, Example 3.1] The abstract's central claim that linear layers can be expressed with O(log^2 d) parameters is not derived. Lemma 1 guarantees a finite representation of any linear map as sum_t a_t x b_t for multivectors a_t,b_t, but the paper replaces these arbitrary multivectors by rotors r,r^†, which are orthogonal/SO-type maps. The paper itself states in §3 that 'our construction does not capture arbitrary linear maps.' No theorem bounds the required number c1,c2 of rotor modules or the Clifford dimension n (subject to 2^n <= min(d_in,d_out)) as a function of the target matrix and approximation error. Example 3.1 silently assumes w ≈ 3 is an admissible width for a rotor composition, but Lemma 1 gives no such bound for rotors. For fixed c1,c2,n the image of the parameter map has dimension at most 2 c1 c2 C(n,2) <= c1 c2 (log2 d)(log2 d - 1), which is O(log^2 d), a measure-zero subset of R^{d_out x d_in}; hence the 'O(log^2 d) versus O(d^2)' comparison cannot be a general expressivity statement without an additional approximation-theoretic result.
  2. [§4.2, Eq. (7); Appendix C; Table 4] The parameter count in Theorem 1/Theorem 4 applies to the bare rotor-sandwich gadget psi_{r,s}, but the implemented layer is not this gadget. Eq. (7) includes a pooling operator sigma, and Appendix C states that each rotor map is followed by fixed permutations, normalizations, and a PReLU nonlinearity. PReLU has learnable slopes, so the '≤896 parameters' in Table 4 omits learnable parameters. More importantly, the nonlinearity and pooling contribute representational power outside the counted rotor parameters, so the experiments do not measure the expressivity of the bivector/rotor composition alone. The empirical rotor layer is not even a linear layer, which conflicts with the paper's stated object of study.
  3. [§5, Tables 1 and 2; Appendix D] The empirical support is not sufficient for the claim that rotor layers 'match or outperform' baselines. No confidence intervals or standard deviations are reported, and the checklist concedes this. Several reported numbers appear implausible (e.g., HellaSwag 6.868 and Arc Challenge 9.01/13.30 in Appendix D), suggesting data or formatting errors. In three-layer replacements on Qwen-2.5 1.5B, Table 1 shows Rotor at 47.28% Arc Challenge versus 60.68% for BH1, and 6.868 HellaSwag versus 13.06 for BH1, contradicting the claim that rotors are 'consistently either the best or second-best.' At minimum these anomalies must be resolved before the main empirical claim can be assessed.
minor comments (5)
  1. [§4.2, Theorem 1] The notation '2n <= min(d_in,d_out)' should be '2^n <= min(d_in,d_out)' to match the Clifford-algebra dimension; the superscript appears to be lost in rendering.
  2. [§5.2] In the sentence 'inference is roughly w×d times slower than dense layers,' the symbol d is used for depth after being used for the input/output dimension; rename the depth variable to avoid ambiguity.
  3. [Appendix D, Tables 8-10] Several accuracy entries such as 9.01, 13.30, and 6.868 are implausible for multiple-choice benchmarks with four choices and should be checked for transcription or averaging errors.
  4. [Figure 4] The caption says 'Six rotors' for c1=3, c2=2, but each psi_{r,s} contains two rotors, so there are twelve rotors; clarify the intended count.
  5. [References] The reference for the torch_ga package ('Alesiani') lacks author, year, and version details; please provide a complete citation.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the O(log^2 d) parameter bound is a definitional count for a fixed rotor gadget, not a fitted prediction, and the core algebra is imported from external published results.

full rationale

The derivation chain is self-contained against external mathematics. Lemma 1 is a textbook representation theorem (Hestenes and Sobczyk); Fact 2 and the closed-form simple-bivector exponential are from Doran and Lasenby; the invariant decomposition used in Section 4 is Theorem 4.8 of Eelbode et al. and the Roelfs/Keninck result, none of which are self-citations by the present authors. No uniqueness theorem from the authors' prior work is invoked to force the rotor parametrization. Theorem 1 (and Theorem 4 in Appendix B) is a direct count of the learnable parameters of a gadget whose architecture is specified in Eq. (7): two rotors per module times c1c2 modules times C(n,2) bivector coordinates. It is therefore a definitional count, not a fitted value relabeled as a prediction, and it does not reduce to its input. The empirical sections train all replacements (Rotor, LR1/LR4, BH1) with the same MSE-plus-Adam protocol on extracted hidden states, so no target-specific constant is fit and then reported as a forecast. Minor self-citations (e.g., Zeng et al. 2023 for the Block-Hadamard baseline; Chytas et al. 2024 and Chen et al. 2023/2025 in related-work or future-work contexts) are not load-bearing. The genuine weakness is that the paper never bounds the number or dimension of rotor modules needed to approximate an arbitrary dense matrix to a given error, and Example 3.1 silently assumes w approximately 3; this makes the abstract's general 'O(log^2 d) versus O(d^2)' claim under-supported, but that is a completeness gap rather than a circular reduction. Appendix A candidly states that the results are feasibility demonstrations, which further supports the non-circular reading.

Assumptions & free parameters 2 free parameters · 4 assumptions · 1 invented entities

The paper introduces no new physical entities. Its parameter-count claim rests on choosing a constant number of rotor modules; the needed module count is not derived, and the nonlinearities used in practice are outside the counted parameters.

free parameters (2)
  • rotor module count c1*c2 (width/depth) = up to 2x3 = 6 in experiments
    The parameter count O(log^2 d) assumes a constant number of modules; the number needed for a given approximation quality is not derived. The paper tunes width and depth by grid search (Appendix C).
  • internal Clifford dimension n = chosen so 2^n <= min(din,dout); e.g., n=8 for 256-dim blocks
    n determines the bivector parameter count per module and the block size; the paper's examples use n such that blocks are 256-2048 dimensional.
assumptions (4)
  • standard math Any linear map can be written as a finite sum of two-sided multivector products (Lemma 1)
    Cited to Hestenes and Sobczyk; standard Clifford algebra result.
  • domain assumption Spin(n) is a double cover of SO(n) and rotors are exponentials of bivectors
    Used to parameterize rotors via bivectors; standard result.
  • domain assumption Any bivector decomposes into at most floor(n/2) commuting simple bivectors (Lemma 2, Eelbode et al. 2024)
    Basis of the differentiable decomposition; imported from cited work.
  • ad hoc to paper A constant number of rotor modules can approximate the target linear layer's behavior on the data distribution
    No expressivity theorem is provided; this is the key empirical assumption behind the O(log^2 d) claim.
invented entities (1)
  • none
    purpose: none
    The paper introduces no new physical entities; rotors and bivectors are standard mathematical objects.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Composing Linear Layers from Irreducibles." pith.science (2026). https://pith.science/paper/IZV5ET6T

@misc{pith2026250711688,
  author       = {Pith},
  title        = {Pith review of: Composing Linear Layers from Irreducibles},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IZV5ET6T}},
  note         = {Machine review of arXiv:2507.11688}
}
read the original abstract

Contemporary large models often exhibit behaviors suggesting the presence of low-level primitives that compose into modules with richer functionality, but these fundamental building blocks remain poorly understood. We investigate this compositional structure in linear layers by asking: can we identify/synthesize linear transformations from a minimal set of geometric primitives? Using Clifford algebra, we show that linear layers can be expressed as compositions of bivectors -- geometric objects encoding oriented planes -- and introduce a differentiable algorithm that decomposes them into products of rotors. This construction uses only O(log^2 d) parameters, versus O(d^2) required by dense matrices. Applied to the key, query, and value projections in LLM attention layers, our rotor-based layers match the performance of strong baselines such as block-Hadamard and low-rank approximations. Our findings provide an algebraic perspective on how these geometric primitives can compose into higher-level functions within deep models.

Figures

Figures reproduced from arXiv: 2507.11688 by the authors.

Figure 1
Figure 1. The basis vectors, bivectors, and trivector for Cl(3) These relations encode the metric and orientation of the underlying space. The algebra has a canonical basis of 2 n elements: the scalar 1 and all distinct products of the basis vectors e1, . . . , en. In particular, basis bivectors are wedge products of two distinct basis vectors, i.e., ei ∧ ej = eiej for i < j. More generally, basis k-vectors are wedge products… view at source ↗
Figure 2
Figure 2. The sandwich product rotat￾ing a vector 60◦ in the e1 ∧ e2 plane. where Cl+(n) ⊂ Cl(n) is the even subalgebra and † denotes grade-wise reversion. The Spin group captures the set of orientation-preserving rotations within Cl(n). The elements r ∈ Spin(n), called rotors, act on multivectors x ∈ Cl(n) through the sandwich product as shown in [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The [bivector → invariant decomposition → rotor de￾composition → rotor] process that enables exact parametrization. Note that a pure rotor is one that corresponds to a simple bivector. Let us check examples. Spin(n) and SO(n) are different Lie groups with the same Lie algebra. Specifically, the Lie algebra of skew-symmetric matrices, so(n) ≜  B ∈ R n×n|B = −BT [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Rotor architecture with c1 = 3 and c2 = 2. An input x is split into  x Ii [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Effect of rotor width and depth. Replac￾ing Layer-13 in Qwen-2.5 1.5B with rotors of vary￾ing depth and width. The dashed line (9.845) indicates convergence to the base model’s perplexity. In [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Number of iterations required by Alg 2 to converge within a tolerance of ϵ = 10−3 , plotted against the number of gradient updates applied to the parameters of rotors (i.e., bivector coefficients). Results are averaged over 50 runs and simple bivectors in the invariant…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

61 extracted references · 35 canonical work pages

  1. [1]

    Adaptivity and modularity for efficient generalization over task complexity, 2023

    Samira Abnar, Omid Saremi, Laurent Dinh, Shantel Wilson, Miguel Angel Bautista, Chen Huang, Vimal Thilak, Etai Littwin, Jiatao Gu, Josh Susskind, and Samy Bengio. Adaptivity and modularity for efficient generalization over task complexity, 2023. URL https://arxiv.org/abs/2310.08866

    arXiv 2023

  2. [2]

    A Walsh Hadamard Derived Linear Vector Symbolic Architecture

    Mohammad Mahmudul Alam, Alexander Oberle, Edward Raff, Stella Biderman, Tim Oates, and James Holt. A walsh hadamard derived linear vector symbolic architecture. arXiv preprint arXiv:2410.22669, 2024

    work page Pith review arXiv 2024

  3. [3]

    Pytorch geometric algebra, 2023

    Francesco Alesiani. Pytorch geometric algebra, 2023. URL https://github.com/falesiani/torch_ga

    work page 2023

  4. [4]

    Zico Kolter, and Vladlen Koltun

    Shaojie Bai, J. Zico Kolter, and Vladlen Koltun. Deep equilibrium models, 2019. URL https://arxiv.org/abs/1909.01377

    arXiv 2019

  5. [5]

    Johannes Brandstetter, Rianne van den Berg, Max Welling, and Jayesh K. Gupta. Clifford neural layers for pde modeling, 2023. URL https://arxiv.org/abs/2209.04934

    arXiv 2023

  6. [6]

    Efficient scaling of large language models with mixture of experts and 3d analog in-memory computing

    Julian B \"u chel, Athanasios Vasilopoulos, William Andrew Simon, Irem Boybat, HsinYu Tsai, Geoffrey W Burr, Hernan Castro, Bill Filipiak, Manuel Le Gallo, Abbas Rahimi, et al. Efficient scaling of large language models with mixture of experts and 3d analog in-memory computing. Nature Computational Science, pages 1--14, 2025

    work page 2025

  7. [7]

    Simpletm: A simple baseline for multivariate time series forecasting

    Hui Chen, Viet Luong, Lopamudra Mukherjee, and Vikas Singh. Simpletm: A simple baseline for multivariate time series forecasting. In The Thirteenth International Conference on Learning Representations, 2025

    work page 2025

  8. [8]

    HarsanyiNet: Computing Accurate Shapley Values in a Single Forward Propagation

    Lu Chen, Siyu Lou, Keyan Zhang, Jin Huang, and Quanshi Zhang. Harsanyinet: Computing accurate shapley values in a single forward propagation. arXiv preprint arXiv:2304.01811, 2023

    work page Pith review arXiv 2023

Show all 61 references
  1. [9]

    Compositional generalization via neural-symbolic stack machines

    Xinyun Chen, Chen Liang, Adams Wei Yu, Dawn Song, and Denny Zhou. Compositional generalization via neural-symbolic stack machines. Advances in Neural Information Processing Systems, 33: 0 1690--1701, 2020

    2020

  2. [10]

    An exploration of parameter redundancy in deep networks with circulant projections

    Yu Cheng, Felix X Yu, Rogerio S Feris, Sanjiv Kumar, Alok Choudhary, and Shi-Fu Chang. An exploration of parameter redundancy in deep networks with circulant projections. In Proceedings of the IEEE international conference on computer vision, pages 2857--2865, 2015

    2015

  3. [11]

    Understanding multi-compositional learning in vision and language models via category theory

    Sotirios Panagiotis Chytas, Hyunwoo J Kim, and Vikas Singh. Understanding multi-compositional learning in vision and language models via category theory. In European Conference on Computer Vision, pages 324--341. Springer, 2024

    2024

  4. [12]

    Think you have solved question answering? try arc, the ai2 reasoning challenge, 2018

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge, 2018. URL https://arxiv.org/abs/1803.05457

    2018 arXiv

  5. [13]

    P. M. Cohn. Basic Algebra: Groups, Rings and Fields. Springer, London, 2003. doi:10.1007/978-0-85729-428-9

    2003 doi

  6. [14]

    Exterior powers

    Keith Conrad. Exterior powers. Lecture notes, Department of Mathematics, University of Connecticut, n.d. Available at https://kconrad.math.uconn.edu/blurbs/linmultialg/extmod.pdf

  7. [15]

    Neural modular control for embodied question answering

    Abhishek Das, Georgia Gkioxari, Stefan Lee, Devi Parikh, and Dhruv Batra. Neural modular control for embodied question answering. In Conference on robot learning, pages 53--62. PMLR, 2018

    2018

  8. [16]

    Documenting large webtext corpora: A case study on the colossal clean crawled corpus, 2021

    Jesse Dodge, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, and Matt Gardner. Documenting large webtext corpora: A case study on the colossal clean crawled corpus, 2021. URL https://arxiv.org/abs/2104.08758

    2021 arXiv

  9. [17]

    Outer and eigen: Tangent concepts, 2024

    David Eelbode, Martin Roelfs, and Steven De Keninck. Outer and eigen: Tangent concepts, 2024. URL https://arxiv.org/abs/2412.20566

    2024

  10. [18]

    Gptq: Accurate post-training quantization for generative pre-trained transformers

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. Gptq: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint arXiv:2210.17323, 2022

    2022 arXiv

  11. [19]

    Recursive sketches for modular deep learning

    Badih Ghazi, Rina Panigrahy, and Joshua Wang. Recursive sketches for modular deep learning. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pag...

    2019

  12. [20]

    Poole, and John L

    Herbert Goldstein, Charles P. Poole, and John L. Safko. Classical Mechanics. Addison-Wesley, 3rd edition, 2002. ISBN 978-0201657029

    2002

  13. [21]

    Golub and Charles F

    Gene H. Golub and Charles F. Van Loan. Matrix Computations. Johns Hopkins University Press, Baltimore, MD, 4th edition, 2013. ISBN 9781421407944

    2013

  14. [22]

    Clifford algebra to geometric calculus: a unified language for mathematics and physics, volume 5

    David Hestenes and Garret Sobczyk. Clifford algebra to geometric calculus: a unified language for mathematics and physics, volume 5. Springer Science & Business Media, 2012

    2012

  15. [23]

    Introduction to clifford's geometric algebra

    Eckhard Hitzer. Introduction to clifford's geometric algebra. arXiv preprint arXiv:1306.1660, 2013. URL https://arxiv.org/abs/1306.1660

    2013 arXiv

  16. [24]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, 2021. URL https://arxiv.org/abs/2106.09685

    2021 arXiv

  17. [25]

    Fox-1: Open small language model for cloud and edge, 2025

    Zijian Hu, Jipeng Zhang, Rui Pan, Zhaozhuo Xu, Shanshan Han, Han Jin, Alay Dilipbhai Shah, Dimitris Stripelis, Yuhang Yao, Salman Avestimehr, Tong Zhang, and Chaoyang He. Fox-1: Open small language model for cloud and edge, 2025. URL https://arxiv.org/abs/2411.05281

    2025 arXiv

  18. [26]

    Perturbation Theory for Linear Operators, volume 132 of Grundlehren der mathematischen Wissenschaften

    Tosio Kato. Perturbation Theory for Linear Operators, volume 132 of Grundlehren der mathematischen Wissenschaften. Springer-Verlag, Berlin, Heidelberg, 2nd edition, 1980. ISBN 3-540-07558-5. URL https://webhomes.maths.ed.ac.uk/ v1ranick/papers/kato1.pdf

    1980

  19. [27]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017. URL https://arxiv.org/abs/1412.6980

    2017 arXiv

  20. [28]

    Kastanenka, and Dmitry Krotov

    Leo Kozachkov, Ksenia V. Kastanenka, and Dmitry Krotov. Building transformers from neurons and astrocytes. Proceedings of the National Academy of Sciences, 120 0 (34): 0 e2219150120, 2023. doi:10.1073/pnas.2219150120. URL https://www.pnas.org/doi/abs/10.1073/pnas.2219150120

    2023 doi

  21. [29]

    On the direct alignment of latent spaces

    Zorah L \"a hner and Michael Moeller. On the direct alignment of latent spaces. In Proceedings of UniReps: the First Workshop on Unifying Representations in Neural Models, pages 158--169. PMLR, 2024

    2024

  22. [30]

    Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks

    Brenden Lake and Marco Baroni. Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks. In International conference on machine learning, pages 2873--2882. PMLR, 2018

    2018

  23. [31]

    Cheap orthogonal constraints in neural networks: A simple parametrization of the orthogonal and unitary group, 2019

    Mario Lezcano-Casado and David Martínez-Rubio. Cheap orthogonal constraints in neural networks: A simple parametrization of the orthogonal and unitary group, 2019. URL https://arxiv.org/abs/1901.08428

    2019 arXiv

  24. [32]

    Neural-symbolic recursive machine for systematic generalization

    Qing Li, Yixin Zhu, Yitao Liang, Ying Nian Wu, Song-Chun Zhu, and Siyuan Huang. Neural-symbolic recursive machine for systematic generalization. arXiv preprint arXiv:2210.01603, 2022

    2022 arXiv

  25. [33]

    Building a large annotated corpus of english: The penn treebank

    Mitch Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. Building a large annotated corpus of english: The penn treebank. Computational linguistics, 19 0 (2): 0 313--330, 1993

    1993

  26. [34]

    Multivariate analysis

    Kanti V Mardia, John T Kent, and Charles C Taylor. Multivariate analysis. John Wiley & Sons, 2024

    2024

  27. [35]

    Mixture of experts: a literature survey

    Saeed Masoudnia and Reza Ebrahimpour. Mixture of experts: a literature survey. Artificial Intelligence Review, 42: 0 275--293, 2014

    2014

  28. [36]

    Vector-based models of semantic composition

    Jeff Mitchell and Mirella Lapata. Vector-based models of semantic composition. In proceedings of ACL-08: HLT, pages 236--244, 2008

    2008

  29. [37]

    Modular deep learning

    Jonas Pfeiffer, Sebastian Ruder, Ivan Vuli \'c , and Edoardo Maria Ponti. Modular deep learning. arXiv preprint arXiv:2302.11529, 2023

    2023 arXiv

  30. [38]

    From complexity to clarity: Analytical expressions of deep neural network weights via clifford's geometric algebra and convexity, 2024

    Mert Pilanci. From complexity to clarity: Analytical expressions of deep neural network weights via clifford's geometric algebra and convexity, 2024. URL https://arxiv.org/abs/2309.16512

    2024 arXiv

  31. [39]

    Smith, and Mike Lewis

    Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A. Smith, and Mike Lewis. Measuring and narrowing the compositionality gap in language models, 2023. URL https://arxiv.org/abs/2210.03350

    2023 arXiv

  32. [40]

    Qwen2.5 technical report, 2025

    Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, ...

    2025 arXiv

  33. [41]

    A practical review of mechanistic interpretability for transformer-based language models

    Daking Rai, Yilun Zhou, Shi Feng, Abulhair Saparov, and Ziyu Yao. A practical review of mechanistic interpretability for transformer-based language models. arXiv preprint arXiv:2407.02646, 2024

    2024

  34. [42]

    Scaling vision with sparse mixture of experts

    Carlos Riquelme, Joan Puigcerver, Basil Mustafa, Maxim Neumann, Rodolphe Jenatton, Andr \'e Susano Pinto, Daniel Keysers, and Neil Houlsby. Scaling vision with sparse mixture of experts. Advances in Neural Information Processing Systems, 34: 0 8583--8595, 2021

    2021

  35. [43]

    Graded symmetry groups: Plane and simple, 2021

    Martin Roelfs and Steven De Keninck. Graded symmetry groups: Plane and simple, 2021. URL https://arxiv.org/abs/2107.03771

    2021

  36. [44]

    Clifford group equivariant neural networks, 2023 a

    David Ruhe, Johannes Brandstetter, and Patrick Forré. Clifford group equivariant neural networks, 2023 a . URL https://arxiv.org/abs/2305.11141

    2023 arXiv

  37. [45]

    Gupta, Steven de Keninck, Max Welling, and Johannes Brandstetter

    David Ruhe, Jayesh K. Gupta, Steven de Keninck, Max Welling, and Johannes Brandstetter. Geometric clifford algebra networks, 2023 b . URL https://arxiv.org/abs/2302.06594

    2023 arXiv

  38. [46]

    Dynamic routing between capsules, 2017

    Sara Sabour, Nicholas Frosst, and Geoffrey E Hinton. Dynamic routing between capsules, 2017. URL https://arxiv.org/abs/1710.09829

    2017 arXiv

  39. [47]

    Discovering modular solutions that generalize compositionally, 2024

    Simon Schug, Seijin Kobayashi, Yassir Akram, Maciej Wołczyk, Alexandra Proca, Johannes von Oswald, Razvan Pascanu, João Sacramento, and Angelika Steger. Discovering modular solutions that generalize compositionally, 2024. URL https://arxiv.org/abs/2312.15001

    2024 arXiv

  40. [48]

    Structured transforms for small-footprint deep learning

    Vikas Sindhwani, Tara Sainath, and Sanjiv Kumar. Structured transforms for small-footprint deep learning. Advances in Neural Information Processing Systems, 28, 2015

    2015

  41. [49]

    Analysis of variance (anova)

    Lars St, Svante Wold, et al. Analysis of variance (anova). Chemometrics and intelligent laboratory systems, 6 0 (4): 0 259--272, 1989

    1989

  42. [50]

    Improved semantic representations from tree-structured long short-term memory networks

    Kai Sheng Tai, Richard Socher, and Christopher D Manning. Improved semantic representations from tree-structured long short-term memory networks. arXiv preprint arXiv:1503.00075, 2015

    2015 arXiv

  43. [51]

    Llama: Open and efficient foundation language models, 2023

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language...

    2023 arXiv

  44. [52]

    Randomized geometric algebra methods for convex neural networks, 2024

    Yifei Wang, Sungyoon Kim, Paul Chu, Indu Subramaniam, and Mert Pilanci. Randomized geometric algebra methods for convex neural networks, 2024. URL https://arxiv.org/abs/2406.02806

    2024 arXiv

  45. [53]

    Thinking like transformers, 2021

    Gail Weiss, Yoav Goldberg, and Eran Yahav. Thinking like transformers, 2021. URL https://arxiv.org/abs/2106.06981

    2021 arXiv

  46. [54]

    Wilkinson

    James H. Wilkinson. The Algebraic Eigenvalue Problem. Oxford University Press, Oxford, 1965. URL https://doi.org/10.1017/S0013091500012104

    1965 doi

  47. [55]

    Safe neurosymbolic learning with differentiable symbolic execution, 2022

    Chenxi Yang and Swarat Chaudhuri. Safe neurosymbolic learning with differentiable symbolic execution, 2022. URL https://arxiv.org/abs/2203.07671

    2022 arXiv

  48. [56]

    Circulant binary embedding

    Felix Yu, Sanjiv Kumar, Yunchao Gong, and Shih-Fu Chang. Circulant binary embedding. In International conference on machine learning, pages 946--954. PMLR, 2014

    2014

  49. [57]

    Hellaswag: Can a machine really finish your sentence?, 2019

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence?, 2019. URL https://arxiv.org/abs/1905.07830

    2019 arXiv

  50. [58]

    Lookupffn: making transformers compute-lite for cpu inference

    Zhanpeng Zeng, Michael Davies, Pranav Pulijala, Karthikeyan Sankaralingam, and Vikas Singh. Lookupffn: making transformers compute-lite for cpu inference. In International Conference on Machine Learning, pages 40707--40718. PMLR, 2023

    2023

  51. [59]

    Shufflenet: An extremely efficient convolutional neural network for mobile devices, 2017

    Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun. Shufflenet: An extremely efficient convolutional neural network for mobile devices, 2017. URL https://arxiv.org/abs/1707.01083

    2017 arXiv

  52. [60]

    Compositional diversity in visual concept learning

    Yanli Zhou, Reuben Feinman, and Brenden M Lake. Compositional diversity in visual concept learning. Cognition, 244: 0 105711, 2024

    2024

  53. [61]

    Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J

    Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J. Zico...

    2025 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.