Pith. sign in

REVIEW 4 major objections 5 minor 34 references

This paper claims that a network learns a group word by finding a low-rank implementation of its word tensor.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 22:51 UTC pith:XJHWQYYD

load-bearing objection The representation-theoretic core is solid and genuinely new, but the paper's broadest claims about standard TLPs rest on an unproven surrogate transfer. the 4 major comments →

arxiv 2509.06931 v1 pith:XJHWQYYD submitted 2025-09-08 cs.LG

Learning words in groups: fusion algebras, tensor ranks and grokking

Maor Shutman , Oren Louidor , Ran Tessler This is my paper
classification cs.LG MSC 15A6920C15
keywords grokkinggroup wordstensor rankrepresentation theoryfusion rulesHadamard modelmatrix multiplicationtwo-layer perceptron
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to explain, in mechanistic terms, how a two-layer network learns an arbitrary word operation in any finite group and why it groks. The explanation is that the learning target is a 3-tensor (the word tensor) that is typically of low rank; the low rank comes from decomposing the tensor along real irreducible-like representations and using the group's fusion rules to discard most triples. The authors show in a simplified Hadamard model that gradient descent converges to these low-rank implementations whenever the width is at least the relevant rank bound, and that for the multiplication word the terminal weights realize fast matrix multiplication. The consequence is a quantitative link between network width, tensor rank, and generalization: the width needed to learn and generalize is controlled by the rank bound, not by the group size.

Core claim

On the paper's own terms: every word operation w on a finite group G is encoded by the 3-tensor δG,w whose (a,b,c) entry is 1 if c=w(a,b) and 0 otherwise. The paper proves this tensor's bsc-3-support is contained in a combinatorial fusion cover (Proposition 4.1), so most representation-triples contribute nothing and the tensor has rank much smaller than |G|². It introduces a box-rank bound rank(δG,w) ≤ rank□(δG,w) obtained by covering the support by products of representation subspaces, and shows numerically that the Hadamard model's terminal weights are dominated by box-covers achieving this bound, or by approximations when the width is too small. For w=ab the projection onto each bsc is eq

What carries the argument

The central object is the word tensor δG,w, and the central mechanism is its bsc-support and box-rank. Basic self-conjugate (bsc) representations are the real-valued irreducible-like building blocks of the group's representation space. Fusion rules determine which triples of bscs can appear in the tensor: a triple (φ,ψ,ζ) can occur only if φ appears in ζ^{⊗n_a(w)} and ψ in ζ^{⊗n_b(w)}. This prunes the support to a small set, and covering that support by 'boxes' Φ×Ψ×Ξ yields the rank bound. In the Hadamard model, each row triple (A_i,B_i,C_i) is a rank-one tensor, so width equals tensor rank; mono-bsc-aligned configurations keep each row inside a single bsc subspace, and Proposition 5.5 shows

Load-bearing premise

The load-bearing premise is that the simplified Hadamard network behaves like the original two-layer perceptron: the paper proves only that the two are equivalent in expressive power, and relies on a Taylor-expansion heuristic for the transfer of the low-rank discovery and generalization dynamics.

What would settle it

Train the Hadamard model on D8 with word a²b at width 32 from many random seeds on the full dataset, and inspect the terminal weights' projections onto the group's basic self-conjugate subspaces. If every high-accuracy run's terminal configuration is not dominated by the predicted thin minimal box cover (rows aligned to the boxes {0}×{0..3}×{0..3}, {0..3}×{5}×{5}, {0,1,5}×{4,6}×{4,6}), then bsc-alignment is not necessary for generalization; if broad-width runs still fail to reach perfect accuracy while aligned, then the rank bound is not sufficient. Either outcome would falsify the proposed me

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • The width at which grokking becomes reliable is set by the box-rank of the word tensor, not by the naive |G|² bound; this bound can be computed from the group's fusion table.
  • Learning multiplication in a group decomposes into independent fast matrix-multiplication problems, one per representation, so a sufficiently wide network is effectively implementing a known efficient algorithm.
  • The fusion-table computation yields explicit low-rank weight configurations, giving a constructive way to initialize or verify solutions for arbitrary group words.
  • Grokking delay is the time the dynamics take to partition hidden rows into bsc-aligned groups and then solve the decoupled subproblems; sharper delay bounds would follow from analyzing the single-bsc dynamics.
  • If the width is below the rank bound, the model can still learn by approximating the low-rank tensor, with accuracy degrading gracefully as approximation error grows.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The box-rank optimization could be used as a search procedure for new low-rank algorithms for word operations, since any covering of the bsc-support directly gives a width-efficient implementation.
  • The bsc decoupling suggests that hidden-layer rows are an interpretable learned decomposition of the group algebra; examining row-to-bsc assignments in larger architectures may reveal the same tensor-rank structure behind grokking there.
  • For activation functions other than squares, the Taylor-expansion heuristic implies terminal solutions should live in higher-order tensor products of bscs; measuring the bsc boxes during training would test whether the activation's polynomial degree is the controlling factor.
  • The paper's terminal-weight alignment could be turned into a numerical method for discovering a finite group's real irreducible representations from data, without knowing them in advance.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies learning group-word operations (e.g., w(a,b)=a·b·a^{-1}) with a two-layer perceptron (TLP). It recasts the task as learning a 3-tensor in (R^G)^{⊗3}, the word tensor, and uses the representation theory of finite groups — specifically basic self-conjugate (bsc) representations and fusion rules — to show that the bsc-support of this tensor is often small. From this it derives upper bounds on tensor rank via box-covers (Propositions 4.1, 4.4). For the special case of group multiplication, it proves structure and rank bounds for each bsc projection, including explicit Strassen-type decompositions for type I, II, and III bscs (Propositions 5.1, 5.2). To connect these rank bounds to neural-network training, the paper introduces the Hadamard (HD) model, in which the hidden layer computes (Ax)⊙(By); width then equals tensor rank exactly (Eq. 38). The paper proves a decoupling/ invariance result for mono-bsc-aligned configurations under gradient descent (Proposition 5.5) and presents extensive experiments showing that GD on the HD model finds low-rank, mono-bsc-aligned terminal weights, often matching the box-cover bounds, and that the multiplication word leads to a Strassen-like efficient matrix-multiplication tensor. The paper also reports initial TLP experiments and argues that a Taylor-expansion heuristic extends the HD picture to standard activations, while explicitly conceding in Section 7.3 that the TLP case is only partially answered.

Significance. The paper offers a genuinely novel bridge between tensor-rank/complexity theory, group representation theory, and the grokking phenomenon. The mathematical core for the HD model — bsc-support sparsity, box-rank upper bounds, the Strassen-type decompositions for non-real representations, and the decoupling lemma — is elegant and, as far as I checked, correct. The empirical finding that the HD network discovers matrix-multiplication tensors of near-optimal rank is striking and well documented. If the transfer from the HD surrogate to standard two-layer networks could be made rigorous (or at least convincingly closed empirically), the paper would be a significant contribution to the mechanistic understanding of grokking and of how networks discover low-rank task representations. The main weakness is exactly this transfer: the rigorous and systematic analysis is for the HD model, whereas the headline claims in the abstract concern standard TLPs with ReLU/sigmoid/square activations. That gap is load-bearing and is acknowledged by the authors only in the discussion section.

major comments (4)
  1. [§7.3, Lemma 4.6, Eq. (38)] The central theoretical claims are about a standard two-layer network, but the analysis is carried out for the Hadamard surrogate. Eq. (38) gives the exact rank–width correspondence only for the HD model. Lemma 4.6 shows that a square-activation TLP can express any HD tensor at twice the width, but it says nothing about gradient descent finding that representation. Section 7.3 explicitly states that the TLP case is 'only partially answered' and that the argument is a Taylor-expansion heuristic supported by preliminary heatmaps (App. B.3). Since the abstract claims that a simple two-layer network with standard activations learns arbitrary group words and grokks, this surrogate gap is load-bearing for the paper's main claim. The authors should either substantially strengthen the TLP analysis or carefully restrict the paper's claims to the HD model plus empirical TLP observations.
  2. [§5.2, Eq. (45), Figure 10] Proposition 5.2 provides upper bounds on rank(δ_{G,ϕ}^{⊗3}) using the matrix-multiplication rank m_d. The text and Figure 10, however, refer to 'theoretical minimal number of rows' and to 'the width of the model, required to fulfill the learning task' (Section 1.2). An upper bound is a sufficient-width condition, not a necessary one. The empirical observation that runs succeed when width reaches the bound does not establish minimality. This conflation appears in Suggested General Principle 3(2) as well. Please rephrase these statements as sufficient-width bounds and remove the implication that the network requires exactly these widths.
  3. [§7.2, Proposition 5.5, Remark A.8] The paper's explanation of why GD finds low-rank implementations relies on convergence to mono-bsc-aligned configurations. Proposition 5.5 shows only that once a configuration is mono-bsc-aligned, it remains so and decouples; it does not prove that GD converges to such a configuration from random initialization. Section 7.2 explicitly states that a proof of global attractiveness is missing 'even in the simple case of group multiplication on Z_p.' Remark A.8 is a heuristic perturbation argument, not a rigorous proof. Thus the claim that the network 'is able to find' these implementations under gradient descent is an empirical observation for the HD model, and the mechanism by which it does so is not established for general words. The paper should distinguish clearly between the proven invariance/decoupling and the empirically observed convergence.
  4. [§4.3, Suggested General Principle 1, Figure 7] The abstract and Section 1.2 assert that word tensors are 'typically of low rank' and that an arbitrary word in any finite group can be learned with limited width. The rigorous results, however, give |G|(|G|-1)+1 as a general bound (Corollary 4.5), which is barely better than |G|^2, and the strong low-rank bounds are demonstrated only for a small set of groups and words in Figure 7. Suggested General Principle 1 is explicitly a suggestion, not a theorem. The paper would be more accurate if it stated the breadth of the claim as an empirical/conjectural principle, or provided a quantified notion of 'typically' (e.g., over random words or with some distribution over groups).
minor comments (5)
  1. [§5.5.1] Typo: 'The results thug suggest' should be 'The results thus suggest'.
  2. [§2.1.6, Figure 2, Appendix C] The main text says optimization is pure gradient descent without regularization, but Figure 2 and Appendix C report using AdamW. Please clarify which optimizer is used in each experiment and whether the pure-GD statement applies only to Section 4/5 experiments.
  3. [Eq. (31)] The displayed definition of box-rank has a missing parenthesis: it reads 'rank□(T)' with an unmatched bracket. Fix the typesetting.
  4. [§5.1, references] The text says the general-group solution was found by 'Nanda [5]' but reference [5] is Chughtai et al., while Section 6 cites [24] for the transformer-based mechanism. Please correct the citation.
  5. [Figure 10] The table column 'Theoretical min rows' should be labeled 'Theoretical sufficient rows (upper bound)' to match Proposition 5.2, as discussed in the major comments.

Circularity Check

0 steps flagged

No circularity: theoretical rank bounds use external constants and empirical width matches are genuine predictions; TLP caveat is a limitation, not a circular step.

full rationale

The paper's derivation chain is self-contained and non-circular. The theoretical rank bounds (Propositions 4.1, 4.4, 5.2) follow from representation-theoretic fusion rules and standard tensor-rank bounds; Proposition 5.2 uses only the external, unfitted matrix-multiplication rank m_d (Strassen's constant), and no fitted parameter is renamed as a prediction. The HD model's expressive equivalence (Eq. 38) is a mathematical identity, but the empirical claim is that gradient descent discovers low-rank configurations whose width matches those bounds; this is an externally checkable prediction, not an artifact of the construction. The paper contains no load-bearing self-citations: references [5], [13], [24], [27] are prior external works by other authors. The explicit caveat in Sec. 7.3 that the TLP case is 'only partially answered' concerns transfer/correctness of the surrogate-model assumption, which is a limitation, not circularity. No step reduces by definition to its own inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 5 axioms · 3 invented entities

The paper introduces no fitted constants in its derivations. It relies on standard representation theory and fusion rules as external mathematics, plus two domain-level assumptions that are flagged by the authors themselves: the HD-TLP surrogate transfer and the low-rank-explains-generalization hypothesis. The HD model and box-cover machinery are new constructs with independent empirical and theoretical support, not ad hoc entities.

axioms (5)
  • standard math Standard representation theory facts: Schur orthogonality, decomposition of the regular representation, and the orthogonal decomposition of R^G into matrix-coefficient spaces of bscs (Equation 13).
    Invoked throughout Section 3 and in the proofs of Proposition 5.1 and Lemma A.7.
  • standard math Fusion rules: tensor products of bscs decompose into direct sums of bscs, with the CFC inclusion of Proposition 4.1 bounding the true bsc-3-support.
    Used to define bscs_CFC in (24) and to prove Proposition 4.1 via Observation A.2.
  • ad hoc to paper The Hadamard (HD) model is phenomenologically similar to the TLP model; the low-rank discovery mechanism transfers.
    The paper motivates this in Section 4.4 but only proves expressiveness (Lemma 4.6); Section 7.3 concedes TLP is only partially answered.
  • ad hoc to paper Word tensors are typically low-rank, and low rank is the key reason the network generalizes with limited width.
    Stated as Suggested General Principle 1 and used to frame the empirical analysis, but not proven.
  • domain assumption Genericity assumption in Remark A.8: at least one of {Ai⊗Bi}, {Ai⊗Ci}, {Bi⊗Ci} is linearly independent, for local attractiveness of bsc-decoupling.
    Needed for the local-attractiveness remark; stated as generic when the number of rows is much smaller than |G|^2.
invented entities (3)
  • Hadamard (HD) model independent evidence
    purpose: Surrogate network in which width exactly equals tensor rank, making the connection between low-rank word tensors and network capacity directly readable from the weights.
    Defined in Section 4.4; empirically validated across groups and words, and expressively equivalent (up to doubling width) to the square-activation TLP by Lemma 4.6.
  • Box-cover and box-rank framework independent evidence
    purpose: Combinatorial machinery to turn the bsc-3-support of a word tensor into upper bounds on its tensor rank (Propositions 4.4 and (33)).
    Gives concrete, checkable rank bounds (Figure 7) that the empirical width requirements match.
  • Mono-bsc-aligned weight configurations independent evidence
    purpose: Idealized terminal states in which each row of A, B, C lies in a single bsc subspace, enabling the loss-decoupling and dynamics-decoupling analysis of Section 5.4.
    Stability under GD is proven in Proposition 5.5, and heatmaps show terminal weights approximate such configurations.

pith-pipeline@v1.3.0-alltime-deepseek · 52211 in / 19057 out tokens · 189746 ms · 2026-08-04T22:51:39.697195+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Learning words in groups: fusion algebras, tensor ranks and grokking." pith.science (2026). https://pith.science/paper/XJHWQYYD

@misc{pith2026250906931,
  author       = {Pith},
  title        = {Pith review of: Learning words in groups: fusion algebras, tensor ranks and grokking},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XJHWQYYD}},
  note         = {Machine review of arXiv:2509.06931}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

In this work, we demonstrate that a simple two-layer neural network with standard activation functions can learn an arbitrary word operation in any finite group, provided sufficient width is available and exhibits grokking while doing so. To explain the mechanism by which this is achieved, we reframe the problem as that of learning a particular $3$-tensor, which we show is typically of low rank. A key insight is that low-rank implementations of this tensor can be obtained by decomposing it along triplets of basic self-conjugate representations of the group and leveraging the fusion structure to rule out many components. Focusing on a phenomenologically similar but more tractable surrogate model, we show that the network is able to find such low-rank implementations (or approximations thereof), thereby using limited width to approximate the word-tensor in a generalizable way. In the case of the simple multiplication word, we further elucidate the form of these low-rank implementations, showing that the network effectively implements efficient matrix multiplication in the sense of Strassen. Our work also sheds light on the mechanism by which a network reaches such a solution under gradient descent.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 19 canonical work pages · 1 internal anchor

  1. [1]

    S.-i. Amari. Feature spaces which admit and detect invariant signal transfor- mations, 1978

    work page 1978

  2. [2]

    Barak, B

    B. Barak, B. Edelman, S. Goel, S. Kakade, E. Malach, and C. Zhang. Hidden progress in deep learning: Sgd learns parities near the computational limit.Advances in Neural Information Processing Systems, 35:21750– 21764, 2022

    work page 2022

  3. [3]

    Bl ¨aser

    M. Bl ¨aser. Complexity of bilinear problems.Lecture notes, 2009

    work page 2009

  4. [4]

    M. M. Bronstein, J. Bruna, T. Cohen, and P. Veli ˇckovi´c. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges.arXiv preprint arXiv:2104.13478, 2021

    Pith/arXiv arXiv 2021

  5. [5]

    Chughtai, L

    B. Chughtai, L. Chan, and N. Nanda. A toy model of universality: Reverse engineering how networks learn group operations. InInternational Conference on Machine Learning, pages 6243–6267. PMLR, 2023

    work page 2023

  6. [6]

    Doshi, A

    D. Doshi, A. Das, T. He, and A. Gromov. To grok or not to grok: Disentangling generalization and memorization on corrupted algorithmic datasets.arXiv preprint arXiv:2310.13061, 2023

    Pith/arXiv arXiv 2023

  7. [7]

    Doshi, T

    D. Doshi, T. He, A. Das, and A. Gromov. Grokking modular polynomials.arXiv preprint arXiv:2406.03495, 2024

    Pith/arXiv arXiv 2024

  8. [8]

    S. Fan, R. Pascanu, and M. Jaggi. Deep grokking: Would deep neural networks generalize better?arXiv preprint arXiv:2405.19454, 2024

    Pith/arXiv arXiv 2024

  9. [9]

    Fawzi, M

    A. Fawzi, M. Balog, A. Huang, T. Hubert, B. Romera-Paredes, M. Barekatain, A. Novikov, F. J. R. Ruiz, J. Schrit- twieser, G. Swirszcz, et al. Discovering faster matrix multiplication algorithms with reinforcement learning.Na- ture, 610(7930):47–53, 2022

    work page 2022

  10. [10]

    Fukushima

    K. Fukushima. Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position.Biological cybernetics, 36(4):193–202, 1980

    work page 1980

  11. [11]

    Fulton and J

    W. Fulton and J. Harris.Representation theory: a first course, volume 129. Springer Science & Business Media, 2013

    work page 2013

  12. [12]

    Goller and A

    C. Goller and A. Kuchler. Learning task-dependent distributed representations by backpropagation through struc- ture. InProceedings of international conference on neural networks (ICNN’96), volume 1, pages 347–352. IEEE, 1996

    work page 1996

  13. [13]

    A. Gromov. Grokking modular arithmetic.arXiv preprint arXiv:2301.02679, 2023

    Pith/arXiv arXiv 2023

  14. [14]

    J. Håstad. Tensor rank is np-complete.Journal of algorithms, 11(4):644–654, 1990

    work page 1990

  15. [15]

    C. J. Hillar and L.-H. Lim. Most tensor problems are np-hard.Journal of the ACM (JACM), 60(6):1–39, 2013

    work page 2013

  16. [16]

    A. I. Humayun, R. Balestriero, and R. Baraniuk. Deep networks always grok and here is why.arXiv preprint arXiv:2402.15555, 2024

    Pith/arXiv arXiv 2024

  17. [17]

    Kanatani.Group-theoretical methods in image understanding, volume 20

    K.-I. Kanatani.Group-theoretical methods in image understanding, volume 20. Springer Science & Business Media, 2012

    work page 2012

  18. [18]

    Kumar, B

    T. Kumar, B. Bordelon, S. J. Gershman, and C. Pehlevan. Grokking as the transition from lazy to rich training dynamics.arXiv preprint arXiv:2310.06110, 2023

    Pith/arXiv arXiv 2023

  19. [19]

    Lenz.Group theoretical methods in image processing, volume 413

    R. Lenz.Group theoretical methods in image processing, volume 413. Springer, 1990

    work page 1990

  20. [20]

    Z. Liu, O. Kitouni, N. S. Nolte, E. Michaud, M. Tegmark, and M. Williams. Towards understanding grokking: An effective theory of representation learning.Advances in Neural Information Processing Systems, 35:34651–34663, 2022

    work page 2022

  21. [21]

    Z. Liu, E. J. Michaud, and M. Tegmark. Omnigrok: Grokking beyond algorithmic data. InThe Eleventh Interna- tional Conference on Learning Representations, 2022

    work page 2022

  22. [22]

    K. Lyu, J. Jin, Z. Li, S. S. Du, J. D. Lee, and W. Hu. Dichotomy of early and late phase implicit biases can provably induce grokking.arXiv preprint arXiv:2311.18817, 2023

    Pith/arXiv arXiv 2023

  23. [23]

    M. A. Mohamadi, Z. Li, L. Wu, and D. J. Sutherland. Why do you grok? a theoretical analysis of grokking modular addition.arXiv preprint arXiv:2407.12332, 2024

    Pith/arXiv arXiv 2024

  24. [24]

    Nanda, L

    N. Nanda, L. Chan, T. Lieberum, J. Smith, and J. Steinhardt. Progress measures for grokking via mechanistic interpretability.arXiv preprint arXiv:2301.05217, 2023

    Pith/arXiv arXiv 2023

  25. [25]

    Tensor Rank and Complexity

    G. Ottaviani and P. Reichenbach. Tensor rank and complexity.arXiv preprint arXiv:2004.01492, 2020. LEARNING WORDS IN GROUPS: FUSION ALGEBRAS, TENSOR RANKS AND GROKKING 22

    work page internal anchor Pith review Pith/arXiv arXiv 2004

  26. [26]

    Is there a faster method to compute the composition of 2 3d rotations than directly using the formula for quaternion product? Mathematics Stack Exchange

    paperclip optimizer (https://math.stackexchange.com/users/1145344/paperclip optimizer). Is there a faster method to compute the composition of 2 3d rotations than directly using the formula for quaternion product? Mathematics Stack Exchange. URL:https://math.stackexchange.com/q/4629959 (version: 2023-11-25)

    work page arXiv 2023

  27. [27]

    Power, Y

    A. Power, Y . Burda, H. Edwards, I. Babuschkin, and V . Misra. Grokking: Generalization beyond overfitting on small algorithmic datasets.arXiv preprint arXiv:2201.02177, 2022

    Pith/arXiv arXiv 2022

  28. [28]

    Rubin, I

    N. Rubin, I. Seroussi, and Z. Ringel. Grokking as a first order phase transition in two layer networks.arXiv preprint arXiv:2310.03789, 2023

    Pith/arXiv arXiv 2023

  29. [29]

    Sperduti

    A. Sperduti. Encoding labeled graphs by labeling raam.Advances in Neural Information Processing Systems, 6, 1993

    work page 1993

  30. [30]

    Strassen

    V . Strassen. Gaussian elimination is not optimal.Numerische mathematik, 13(4):354–356, 1969

    work page 1969

  31. [31]

    Thilak, E

    V . Thilak, E. Littwin, S. Zhai, O. Saremi, R. Paiss, and J. Susskind. The slingshot mechanism: An empirical study of adaptive optimizers and the grokking phenomenon.arXiv preprint arXiv:2206.04817, 2022

    Pith/arXiv arXiv 2022

  32. [32]

    Varma, R

    V . Varma, R. Shah, Z. Kenton, J. Kram´ar, and R. Kumar. Explaining grokking through circuit efficiency, september 2023.URL http://arxiv. org/abs/2309.02390

    Pith/arXiv arXiv 2023

  33. [33]

    Wood and J

    J. Wood and J. Shawe-Taylor. Representation theory and invariant neural networks.Discrete applied mathematics, 69(1-2):33–60, 1996

    work page 1996

  34. [34]

    Z. Xu, Y . Wang, S. Frei, G. Vardi, and W. Hu. Benign overfitting and grokking in relu networks for xor cluster data.arXiv preprint arXiv:2310.02541, 2023. AppendixA. Proofs of statements Proof of Lemma 3.3.Assume without loss of generalitya≤b≤c.ThenTcan be written asP i∈[a] 1i⊗T i,where 1 i is theith standard basis element ofR a andT i is a 2−tensor inR ...

    Pith/arXiv arXiv 2023