Pith. sign in

REVIEW 3 major objections 4 minor 60 references

Flopping for FLOPs: Leveraging equivariance for computational efficiency

T0 review · 3 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Making vision networks equivariant to horizontal mirroring (flopping) lets ResMLP, ConvNeXt and ViT classifiers match or beat their ImageNet-1K baselines at scale while using about half the FLOPs and half the parameters.

desk verdict The block-diagonal FLOPs construction is correct and useful, but the accuracy-parity claim at scale rests on uncontrolled baselines and single runs and is not yet established. read the letter →

arxiv 2502.05169 v2 pith:GXSQSUQD submitted 2025-02-07 cs.CV cs.LG

Georg Bökman , David Nordström , Fredrik Kahl This is my paper · ORCID
classification cs.CVcs.LG
keywords equivariantneuralnetworksfloppinginvariancemirrorsymmetryirreduciblerepresentationsSchur'slemmaFLOPsefficiencymodelscalingImageNet-1Kclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper attacks the assumption that symmetry in a neural network must be bought with extra compute. It studies horizontal mirroring ('flopping'), whose symmetry group admits exactly two kinds of features: those unchanged by a mirror flip and those that change sign. If every layer stores its activations in this two-part form, any equivariant linear map becomes block-diagonal, so one full-size matrix product turns into two half-size products, which is why the equivariant networks use about half the FLOPs of their baselines. The authors build flopping-equivariant versions of ResMLP, ConvNeXt and ViT using the same ImageNet-1K training recipes as the ordinary models and report that at the largest sizes the equivariant versions match or beat baseline accuracy while running faster. The intended lesson is that geometric prior knowledge and compute scaling are complementary: hard-coding one small symmetry yields a constant-factor efficiency gain that grows as networks scale.

What carries the argument

The engine is the block-diagonalization identity (2): in a basis split between mirror-invariant and mirror-antisymmetric channels, equivariant linear maps act as $\mathrm{diag}(W_{1,1}, W_{-1,-1})$, a special case of Schur's lemma stating that equivariant maps between distinct irreducible representations must vanish. Around that identity the paper fits the remaining layers: an equivariant GELU computed as $\sigma((x_1+x_{-1})/\sqrt{2})$ and $\sigma((x_1-x_{-1})/\sqrt{2})$ recombined with a $1/\sqrt{2}$ factor (Eq. 3), invariant attention logits formed by pairing invariant with invariant and sign-flip with sign-flip features (Eq. 4), and symmetric/antisymmetric kernels in the patch embedding and depthwise convolutions. The reason the savings show up in wall-clock time, unlike earlier steerable-convolution implementations that reused ordinary convolution routines, is that the chosen backbones are dominated by dense linear layers — fully connected layers, 1x1 convolutions and attention projections — where the block-diagonal split halves FLOPs directly.

What would settle it

Train a large flopping-equivariant model (say, the ViT-H or ResMLP-L24 variant) in which the fraction of invariant versus sign-flip channels is learned per layer rather than fixed at 50/50; if the learned allocation departs strongly from equality and simultaneously improves top-1 accuracy by more than about a point, the premise that the fixed equal split is harmless at scale would be refuted. A weaker test: rerun the largest models with the second half of the network also equivariantized, since the paper's hybrid design equivariantizes only the first half of blocks, and check whether accuracy drops.

Watch

Extended reading notes

Core claim

Horizontal mirroring is the two-element group $D_2$, whose only irreducible representations are the trivial one ($+1$, mirror-invariant features) and the sign-flip one ($-1$, mirror-antisymmetric features). The paper's central discovery is that parametrizing every feature map as half invariant and half $(-1)$-equivariant channels forces every equivariant linear layer into block-diagonal form, $W = \mathrm{diag}(W_{1,1}, W_{-1,-1})$, by Schur's lemma: a $d \times c$ matrix-vector product becomes two $(d/2) \times (c/2)$ products, halving FLOPs while also halving trainable parameters, so the equivariant network keeps the same FLOPs-per-parameter ratio as an ordinary network even though its total compute is cut in half. Around this core the paper assembles the rest of an equivariant network: patch embeddings built from symmetric and antisymmetric filters, an equivariant GELU that applies the nonlinearity in a mirrored sum/difference basis and transforms back, attention scores that are invariant because they pair like features, and mirror-adapted positional encodings and class tokens for transformers. Applied to ResMLP, ViT and isotropic ConvNeXt, the recipe yields on ImageNet-1K a set of equivariant models that at L/H scale match or beat their baselines — E(ResMLP-L24) reaches 81.5% against 80.4%, E(ConvNeXt-L) ties 82.6% using 30.3 vs 60.0 GFLOPs, and H(ViT-H) reaches 85.0% against 84.6% — leading the authors to conclude that equivariant networks can be simple models that scale well.

Load-bearing premise

Forcing every layer to commit exactly half its channels to mirror-unchanged features and half to mirror-flipped features — with no freedom to learn a better split — must not meaningfully reduce what the network can learn once it is large, and the paper's own small-model results (3 to 5 point losses) show the premise is not automatically true.

Editorial extensions

If this is right

  • Equivariance need not cost compute: for the two-element flop group in linear-layer-dominated architectures, FLOPs per parameter match ordinary networks while total FLOPs and parameters are halved.
  • At scale, hard-coding flopping does not cost accuracy: the largest equivariant models match or beat their baselines on ImageNet-1K (E(ResMLP-L24) 81.5 vs 80.4; E(ConvNeXt-L) 82.6 vs 82.6; E(ViT-H) 84.4 vs 84.6), and the hybrid H(ViT-H) at 85.0 is the best model reported.
  • Throughput gains widen with model size, since dense linear layers come to dominate runtime; the paper reports throughput improvements up to roughly 59% for the largest equivariant ResMLP, while small models can be slightly slower.
  • The block-diagonal construction generalizes to any finite symmetry group via Schur's lemma and Maschke's theorem, with savings set by the number, dimension and type of the group's irreps; the paper singles out $D_8$, the symmetry group of the square grid, as a promising next step.
  • Equivariant backbones run on the unmodified training recipes of their baselines, so the efficiency gain comes without new training procedures or hyperparameter tuning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An extrapolation the authors do not make: on a fixed compute budget, adopting a flopping-equivariant backbone resembles doubling the effective model size, since half the FLOPs and half the parameters buy the same ImageNet accuracy at scale.
  • The recipe should transfer wherever a feature extractor is dominated by dense linear layers — self-supervised visual pretraining, dense prediction heads, or keypoint descriptors — but nothing in the paper measures those settings, so the efficiency gain outside classification is untested.
  • The small-versus-large divergence (3–5 point losses at S/T sizes, parity at L/H, and the failure of E(ViT-S) to converge) suggests that the standard heavy regularization recipes over-constrain heavily symmetrized small networks; tuning the invariant/equivariant split or the recipe per model size is a concrete follow-up experiment.
  • The exact factor-of-two saving is specific to two-element mirror symmetry, because both $D_2$ irreps are one-dimensional; for rotation groups with higher-dimensional irreps the per-parameter compute ratio worsens, so the headline result should not be assumed to carry over to full rotational equivariance.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces flopping-equivariant versions of ResMLP, ViT, and ConvNeXt for image classification. The core construction is to split every feature space into D2-invariant and (-1)-equivariant channels, so that equivariant linear maps between these spaces are block-diagonal (Eq. 2). The paper shows that this block structure halves the FLOPs of the dense linear layers, gives an equivariant patch embedding, normalization, GELU, and attention mechanism, and supports hybrid models in which only the first half of the blocks is equivariant. On ImageNet-1K, the equivariant models are reported to reach comparable or better accuracy than the baselines at roughly half the parameters and FLOPs, with increasing throughput gains at larger scales. The authors are candid that baselines were not rerun, that each configuration is a single run, that E(ViT-S) failed to converge, and that the equal split between invariant and equivariant features is not known to be optimal.

Significance. The analytical contribution is valuable and appears sound: the FLOPs-per-parameter parity is a direct, parameter-free consequence of block-diagonalizing linear maps between D2 irreps, not a number fitted to data. If the accuracy-parity claim held under controlled comparisons, this would be an important practical result, since equivariant networks are often assumed to be compute-inefficient per parameter. The paper also demonstrates the value of separating the equivariant-design question from the training-recipe question, and it reports negative small-scale results honestly. However, the empirical evidence for the headline claim is currently not at the standard needed for a journal publication: the large-scale comparisons rest on single runs against literature baselines, with differences that are comparable to typical ImageNet run-to-run noise. The significance of the paper is therefore conditional on a controlled experimental verification.

major comments (3)
  1. [Section 6, Table 1, Section 6.2] The central claim in Section 1 and Figure 5, that scaled equivariant models achieve comparable or improved accuracy at half FLOPs, is not supported by a controlled comparison. Table 1 states that baselines were not rerun, and Section 6.2 states that each configuration was run once. The large-scale differences are small: E(ViT-H) is 84.4 vs. 84.6, E(ConvNeXt-L) is 82.6 vs. 82.6, H(ViT-H) is 85.0 vs. 84.6, and E(ResMLP-L24) is 81.5 vs. 80.4 against a baseline that the authors themselves call unexpectedly low. With ImageNet-1K run-to-run standard deviations typically around 0.2-0.3 points, these gaps are within noise. Please rerun the baselines under the same training recipe, hardware, and compiler settings, and report means and standard deviations over multiple seeds; without this, the accuracy-parity conclusion is unverified.
  2. [Section 3.5, Table 1] The assumption that a fixed 1:1 split of channels into invariant and (-1)-equivariant features does not materially limit accuracy is load-bearing for the scaling claim, and the paper's own data show it can fail. Section 3.5 acknowledges that it is not obvious that the equal split is optimal, and the small models lose substantial accuracy: E(ResMLP-T12) is 72.0 vs. 76.6, E(ConvNeXt-S) is 76.8 vs. 79.7, and E(ViT-S) fails to converge. Because the large-scale comparisons are uncontrolled, the paper does not establish that the split becomes harmless at scale; the apparent improvement could equally be explained by baseline underperformance or by noise. Please provide a sensitivity analysis over the invariant/equivariant channel ratio, or otherwise give evidence from trained large models that the equal split is capacity-preserving.
  3. [Section B, Table 1 caption, Figure 6] The wall-clock time claim is confounded by implementation and hardware differences. Section B states that the baselines were trained on V100 GPUs by their respective authors, while the reported throughput and peak memory are measured on a single A100-40GB GPU with PyTorch 2.5.1, Apex, torch.compile, and FlashAttention. It is unclear from the Table 1 caption whether the baseline throughput numbers were also measured on the same A100 machine under the same compilation and mixed-precision settings. If they were not, the throughput gains in Figure 6 may reflect the measurement harness rather than the equivariant construction. Please state explicitly whether all throughput and memory numbers were measured in one environment for both E(X) and X, and if not, provide such same-environment measurements.
minor comments (4)
  1. [Eq. (4)] The last expression in Eq. (4) appears to have a typo: it should use the same index on the key as on the query, e.g., q_{i,-1} · k_{j,-1} = (-q_{i,-1}) · (-k_{j,-1}); the current text writes -k_{i,-1} with the query index.
  2. [Section C] The four tensors x_{1,1}, x_{1,-1}, x_{-1,1}, and x_{-1,-1} are easy to confuse; a small table or a more systematic naming scheme with explicit signs would improve readability.
  3. [Table 2] Several learning-rate entries such as "3.10−3" and "5.10−3" are ambiguous; please use standard scientific notation such as 3×10⁻³.
  4. [Section 6.2] The failed convergence of E(ViT-S) is reported only in the table footnote; since this is one of the smallest-scale points and directly relevant to the scaling argument, it deserves at least a sentence in the main text and a marker in Figure 5 if included there.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the half-FLOPs property is an analytic consequence of D2 block-diagonalization; the self-citations in Section 3.5 are explicitly hedged and not load-bearing.

full rationale

The central FLOPs claim is derived, not fitted. Equation (2) follows from the definition of D2-equivariant linear maps (or Schur's lemma, Lemma A.2): off-diagonal blocks would break the sign transformation law, so an equivariant linear layer is block-diagonal by construction. No data-dependent parameter is fitted and then renamed as a prediction; the half-parameters/half-FLOPs relation is an architectural identity. The accuracy-parity claim at scale is an empirical assertion based on Table 1, whose limitations the paper discloses in Section 6.2: baselines are not rerun, hardware and software differ, and each configuration is a single run. Those are evidence-quality concerns, not circularity. The only author self-citations occur in Section 3.5, where Bökman & Kahl (2023) and Bökman et al. (2024) are used to note that learned representations may have more invariant than equivariant features and that the equal split is not obviously optimal. The paper explicitly states 'it is not obvious that the choice with an equal number of invariant and (-1)-equivariant features is optimal' and defers optimal design to future work. The FLOPs derivation does not depend on that split being optimal, so the self-citation is not load-bearing. No uniqueness theorem from the authors' prior work is imported, no ansatz is smuggled via citation, and known steerable-CNN results are attributed rather than renamed. Score 2 reflects the presence of non-load-bearing self-citation without any circular derivation.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The FLOPs-halving construction uses only standard representation theory (Maschke, Schur) and the definition of D2; no constants are fitted to data. The empirical accuracy claim additionally assumes the hand-chosen 1:1 feature split and the transfer of baseline training recipes to the equivariant variants. No new entities are postulated.

free parameters (2)
  • Invariant/equivariant channel split ratio = 1:1
    Hand-chosen in Section 3.2 (PatchEmbed) and applied to all linear layers. The paper itself notes in Section 3.5 that an equal split is not obviously optimal and cites Bokman & Kahl 2023 and Bokman et al. 2024 showing better allocations may exist. Affects accuracy but not the FLOPs-per-parameter derivation.
  • Hybrid equivariant layer split = first half of residual blocks equivariant, second half ordinary
    Hand-chosen for H(ViT) variants following Weiler & Cesa 2019; affects the reported H(ViT) accuracy numbers.
assumptions (6)
  • standard math Maschke's theorem: every real representation of a finite group decomposes into a direct sum of irreps.
    Used in Section A (Theorem A.1) and Section 3.5 to justify isotypical decomposition of the steering matrix.
  • standard math Schur's lemma: linear equivariant maps between non-isomorphic irreps vanish.
    Used in Section A (Lemma A.2) and Section 3.1 to derive block-diagonal linear layers (Eq. 2).
  • standard math D2 has exactly two one-dimensional real irreps, the trivial and sign-flip representations.
    Used in Section A and Section 3.1; this is what makes the block structure two blocks of half size.
  • domain assumption Upright image classification is invariant to horizontal mirroring.
    Stated in Section 1 and Figure 1; motivates the whole approach.
  • domain assumption Ordinary networks trained on symmetric data become approximately equivariant, so hard-coding equivariance does not limit representational power.
    Argued in Section 3.5 using prior work including the authors' own (Bokman & Kahl 2023); it is a motivation for the accuracy-parity claim, not an input to the FLOPs derivation.
  • domain assumption The same training recipes as the non-equivariant baselines transfer to the equivariant variants without retuning.
    Section 6 and Table 2; the paper reports instability and convergence failure for some variants, so this transferability is not guaranteed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Flopping for FLOPs: Leveraging equivariance for computational efficiency." pith.science (2026). https://pith.science/paper/GXSQSUQD

@misc{pith2026250205169,
  author       = {Pith},
  title        = {Pith review of: Flopping for FLOPs: Leveraging equivariance for computational efficiency},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GXSQSUQD}},
  note         = {Machine review of arXiv:2502.05169}
}
read the original abstract

Incorporating geometric invariance into neural networks enhances parameter efficiency but typically increases computational costs. This paper introduces new equivariant neural networks that preserve symmetry while maintaining a comparable number of floating-point operations (FLOPs) per parameter to standard non-equivariant networks. We focus on horizontal mirroring (flopping) invariance, common in many computer vision tasks. The main idea is to parametrize the feature spaces in terms of mirror-symmetric and mirror-antisymmetric features, i.e., irreps of the flopping group. This decomposes the linear layers to be block-diagonal, requiring half the number of FLOPs. Our approach reduces both FLOPs and wall-clock time, providing a practical solution for efficient, scalable symmetry-aware architectures.

Figures

Figures reproduced from arXiv: 2502.05169 by the authors.

Figure 1
Figure 1. Common image classification tasks are invariant to [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Patch embedding layer. The patch embed￾ding (PatchEmbed) layer is common in modern networks, proposed with the ViT-model (Dosovitskiy et al., 2021). PatchEmbed is a convolutional layer with stride equal to the kernel size of the convolution filters. In our equivariant architectures, we enforce half of the filters to be symmet￾ric and half to be antisymmetric. When the input image is flopped, the output feature map o… view at source ↗
Figure 3
Figure 3. shows a schematic of the network design. All three network families incorporate a patch embedding layer (“PatchEmbed”) as the first layer, as outlined in Section 3.2. We replace all linear layers (except the depthwise convolu￾tions in ConvNeXt) by block diagonal linear layers as in (2) and use equivariant layer norm, attention and GELU￾nonlinearities as discussed in Section 3. Further special layers will be discusse… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: ). 101 102 Number of Parameters (×106 ) 101 102 FLOPs ( ×10 9 ) DeiT III ResMLP ConvNeXt-ISO Equivariant [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Validation accuracy on ImageNet-1K versus model complexity as measured by the number of FLOPs required per [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Throughput gain based on model size. The throughput gain as measured by the percentage differ￾ence between the equivariant and baseline implementations, placed according to the baseline’s number of FLOPs. models. The hardware and software used for our training runs dif…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

60 extracted references · 38 canonical work pages

  1. [1]

    L., Kiros, J

    Ba, J. L., Kiros, J. R., and Hinton, G. E. Layer normalization. arXiv preprint arXiv:1607.06450, 2016

    arXiv 2016

  2. [2]

    Neural machine translation by jointly learning to align and translate

    Bahdanau, D., Cho, K., and Bengio, Y. Neural machine translation by jointly learning to align and translate. ICLR, 2015

    work page 2015

  3. [3]

    J., Lafarge, M

    Bekkers, E. J., Lafarge, M. W., Veta, M., Eppenhof, K. A., Pluim, J. P., and Duits, R. Roto-translation covariant convolutional networks for medical image analysis. In Medical Image Computing and Computer Assisted Intervention--MICCAI 2018: 21st International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part I, pp.\ 440--448. Springer, 2018

    work page 2018

  4. [4]

    J., Vadgama, S., Hesselink, R., der Linden, P

    Bekkers, E. J., Vadgama, S., Hesselink, R., der Linden, P. A. V., and Romero, D. W. Fast, expressive \ mathrm\ SE \ (n)\ equivariant networks through weight-sharing in position-orientation space. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=dPHLbUqGbr

    work page 2024

  5. [5]

    Boardman, J. M. Real and Complex Representations , 2007. URL https://web.archive.org/web/20240419161331/https://math.jhu.edu/ jmb/note/rschur.pdf

    work page arXiv 2007

  6. [6]

    and Kahl, F

    B\" o kman, G. and Kahl, F. Investigating how relu-networks encode symmetries. In Advances in Neural Information Processing Systems, volume 36, pp.\ 13720--13744. Curran Associates, Inc., 2023

    work page 2023

  7. [7]

    Steerers: A framework for rotation equivariant keypoint descriptors

    B \"o kman, G., Edstedt, J., Felsberg, M., and Kahl, F. Steerers: A framework for rotation equivariant keypoint descriptors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 4885--4895, 2024

    work page 2024

  8. [8]

    Brehmer, J., de Haan, P., Behrends, S., and Cohen, T. S. Geometric algebra transformer. In Advances in Neural Information Processing Systems, volume 36, pp.\ 35472--35496. Curran Associates, Inc., 2023

    work page 2023

Show all 60 references
  1. [9]

    Does equivariance matter at scale? arXiv preprint arXiv:2410.23179, 2024

    Brehmer, J., Behrends, S., de Haan, P., and Cohen, T. Does equivariance matter at scale? arXiv preprint arXiv:2410.23179, 2024

    2024 arXiv

  2. [10]

    M., Bruna, J., Cohen, T., and Veli c kovi \'c , P

    Bronstein, M. M., Bruna, J., Cohen, T., and Veli c kovi \'c , P. Geometric Deep Learning : Grids, Groups , Graphs , Geodesics , and Gauges . arXiv:2104.13478 [cs, stat], May 2021

    2021 arXiv

  3. [11]

    Bruintjes, R.-J., Motyka, T., and van Gemert, J. What affects learned equivariance in deep image recognition models? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp.\ 4838--4846, June 2023

    2023

  4. [12]

    and Welling, M

    Cohen, T. and Welling, M. Group equivariant convolutional networks. In Int. Conf. Machine Learning, 2016

    2016

  5. [13]

    and Welling, M

    Cohen, T. and Welling, M. Steerable CNN s. In Int. Conf. Learn. Represent., 2017

    2017

  6. [14]

    Flashattention-2: Faster attention with better parallelism and work partitioning, 2023

    Dao, T. Flashattention-2: Faster attention with better parallelism and work partitioning, 2023. URL https://arxiv.org/abs/2307.08691

    2023 arXiv

  7. [15]

    Imagenet: A large-scale hierarchical image database

    Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp.\ 248--255, 2009. doi:10.1109/CVPR.2009.5206848

    2009

  8. [16]

    Exploiting cyclic symmetry in convolutional neural networks

    Dieleman, S., De Fauw, J., and Kavukcuoglu, K. Exploiting cyclic symmetry in convolutional neural networks. In International conference on machine learning, pp.\ 1889--1898. PMLR, 2016

    2016

  9. [17]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on...

    2021

  10. [18]

    Cognitron: A self-organizing multilayered neural network

    Fukushima, K. Cognitron: A self-organizing multilayered neural network. Biological cybernetics, 20 0 (3): 0 121--136, 1975

    1975

  11. [19]

    Learning and leveraging world models in visual representation learning

    Garrido, Q., Assran, M., Ballas, N., Bardes, A., Najman, L., and LeCun, Y. Learning and leveraging world models in visual representation learning. arXiv preprint arXiv:2403.00504, 2024

    2024 arXiv

  12. [20]

    E., Aronsson, J., Carlsson, O., Linander, H., Ohlsson, F., Petersson, C., and Persson, D

    Gerken, J. E., Aronsson, J., Carlsson, O., Linander, H., Ohlsson, F., Petersson, C., and Persson, D. Geometric deep learning and equivariant neural networks. Artificial Intelligence Review, 56 0 (12): 0 14605--14662, 2023

    2023

  13. [21]

    A., Goldblum, M., and Wilson, A

    Gruver, N., Finzi, M. A., Goldblum, M., and Wilson, A. G. The lie derivative for measuring learned equivariance. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=JL7Va5Vy15J

    2023

  14. [22]

    Deep residual learning for image recognition

    He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 770--778, 2016 a

    2016

  15. [23]

    Identity mappings in deep residual networks

    He, K., Zhang, X., Ren, S., and Sun, J. Identity mappings in deep residual networks. CoRR, abs/1603.05027, 2016 b . URL http://arxiv.org/abs/1603.05027

    2016 arXiv

  16. [24]

    and Gimpel, K

    Hendrycks, D. and Gimpel, K. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415, 2016

    2016 arXiv

  17. [25]

    B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D

    Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020

    2001 arXiv

  18. [26]

    Y., Platt, R., and Walters, R

    Klee, D., Park, J. Y., Platt, R., and Walters, R. A comparison of equivariant vision models with imagenet pre-training. In NeurIPS 2023 Workshop on Symmetry and Geometry in Neural Representations, 2023. URL https://openreview.net/forum?id=3ItzNHPov9

    2023

  19. [27]

    M., Romero, D

    Knigge, D. M., Romero, D. W., and Bekkers, E. J. Exploiting redundancy: Separable group convolutional networks on lie groups. In International Conference on Machine Learning, pp.\ 11359--11386. PMLR, 2022

    2022

  20. [28]

    Clebsch gordan nets: a fully fourier space spherical convolutional neural network

    Kondor, R., Lin, Z., and Trivedi, S. Clebsch gordan nets: a fully fourier space spherical convolutional neural network. In Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018

    2018

  21. [29]

    and Kondor, R

    Kundu, S. and Kondor, R. Steerable transformers, 2024. URL https://arxiv.org/abs/2405.15932

    2024

  22. [30]

    S., Henderson, D., Howard, R

    LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., and Jackel, L. D. Backpropagation applied to handwritten zip code recognition. Neural Computation, 1 0 (4): 0 541--551, 1989

    1989

  23. [31]

    Learning rotation-equivariant features for visual correspondence

    Lee, J., Kim, B., Kim, S., and Cho, M. Learning rotation-equivariant features for visual correspondence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 21887--21897, 2023

    2023

  24. [32]

    and Vedaldi, A

    Lenc, K. and Vedaldi, A. Understanding image representations by measuring their equivariance and equivalence. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 991--999, 2015

    2015

  25. [33]

    and van Gemert, J

    Lengyel, A. and van Gemert, J. Exploiting learned symmetries in group equivariant convolutions. In 2021 IEEE International Conference on Image Processing (ICIP), pp.\ 759--763. IEEE, 2021

    2021

  26. [34]

    A convnet for the 2020s

    Liu, Z., Mao, H., Wu, C.-Y., Feichtenhofer, C., Darrell, T., and Xie, S. A convnet for the 2020s. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 11976--11986, 2022

    2022

  27. [35]

    Lowe, D. G. Distinctive Image Features from Scale - Invariant Keypoints . Int. J. Comput. Vis., 60 0 (2): 0 91--110, November 2004. doi:10/bqrmsp

    2004

  28. [36]

    and Eklundh, J.-O

    Loy, G. and Eklundh, J.-O. Detecting symmetry and symmetric constellations of features. In Computer Vision--ECCV 2006: 9th European Conference on Computer Vision, Graz, Austria, May 7-13, 2006. Proceedings, Part II 9, pp.\ 508--521. Springer, 2006

    2006

  29. [37]

    L., Hillar, C

    Marchetti, G. L., Hillar, C. J., Kragic, D., and Sanborn, S. Harmonics of learning: Universal fourier features emerge in invariant networks. In The Thirty Seventh Annual Conference on Learning Theory, pp.\ 3775--3797. PMLR, 2024

    2024

  30. [38]

    Do you even need attention? a stack of feed-forward layers does surprisingly well on imagenet

    Melas-Kyriazi, L. Do you even need attention? a stack of feed-forward layers does surprisingly well on imagenet. arXiv preprint arXiv:2105.02723, 2021

    2021 arXiv

  31. [39]

    Naturally occurring equivariance in neural networks

    Olah, C., Cammarata, N., Voss, C., Schubert, L., and Goh, G. Naturally occurring equivariance in neural networks. Distill, 2020. doi:10.23915/distill.00024.004. https://distill.pub/2020/circuits/equivariance

    2020 doi

  32. [40]

    Pytorch: An imperative style, high-performance deep learning library

    Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S. Pytorch: An impera...

    2019

  33. [41]

    Improving equivariant model training via constraint relaxation

    Pertigkiozoglou, S., Chatzipantazis, E., Trivedi, S., and Daniilidis, K. Improving equivariant model training via constraint relaxation. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  34. [42]

    Do imagenet classifiers generalize to imagenet? CoRR, abs/1902.10811, 2019

    Recht, B., Roelofs, R., Schmidt, L., and Shankar, V. Do imagenet classifiers generalize to imagenet? CoRR, abs/1902.10811, 2019. URL http://arxiv.org/abs/1902.10811

    1902 arXiv

  35. [43]

    Attentive group equivariant convolutional networks

    Romero, D., Bekkers, E., Tomczak, J., and Hoogendoorn, M. Attentive group equivariant convolutional networks. In International Conference on Machine Learning, pp.\ 8188--8199. PMLR, 2020

    2020

  36. [44]

    and Kroon, R

    Roos, L. and Kroon, R. S. On fairly comparing group equivariant networks. In preprintX 2024 Workshop on Geometry-grounded Representation Learning and Generative Modeling, 2024

    2024

  37. [45]

    Imagenet large scale visual recognition challenge

    Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115: 0 211--252, 2015

    2015

  38. [46]

    Linear representations of finite groups

    Serre, J.-P. Linear representations of finite groups. Springer, 1977

    1977

  39. [47]

    K., Greff, K., and Schmidhuber, J

    Srivastava, R. K., Greff, K., and Schmidhuber, J. Training very deep networks. Advances in neural information processing systems, 28, 2015

    2015

  40. [48]

    The bitter lesson

    Sutton, R. The bitter lesson. Incomplete Ideas (blog), 2019

    2019

  41. [49]

    O., Houlsby, N., Kolesnikov, A., Beyer, L., Zhai, X., Unterthiner, T., Yung, J., Steiner, A., Keysers, D., Uszkoreit, J., et al

    Tolstikhin, I. O., Houlsby, N., Kolesnikov, A., Beyer, L., Zhai, X., Unterthiner, T., Yung, J., Steiner, A., Keysers, D., Uszkoreit, J., et al. Mlp-mixer: An all-mlp architecture for vision. Advances in neural information processing systems, 34: 0 24261--24272, 2021

    2021

  42. [50]

    Going deeper with image transformers, 2021

    Touvron, H., Cord, M., Sablayrolles, A., Synnaeve, G., and Jégou, H. Going deeper with image transformers, 2021. URL https://arxiv.org/abs/2103.17239

    2021 arXiv

  43. [51]

    Deit iii: Revenge of the vit

    Touvron, H., Cord, M., and J \'e gou, H. Deit iii: Revenge of the vit. In European conference on computer vision, pp.\ 516--533. Springer, 2022

    2022

  44. [52]

    Resmlp: Feedforward networks for image classification with data-efficient training

    Touvron, H., Bojanowski, P., Caron, M., Cord, M., El-Nouby, A., Grave, E., Izacard, G., Joulin, A., Synnaeve, G., Verbeek, J., and Jégou, H. Resmlp: Feedforward networks for image classification with data-efficient training. IEEE Transactions on Pattern Analysis and Machine In...

    2023

  45. [53]

    N., Kaiser, ., and Polosukhin, I

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, ., and Polosukhin, I. Attention is All you Need . In Adv. Neural Inform. Process. Syst., 2017

    2017

  46. [54]

    and Cesa, G

    Weiler, M. and Cesa, G. General E(2) -equivariant steerable CNN s. In Adv. Neural Inform. Process. Syst., 2019. URL https://proceedings.neurips.cc/paper/2019/file/45d6637b718d0f24a237069fe41b0db4-Paper.pdf

    2019

  47. [55]

    Equivariant and Coordinate Independent Convolutional Networks

    Weiler, M., Forré, P., Verlinde, E., and Welling, M. Equivariant and Coordinate Independent Convolutional Networks . 2023

    2023

  48. [56]

    Pytorch image models

    Wightman, R. Pytorch image models. https://github.com/rwightman/pytorch-image-models, 2019

    2019

  49. [57]

    Arithmetic complexity of computations, volume 33

    Winograd, S. Arithmetic complexity of computations, volume 33. Siam, 1980

    1980

  50. [58]

    and Shawe-Taylor , J

    Wood, J. and Shawe-Taylor , J. Representation theory and invariant neural networks. Discrete Applied Mathematics, 69 0 (1-2): 0 33--60, August 1996. ISSN 0166218X. doi:10/c3qmr6

    1996

  51. [59]

    e (2) -equivariant vision transformer

    Xu, R., Yang, K., Liu, K., and He, F. e (2) -equivariant vision transformer. In Uncertainty in Artificial Intelligence, pp.\ 2356--2366. PMLR, 2023

    2023

  52. [60]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.