Pith. sign in

REVIEW 3 major objections 5 minor 44 references

Quotient Network -- A Network Similar to ResNet but Learning Quotients

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Learning ratios instead of differences improves on ResNet

desk verdict A genuinely different block parameterization, but the headline claim is undercut by a confound: the gains could come from the activation swap rather than the quotient. read the letter →

arxiv 2506.00992 v1 pith:AKQOIQLK submitted 2025-06-01 cs.CV cs.AI

Peng Hui , Jiamuyang Zhao , Changxin Li , Qingzhen Zhu This is my paper
classification cs.CVcs.AI
keywords quotientnetworkresiduallearningmultiplicativegatingactivationfunctiondesignCIFAR-10CIFAR-100SVHNdeepconvolutionalnetworks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that residual learning in ResNet addresses the wrong quantity: the absolute difference $H(x)-x$ between new and old features is sensitive to feature scale and does not carry clear semantic meaning. The proposed quotient network instead learns the ratio $H(x)/x$ and reconstructs features by multiplication, $H(x)=F(x)\times x$. The authors give design rules, including a positive, globally differentiable activation that passes through $(0,1)$ and placement of that activation in the network head and channel-changing shortcuts, and report that on CIFAR10, CIFAR100, and SVHN the quotient network stably outperforms ResNet with the same layer count and parameter count. The point of the claim is that relative change is a more natural learning target than absolute change.

What carries the argument

The quotient module replaces the residual addition $H=F+x$ with the multiplication $H=F(x)\times x$, and it uses the activation $\mathrm{activate}(x)=\operatorname{sigmoid}(x-\ln(\alpha-1))\cdot\alpha$ before that multiplication. This function is positive, bounded on $(0,\alpha)$, globally differentiable, and passes through the point $(0,1)$, which lets the layer approximate an identity mapping when weights are small and prevents multiplicative explosion of feature magnitudes. The same activation is also placed in the first convolution of the network and in shortcut convolutions that change channel counts, where zero-padding and ReLU would otherwise corrupt multiplicative learning.

What would settle it

Train two otherwise identical networks on CIFAR10, one residual with $H=F(x)+x$ and one quotient with $H=F(x)\times x$, using the same sigmoid-style activation of Equation 1 in the head and shortcuts for both. If the residual network with the new activation matches or beats the quotient network's accuracy, the paper's attribution of gains to quotient learning is falsified; if the quotient network still wins, the claim survives.

Watch

Extended reading notes

Core claim

The central claim is that a network block can learn the quotient of its target and current features instead of their difference, and that doing so makes deep networks easier to train and more accurate. Formally, where ResNet computes $H(x)=F(x)+x$ with $F(x)$ learned as a residual, the quotient network computes $H(x)=F(x)\times x$ with $F(x)$ learned as a quotient, using the activation of Equation 1 as the final nonlinearity so the multiplier stays positive, bounded, and close to 1 when the pre-activation is near 0. On CIFAR10, CIFAR100, and SVHN, with the scale parameter $\alpha$ tuned per depth ($1.8$, $1.7$, and $1.5$ for 44-, 56-, and 110-layer networks), the quotient versions of ResNet are reported to be consistently more accurate than the corresponding ResNets without adding any parameters. The paper also reports that the learned quotient feature maps are visually clearer than residual feature maps, supporting the motivation that quotients carry independent meaning.

Load-bearing premise

The load-bearing premise is that the accuracy gains come specifically from learning quotients rather than from the accompanying switch in activation function and shortcut design; no experiment isolates the division operation while keeping the activation identical.

Editorial extensions

If this is right

  • On CIFAR10, the 44-layer quotient network reaches 92.78% versus 92.61% for ResNet-44, and the 56-layer quotient network reaches 93.1%, which is above the 93.02% of ResNet-110.
  • The pattern holds on CIFAR100 and SVHN, where the 44-layer quotient network already outperforms all tested ResNet depths, e.g., 73.25% versus 72.66% for ResNet-44 on CIFAR100.
  • The gains come without increasing parameter count; the extra cost is only pointwise multiplication and the sigmoid-style activation, adding about three percent to training time for a 56-layer CIFAR10 model.
  • The design rules are stated as a general recipe: any ResNet can be converted into a quotient network by replacing residual blocks and adjusting activation placement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The experiments do not isolate the quotient operation from the activation change, because the quotient network also swaps ReLU for the sigmoid-style activation in the head and shortcut paths; a fair test would hold the activation fixed and vary only addition versus multiplication.
  • If the quotient mechanism is what drives the gains, the same transformation could apply to other residual-style architectures, including transformer blocks where $H = X + \mathrm{Attention}(X)$ could become a multiplicative update with a learned positive gate; this is not tested in the paper.
  • The optimal $\alpha$ decreasing with depth hints at a scaling rule worth testing: deeper layers may need a smaller multiplier range to counteract growing feature magnitudes, and $\alpha$ might be scheduled rather than tuned per architecture.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a 'quotient network' in which a residual block computes H(x) = F(x) * x instead of ResNet's H(x) = F(x) + x, with F(x) learned as a quotient. To make multiplication-based blocks trainable, the authors introduce design rules: a bounded, positive, globally differentiable activation function (Eq. 1) that passes through (0,1), use of this activation in the network head and in channel-increasing shortcut convolutions, and 3x3 convolutional projections for channel changes. Experiments on CIFAR-10, CIFAR-100, and SVHN with 44/56/110-layer variants report small consistent accuracy gains over ResNet with the same parameter count, plus visualizations of intermediate feature maps. The paper also provides an appendix ablation of the design-rule choices and an analysis of the increased computational cost.

Significance. The idea of learning a quotient rather than a difference is conceptually interesting and, if the gains are real and attributable to the quotient mechanism, it would be a meaningful architectural contribution. The paper includes useful elements: an explicit design-rule ablation in Appendix A, an honest account of the added computation in Section 3.5, and a genuine out-of-sample transfer of the alpha hyperparameter from CIFAR-10 to CIFAR-100 and SVHN. However, the central empirical claim that quotient learning itself yields the improvements is not yet supported because the comparison changes more than the residual/quotient operation.

major comments (3)
  1. [Sections 4.3 and Tables 1-3] The comparison of the quotient network to ResNet changes two factors simultaneously: the combine operation (addition vs multiplication) and the activation function. The quotient network uses Eq. 1 not only in the final convolution of each block (Figure 1), but also in the network head (Figure 2) and in channel-increasing shortcut convolutions (Figure 3), whereas the ResNet baseline uses ReLU. No experiment holds the activation fixed and varies only the combine operation. The appendix ablation in Table 8 shows that placement of Eq. 1 inside quotient networks changes accuracy by about one percentage point (e.g., sigmoid 90.67 with no placement vs 91.72 with head+shortcut placement), so the activation change is a plausible alternative explanation for the gains in Tables 1-3. I request an ablation that applies Eq. 1 to a ResNet while keeping addition (i.e., swap only the activation in the head, shortcuts, and block output), or a quotient network that uses the same activation as the corresponding ResNet, to isolate the contribution of multiplication. As written, the abstract's claim that improvements come from quotient learning is not supported.
  2. [Sections 4.3 and Tables 1-3] The claimed 'stable' and 'considerable' improvements are not backed by sufficient statistical evidence. The paper reports mean ± std but never states the number of runs, whether the variation is over random seeds, data splits, or both, and it does not report any significance test or confidence interval. Several key comparisons have overlapping standard-deviation ranges, for example CIFAR-10-44 (92.78±0.25 vs 92.61±0.33) and CIFAR-100-44 (73.25±0.27 vs 72.66±1.24). Given the small margins, the authors should state the number of runs and either report paired significance tests (e.g., matched-seed t-tests) or show that the improvement holds for every run. Otherwise the wording 'prove that this network can stably achieve considerable improvements' overstates the evidence.
  3. [Section 3.4] The text describing the shortcut projection is ambiguous and potentially misleading. The sentence 'Unlike the original paper shortcuts, which add new channels with all zeros when the number of channels increases, we use a 3x3 convolution with stride 2 to increase the number of channels' appears to describe the base network used for both ResNet and the quotient network, but it could be read as a change introduced specifically for the quotient network. If the ResNet baseline uses the same 3x3 convolutional projections, please say so explicitly; if it does not, then the comparison in Tables 1-3 is confounded by a shortcut-type difference and the parameter-count claim needs reconciliation.
minor comments (5)
  1. [Abstract and Section 1] The abstract and introduction use the word 'perfectly solves' and 'prove'; given the experimental margins and the confounds discussed above, these claims should be softened to 'addresses' and 'suggests' or 'indicates'.
  2. [Throughout] Please replace 'Formula 1' with 'Equation (1)' and consistently number equations; the manuscript currently uses only a single unnumbered equation.
  3. [Section 4.3] Please state explicitly how many independent runs produced the mean ± std values in Tables 1-3, and whether the reported value is the standard deviation or the standard error of the mean.
  4. [Appendix Table 8] The ablation in Table 8 is informative, but the 'null' condition should be described more precisely: does it mean ReLU is used in the head and shortcuts, or that no activation is applied after the final convolution? Clarify to make the baseline unambiguous.
  5. [Section 3.3.1] The claim that 'useful feature representations are often positive numbers' is stated without support; a brief citation or a short justification would help.

Circularity Check

0 steps flagged · score 1.0 of 10

No circularity: the accuracy claims rest on external ResNet baselines and out-of-sample alpha transfer; the activation and shortcut confounds are an experimental-validity concern, not a circular derivation.

full rationale

The paper makes no derivation that reduces to its own inputs. The central claim, that a quotient block H(x)=F(x)*x trained with the Eq. 1 activation outperforms ResNet's H(x)=F(x)+x, is supported purely by measured accuracy comparisons against ResNet models trained with the same protocol (Section 4), i.e., an external benchmark. The activation hyperparameter alpha is tuned per depth on CIFAR10 (Section 4.3) and then transferred unchanged to CIFAR100 and SVHN (Section 4.4), so those results are genuinely out-of-sample, and alpha does not encode the target accuracies. No parameter is fitted to a subset and then 'predicted' on the same subset: even the CIFAR10 comparisons use different depths (44, 56, 110) than the 20- and 32-layer networks used to motivate the design rules in Section A, and the main claim is comparative rather than a predicted value. The reference list contains no papers by the present authors, so no self-citation chain is load-bearing, and no uniqueness theorem is imported from prior work. The design rules in Section 3.3 are explicitly presented as empirical heuristics derived from failed attempts ('After many attempts and failures...'), not as consequences of a theory, so appendix validation of those rules is weak but not circular support. The genuine weakness is a missing control: the quotient network simultaneously replaces ReLU with the Eq. 1 activation in the head, channel-increasing shortcuts, and block outputs (Sections 3.3.2–3.3.3 and Figure 1), and it also adopts convolutional projections where the ResNet baseline uses zero-padded shortcuts (Section 3.4), so the measured gains are not unambiguously attributable to the quotient operation itself. That is a confound that threatens the attribution of the improvement, but it is an experimental-validity and correctness concern, not circularity under any of the enumerated patterns.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

No new particles, mediators, forces, or conserved quantities are introduced. The only new entity is the architectural module itself, which is a construction rather than a physical postulate. The free parameter alpha is fitted per network depth and is load-bearing for training stability and performance.

free parameters (1)
  • alpha (activation upper bound) = 1.8 (44-layer), 1.7 (56-layer), 1.5 (110-layer)
    Controls the upper bound of the sigmoid-style activation in Eq. 1. Tuned per network depth on CIFAR10, then transferred unchanged to CIFAR100 and SVHN. Not theoretically derived.
assumptions (3)
  • domain assumption Useful feature representations in CNNs are typically positive values.
    Used in Sec 3.3.1 to justify banning negative activation outputs; not empirically verified with the quotient architecture.
  • domain assumption The quotient of two feature maps is more likely than their difference to be a meaningful intermediate feature.
    Core motivation from Sec 1 and 3.2, illustrated with mass/volume and voltage/current analogies; only qualitative feature-map visualizations are offered as evidence.
  • domain assumption An activation bounded in (0, alpha), positive and passing through (0,1), keeps multiplicative updates stable and identity-preserving.
    Design principle from Sec 3.3.1; supported by ablations on 20/32 layer networks but not by a theoretical proof.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Quotient Network -- A Network Similar to ResNet but Learning Quotients." pith.science (2026). https://pith.science/paper/AKQOIQLK

@misc{pith2026250600992,
  author       = {Pith},
  title        = {Pith review of: Quotient Network -- A Network Similar to ResNet but Learning Quotients},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AKQOIQLK}},
  note         = {Machine review of arXiv:2506.00992}
}
read the original abstract

The emergence of ResNet provides a powerful tool for training extremely deep networks. The core idea behind it is to change the learning goals of the network. It no longer learns new features from scratch but learns the difference between the target and existing features. However, the difference between the two kinds of features does not have an independent and clear meaning, and the amount of learning is based on the absolute rather than the relative difference, which is sensitive to the size of existing features. We propose a new network that perfectly solves these two problems while still having the advantages of ResNet. Specifically, it chooses to learn the quotient of the target features with the existing features, so we call it the quotient network. In order to enable this network to learn successfully and achieve higher performance, we propose some design rules for this network so that it can be trained efficiently and achieve better performance than ResNet. Experiments on the CIFAR10, CIFAR100, and SVHN datasets prove that this network can stably achieve considerable improvements over ResNet by simply making tiny corresponding changes to the original ResNet network without adding new parameters.

Figures

Figures reproduced from arXiv: 2506.00992 by the authors.

Figure 1
Figure 1. The residual module(left) and the quotient module(right) [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Convolution processing before stacking quotient modules [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. The residual module(left) and the quotient module(right) when changing the number of [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: The middle feature maps when the input image is a frog. The left is for the quotient network, [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: The middle feature maps when the input image is a bird. The left is for the quotient network, [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: The middle feature maps when the input image is a plane. The left is for the quotient [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: The middle feature maps when the input image is a dog. The left is for the quotient network, [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: The middle feature maps when the input image is a ship. The left is for the quotient network, [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: The middle feature maps when the input image is a horse. The left is for the quotient [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

44 extracted references · 15 canonical work pages

  1. [1]

    Imagenet classification with deep convolutional neural networks,

    A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,”Advances in neural information processing systems, vol. 25, 2012

    2012

  2. [2]

    You only look once: Unified, real-time object detection,

    J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 779–788

    work page 2016

  3. [3]

    Fast r-cnn,

    R. Girshick, “Fast r-cnn,” inProceedings of the IEEE international conference on computer vision, 2015, pp. 1440–1448

    2015

  4. [4]

    U-net: Convolutional networks for biomedical image segmenta- tion,

    O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmenta- tion,” inMedical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18. Springer, 2015, pp. 234–241

    work page 2015

  5. [5]

    Very deep convolutional networks for large-scale image recognition,

    K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014

    arXiv 2014

  6. [6]

    Going deeper with convolutions,

    C. Szegedy, W. Liu, Y . Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V . Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1–9

    2015

  7. [7]

    Understanding the difficulty of training deep feedforward neural networks,

    X. Glorot and Y . Bengio, “Understanding the difficulty of training deep feedforward neural networks,” in Proceedings of the thirteenth international conference on artificial intelligence and statistics. JMLR Workshop and Conference Proceedings, 2010, pp. 249–256

    work page 2010

  8. [8]

    Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,

    K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,” inProceedings of the IEEE international conference on computer vision, 2015, pp. 1026–1034

    2015

Show all 44 references
  1. [9]

    Batch normalization: Accelerating deep network training by reducing internal covariate shift,

    S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” inInternational conference on machine learning. pmlr, 2015, pp. 448–456

    2015

  2. [10]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778

    2016

  3. [11]

    Visualizing and understanding convolutional networks,

    M. D. Zeiler and R. Fergus, “Visualizing and understanding convolutional networks,” inComputer Vision– ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I

    2014

  4. [12]

    Learning multiple layers of features from tiny images,

    A. Krizhevsky, G. Hintonet al., “Learning multiple layers of features from tiny images,” 2009

    2009

  5. [13]

    Springer, 2014, pp. 818–833

    2014

  6. [14]

    Reading digits in natural images with unsupervised feature learning,

    Y . Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, A. Y . Nget al., “Reading digits in natural images with unsupervised feature learning,” inNIPS workshop on deep learning and unsupervised feature learning, vol. 2011, no. 5. Granada, Spain, 2011, p. 7. 9

    2011

  7. [15]

    Squeeze-and-excitation networks,

    J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 7132–7141

    2018

  8. [16]

    Cbam: Convolutional block attention module,

    S. Woo, J. Park, J.-Y . Lee, and I. S. Kweon, “Cbam: Convolutional block attention module,” inProceedings of the European conference on computer vision (ECCV), 2018, pp. 3–19

    2018

  9. [17]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,”Advances in neural information processing systems, vol. 30, 2017

    2017

  10. [18]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Min- derer, G. Heigold, S. Gellyet al., “An image is worth 16x16 words: Transformers for image recognition at scale,”arXiv preprint arXiv:2010.11929, 2020

    2010 arXiv

  11. [19]

    Rethinking the inception architecture for computer vision,

    C. Szegedy, V . Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2818–2826

    2016

  12. [20]

    Inception-v4, inception-resnet and the impact of residual connections on learning,

    C. Szegedy, S. Ioffe, V . Vanhoucke, and A. Alemi, “Inception-v4, inception-resnet and the impact of residual connections on learning,” inProceedings of the AAAI conference on artificial intelligence, vol. 31, no. 1, 2017

    2017

  13. [21]

    Densely connected convolutional networks,

    G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 4700–4708

    2017

  14. [22]

    Aggregated residual transformations for deep neural networks,

    S. Xie, R. Girshick, P. Dollár, Z. Tu, and K. He, “Aggregated residual transformations for deep neural networks,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1492–1500

    2017

  15. [23]

    Feature pyramid networks for object detection,

    T.-Y . Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 2117–2125

    2017

  16. [24]

    Mobilenets: Efficient convolutional neural networks for mobile vision applications,

    A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convolutional neural networks for mobile vision applications,”arXiv preprint arXiv:1704.04861, 2017

    2017 arXiv

  17. [25]

    Shufflenet: An extremely efficient convolutional neural network for mobile devices,

    X. Zhang, X. Zhou, M. Lin, and J. Sun, “Shufflenet: An extremely efficient convolutional neural network for mobile devices,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 6848–6856

    2018

  18. [26]

    Xception: Deep learning with depthwise separable convolutions,

    F. Chollet, “Xception: Deep learning with depthwise separable convolutions,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1251–1258

    2017

  19. [27]

    Mobilenetv2: Inverted residuals and linear bottlenecks,

    M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 4510–4520

    2018

  20. [28]

    Long short-term memory,

    S. Hochreiter and J. Schmidhuber, “Long short-term memory,”Neural computation, vol. 9, no. 8, pp. 1735–1780, 1997

    1997

  21. [29]

    Learning phrase representations using rnn encoder-decoder for statistical machine translation,

    K. Cho, B. Van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y . Bengio, “Learning phrase representations using rnn encoder-decoder for statistical machine translation,”arXiv preprint arXiv:1406.1078, 2014

    2014 arXiv

  22. [30]

    Global second-order pooling convolutional networks,

    Z. Gao, J. Xie, Q. Wang, and P. Li, “Global second-order pooling convolutional networks,” inProceedings of the IEEE/CVF Conference on computer vision and pattern recognition, 2019, pp. 3024–3033

    2019

  23. [31]

    Fcanet: Frequency channel attention networks,

    Z. Qin, P. Zhang, F. Wu, and X. Li, “Fcanet: Frequency channel attention networks,” inProceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 783–792

    2021

  24. [32]

    Eca-net: Efficient channel attention for deep convolutional neural networks,

    Q. Wang, B. Wu, P. Zhu, P. Li, W. Zuo, and Q. Hu, “Eca-net: Efficient channel attention for deep convolutional neural networks,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 11 534–11 542

    2020

  25. [33]

    Bam: Bottleneck attention module,

    J. Park, S. Woo, J.-Y . Lee, and I. S. Kweon, “Bam: Bottleneck attention module,”arXiv preprint arXiv:1807.06514, 2018

    2018 arXiv

  26. [34]

    Training data-efficient image transformers & distillation through attention,

    H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. Jégou, “Training data-efficient image transformers & distillation through attention,” inInternational conference on machine learning. PMLR, 2021, pp. 10 347–10 357. 10

    2021

  27. [35]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” inProceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 10 012–10 022

    2021

  28. [36]

    Pyramid vision transformer: A versatile backbone for dense prediction without convolutions,

    W. Wang, E. Xie, X. Li, D.-P. Fan, K. Song, D. Liang, T. Lu, P. Luo, and L. Shao, “Pyramid vision transformer: A versatile backbone for dense prediction without convolutions,” inProceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 568–578

    2021

  29. [37]

    End-to-end object detection with transformers,

    N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” inEuropean conference on computer vision. Springer, 2020, pp. 213–229

    2020

  30. [38]

    Deformable detr: Deformable transformers for end-to-end object detection,

    X. Zhu, W. Su, L. Lu, B. Li, X. Wang, and J. Dai, “Deformable detr: Deformable transformers for end-to-end object detection,”arXiv preprint arXiv:2010.04159, 2020

    2010 arXiv

  31. [39]

    Exploring plain vision transformer backbones for object detection,

    Y . Li, H. Mao, R. Girshick, and K. He, “Exploring plain vision transformer backbones for object detection,” inEuropean Conference on Computer Vision. Springer, 2022, pp. 280–296

    2022

  32. [40]

    Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers,

    S. Zheng, J. Lu, H. Zhao, X. Zhu, Z. Luo, Y . Wang, Y . Fu, J. Feng, T. Xiang, P. H. Torret al., “Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2...

    2021

  33. [41]

    Transunet: Transformers make strong encoders for medical image segmentation,

    J. Chen, Y . Lu, Q. Yu, X. Luo, E. Adeli, Y . Wang, L. Lu, A. L. Yuille, and Y . Zhou, “Transunet: Transformers make strong encoders for medical image segmentation,”arXiv preprint arXiv:2102.04306, 2021

    2021 arXiv

  34. [42]

    Segformer: Simple and efficient design for semantic segmentation with transformers,

    E. Xie, W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo, “Segformer: Simple and efficient design for semantic segmentation with transformers,”Advances in neural information processing systems, vol. 34, pp. 12 077–12 090, 2021

    2021

  35. [43]

    Rectified linear units improve restricted boltzmann machines,

    V . Nair and G. E. Hinton, “Rectified linear units improve restricted boltzmann machines,” inProceedings of the 27th international conference on machine learning (ICML-10), 2010, pp. 807–814. Appendix / supplemental material In the Appendix, we first provide validation experim...

    2010

  36. [44]

    Guidelines: • The answer NA means that the paper does not involve crowdsourcing nor research with human subjects

    Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.