Pith. sign in

REVIEW 3 major objections 6 minor 53 references

STEAM: Squeeze and Transform Enhanced Attention Module

T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read STEAM is a constant-parameter attention module that models channel and spatial attention as graph message passing, adding only 320 parameters and 3.57e-3 GFLOPs to ResNet-50 while improving ImageNet Top-1 accuracy by about two percentage…

desk verdict STEAM is a neat constant-parameter attention module with a plausible graph-attention design, but its claimed edge over GCT/MCA rests on single-seed margins that are within ImageNet run-to-run noise. read the letter →

arxiv 2412.09023 v3 pith:HN2Y2YZY submitted 2024-12-12 cs.CV

Rishabh Sabharwal , Ram Samarth B B , Parikshit Singh Rathore , Punit Rathore This is my paper · ORCID
classification cs.CV
keywords attentionmechanismgraphchannelspatialconstant-parametermoduleOutputGuidedPoolingimageclassificationobjectdetection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

STEAM is an attention module for convolutional networks that models both channel and spatial attention as message passing on two small graphs. Its parameter count depends only on a hidden dimension $d$, not on the number of channels or spatial positions, so it can be dropped into ResNets and lightweight networks at negligible cost. On ImageNet-1K, the authors report a 2% Top-1 accuracy gain over ResNet-50 with only 320 added parameters and $3.57\times 10^{-3}$ added GFLOPs, and they report consistent gains in object detection and instance segmentation on COCO. The point for a reader is that a graph-based, constant-parameter module may reach the accuracy of heavier attention blocks such as SE and CBAM while using a tiny fraction of their added computation.

What carries the argument

The load-bearing object is a pair of small graphs, a channel graph $G_c$ and a spatial graph $G_s$, together with multi-head scaled dot-product graph attention. For the channel graph, each channel is a node connected to its two immediate neighbors in a cycle, initialized by global average pooling. For the spatial graph, Output Guided Pooling (OGP) collapses any feature map to a fixed $m\times m$ spatial map ($7\times 7$ in ResNets), each cell is a node connected to adjacent cells, and one edge is randomly dropped per central node during training to counter over-smoothing. Because keys and queries are computed from single-dimensional node features with hidden dimension $d$, the module's parameter count ($8d$ per unit) never depends on $C$, $H$, or $W$; that constant-parameter property is what lets STEAM add representation power at a cost that is essentially independent of the backbone's width.

What would settle it

Train ResNet-50 with and without STEAM on the full ImageNet-1K using at least three random seeds under the paper's exact schedule, and compare the Top-1 distributions against ECA and GCT; if the reported 0.15-0.20 point gaps over those baselines fall inside the seed-to-seed spread, the claimed advantage is not established.

Watch

Extended reading notes

Core claim

The central claim is that both channel and spatial attention can be modeled as graph attention and fused in one module without any growth in parameters as the backbone's channel count grows. Each channel is a node in a cyclic degree-2 channel graph whose features come from global average pooling; each spatial location in a fixed $7\times 7$ grid is a node in a spatial graph built by the novel Output Guided Pooling. Multi-head scaled dot-product attention over these graphs produces channel scores and spatial scores, and a channel-first sequential arrangement with a tanh between the two stages performs best. Integrated before residual connections, one STEAM unit costs $8d$ parameters, so five units add 320 parameters and $3.57\times 10^{-3}$ GFLOPs to ResNet-50 while improving Top-1 accuracy from 75.22 to 77.20. The paper further claims this beats the leading lightweight attention modules ECA, GCT, and MCA on image classification and on COCO detection and segmentation while adding roughly one-third the GFLOPs of ECA or GCT; the authors note one exception where GCT edges ahead by 0.1 AP on instance segmentation with ResNet-101.

Load-bearing premise

The reported gains depend on hyperparameters tuned on a 100-class ImageNet subset (hidden size 8, four heads, degree-2 channel graph, 7x7 spatial grid, edge drop, and unit counts [1,1,2,1]), and the paper gives no error bars, so run-to-run variance could erase the small margins over ECA and GCT on the full benchmarks.

Editorial extensions

If this is right

  • Adding five STEAM units to ResNet-50 raises Top-1 accuracy on ImageNet by about two percentage points while adding only 320 parameters and $3.57\times 10^{-3}$ GFLOPs.
  • On ResNet-18, ResNet-101, and ShuffleNet-V2, STEAM improves Top-1 accuracy over the strongest prior module GCT by 0.30, 0.30, and 0.27 points respectively, and beats the previous best MCA by 0.15 and 0.21 points on ResNet-50 and ResNet-101.
  • On COCO, STEAM improves AP over the backbone across Faster R-CNN, Mask R-CNN, and RetinaNet with both ResNet-50 and ResNet-101, with the largest reported gain being 2.2 AP over RetinaNet with ResNet-50.
  • The added GFLOPs of a STEAM insertion is about one-third that of ECA and GCT on ResNet-50, so the accuracy-computation frontier shifts toward constant-parameter attention.
  • Sequential channel-then-spatial attention with a tanh between stages outperforms parallel fusion, and a degree-2 cyclic channel graph outperforms degree-4, which gives a concrete design rule for graph-based attention.
  • The authors note one exception: on instance segmentation with ResNet-101, GCT surpasses STEAM by 0.1 AP, so the advantage over GCT is not universal across all tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's explicit experiments, the constant-parameter property makes STEAM-like graph attention a candidate for very wide or high-resolution backbones where SE and CBAM overhead grows quickly; the paper does not test this directly.
  • OGP appears to be transferable: the authors show that plugging OGP into CBAM's spatial branch improves Top-1 by 0.22 points on ImageNet-100, which suggests other spatial-attention modules might benefit from the same fixed-size spatial pooling idea.
  • The fixed degree-2 channel topology discards long-range channel relationships by design; a learnable or adaptive graph could improve further, but the paper's results indicate that local channel neighborhoods already capture most of the benefit.
  • Because all hyperparameters were tuned on a 100-class subset of ImageNet, a full-ImageNet hyperparameter search could shift the optimal hidden size, number of heads, edge-drop choice, or unit placement, and with it the reported margins.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. STEAM proposes a combined channel-spatial attention module built on graph attention. Channel Interaction Attention (CIA) treats each channel as a node in a cyclic degree-2 graph, applies multi-head scaled dot-product attention over the globally pooled channel features, and produces channel scores via sigmoid. Spatial Interaction Attention (SIA) constructs a grid graph over a fixed-size (7x7) spatial map obtained by the proposed Output Guided Pooling (OGP), applies the same multi-head graph attention with a random edge-drop regularizer, and upsamples the resulting scores. The two sub-modules are placed in sequence (channel-first) with a tanh non-linearity between them. The module has 8d parameters per unit, independent of channel count, and units are placed per stage according to ceil(Nbs/4); for ResNet-50 this yields 5 units and a total of 320 added parameters. The paper reports ImageNet-1K classification, COCO object detection, and instance segmentation results, claiming a ~2% Top-1 gain over ResNet-50 and consistent gains over SE, CBAM, ECA, GCT, and MCA at fractions of the parameter and FLOP overhead.

Significance. If the empirical results are reliable, the paper makes a useful contribution: it demonstrates that a parameter-efficient, graph-based combination of channel and spatial attention can improve accuracy over strong efficient-attention baselines, and it provides a consistent training pipeline for all compared modules, which strengthens internal validity. The ablations on module arrangement, graph degrees, unit placement, and hyperparameters are informative. However, the central claim of superiority over GCT and MCA rests on very small Top-1 margins from single training runs, and one of the reported instance-segmentation results (Table 5) actually shows GCT beating STEAM. The paper's value would be substantially increased by multiple-seed experiments with error bars and a more transparent treatment of the comparative claims.

major comments (3)
  1. [Table 3 / Image Classification on ImageNet-1K] The core claim that STEAM outperforms GCT and MCA is supported only by margins of 0.15–0.30 Top-1 points (e.g., ResNet-50: 77.20 vs 77.03 for GCT and 77.05 for MCA; ResNet-101: 78.38 vs 78.08 for GCT; ShuffleNet-V2: 67.48 vs 67.21 for GCT). These values are reported from single training runs with no seeds, error bars, or significance testing. ImageNet-1K Top-1 accuracy for a fixed architecture and schedule commonly varies by about ±0.1–0.2 points across seeds, so the observed differences between STEAM and its strongest baselines are within the range of random variation. Since the abstract and introduction explicitly claim that STEAM 'outperforms leading modules ECA and GCT', this missing statistical support is load-bearing. The authors should run at least 3–5 seeds per configuration, report mean ± standard deviation, and ideally perform a paired significance test. Without this, the reported gains over GCT and MCA cannot be considered established.
  2. [Table 5 / Instance Segmentation on MS-COCO] The claim that STEAM 'consistently outperforms' prior state-of-the-art modules is contradicted by the Mask R-CNN / ResNet-101 row of Table 5, where GCT reaches AP 37.5 while STEAM reaches AP 37.4. The text acknowledges this with 'GCT surpasses STEAM by a marginally higher 0.1% AP score', but the preceding paragraph then lists gains over the baseline, SE, CBAM, ECA, and MCA while omitting GCT, which is misleading. The paper should either present the full comparison transparently or qualify the headline claim so that it does not assert universal superiority over GCT in all tasks and backbones.
  3. [Appendix, Ablations and Implementation Details] Several influential design choices (d=8, H=4, tanh non-linearity between CIA and SIA, edge drop in SIA, OGP output size m=7, and the ceil(Nbs/4) placement rule) are selected using a 100-class ImageNet subset (the appendix reports ResNet-18 ablations on this subset), while all final ImageNet-1K and COCO results are obtained with only this single configuration. Meanwhile, the baseline modules (SE, CBAM, ECA, GCT, MCA) are used with their default hyperparameters without analogous tuning on the same subset. This gives STEAM the advantage of dataset-specific tuning and raises the possibility that the selected configuration is a peak on the subset that does not generalize to the full dataset. The authors should report results on ImageNet-1K for a small grid of neighboring configurations (e.g., d ∈ {4,8,16}, H ∈ {1,4,8}, with and without edge drop) to demonstrate that the margins over baselines are not an artifact of configuration selection, and they should explicitly state which hyperparameters were chosen on the subset.
minor comments (6)
  1. [Appendix, STEAM Configuration] The sentence 'Readers are encouraged to refer to (He et al. 2016) and (?) for the exact architectural details' contains a missing citation, and a similar '(?)' placeholder appears in the ImageNet-100 class description. These should be filled in.
  2. [Appendix, Ablations] The appendix text says results are reported in 'Tables 3-11' but the actual table numbers are 7–16. Please renumber the tables or correct the cross-references for consistency.
  3. [Equations (1) and (4)] In Equation (1) and Equation (4), the softmax is written as applied to a single scalar, but the intended operation is a softmax over the neighbor set j ∈ N_i for each node i. Please make the notation explicit, e.g., with a subscript on the softmax or a note that normalization is over N_i.
  4. [Abstract and Table 3] The phrase 'threefold reduction in GFLOPs' is ambiguous: it refers to the added GFLOPs of STEAM compared with the added GFLOPs of ECA and GCT (e.g., 3.57e-3 vs ~1.1e-2 for ResNet-50), not to total GFLOPs. Please clarify this in the abstract and main text.
  5. [Table 3] In the ResNet-50 block of Table 3, +MCA reports 4.1327 GFLOPs, which is lower than the baseline of 4.1324 GFLOPs. Adding a module should not reduce GFLOPs; this is likely a typo and should be corrected.
  6. [Proposed Method, OGP] Output Guided Pooling is described as applying global average pooling along the channel dimension and then producing a fixed-size m×m spatial map; in the appendix it is stated to have 'output size fixed to 7×7'. This is effectively adaptive average pooling. The paper should explicitly state the implementation (e.g., F.adaptive_avg_pool2d) and discuss the novelty relative to existing global and adaptive pooling modules, beyond the single CBAM comparison in Table 14.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: reported STEAM gains are held-out evaluations on ImageNet-1K and COCO, with hyperparameters tuned on a disjoint 100-class subset.

full rationale

The paper makes no derivation claim that reduces to its inputs. The central comparison (Table 3) reports Top-1 accuracy on ImageNet-1K validation for STEAM and reproduced baselines under the same training schedule; STEAM's hyperparameters (d=8, H=4, degree-2 channel graph, OGP 7x7, edge drop, unit counts [1,1,2,1]) were chosen via ablations on a 100-class ImageNet subset reported in Appendix Tables 8-16, and the full-table numbers are therefore predictions rather than fits to the reported target. The parameter count 320 = 5*(8*8) (for ResNet-50) is computed from the module definition and hidden dimension, not from accuracy outcomes, and the module's parameters are stated to be independent of channel count C even though total units depend on the backbone's stage widths. No load-bearing argument rests on a self-citation: the paper cites ECA, GAT, CBAM, and related prior work for design inspiration, but the authors cite none of their own papers. Qualitative claims such as 'first graph-based approach for both channel and spatial attention' are novelty assertions, not circular reductions. The appendix contains two unresolved placeholder citations '( ?)' for architectural details and the ImageNet-100 class selection, and the single-seed runs lack error bars; both are completeness/reproducibility concerns that belong in correctness risk, not circularity. Consequently no step satisfies the standard of being equivalent to its input by definition or by self-citation chain.

Assumptions & free parameters 7 free parameters · 5 assumptions · 1 invented entities

The central claim depends on several hand-chosen hyperparameters (d, H, node degree, unit placement rule, OGP size, edge drop, activation), all selected via ablations on a 100-class ImageNet subset. These are model-selection choices rather than fitted constants in a derivation, but they carry the burden of generalizing to ImageNet-1K and COCO. No physics-style free parameters or exotic invented entities are used; the only new component is OGP, which is a simple pooling operation.

free parameters (7)
  • hidden dimension d = 8
    Chosen via ablation on ImageNet-100 (Table 12); controls the constant parameter count of the module.
  • number of attention heads H = 4
    Chosen via ablation on ImageNet-100 (Table 12); affects the diversity of attention patterns.
  • CIA node degree = 2
    Chosen via ablation (Table 2, Ablation-b); degree 2 outperformed degree 4 in the cyclic channel graph.
  • STEAM unit placement rule = [1, 1, 2, 1] for ResNet-50
    Chosen via ablation (Table 2, Ablation-c); the rule ceil(Nbs/4) is applied per stage and may not generalize across backbones.
  • OGP output spatial size m = 7
    Fixed to the spatial dimension of the deepest stage (7x7 for ResNets); not tuned but load-bearing for spatial graph node count.
  • random edge drop in SIA = yes
    Chosen via ablation (Table 10); improves accuracy on ImageNet-100 and is applied only to the central submatrix of the spatial graph.
  • activation between CIA and SIA = Tanh
    Chosen via ablation (Table 11); Tanh outperformed ReLU, Sigmoid, and no activation on ImageNet-100.
assumptions (5)
  • domain assumption Multi-head scaled dot-product attention is an effective relational modeling mechanism.
    Invoked by citing Vaswani et al. 2017 and GAT in the proposed method section; the paper relies on this assumption without re-deriving it.
  • domain assumption Treating each channel as a node with GAP features captures useful channel dependencies.
    The channel graph Gc is built from per-channel GAP features (Section Proposed Method, CIA). The validity of this representation is assumed from prior attention work.
  • domain assumption Local graph connectivity is sufficient; deeper GNN layers would oversmooth.
    The paper avoids dense graphs citing oversmoothing (Nt and Maehara 2019; Oono and Suzuki 2019), and uses degree-2 and grid neighborhoods without further justification.
  • domain assumption GAP along channels (OGP) provides a representative fixed-size spatial map.
    The OGP description claims a fixed m x m map is obtained from any HxW input; this requires an additional pooling or resize step that is not explicitly defined, making the assumption underspecified.
  • ad hoc to paper The adaptive placement rule ceil(Nbs/4) generalizes across backbones and datasets.
    The rule is derived from ablations on ResNet-50 with ImageNet-100 (Ablation-c) and then applied to ResNet-18/101, ShuffleNet-V2, and COCO without further tuning.
invented entities (1)
  • Output Guided Pooling (OGP) independent evidence
    purpose: Reduces any spatial feature map to a fixed m x m grid so that the spatial graph has a constant number of nodes across layers and backbones.
    The paper shows that adding OGP to CBAM's spatial attention improves Top-1 accuracy by 0.22% on ImageNet-100 (Table 14), providing a falsifiable handle that other researchers can test. However, the exact implementation is ambiguous because GAP along channels preserves spatial size, and the paper does not specify the adaptive pooling step.

how reviews work

0 comments
Cite this review

Pith. "Pith review of STEAM: Squeeze and Transform Enhanced Attention Module." pith.science (2026). https://pith.science/paper/HN2Y2YZY

@misc{pith2026241209023,
  author       = {Pith},
  title        = {Pith review of: STEAM: Squeeze and Transform Enhanced Attention Module},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HN2Y2YZY}},
  note         = {Machine review of arXiv:2412.09023}
}
abstract

Channel and spatial attention mechanisms introduced in earlier work enhance the representational capabilities of deep convolutional neural networks (CNNs) but often increase parameter and computational costs. While recent approaches focus solely on efficient feature context modeling for channel attention, we aim to model both channel and spatial attention comprehensively with minimal parameters and reduced computation. Leveraging the principles of relational modeling in graphs, we introduce a constant-parameter module, \textit{STEAM: Squeeze and Transform Enhanced Attention Module}, which integrates channel and spatial attention to enhance the representation power of CNNs. To our knowledge, we are the first to propose a graph-based approach for modeling both channel and spatial attention, utilizing concepts from multi-head graph transformers. Additionally, we introduce \textit{Output Guided Pooling} (OGP), which efficiently captures spatial context to further enhance spatial attention. We extensively evaluate STEAM for large-scale image classification, object detection and instance segmentation on standard benchmark datasets. STEAM achieves a \(2\%\) increase in accuracy over the standard ResNet-50 model with only a meager increase in GFLOPs. Furthermore, STEAM outperforms the leading modules, ECA and GCT, in terms of accuracy while achieving a threefold reduction in GFLOPs. The code will be made available upon acceptance.

Figures

Figures reproduced from arXiv: 2412.09023 by the authors.

Figure 1
Figure 1. Detailed Overview of STEAM. Channel Interaction Attention (CIA) and Spatial Interaction Attention (SIA). We introduce a new Output Guided Pooling (OGP) to capture spatial context efficiently which enhances our SIA. We name our approach that integrates both CIA and SIA as STEAM: Squeeze and Transform Enhanced At￾tention Module. As evident in [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Spatial Interactional Attention Module. STEAM: Squeeze and Transform Enhanced Attention Module In this section, we introduce the final architecture. Tradi￾tional channel attention mechanisms leverage spatial rep￾resentation maps, while spatial attention mechanisms draw on global channel information, highlighting the interdepen￾dence between the two. Building on this insight, we integrate our CIA and SIA modules to e… view at source ↗
Figure 3
Figure 3. Overview of STEAM Appendix STEAM Configuration In this section, we discuss the overall configuration of our proposed method, STEAM [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: ResNet-18 integrated with STEAM 1x1 Conv, 64 3x3 Conv, 64 1x1 Conv, 256 1x1 Conv, 64 3x3 Conv, 64 1x1 Conv, 256 1x1 Conv, 64 3x3 Conv, 64 1x1 Conv, 256 STEAM [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 8
Figure 8. Figure 8: Stage c5 of ResNet-50 integrated with STEAM. [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 6
Figure 6. Figure 6: Stage c3 of ResNet-50 integrated with STEAM. [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Stage c4 of ResNet-50 integrated with STEAM. [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 9
Figure 9. Figure 9: AlexNet integrated with STEAM [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 16 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Aflalo, A.; Bagon, S.; Kashti, T.; and Eldar, Y. 2023. Deepcut: Unsupervised segmentation using graph neural networks clustering. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 32--41

    work page 2023

  4. [4]

    Bahdanau, D.; Cho, K.; and Bengio, Y. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473

    arXiv 2014

  5. [5]

    W.; Hamrick, J

    Battaglia, P. W.; Hamrick, J. B.; Bapst, V.; Sanchez-Gonzalez, A.; Zambaldi, V.; Malinowski, M.; Tacchetti, A.; Raposo, D.; Santoro, A.; Faulkner, R.; et al. 2018. Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv:1806.01261

    arXiv 2018

  6. [6]

    Brody, S.; Alon, U.; and Yahav, E. 2021. How attentive are graph attention networks? arXiv preprint arXiv:2105.14491

    arXiv 2021

  7. [7]

    Cao, Y.; Xu, J.; Lin, S.; Wei, F.; and Hu, H. 2019. Gcnet: Non-local networks meet squeeze-excitation networks and beyond. In Proceedings of the IEEE/CVF international conference on computer vision workshops, 0--0

    work page 2019

  8. [8]

    Chen, K.; Wang, J.; Pang, J.; Cao, Y.; Xiong, Y.; Li, X.; Sun, S.; Feng, W.; Liu, Z.; Xu, J.; et al. 2019. MMDetection: Open mmlab detection toolbox and benchmark. arXiv preprint arXiv:1906.07155

    arXiv 2019

Show all 53 references
  1. [9]

    Chen, Y.; Kalantidis, Y.; Li, J.; Yan, S.; and Feng, J. 2018. A\^ 2-nets: Double attention networks. Advances in neural information processing systems, 31

    2018

  2. [10]

    Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; and Fei-Fei, L. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, 248--255. Ieee

    2009

  3. [11]

    Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929

    2020 arXiv

  4. [12]

    P.; and Bresson, X

    Dwivedi, V. P.; and Bresson, X. 2020. A generalization of transformer networks to graphs. arXiv preprint arXiv:2012.09699

    2020 arXiv

  5. [13]

    Gao, Z.; Xie, J.; Wang, Q.; and Li, P. 2019. Global second-order pooling convolutional networks. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, 3024--3033

    2019

  6. [14]

    S.; Riley, P

    Gilmer, J.; Schoenholz, S. S.; Riley, P. F.; Vinyals, O.; and Dahl, G. E. 2017. Neural message passing for quantum chemistry. In International conference on machine learning, 1263--1272. PMLR

    2017

  7. [15]

    Hamilton, W.; Ying, Z.; and Leskovec, J. 2017. Inductive representation learning on large graphs. Advances in neural information processing systems, 30

    2017

  8. [16]

    Han, K.; Wang, Y.; Guo, J.; Tang, Y.; and Wu, E. 2022. Vision gnn: An image is worth graph of nodes. Advances in neural information processing systems, 35: 8291--8303

    2022

  9. [17]

    He, K.; Gkioxari, G.; Doll \'a r, P.; and Girshick, R. 2017. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, 2961--2969

    2017

  10. [18]

    He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770--778

    2016

  11. [19]

    Hu, J.; Shen, L.; Albanie, S.; Sun, G.; and Vedaldi, A. 2018. Gather-excite: Exploiting feature context in convolutional neural networks. Advances in neural information processing systems, 31

    2018

  12. [20]

    Hu, J.; Shen, L.; and Sun, G. 2018. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, 7132--7141

    2018

  13. [21]

    A.; Jia, S.; and Bruce, N

    Islam, M. A.; Jia, S.; and Bruce, N. D. 2020. How much position information do convolutional neural networks encode? arXiv preprint arXiv:2001.08248

    2020 arXiv

  14. [22]

    Jaderberg, M.; Simonyan, K.; Zisserman, A.; et al. 2015. Spatial transformer networks. Advances in neural information processing systems, 28

    2015

  15. [23]

    Jiang, Y.; Jiang, Z.; Han, L.; Huang, Z.; and Zheng, N. 2024. MCA: Moment Channel Attention Networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 2579--2588

    2024

  16. [24]

    Jiao, L.; Chen, J.; Liu, F.; Yang, S.; You, C.; Liu, X.; Li, L.; and Hou, B. 2022. Graph representation learning meets computer vision: A survey. IEEE Transactions on Artificial Intelligence, 4(1): 2--22

    2022

  17. [25]

    N.; and Welling, M

    Kipf, T. N.; and Welling, M. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907

    2016 arXiv

  18. [26]

    Kreuzer, D.; Beaini, D.; Hamilton, W.; L \'e tourneau, V.; and Tossou, P. 2021. Rethinking graph transformers with spectral attention. Advances in Neural Information Processing Systems, 34: 21618--21629

    2021

  19. [27]

    Lee, H.; Kim, H.-E.; and Nam, H. 2019. Srm: A style-based recalibration module for convolutional neural networks. In Proceedings of the IEEE/CVF International conference on computer vision, 1854--1862

    2019

  20. [28]

    Li, X.; Wang, W.; Hu, X.; and Yang, J. 2019. Selective kernel networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 510--519

    2019

  21. [29]

    Lin, T.-Y.; Doll \'a r, P.; Girshick, R.; He, K.; Hariharan, B.; and Belongie, S. 2017 a . Feature pyramid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2117--2125

    2017

  22. [30]

    Lin, T.-Y.; Goyal, P.; Girshick, R.; He, K.; and Doll \'a r, P. 2017 b . Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, 2980--2988

    2017

  23. [31]

    Lin, T.-Y.; Maire, M.; Belongie, S.; Hays, J.; Perona, P.; Ramanan, D.; Doll \'a r, P.; and Zitnick, C. L. 2014. Microsoft coco: Common objects in context. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 1...

    2014

  24. [32]

    Ma, N.; Zhang, X.; Zheng, H.-T.; and Sun, J. 2018. Shufflenet v2: Practical guidelines for efficient cnn architecture design. In Proceedings of the European conference on computer vision (ECCV), 116--131

    2018

  25. [33]

    Nt, H.; and Maehara, T. 2019. Revisiting graph neural networks: All we have is low-pass filters. arXiv preprint arXiv:1905.09550

    2019 arXiv

  26. [34]

    Oono, K.; and Suzuki, T. 2019. Graph neural networks exponentially lose expressive power for node classification. arXiv preprint arXiv:1905.10947

    2019 arXiv

  27. [35]

    Park, J.; Woo, S.; Lee, J.-Y.; and Kweon, I. S. 2018. Bam: Bottleneck attention module. arXiv preprint arXiv:1807.06514

    2018 arXiv

  28. [36]

    Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L.; et al. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32

    2019

  29. [37]

    P.; Luu, A

    Ramp \'a s ek, L.; Galkin, M.; Dwivedi, V. P.; Luu, A. T.; Wolf, G.; and Beaini, D. 2022. Recipe for a general, powerful, scalable graph transformer. Advances in Neural Information Processing Systems, 35: 14501--14515

    2022

  30. [38]

    Ren, S.; He, K.; Girshick, R.; and Sun, J. 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems, 28

    2015

  31. [39]

    T.; and Saul, L

    Roweis, S. T.; and Saul, L. K. 2000. Nonlinear dimensionality reduction by locally linear embedding. science, 290(5500): 2323--2326

    2000

  32. [40]

    squeeze and excitation

    Roy, A. G.; Navab, N.; and Wachinger, C. 2018. Recalibrating fully convolutional networks with spatial and channel “squeeze and excitation” blocks. IEEE transactions on medical imaging, 38(2): 540--549

    2018

  33. [41]

    Ruan, D.; Wang, D.; Zheng, Y.; Zheng, N.; and Zheng, M. 2021. Gaussian context transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 15129--15138

    2021

  34. [42]

    C.; Hagenbuchner, M.; and Monfardini, G

    Scarselli, F.; Gori, M.; Tsoi, A. C.; Hagenbuchner, M.; and Monfardini, G. 2008. The graph neural network model. IEEE transactions on neural networks, 20(1): 61--80

    2008

  35. [43]

    Srinivas, A.; Lin, T.-Y.; Parmar, N.; Shlens, J.; Abbeel, P.; and Vaswani, A. 2021. Bottleneck transformers for visual recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 16519--16529

    2021

  36. [44]

    N.; Kaiser, .; and Polosukhin, I

    Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention is all you need. Advances in neural information processing systems, 30

    2017

  37. [45]

    Veli c kovi \'c , P.; Cucurull, G.; Casanova, A.; Romero, A.; Lio, P.; and Bengio, Y. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903

    2017 arXiv

  38. [46]

    Wang, F.; Jiang, M.; Qian, C.; Yang, S.; Li, C.; Zhang, H.; Wang, X.; and Tang, X. 2017. Residual attention network for image classification. In Proceedings of the IEEE conference on computer vision and pattern recognition, 3156--3164

    2017

  39. [47]

    Wang, Q.; Wu, B.; Zhu, P.; Li, P.; Zuo, W.; and Hu, Q. 2020. ECA-Net: Efficient channel attention for deep convolutional neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 11534--11542

    2020

  40. [48]

    Wang, X.; Girshick, R.; Gupta, A.; and He, K. 2018. Non-local neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, 7794--7803

    2018

  41. [49]

    Woo, S.; Park, J.; Lee, J.-Y.; and Kweon, I. S. 2018. Cbam: Convolutional block attention module. In Proceedings of the European conference on computer vision (ECCV), 3--19

    2018

  42. [50]

    Xie, S.; Girshick, R.; Doll \'a r, P.; Tu, Z.; and He, K. 2017. Aggregated residual transformations for deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, 1492--1500

    2017

  43. [51]

    Zagoruyko, S.; and Komodakis, N. 2016. Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. arXiv preprint arXiv:1612.03928

    2016 arXiv

  44. [52]

    Zhang, Q.-L.; and Yang, Y.-B. 2021. Sa-net: Shuffle attention for deep convolutional neural networks. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2235--2239. IEEE

    2021

  45. [53]

    Zhang, X.; Zhou, X.; Lin, M.; and Sun, J. 2018. Shufflenet: An extremely efficient convolutional neural network for mobile devices. In Proceedings of the IEEE conference on computer vision and pattern recognition, 6848--6856

    2018

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.