Pith. sign in

REVIEW 3 major objections 6 minor 50 references

Attentive Normalization

T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Attentive Normalization replaces the single learned affine transformation inside a feature-normalization layer with an instance-specific mixture of K such transformations, and the paper reports consistent accuracy gains in ImageNet-1000…

desk verdict Attentive Normalization is a clean, useful extension of normalization with attention-weighted mixtures of affine transforms, but the strength of the empirical claim is under-supported by single-run, validation-tuned numbers. read the letter →

arxiv 1908.01259 v3 pith:XFKFT6SJ submitted 2019-08-04 cs.CV

Xilai Li , Wei Sun , Tianfu Wu This is my paper · ORCID
classification cs.CV
keywords attentivenormalizationfeaturechannel-wiseattentionmixtureofaffinetransformationsrelativestandarddeviationimageclassificationobjectdetectioninstancesegmentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Attentive Normalization (AN) is a proposed replacement for the affine-transformation step inside feature normalization layers. Instead of learning one scale and shift per channel, AN learns K affine transformations per channel and produces the final transform as their weighted sum, with the weights computed per image from a channel-level variability statistic. The paper presents this as a single lightweight module that integrates feature normalization and channel-wise attention, and reports that it beats the vanilla batch/group normalization baselines and the widely used Squeeze-and-Excitation module across four architectures in ImageNet-1000 and MS-COCO. If right, this means a modest change to a standard building block, not a new architecture, can yield consistent accuracy gains in classification, detection, and instance segmentation.

What carries the argument

The carrying object is the mixture-of-affine re-calibration identity $\tilde{x}_i = \sum_k \lambda_{n,k}(\gamma_{k,c}\hat{x}_i+\beta_{k,c})$, which turns the static per-channel scale-and-shift into a weighted combination of K such maps. The per-instance weights $\lambda_{n,k}$ are the attention mechanism: they are computed from the coefficient of variation of each channel (RSD) rather than from the channel mean that SE-style attention uses, passed through a light network (one FC layer plus batch normalization and hsigmoid in the best variant). The stage-dependent hyperparameter K (typically 10, 10, 20, 20 across four stages) controls the size of the mixture. What the machinery does is decouple population-level knowledge (the $\gamma,\beta$ mixture components, shared across images) from individual-level knowledge (the attention weights, computed fresh from each input), letting the normalization layer adapt at test time instead of freezing its affine transform.

What would settle it

Train the exact AN recipe with the RSD descriptor replaced by a constant input, or by the SE mean descriptor, under identical hyperparameters; if top-1 accuracy stays within the reported margin, the RSD-driven attention is not what is doing the work. A sharper test is to shuffle the per-instance weights $\lambda_{n,k}$ across images at inference time; if accuracy is unchanged, the instance-specific component of the mixture is not load-bearing.

Watch

Extended reading notes

Core claim

AN keeps the standardization step of a chosen normalization backbone (e.g., batch or group normalization) untouched and replaces the channel-wise affine transform with an instance-specific mixture of K affine maps. The re-calibrated response is $\tilde{x}_i = \sum_{k=1}^K \lambda_{n,k}(\gamma_{k,i_C}\hat{x}_i+\beta_{k,i_C})$, where the attention weight $\lambda_{n,k}$ for instance $n$ and component $k$ is produced by a squeeze-style subnetwork whose input descriptor is the relative standard deviation (RSD), $\sigma_c/\mu_c$, of each channel; in the authors' best configuration this is a fully connected layer followed by batch normalization and a hard-sigmoid activation. On ImageNet-1000, with BN as the backbone, AN improves top-1 accuracy over the vanilla network by 0.5 to 2.7 percentage points across ResNet-34/50/101, DenseNet-121, MobileNetV2, and two AOGNet sizes, under two training regimes; on MS-COCO it improves bounding-box AP by up to 1.8 points and mask AP by up to 2.2 points. The paper's central contention is that this one integration gives a stronger re-calibration path than either vanilla affine normalization or a separate attention module such as SE, at a parameter cost far below adding SE.

Load-bearing premise

The claim stands on the assumption that a channel's relative standard deviation, run through one fully connected layer and a hard-sigmoid nonlinearity, is a reliable per-image signal for how that channel should be rescaled; if that statistic carries no useful information, the attention weights are noise and the reported gains should not appear.

Editorial extensions

If this is right

  • In ImageNet-1000, switching the affine layer of BN to AN raises top-1 accuracy by 0.5 to 2.7 absolute points across ResNet34/50/101, DenseNet121, MobileNetV2, and two AOGNet variants, in both the vanilla and state-of-the-art training setups.
  • In MS-COCO Mask R-CNN, an AN backbone beats a frozen-BN backbone by up to 1.8 absolute box AP and 1.6 mask AP; adding AN with GN to the box head beats GN and SN baselines by up to 1.4 box AP and 2.2 mask AP.
  • AN is a parameter-cheap replacement for SE: for ResNet50 it adds about 0.2M parameters while SE adds about 2.5M, and AN matches or exceeds SE's accuracy when both are placed after the same BN position.
  • AN transfers across tasks: when a backbone trained with BN is fine-tuned for detection, freezing the standardization and the mixture coefficients but fine-tuning the attention subnetwork is enough to obtain the reported gains.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One testable extension is to build AN on layer or instance normalization and measure whether the same RSD-driven mixture gains appear; the paper's formulation is backbone-agnostic, but the evidence only covers BN and GN.
  • Because the attention weights are computed from unstandardized features, AN may be implicitly reading global image statistics such as contrast; a domain-shift or image-corruption benchmark would show whether instance-specific re-calibration is most valuable when test images drift from training.
  • The mixture-of-affine form is compatible with conditioning; adding a class or style signal to the attention weights would connect this image-classification method to conditional normalization used in generation.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes Attentive Normalization (AN), a module that replaces the single affine transformation in feature normalization with a mixture of K affine transformations whose per-instance weights are predicted by a lightweight channel-attention sub-network. The standardization step of a base normalization (e.g., BN or GN) is kept unchanged. The authors evaluate AN with four CNN architectures (ResNets, DenseNets, MobileNetV2, AOGNets) on ImageNet-1000 classification and on MS-COCO object detection/instance segmentation with Mask R-CNN and Cascade Mask R-CNN. They report consistent top-1 accuracy gains of 0.5%–2.7% over BN in ImageNet and AP gains up to 1.8% (box) and 2.2% (mask) on COCO, and they position AN as a stronger and more parameter-efficient alternative to the Squeeze-and-Excitation module. The paper includes an ablation study over attention inputs, attention network choices, activation functions, and mixture sizes K, and releases code for both benchmarks.

Significance. If the reported gains are reproducible, AN is a practically useful and conceptually clean integration of feature normalization and channel attention: it adds only a small number of parameters, keeps FLOPs essentially unchanged, and applies uniformly to multiple architectures and tasks. The paper's strengths are the breadth of architectures and benchmarks, the fact that the BN and SE baselines used in most comparisons are retrained under the same settings as AN, the inclusion of an ablation study, and the public release of code. The central claim is empirical rather than theoretical, and its reliability currently rests on single-run point estimates; the significance of the contribution therefore depends on whether the reported consistency across configurations survives repeated training runs and a more controlled comparison with GN/SN baselines.

major comments (3)
  1. [Sec. 4.2, Table 2; Sec. 4.3, Tables 4–5] The central claim of the abstract—consistent absolute top-1 gains between 0.5% and 2.7% in ImageNet and up to 1.8%/2.2% AP in COCO—is supported only by single-run point estimates. No seeds, repeated trials, or error bars are reported. The smallest gains (AOGNet40M: 0.51% and 0.57% top-1 in Table 2; 0.7% and 0.5% AP in Table 4) are within the range that typical run-to-run variation can produce in large-scale training under identical settings. To establish the claimed consistency, please report mean and standard deviation over at least three independent runs for the key configurations, especially those with gains below 1%, or otherwise provide evidence that the noise floor is smaller than the smallest claimed improvement.
  2. [Sec. 4.1, Table 1; Sec. 4.2, Table 2] The design choices for AN—the channel statistic (mean-only, mean+std, or RSD), the attention network variant (A1 vs. A2), the activation function (hsigmoid vs. others), and the per-stage mixture size K—were selected by evaluating on the ImageNet validation set (Table 1, plus the 'small experiments (a few epochs)' mentioned in Sec. 4.1), and the same validation set is then used to report the headline results in Table 2. This selection-on-the-evaluation-set procedure makes the lower end of the claimed 0.5%–2.7% range depend on hyperparameters that could be overfit to that split. Please demonstrate robustness by evaluating the selected configuration on a held-out split or an independent test set, or by reporting the full set of ablated configurations with their final validation numbers and training epochs, so readers can assess the sensitivity of the headline gains.
  3. [Sec. 4.2, Table 2; Sec. 4.3, Table 4] The comparisons against Group Normalization and Switchable Normalization are not apples-to-apples: the GN and SN rows are marked as not trained by the authors and their numbers are taken from the original papers, which used different training schedules, data augmentation, batch sizes, and possibly different pre-training protocols than the retrained BN and AN models. Consequently, the claims that AN 'outperforms GN and SN by 1.93% and 0.83%' (Sec. 4.2) and that AN w/ GN is better than GN in COCO (Sec. 4.3) may be confounded by training-setup differences rather than by the normalization module itself. Please retrain GN and SN under the exact same settings as AN (including the same backbone pre-training for COCO), or restrict the claims to the baselines that were retrained by the authors.
minor comments (6)
  1. [Sec. 4 (Data section)] The text says evaluation is on 'val2107'; this should be 'val2017'.
  2. [Table 1] The rows of Table 1 are not labeled with column headers, making it difficult to determine which design choice is varied in each row. Please add explicit column headers for the attention input, attention network variant, activation function, and K configuration.
  3. [Sec. 3.3, Eq. (11)] The notation 'Choice 1' and 'Choice 2' for A1 and A2 is used inconsistently in Sec. 4.3 (where 'Choice 1, A1(·)' appears). Please unify the notation and define A1 and A2 explicitly before first use.
  4. [Sec. 4.2, Common Training Settings] The text states a batch size of 128 per GPU and that the mean/standard deviation for standardization are computed within each GPU. Please clarify whether the effective BN batch size is 128 per GPU (i.e., no synchronization across GPUs) or 1024 in total, and whether synchronized BN is not used.
  5. [Sec. 3.3, footnote 1] The footnote states that learning the affine parameters from standardized features 'works worse' than the presented design, but no quantitative result or experimental protocol is given. Please provide the supporting numbers in an appendix or supplementary material.
  6. [Table 4] The caption of Table 4 does not explain the meaning of the down-arrow notation; in particular, the SN row contains '62.3↓(−0.3)', which is confusing because a negative value inside the parentheses is not an improvement. Please define the notation and clarify how the reported deltas are computed.

Circularity Check

1 steps flagged · score 2.0 of 10

Mild fitting-to-benchmark, not full circularity: ImageNet headline gains are reported on the same validation set used to select AN's hyperparameters; COCO gives independent support.

  1. fitted input called prediction [Section 4.1, Table 1; Section 4.2 'Common Training Settings' and Table 2]
    "During our development, we first observed the best combination based on our intuitive reasoning and small experiments (a few epochs) in the process, and then design this ablation study to verify the design choices. ... The best combination is RSD + A2(·) + hsigmoid + K = (10,10,20,20). ... For AN, the best practice observed in our ablation study (Table 1) is used."

    The AN configuration (input statistic, A1/A2 choice, activation, and per-stage K) was selected because it produced the lowest top-1 error on the ImageNet-1000 validation set in Table 1. The paper's headline claim of 'absolute increase of top-1 accuracy in ImageNet-1000 between 0.5% and 2.7%' is then measured on that same validation set in Table 2 using the selected configuration. The ImageNet part of the claimed consistent improvement is therefore a report of the fitted configuration's performance on the fitting set, not an out-of-sample prediction of AN as a method. The MS-COCO results, which were not used for this selection, remain independent and keep the circularity mild.

full rationale

The paper contains no derivation chain whose conclusion is equivalent to its assumptions: Eq. (8) is a constructive parameterization of an instance-specific mixture of affine transformations, and the reported gains are empirical measurements, not consequences of the equations. The main circularity-adjacent issue is benchmark fitting: K, the attention input (RSD vs. mean vs. mean+std), the activation, and Choice 1 vs. Choice 2 were all chosen on the ImageNet-1000 validation set (Section 4.1), and the abstract's ImageNet improvement range is computed on the same validation set (Section 4.2). That makes the ImageNet headline partly an artifact of selection rather than an independent test. The MS-COCO val2017 results, however, were obtained by transferring the configuration without reselecting it on COCO, so they provide genuinely independent support for the method. The only notable self-citation is AOGNets [24] (by two of the present authors), but it is used simply as one of four testbed architectures; ResNet, DenseNet, and MobileNetV2 would still test the claim, so the self-citation is not load-bearing. Concerns about single runs and small reported margins are reproducibility risks, not circularity, and do not by themselves raise the circularity score further.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The central claim is empirical, so the ledger captures the design choices the method depends on. The free parameters are the hyperparameters selected by ablation on ImageNet-1000: per-stage K, the RSD statistic, the hsigmoid activation, and the A2 architecture with post-FC BN. The axioms are domain assumptions about the value of keeping standardization unchanged, of using a small FC network on RSD to produce attention weights, and of sharing one attention weight between scale and shift. No physical or metaphysical entities are introduced; Attentive Normalization is a designed network module whose validity is directly testable on the benchmarks and via the public code links.

free parameters (4)
  • per-stage mixture size K = [10, 10, 20, 20]
    Selected by ablation in Section 4.1 (Table 1); the authors state they 'empirically select three options for simplicity' and K=(10,10,20,20) for the four stages gave the best validation error.
  • activation function for attention weights = hsigmoid
    Selected by ablation in Table 1; the paper reports 'The hsigmoid function is observed to work better in our experiments' (Section 3.3).
  • channel statistic used as attention input = RSD (sigma_c / mu_c)
    Selected by ablation in Table 1; footnote 2 says 'RSD is indeed observed to work better in our experiments.'
  • attention sub-network variant = Choice 2 (BN after FC)
    Selected by ablation; the paper states 'The Choice 2 improves performance in our experiments in ImageNet' (Section 3.3).
assumptions (4)
  • domain assumption The block-wise standardization component of base feature normalization is kept unchanged and is beneficial.
    Section 3.3: AN retains standardization (Eq. 1) and only changes the affine transform (Eq. 8); no analysis is given for why the existing standardization is appropriate.
  • domain assumption Global average pooling plus a small fully-connected network is sufficient to produce useful instance-specific attention weights.
    Section 3.3, Eq. 11: the attention weights are computed as Act(BN(fc(RSD(x)))), a lightweight network borrowed from SE design; its sufficiency is validated by ablation, not derived.
  • ad hoc to paper The coefficient of variation is a more informative channel statistic than the mean alone for computing these weights.
    Section 3.3, footnote 2: 'RSD is indeed observed to work better in our experiments'; this is an empirical assertion on the target benchmark.
  • ad hoc to paper Sharing the attention weight between the scale and the shift of each mixture component is sufficient for the claimed gains.
    Section 3.3: 'lambda_{i_N,k} is shared by the re-scaling parameter and the re-shifting parameter for simplicity'; the paper provides no experiment relaxing this coupling.
invented entities (1)
  • Attentive Normalization module (mixture of K affine transforms with instance-specific attention weights) independent evidence
    purpose: Provides dynamic, input-dependent affine re-calibration after feature standardization in deep networks.
    This is an architectural design rather than a physical postulate; it is falsifiable through the reported benchmarks and the provided public code repositories.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Attentive Normalization." pith.science (2026). https://pith.science/paper/XFKFT6SJ

@misc{pith2026190801259,
  author       = {Pith},
  title        = {Pith review of: Attentive Normalization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XFKFT6SJ}},
  note         = {Machine review of arXiv:1908.01259}
}
read the original abstract

In state-of-the-art deep neural networks, both feature normalization and feature attention have become ubiquitous. % with significant performance improvement shown in a vast amount of tasks. They are usually studied as separate modules, however. In this paper, we propose a light-weight integration between the two schema and present Attentive Normalization (AN). Instead of learning a single affine transformation, AN learns a mixture of affine transformations and utilizes their weighted-sum as the final affine transformation applied to re-calibrate features in an instance-specific way. The weights are learned by leveraging channel-wise feature attention. In experiments, we test the proposed AN using four representative neural architectures in the ImageNet-1000 classification benchmark and the MS-COCO 2017 object detection and instance segmentation benchmark. AN obtains consistent performance improvement for different neural architectures in both benchmarks with absolute increase of top-1 accuracy in ImageNet-1000 between 0.5\% and 2.7\%, and absolute increase up to 1.8\% and 2.2\% for bounding box and mask AP in MS-COCO respectively. We observe that the proposed AN provides a strong alternative to the widely used Squeeze-and-Excitation (SE) module. The source codes are publicly available at https://github.com/iVMCL/AOGNet-v2 (the ImageNet Classification Repo) and https://github.com/iVMCL/AttentiveNorm\_Detection (the MS-COCO Detection and Segmentation Repo).

Figures

Figures reproduced from arXiv: 1908.01259 by the authors.

Figure 1
Figure 1. Illustration of the proposed Attentive Normalization (AN). AN aims to [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Illustration of integrating the proposed AN in different building blocks. [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 23 canonical work pages

  1. [1]

    CoRR abs/1607.06450 (2016), http://arxiv.org/abs/1607.06450 1, 3

    Ba, L.J., Kiros, R., Hinton, G.E.: Layer normalization. CoRR abs/1607.06450 (2016), http://arxiv.org/abs/1607.06450 1, 3

    arXiv 2016

  2. [2]

    arXiv preprint arXiv:1809.11096 (2018) 2, 4, 10

    Brock, A., Donahue, J., Simonyan, K.: Large scale gan training for high fidelity natural image synthesis. arXiv preprint arXiv:1809.11096 (2018) 2, 4, 10

    arXiv 2018

  3. [3]

    In: 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018

    Cai, Z., Vasconcelos, N.: Cascade R-CNN: delving into high quality object detec- tion. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018. pp. 6154–6162 (2018). https://doi.org/10.1109/CVPR.2018.00644, http://openaccess.thecvf.com/ content_cvpr_2018/html/Cai_Cascade_R-CNN_Delving_CVPR_...

    arXiv 2018

  4. [4]

    arXiv preprint arXiv:1906.07155 (2019) 9, 12

    Chen, K., Wang, J., Pang, J., Cao, Y., Xiong, Y., Li, X., Sun, S., Feng, W., Liu, Z., Xu, J., Zhang, Z., Cheng, D., Zhu, C., Cheng, T., Zhao, Q., Li, B., Lu, X., Zhu, R., Wu, Y., Dai, J., Wang, J., Shi, J., Ouyang, W., Loy, C.C., Lin, D.: MMDetection: Open mmlab detection toolbox and benchmark. arXiv preprint arXiv:1906.07155 (2019) 9, 12

    arXiv 2019

  5. [5]

    In: 7th International Con- ference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 (2019), https://openreview.net/forum?id=HyN-M2Rctm 1, 3, 4

    Deecke, L., Murray, I., Bilen, H.: Mode normalization. In: 7th International Con- ference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 (2019), https://openreview.net/forum?id=HyN-M2Rctm 1, 3, 4

    work page 2019

  6. [6]

    CoRR abs/1606.00704 (2016), http://arxiv.org/abs/1606.00704 2, 4

    Dumoulin, V., Belghazi, I., Poole, B., Lamb, A., Arjovsky, M., Mastropietro, O., Courville, A.C.: Adversarially learned inference. CoRR abs/1606.00704 (2016), http://arxiv.org/abs/1606.00704 2, 4

    arXiv 2016

  7. [7]

    In: Proceedings of the International Conference on Computer Vision (ICCV) (2015) 12

    Girshick, R.: Fast R-CNN. In: Proceedings of the International Conference on Computer Vision (ICCV) (2015) 12

    work page 2015

  8. [8]

    In: IEEE In- ternational Conference on Computer Vision, ICCV 2017, Venice, Italy, Octo- ber 22-29, 2017

    He, K., Gkioxari, G., Doll´ ar, P., Girshick, R.B.: Mask R-CNN. In: IEEE In- ternational Conference on Computer Vision, ICCV 2017, Venice, Italy, Octo- ber 22-29, 2017. pp. 2980–2988 (2017). https://doi.org/10.1109/ICCV.2017.322, https://doi.org/10.1109/ICCV.2017.322 9, 12, 13

    doi:10.1109/iccv.2017.322 2017

Show all 50 references
  1. [9]

    In: 2015 IEEE International Confer- ence on Computer Vision, ICCV 2015, Santiago, Chile, December 7-13, 2015

    He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: Surpassing human- level performance on imagenet classification. In: 2015 IEEE International Confer- ence on Computer Vision, ICCV 2015, Santiago, Chile, December 7-13, 2015. pp. 1026–1034 (2015). https://doi.org/...

    2015 doi

  2. [10]

    In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016) 5, 9, 11

    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016) 5, 9, 11

    2016

  3. [11]

    CoRR abs/1812.01187 (2018), http://arxiv.org/abs/1812.01187 11

    He, T., Zhang, Z., Zhang, H., Zhang, Z., Xie, J., Li, M.: Bag of tricks for image classification with convolutional neural networks. CoRR abs/1812.01187 (2018), http://arxiv.org/abs/1812.01187 11

    2018 arXiv

  4. [12]

    CoRR abs/1905.02244 (2019), http://arxiv.org/abs/1905.02244 8

    Howard, A., Sandler, M., Chu, G., Chen, L., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., Vasudevan, V., Le, Q.V., Adam, H.: Searching for mobilenetv3. CoRR abs/1905.02244 (2019), http://arxiv.org/abs/1905.02244 8

    2019 arXiv

  5. [13]

    CoRR abs/1709.01507 (2017), http://arxiv.org/abs/1709.01507 2, 4, 5, 7, 11

    Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. CoRR abs/1709.01507 (2017), http://arxiv.org/abs/1709.01507 2, 4, 5, 7, 11

    2017 arXiv

  6. [14]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017) 9

    Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017) 9

    2017

  7. [15]

    Huang, L., Liu, X., Lang, B., Yu, A.W., Wang, Y., Li, B.: Orthogonal weight normalization: Solution to optimization over multiple dependent stiefel manifolds 16 Li, Sun and Wu in deep neural networks. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligen...

    2018

  8. [16]

    In: 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018

    Huang, L., Yang, D., Lang, B., Deng, J.: Decorrelated batch normalization. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018. pp. 791–800 (2018) 1, 3

    2018

  9. [17]

    CoRR abs/1811.11721 (2018), http: //arxiv.org/abs/1811.11721 2

    Huang, Z., Wang, X., Huang, L., Huang, C., Wei, Y., Liu, W.: Ccnet: Criss- cross attention for semantic segmentation. CoRR abs/1811.11721 (2018), http: //arxiv.org/abs/1811.11721 2

    2018 arXiv

  10. [18]

    In: Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA

    Ioffe, S.: Batch renormalization: Towards reducing minibatch dependence in batch- normalized models. In: Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA. pp. 1945–1953 ...

    2017

  11. [19]

    In: Blei, D., Bach, F

    Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: Blei, D., Bach, F. (eds.) Proceedings of the 32nd International Conference on Machine Learning (ICML-15). pp. 448–

  12. [20]

    In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019

    Jia, S., Chen, D., Chen, H.: Instance-level meta normalization. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019. pp. 4865–4873 (2019), http: //openaccess.thecvf.com/content_CVPR_2019/html/Jia_Instance-Level_ Meta...

    2019

  13. [21]

    IEEE Transactions on Pattern Analysis and Machine Intelligence pp

    Kalayeh, M.M., Shah, M.: Training faster by separating modes of variation in batch-normalized models. IEEE Transactions on Pattern Analysis and Machine Intelligence pp. 1–1 (2019). https://doi.org/10.1109/TPAMI.2019.2895781 1, 3, 4

    2019

  14. [22]

    arXiv preprint arXiv:1812.04948 (2018) 2, 4

    Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. arXiv preprint arXiv:1812.04948 (2018) 2, 4

    2018 arXiv

  15. [23]

    In: Neural Information Processing Systems (NIPS)

    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep con- volutional neural networks. In: Neural Information Processing Systems (NIPS). pp. 1106–1114 (2012) 5

    2012

  16. [24]

    In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019

    Li, X., Song, X., Wu, T.: Aognets: Compositional grammatical architectures for deep learning. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019. pp. 6220–6230 (2019) 9

    2019

  17. [25]

    In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017

    Lin, T., Doll´ ar, P., Girshick, R.B., He, K., Hariharan, B., Belongie, S.J.: Feature pyramid networks for object detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017. pp. 936–944 (2017). https://doi.org...

    2017 doi

  18. [26]

    CoRR abs/1405.0312 (2014), http://arxiv.org/abs/1405.0312 8, 13

    Lin, T., Maire, M., Belongie, S.J., Bourdev, L.D., Girshick, R.B., Hays, J., Perona, P., Ramanan, D., Doll´ ar, P., Zitnick, C.L.: Microsoft COCO: common objects in context. CoRR abs/1405.0312 (2014), http://arxiv.org/abs/1405.0312 8, 13

    2014 arXiv

  19. [27]

    CoRR abs/1608.03983 (2016), http://arxiv.org/abs/1608.03983 11

    Loshchilov, I., Hutter, F.: SGDR: stochastic gradient descent with restarts. CoRR abs/1608.03983 (2016), http://arxiv.org/abs/1608.03983 11

    2016 arXiv

  20. [28]

    CoRR abs/1806.10779 (2018), http://arxiv.org/abs/1806.10779 2, 3, 9, 11, 12, 13, 14 Attentive Normalization 17

    Luo, P., Ren, J., Peng, Z.: Differentiable learning-to-normalize via switchable nor- malization. CoRR abs/1806.10779 (2018), http://arxiv.org/abs/1806.10779 2, 3, 9, 11, 12, 13, 14 Attentive Normalization 17

    2018 arXiv

  21. [29]

    arXiv preprint arXiv:1802.05637 (2018) 2

    Miyato, T., Koyama, M.: cgans with projection discriminator. arXiv preprint arXiv:1802.05637 (2018) 2

    2018 arXiv

  22. [30]

    In: 2019 IEEE/CVF International Conference on Com- puter Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019

    Pan, X., Zhan, X., Shi, J., Tang, X., Luo, P.: Switchable whitening for deep representation learning. In: 2019 IEEE/CVF International Conference on Com- puter Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019. pp. 1863–1871. IEEE (2019). https://doi.org/10...

    2019

  23. [31]

    In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019

    Park, T., Liu, M., Wang, T., Zhu, J.: Semantic image synthesis with spatially- adaptive normalization. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019. pp. 2337–2346 (2019) 2, 4

    2019

  24. [32]

    In: 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018

    Peng, C., Xiao, T., Li, Z., Jiang, Y., Zhang, X., Jia, K., Yu, G., Sun, J.: Megdet: A large mini-batch object detector. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018. pp. 6181–6189 (2018) 3

    2018

  25. [33]

    CoRR abs/1707.03017 (2017), http://arxiv

    Perez, E., de Vries, H., Strub, F., Dumoulin, V., Courville, A.C.: Learning visual reasoning without strong priors. CoRR abs/1707.03017 (2017), http://arxiv. org/abs/1707.03017 2, 4

    2017 arXiv

  26. [34]

    In: Neural Information Processing Systems (NIPS) (2015) 12

    Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: Towards real-time object de- tection with region proposal networks. In: Neural Information Processing Systems (NIPS) (2015) 12

    2015

  27. [35]

    Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: ImageNet Large Scale Visual Recognition Challenge. Int. J. Comput. Vision (IJCV) 115(3), 211– 252 (2015). https://doi.org/10.1007/s...

    2015 doi

  28. [36]

    In: Advances in Neural Informa- tion Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain

    Salimans, T., Kingma, D.P.: Weight normalization: A simple reparameterization to accelerate training of deep neural networks. In: Advances in Neural Informa- tion Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barce...

    2016

  29. [37]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

    Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: Mobilenetv2: In- verted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 4510–4520 (2018) 9

    2018

  30. [38]

    Santurkar, S., Tsipras, D., Ilyas, A., Madry, A.: How does batch normalization help optimization? In: Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, 3-8 December 2018, Montr´ eal, Canada. pp....

    2018

  31. [39]

    CoRR abs/1903.03793 (2019), http://arxiv.org/abs/1903.03793 2, 3, 4

    Shao, W., Meng, T., Li, J., Zhang, R., Li, Y., Wang, X., Luo, P.: Ssn: Learning sparse switchable normalization via sparsestmax. CoRR abs/1903.03793 (2019), http://arxiv.org/abs/1903.03793 2, 3, 4

    2019 arXiv

  32. [40]

    In: Inter- national Conference on Computer Vision, ICCV (2019) 2, 4

    Sun, W., Wu, T.: Image synthesis from reconfigurable layout and style. In: Inter- national Conference on Computer Vision, ICCV (2019) 2, 4

    2019

  33. [41]

    CoRR abs/1512.00567 (2015), http: //arxiv.org/abs/1512.00567 11

    Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the in- ception architecture for computer vision. CoRR abs/1512.00567 (2015), http: //arxiv.org/abs/1512.00567 11

    2015 arXiv

  34. [42]

    CoRRabs/1607.08022 (2016), http://arxiv.org/ abs/1607.08022 1, 3

    Ulyanov, D., Vedaldi, A., Lempitsky, V.S.: Instance normalization: The missing ingredient for fast stylization. CoRRabs/1607.08022 (2016), http://arxiv.org/ abs/1607.08022 1, 3

    2016 arXiv

  35. [43]

    de Vries, H., Strub, F., Mary, J., Larochelle, H., Pietquin, O., Courville, A.C.: Modulating early visual processing by language. In: Advances in 18 Li, Sun and Wu Neural Information Processing Systems 30: Annual Conference on Neu- ral Information Processing Systems 2017, 4-9 ...

    2017

  36. [44]

    In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017

    Wang, F., Jiang, M., Qian, C., Yang, S., Li, C., Zhang, H., Wang, X., Tang, X.: Residual attention network for image classification. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017. pp. 6450–6458 (2017). https:...

    2017 doi

  37. [45]

    In: 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22,

    Wang, X., Girshick, R.B., Gupta, A., He, K.: Non-local neural net- works. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22,

    2018

  38. [46]

    In: Computer Vision - ECCV 2018 - 15th European Confer- ence, Munich, Germany, September 8-14, 2018, Proceedings, Part VII

    Woo, S., Park, J., Lee, J., Kweon, I.S.: CBAM: convolutional block atten- tion module. In: Computer Vision - ECCV 2018 - 15th European Confer- ence, Munich, Germany, September 8-14, 2018, Proceedings, Part VII. pp. 3–19 (2018). https://doi.org/10.1007/978-3-030-01234-2 1, http...

    2018 doi

  39. [47]

    In: Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part XIII

    Wu, Y., He, K.: Group normalization. In: Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part XIII. pp. 3–19 (2018). https://doi.org/10.1007/978-3-030-01261-8 1, https://doi. org/10.1007/978-3-030-01261-8_1 1, 3, 7, 9...

    2018 doi

  40. [48]

    Zhang, H., Ciss´ e, M., Dauphin, Y.N., Lopez-Paz, D.: mixup: Beyond empirical risk minimization. In: 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings (2018), https://openreview.net/...

    2018

  41. [456]

    JMLR Workshop and Conference Proceedings (2015), http://jmlr.org/ proceedings/papers/v37/ioffe15.pdf 1, 3, 8, 9

    2015

  42. [2018]

    7794–7803 (2018)

    pp. 7794–7803 (2018). https://doi.org/10.1109/CVPR.2018.00813, http://openaccess.thecvf.com/content_cvpr_2018/html/Wang_Non-Local_ Neural_Networks_CVPR_2018_paper.html 2

    2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.