Pith. sign in

REVIEW 2 cited by

Rethinking "Batch" in BatchNorm

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2105.07576 v1 pith:FX6CO5FV submitted 2021-05-17 cs.CV

Yuxin Wu , Justin Johnson This is my paper · ORCID
classification cs.CV
keywords batchnormbatchcaveatsdifferentaddressbatchesbehaviorsblock
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

BatchNorm is a critical building block in modern convolutional neural networks. Its unique property of operating on "batches" instead of individual samples introduces significantly different behaviors from most other operations in deep learning. As a result, it leads to many hidden caveats that can negatively impact model's performance in subtle ways. This paper thoroughly reviews such problems in visual recognition tasks, and shows that a key to address them is to rethink different choices in the concept of "batch" in BatchNorm. By presenting these caveats and their mitigations, we hope this review can help researchers use BatchNorm more effectively.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Spooky Action at a Distance: Normalization Layers Enable Side-Channel Spatial Communication

    cs.LG 2025-07 conditional novelty 7.0 of 10

    InstanceNorm, GroupNorm, and BatchNorm can act as spatial communication channels that let CNNs aggregate information from well beyond their local receptive field.

  2. Adapt in the Wild: Test-Time Entropy Minimization with Sharpness and Feature Regularization

    cs.LG 2025-09 conditional novelty 6.0 of 10

    Test-time adaptation is stabilized by filtering unreliable samples, seeking flat entropy minima, and applying redundancy and inequity regularizers to pseudo-labeled class centroids.

Pith tools