Pith. sign in

REVIEW 3 major objections 4 minor 21 references

Mix & Match: training convnets with mixed image sizes for improved accuracy, speed and scale resiliency

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Training on a random mix of image sizes lets a convolutional network match baseline accuracy at half the compute.

desk verdict Worth a serious referee: the B+/D+ recipes and BN calibration are useful and reproducible, but the D+ results are confounded with the authors' own batch augmentation and the abstract's 14% is wrong. read the letter →

arxiv 1908.08986 v1 pith:QZESQVPF submitted 2019-08-12 cs.CV cs.LGstat.ML

Elad Hoffer , Berry Weinstein , Itay Hubara , Tal Ben-Nun , Torsten Hoefler , Daniel Soudry This is my paper · ORCID
classification cs.CVcs.LGstat.ML
keywords mixed-sizetrainingMixSizeimagescaleresiliencydataaugmentationbatchefficientinferenceconvolutionalneuralnetworksNet
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that training a convolutional neural network on a stochastic mixture of image sizes, rather than one fixed size, makes the finished model nearly indifferent to the input resolution it sees at test time. That resiliency is the useful property: after MixSize training, a model can be evaluated on smaller images and still match the accuracy of a conventional fixed-size model, at roughly half the floating-point work. The authors demonstrate this on ImageNet with ResNet-50, reporting 76.43% top-1 accuracy at a 160-pixel evaluation size, matching the baseline's 76.40% at 224 pixels, and a 79.27% top-1 at 288 pixels when the training size distribution is centered near 224. The same regime can be steered toward faster training or higher accuracy by converting the freed compute into larger batches or more duplicate augmentations. If correct, this turns image size from a fixed architectural choice into a tunable axis of the compute-accuracy trade-off.

What carries the argument

The carrying mechanism is the MixSize sampling distribution: a jointly random choice of spatial size $S$, batch size $B$, and duplicate count $D$ at every optimization step, with the constraint $S^2 \cdot B \cdot D \approx \text{const}$ so each step costs about the same compute and memory. Two auxiliary pieces make the regime work in practice. Gradient smoothing is an exponentially weighted running estimate of the gradient norm, used to rescale each optimization step so that steps taken with small images and large batches do not dominate. Batch-norm calibration is a short feed-forward pass over a few hundred training examples at a chosen evaluation size, recomputing the running statistics of each batch-normalization layer; this removes most of the train-test discrepancy that appears when the evaluation size differs from the training size.

What would settle it

Train a fixed-size ResNet-50 at 224 input with the same number of duplicate augmentations and the same total floating-point budget per step as the mixed-size D+ run, reducing batch size accordingly, then measure top-1 accuracy at 96, 128, 160, 224, and 288. If this fixed-size duplicated model matches or exceeds the mixed-size results, 76.43% at 160 or 78.04% at 224, then size mixing itself is not the cause of the gain.

Watch

Extended reading notes

Core claim

The central discovery is that input resolution during training can be treated as a random variable without hurting final accuracy, and with clear benefits. In the MixSize regime, each training step samples a spatial size $S$, a batch size $B$, and a number of per-sample duplicates $D$ from a discrete distribution constrained so that $S^2 \cdot B \cdot D$ stays approximately constant, keeping the computational and memory cost of a step fixed. Because gradients computed from small and large versions of the same image are strongly correlated early in training, the small-image gradients serve as a cheap approximation of the large-image gradients. The authors find that models trained this way generalize well on the original 224-pixel size, improve markedly on sizes below 224, and can exceed the baseline at larger sizes too, reaching 79.27% top-1 accuracy at 288 pixels when the training distribution is balanced near 224. With the freed per-step budget, increasing duplicates raises top-1 accuracy on ImageNet from 76.40% to 78.04%, while increasing batch size reaches 76.61% in about 2.7 times fewer steps.

Load-bearing premise

The claim rests on mixing image sizes being the active ingredient: the regime that produces the largest gains also adds duplicate-augmented copies of each training sample, and the paper compares it only against a fixed-size baseline without such duplicates, so the improvement is not cleanly attributed to size mixing.

Editorial extensions

If this is right

  • A fixed-size model can be replaced by a mixed-size model evaluated at a smaller resolution with equal accuracy and roughly half the inference compute: the headline ResNet-50 result is 76.43% top-1 at size 160 versus 76.40% at size 224.
  • Mixed-size training produces a flatter accuracy-versus-resolution curve, so practitioners can choose a test resolution after training without retraining or fine-tuning.
  • The same per-step compute budget can be redirected: increasing batch size shortens training time, about 2.7 times fewer steps on ImageNet at comparable accuracy, while increasing duplicate augmentations improves final accuracy from 76.40% to 78.04%.
  • The average training size sets the center of the accuracy-versus-resolution curve, so targeting a deployment resolution is a distribution choice rather than a separate training run.
  • Batch-norm calibration at evaluation time removes most of the train-test resolution discrepancy, making the approach usable without per-size fine-tuning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct control experiment the paper does not report would train the same fixed-size baseline with the same duplicate augmentation and compute budget, separating the size-mixing effect from the batch-augmentation effect.
  • Because small and large image gradients are most correlated early in training, a schedule that starts with small images and gradually grows them could capture most of the benefit with less total compute than uniform per-step mixing.
  • The batch-norm calibration step is cheap enough, only hundreds of forward passes, that it could be used at deployment time to adapt a model to a new resolution on the fly, beyond the sizes sampled during training.
  • If size mixing acts as a form of input-space augmentation, it may combine with existing augmentations; the CIFAR experiments already include cutout and autoaugment, so a factorial study would reveal whether the gains are additive or overlapping.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The manuscript introduces MixSize, a stochastic training regime in which each optimization step samples an input image size (and, correspondingly, a batch size or number of duplicate augmentations) from a fixed discrete distribution, keeping per-step computational cost approximately constant. The authors report two regimes: B+, which increases the batch size at smaller image sizes to accelerate training, and D+, which increases the number of batch-augmentation duplicates to improve accuracy. Experiments on CIFAR-10/100 and ImageNet with ResNet, WideResNet, AmoebaNet, and EfficientNet are used to claim that MixSize improves scale resiliency, enables faster inference at smaller test sizes, and yields a better accuracy-compute trade-off than fixed-size training. The paper also proposes gradient smoothing and batch-norm calibration as supporting techniques. The repository with code and pretrained models is made public.

Significance. If the central claims hold, the paper would offer a simple, practical training recipe for making convolutional networks more robust to test-time resolution changes, with a clear computational benefit. The qualitative finding that models trained on mixed sizes exhibit broader accuracy-versus-size curves is plausible and is visually supported by the reported figures. The gradient-correlation analysis in Table 1 is a useful motivating observation. However, the quantitative attribution of the reported gains to stochastic size mixing is not yet established, because the D+ regime is built on the authors' prior batch-augmentation method and is compared only against fixed-size baselines without duplicate augmentation. The paper's headline numerical claims also need correction. With an additional fixed-size duplicate-augmentation control and a fair batch-norm calibration protocol, the central claim would be testable; as it stands, the evidence is suggestive but confounded.

major comments (3)
  1. [§5.1, Table 2; §5.2, Figure 4] The D+ regime, which produces the main accuracy improvements (78.04% on ImageNet, 76.43% at S=160), is trained using batch augmentation with duplicates (Hoffer et al. 2019), yet the comparison baseline is a fixed-size model trained without duplicates. No fixed-size model is trained with the same duplicate count, step budget, per-step compute, and batch-norm calibration. Consequently, the observed gains cannot be attributed to stochastic size mixing; they may be fully explained by batch augmentation alone, which the paper itself notes improves generalization (§2.2). This is the load-bearing gap in the central claim, and it must be closed with a fixed-size D+ control before the MixSize-specific contribution can be assessed.
  2. [Abstract; §5.2, Figure 4a] The abstract states that a 79.27% accuracy at a 288×288 evaluation size is a 'relative improvement of 14% over the baseline.' The reported baseline at 288 is approximately 76.8% (Figure 4a), which gives a relative improvement of about 3.2%, not 14%. Even if the baseline were taken at 224 (76.4%), the relative improvement would be about 3.8%. The paper should either restate the baseline and recompute the claimed improvement, or remove the number; the current claim is numerically unsupported and inconsistent with the reported curves.
  3. [§4.2 and §5.2] Batch-norm calibration is applied to mixed-size models but not to fixed-size baselines: the text says that for fixed-size regimes calibration 'resulted with degraded results' and so those models are reported without calibration. This asymmetry is not backed by any reported experiment, and it could inflate the apparent advantage of mixed-size models, especially because fixed-size models evaluated at off-training sizes also suffer from batch-norm statistics mismatch. The authors should report fixed-size results both with and without calibration, or provide the supporting data for the claim that calibration does not help fixed-size models.
minor comments (4)
  1. [Abstract and §5.2] The phrase 'we receive a 76.43% top-1 accuracy' is unidiomatic; 'we achieve' is clearer.
  2. [§5.2] The sentence beginning 'We analyzed an alternative regime S(208)' would benefit from stating explicitly that the comparison in Figure 4a is between D+ regimes and the fixed-size baseline, since the text does not clearly separate the B+ and D+ results.
  3. [Appendix D] The distributions S(208) and S(224) are listed but no results are shown for S(224) in the main text beyond a single point in Figure 4b; adding the accuracy-versus-size curve for this regime would make the comparison complete.
  4. [§3, Eq. (1)] The notation (S,B,D)_i with probabilities p_i is slightly ambiguous: it should be clarified whether the tuple is sampled jointly per step or each component is sampled independently.

Circularity Check

1 steps flagged · score 4.0 of 10

D+ gains cannot be separated from the authors' prior Batch Augmentation method; no fixed-size BA control is reported.

  1. other [Section 3 (Eq. 1), Section 5.1 (ImageNet D+ rows in Table 2), Section 5.2 (Figure 4a)]
    "The batch size is changed either by the number of samples used, denoted B, or the number of batch-augmentations for each sample (Hoffer et al., 2019), denoted D (”duplicates”). ... We compare these results with a D+ regime, trained with an average size of S = 144. ... due to the decreased average size, we were able to leverage more than 1 duplicates per batch on average, which improved the model’s top-1 accuracy to 77.14% at size 224."

    The headline MixSize numbers, including the 78.04% ImageNet result and the 76.43% accuracy at S=160, are all D+ results. By the paper's own definitions, D+ couples stochastic image sizes with duplicate batch-augmentation from Hoffer et al. (2019), a self-cited prior method. Table 2 and Figure 4a compare D+ only to fixed-size models without duplicates; no fixed-size D+ control is trained. Since BA alone is acknowledged to improve generalization, the observed gains cannot be attributed specifically to stochastic size mixing: the central claim that MixSize improves scale resiliency is statistically confounded with the self-cited BA effect.

full rationale

This is partial, not full, circularity. No equation-level tautology is present: the paper does not define MixSize in terms of the accuracies it later reports, and no fitted parameter is renamed as a prediction. The B+ speedup results and the small-image resiliency curves are independent empirical contributions, and the paper is self-contained against external baselines such as Touvron et al. (2019). However, the flagship D+ accuracy claims are load-bearing on the authors' prior Batch Augmentation method without the necessary fixed-size BA control, so the unique contribution of stochastic size mixing is never isolated. The ImageNet size distribution was also selected by cross-validation over alternatives, which is tuning rather than circularity but further softens the claim that the regime is first-principles. Separately, the abstract's stated 14% relative improvement for the 79.27% result is numerically inconsistent with the reported 76.4-76.8% baselines, though this is an arithmetic issue rather than a circularity. Overall score 4 reflects a self-citation-flavored confound in the central attribution while other results retain independent content.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on empirical comparisons between training regimes. The key free parameters are the size distributions chosen by cross-validation and the smoothing coefficient; no new physical or mathematical entities are introduced.

free parameters (4)
  • ImageNet MixSize distribution for S(144) = {256:0.1, 224:0.1, 128:0.6, 96:0.2}
    Chosen by cross-validation over alternative regimes (Appendix D). Determines the average training size S=144 and directly affects the reported speed/accuracy trade-offs.
  • Alternative size distributions S(208) and S(224) = probabilities per size listed in Appendix D
    Used in the scale-resiliency comparisons; these are hand-selected and validated alternatives, so they are also fitted to the problem.
  • Gradient smoothing coefficient alpha = not reported
    Exponential moving average factor for gradient-norm smoothing used in B+ runs; the value is a tunable hyperparameter.
  • Batch-norm calibration batch count = 200 batches
    Chosen to estimate BN running statistics at each evaluation size; the number of batches is a free experimental choice.
assumptions (4)
  • domain assumption CNN computational cost per training step is approximately proportional to S^2 * B * D
    Section 3 uses this to construct regimes with constant per-step budget. It ignores fixed overheads, padding, and layer-specific cost differences.
  • domain assumption Gradients computed on small images are strongly correlated with full-size gradients
    Table 1 measures this on CIFAR10 with ResNet-44; the paper assumes the effect transfers to ImageNet-scale models and datasets.
  • domain assumption Batch-norm statistics calibrated on 200 batches generalize to the whole validation set at that size
    Section 4.2 and Section 5.2 assume this makes the mixed-size evaluation comparable to fixed-size evaluation at each image size.
  • standard math The standard training setup of Goyal et al. (2017) is a valid fixed-size baseline
    Accepted as a baseline in the field; the paper does not derive it, but uses it as the reference point for comparisons.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Mix & Match: training convnets with mixed image sizes for improved accuracy, speed and scale resiliency." pith.science (2026). https://pith.science/paper/QZESQVPF

@misc{pith2026190808986,
  author       = {Pith},
  title        = {Pith review of: Mix & Match: training convnets with mixed image sizes for improved accuracy, speed and scale resiliency},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QZESQVPF}},
  note         = {Machine review of arXiv:1908.08986}
}
read the original abstract

Convolutional neural networks (CNNs) are commonly trained using a fixed spatial image size predetermined for a given model. Although trained on images of aspecific size, it is well established that CNNs can be used to evaluate a wide range of image sizes at test time, by adjusting the size of intermediate feature maps. In this work, we describe and evaluate a novel mixed-size training regime that mixes several image sizes at training time. We demonstrate that models trained using our method are more resilient to image size changes and generalize well even on small images. This allows faster inference by using smaller images attest time. For instance, we receive a 76.43% top-1 accuracy using ResNet50 with an image size of 160, which matches the accuracy of the baseline model with 2x fewer computations. Furthermore, for a given image size used at test time, we show this method can be exploited either to accelerate training or the final test accuracy. For example, we are able to reach a 79.27% accuracy with a model evaluated at a 288 spatial size for a relative improvement of 14% over the baseline.

Figures

Figures reproduced from arXiv: 1908.08986 by the authors.

Figure 1
Figure 1. Test accuracy per image size, models trained on specific sizes (ResNet50, ImageNet). Convolutional neural networks are successfully used to solve various tasks across multiple do￾mains such as visual (Krizhevsky et al., 2012; Ren et al., 2015), audio (van den Oord et al., 2016), language (Gehring et al., 2017) and speech (Abdel-Hamid et al., 2014). While scale-invariance is considered important for vi￾sual represent… view at source ↗
Figure 2
Figure 2. Training (dotted) and test accuracy vs optimization step (ResNet44, CIFAR10). We com [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Training (dotted) and test accuracy on ImageNet using the Baseline, B+ and D+ regimes (224 × 224 evaluation size). All regimes required similar computational resources per step. B+ regime required ≈ 2.7× less steps per epoch. As can be seen in [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Left: Test accuracy on validation set per image size, all models trained using the same [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Impact of gradient smoothing on CIFAR10, ResNet-44. The training regime includes two [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Test accuracy vs step for 3 size sampling regimes: (1) From small to large (2) Sample each [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 5 canonical work pages

  1. [3]

    Autoaugment: Learning augmentation policies from data

    Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V Le. Autoaugment: Learning augmentation policies from data. arXiv preprint arXiv:1805.09501,

    arXiv

  2. [6]

    Accurate, large minibatch sgd: Training imagenet in 1 hour

    Priya Goyal, Piotr Doll´ar, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, An- drew Tulloch, Yangqing Jia, and Kaiming He. Accurate, large minibatch sgd: Training imagenet in 1 hour. arXiv preprint arXiv:1706.02677,

    arXiv

  3. [7]

    Augment your batch: better training with larger batches

    Elad Hoffer, Tal Ben-Nun, Itay Hubara, Niv Giladi, Torsten Hoefler, and Daniel Soudry. Augment your batch: better training with larger batches. arXiv preprint arXiv:1901.09335,

    arXiv 1901

  4. [8]

    Some improvements on deep convolutional neural network based image classi- fication

    9 Andrew G Howard. Some improvements on deep convolutional neural network based image classi- fication. arXiv preprint arXiv:1312.5402,

    arXiv

  5. [11]

    Batch renormalization: Towards reducing minibatch dependence in batch-normalized models

    Sergey Ioffe. Batch renormalization: Towards reducing minibatch dependence in batch-normalized models. In Advances in neural information processing systems, pp. 1945–1953,

    work page 1945

  6. [14]

    Sgdr: Stochastic gradient descent with warm restarts

    Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983,

    arXiv

  7. [15]

    Measuring the effects of data parallelism on neural network training

    Christopher J Shallue, Jaehoon Lee, Joe Antognini, Jascha Sohl-Dickstein, Roy Frostig, and George E Dahl. Measuring the effects of data parallelism on neural network training. arXiv preprint arXiv:1811.03600,

    arXiv

  8. [16]

    Very deep convolutional networks for large-scale image recognition

    Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556,

    arXiv

Show all 21 references
  1. [18]

    A¨aron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alexander Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu

    URL http://arxiv.org/abs/1906.06423. A¨aron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alexander Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. Wavenet: A generative model for raw audio

    1906 arXiv

  2. [20]

    Large batch training of convolutional networks.arXiv preprint arXiv:1708.03888,

    Yang You, Igor Gitman, and Boris Ginsburg. Large batch training of convolutional networks.arXiv preprint arXiv:1708.03888,

    arXiv

  3. [21]

    11 Appendix A E XPERIMENTAL SETTINGS A.1 CIFAR We used the common data augmentation technique as described by He et al. (2016). In this method, the input image is padded with 4 zero-valued pixels at each side, top and bottom. A random32× 32 part of the padded image is then cro...

    2016

  4. [22]

    Solid lines are test errors while dotted lines are for training

    (b) Training and test error with and with out gradient smoothing. Solid lines are test errors while dotted lines are for training. Figure 5: Impact of gradient smoothing on CIFAR10, ResNet-44. The training regime includes two image sizes: 32× 32 and 16× 16 (average size isS = ...

    2018

  5. [2009]

    Improved regularization of convolutional neural networks with cutout

    Terrance DeVries and Graham W Taylor. Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552,

    arXiv

  6. [2012]

    Network in network

    Min Lin, Qiang Chen, and Shuicheng Yan. Network in network. arXiv preprint arXiv:1312.4400,

    arXiv

  7. [2013]

    Fastai - progressive resizing

    Jeremy Howard. Fastai - progressive resizing. https://www.fast.ai/2018/04/30/ dawnbench-fastai/,

    2018

  8. [2014]

    Multigrain: a unified image embedding for classes and instances

    Maxim Berman, Herv´e J´egou, Andrea Vedaldi, Iasonas Kokkinos, and Matthijs Douze. Multigrain: a unified image embedding for classes and instances. arXiv preprint arXiv:1902.05509,

    1902 arXiv

  9. [2015]

    Progressive growing of gans for im- proved quality, stability, and variation

    Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for im- proved quality, stability, and variation. arXiv preprint arXiv:1710.10196,

    arXiv

  10. [2016]

    Scale-invariant con- volutional neural networks

    Yichong Xu, Tianjun Xiao, Jiaxing Zhang, Kuiyuan Yang, and Zheng Zhang. Scale-invariant con- volutional neural networks. arXiv preprint arXiv:1411.6369,

    arXiv

  11. [2017]

    Stochastic gradient methods with layer- wise adaptive moments for training of deep networks

    Boris Ginsburg, Patrice Castonguay, Oleksii Hrinchuk, Oleksii Kuchaiev, Vitaly Lavrukhin, Ryan Leary, Jason Li, Huyen Nguyen, and Jonathan M Cohen. Stochastic gradient methods with layer- wise adaptive moments for training of deep networks. arXiv preprint arXiv:1905.11286,

    1905 arXiv

  12. [2018]

    Gpipe: Efficient training of giant neural networks using pipeline parallelism.arXiv preprint arXiv:1811.06965,

    Yanping Huang, Yonglong Cheng, Dehao Chen, HyoukJoong Lee, Jiquan Ngiam, Quoc V Le, and Zhifeng Chen. Gpipe: Efficient training of giant neural networks using pipeline parallelism.arXiv preprint arXiv:1811.06965,

    arXiv

  13. [2019]

    Faster neural network training with data echoing

    Dami Choi, Alexandre Passos, Christopher J Shallue, and George E Dahl. Faster neural network training with data echoing. arXiv preprint arXiv:1907.05550,

    1907 arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.