Pith. sign in

REVIEW 3 major objections 4 minor 22 references

Learning Faster without Deeper Networks: A*-Inspired Batch Selection for Efficient CNN Training

T0 review · 3 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that training a CNN with A*-inspired batch selection outperforms random batch shuffling on all twelve MedMNIST-v2 2D tasks, and that a lightweight network using it matches or beats ResNet-18 and ResNet-50 on six of those ta

desk verdict The A*-BS mechanism, as written, reduces to random shuffling because the λ formula zeros out the loss term when reuse counts are equal; the central claim is unsupported until this is fixed. read the letter →

arxiv 2607.15745 v1 pith:5662KTVR submitted 2026-07-17 cs.LG

Anxhelo Shehu , Enes Stastoli , Arben Cela This is my paper
classification cs.LG
keywords batchselectionA*algorithmCNNtrainingcurriculumlearningMedMNIST-v2efficiencyablationstudyordering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that the order in which mini-batches are fed to a CNN matters as much as architecture. It proposes A*-BS, which treats each batch as a node in a search space, ranks batches each epoch by combining a loss-based difficulty score with a reuse penalty, and trains on high-loss batches while discouraging over-reuse. Under identical architecture and hyperparameters, A*-BS improves accuracy and AUC over random shuffling on all twelve MedMNIST 2D tasks in a single-run ablation. Combined with a roughly 225-thousand-parameter CNN, it reaches higher accuracy and AUC than the much deeper ResNet-18 and ResNet-50 on half of the tasks, with relative gains up to 15%. If the results hold, they imply that batch scheduling can partially compensate for reduced network capacity.

What carries the argument

A*-inspired batch selection (A*-BS): a batch-ordering heuristic that scores each mini-batch as f(B) = g(B) + λ h(B). The heuristic h(B) is the average per-sample cross-entropy (or binary cross-entropy per label for multi-label tasks) computed on a 32-sample subsample of the batch; g(B) is the cumulative reuse count of that batch; λ is automatically computed as the ratio of standard deviations of g and h across batches, balancing exploitation of hard batches with exploration of underused ones. This converts mini-batch scheduling into a search problem and is the mechanism that carries the paper's empirical claims.

What would settle it

Run the controlled ablation with at least five seeds per (dataset, strategy) cell and a paired statistical test: if the accuracy/AUC advantage of A*-BS over random shuffling does not persist across seeds on a majority of the twelve tasks, the central claim fails. A concrete starting point is PathMNIST, where the reported gap is 0.745 vs 0.835 in accuracy.

Watch

Extended reading notes

Core claim

The paper's central claim is that the batch-ordering policy itself is a real axis of training efficiency. The proposed A*-BS mechanism scores each candidate mini-batch with f(B) = g(B) + λ h(B), where h(B) is the average loss of the batch under the current model, g(B) is the number of times that batch has already been selected, and λ is automatically set from the standard deviations of g and h. Each epoch, batches are ranked and presented to the optimizer in order of this score, so high-loss, informative batches are prioritized while over-used batches become less attractive. The authors report that this ordering alone—with no change to the network, optimizer, or loss function—improves test a

Load-bearing premise

The controlled ablation compares a single training run per strategy per dataset, so the claimed all-twelve improvement could be run-to-run noise rather than a true effect of batch ordering.

Editorial extensions

If this is right

  • A*-BS can be added to existing CNN training pipelines without changing architecture, loss, or optimizer, making the efficiency gain nearly free to adopt.
  • On low-resolution medical imaging tasks, a roughly 2.25×10^5-parameter CNN combined with A*-BS can match or exceed the reported accuracy and AUC of ResNet-18 and ResNet-50 on 6 of 12 datasets, with relative gains up to 15%.
  • The extra wall-clock cost of ranking batches is modest—about 22–35% over random shuffling on the lightweight CNN—and remains orders of magnitude cheaper than training ResNet-18 or ResNet-50 on the same hardware.
  • If the ablation generalizes, batch ordering deserves to be treated as a first-class hyperparameter alongside learning rate and batch size, since it improved both accuracy and AUC on all twelve ablations.
  • The method can be combined with deeper architectures, since the paper notes compatibility but does not test it directly.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If confirmed with multi-seed runs, A*-BS would provide a weak-form curriculum that is fully internal: the difficulty signal is recomputed each epoch from the current model state, and it keeps emphasizing high-loss batches even late in training, unlike classical curricula that present easy examples first.
  • Because the heuristic uses only a 32-sample subsample per batch, the mechanism should scale to much larger datasets than MedMNIST, but the paper does not test this regime.
  • The pattern of gains—largest on OCTMNIST, OrganAMNIST, OrganCMNIST, and OrganSMNIST—suggests that A*-BS may help most on small or class-imbalanced tasks where informative gradients are scarce; the paper does not isolate this cause.
  • Combining A*-BS with deeper networks may compound the benefit, since the paper states it can be used with deeper architectures but reports no experiment doing so; this is a natural next test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes A*-Inspired Batch Selection (A*-BS), a method that reorders mini-batches during CNN training using an A*-like score f(B_i)=g(B_i)+λh(B_i), where h is a loss-based difficulty estimate and g is a reuse penalty. The method is evaluated on the twelve 2D MedMNIST-v2 tasks with a lightweight CNN (~2.25×10^5 parameters). The authors report that A*-BS improves accuracy and AUC over random shuffling on all twelve tasks in a same-architecture ablation, and that on six tasks the lightweight CNN plus A*-BS exceeds the published ResNet-18/50 benchmark numbers from [4]. Wall-clock measurements show large speedups relative to ResNet-18/50. The paper explicitly acknowledges that the ResNet comparison is not a controlled experiment and that the ablation uses a single run per condition.

Significance. The core idea — that pure batch-ordering heuristics can partially compensate for reduced network capacity — is interesting and potentially useful for low-resource settings. The method is lightweight, model-agnostic, and would integrate easily into existing pipelines. The authors are also commendably explicit about the uncontrolled nature of the ResNet comparison and about the lack of multi-seed statistics. However, the reported results are not currently supported by the evidence: the mechanism as written degenerates to random ordering, and the only controlled ablation rests on single runs without error bars or paired tests. If these issues are fixed with a clear algorithmic description and proper statistical evaluation, the contribution could be valuable, but in its present form the central claims are not established.

major comments (3)
  1. [Sec. III-C, Eqs. (3)–(4), and Sec. IV-B] The selection score as defined degenerates under the stated training protocol. Eq. (4) sets λ=σ_g/(σ_h+ε). If, as implied by Sec. IV-B ('the only varying factor is the batch-ordering policy') and Sec. III-D ('ranking once per epoch'), every mini-batch is used exactly once per epoch, then at the start of each epoch all g(B_i) are equal, σ_g=0, λ=0, and f(B_i)=g(B_i) is identical across batches. The ranking is then arbitrary — i.e., equivalent to random shuffling. This makes the large improvements in Table II (e.g., ChestMNIST ACC 0.199→0.640) unexplained by the mechanism as written. The authors must either (a) specify a selection mechanism that makes reuse counts unequal within an epoch (e.g., over-sampling high-loss batches and skipping low-loss batches) and analyze its effect on the number of gradient updates per epoch, or (b) show a different operational definition of g. As it stands,
  2. [Sec. IV-B, Table II] Each (dataset, strategy) cell contains a single training run, with no seeds, no error bars, and no paired statistical test. The claim that A*-BS 'improves ACC and AUC over random shuffling on all twelve datasets' cannot be distinguished from run-to-run noise; for example, RetinaMNIST ACC changes from 0.513 to 0.533, a 2% relative shift that is well within typical seed variance. The Limitations section correctly identifies a multi-seed expansion (R≥5) and a paired test as next steps, but the Conclusions still describe the improvement as 'significant' without such evidence. This is load-bearing for the paper's main contribution and must be addressed before publication.
  3. [Sec. IV-A and Sec. V] The abstract and conclusions state that the lightweight CNN with A*-BS 'reaches higher accuracy and AUC than both ResNet baselines' on half the tasks, but Sec. IV-A correctly notes that the ResNet numbers are taken from [4] and were not re-trained in a controlled pipeline. The text in Sec. V ('outperformed ResNet-18 and ResNet-50...') repeats the uncontrolled comparison without the caveat. Because the experimental protocol differs (training time, augmentation, checkpoint selection, infrastructure), these statements should be consistently qualified as benchmark comparisons, not head-to-head wins. This is a presentation issue, but it affects the paper's headline claims.
minor comments (4)
  1. [Sec. III-C, Eq. (2)] For multi-label tasks, h_b is defined per label, but it is not specified how the per-label values are combined into the scalar h(B_i) used in Eq. (3) (e.g., mean, max, sum). Please clarify.
  2. [Sec. IV-C, Table III] The table lists four datasets (PathMNIST, RetinaMNIST, BreastMNIST, PneumoniaMNIST) but the paragraph says 'on three different datasets.' Please correct.
  3. [Sec. V] The statement that A*-BS incurs 'approximately 22–35% overhead over random shuffling' is not consistent with Table III: RetinaMNIST shows A*-BS is faster (1.95 s/epoch vs 2.64 s/epoch), and the other three datasets show 22%, 22%, and 36% overhead. Please qualify the claim.
  4. [Sec. III-D] The description says the heuristic is computed on a subsample of 32 examples per batch. It would be helpful to state whether the same subsample is reused across epochs, and how the random selection of these 32 examples interacts with the ranking determinism.

Circularity Check

0 steps flagged · score 2.0 of 10

No load-bearing circularity: A*-BS is an empirical heuristic whose gains are measured, not fitted; the only self-citation ([11]) is background and non-load-bearing.

full rationale

The paper does not derive A*-BS from first principles; it proposes f(B_i)=g(B_i)+λh(B_i) and then measures ACC/AUC. Eq. 4 is a scale-balancing heuristic, not a fit to the target metric, and Table II is an ablation with identical architecture and hyperparameters, so the random-vs-A*-BS result is an empirical comparison rather than a quantity reconstructed from its inputs. The ResNet comparison (Table I) explicitly uses reference numbers from [4] and is correctly labeled non-controlled, so it does not claim to derive superiority from the method. The only self-citation is [11] in Related Work, used as an example of loss-aware sampling; it does not justify the core mechanism and is not load-bearing. The paper's own Limitations section flags the single-run ablation and missing paired test; this weakens evidence quality but is not circularity. A reader concern that λ=0 when g(B_i) is equal across batches (making Eq. 3 constant) is a substantive mechanism/validity issue to check against the released code, not a circular reduction of the reported numbers to the method's inputs. Score 2 reflects only the minor non-load-bearing self-citation.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim is empirical. There are no physically invented entities. The main free parameters are design choices in the heuristic (λ schedule, ε, subsample size, reuse increment). The axioms are domain assumptions about loss as informativeness and the comparability of external benchmark numbers.

free parameters (4)
  • λ balance coefficient = σ_g/(σ_h+1e-8), Eq. (4)
    The weight of the loss heuristic is fixed by an ad hoc ratio of standard deviations rather than derived or tuned; it shapes selection throughout training.
  • ε numerical stabilizer = 1e-8
    Small constant added to σ_h to avoid division by zero; arbitrary but standard.
  • Heuristic subsample size = 32 examples per batch
    h(B_i) is estimated from 32 subsampled examples per batch, reducing overhead but making the difficulty estimate stochastic and dependent on this hand-chosen size.
  • Reuse penalty increment = 1 per selection
    g(B_i) increments by 1 each time a batch is selected; the increment size is an arbitrary design choice of the exploration penalty.
assumptions (4)
  • domain assumption High average loss of a batch is a valid proxy for informativeness of its gradient update.
    Underlies h(B_i) in Eqs. (1)-(3); the paper does not prove that high-loss batches produce more useful gradients than random batches for CNN training.
  • domain assumption A 32-sample subsample is sufficient to rank batch losses.
    Stated in Sec. III-D; without this, the h estimate could misorder batches and change the results.
  • domain assumption Published MedMNIST ResNet-18/50 numbers from [4] are comparable to the authors' runs despite different architectures and training protocols.
    Sec. IV-A acknowledges the comparison is not controlled; the 'beats ResNet on six tasks' claim depends on this comparability.
  • ad hoc to paper f(B)=g(B)+λh(B) retains A*-like optimality or search properties relevant to training.
    The analogy with A* is motivational; no admissibility, consistency, or optimality conditions are established.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning Faster without Deeper Networks: A*-Inspired Batch Selection for Efficient CNN Training." pith.science (2026). https://pith.science/paper/5662KTVR

@misc{pith2026260715745,
  author       = {Pith},
  title        = {Pith review of: Learning Faster without Deeper Networks: A*-Inspired Batch Selection for Efficient CNN Training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5662KTVR}},
  note         = {Machine review of arXiv:2607.15745}
}
read the original abstract

Common practice when training Convolutional Neural Networks (CNNs) is to use randomly shuffled mini-batches. This creates two limitations: slower convergence, and a diminishing learning signal, since many samples are quickly classified as easy during training. We address these inefficiencies with A*-Inspired Batch Selection (A*-BS), a lightweight, model-agnostic strategy that formulates mini-batch scheduling as a heuristic search problem. Each batch is treated as a node in a search space and ranked using an A*-like score combining a loss-based difficulty measure with a reuse penalty. This encourages informative gradient updates and batch diversity throughout training, without modifying network architectures or optimization algorithms, so it integrates seamlessly into existing pipelines. We evaluate A*-BS on the twelve 2D classification tasks of the MedMNIST-v2 benchmark, using a deliberately simple architecture of approximately 2.25x10^5 parameters, compared against the ResNet-18 and ResNet-50 baselines reported by the benchmark. On half of these tasks, the lightweight model with A*-BS reaches higher accuracy and AUC than both ResNet baselines, with relative gains of up to 15%. An ablation under identical architecture and hyperparameters shows A*-BS outperforms random batch shuffling on all twelve tasks. Wall-clock measurements further show the lightweight CNN with A*-BS trains substantially faster than ResNet-18 and ResNet-50 on identical hardware. These results indicate that intelligent batch ordering can partially compensate for reduced architectural complexity, offering a computationally efficient alternative to deeper models, with reliability reinforced by strong performance even against deeper, more sophisticated architectures.

Figures

Figures reproduced from arXiv: 2607.15745 by the authors.

Figure 1
Figure 1. Representative visualization of the MedMNIST benchmark. For each sub-dataset, five sample images from the training set are presented, providing [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

22 extracted references · 4 linked inside Pith

  1. [4]

    MedMNIST v2: A Large-Scale Lightweight Benchmark for 2D and 3D Biomedical Image Classification,

    J. Yang, R. Shi, D. Wei, Z. Liu, L. Zhao, B. Ke, H. Pfister, and B. Ni, “MedMNIST v2: A Large-Scale Lightweight Benchmark for 2D and 3D Biomedical Image Classification,” inScientific Data, vol. 10, no. 1, p. 41, 2023

    2023

  2. [1]

    Review of deep learning: concepts, convolutional neural network architectures, challenges, and applications,

    L. Alzubaidi, J. Zegnoun, R. M. A. Mrabet, and A. Abdelouahid, “Review of deep learning: concepts, convolutional neural network architectures, challenges, and applications,”Journal of Big Data, vol. 8, no. 1, pp. 1–74, 2021

    2021

  3. [2]

    A formal basis for the heuristic determination of minimum cost paths,

    P. E. Hart, N. J. Nilsson, and B. Raphael, “A formal basis for the heuristic determination of minimum cost paths,”IEEE Transactions on Systems Science and Cybernetics, vol. 4, no. 2, pp. 100–107, 1968

    1968

  4. [3]

    A Systematic Literature Review of A* Pathfinding,

    M. I. Foeadet al., “A Systematic Literature Review of A* Pathfinding,” Procedia Computer Science, vol. 179, pp. 507–514, 2021

    2021

  5. [5]

    Curriculum Learning,

    Y . Bengio, J. Louradour, R. Collobert, and J. Weston, “Curriculum Learning,” inProceedings of the 26th International Conference on Machine Learning (ICML), pp. 41–48, 2009

    2009

  6. [6]

    Self-Paced Learning for Latent Variable Models,

    M. P. Kumar, B. Packer, and D. Koller, “Self-Paced Learning for Latent Variable Models,” inAdvances in Neural Information Processing Systems (NeurIPS), 2010

    2010

  7. [7]

    Training Region-Based Object Detectors with Online Hard Example Mining,

    A. Shrivastava, A. Gupta, and R. Girshick, “Training Region-Based Object Detectors with Online Hard Example Mining,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 761–769, 2016

    2016

  8. [8]

    Focal Loss for Dense Object Detection,

    T.-Y . Lin, P. Goyal, R. Girshick, K. He, and P. Dollar, “Focal Loss for Dense Object Detection,” inProceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 2980–2988, 2017

    2017

Show all 22 references
  1. [9]

    Online Batch Selection for Faster Train- ing of Neural Networks,

    I. Loshchilov and F. Hutter, “Online Batch Selection for Faster Train- ing of Neural Networks,” inInternational Conference on Learning Representations (ICLR) Workshop, 2016

    2016

  2. [10]

    Not All Samples Are Created Equal: Deep Learning with Importance Sampling,

    A. Katharopoulos and F. Fleuret, “Not All Samples Are Created Equal: Deep Learning with Importance Sampling,” inProceedings of the 35th International Conference on Machine Learning (ICML), pp. 2525– 2534, 2018

    2018

  3. [11]

    Ro- bust Time Series Forecasting with Non-Heavy-Tailed Gaussian Loss- Weighted Sampler,

    J. You, A. C ¸ ela, R. Natowicz, J. Ouanounou, and P. Siarry, “Ro- bust Time Series Forecasting with Non-Heavy-Tailed Gaussian Loss- Weighted Sampler,” arXiv:2406.13871, 2024

    2024 arXiv

  4. [12]

    An empirical study of example forgetting during deep neural network learning,

    M. Toneva, A. Sordoni, R. T. des Combes, A. Trischler, Y . Bengio, and G. J. Gordon, “An empirical study of example forgetting during deep neural network learning,” inProc. International Conference on Learning Representations (ICLR), 2019

    2019

  5. [13]

    Beyond neural scaling laws: Beating power law scaling via data pruning,

    B. Sorscher, R. Geirhos, S. Shekhar, S. Ganguli, and A. S. Morcos, “Beyond neural scaling laws: Beating power law scaling via data pruning,” inAdvances in Neural Information Processing Systems (NeurIPS), vol. 35, pp. 19523–19536, 2022

    2022

  6. [14]

    Prioritized training on points that are learnable,

    S. Mindermann, M. T. Razzak, W. Xu, A. Kirsch, et all, “Prioritized training on points that are learnable,” inProc. International Conference on Machine Learning (ICML), vol. 162, pp. 15630–15642, 2022

    2022

  7. [15]

    InfoBatch: Lossless Train- ing Speed Up by Unbiased Dynamic Data Pruning,

    Z. Qin, K. Wang, Z. Zheng, J. Gu, X. Peng, Z. Xu, D. Zhou, L. Shang, B. Sun, X. Xie, and Y . You, “InfoBatch: Lossless Train- ing Speed Up by Unbiased Dynamic Data Pruning,” arXiv preprint arXiv:2303.04947, 2023

    2023 arXiv

  8. [16]

    DeepCore: A comprehensive library for coreset selection in deep learning,

    C. Guo, B. Zhao, and Y . Bai, “DeepCore: A comprehensive library for coreset selection in deep learning,”arXiv preprint arXiv:2204.08499, 2022

    2022 arXiv

  9. [17]

    Adaptive Subgradient Methods for Online Learning and Stochastic Optimization,

    J. Duchi, E. Hazan, and Y . Singer, “Adaptive Subgradient Methods for Online Learning and Stochastic Optimization,”Journal of Machine Learning Research, vol. 12, pp. 2121–2159, 2011

    2011

  10. [18]

    Lecture 6.5 — RMSProp,

    T. Tieleman and G. Hinton, “Lecture 6.5 — RMSProp,”Neural Networks for Machine Learning, Coursera, 2012

    2012

  11. [19]

    Adam: A Method for Stochastic Opti- mization,

    D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Opti- mization,” inInternational Conference on Learning Representations (ICLR), 2015

    2015

  12. [20]

    A Disciplined Approach to Neural Network Hyperparameters: Part 1 – Learning Rate, Batch Size, Momentum, and Weight Decay,

    L. N. Smith and Q. V . Le, “A Disciplined Approach to Neural Network Hyperparameters: Part 1 – Learning Rate, Batch Size, Momentum, and Weight Decay,”arXiv preprint arXiv:1803.09820, 2018

    2018 arXiv

  13. [21]

    On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima,

    N. S. Keskaret al., “On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima,” inInternational Conference on Learning Representations (ICLR), 2017

    2017

  14. [22]

    Deep Residual Learning for Image Recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778, 2016

    2016

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.