Pith. sign in

REVIEW 4 major objections 6 minor 53 references

Partial Forward Blocking: A Novel Data Pruning Paradigm for Lossless Training Acceleration

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A per-batch density estimate computed from a model's own shallow features can prune 40% of training data, cut training time by a third, and slightly improve accuracy on ImageNet.

desk verdict PFB's speedup claim is more plausible than the stress-test note suggests; the real weakness is the 'lossless' framing, not the partial-forward accounting. read the letter →

arxiv 2506.23674 v1 pith:DKHJMK2W submitted 2025-06-30 cs.CV

Dongyue Wu , Zilin Guo , Jialong Zuo , Nong Sang , Changxin Gao This is my paper
classification cs.CV
keywords datapruningonlinebatchselectiontrainingaccelerationlosslesspartialforwardblockingprobabilitydensityimportanceadaptivedistributionestimationkernel
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Partial Forward Blocking (PFB) is a training-time data pruning method that decides, inside each batch and before the deep layers of the network run, which images are worth continuing. It derives an importance score from the probability density of features at a shallow layer of the target model itself, treating densely crowded feature regions as redundant and rare regions as valuable, and then simply blocks the pruned samples from the rest of the forward pass and from backpropagation. The paper's central claim is that this early blocking turns pruning into real wall-clock savings because no proxy model needs to be trained and no gradient computation is needed for scoring. On ImageNet-1k at a 40% prune ratio, the method is reported to improve top-1 accuracy by 0.5 points while cutting total training time by 33.2%, with similar accuracy retention on CIFAR-10/100 and larger time savings on semantic segmentation. If the claim holds, it gives a path to cheaper large-scale training that scales with the cost of the layers being skipped.

What carries the argument

The machinery is a cheap early-exit scorer paired with a density estimate that adapts as training progresses. Shallow features of the target network are pooled down to a $D$-dimensional representation, and their distribution is approximated by a weighted kernel density estimate, a smoothed histogram of where samples sit in feature space, placed on $N_C$ cluster centroids that are updated by exponential moving average from retained samples. The importance of a sample is the reciprocal of this estimated density plus a small random term, so high-density samples fall below the per-batch percentile threshold and are pruned. The same pooled features are then reused by the surviving samples, and the pruned samples' deep forward and backward paths are cut, which is what converts the selection rule into actual training-time savings.

What would settle it

On a small benchmark, rank PFB's inverse-density importance scores against an approximate leave-one-out generalization influence for each sample: the premise predicts low-density samples are high-influence, so a near-zero or negative rank correlation would falsify the density-to-importance mapping.

Watch

Extended reading notes

Core claim

The core discovery is that sample importance can be read off the target model's own shallow features during training, without auxiliary models or gradients. PFB pools the shallow feature maps to a low-dimensional vector, estimates their probability density with an adaptive kernel density estimator whose kernels sit on a small set of evolving cluster centroids, and assigns each sample the importance $I(z_i)=1/(f_X(x_i)+r)$, where $f_X$ is the estimated density and $r$ is a small random term that prevents the retained set from collapsing. Within each batch the least important fraction is pruned, and pruned samples are blocked from the deep sub-network and from backpropagation, while retained samples reuse their extracted shallow features for the rest of the forward pass. An Adaptive Distribution Estimation module updates the centroids and their weights from the retained samples, so the density estimate tracks the distribution shift caused by learning. The paper reports that this yields accuracy equal to or better than full-data training at moderate prune ratios, 0.5% higher on ImageNet with 40% pruned, while cutting total training time by roughly a third.

Load-bearing premise

The load-bearing assumption is that a sample sitting in a crowded region of shallow-feature space is redundant while a rare sample is informative, so if rarity does not track usefulness, pruning removes exactly the wrong data.

Editorial extensions

If this is right

  • If the reported ImageNet result holds, a 40% batch-wise prune ratio can remove roughly a third of the wall-clock training cost while keeping accuracy at or above full-data training, so PFB is a practical lossless speedup rather than a trade-off.
  • Since scoring uses only shallow features of the target model, the two most expensive components of prior pruning methods, proxy-model training and gradient back-propagation for scoring, are eliminated, which is the paper's main efficiency argument.
  • The savings grow with the cost of the skipped layers: on semantic segmentation, where deep forward passes and loss computation are expensive, PFB reports larger time reductions than on classification, so the method should be increasingly attractive for deeper models and denser tasks.
  • The method transfers across architecture families in the reported experiments, a CNN and a vision transformer, with the main tuning being where to block and how many centroids to keep.
  • At high prune ratios of 70 to 90 percent, accuracy degrades but by less than the compared methods, indicating that the density criterion remains a serviceable importance signal even under aggressive pruning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the density-as-importance premise suggests PFB should be especially effective on long-tailed or class-imbalanced datasets, because rare classes occupy low-density feature regions and would be preferentially retained; the paper does not test this directly.
  • Beyond the paper, the same shallow-feature density score could be reused as a soft curriculum signal, ordering samples or weighting their losses, rather than a hard prune, which would test whether the value comes from selection or from the blocking itself.
  • Beyond the paper, a direct comparison of the adaptive density estimate against a fixed, precomputed feature bank would isolate the contribution of the Adaptive Distribution Estimation module; the paper's claim implies the adaptive version should win, especially in later training epochs.
  • Beyond the paper, the reported accuracy gains at 30% pruning suggest a regularizing effect from removing redundant samples, which implies the method might help most on smaller or noisier datasets; that extrapolation is not in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Partial Forward Blocking (PFB), a batch-wise data pruning method that scores sample importance using shallow-layer features of the target model, prunes low-importance samples, and blocks their forward pass through the deeper layers and their backpropagation. A probability-density importance metric, computed via an adaptive Kernel Density Estimation module, is used to retain rare samples. Experiments on CIFAR-10/100, ImageNet-1k, and semantic segmentation benchmarks report accuracy improvements at low pruning ratios and significant training time reductions, with a headline claim of 0.5% accuracy improvement and 33.2% training time reduction on ImageNet at 40% pruning.

Significance. If the results hold, PFB is a practically attractive data pruning scheme: it avoids expensive gradient computations and proxy-model training, is adaptive to the training state, and is simple to implement. The paper is thorough in its experimental coverage, including ablations, error bars, and comparisons with many recent methods. However, the two central quantitative claims—'lossless' accuracy and a 33% wall-clock speedup—need careful scrutiny. The speedup figure appears inconsistent with the amount of computation actually skipped, and the lossless property is not universal across the reported pruning ratios. The method's core heuristic (rarity equals importance) is plausible but not isolated by ablation.

major comments (4)
  1. [Section 4.2, Table 3] The reported 33.2% training time reduction on ImageNet with ResNet-50 at 40% pruning appears inconsistent with the computation actually saved. With blocking after stage-2, every sample still runs the stem, stage-1, and stage-2; only stage-3, stage-4, and the classifier are skipped for pruned samples. By standard ResNet-50 FLOP estimates, the layers after stage-2 account for roughly 40% of forward FLOPs, so pruning 40% of samples after stage-2 saves at most about 16% of forward (and a similar proportion of backward) compute, which does not obviously translate into a 33% wall-clock reduction. The authors should provide a per-component time breakdown (forward, backward, scoring overhead, data loading, synchronization), report MAC/FLOP counts, and ideally include a control that computes the same shallow features for all samples without pruning to isolate the savings. Without such evidence, the headline acceleration claim is not sufficiently supported.
  2. [Title, Abstract, Section 4.2, Tables 1-2] The phrase 'lossless training acceleration' is used without qualification, but the paper's own results show accuracy drops at higher pruning ratios: CIFAR-10 at 70% pruning (95.2 vs 95.6, Table 1), Swin-T at 40% (79.2 vs 79.6, Table 2), and Swin-T at 50% (78.2 vs 79.6). The lossless property is only clearly supported at 30% pruning and at 40% for ResNet-50 on ImageNet. The authors should either scope the 'lossless' claim to the regime where it is empirically supported, or define a lossless criterion that accounts for the error bars reported in Table 8 (e.g., within one standard deviation of full-data accuracy).
  3. [Section 4.1, Figure 3, Table 9] The key hyperparameters (b, NC, D, blocking location, and pruning start/stop epochs) are selected through ablations on CIFAR-100 and ImageNet and then applied to the other datasets. Because the same datasets are used for evaluation, this creates a risk of selection bias that could inflate the reported accuracies. The authors should report results with a fixed hyperparameter setting across all datasets or perform a proper cross-validation, and they should explicitly acknowledge this limitation in the text.
  4. [Section 3.3] The core premise that low probability density (rarity) implies high sample importance is not validated independently of the full pipeline. The paper shows that PFB outperforms several baselines, but no ablation isolates the contribution of the density-based criterion from the random term (Eq. 3) and the adaptive distribution module. Replacing the density score with a simpler alternative, such as distance to the nearest centroid or a uniform random score with the same computational structure, would clarify whether the density criterion is essential. This is not a blocking issue for the empirical claims, but it would strengthen the causal interpretation.
minor comments (6)
  1. [Author affiliation (page 1)] The word 'Artifcial' in the affiliation is a typo and should read 'Artificial'.
  2. [Algorithm 1, line 1] The phrase 'which can be dived into' should be 'which can be divided into'.
  3. [Eq. (6)] The equation for the bandwidth is formatted incorrectly: 'p hd' appears to be a square root symbol rendered as 'p'; it should read '√hd = (4/((D+2)NC))^(1/(D+4)) σd'.
  4. [Section 3.2, second paragraph] The statement that 'most methods perform B&PU only on retained samples' is misleading for gradient-based online batch selection methods, which compute gradients on all samples to obtain importance scores before selecting a subset for update; the sentence should be rephrased to reflect that distinction.
  5. [Section 3.3, Eq. (3)] The main text states that the upper bound b is set to 0.01, but Table 9 shows b=0.001 for ImageNet experiments and Figure 3(b) sweeps b over a range. Please reconcile these values in the main text or explain that the default setting varies by dataset.
  6. [Algorithm 1, line 9] The pruning condition uses strict inequality I(zi) < τ, which may prune fewer than p fraction of samples in the presence of ties; please specify how ties are broken or how the threshold is adjusted to ensure exactly pNB samples are pruned.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: PFB's importance criterion, pruning rule, and speedup measurements are empirical heuristics that do not reduce by construction to their inputs.

full rationale

The paper makes no first-principles derivation that collapses into its inputs. The importance score in Eq. (2), I(z_i^t) = 1/(f_X(x_i^t)+r), is a heuristic criterion that treats samples in dense feature regions as redundant; this is an assumption, not a consequence of the reported accuracy or speedup. The pruning decision in Eq. (1) is by definition the p fraction with the lowest importance, and the ADE/KDE update in Eqs. (4)-(9) is an algorithmic mechanism rather than a fitted proxy for the target result. No load-bearing self-citations appear: the method-related references are to standard KDE bandwidth rules (Scott, Silverman), InfoBatch, DivBS, and other external works with no author overlap. The headline claim that PFB achieves a 0.5% accuracy improvement and 33% training time reduction on ImageNet is an empirical measurement (Tab. 3), not a consequence entailed by the importance formula; whether the wall-clock speedup is robust is a hardware and implementation question, not a circularity concern. The main caveat is that hyperparameters such as the blocking location, number of centroids N_C, feature dimension D, and the random-term bound b were chosen via ablations on the same evaluation benchmarks (e.g., Fig. 3(a) on ImageNet and Fig. 3(b)-(c) on CIFAR-100), so the reported numbers are partly in-sample. This is a model-selection and reporting limitation, not a circular derivation, and it does not undermine the self-contained nature of the method's logic.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central method rests on the unproven domain assumption that low feature density equals informativeness, plus several hand-tuned hyperparameters (b, NC, D, beta, blocking location) chosen via ablations on the same datasets used for the final results. No new physical entities are introduced.

free parameters (6)
  • alpha upper bound b = 0.01 (CIFAR), 0.001 (ImageNet)
    Controls the scale of the random term r in Eq. (2)-(3); selected via ablation in Fig. 3(b), not derived.
  • number of centroids NC = 64
    Number of kernels in KDE; chosen via ablation in Fig. 3(c).
  • feature dimension D = 128
    Dimensionality of the pooled feature representation; chosen via ablation in Fig. 3(f).
  • EMA coefficient beta = 0.01
    Momentum for centroid updates in Eq. (8); set by hand, no ablation reported.
  • blocking location = stage-2 for ResNet, stage-1 for Swin
    Where to split the network; chosen via ablation in Fig. 3(a).
  • pruning start/stop epochs = e.g., 5/180 for CIFAR, 5/80 for ImageNet ResNet-50, 15/265 for Swin-T
    Epoch range during which pruning is active; listed in Tab. 9 without analysis.
assumptions (5)
  • standard math Kernel density estimation with Gaussian kernel and Silverman's rule yields a valid estimate of the feature distribution
    Used in Sec. 3.4, Eqs. (4)-(6).
  • standard math Exponential moving average update of centroids (Eq. 8) tracks the evolving feature distribution
    Assumed to provide adaptivity; no proof that the EMA centroids converge to the true distribution.
  • domain assumption Samples with low probability density in the shallow-feature space are more informative for generalization
    Core premise of the importance criterion in Sec. 3.3; intuitively motivated but not proven.
  • domain assumption Shallow-layer features are sufficient to judge sample redundancy
    The method prunes after stage-1/2 features; ablation Fig. 3(a) shows performance varies with block location.
  • ad hoc to paper Adding a uniform random term r = alpha * max f_X with alpha~U(0,b) improves diversity
    Eq. (3); the specific form is introduced to add randomness, with b chosen by hand.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Partial Forward Blocking: A Novel Data Pruning Paradigm for Lossless Training Acceleration." pith.science (2026). https://pith.science/paper/DKHJMK2W

@misc{pith2026250623674,
  author       = {Pith},
  title        = {Pith review of: Partial Forward Blocking: A Novel Data Pruning Paradigm for Lossless Training Acceleration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DKHJMK2W}},
  note         = {Machine review of arXiv:2506.23674}
}
read the original abstract

The ever-growing size of training datasets enhances the generalization capability of modern machine learning models but also incurs exorbitant computational costs. Existing data pruning approaches aim to accelerate training by removing those less important samples. However, they often rely on gradients or proxy models, leading to prohibitive additional costs of gradient back-propagation and proxy model training. In this paper, we propose Partial Forward Blocking (PFB), a novel framework for lossless training acceleration. The efficiency of PFB stems from its unique adaptive pruning pipeline: sample importance is assessed based on features extracted from the shallow layers of the target model. Less important samples are then pruned, allowing only the retained ones to proceed with the subsequent forward pass and loss back-propagation. This mechanism significantly reduces the computational overhead of deep-layer forward passes and back-propagation for pruned samples, while also eliminating the need for auxiliary backward computations and proxy model training. Moreover, PFB introduces probability density as an indicator of sample importance. Combined with an adaptive distribution estimation module, our method dynamically prioritizes relatively rare samples, aligning with the constantly evolving training state. Extensive experiments demonstrate the significant superiority of PFB in performance and speed. On ImageNet, PFB achieves a 0.5% accuracy improvement and 33% training time reduction with 40% data pruned.

Figures

Figures reproduced from arXiv: 2506.23674 by the authors.

Figure 1
Figure 1. Computation cost comparison between Partial Forward [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the proposed Partial Forward Blocking pipeline. The [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Ablation studies of PFB on (a) the blocking location of PFB strategy on ImageNet-1k, (b) the upper bound of the uniformly [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

53 extracted references · 41 canonical work pages

  1. [1]

    Youtube-8m: A large- scale video classification benchmark

    Sami Abu-El-Haija, Nisarg Kothari, Joonseok Lee, Paul Natsev, George Toderici, Balakrishnan Varadarajan, and Sudheendra Vijayanarasimhan. Youtube-8m: A large- scale video classification benchmark. arXiv preprint arXiv:1609.08675, 2016. 1

    arXiv 2016

  2. [2]

    Fast kernel classifiers with online and active learn- ing

    Antoine Bordes, Seyda Ertekin, Jason Weston, and L ´eon Bottou. Fast kernel classifiers with online and active learn- ing. Journal of machine learning research , 6(Sep):1579– 1619, 2005. 2

    work page 2005

  3. [3]

    Dataset distillation by matching training trajectories

    George Cazenavette, Tongzhou Wang, Antonio Torralba, Alexei A Efros, and Jun-Yan Zhu. Dataset distillation by matching training trajectories. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4750–4759, 2022. 1

    work page 2022

  4. [4]

    Conceptual 12m: Pushing web-scale image-text pre- training to recognize long-tail visual concepts

    Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut. Conceptual 12m: Pushing web-scale image-text pre- training to recognize long-tail visual concepts. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3558–3568, 2021. 1

    work page 2021

  5. [5]

    Rethinking atrous convolution for seman- tic image segmentation

    Liang-Chieh Chen, George Papandreou, Florian Schroff, and Hartwig Adam. Rethinking atrous convolution for seman- tic image segmentation. arXiv preprint arXiv:1706.05587 ,

    arXiv

  6. [6]

    Encoder-decoder with atrous separable convolution for semantic image segmentation

    Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference on computer vision (ECCV), pages 801–818, 2018. 6, 12

    work page 2018

  7. [7]

    Super-samples from kernel herding

    Yutian Chen, Max Welling, and Alex Smola. Super-samples from kernel herding. arXiv preprint arXiv:1203.3472, 2012. 2, 6, 7

    arXiv 2012

  8. [8]

    Selection via proxy: Efficient data se- lection for deep learning

    Cody Coleman, Christopher Yeh, Stephen Mussmann, Baha- ran Mirzasoleiman, Peter Bailis, Percy Liang, Jure Leskovec, and Matei Zaharia. Selection via proxy: Efficient data se- lection for deep learning. In International Conference on Learning Representations, 2019. 1, 2, 6, 7

    work page 2019

Show all 53 references
  1. [9]

    MMSegmentation: Openmmlab semantic segmentation toolbox and benchmark

    MMSegmentation Contributors. MMSegmentation: Openmmlab semantic segmentation toolbox and benchmark. https : / / github . com / open - mmlab/mmsegmentation, 2020. 12

    2020

  2. [10]

    The cityscapes dataset for semantic urban scene understanding

    Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In CVPR, pages 3213–3223, 2016. 6, 12

    2016

  3. [11]

    Autoaugment: Learning augmentation policies from data

    Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasude- van, and Quoc V Le. Autoaugment: Learning augmentation policies from data. arXiv preprint arXiv:1805.09501, 2018. 12

    2018 arXiv

  4. [12]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009. 6

    2009

  5. [13]

    Towards accelerated model training via bayesian data selection.Advances in Neu- ral Information Processing Systems, 36, 2024

    Zhijie Deng, Peng Cui, and Jun Zhu. Towards accelerated model training via bayesian data selection.Advances in Neu- ral Information Processing Systems, 36, 2024. 2, 3

    2024

  6. [14]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, pages 770–778, 2016. 6

    2016

  7. [15]

    Large- scale dataset pruning with dynamic uncertainty

    Muyang He, Shuo Yang, Tiejun Huang, and Bo Zhao. Large- scale dataset pruning with dynamic uncertainty. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7713–7722, 2024. 2, 6, 7, 12

    2024

  8. [16]

    You only condense once: Two rules for pruning condensed datasets

    Yang He, Lingao Xiao, and Joey Tianyi Zhou. You only condense once: Two rules for pruning condensed datasets. Advances in Neural Information Processing Systems , 36: 39382–39394, 2023. 2

    2023

  9. [17]

    Diversified batch selection for training acceleration

    Feng Hong, Yueming Lyu, Jiangchao Yao, Ya Zhang, Ivor Tsang, and Yanfeng Wang. Diversified batch selection for training acceleration. In Forty-first International Conference on Machine Learning, 2024. 1, 2, 3, 6, 7, 8, 11, 12

    2024

  10. [18]

    Accelerating deep learning by focusing on the biggest losers

    Angela H Jiang, Daniel L-K Wong, Giulio Zhou, David G Andersen, Jeffrey Dean, Gregory R Ganger, Gauri Joshi, Michael Kaminksy, Michael Kozuch, Zachary C Lipton, et al. Accelerating deep learning by focusing on the biggest losers. arXiv preprint arXiv:1910.00762, 2019. 3, 6

    1910 arXiv

  11. [19]

    Not all sam- ples are created equal: Deep learning with importance sam- pling

    Angelos Katharopoulos and Franc ¸ois Fleuret. Not all sam- ples are created equal: Deep learning with importance sam- pling. In International conference on machine learning , pages 2525–2534. PMLR, 2018. 3, 6, 7

    2018

  12. [20]

    Ordered sgd: A new stochastic optimization framework for empirical risk mini- mization

    Kenji Kawaguchi and Haihao Lu. Ordered sgd: A new stochastic optimization framework for empirical risk mini- mization. In International Conference on Artificial Intelli- gence and Statistics, pages 669–679. PMLR, 2020. 6

    2020

  13. [21]

    Glister: Generalization based data subset selection for efficient and robust learning

    Krishnateja Killamsetty, Durga Sivasubramanian, Ganesh Ramakrishnan, and Rishabh Iyer. Glister: Generalization based data subset selection for efficient and robust learning. In Proceedings of the AAAI Conference on Artificial Intelli- gence, pages 8110–8118, 2021. 6

    2021

  14. [22]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In Proceedings of the IEEE/CVF international confer- ence on computer vision, pages 4015–4026, 2023. 1

    2023

  15. [23]

    Understanding black-box predictions via influence functions

    Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International confer- ence on machine learning, pages 1885–1894. PMLR, 2017. 6

    2017

  16. [24]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009. 6 9

    2009

  17. [25]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV, pages 10012–10022, 2021. 6

    2021

  18. [26]

    Online batch selec- tion for faster training of neural networks

    Ilya Loshchilov and Frank Hutter. Online batch selec- tion for faster training of neural networks. arXiv preprint arXiv:1511.06343, 2015. 1, 3

    2015 arXiv

  19. [27]

    Mixed precision training

    Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory Diamos, Erich Elsen, David Garcia, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, et al. Mixed precision training. In International Conference on Learning Representations, 2018. 8

    2018

  20. [28]

    Prioritized training on points that are learnable, worth learning, and not yet learnt

    S ¨oren Mindermann, Jan M Brauner, Muhammed T Raz- zak, Mrinank Sharma, Andreas Kirsch, Winnie Xu, Benedikt H¨oltgen, Aidan N Gomez, Adrien Morisot, Sebastian Far- quhar, et al. Prioritized training on points that are learnable, worth learning, and not yet learnt. In Internati...

  21. [29]

    Coresets for data-efficient training of machine learning mod- els

    Baharan Mirzasoleiman, Jeff Bilmes, and Jure Leskovec. Coresets for data-efficient training of machine learning mod- els. In International Conference on Machine Learning, pages 6950–6960. PMLR, 2020. 6

    2020

  22. [30]

    Coresets for robust training of deep neural networks against noisy labels

    Baharan Mirzasoleiman, Kaidi Cao, and Jure Leskovec. Coresets for robust training of deep neural networks against noisy labels. Advances in Neural Information Processing Systems, 33:11465–11477, 2020. 2

    2020

  23. [31]

    Reading digits in natural images with unsupervised feature learning

    Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bis- sacco, Baolin Wu, Andrew Y Ng, et al. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learn- ing, page 4. Granada, 2011. 1

    2011

  24. [32]

    Dataset meta-learning from kernel ridge-regression

    Timothy Nguyen, Zhourong Chen, and Jaehoon Lee. Dataset meta-learning from kernel ridge-regression. International Conference on Learning Representations, 2020. 1, 2

    2020

  25. [33]

    Deep learning on a data diet: Finding important ex- amples early in training

    Mansheej Paul, Surya Ganguli, and Gintare Karolina Dziu- gaite. Deep learning on a data diet: Finding important ex- amples early in training. Advances in neural information processing systems, 34:20596–20607, 2021. 1, 2, 6, 7, 8

    2021

  26. [34]

    Identifying mislabeled data using the area under the margin ranking

    Geoff Pleiss, Tianyi Zhang, Ethan Elenberg, and Kilian Q Weinberger. Identifying mislabeled data using the area under the margin ranking. Advances in Neural Information Pro- cessing Systems, 33:17044–17056, 2020. 6

    2020

  27. [35]

    Infobatch: Lossless training speed up by unbiased dynamic data pruning

    Ziheng Qin, Kai Wang, Zangwei Zheng, Jianyang Gu, Xi- angyu Peng, Daquan Zhou, Lei Shang, Baigui Sun, Xuan- song Xie, Yang You, et al. Infobatch: Lossless training speed up by unbiased dynamic data pruning. In The Twelfth Inter- national Conference on Learning Representations,...

    2023

  28. [36]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  29. [37]

    Accelerat- ing deep learning with dynamic data pruning

    Ravi S Raju, Kyle Daruwalla, and Mikko Lipasti. Accelerat- ing deep learning with dynamic data pruning. arXiv preprint arXiv:2111.12621, 2021. 2, 3, 6, 8

    2021 arXiv

  30. [38]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1

    2022

  31. [39]

    Imagenet large scale visual recognition challenge

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, San- jeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115:211–252, 2015. 1

    2015

  32. [40]

    Multivariate density estimation: theory, practice, and visualization

    David W Scott. Multivariate density estimation: theory, practice, and visualization . John Wiley & Sons, 2015. 5, 11

    2015

  33. [41]

    Active learning for convolu- tional neural networks: A core-set approach

    Ozan Sener and Silvio Savarese. Active learning for convolu- tional neural networks: A core-set approach. arXiv preprint arXiv:1708.00489, 2017. 2

    2017 arXiv

  34. [42]

    Active learning for convo- lutional neural networks: A core-set approach

    Ozan Sener and Silvio Savarese. Active learning for convo- lutional neural networks: A core-set approach. ICLR, 2018. 6

    2018

  35. [43]

    Density estimation for statistics and data analysis

    Bernard W Silverman. Density estimation for statistics and data analysis. Routledge, 2018. 5, 11

    2018

  36. [44]

    Beyond neural scaling laws: beat- ing power law scaling via data pruning

    Ben Sorscher, Robert Geirhos, Shashank Shekhar, Surya Ganguli, and Ari Morcos. Beyond neural scaling laws: beat- ing power law scaling via data pruning. Advances in Neural Information Processing Systems, 35:19523–19536, 2022. 1

    2022

  37. [45]

    Data pruning via moving-one- sample-out

    Haoru Tan, Sitong Wu, Fei Du, Yukang Chen, Zhibin Wang, Fan Wang, and Xiaojuan Qi. Data pruning via moving-one- sample-out. Advances in Neural Information Processing Sys- tems, 36, 2024. 2, 6, 7

    2024

  38. [46]

    An empirical study of example forgetting during deep neural network learning

    Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geoffrey J Gordon. An empirical study of example forgetting during deep neural network learning. In International Conference on Learning Representations, 2018. 2, 6, 7

    2018

  39. [47]

    Cafe: Learning to condense dataset by align- ing features

    Kai Wang, Bo Zhao, Xiangyu Peng, Zheng Zhu, Shuo Yang, Shuo Wang, Guan Huang, Hakan Bilen, Xinchao Wang, and Yang You. Cafe: Learning to condense dataset by align- ing features. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12196– ...

    2022

  40. [48]

    Moderate coreset: A universal method of data selection for real-world data-efficient deep learning

    Xiaobo Xia, Jiale Liu, Jun Yu, Xu Shen, Bo Han, and Tongliang Liu. Moderate coreset: A universal method of data selection for real-world data-efficient deep learning. In The Eleventh International Conference on Learning Repre- sentations, 2022. 6, 7

    2022

  41. [49]

    Dataset pruning: Reducing training data by ex- amining generalization influence

    Shuo Yang, Zeke Xie, Hanyu Peng, Min Xu, Mingming Sun, and Ping Li. Dataset pruning: Reducing training data by ex- amining generalization influence. In The Eleventh Interna- tional Conference on Learning Representations, 2022. 1, 2, 6

    2022

  42. [50]

    Online coreset selection for rehearsal-based contin- ual learning

    Jaehong Yoon, Divyam Madaan, Eunho Yang, and Sung Ju Hwang. Online coreset selection for rehearsal-based contin- ual learning. In 10th International Conference on Learning Representations, ICLR 2022, 2022. 2

    2022

  43. [51]

    Dataset condensation with dis- tribution matching

    Bo Zhao and Hakan Bilen. Dataset condensation with dis- tribution matching. In Proceedings of the IEEE/CVF Win- ter Conference on Applications of Computer Vision , pages 6514–6523, 2023. 1 10

    2023

  44. [52]

    Dataset condensation with gradient matching

    Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. Dataset condensation with gradient matching. In International Con- ference on Learning Representations, 2020. 1

    2020

  45. [53]

    Coverage-centric coreset selection for high pruning rates

    Haizhong Zheng, Rui Liu, Fan Lai, and Atul Prakash. Coverage-centric coreset selection for high pruning rates. In 11th International Conference on Learning Representations, ICLR 2023, 2023. 6 A. More Experimental Results In this section, we provide more experimental results to...

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.