Pith. sign in

REVIEW 4 major objections 5 minor 102 references

RL-Selector: Reinforcement Learning-Guided Data Selection via Redundancy Assessment

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that training-data redundancy can be captured by mutual feature closeness (epsilon-sample cover), and that an RL agent optimizing a coverage-based reward selects smaller training subsets that train models to equal or…

desk verdict Plausible RL-based data selection with an inverted reward as written; the experiments are extensive, but the method's stated pruning objective is contradicted by its own equations. read the letter →

arxiv 2506.21037 v1 pith:44VSRTPB submitted 2025-06-26 cs.LG cs.CV

Suorong Yang , Peijia Li , Furao Shen , Jian Zhao This is my paper
classification cs.LGcs.CV
keywords dataselectionpruningcore-setreinforcementlearningepsilon-samplecoverredundancyassessmenttrainingdynamicsA2C
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that data redundancy, not just sample importance, is the right target for data selection, and that redundancy can be quantified by mutual closeness of same-class feature vectors. It introduces the epsilon-sample cover: two samples are epsilon-covered when their intermediate-layer features are within distance epsilon, and proves that mutually covered samples produce similar outputs, losses, and gradients, so deleting one costs little. The method wraps this coverage signal in an RL loop: a lightweight A2C agent learns per-sample scores during training, balancing a target selection ratio with a coverage-based pruning reward, and the final scores decide what to keep. If right, this gives a training-dynamics-aware, ratio-adaptive selection procedure that consistently beats static scoring baselines, and selected subsets even beat full-data training on ImageNet-style benchmarks.

What carries the argument

The epsilon-sample cover (Def. 1) is the central object: sample $x_i$ is $\epsilon$-covered by same-class $x_j$ when $\|\tilde{x}_i - \tilde{x}_j\| \le \epsilon$ for intermediate-layer feature maps. The theory (Props. 1-3) shows mutual coverage implies similar outputs, losses, and weight gradients, with $\lim_{\epsilon\to 0}\|\Delta g_W\| = 0$; Prop. 4 then bounds the influence-function gap between pruned and kept samples. The algorithm never sets an actual epsilon; instead it uses the unthresholded sum of pairwise distances in Eq. (6) as a soft coverage score, and feeds that score, combined with a ratio-alignment term, as reward $r = r_1 + r_2$ to a three-linear-layer A2C actor-critic that learns sample-wise scores over training epochs. The scores are binarized at 0.5 to produce the selected subset.

What would settle it

Train a model on CIFAR-100, locate the same-class pair with the smallest feature distance under the selection network, delete exactly one of the two, retrain from scratch, and measure test accuracy against the unpruned baseline: Props. 1-3 predict a negligible change, so a large drop would refute the epsilon-cover redundancy claim.

Watch

Extended reading notes

Core claim

The paper's central discovery is that mutually epsilon-covered samples are nearly interchangeable for training: their outputs (Prop. 1), losses (Prop. 2), and gradient contributions (Prop. 3) coincide as epsilon tends to 0, and consequently pruning covered samples barely moves the model parameters (Prop. 4). This turns data selection into a redundancy-removal problem. The method measures each sample's coverage by the sum of pairwise feature distances to same-class samples (Eq. 6), uses this as a reward to train an A2C agent that outputs sample-wise continuous scores, and binarizes the scores at 0.5 to hit a target selection ratio within 1% tolerance. The RL sweep over training epochs captures how redundancy evolves as features change, addressing the group effect that static scoring misses. Empirically the paper reports consistent improvements over ten baselines across CIFAR-10/100, Tiny-ImageNet and ImageNet-1k, with selected subsets transferring to unseen architectures and improving accuracy on ImageNet-A/R/Hard.

Load-bearing premise

The load-bearing premise is that same-class samples with close intermediate-layer features are interchangeable for training, so deleting them costs little; the proof holds only for a linear-model approximation in the limit as the distance goes to zero, and the paper itself flags that noisy or outlier samples break the premise.

Editorial extensions

If this is right

  • At selection ratios down to 20-40%, models trained on RL-Selector subsets match or exceed full-data accuracy on CIFAR-10/100, Tiny-ImageNet and ImageNet-1k.
  • Subsets chosen with a small proxy network transfer to larger and different architectures (ResNet-50, VGG-16, DenseNet-121, ViT, Swin-Transformer) without retraining the selector.
  • Training on selected ImageNet-1k subsets improves accuracy on ImageNet-A, ImageNet-R and ImageNet-Hard relative to full-data training, so redundancy removal can improve robustness.
  • A pretrained selection policy can be fine-tuned for 15 epochs to select at a new ratio, giving roughly a 7x speedup in selection cost with only a small accuracy drop.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's claims, the reward design suggests a direct stress test: if coverage scores are computed with a self-supervised feature extractor instead of a label-trained one, the method should remain effective for transfer tasks where labels are scarce; the paper does not test this.
  • The theory's epsilon-to-zero limit implies a testable prediction the paper does not run: for a fixed dataset, the benefit of pruning should increase as retained pairs are restricted to smaller feature distances; measuring pruning gain against distance thresholds would isolate when the redundancy assumption begins to fail.
  • Because the method prunes within classes, it implicitly assumes redundancy is class-local; an extension to cross-class coverage distances could address class-imbalanced data, but that is not claimed here.
  • The reported speedup applies to training the downstream model on the smaller subset, not to the selection cost itself; a full lifecycle comparison including the surrogate-training and RL epochs would clarify the net compute trade-off.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes RL-Selector, a data-selection method that uses a reinforcement learning agent (A2C) to choose training subsets. The selection signal is based on an "epsilon-sample cover" notion: for same-class samples, the sum of pairwise feature distances (Eq. 6) is used as a proxy for redundancy, and the RL reward combines a selection-ratio alignment term (Eq. 5) with a coverage-based selection term (Eq. 7). The authors prove that, under a linear-model approximation and in the epsilon-to-zero limit, mutually epsilon-covered samples have similar outputs, losses, and gradients (Props. 1-3), and they use an influence-function argument to claim that pruning covered samples is safe (Prop. 4). Experiments on CIFAR-10/100, Tiny-ImageNet, and ImageNet-1k report consistent improvements over baselines, including cross-architecture transfer and efficiency gains.

Significance. The empirical scope is substantial: the paper evaluates across multiple datasets, architectures, and selection ratios and reports gains on ImageNet-1k and on robustness benchmarks. If the method works as described, the idea of adapting selection through training dynamics with a lightweight RL agent is of interest to the data-centric AI community. However, the formal contribution is modest: Propositions 1-3 are essentially Lipschitz-style continuity statements that follow from Definition 1, and Proposition 4 relies on a positive-definite Hessian assumption that is typically false for overparameterized networks. More importantly, the reward definitions in the main text appear to have inverted signs relative to the stated pruning objective, and correcting them would change the behavior of the algorithm. As written, the theoretical justification and the actual optimized objective are not aligned.

major comments (4)
  1. [Section 4, Eq. (5)] The reward r1 is defined as a positive normalized discrepancy between the current selection ratio and the target ratio sr. Since A2C maximizes cumulative reward (Eq. 8 minimizes -log pi A, so positive rewards increase the probability of actions leading to them), maximizing r1 encourages the selection ratio to move away from sr, not toward it. If the 1% ratio constraint is enforced as a hard post-processing step, r1 may only push the ratio to the boundary of the allowed interval, but the printed reward does not implement ratio alignment. The text says the reward 'aligns the current selection ratio with the target one,' which is the opposite of what Eq. (5) does.
  2. [Section 4, Eq. (7) and Eq. (6)] The coverage degree Ec is defined as the sum of pairwise feature distances within the class (Eq. 6), so a high Ec means a sample is far from its same-class neighbors and has low epsilon-coverage. The selection reward r2 = Ec(x) * pi(x) is then maximized, which rewards selecting the least-covered, most outlier-like samples. This is the opposite of the stated goal of 'pruning samples with a high degree of epsilon coverage' and contradicts the theory in Props. 1-3, which says mutually covered samples are redundant and should be removed. The ablation in Table 3 makes the issue concrete: the no-RL baseline selects samples with the smallest Ec and underperforms, while the RL agent under the printed reward is pushed toward largest Ec; the reported gains therefore come from selecting the samples the theory identifies as non-redundant.
  3. [Section 3, Props. 1-3 and Section 4] The formal results are proved only in the limit epsilon -> 0 and under a linear-model approximation of the ReLU network (Appendix 1 uses y = wx + b, and Lemma 1 uses a first-order Taylor expansion). The algorithm never sets an epsilon and uses the unthresholded sum of pairwise distances from Eq. (6), so no finite-epsilon guarantee is provided for the actual operating regime. Furthermore, Proposition 4 assumes the Hessian H_theta is positive definite (Eq. 10), which is generally false for overparameterized deep networks. These gaps mean the theory does not establish the safety of pruning under the conditions in which RL-Selector is deployed.
  4. [Section 3 and Section 4 (circularity of the argument)] The conclusions (C.1)-(C.3) are essentially a restatement of Definition 1 plus continuity of the network map: if two feature vectors are close, the outputs, losses, and gradients are close. This conclusion is then used to design the reward in Eq. (7), so the theory does not independently validate the selection criterion; the load-bearing assumption is the empirical correlation between feature-space distance and training redundancy. This would be acceptable if the reward sign were consistent with the theory, but combined with the sign inversion above, the theoretical section does not support the algorithm actually optimized.
minor comments (5)
  1. [Section 2] There is a typo in the related-work section: 'dynamic dataest prun-ing' should read 'dynamic data-set pruning'.
  2. [Section 4, Eq. (6)] The summation in Eq. (6) runs from j = 0 to n for an n x n matrix D^k, which appears to be an off-by-one indexing error; the diagonal self-distance should also be excluded from the coverage estimate.
  3. [Section 5.3] The sentence 'the performance differences among methods are marginal at high selection ratios... yet our method demonstrates more substantial gains at these ratios' is internally contradictory; please clarify whether the gains are at high or low selection ratios.
  4. [Section 4, Policy Optimization] The binarization threshold of 0.5 and the 1% selection-ratio tolerance are introduced without justification or sensitivity analysis; since the ratio constraint interacts with the reward in Eq. (5), its role should be stated explicitly.
  5. [Figure 3] The selection-cost comparison in Figure 3 lacks units and a precise protocol in the main text; please specify whether the reported numbers are wall-clock hours, GPU-hours, or FLOP counts, and how the costs of the RL module are amortized.

Circularity Check

3 steps flagged · score 6.0 of 10

The formal results (Props. 1–3) reduce to Def. 1 by construction: 'ε-covered' means close features, the proofs only apply continuity, and the 'safe to prune' conclusion is assumed in the reward (Eqs. 6–7), then read back as a validated prediction; Eq. (5) and the Section 5.5 ablation are additionally sign-inconsistent with the stated mechanism.

  1. self definitional [Section 3, Definition 1 and Propositions 1–3; Appendix 1–4 (proofs of Props. 1–3)]
    "Definition 1 (ϵ-sample cover) Let x̃_i and x̃_j denote the feature maps of x_i and x_j, respectively, with y_i = y_j. We say that x_i is ϵ-covered by x_j if ‖x̃_i − x̃_j‖ ≤ ϵ. ... Proposition 1 If x_i and x_j are mutually ϵ-covered ... we have, ‖ŷ_i − ŷ_j‖ ≤ O(ϵ). (1) ... Proof 1 ... ‖ŷ_i − ŷ_j‖ = ‖w(x_i − x_j)‖ ≤ ‖w‖ ‖x_i − x_j‖ ≤ ϵ‖w‖."

    The 'prediction' of Props. 1–3 — that mutually ε-covered samples have similar outputs, losses, and gradients — is obtained by inserting the defining inequality of Def. 1 into Hölder/norm bounds on a linearized model (output gap ≤ ε‖w‖; gradient gap ≤ ε(‖x̃‖‖H‖+‖g‖)). This is uniform continuity: 'close features ⇒ close behavior' is what the definition already says for smooth maps; no fact about redundancy is derived. The leap to Contribution 2 ('removing them from the training dataset incurs minimal impact on model generalization') is not proven — pairwise closeness of gradients/influence does not imply safe deletion from the ERM — and is instead an assumption that the method encodes in its reward. The formal support therefore restates its input (feature closeness) by construction.

  2. fitted input called prediction [Section 4, 'Reward Modeling' (Eqs. 6–7), with Sections 5.2–5.5 (validation)]
    "Based on Proposition 1-3, we estimate the degree of ϵ-coverage for each sample, which serves as another reward signal. ... The degree of ϵ-cover is then estimated as: Ec(x_i^k) = Σ_{j=0}^n D^k_{ij} (6). ... The selection reward is expressed as: r2 = Ec(x) ⊙ π(x), (7)."

    E_c in Eq. (6) is the unthresholded sum of the very pairwise feature distances that define ε-cover (Def. 1), and r2 = E_c ⊙ π feeds that quantity back as the reward with a fixed direction (covered/low-distance samples get near-zero selection reward and are pruned). Props. 1–3 only bound pairwise output/loss/gradient agreement; they never establish the selection direction or that dropping covered samples is near-lossless. The criterion is thus an input to the reward, and the test-accuracy gains (Fig. 2, Tables 1–5) are reported as validating the redundancy premise ('validating the existence of redundant samples in large-scale datasets').

1 more flagged steps
  1. other [Section 4, 'Reward Modeling' (Eqs. 5 and 7); Section 5.5, 'Ablation Study']
    "The reward for aligning the selection ratio discrepancy is scaled by: r1 = ( |ΣI(π(x))/N − s_r| / s_r if ΣI(π(x))/N < s_r, |ΣI(π(x))/N − s_r| / (1−s_r) if ΣI(π(x))/N ≥ s_r, (5) ... r2 = Ec(x) ⊙ π(x), (7). ... Without using RL, we employ a pretrained model to estimate the degree of ϵ-coverage. We select samples with the smallest degree of ϵ-coverage."

    Flagged per the manuscript-reviewing rule as missing support in the claimed derivation chain, not as a fitted-input inversion: since A2C maximizes r = r1 + r2, Eq. (5) as printed rewards a larger ratio discrepancy, pulling the selection ratio away from the target s_r — the opposite of 'aligning the current selection ratio with the target one.' Independently, the Section 5.5 ablation keeps the samples with the smallest E_c, i.e., by Eq. (6) the samples closest to their classmates (the ones Def. 1 calls ε-covered), whereas r2 rewards selecting large-E_c samples; the two directions are complements, and the ablation implements the stated pruning logic backwards.

full rationale

The paper's headline theoretical claim is that Props. 1–3 prove ε-covered samples are redundant and can be safely pruned (Contribution 2, Section 1). Inspection of the proofs shows each proposition is obtained by feeding the defining inequality of Def. 1 (‖x̃_i − x̃_j‖ ≤ ε) through Hölder/norm inequalities on a linearized model: outputs differ by ≤ ε‖w‖, losses differ by → 0, gradient norms ≤ ε(‖x̃‖‖H‖+‖g‖). This is continuity — 'close features ⇒ close behavior' is what 'close features' means for smooth maps. No step of the proof establishes that deleting a covered sample leaves the ERM solution or generalization unchanged; that step (C.3, Contribution 2) is an assumption. The assumption is then loaded into the reward: r2 = E_c ⊙ π with E_c defined as the unthresholded sum of the same pairwise feature distances (Eq. 6 vs. Def. 1), and the empirical gains (Fig. 2, Tables 3–5) are presented as validating the redundancy premise — reading the input of r2 back as a confirmed prediction. Two printed-text defects deserve note per the reviewing rule: (a) Eq. (5) defines r1 as the positive ratio discrepancy and the agent maximizes r1+r2, which pushes the selection ratio away from s_r rather than aligning it; (b) the no-RL ablation in Section 5.5 selects the samples with the SMALLEST E_c, which by Eq. (6) are the samples closest to their classmates (the ones Def. 1 designates as ε-covered), while r2 rewards the opposite direction. The mechanism text ('encourages pruning samples with a high degree of ε coverage') cannot be traced through the equations as printed; if the implemented code uses opposite signs, the equations are misprinted, but as written the support is missing or inverted. Corrective context: there is no load-bearing self-citation (the authors' earlier works [24, 74–79] are not used to justify the ε-cover concept or the reward), and the benchmark results are independent external evidence (test accuracy on held-out sets, cross-architecture transfer). Hence the paper is not wholly circular; but the central formal 'derivation' reduces to its definition, and the pruning-safety claim is an assumed input to the reward rather than a derived output, placing the analysis at partial circularity.

Assumptions & free parameters 3 free parameters · 5 assumptions · 1 invented entities

The central method relies on a small set of hand-made choices: a fixed binarization threshold, a fixed A2C architecture, and a 1% selection-ratio tolerance. The formal results depend on linear-model and smoothness assumptions, on an ad hoc sum-of-distances proxy for coverage, and on an often-false positive definite Hessian assumption. The only independent check is standard test-set accuracy, which supports the method empirically but does not validate the redundancy theory.

free parameters (3)
  • binarization threshold for policy scores = 0.5
    The continuous policy scores are binarized at 0.5 to decide selection; this fixed threshold is a hand-chosen design decision that affects the final subset.
  • A2C network widths = 512, 256, 1 for both actor and critic
    The architecture is chosen without an ablation study; the paper states only that the networks are lightweight, with three linear layers each.
  • selection-ratio tolerance = plus or minus 1% around expected ratio
    The paper constrains the achieved selection ratio to within 1% of the target; this hand-set tolerance controls how strictly the ratio is enforced.
assumptions (5)
  • domain assumption A ReLU network can be approximated by a linear model.
    Section 3, Preliminary, justifies the derivation of Props. 1-3; this approximation is not validated for finite feature distances or deep networks.
  • domain assumption Feature-space closeness propagates to output and gradient closeness.
    Props. 1-3 assume that if two feature vectors are close, logits, losses, and gradients are close; this requires Lipschitz smoothness of the network and loss that is not established for ReLU networks.
  • ad hoc to paper The sum of pairwise feature distances estimates the degree of epsilon-cover.
    Eq. (6) replaces the binary cover relation of Definition 1 with an unthresholded sum of all distances; no derivation connects this sum to the epsilon-cover concept.
  • domain assumption The dataset has limited noise.
    Section 13 states that the method assumes the amount of noise in datasets is limited; if noisy samples dominate, covered samples are not necessarily safe to prune.
  • domain assumption The Hessian of the loss is positive definite and invertible.
    Proposition 4 and Eq. (10) use the influence function with H^{-1}; for overparameterized neural networks the Hessian is typically singular, so the bound may not hold.
invented entities (1)
  • epsilon-sample cover
    purpose: Quantify sample redundancy from pairwise feature distances and serve as the RL reward signal.
    The concept is defined by this paper; no external measurement or falsifiable prediction validates it as the true redundancy measure beyond the paper's own benchmark results.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RL-Selector: Reinforcement Learning-Guided Data Selection via Redundancy Assessment." pith.science (2026). https://pith.science/paper/44VSRTPB

@misc{pith2026250621037,
  author       = {Pith},
  title        = {Pith review of: RL-Selector: Reinforcement Learning-Guided Data Selection via Redundancy Assessment},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/44VSRTPB}},
  note         = {Machine review of arXiv:2506.21037}
}
read the original abstract

Modern deep architectures often rely on large-scale datasets, but training on these datasets incurs high computational and storage overhead. Real-world datasets often contain substantial redundancies, prompting the need for more data-efficient training paradigms. Data selection has shown promise to mitigate redundancy by identifying the most representative samples, thereby reducing training costs without compromising performance. Existing methods typically rely on static scoring metrics or pretrained models, overlooking the combined effect of selected samples and their evolving dynamics during training. We introduce the concept of epsilon-sample cover, which quantifies sample redundancy based on inter-sample relationships, capturing the intrinsic structure of the dataset. Based on this, we reformulate data selection as a reinforcement learning (RL) process and propose RL-Selector, where a lightweight RL agent optimizes the selection policy by leveraging epsilon-sample cover derived from evolving dataset distribution as a reward signal. Extensive experiments across benchmark datasets and diverse architectures demonstrate that our method consistently outperforms existing state-of-the-art baselines. Models trained with our selected datasets show enhanced generalization performance with improved training efficiency.

Figures

Figures reproduced from arXiv: 2506.21037 by the authors.

Figure 1
Figure 1. Our framework utilizes a target model fθ and an A2C policy model to optimize the selection strategy based on specific selection ratios. fθ is used to capture the evolving training dynamics and will not be used for subsequent tasks. The adaptively optimized policy π ∗ ultimately determines the selected data. 4. The Proposed Method Overview. Through theoretical analyses, redundancy is observed among mutually-covered s… view at source ↗
Figure 2
Figure 2. Comparisons of our method with various data selection baselines on CIFAR-100 (a), Tiny-ImageNet (b), and ImageNet-1k (c) [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Selection costs vs. perfor￾mance. Results are reported with ResNet￾50 under 40% selection ratio on a 2 NVIDIA RTX2080TI GPUs server [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Illustrations of the selected results using CIFAR-10. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

102 extracted references · 55 canonical work pages

  1. [1]

    A definition of continual reinforcement learning

    David Abel, Andre Barreto, Benjamin Van Roy, Doina Pre- cup, Hado van Hasselt, and Satinder Singh. A definition of continual reinforcement learning. InThirty-seventh Confer- ence on Neural Information Processing Systems, 2023

    2023

  2. [2]

    Vlmo: Unified vision-language pre-training with mixture-of-modality-experts.Advances in Neural Information Processing Systems, 35:32897–32912, 2022

    Hangbo Bao, Wenhui Wang, Li Dong, Qiang Liu, Owais Khan Mohammed, Kriti Aggarwal, Subhojit Som, Songhao Piao, and Furu Wei. Vlmo: Unified vision-language pre-training with mixture-of-modality-experts.Advances in Neural Information Processing Systems, 35:32897–32912, 2022

    2022

  3. [3]

    Scail: Classifier weights scaling for class incremental learning

    Eden Belouadah and Adrian Popescu. Scail: Classifier weights scaling for class incremental learning. InProceed- ings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1266–1275, 2020

    2020

  4. [4]

    Efros, and Jun-Yan Zhu

    George Cazenavette, Tongzhou Wang, Antonio Torralba, Alexei A. Efros, and Jun-Yan Zhu. Dataset distillation by matching training trajectories. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pages 4750–4759, 2022

    2022

  5. [5]

    Why Adversarial Training of ReLU Networks Is Difficult?

    Xu Cheng, Hao Zhang, Yue Xin, Wen Shen, Jie Ren, and Quanshi Zhang. Why adversarial training of relu networks is difficult?arXiv preprint arXiv:2205.15130, 2022

    work page Pith review arXiv 2022

  6. [6]

    Palm: Scaling language modeling with pathways.Journal of Machine Learning Research, 24(240): 1–113, 2023

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways.Journal of Machine Learning Research, 24(240): 1–113, 2023

    2023

  7. [7]

    A downsampled variant of imagenet as an alternative to the ci- far datasets.arXiv preprint arXiv:1707.08819, 2017

    Patryk Chrabaszcz, Ilya Loshchilov, and Frank Hutter. A downsampled variant of imagenet as an alternative to the ci- far datasets.arXiv preprint arXiv:1707.08819, 2017

    arXiv 2017

  8. [8]

    Selection via proxy: Efficient data se- lection for deep learning.arXiv preprint arXiv:1906.11829, 2019

    Cody Coleman, Christopher Yeh, Stephen Mussmann, Baha- ran Mirzasoleiman, Peter Bailis, Percy Liang, Jure Leskovec, and Matei Zaharia. Selection via proxy: Efficient data se- lection for deep learning.arXiv preprint arXiv:1906.11829, 2019

    arXiv 1906

Show all 102 references
  1. [9]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009

    2009

  2. [10]

    An image is worth 16x16 words: Trans- formers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale.arXiv preprint ar...

    2010 arXiv

  3. [11]

    Minimizing the accumulated trajectory error to improve dataset distillation

    Jiawei Du, Yidi Jiang, Vincent YF Tan, Joey Tianyi Zhou, and Haizhou Li. Minimizing the accumulated trajectory error to improve dataset distillation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3749–3758, 2023

    2023

  4. [12]

    J. C. Dunn. A fuzzy relative of the isodata process and its use in detecting compact well-separated clusters.Journal of Cybernetics, 3(3):32–57, 1973

    1973

  5. [13]

    What neural networks memorize and why: Discovering the long tail via influence estimation.Advances in Neural Information Processing Sys- tems, 33:2881–2891, 2020

    Vitaly Feldman and Chiyuan Zhang. What neural networks memorize and why: Discovering the long tail via influence estimation.Advances in Neural Information Processing Sys- tems, 33:2881–2891, 2020

    2020

  6. [14]

    Vissl.https://github.com/ facebookresearch/vissl, 2021

    Priya Goyal, Quentin Duval, Jeremy Reizenstein, Matthew Leavitt, Min Xu, Benjamin Lefaudeux, Mannat Singh, Vinicius Reis, Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Ishan Misra. Vissl.https://github.com/ facebookresearch/vissl, 2021

    2021

  7. [15]

    Ivo Grondman, Lucian Busoniu, Gabriel AD Lopes, and Robert Babuska. A survey of actor-critic reinforcement learning: Standard and natural policy gradients.IEEE Trans- actions on Systems, Man, and Cybernetics, Part C (Applica- tions and Reviews), 42(6):1291–1307, 2012

    2012

  8. [16]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  9. [17]

    Towards lossless dataset distillation via difficulty-aligned trajectory matching.arXiv preprint arXiv:2310.05773, 2023

    Ziyao Guo, Kai Wang, George Cazenavette, Hui Li, Kaipeng Zhang, and Yang You. Towards lossless dataset distillation via difficulty-aligned trajectory matching.arXiv preprint arXiv:2310.05773, 2023

    2023 arXiv

  10. [18]

    Data-efficient training of cnns and trans- formers with coresets: A stability perspective.arXiv preprint arXiv:2303.02095, 2023

    Animesh Gupta, Irtiza Hassan, Dilip K Prasad, and Deepak K Gupta. Data-efficient training of cnns and trans- formers with coresets: A stability perspective.arXiv preprint arXiv:2303.02095, 2023

    2023 arXiv

  11. [19]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pages 770– 778, 2016

    2016

  12. [20]

    You only con- dense once: Two rules for pruning condensed datasets.Ad- vances in Neural Information Processing Systems, 36, 2024

    Yang He, Lingao Xiao, and Joey Tianyi Zhou. You only con- dense once: Two rules for pruning condensed datasets.Ad- vances in Neural Information Processing Systems, 36, 2024

    2024

  13. [21]

    The many faces of robust- ness: A critical analysis of out-of-distribution generalization

    Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kada- vath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robust- ness: A critical analysis of out-of-distribution generalization. InProceedings of the IEEE/CVF internationa...

    2021

  14. [22]

    Natural adversarial examples

    Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Stein- hardt, and Dawn Song. Natural adversarial examples. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 15262–15271, 2021

    2021

  15. [23]

    Diversified batch selection for training acceleration.arXiv preprint arXiv:2406.04872, 2024

    Feng Hong, Yueming Lyu, Jiangchao Yao, Ya Zhang, Ivor W Tsang, and Yanfeng Wang. Diversified batch selection for training acceleration.arXiv preprint arXiv:2406.04872, 2024

    2024 arXiv

  16. [24]

    Donod: Robust and generalizable instruction fine-tuning for llms via model-intrinsic dataset pruning.arXiv preprint arXiv:2504.14810, 2025

    Jucheng Hu, Surong Yang, Dongzhan Zhou, and Lijun Wu. Donod: Robust and generalizable instruction fine-tuning for llms via model-intrinsic dataset pruning.arXiv preprint arXiv:2504.14810, 2025

    2025 arXiv

  17. [25]

    Densely connected convolutional net- works

    Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kil- ian Q Weinberger. Densely connected convolutional net- works. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 4700–4708, 2017

    2017

  18. [26]

    Polynomial bounds for vc dimension of sigmoidal and general pfaffian neural networks.Journal of Computer and System Sciences, 54(1): 169–176, 1997

    Marek Karpinski and Angus Macintyre. Polynomial bounds for vc dimension of sigmoidal and general pfaffian neural networks.Journal of Computer and System Sciences, 54(1): 169–176, 1997

    1997

  19. [27]

    Grad-match: Gradient matching based data subset selection for efficient deep model training

    Krishnateja Killamsetty, S Durga, Ganesh Ramakrishnan, Abir De, and Rishabh Iyer. Grad-match: Gradient matching based data subset selection for efficient deep model training. InInternational Conference on Machine Learning, pages 5464–5474. PMLR, 2021

    2021

  20. [28]

    Glister: Generalization based data subset selection for efficient and robust learning

    Krishnateja Killamsetty, Durga Sivasubramanian, Ganesh Ramakrishnan, and Rishabh Iyer. Glister: Generalization based data subset selection for efficient and robust learning. InProceedings of the AAAI Conference on Artificial Intelli- gence, pages 8110–8118, 2021

    2021

  21. [29]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. InProceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4015–4026, 2023

    2023

  22. [30]

    Understanding black-box predictions via influence functions

    Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. InProceedings of the 34th International Conference on Machine Learning - Vol- ume 70, page 1885–1894. JMLR.org, 2017

    2017

  23. [31]

    Understanding black-box predictions via influence functions

    Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. InInternational confer- ence on machine learning, pages 1885–1894. PMLR, 2017

    2017

  24. [32]

    Prism: A unified framework of parameterized submodular information measures for tar- geted data subset selection and summarization

    Suraj Kothawade, Vishal Kaushal, Ganesh Ramakrishnan, Jeff Bilmes, and Rishabh Iyer. Prism: A unified framework of parameterized submodular information measures for tar- geted data subset selection and summarization. InThirty- Sixth AAAI Conference on Artificial Intelligence, ...

    2022

  25. [33]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009

    2009

  26. [34]

    Dr3: Value- based deep reinforcement learning requires explicit regular- ization.arXiv preprint arXiv:2112.04716, 2021

    Aviral Kumar, Rishabh Agarwal, Tengyu Ma, Aaron Courville, George Tucker, and Sergey Levine. Dr3: Value- based deep reinforcement learning requires explicit regular- ization.arXiv preprint arXiv:2112.04716, 2021

    2021 arXiv

  27. [35]

    Super- vised pretraining can learn in-context reinforcement learn- ing.Advances in Neural Information Processing Systems, 36, 2024

    Jonathan Lee, Annie Xie, Aldo Pacchiano, Yash Chandak, Chelsea Finn, Ofir Nachum, and Emma Brunskill. Super- vised pretraining can learn in-context reinforcement learn- ing.Advances in Neural Information Processing Systems, 36, 2024

    2024

  28. [36]

    A comprehensive survey to dataset distillation.arXiv preprint arXiv:2301.05603, 2023

    Shiye Lei and Dacheng Tao. A comprehensive survey to dataset distillation.arXiv preprint arXiv:2301.05603, 2023

    2023 arXiv

  29. [37]

    Align before fuse: Vision and language representation learn- ing with momentum distillation.Advances in neural infor- mation processing systems, 34:9694–9705, 2021

    Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. Align before fuse: Vision and language representation learn- ing with momentum distillation.Advances in neural infor- mation processing systems, 34:9694–9705, 2021

    2021

  30. [38]

    Design from policies: Con- servative test-time adaptation for offline policy optimiza- tion.Advances in Neural Information Processing Systems, 36, 2024

    Jinxin Liu, Hongyin Zhang, Zifeng Zhuang, Yachen Kang, Donglin Wang, and Bin Wang. Design from policies: Con- servative test-time adaptation for offline policy optimiza- tion.Advances in Neural Information Processing Systems, 36, 2024

    2024

  31. [39]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021

    2021

  32. [40]

    Struc- tured state space models for in-context reinforcement learn- ing.Advances in Neural Information Processing Systems, 36, 2024

    Chris Lu, Yannick Schroecker, Albert Gu, Emilio Parisotto, Jakob Foerster, Satinder Singh, and Feryal Behbahani. Struc- tured state space models for in-context reinforcement learn- ing.Advances in Neural Information Processing Systems, 36, 2024

    2024

  33. [41]

    D2 pruning: Message passing for balancing diversity and dif- ficulty in data pruning.arXiv preprint arXiv:2310.07931, 2023

    Adyasha Maharana, Prateek Yadav, and Mohit Bansal. D2 pruning: Message passing for balancing diversity and dif- ficulty in data pruning.arXiv preprint arXiv:2310.07931, 2023

    2023 arXiv

  34. [42]

    Language models are few-shot learners.arXiv preprint arXiv:2005.14165, 2020

    Ben Mann, N Ryder, M Subbiah, J Kaplan, P Dhariwal, A Neelakantan, P Shyam, G Sastry, A Askell, S Agarwal, et al. Language models are few-shot learners.arXiv preprint arXiv:2005.14165, 2020

    2005 arXiv

  35. [43]

    Schulze Buschoff, Robert Geirhos, and Felix A

    Kristof Meding, Luca M. Schulze Buschoff, Robert Geirhos, and Felix A. Wichmann. Trivial or impossible — dichoto- mous data difficulty masks model differences (on imagenet and beyond). InInternational Conference on Learning Rep- resentations, 2022

    2022

  36. [44]

    Coresets for data-efficient training of machine learning mod- els

    Baharan Mirzasoleiman, Jeff Bilmes, and Jure Leskovec. Coresets for data-efficient training of machine learning mod- els. InInternational Conference on Machine Learning, pages 6950–6960. PMLR, 2020

    2020

  37. [45]

    Asynchronous methods for deep rein- forcement learning.arXiv preprint arXiv:1602.01783, 2016

    V olodymyr Mnih. Asynchronous methods for deep rein- forcement learning.arXiv preprint arXiv:1602.01783, 2016

    2016 arXiv

  38. [46]

    Asynchronous methods for deep reinforcement learning

    V olodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learning. InInternational conference on ma- chine learning, pages 1928–1937. PMLR, 2016

    1928

  39. [47]

    Performance bounds for policy-based average reward rein- forcement learning algorithms.Advances in Neural Infor- mation Processing Systems, 36:19386–19396, 2023

    Yashaswini Murthy, Mehrdad Moharrami, and R Srikant. Performance bounds for policy-based average reward rein- forcement learning algorithms.Advances in Neural Infor- mation Processing Systems, 36:19386–19396, 2023

    2023

  40. [48]

    Bridging the gap between value and policy based reinforcement learning.Advances in neural informa- tion processing systems, 30, 2017

    Ofir Nachum, Mohammad Norouzi, Kelvin Xu, and Dale Schuurmans. Bridging the gap between value and policy based reinforcement learning.Advances in neural informa- tion processing systems, 30, 2017

    2017

  41. [49]

    Data valu- ation without training of a model

    Ki Nohyun, Hoyong Choi, and Hye Won Chung. Data valu- ation without training of a model. InThe Eleventh Interna- tional Conference on Learning Representations, 2023

    2023

  42. [50]

    Deep learning on a data diet: Finding important ex- amples early in training.Advances in Neural Information Processing Systems, 34:20596–20607, 2021

    Mansheej Paul, Surya Ganguli, and Gintare Karolina Dziu- gaite. Deep learning on a data diet: Finding important ex- amples early in training.Advances in Neural Information Processing Systems, 34:20596–20607, 2021

    2021

  43. [51]

    Adaptive second order coresets for data-efficient machine learning

    Omead Pooladzandi, David Davini, and Baharan Mirza- soleiman. Adaptive second order coresets for data-efficient machine learning. InInternational Conference on Machine Learning, pages 17848–17869. PMLR, 2022

    2022

  44. [52]

    Infobatch: Lossless training speed up by unbiased dynamic data pruning.arXiv preprint arXiv:2303.04947, 2023

    Ziheng Qin, Kai Wang, Zangwei Zheng, Jianyang Gu, Xi- angyu Peng, Zhaopan Xu, Daquan Zhou, Lei Shang, Baigui Sun, Xuansong Xie, et al. Infobatch: Lossless training speed up by unbiased dynamic data pruning.arXiv preprint arXiv:2303.04947, 2023

    2023 arXiv

  45. [53]

    Language models are unsu- pervised multitask learners.OpenAI blog, 1(8):9, 2019

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsu- pervised multitask learners.OpenAI blog, 1(8):9, 2019

    2019

  46. [54]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational conference on machine learning, p...

    2021

  47. [55]

    Accelerat- ing deep learning with dynamic data pruning.arXiv preprint arXiv:2111.12621, 2021

    Ravi S Raju, Kyle Daruwalla, and Mikko Lipasti. Accelerat- ing deep learning with dynamic data pruning.arXiv preprint arXiv:2111.12621, 2021

    2021 arXiv

  48. [56]

    Data-centric green artifi- cial intelligence: A survey.IEEE Transactions on Artificial Intelligence, 2023

    Shirin Salehi and Anke Schmeink. Data-centric green artifi- cial intelligence: A survey.IEEE Transactions on Artificial Intelligence, 2023

    2023

  49. [57]

    Laion-5b: An open large-scale dataset for training next generation image-text models.Advances in Neural In- formation Processing Systems, 35:25278–25294, 2022

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, et al. Laion-5b: An open large-scale dataset for training next generation image-text models.Advances in Neural In- fo...

    2022

  50. [58]

    Active learning for convolu- tional neural networks: A core-set approach

    Ozan Sener and Silvio Savarese. Active learning for convolu- tional neural networks: A core-set approach. InInternational Conference on Learning Representations, 2018

    2018

  51. [59]

    Reinforcement learning algorithms: A brief survey.Expert Systems with Applications, page 120495, 2023

    Ashish Kumar Shakya, Gopinatha Pillai, and Sohom Chakrabarty. Reinforcement learning algorithms: A brief survey.Expert Systems with Applications, page 120495, 2023

    2023

  52. [60]

    Very deep convo- lutional networks for large-scale image recognition.arXiv preprint arXiv:1409.1556, 2014

    Karen Simonyan and Andrew Zisserman. Very deep convo- lutional networks for large-scale image recognition.arXiv preprint arXiv:1409.1556, 2014

    2014 arXiv

  53. [61]

    Ben Sorscher, Robert Geirhos, Shashank Shekhar, Surya Ganguli, and Ari S. Morcos. Beyond neural scaling laws: beating power law scaling via data pruning. InAdvances in Neural Information Processing Systems, 2022

    2022

  54. [62]

    On the depth of deep neural networks: A the- oretical view

    Shizhao Sun, Wei Chen, Liang Wang, Tie-Yan Liu, and Dacheng Tao. On the depth of deep neural networks: A the- oretical view. InProceedings of the AAAI Conference on Artificial Intelligence, 2016

    2016

  55. [63]

    Mohammad Reza Taesiri, Giang Nguyen, Sarra Habchi, Cor- Paul Bezemer, and Anh Nguyen. Imagenet-hard: The hard- est images remaining from a study of the power of zoom and spatial biases in image classification.Advances in Neural Information Processing Systems, 36, 2024

    2024

  56. [64]

    Data pruning via moving-one- sample-out.Advances in Neural Information Processing Sys- tems, 36, 2024

    Haoru Tan, Sitong Wu, Fei Du, Yukang Chen, Zhibin Wang, Fan Wang, and Xiaojuan Qi. Data pruning via moving-one- sample-out.Advances in Neural Information Processing Sys- tems, 36, 2024

    2024

  57. [65]

    D4: Improving llm pretraining via document de- duplication and diversification

    Kushal Tirumala, Daniel Simig, Armen Aghajanyan, and Ari Morcos. D4: Improving llm pretraining via document de- duplication and diversification. InAdvances in Neural In- formation Processing Systems, pages 53983–53995. Curran Associates, Inc., 2023

    2023

  58. [66]

    Policy-based reinforcement learning for generalisation in interactive text- based environments

    Edan Toledo, Jan Buys, and Jonathan Shock. Policy-based reinforcement learning for generalisation in interactive text- based environments. InProceedings of the 17th Conference of the European Chapter of the Association for Computa- tional Linguistics, pages 1230–1242, 2023

    2023

  59. [67]

    An empirical study of example forget- ting during deep neural network learning.arXiv preprint arXiv:1812.05159, 2018

    Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geof- frey J Gordon. An empirical study of example forget- ting during deep neural network learning.arXiv preprint arXiv:1812.05159, 2018

    2018 arXiv

  60. [68]

    Visualizing data using t-sne.J

    Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne.J. Machine Learning Research, 9(11), 2008

    2008

  61. [69]

    Submodularity in data subset selection and active learning

    Kai Wei, Rishabh Iyer, and Jeff Bilmes. Submodularity in data subset selection and active learning. InInternational conference on machine learning, pages 1954–1963. PMLR, 2015

    1954

  62. [70]

    Herding dynamical weights to learn

    Max Welling. Herding dynamical weights to learn. InPro- ceedings of the 26th Annual International Conference on Machine Learning, pages 1121–1128, 2009

    2009

  63. [71]

    Scalable trust-region method for deep re- inforcement learning using kronecker-factored approxima- tion.Advances in neural information processing systems, 30, 2017

    Yuhuai Wu, Elman Mansimov, Roger B Grosse, Shun Liao, and Jimmy Ba. Scalable trust-region method for deep re- inforcement learning using kronecker-factored approxima- tion.Advances in neural information processing systems, 30, 2017

    2017

  64. [72]

    Moderate coreset: A universal method of data selection for real-world data-efficient deep learning

    Xiaobo Xia, Jiale Liu, Jun Yu, Xu Shen, Bo Han, and Tongliang Liu. Moderate coreset: A universal method of data selection for real-world data-efficient deep learning. In The Eleventh International Conference on Learning Repre- sentations, 2023

    2023

  65. [73]

    Dataset pruning: Reducing training data by ex- amining generalization influence

    Shuo Yang, Zeke Xie, Hanyu Peng, Min Xu, Mingming Sun, and Ping Li. Dataset pruning: Reducing training data by ex- amining generalization influence. InInternational Confer- ence on Learning Representations, 2023

    2023

  66. [74]

    Not all data matters: An end-to-end adap- tive dataset pruning framework for enhancing model per- formance and efficiency.arXiv preprint arXiv:2312.05599, 2023

    Suorong Yang, Hongchao Yang, Suhan Guo, Furao Shen, and Jian Zhao. Not all data matters: An end-to-end adap- tive dataset pruning framework for enhancing model per- formance and efficiency.arXiv preprint arXiv:2312.05599, 2023

    2023 arXiv

  67. [75]

    In- vestigating the effectiveness of data augmentation from simi- larity and diversity: An empirical study.Pattern Recognition, 148:110204, 2024

    Suorong Yang, Suhan Guo, Jian Zhao, and Furao Shen. In- vestigating the effectiveness of data augmentation from simi- larity and diversity: An empirical study.Pattern Recognition, 148:110204, 2024

    2024

  68. [76]

    Adaaugment: A tuning-free and adaptive approach to enhance data augmentation.arXiv preprint arXiv:2405.11467, 2024

    Suorong Yang, Peijia Li, Xin Xiong, Furao Shen, and Jian Zhao. Adaaugment: A tuning-free and adaptive approach to enhance data augmentation.arXiv preprint arXiv:2405.11467, 2024

    2024 arXiv

  69. [77]

    Entaugment: Entropy-driven adaptive data augmentation framework for image classification

    Suorong Yang, Furao Shen, and Jian Zhao. Entaugment: Entropy-driven adaptive data augmentation framework for image classification. InEuropean Conference on Computer Vision, pages 197–214. Springer, 2024

    2024

  70. [78]

    A clip-powered framework for ro- bust and generalizable data selection.arXiv preprint arXiv:2410.11215, 2024

    Suorong Yang, Peng Ye, Wanli Ouyang, Dongzhan Zhou, and Furao Shen. A clip-powered framework for ro- bust and generalizable data selection.arXiv preprint arXiv:2410.11215, 2024

    2024 arXiv

  71. [79]

    When dynamic data selection meets data augmentation

    Suorong Yang, Peng Ye, Furao Shen, and Dongzhan Zhou. When dynamic data selection meets data augmentation. arXiv preprint arXiv:2505.03809, 2025

    2025 arXiv

  72. [80]

    Nearly op- timal vc-dimension and pseudo-dimension bounds for deep neural network derivatives

    Yahong Yang, Haizhao Yang, and Yang Xiang. Nearly op- timal vc-dimension and pseudo-dimension bounds for deep neural network derivatives. InAdvances in Neural Informa- tion Processing Systems, 2024

    2024

  73. [81]

    Zishun Yu, Yunzhe Tao, Liyu Chen, Tao Sun, and Hongxia Yang.B-coder: Value-based deep reinforcement learning for program synthesis.arXiv preprint arXiv:2310.03173, 2023

    2023 arXiv

  74. [82]

    Metalight: Value-based meta-reinforcement learning for traffic signal control

    Xinshi Zang, Huaxiu Yao, Guanjie Zheng, Nan Xu, Kai Xu, and Zhenhui Li. Metalight: Value-based meta-reinforcement learning for traffic signal control. InProceedings of the AAAI conference on artificial intelligence, pages 1153–1160, 2020

    2020

  75. [83]

    Accelerating dataset distillation via model augmenta- tion

    Lei Zhang, Jie Zhang, Bowen Lei, Subhabrata Mukherjee, Xiang Pan, Bo Zhao, Caiwen Ding, Yao Li, and Dongkuan Xu. Accelerating dataset distillation via model augmenta- tion. InProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 11950–11959, 2023

    2023

  76. [84]

    Spanning training progress: Temporal dual-depth scoring (tdds) for enhanced dataset pruning

    Xin Zhang, Jiawei Du, Yunsong Li, Weiying Xie, and Joey Tianyi Zhou. Spanning training progress: Temporal dual-depth scoring (tdds) for enhanced dataset pruning. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 26223–26232, 2024

    2024

  77. [85]

    Selectivity drives productivity: Efficient dataset prun- ing for enhanced transfer learning.Advances in Neural In- formation Processing Systems, 36, 2024

    Yihua Zhang, Yimeng Zhang, Aochuan Chen, Jiancheng Liu, Gaowen Liu, Mingyi Hong, Shiyu Chang, Sijia Liu, et al. Selectivity drives productivity: Efficient dataset prun- ing for enhanced transfer learning.Advances in Neural In- formation Processing Systems, 36, 2024

    2024

  78. [86]

    Coverage-centric coreset selection for high pruning rates

    Haizhong Zheng, Rui Liu, Fan Lai, and Atul Prakash. Coverage-centric coreset selection for high pruning rates. In The Eleventh International Conference on Learning Repre- sentations, 2023

    2023

  79. [87]

    Dataset quantization

    Daquan Zhou, Kai Wang, Jianyang Gu, Xiangyu Peng, Dongze Lian, Yifan Zhang, Yang You, and Jiashi Feng. Dataset quantization. InProceedings of the IEEE/CVF In- ternational Conference on Computer Vision, pages 17205– 17216, 2023

    2023

  80. [88]

    Dataset distillation using neural feature regression.arXiv preprint arXiv:2206.00719, 2022

    Yongchao Zhou, Ehsan Nezhadarya, and Jimmy Ba. Dataset distillation using neural feature regression.arXiv preprint arXiv:2206.00719, 2022. RL-Selector: Reinforcement Learning-Guided Data Selection via Redundancy Assessment Supplementary Material

    2022 arXiv

  81. [89]

    ˆyi − ˆyj =∥w(x i −x j)∥(12) ≤ ∥w∥ ∥xi −x j∥(13) ≤ϵ∥w∥(14) (15) In the above, the Inequality(13)follows from H ¨older’s inequality

    Proof of Proposition 1 Proof 1For simplicity, we approximate the fully connected layer as a linear model, i.e., ˆyi =wx i +b. ˆyi − ˆyj =∥w(x i −x j)∥(12) ≤ ∥w∥ ∥xi −x j∥(13) ≤ϵ∥w∥(14) (15) In the above, the Inequality(13)follows from H ¨older’s inequality. For a given model w...

  82. [90]

    Proof of Proposition 2 Proof 2Letx i andx j are two samples, andxi isϵ-covered byx j. The cross-entropy loss ofx i is given as loss(xi) =L(wx i +b) =− KX c=1 yic log(wxi +b) c.(16) Letˆyic = (wx i +b) c, thenloss(x i)−loss(x j)can be computed as loss(xi)−loss(x j) =L(wx i +b)−...

  83. [91]

    Proof of Lemma 1 Proof 3Without loss of generality, the gradient of loss for xw.r.t. the weightW h of theh-th intermediate layer is given as gWh = ( ∂ ∂W T h L(fθ(x), y))T .(22) According to the chain rule, the gradient of thei-th element of the weightW h can be written as gW ...

  84. [92]

    Proof of Proposition 3 Proof 4According to Eq.(29), we have ∥∆gWh ∥= ˜xi,h∆xT H T ˜x + ∆xgT ˜x (30) ≤ ˜xi,h∆xT H T ˜x + ∆xgT ˜x (31) ≤ ∥∆x∥(∥˜xi,h∥ ∥H˜x∥+∥g ˜x∥).(32) Given a modelf θ, since∥∆x∥ ≤ϵ, the change of the gra- dient can be formulated as follows, lim ϵ→0 ∥∆gWh ∥= 0. (33)

  85. [93]

    Proof of Proposition 4 Proof 5According to Lemma 2,∀x ′ ∈ −ˆD, we can find ax∈ Dsuch thatx ′ isϵ-covered byx. The gap in the parameter changes betweenx ′ andxcan be estimated as follows ∆Iup,params =I up,params(x)− Iup,params(x′)(34) =−(H −1 ˆθ gx θ −H −1 ˆθ gx′ θ )(35) =H −1 ...

  86. [94]

    The total epoch is 200, and no warm-up schedule is used

    The General Workflow of Our Proposed Method To better understand the workflow of our proposed method, we summarize the detailed algorithm in Algo- rithm 1. The total epoch is 200, and no warm-up schedule is used. The Adam optimizer is used with a weight decay of1e−4and an init...

  87. [95]

    The A2C network architecture details

    Choice of A2C Network Table 6. The A2C network architecture details. index Layer Dimension Actor 1 Linear (512, 512) 2 Linear (512, 256) 3 Linear (256, 1) Critic 1 Linear (512, 512) 2 Linear (512, 256) 3 Linear (256, 1) In Table 6, we provide the details of the A2C network. Bo...

  88. [96]

    The complexities of the first two steps areO N 2 k d andO N 2 k , respectively, whereN k is the number of sam- ples in classkanddis the feature dimension (e.g., 512 for ResNet-18)

    Complexity Analysis According to the algorithm pipeline in Algorithm 1, the main computational costs can be divided into three compo- nents: 1) distance metric calculation, 2) degree ofϵ-cover calculation, and 3) the RL network forward passes and up- dates. The complexities of...

  89. [97]

    Implementation Details Our RL module follows the design and parameter settings from [45], without introducing any additional hyperparam- eters. For experiments on CIFAR-10 and CIFAR-100, fol- lowing [72, 77, 86], we train ResNet-50 models for 200 epochs with a batch size of 25...

  90. [98]

    For experiments on Tiny-ImageNet, following [72], we adopt a batch size of 256, an SGD opti- mizer with a momentum of 0.9, weight decay of 1e-4, and an initial learning rate of 0.1

    Data augmentation of random crop and random hori- zontal flip is added. For experiments on Tiny-ImageNet, following [72], we adopt a batch size of 256, an SGD opti- mizer with a momentum of 0.9, weight decay of 1e-4, and an initial learning rate of 0.1. The learning rate is di...

  91. [99]

    Specifically, once the selected datasets are obtained, only a subset needs to be stored as a replacement for the full dataset, leading to savings in memory costs

    Data Selection Improves Training Effi- ciency Data selection substantially enhances the efficiency of model training efficiency for subsequent tasks. Specifically, once the selected datasets are obtained, only a subset needs to be stored as a replacement for the full dataset, ...

  92. [100]

    As shown in Table 8, our method can achieve superior results

    Experiment Results on CIFAR-10 Due to space constraints, we present experiment results on CIFAR-10. As shown in Table 8, our method can achieve superior results. Notably, with relatively high selection ra- tios (e.g.,≥90%), our method exhibits performance sur- passing that of ...

  93. [101]

    Following [72], the implementation is based on the public Github repository 1, where ViT small is used

    More Experimental Results on Vision Transformer To further demonstrate the superiority of our proposed method, we employ Vision Transformer [10] to train with selected datasets. Following [72], the implementation is based on the public Github repository 1, where ViT small is u...

  94. [102]

    This may limit its ap- plicability to extremely large-scale datasets, which is also a primary challenge for all score-based data selection meth- ods

    Limitation and Future Work First, the proposed method focuses on optimizing sample- wise importance scores for selection. This may limit its ap- plicability to extremely large-scale datasets, which is also a primary challenge for all score-based data selection meth- ods. Futur...

    1996

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.