REVIEW 10 cited by
Dataset Pruning: Reducing Training Data by Examining Generalization Influence
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Dataset Pruning: Reducing Training Data by Examining Generalization Influence
read the original abstract
The great success of deep learning heavily relies on increasingly larger training data, which comes at a price of huge computational and infrastructural costs. This poses crucial questions that, do all training data contribute to model's performance? How much does each individual training sample or a sub-training-set affect the model's generalization, and how to construct the smallest subset from the entire training data as a proxy training set without significantly sacrificing the model's performance? To answer these, we propose dataset pruning, an optimization-based sample selection method that can (1) examine the influence of removing a particular set of training samples on model's generalization ability with theoretical guarantee, and (2) construct the smallest subset of training data that yields strictly constrained generalization gap. The empirically observed generalization gap of dataset pruning is substantially consistent with our theoretical expectations. Furthermore, the proposed method prunes 40% training examples on the CIFAR-10 dataset, halves the convergence time with only 1.3% test accuracy decrease, which is superior to previous score-based sample selection methods.
Forward citations
Cited by 10 Pith papers
-
Knowledge Cascade: Reverse Knowledge Distillation on Nonparametric Multivariate Functional Estimation
KCas transfers student-selected smoothing parameters to full-sample teacher models via asymptotic scaling laws in smoothing splines and kernel methods, cutting computation while retaining performance guarantees.
-
Towards Multimodal Active Learning: Efficient Learning with Limited Paired Data
Introduces the first active learning framework for unaligned multimodal data that selects alignments using uncertainty and diversity to cut annotation costs by up to 40% on benchmarks while preserving accuracy.
-
Representation-Guided Parameter-Efficient LLM Unlearning
REGLU guides LoRA-based unlearning via representation subspaces and orthogonal regularization to outperform prior methods on forget-retain trade-off in LLM benchmarks.
-
OPERA: Online Data Pruning for Efficient Retrieval Model Adaptation
Dynamic hierarchical data pruning improves NDCG@10 and Recall@20 for dense retrievers while reaching full performance in half the iterations.
-
Surprisingly High Redundancy in Electronic Structure Data Across Materials Explained by Low Intrinsic Dimensionality
Electronic structure datasets across materials show high redundancy from low intrinsic dimensionality, allowing pruning to 1/100th size with preserved chemical accuracy.
-
SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation
SalUn uses gradient-based weight saliency to achieve effective machine unlearning of data, classes, or concepts in image classification and generation, narrowing the gap to exact retraining.
-
OrderDP: A Theoretically Guaranteed Lossless Dynamic Data Pruning Framework
OrderDP is a plug-and-play data pruning method that selects a random subset then top-q samples to guarantee unbiased surrogate-loss training with convergence analysis and over 40% training cost reduction on CIFAR and ...
-
SLAP: Stratified Loss-based Pruning for On-Policy Data-Efficient Instruction Tuning
SLAP is a new batch-aware pruning framework that uses distribution-aware stratified sampling and Hessian-approximated gradients to select data, claiming 20-40% less data while matching or exceeding full-dataset perfor...
-
MEGG: Replay via Maximally Extreme GGscore in Incremental Learning for Neural Recommendation Models
A gradient-alignment influence score (GGscore) that selects the highest- and lowest-scoring old interactions for replay improves incremental neural recommendation slightly over random replay, mainly at large replay ratios.
-
Adaptive Data Dropout: Towards Self-Regulated Learning in Deep Neural Networks
Adaptive Data Dropout uses performance feedback to dynamically modulate training data exposure, reducing effective steps while matching static dropout accuracy on image benchmarks.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.