REVIEW 8 cited by
The Evolution of Dataset Distillation: Toward Scalable and Generalizable Solutions
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Dataset distillation, which condenses large-scale datasets into compact synthetic representations, has emerged as a critical solution for training modern deep learning models efficiently. While prior surveys focus on developments before 2023, this work comprehensively reviews recent advances, emphasizing scalability to large-scale datasets such as ImageNet-1K and ImageNet-21K. We categorize progress into a few key methodologies: trajectory matching, gradient matching, distribution matching, scalable generative approaches, and decoupling optimization mechanisms. As a comprehensive examination of recent dataset distillation advances, this survey highlights breakthrough innovations: the SRe2L framework for efficient and effective condensation, soft label strategies that significantly enhance model accuracy, and lossless distillation techniques that maximize compression while maintaining performance. Beyond these methodological advancements, we address critical challenges, including robustness against adversarial and backdoor attacks, effective handling of non-IID data distributions. Additionally, we explore emerging applications in video and audio processing, multi-modal learning, medical imaging, and scientific computing, highlighting its domain versatility. By offering extensive performance comparisons and actionable research directions, this survey equips researchers and practitioners with practical insights to advance efficient and generalizable dataset distillation, paving the way for future innovations.
Forward citations
Cited by 8 Pith papers
-
Efficient Video Dataset Distillation via Cluster-Guided Prototype Blending
A construction-based video distillation pipeline selects teacher-confident clips, allocates slots to feature-space clusters, and blends prototype-anchor pairs with matched soft labels, avoiding gradient updates of sto...
-
Self-Supervised Representation-Guided Generative Dataset Distillation
SRG guides diffusion-based dataset distillation with self-supervised representation prototypes, beating generative baselines for frozen pretrained encoders.
-
Dataset Distillation Based on Saliency-Driven Prototype Alignment
Saliency-guided latent prototypes plus confidence-based hard-prototype refinement improve diffusion-based dataset distillation accuracy on ImageNet subsets, CIFAR, and ImageNet-1K without fine-tuning the generative backbone.
-
Hard Labels In! Rethinking the Role of Hard Labels in Mitigating Local Semantic Drift
A soft-hard-soft training schedule uses hard labels as an intermediate anchor to correct local semantic drift and improves accuracy under 100x-reduced soft-label storage.
-
Dataset Distillation as Data Compression: A Rate-Utility Perspective
A new method frames dataset distillation as joint rate-utility optimization, storing distilled images as entropy-coded latent codes plus tiny decoders, and it reports better storage-accuracy trade-offs than prior methods.
-
Task-Specific Generative Dataset Distillation with Difficulty-Guided Sampling
Difficulty-guided sampling from a generated image pool, aided by a logarithmic distribution correction, yields modest classification accuracy gains in dataset distillation.
-
FADRM: Fast and Accurate Data Residual Matching for Dataset Distillation
A dataset distillation method combining data-level residual connections, mixed precision, and multi-resolution optimization achieves new state-of-the-art accuracy with roughly half the compute.
-
Dataset Distillation via Vision-Language Category Prototype
A dataset distillation method that combines K-means image prototypes with LLM-generated text prototypes to synthesize small, high-accuracy training sets.
Discussion (0). Sign in to comment.