REVIEW 20 cited by
Dataset Condensation with Gradient Matching
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Dataset Condensation with Gradient Matching
read the original abstract
As the state-of-the-art machine learning methods in many fields rely on larger datasets, storing datasets and training models on them become significantly more expensive. This paper proposes a training set synthesis technique for data-efficient learning, called Dataset Condensation, that learns to condense large dataset into a small set of informative synthetic samples for training deep neural networks from scratch. We formulate this goal as a gradient matching problem between the gradients of deep neural network weights that are trained on the original and our synthetic data. We rigorously evaluate its performance in several computer vision benchmarks and demonstrate that it significantly outperforms the state-of-the-art methods. Finally we explore the use of our method in continual learning and neural architecture search and report promising gains when limited memory and computations are available.
Forward citations
Cited by 20 Pith papers
-
Condensing Large-Scale Datasets Directly with Minimal Information Loss
CIM directly aligns data distributions to condense large-scale datasets with minimal information loss, achieving new SOTA results on ImageNet-1K distillation at IPC=10.
-
DMGD: Train-Free Dataset Distillation with Semantic-Distribution Matching in Diffusion Models
DMGD achieves better performance than fine-tuned SOTA methods in dataset distillation on ImageNet subsets by using semantic matching through conditional likelihood optimization and OT-based distribution matching in a ...
-
Direct Discrepancy Replay: Distribution-Discrepancy Condensation and Manifold-Consistent Replay for Continual Face Forgery Detection
A replay method for continual face forgery detection condenses real-fake distribution discrepancies into compact maps and synthesizes compatible samples from current real faces to reduce forgetting under tight memory ...
-
Robust Trajectory Distillation: Hybrid Reweighting Meets Teacher-Inspired Targets
Introduces SGR and TIAT for robust dataset distillation that suppresses noise while preserving knowledge under noisy supervision.
-
Rank-Aware Hyperbolic Alignment for Vision-Language Dataset Distillation
RAHA applies rank-aware hyperbolic alignment to vision-language dataset distillation by enforcing geodesic alignment in the shared low-rank range and regularizing the residual subspace for improved transfer.
-
Pool-Select-Refine for Allocation-Aware Generative Dataset Distillation
Pool-Select-Refine decouples generation, selection, and refinement in diffusion-based dataset distillation to allocate a fixed synthetic sample budget more effectively, delivering consistent gains over prior baselines...
-
Pool-Select-Refine for Allocation-Aware Generative Dataset Distillation
A two-stage framework that decouples generation, selection, and refinement to improve budget use in diffusion-based dataset distillation.
-
Multimodal Distribution Matching for Vision-Language Dataset Distillation
MDM distills vision-language datasets via joint embedding clustering, weight-space model interpolation, and geometry-aware distribution matching on the unit hypersphere.
-
DIVER:Diving Deeper into Distilled Data via Expressive Semantic Recovery
DIVER is a dual-stage distillation method using diffusion models to enhance semantic preservation and cross-architecture generalization in dataset distillation.
-
Fair Dataset Distillation via Cross-Group Barycenter Alignment
Dataset distillation introduces fairness gaps from subgroup pattern mismatches rather than just imbalance; distilling to a group-agnostic barycenter of predictive information reduces these gaps.
-
Omnimodal Dataset Distillation via High-order Proxy Alignment
HoPA captures high-order cross-modal alignments via a shared proxy to enable scalable omnimodal dataset distillation with better performance-compression trade-offs.
-
ROAST: Risk-aware Outlier-exposure for Adversarial Selective Training of Anomaly Detectors Against Evasion Attacks
ROAST selectively trains anomaly detectors on less vulnerable patient data with targeted outlier exposure, boosting recall by 16.2% in black-box settings and reducing training time by 88.3%.
-
Hard Labels In! Rethinking the Role of Hard Labels in Mitigating Local Semantic Drift
A soft-hard-soft training schedule uses hard labels as an intermediate anchor to correct local semantic drift and improves accuracy under 100x-reduced soft-label storage.
-
Distill Once, Adapt Life-Long: Exploring Dataset Distillation for Continual Test-Time Adaptation
DO-ALL uses dataset distillation to create synthetic source anchors that enable stable long-term continual test-time adaptation without storing original source data.
-
Distill Once, Adapt Life-Long: Exploring Dataset Distillation for Continual Test-Time Adaptation
DO-ALL applies dataset distillation to generate synthetic source anchors that stabilize continual test-time adaptation under evolving domains without storing original source data.
-
An Efficient and Scalable Graph Condensation with Structure-Preserving
SP-ESGC decouples graph condensation into heat-kernel node condensation and pre-trained edge prediction for structure, claiming high efficiency and cross-GNN generalization on real-world datasets.
-
DIVER:Diving Deeper into Distilled Data via Expressive Semantic Recovery
DIVER applies a pre-trained diffusion model in a dual-stage process of semantic inheritance, guidance, and fusion to improve semantic expression and cross-architecture generalization in dataset distillation.
-
CAST: Collapse-Aware multi-Scale Topology Fusion for Multimodal Coreset Selection
CAST selects better multimodal coresets by fusing collapse-aware topologies across modalities and matching distributions at multiple scales in the diffusion wavelet domain.
-
Data-Efficient Ensemble Weather Forecasting with Diffusion Models
Training an autoregressive diffusion weather forecaster on 20% of ERA5 data selected uniformly by calendar month matches full-data CRPS/RMSE and improves the spread-skill ratio on the 2018 test year.
-
Domain-Generalization to Improve Learning in Meta-Learning Algorithms
DGS-MAML layers gradient matching onto SharpMAML and claims O(1/T) convergence and tighter PAC-Bayes bounds, but the displayed theorems give O(1/sqrt T) under the paper's own parameter choices.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.