REVIEW 3 cited by
Is Synthetic Data From Diffusion Models Ready for Knowledge Distillation?
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Diffusion models have recently achieved astonishing performance in generating high-fidelity photo-realistic images. Given their huge success, it is still unclear whether synthetic images are applicable for knowledge distillation when real images are unavailable. In this paper, we extensively study whether and how synthetic images produced from state-of-the-art diffusion models can be used for knowledge distillation without access to real images, and obtain three key conclusions: (1) synthetic data from diffusion models can easily lead to state-of-the-art performance among existing synthesis-based distillation methods, (2) low-fidelity synthetic images are better teaching materials, and (3) relatively weak classifiers are better teachers. Code is available at https://github.com/zhengli97/DM-KD.
Forward citations
Cited by 3 Pith papers
-
When Model Knowledge meets Diffusion Model: Diffusion-assisted Data-free Image Synthesis with Alignment of Domain and Class
DDIS generates training-like images from a frozen classifier by steering Stable Diffusion with batch-normalization statistics and an optimized per-class token, improving data-free distillation and pruning.
-
Diffusion-based Data Augmentation and Knowledge Distillation with Generated Soft Labels Solving Data Scarcity Problems of SAR Oil Spill Segmentation
DAKTer jointly generates SAR oil spill images and soft labels with a diffusion model, using an SNR balancing factor and cross-entropy training, and distills these soft labels into segmentation models to improve perfor...
-
Feature Alignment and Representation Transfer in Knowledge Distillation for Large Language Models
A broad survey of knowledge distillation for LLMs that summarizes published methods but contains no new results and several citation errors.
Discussion (0). Continue with ORCID to comment.