DAKTer jointly generates SAR oil spill images and soft labels with a diffusion model, using an SNR balancing factor and cross-entropy training, and distills these soft labels into segmentation models to improve performance under data scarcity.
Is Synthetic Data From Diffusion Models Ready for Knowledge Distillation?
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Diffusion models have recently achieved astonishing performance in generating high-fidelity photo-realistic images. Given their huge success, it is still unclear whether synthetic images are applicable for knowledge distillation when real images are unavailable. In this paper, we extensively study whether and how synthetic images produced from state-of-the-art diffusion models can be used for knowledge distillation without access to real images, and obtain three key conclusions: (1) synthetic data from diffusion models can easily lead to state-of-the-art performance among existing synthesis-based distillation methods, (2) low-fidelity synthetic images are better teaching materials, and (3) relatively weak classifiers are better teachers. Code is available at https://github.com/zhengli97/DM-KD.
citation-role summary
baseline 1
citation-polarity summary
fields
cs.CV 1years
2024 1verdicts
CONDITIONAL 1roles
baseline 1polarities
baseline 1representative citing papers
citing papers explorer
-
Diffusion-based Data Augmentation and Knowledge Distillation with Generated Soft Labels Solving Data Scarcity Problems of SAR Oil Spill Segmentation
DAKTer jointly generates SAR oil spill images and soft labels with a diffusion model, using an SNR balancing factor and cross-entropy training, and distills these soft labels into segmentation models to improve performance under data scarcity.