Pith. sign in

REVIEW 10 cited by

Data-Free Adversarial Distillation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1912.11006 v3 pith:MYSXRZRC submitted 2019-12-23 cs.LG cs.CVstat.ML

Gongfan Fang , Jie Song , Chengchao Shen , Xinchao Wang , Da Chen , Mingli Song This is my paper · ORCID
classification cs.LGcs.CVstat.ML
keywords datadistillationmodelstudentadversarialapproachdata-drivendata-free
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Knowledge Distillation (KD) has made remarkable progress in the last few years and become a popular paradigm for model compression and knowledge transfer. However, almost all existing KD algorithms are data-driven, i.e., relying on a large amount of original training data or alternative data, which is usually unavailable in real-world scenarios. In this paper, we devote ourselves to this challenging problem and propose a novel adversarial distillation mechanism to craft a compact student model without any real-world data. We introduce a model discrepancy to quantificationally measure the difference between student and teacher models and construct an optimizable upper bound. In our work, the student and the teacher jointly act the role of the discriminator to reduce this discrepancy, when a generator adversarially produces some "hard samples" to enlarge it. Extensive experiments demonstrate that the proposed data-free method yields comparable performance to existing data-driven methods. More strikingly, our approach can be directly extended to semantic segmentation, which is more complicated than classification, and our approach achieves state-of-the-art results. Code and pretrained models are available at https://github.com/VainF/Data-Free-Adversarial-Distillation.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Towards Efficient and Effective Alignment of Large Language Models

    cs.CL 2025-06 conditional novelty 7.0 of 10

    A thesis presenting Lion, WebR, LTE, BMC, and FollowBench, five empirical methods that together address LLM alignment data, training, and evaluation.

  2. Stealix: Model Stealing via Prompt Evolution

    cs.CR 2025-06 conditional novelty 7.0 of 10

    An attacker with one seed image per class can steal a black-box image classifier by genetically evolving text prompts, guided only by the victim's hard-label predictions.

  3. Medical Multimodal Model Stealing Attacks via Adversarial Domain Alignment

    cs.CR 2025-02 conditional novelty 7.0 of 10

    An adversarial domain alignment method steals a medical multimodal LLM's radiology report generation using natural images and an oracle LLM, without medical data.

  4. Toward Efficient Data-Free Unlearning

    cs.LG 2024-12 conditional novelty 7.0 of 10

    ISPF combines a new generator loss that discourages synthesis of forgetting-class samples with a logit-redistribution step that lets every synthetic sample contribute to distillation, improving retained accuracy over GKT.

  5. When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need

    cs.LG 2025-07 conditional novelty 6.0 of 10

    Data-free distillation from non-transferable teachers fails because synthesized samples drift toward the OOD domain; ATEsc separates ID-like from OOD-like samples via adversarial robustness and improves distillation.

  6. Forget the Data and Fine-Tuning! Just Fold the Network to Compress

    cs.LG 2025-02 conditional novelty 6.0 of 10

    Model folding compresses a network by k-means clustering similar neurons across adjacent layers and repairing activation statistics without data (Fold-AR, Fold-DIR), surpassing prior data-free methods at high sparsity.

  7. Training-Free Restoration of Pruned Neural Networks

    cs.LG 2025-02 conditional novelty 6.0 of 10

    LBYL restores pruned CNNs by spreading each pruned filter's contribution across multiple preserved filters via a closed-form delivery matrix, outperforming one-to-one neuron merging without data or fine-tuning.

  8. Towards Communication-Efficient Adversarial Federated Learning for Robust Edge Intelligence

    cs.CV 2025-01 conditional novelty 6.0 of 10

    A pre-trained teacher-guided distillation framework, PM-AFL++, improves clean and adversarial accuracy of federated models while reducing communication rounds and parameters.

  9. Stabilizing Data-Free Model Extraction

    cs.LG 2025-09 conditional novelty 5.0 of 10

    MetaDFME stabilizes data-free model extraction by training the generator with Reptile-style meta-learning, achieving higher and less oscillating substitute accuracy.

  10. Federated Hybrid Training and Self-Adversarial Distillation: Towards Robust Edge Networks

    cs.CV 2024-12 conditional novelty 5.0 of 10

    FedBAT combines hybrid adversarial training with augmentation-invariant self-distillation to improve both clean and robust accuracy in federated learning under non-IID data.

Pith tools