Pith. sign in

REVIEW 4 cited by

Reliable Adversarial Distillation with Unreliable Teachers

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2106.04928 v3 pith:7G7LMH3Z submitted 2021-06-09 cs.LG

Jianing Zhu , Jiangchao Yao , Bo Han , Jingfeng Zhang , Tongliang Liu , Gang Niu , Jingren Zhou , Jianliang Xu
show 1 more author
Hongxia Yang
This is my paper
classification cs.LG
keywords adversarialteachersdatadistillationgoodstudentstudentsteacher
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

In ordinary distillation, student networks are trained with soft labels (SLs) given by pretrained teacher networks, and students are expected to improve upon teachers since SLs are stronger supervision than the original hard labels. However, when considering adversarial robustness, teachers may become unreliable and adversarial distillation may not work: teachers are pretrained on their own adversarial data, and it is too demanding to require that teachers are also good at every adversarial data queried by students. Therefore, in this paper, we propose reliable introspective adversarial distillation (IAD) where students partially instead of fully trust their teachers. Specifically, IAD distinguishes between three cases given a query of a natural data (ND) and the corresponding adversarial data (AD): (a) if a teacher is good at AD, its SL is fully trusted; (b) if a teacher is good at ND but not AD, its SL is partially trusted and the student also takes its own SL into account; (c) otherwise, the student only relies on its own SL. Experiments demonstrate the effectiveness of IAD for improving upon teachers in terms of adversarial robustness.

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Attention-based Adversarial Robust Distillation in Radio Signal Classifications for Low-Power IoT Devices

    cs.LG 2025-06 conditional novelty 6.0 of 10

    A compact transformer trained with adversarial attention-map distillation achieves higher accuracy under white-box FGM and PGD attacks than existing adversarial distillation baselines.

  2. Towards Class-wise Fair Adversarial Training via Anti-Bias Soft Label Distillation

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A label-based adversarial robustness distillation method, ABSLD, re-temperates teacher soft labels per class to shrink the student's class-wise robust error gap and improves worst-class robustness and normalized stand...

  3. SHIELD: Secure Hypernetworks for Incremental Expansion Learning Defense

    cs.LG 2025-06 conditional novelty 6.0 of 10

    SHIELD uses a hypernetwork with IBP training and a new Interval MixUp technique to achieve certified robustness in continual learning, reporting state-of-the-art adversarial accuracy on MNIST, CIFAR-100, and miniImage...

  4. Improving Adversarial Robustness Through Adaptive Learning-Driven Multi-Teacher Knowledge Distillation

    cs.CV 2025-07 conditional novelty 4.0 of 10

    A multi-teacher adversarial robustness distillation method (MTKD-AR) trains a clean-data student using cosine-similarity-weighted logits from adversarially trained teachers, reporting improved robustness on MNIST and ...

Pith tools