Pith. sign in

REVIEW 4 cited by

Training Shallow and Thin Networks for Acceleration via Knowledge Distillation with Conditional Adversarial Networks

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1709.00513 v2 pith:CUOP5BYY submitted 2017-09-02 cs.LG cs.AIcs.CV

Zheng Xu , Yen-Chang Hsu , Jiawei Huang This is my paper · ORCID
classification cs.LGcs.AIcs.CV
keywords networksstudentnetworkadversarialconditionalknowledgesmallteacher
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

There is an increasing interest on accelerating neural networks for real-time applications. We study the student-teacher strategy, in which a small and fast student network is trained with the auxiliary information learned from a large and accurate teacher network. We propose to use conditional adversarial networks to learn the loss function to transfer knowledge from teacher to student. The proposed method is particularly effective for relatively small student networks. Moreover, experimental results show the effect of network size when the modern networks are used as student. We empirically study the trade-off between inference time and classification accuracy, and provide suggestions on choosing a proper student network.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Scaling and Distilling Transformer Models for sEMG

    eess.AS 2025-07 accept novelty 6.0 of 10

    Vanilla transformers on the emg2qwerty dataset improve cross-user typing accuracy up to 109M parameters, and simple logit distillation recovers most of the gain in a 2.2M-parameter student.

  2. Bidirectional Knowledge Distillation for Enhancing Sequential Recommendation with Large Language Models

    cs.IR 2025-05 conditional novelty 6.0 of 10

    An alternating distillation loop between a conventional recommender and an LLM recommender improves top-K accuracy on four datasets without adding inference-time parameters.

  3. Latent-Space Mean-Field Theory for Deep BitNet-like Training: Constrained Gradient Flows with Smooth Quantization and STE Limits

    math.OC 2025-08 reject novelty 5.0 of 10

    The paper's main mean-field theorem for BitNet-like training is not proven: it assumes key regularity and uses an invalid distributional limit step.

  4. MASTER: Enhancing Large Language Model via Multi-Agent Simulated Teaching

    cs.CL 2025-06 conditional novelty 4.0 of 10

    A multi-agent simulated teaching pipeline creates BOOST-QA, and fine-tuning on it lifts reported LLM benchmark scores by up to 31 points over the original data.

Pith tools