Adaptive Knowledge Distillation using a Device-Aware Teacher for Low-Complexity Acoustic Scene Classification

Seong Eun Kim; Seung Gyu Jeong

REVIEW 5 major objections 5 minor 18 references

A two-teacher distillation with device-aware alignment lifts a 128 kB sound-scene model to 57.93% accuracy.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A low-complexity scene classifier trained by two-teacher knowledge distillation and device-specific fine-tuning reaches 57.93% accuracy on the DCASE 2025 development set.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection A solid engineering write-up for DCASE 2025 Task 1, but the headline 'significant improvement over baseline' is unverifiable as written and the dev set does double duty as both selection and evaluation. the 5 major comments →

arxiv 2509.09262 v1 pith:ZPG46SI2 submitted 2025-09-11 cs.SD cs.AI

Adaptive Knowledge Distillation using a Device-Aware Teacher for Low-Complexity Acoustic Scene Classification

Seung Gyu Jeong , Seong Eun Kim This is my paper

classification cs.SD cs.AI

keywords acoustic scene classificationknowledge distillationdevice robustnessdevice-aware feature alignmentlow-complexity modelCP-MobileNetPaSSTDCASE 2025

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a low-complexity acoustic scene classifier can reach 57.93% accuracy on the DCASE 2025 development set, a large gain over the official baseline, by combining two ideas. First, instead of distilling from a single teacher, a CP-MobileNet student learns from an ensemble of two PaSST teachers: one trained with plain cross-entropy and one trained with a Device-Aware Feature Alignment (DAFA) loss that shapes the feature space for device robustness. Second, because the challenge now supplies device labels at test time, the distilled student is fine-tuned separately for each of the six known devices. The authors find that the DAFA-inclusive ensemble matches a two-baseline ensemble overall but is better on unseen devices, and that device-specific fine-tuning adds 3.32 points overall, entirely on the known devices. The result matters because it suggests that robustness to unseen recording conditions can be instilled during distillation without violating strict parameter and compute limits.

Core claim

On its own terms, this work establishes the following: a two-teacher distillation, in which one teacher is a standard cross-entropy PaSST and the other is a PaSST trained with the proposed DAFA loss, transfers more device-robust knowledge to a 128 kB CP-MobileNet student than any single teacher, and as much overall as a two-standard-teacher ensemble while generalizing better to devices not seen in training (58.35% vs 58.31% on S4-S6). A subsequent device-specific fine-tuning stage, using the test-time device labels allowed by the 2025 task rules, lifts the distilled student from 54.61% to 57.93% overall, with gains of 2.16-8.24 points on the six known devices and no change on the unseen devi

What carries the argument

The core mechanism is the Device-Aware Feature Alignment (DAFA) loss, a regularizer added to the cross-entropy objective of the 'generalization expert' teacher. DAFA combines two terms: a Device Cohesion-Separation Loss (DCSL), defined as within-device feature scatter divided by between-device scatter (plus a small epsilon), which pulls each device's features into tight, well-separated clusters; and a Global Device Alignment Loss (GDAL), the mean squared distance of device centroids from the global batch centroid, which stops those clusters from fragmenting the shared embedding space. The teacher's soft logits are averaged with those of a baseline cross-entropy teacher to form the distillati

Load-bearing premise

The teacher ensemble and the DAFA hyperparameters were selected using the same development set that includes the 'unseen' devices, so the reported robustness to unseen devices assumes that those devices are representative enough that tuning on them does not just overfit to them.

What would settle it

Train the complete pipeline exactly as described, but choose the teacher ensemble and DAFA weights using only the six seen devices (A-C, S1-S3), then evaluate on S4-S6; if the final S4-S6 accuracy is no better than the two-CE-teacher ensemble, the unseen-device claim is a selection artifact.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

A 128 k-parameter model, running under 30 MMACs, can inherit Transformer-level knowledge through an ensemble, so the method is compatible with strict edge-device budgets.
The DAFA expert, not just ensembling, is what buys unseen-device robustness: the T_CE1+T_DAFA ensemble beats T_CE1+T_CE2 on S4-S6 even though both ensembles score exactly 60.73% overall as teachers.
Test-time device labels are turned into a 3.32-point accuracy gain by fine-tuning the distilled student per known device, with the largest gains on the devices where the distilled model was weakest.
If the official test distribution resembles the development set, the reported 57.93% should put the submission at the top end of the task's low-complexity systems.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Because the S4-S6 'unseen' devices were used to select the teacher ensemble and DAFA weights, the reported unseen-device edge is only as strong as the assumption that those devices represent unseen conditions; holding them out from all design decisions would give a cleaner estimate.
The DAFA loss's two components are not ablated separately; an ablation that removes DCSL or GDAL alone would identify which term actually drives the 58.35% unseen-device result.
Since DSFT does not change S4-S6 accuracy, an obvious extension is to learn a lightweight adapter per unseen device at test time, or to cluster devices and share fine-tuned adapters, which could extend the adaptive benefit beyond the six known devices.
One could test the pipeline's generality by training the same student from a single DAFA teacher and from the two-teacher ensemble on a different ASC dataset; if the ensemble advantage disappears, the benefit is task-specific.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

A solid engineering write-up for DCASE 2025 Task 1, but the headline 'significant improvement over baseline' is unverifiable as written and the dev set does double duty as both selection and evaluation.

read the letter

Short version: this is a decent challenge technical report, and the pipeline is coherent: two-teacher KD with a DAFA-regularized 'generalization expert,' plus device-specific fine-tuning to exploit test-time device labels. The per-device numbers in Table 2 are internally consistent, and the DSFT stage shows a plausible, uniform gain on known devices. What is new is the specific combination—not the individual pieces. The DAFA loss is an adaptation of your prior lung-sound work, and two-teacher averaging and test-time label fine-tuning have appeared in the challenge literature. Still, the extension to unseen-device robustness is legitimate and the results are directly tabulated.

The soft spots are real and match the stress-test. The official baseline accuracy is never reported, so the phrase 'significant improvement over the official baseline' is not checkable from the manuscript. Table 2 gives 57.93% overall and per-device numbers, but without baseline per-device values, the 'particularly on unseen devices' claim is just assertion. More importantly, the same development set is used for model selection (teacher ensemble choice, loss weights, fine-tuning schedule) and for the final headline number. Table 1 shows the two ensembles tie at 60.73% overall, and the chosen ensemble wins on the unseen subset by 0.04%—that margin is likely noise, yet it drives the selection. Without repeated seeds or confidence intervals, we cannot tell whether the 3.32% DSFT gain is stable or whether the unseen-device advantage over the alternative ensemble is meaningful. Eq. (3) is also underspecified: S_W and S_B are never defined, and a ratio with additive epsilon can be unstable if S_B is small. These are not fatal flaws—this is a challenge tech report, not a journal paper—but they do mean the central claim is not yet independently verifiable.

I agree with the reader's conditional verdict. The work is honest and the empirical core looks real, but the missing baseline and the selection-on-the-same-set issue are load-bearing for the 'significant improvement' claim. If the authors release code and report the baseline, log loss, and variance, this becomes a solid system description. As it stands, it is a useful data point for anyone entering DCASE 2025 Task 1, but I would not treat the 57.93% as a robust estimate of test performance. If this was expanded into a full paper with the missing details, it would deserve a serious referee. I would not desk-reject it; I would send it to review with a request for the baseline numbers, error bars, and precise definitions of the DAFA terms.

Referee Report

5 major / 5 minor

Summary. The paper describes a submission to the DCASE 2025 Task 1 low-complexity acoustic scene classification challenge. The proposed system trains a CP-MobileNet student via knowledge distillation from a two-teacher ensemble consisting of a standard cross-entropy PaSST teacher and a 'generalization expert' PaSST teacher trained with a Device-Aware Feature Alignment (DAFA) loss. After distillation, the student is fine-tuned separately for each of the six known device types, exploiting the challenge rule that device labels are available at test time. The authors report 57.93% accuracy on the development set and claim a significant improvement over the official baseline, especially on unseen devices.

Significance. If the reported results are reliable, the paper demonstrates a practical low-complexity ASC system that respects the 128 kB / 30 MMAC constraints and makes constructive use of test-time device labels. The per-device breakdown in Table 2 and the teacher-ensemble ablation in Table 1 are useful empirical contributions. However, the central claim is currently not independently verifiable because the official baseline is never reported, the DAFA loss is underspecified, and the statistical robustness of small performance gaps is not established. The work would be more significant if accompanied by the baseline comparison, precise loss definitions, and repeated-run variability.

major comments (5)

[Abstract and Section 5.3] The headline claim 'significant improvement over the official baseline, particularly on unseen devices' is unverifiable because the official baseline accuracy is never reported. Table 2 gives the proposed system's overall and per-device accuracy, but without the baseline's overall, per-device, and log-loss values, the improvement cannot be quantified. Please include the official baseline numbers and, if possible, the official evaluation metric.
[Section 4.1, Eq. (3)] The DCSL loss is defined as S_W/(S_B + epsilon), but S_W and S_B are never formally defined. The verbal descriptions of intra-device and inter-device scatter are insufficient for reproduction. Please specify the exact matrix definitions (e.g., trace of within-class and between-class scatter matrices), the feature layer used, the batch size, how centroids are computed, and the value of epsilon.
[Section 5.2 and Table 1] The choice of T_CE1 + T_DAFA over T_CE1 + T_CE2 rests on a 0.04% unseen-device advantage (58.35% vs. 58.31%) while both ensembles tie at 60.73% overall. Without repeated runs, confidence intervals, or significance tests, this difference is within run-to-run noise. Please report variance across multiple seeds or otherwise justify that the selected ensemble is statistically preferred.
[Section 5.3 and Table 2] The DSFT stage improves only the six known devices; the unseen devices S4-S6 show no change after fine-tuning. The abstract's claim of improvement 'particularly on unseen devices' is therefore not supported by the DSFT stage. If that claim refers to the teacher-ensemble comparison in Table 1, the text should say so explicitly and separate the two claims. Also, the manuscript does not specify the inference procedure when a test sample comes from an unseen device: is the un-fine-tuned distilled model used? Please clarify.
[Sections 5.2 and 5.3] The DAFA loss weights, the teacher ensemble, and the DSFT hyperparameters were selected using the same development set whose S4-S6 devices are used to claim robustness. This selection procedure risks overfitting to the development set's 'unseen' devices. The authors should acknowledge this limitation explicitly and, if possible, validate on the official test set or a nested split.

minor comments (5)

[Section 4 and throughout] The acronym is introduced as DAFA but Section 4 first uses 'DFA loss' and later 'DAFA'. Please use one consistent acronym.
[Table 1] In the T_CE1 + T_CE2 row, the numbers '66.4757.41' are concatenated without a space; please fix the formatting.
[Section 5.1] The text states that the primary evaluation metric is log loss, but all reported results are accuracy. Please report log-loss values as well, or justify the exclusive use of accuracy.
[References] Reference [8] is titled 'Domain Generalization with Relaxed Instance Frequency-wise Normalization...' while the text calls the method 'Freq-MixStyle'; clarify the relationship to the cited method.
[General] No code or implementation details are provided for the DAFA loss or the DSFT procedure. Given the underspecified Eq. (3), releasing code or a precise pseudocode would materially improve reproducibility.

Circularity Check

0 steps flagged

No significant circularity: the headline accuracy is an empirical development-set result, and the only self-citation (DAFA loss source) is not load-bearing.

full rationale

The paper's strongest claim is an empirical number (57.93% on the development set) obtained by a standard challenge pipeline; it is not derived from the method's equations by construction. The DAFA loss is the one component explicitly imported from the authors' own prior work [9], and no external benchmark for that loss is provided, so it carries a minor self-citation burden. However, its role is also supported by a within-paper teacher-ablation (Table 1) and by the method description, so the citation is not the sole load-bearing justification. Model and hyperparameter selection on the same development set used for reporting, and the omitted official baseline number, are statistical/verifiability concerns rather than circular reductions. No equation in the paper reduces to another by definition, and no fitted parameter is renamed as a prediction. Accordingly, the circularity score is low.

Axiom & Free-Parameter Ledger

7 free parameters · 5 axioms · 0 invented entities

The central result depends on conventional ML hyperparameters and domain assumptions. No new physical entities are introduced. The most fragile items are the transfer of DAFA from lung-sound to acoustic scene classification and the assumption that the development set's unseen devices are representative.

free parameters (7)

KD loss weight lambda = 0.98
Balances cross-entropy and KL distillation in Eq. (1); selected by experiment.
Distillation temperature tau = 2
Softens teacher logits; fixed for all runs.
DCSL loss weight lambda_dcsl = 0.01
Set through empirical validation on the development set, per Section 4.
GDAL loss weight lambda_gdal = 0.01
Set through empirical validation on the development set, per Section 4.
Student model width configuration = 32 base channels, expansion 3, multiplier 2.3
Chosen to stay under 128 kB and 30 MMACs while preserving accuracy.
DSFT learning rate and epochs = 1e-5, 100 epochs
Fine-tuning schedule for the six known devices after distillation.
Main distillation schedule = 500 epochs, peak LR 5e-4, batch 128
Training schedule; no sensitivity analysis is reported.

axioms (5)

domain assumption Knowledge distillation from large transformer teachers transfers to a CP-MobileNet student under the challenge complexity budget.
Invoked in Section 3.2; based on references [3] and [4], not proven in this report.
ad hoc to paper The DAFA loss developed for lung sound classification transfers to acoustic scene classification.
The report states the loss is adapted from prior work [9] but gives no independent verification for ASC; this is the paper-specific transfer assumption.
domain assumption Freq-MixStyle and Mixup improve device robustness without harming scene classification.
Techniques from [6]-[8] are used in training; no ablation isolates their contribution.
domain assumption Test-time device labels are available and correct, and the matching fine-tuned copy can be selected.
The DSFT stage depends on this challenge rule; if labels are noisy or unavailable, the final stage fails.
domain assumption The official development and validation split is representative of the final evaluation conditions.
Used to select the teacher ensemble and report the final accuracy; if this fails, the headline result will not transfer.

reviewed 2026-08-04 · how reviews work

0 comments

Cite this review

Pith. "Pith review of Adaptive Knowledge Distillation using a Device-Aware Teacher for Low-Complexity Acoustic Scene Classification." pith.science (2026). https://pith.science/paper/ZPG46SI2

@misc{pith2026250909262,
  author       = {Pith},
  title        = {Pith review of: Adaptive Knowledge Distillation using a Device-Aware Teacher for Low-Complexity Acoustic Scene Classification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZPG46SI2}},
  note         = {Machine review of arXiv:2509.09262}
}

Share X Bluesky LinkedIn Reddit HN

read the original abstract

In this technical report, we describe our submission for Task 1, Low-Complexity Device-Robust Acoustic Scene Classification, of the DCASE 2025 Challenge. Our work tackles the dual challenges of strict complexity constraints and robust generalization to both seen and unseen devices, while also leveraging the new rule allowing the use of device labels at test time. Our proposed system is based on a knowledge distillation framework where an efficient CP-MobileNet student learns from a compact, specialized two-teacher ensemble. This ensemble combines a baseline PaSST teacher, trained with standard cross-entropy, and a 'generalization expert' teacher. This expert is trained using our novel Device-Aware Feature Alignment (DAFA) loss, adapted from prior work, which explicitly structures the feature space for device robustness. To capitalize on the availability of test-time device labels, the distilled student model then undergoes a final device-specific fine-tuning stage. Our proposed system achieves a final accuracy of 57.93\% on the development set, demonstrating a significant improvement over the official baseline, particularly on unseen devices.

discussion (0)

Reference graph

Works this paper leans on

18 extracted references · 10 linked inside Pith

[1]

This field has gained prominence through the Detection and Classification of Acoustic Scenes and Events (DCASE) Challenge, which takes place annu- ally [1]

INTRODUCTION Acoustic Scene Classification (ASC) systems aim to categorize au- dio recordings into predefined scene classes. This field has gained prominence through the Detection and Classification of Acoustic Scenes and Events (DCASE) Challenge, which takes place annu- ally [1]. In this report, we describe our submission for Task 1 of the DCASE 2025 edi...

2025
[2]

SYSTEM COMPONENTS 2.1. Student Network: CP-MobileNet For the student model, we selected CP-MobileNet (CPM) [4], an architecture that has demonstrated a remarkable balance between performance and computational cost in previous ASC tasks. It in- corporates efficiency-focused designs, such as depth-wise separable convolutions inspired by MobileNets [10], to ...

Pith/arXiv arXiv 2025
[3]

This section details our training process, which is centered on a special- ized ensemble knowledge distillation (KD) framework designed to enhance model generalization

TRAINING METHODOLOGY Building upon the components described in the previous section, our core contribution lies in a novel training methodology. This section details our training process, which is centered on a special- ized ensemble knowledge distillation (KD) framework designed to enhance model generalization. We will describe the composition of our uni...

arXiv
[4]

The purpose of this loss is to structure the embedding space to be robust against device variations

DEVICE-A W ARE FEATURE ALIGNMENT LOSS Our Device-Aware Feature Alignment (DFA) loss is introduced as a regularization term, added to the primary cross-entropy objective (LCE ), to train our generalization expert teacher. The purpose of this loss is to structure the embedding space to be robust against device variations. The DFA loss itself consists of two...

2025
[5]

We first describe the experimental setup, then ana- lyze the performance based on different teacher ensemble composi- tions to identify the optimal configuration

EXPERIMENTS AND RESULTS In this section, we present the experimental validation of our pro- posed system. We first describe the experimental setup, then ana- lyze the performance based on different teacher ensemble composi- tions to identify the optimal configuration. Finally, we present the detailed performance of our final proposed system, highlighting ...

2025
[6]

The main contribution of this work is a novel training frame- work that combines specialized knowledge distillation with an adaptive fine-tuning strategy

and Freq-MixStyle [7, 8] are also used to enhance overall ro- bustness. The main contribution of this work is a novel training frame- work that combines specialized knowledge distillation with an adaptive fine-tuning strategy. We distill knowledge from a com- pact teacher ensemble composed of two PaSST models: 1) a base- line teacher trained with standard...
[7]

Our proposed method is cen- tered on a knowledge distillation framework that leverages a com- pact, specialized two-teacher ensemble to train an efficient CP- MobileNet student

CONCLUSION In this technical report, we presented our system for the DCASE 2025 Task 1, addressing the challenge of low-complexity, device- robust acoustic scene classification. Our proposed method is cen- tered on a knowledge distillation framework that leverages a com- pact, specialized two-teacher ensemble to train an efficient CP- MobileNet student. T...

2025
[8]

Low-complexity acoustic scene classification with device information in the dcase 2025 challenge,

F. Schmid, P. Primus, T. Heittola, A. Mesaros, I. Mart ´ın- Morat´o, and G. Widmer, “Low-complexity acoustic scene classification with device information in the dcase 2025 challenge,” 2025. [Online]. Available: https://arxiv.org/abs/ 2505.01747

Pith/arXiv arXiv 2025
[9]

Acoustic scene classification in dcase 2020 challenge: generalization across devices and low complexity solutions,

T. Heittola, A. Mesaros, and T. Virtanen, “Acoustic scene classification in dcase 2020 challenge: generalization across devices and low complexity solutions,” inProceedings of the Detection and Classification of Acoustic Scenes and Events 2020 Workshop (DCASE2020), 2020, pp. 56–60. [Online]. Available: https://arxiv.org/abs/2005.14623

Pith/arXiv arXiv 2020
[10]

Distilling the knowledge in a neural network,

G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,”arXiv preprint arXiv:1503.02531, 2015

Pith/arXiv arXiv 2015
[11]

Distilling the knowledge of transformers and CNNs with CP-mobile,

F. Schmid, T. Morocutti, S. Masoudian, K. Koutini, and G. Widmer, “Distilling the knowledge of transformers and CNNs with CP-mobile,” inProceedings of the Detection and Classification of Acoustic Scenes and Events 2023 Workshop (DCASE2023), 2023, pp. 161–165

2023
[12]

Efficient training of audio transformers with patchout,

K. Koutini, J. Schl ¨uter, H. Eghbal-Zadeh, and G. Widmer, “Efficient training of audio transformers with patchout,”arXiv preprint arXiv:2110.05069, 2021

Pith/arXiv arXiv 2021
[13]

mixup: Beyond empirical risk minimization,

H. Zhang, M. Cisse, Y . N. Dauphin, and D. Lopez-Paz, “mixup: Beyond empirical risk minimization,”arXiv preprint arXiv:1710.09412, 2017

Pith/arXiv arXiv 2017
[14]

Domain generaliza- tion with mixstyle,

K. Zhou, Y . Yang, Y . Qiao, and T. Xiang, “Domain generaliza- tion with mixstyle,”arXiv preprint arXiv:2104.02008, 2021

Pith/arXiv arXiv 2021
[15]

Domain generalization with relaxed instance frequency-wise normalization for multi-device acoustic scene classification,

B. Kim, S. Yang, J. Kim, H. Park, J. Lee, and S. Chang, “Domain generalization with relaxed instance frequency-wise normalization for multi-device acoustic scene classification,” arXiv preprint arXiv:2206.12513, 2022

Pith/arXiv arXiv 2022
[16]

Patient-aware feature alignment for robust lung sound classification: Cohesion-separation and global alignment losses,

S. G. Jeong and S. E. Kim, “Patient-aware feature alignment for robust lung sound classification: Cohesion-separation and global alignment losses,”arXiv preprint arXiv:2505.23834, 2025

Pith/arXiv arXiv 2025
[17]

Mobilenets: Effi- cient convolutional neural networks for mobile vision appli- cations,

A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Effi- cient convolutional neural networks for mobile vision appli- cations,”arXiv preprint arXiv:1704.04861, 2017

Pith/arXiv arXiv 2017
[18]

Audio set: An ontology and human-labeled dataset for audio events,

J. F. Gemmeke, D. P. Ellis, D. Freedman, A. Jansen, W. Lawrence, R. C. Moore, M. Plakal, and M. Ritter, “Audio set: An ontology and human-labeled dataset for audio events,” in2017 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, 2017, pp. 776–780

2017

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.

[1] [1]

This field has gained prominence through the Detection and Classification of Acoustic Scenes and Events (DCASE) Challenge, which takes place annu- ally [1]

INTRODUCTION Acoustic Scene Classification (ASC) systems aim to categorize au- dio recordings into predefined scene classes. This field has gained prominence through the Detection and Classification of Acoustic Scenes and Events (DCASE) Challenge, which takes place annu- ally [1]. In this report, we describe our submission for Task 1 of the DCASE 2025 edi...

2025

[2] [2]

SYSTEM COMPONENTS 2.1. Student Network: CP-MobileNet For the student model, we selected CP-MobileNet (CPM) [4], an architecture that has demonstrated a remarkable balance between performance and computational cost in previous ASC tasks. It in- corporates efficiency-focused designs, such as depth-wise separable convolutions inspired by MobileNets [10], to ...

Pith/arXiv arXiv 2025

[3] [3]

This section details our training process, which is centered on a special- ized ensemble knowledge distillation (KD) framework designed to enhance model generalization

TRAINING METHODOLOGY Building upon the components described in the previous section, our core contribution lies in a novel training methodology. This section details our training process, which is centered on a special- ized ensemble knowledge distillation (KD) framework designed to enhance model generalization. We will describe the composition of our uni...

arXiv

[4] [4]

The purpose of this loss is to structure the embedding space to be robust against device variations

DEVICE-A W ARE FEATURE ALIGNMENT LOSS Our Device-Aware Feature Alignment (DFA) loss is introduced as a regularization term, added to the primary cross-entropy objective (LCE ), to train our generalization expert teacher. The purpose of this loss is to structure the embedding space to be robust against device variations. The DFA loss itself consists of two...

2025

[5] [5]

We first describe the experimental setup, then ana- lyze the performance based on different teacher ensemble composi- tions to identify the optimal configuration

EXPERIMENTS AND RESULTS In this section, we present the experimental validation of our pro- posed system. We first describe the experimental setup, then ana- lyze the performance based on different teacher ensemble composi- tions to identify the optimal configuration. Finally, we present the detailed performance of our final proposed system, highlighting ...

2025

[6] [6]

The main contribution of this work is a novel training frame- work that combines specialized knowledge distillation with an adaptive fine-tuning strategy

and Freq-MixStyle [7, 8] are also used to enhance overall ro- bustness. The main contribution of this work is a novel training frame- work that combines specialized knowledge distillation with an adaptive fine-tuning strategy. We distill knowledge from a com- pact teacher ensemble composed of two PaSST models: 1) a base- line teacher trained with standard...

[7] [7]

Our proposed method is cen- tered on a knowledge distillation framework that leverages a com- pact, specialized two-teacher ensemble to train an efficient CP- MobileNet student

CONCLUSION In this technical report, we presented our system for the DCASE 2025 Task 1, addressing the challenge of low-complexity, device- robust acoustic scene classification. Our proposed method is cen- tered on a knowledge distillation framework that leverages a com- pact, specialized two-teacher ensemble to train an efficient CP- MobileNet student. T...

2025

[8] [8]

Low-complexity acoustic scene classification with device information in the dcase 2025 challenge,

F. Schmid, P. Primus, T. Heittola, A. Mesaros, I. Mart ´ın- Morat´o, and G. Widmer, “Low-complexity acoustic scene classification with device information in the dcase 2025 challenge,” 2025. [Online]. Available: https://arxiv.org/abs/ 2505.01747

Pith/arXiv arXiv 2025

[9] [9]

Acoustic scene classification in dcase 2020 challenge: generalization across devices and low complexity solutions,

T. Heittola, A. Mesaros, and T. Virtanen, “Acoustic scene classification in dcase 2020 challenge: generalization across devices and low complexity solutions,” inProceedings of the Detection and Classification of Acoustic Scenes and Events 2020 Workshop (DCASE2020), 2020, pp. 56–60. [Online]. Available: https://arxiv.org/abs/2005.14623

Pith/arXiv arXiv 2020

[10] [10]

Distilling the knowledge in a neural network,

G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,”arXiv preprint arXiv:1503.02531, 2015

Pith/arXiv arXiv 2015

[11] [11]

Distilling the knowledge of transformers and CNNs with CP-mobile,

F. Schmid, T. Morocutti, S. Masoudian, K. Koutini, and G. Widmer, “Distilling the knowledge of transformers and CNNs with CP-mobile,” inProceedings of the Detection and Classification of Acoustic Scenes and Events 2023 Workshop (DCASE2023), 2023, pp. 161–165

2023

[12] [12]

Efficient training of audio transformers with patchout,

K. Koutini, J. Schl ¨uter, H. Eghbal-Zadeh, and G. Widmer, “Efficient training of audio transformers with patchout,”arXiv preprint arXiv:2110.05069, 2021

Pith/arXiv arXiv 2021

[13] [13]

mixup: Beyond empirical risk minimization,

H. Zhang, M. Cisse, Y . N. Dauphin, and D. Lopez-Paz, “mixup: Beyond empirical risk minimization,”arXiv preprint arXiv:1710.09412, 2017

Pith/arXiv arXiv 2017

[14] [14]

Domain generaliza- tion with mixstyle,

K. Zhou, Y . Yang, Y . Qiao, and T. Xiang, “Domain generaliza- tion with mixstyle,”arXiv preprint arXiv:2104.02008, 2021

Pith/arXiv arXiv 2021

[15] [15]

Domain generalization with relaxed instance frequency-wise normalization for multi-device acoustic scene classification,

B. Kim, S. Yang, J. Kim, H. Park, J. Lee, and S. Chang, “Domain generalization with relaxed instance frequency-wise normalization for multi-device acoustic scene classification,” arXiv preprint arXiv:2206.12513, 2022

Pith/arXiv arXiv 2022

[16] [16]

Patient-aware feature alignment for robust lung sound classification: Cohesion-separation and global alignment losses,

S. G. Jeong and S. E. Kim, “Patient-aware feature alignment for robust lung sound classification: Cohesion-separation and global alignment losses,”arXiv preprint arXiv:2505.23834, 2025

Pith/arXiv arXiv 2025

[17] [17]

Mobilenets: Effi- cient convolutional neural networks for mobile vision appli- cations,

A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Effi- cient convolutional neural networks for mobile vision appli- cations,”arXiv preprint arXiv:1704.04861, 2017

Pith/arXiv arXiv 2017

[18] [18]

Audio set: An ontology and human-labeled dataset for audio events,

J. F. Gemmeke, D. P. Ellis, D. Freedman, A. Jansen, W. Lawrence, R. C. Moore, M. Plakal, and M. Ritter, “Audio set: An ontology and human-labeled dataset for audio events,” in2017 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, 2017, pp. 776–780

2017

Adaptive Knowledge Distillation using a Device-Aware Teacher for Low-Complexity Acoustic Scene Classification

The reading

Core claim §

What carries the argument

Load-bearing premise §

What would settle it §

If this is right

Where Pith is reading between the lines

discussion (0)

Core claim

Load-bearing premise

What would settle it