Pith. sign in

REVIEW 3 major objections 6 minor 77 references

Variational Bayesian Adaptive Learning of Deep Latent Variables for Acoustic Knowledge Transfer

T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Bayesian hidden-layer transfer beats 13 acoustic adaptation methods.

desk verdict Solid, broad experiments, but the paper's Bayesian identity is overstated: with equal fixed variances the KL term is exactly a scaled squared-L2 feature-matching penalty, and the authors' denial of that is algebraically wrong. read the letter →

arxiv 2501.15496 v1 pith:JIQ6OWVL submitted 2025-01-26 eess.AS cs.SD

Hu Hu , Sabato Marco Siniscalchi , Chao-Han Huck Yang , Chin-Hui Lee This is my paper · ORCID
classification eess.AScs.SD
keywords BayesianadaptationVariationalinferencelatentvariabledomainknowledgedistillationacousticsceneclassificationspeechcommandrecognitiondevicemismatch
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that acoustic models can be adapted to new devices and noise conditions more effectively by treating a network's hidden embeddings as Bayesian latent variables and updating their distributions, rather than updating the many network weights or just copying point estimates from a teacher. The authors propose variational Bayesian knowledge transfer (VBKT) with two posterior estimators: Gaussian mean-field variational inference for parallel source-target data, and empirical Bayes when no parallel data exists. On device adaptation for acoustic scene classification and noise adaptation for spoken command recognition, the method improves over a one-hot fine-tuned baseline by about 4-6 percentage points and beats thirteen listed knowledge-transfer baselines. If correct, the approach offers a parameter-efficient recipe for adapting deep acoustic models with small target sets.

What carries the argument

The load-bearing object is the deep latent variable $Z$, the hidden embedding before the final layer. The central identity is Eq. (3), $p(Z_T \mid \theta_T) = p(Z_S \mid \theta_S, D_S)$, which says the target latent prior equals the source latent posterior, turning Bayes update into distribution matching in embedding space. The optimization object is the ELBO in Eq. (7), whose KL term has closed form under diagonal Gaussian assumptions; with fixed variance it becomes a mean-matching $\ell^2$ term, trained by stochastic gradient variational Bayes with reparameterization. The structural-relationship variant adds a Huber-smoothed penalty on pairwise KL distances within each mixture, Eq. (20), to preserve interdependencies among latent components.

What would settle it

Take a trained source model, extract hidden embeddings from source and target data for one class, fit Gaussians to each, and measure the KL divergence between them. If that KL is large or the per-class embeddings are visibly non-Gaussian and the fixed-variance mean-matching penalty still claims to succeed, the central assumption fails; a direct test is whether accuracy on that class drops when the penalty is removed.

Watch

Extended reading notes

Core claim

The central claim is that the posterior of target latent variables, $p(Z_T \mid \theta_T, D_T)$, can be approximated by a variational distribution whose prior comes from the source domain, $p(Z_S \mid \theta_S, D_S)$, and that optimizing the resulting ELBO transfers knowledge better than teacher-student output matching or feature matching. Under Gaussian mean-field and empirical-Bayes approximations, the KL term reduces to a squared-distance penalty between means (with fixed variances), so training pulls target hidden embeddings toward source-consistent locations while preserving classification likelihood. The paper reports consistent accuracy gains over thirteen listed transfer methods on both tasks, with the structural-relationship variant adding a small further improvement and lower variance.

Load-bearing premise

The whole update relies on Eq. (3): before adaptation, target hidden embeddings are assumed to follow the same distribution as source hidden embeddings; if device or noise mismatch changes that distribution in ways the Gaussian mean-matching cannot capture, the Bayesian pull moves target features toward the wrong values.

Editorial extensions

If this is right

  • VBKT-GMF can be combined with teacher-student learning, and the best accuracies in the paper come from that combination, suggesting the two forms of transfer are complementary.
  • Deeper hidden embeddings transfer better than shallower ones: the paper's ablation across convolutional layers shows accuracy rises as the embedding moves toward the output.
  • VBKT-EB removes the need for parallel data, so the method can be applied to any target domain where a source model and a small labeled target set exist.
  • Structural relationship modeling consistently adds a small accuracy boost and reduces standard deviation across repeated trials, implying the independence assumption in the plain Gaussian approximation is the main loss.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One extension the paper does not explore: replace the fixed variance $\sigma^2$ in Eq. (10) with a learned, input-dependent variance; if the method's gains come from matching means, a learned variance should improve calibration on classes with broader source spread.
  • The Gaussian-per-class prior suggests a straightforward diagnostic: estimate per-class source and target embedding Gaussians before adaptation; classes where the KL between them is large should be exactly where adaptation helps most, or fails if mean matching is insufficient.
  • The diagonal-covariance assumption could be lifted using low-rank plus diagonal covariances; the closed-form KL would generalize and might matter when device mismatch changes correlations between embedding dimensions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a variational Bayesian knowledge transfer (VBKT) framework for acoustic domain adaptation. Instead of placing priors on network parameters, the authors place Gaussian priors on deep latent variables (hidden embeddings) and combine source knowledge with target data through an ELBO objective. Two estimation strategies are presented: Gaussian mean-field variational inference (VBKT-GMF) for parallel source-target data, and empirical Bayes (VBKT-EB) for non-parallel data, with an optional structural-relationship regularizer (VBKT-GMF-rela and VBKT-EB-rela). Experiments are conducted on device adaptation for DCASE 2020 acoustic scene classification (using RESNET, FCNN, and INCEPTION models) and noise adaptation for Google Speech Commands v2 (using CRNN-att). The reported results show consistent accuracy improvements over thirteen knowledge transfer baselines. The paper also includes layer-depth ablation and feature-visualization analyses.

Significance. If the results hold, the paper makes a useful empirical contribution: it shows that modeling distributions of deep latent variables can outperform standard feature-distillation and fine-tuning methods on two acoustic adaptation tasks. The study is extensive in its baseline coverage and in the range of model architectures tested, and the ELBO derivation in Eqs. (4)-(7) follows standard variational inference. However, the claimed Bayesian identity of the method is weakened by the reduction of the KL term to an L2 penalty, and the lack of comparison with the authors' own prior VBKT work [34] makes the incremental novelty difficult to assess. The contribution is therefore promising but currently under-substantiated in its central interpretation.

major comments (3)
  1. [Section IV-B, Eq. (10) and paragraph after Eq. (11)] The assertion that the KL term 'differs from a simple L2 distance between hidden features since the measurement is carried out over a probabilistic distribution space' is algebraically incorrect. With fixed and equal variances, Eq. (9) reduces exactly to (1/2σ^2) Σ_i ||μ_T^(i) − μ_S^(i)||^2, a scaled squared Euclidean distance. The covariance terms cancel, so no distributional information beyond the mean difference remains in the KL term. This does not invalidate the reported accuracy gains, but it means the central claim that the method performs Bayesian distributional matching rather than deterministic feature matching is not supported. The authors should either correct this statement, introduce learnable/unequal variances so that covariance information is retained, or provide an ablation comparing with a deterministic L2 feature-matching objective (e.g., Fitnets/NST) under identical sampling and architecture choices.
  2. [Section V, Tables III and IV] The paper builds directly on the authors' prior VBKT method introduced in [34], yet no comparison with that method is reported. Since the abstract claims consistent improvement over state-of-the-art knowledge transfer methods, and [34] is the immediate predecessor, the reader cannot determine what the new GMFVI/EB formulations and the structural relationship modeling contribute. Please add VBKT [34] as a baseline in both tables, and discuss any differences in setup that might affect comparability.
  3. [Section V, Tables III and IV] No statistical significance tests are reported. Some improvements over the best baselines are modest relative to the listed standard deviations, for example Table III, RESNET columns: VBKT-GMF 69.58±0.49 vs. NST 68.35±0.28, and VBKT-GMF-rela 69.86±0.35 vs. NST 68.35±0.28. Given that the abstract claims the method 'consistently outperforms' state-of-the-art methods, paired tests (or a statement of effect size and confidence intervals) across the 32 experimental runs are needed to substantiate that claim.
minor comments (6)
  1. [Section III, Eq. (3)] The distributional invariance assumption p(ZT|θT)=p(ZS|θS,DS) is asserted without empirical justification for the two tasks. A brief analysis of source and target feature statistics (e.g., per-device or per-noise-condition means and variances) would strengthen the Bayesian interpretation and calibrate expectations about when the method may fail.
  2. [Section V-B] The sensitivity of the reported results to the fixed variance σ and the structural-relationship weight β (set to 0.1) is not studied. Since these hyperparameters directly control the strength of the adaptation loss, a sensitivity analysis or a description of how they were selected would improve reproducibility and confidence in the reported margins.
  3. [Section IV-B] The sentence 'the lower bound in Eq. (7) can be approximated empirically approximated as follows' contains a duplicated phrase; please revise.
  4. [Section VI] The word 'scenorios' in the conclusion is a typo for 'scenarios'.
  5. [Section I and Section IV] The text refers to GMFVI and EB as 'prior approximation mechanisms,' but these are methods for approximating the posterior distribution; the prior is the source-domain distribution in Eq. (3). Please reword to avoid terminological confusion.
  6. [Table I] The NLE setting 'S-KLD' is not defined in the text or in the table caption; please clarify.

Circularity Check

2 steps flagged · score 2.0 of 10

No load-bearing circularity: the ELBO derivation is explicit and the accuracy comparisons are genuine; however, with equal fixed variances the 'distributional' KL term is exactly an L2 feature-matching loss, so the Bayesian framing is partly a relabeling of known feature-distillation objectives.

  1. renaming known result [Section IV-B, Eq. (10)-(11) and the paragraph after Eq. (11)]
    "In this work, we assign a fixed value σ2 to both (σ(i) S )2 and (σ(i) T )2. We can now rewrite Eq. (9) as follows: KLD(q(ZT |θT , DT ) ∥ p(ZS|θS, DS)) = 1/2σ2 PNT i ∥µ(i) T − µ(i) S ∥2 2. ... Moreover, it should be noted the second term differs from a simple L2 distance between hidden features since the measurement is carried out over a probabilistic distribution space."

    With σ_S=σ_T=σ, Eq. (9) collapses term-by-term: log(σ_S/σ_T)=0 and [(σ_T)^2+(μ_T−μ_S)^2]/(2σ_S^2)−1/2 = (μ_T−μ_S)^2/(2σ^2). So Eq. (10) is exactly a scaled squared-Euclidean feature-matching penalty. The sentence after Eq. (11) denies this, but no covariance information survives; the only 'probabilistic' element is Gaussian sampling noise in the likelihood term. Thus the claimed distribution-space measurement is arithmetically identical to the L2 distance used by Fitnets/NST, and the Bayesian framing is a relabeling of a known feature-matching objective.

  2. renaming known result [Section IV-C, Eq. (16) and surrounding text]
    "Moreover, (σ(c,i) T )2 is set equal to (σ(c) MLE)2 for all the target input samples, x(c,i) T , from class c. Re-arranging the terms and adopting Monte Carlo sampling in Eq. (15) allows us to empirically approximate the ELBO in Eq. (7) as follows: ... − 1/2(σ(c) MLE)2 PC c PN(c)T i ∥µ(c,i) T − µ(c) MLE∥2 2."

    With the target posterior variance fixed to the source MLE variance, the KL term in Eq. (15) again reduces to (1/(2σ_MLE^2)) Σ ||μ_T^(c,i)−μ_MLE^(c)||^2. The 'empirical Bayes prior' contributes only source class-conditional means and a scalar scale; the adaptation loss is class-wise L2 matching to those means. No component of the loss depends on a posterior distribution beyond this deterministic mean-matching plus sampling noise. This is the same renaming of L2 feature distillation, not a circular prediction of the accuracy numbers, but it does undermine the distinctive Bayesian content.

full rationale

Score 2. The derivation chain is not circular in the sense that a predicted result is identical to an input by construction. Eqs. (1)-(7) define a standard ELBO, and Eq. (3) is an explicit domain-invariance assumption rather than an output smuggled in. No parameter is fitted to final target accuracy: σ is fixed from augmented-data statistics (Section V-B) or source MLE, and β=0.1 is a stated constant, so the reported improvements over the thirteen baselines are genuine empirical outcomes. The self-citation of [34] is normal prior work and is not load-bearing: the paper re-derives the objective without relying on an unverified uniqueness claim. The real problem is a mathematical overstatement: with equal fixed variances Eq. (10) is exactly an L2 feature-matching loss, and the text's assertion that it differs from simple L2 distance is false; Eq. (16) collapses similarly to class-conditional L2 matching. This makes the 'variational Bayesian' branding largely a relabeling of known feature-distillation losses, but it does not make the experiments or the accuracy comparisons circular. Hence a low circularity score with a flagged correctness/overstatement issue.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new physical entities are introduced. 'Deep latent variables' are standard hidden embeddings; the contributions are modeling choices and losses.

free parameters (3)
  • Fixed Gaussian variance σ² for GMFVI = not reported (average std over augmented data)
    Controls the weight of the latent matching penalty in Eq. (10); chosen from augmented-data statistics, no sensitivity analysis.
  • Structural relationship weight β = 0.1
    Added in Eq. (20) to balance the relational Huber loss; a single value is used, with no sweep or analysis.
  • Hidden embedding layer index = second-last conv layer (ASC); pre-final activation (SCR)
    Defines which features are treated as latent variables; Figure 3 shows the choice affects accuracy.
assumptions (4)
  • domain assumption p(ZT|θT)=p(ZS|θS,DS): target latent variables share the source latent distribution before adaptation.
    Eq. (3) is the mechanism that makes source embeddings a valid prior for target features; if the mismatch changes feature statistics, the prior is misspecified.
  • domain assumption Each class's hidden embeddings form a multivariate Gaussian with diagonal covariance.
    Section IV-C uses histograms to argue this; no quantitative fit test, and the closed-form KL depends on it.
  • domain assumption Non-informative priors on the network weights θT and ωT, which are never updated.
    Section IV-A states this; all adaptation capacity is forced into the latent variable distributions.
  • ad hoc to paper Source and target Gaussian variances are equal and fixed in the mean-field variant.
    Section IV-B, following Eq. (9), sets both σ_S and σ_T to a fixed σ, which cancels the variance terms and leaves only mean matching.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Variational Bayesian Adaptive Learning of Deep Latent Variables for Acoustic Knowledge Transfer." pith.science (2026). https://pith.science/paper/JIQ6OWVL

@misc{pith2026250115496,
  author       = {Pith},
  title        = {Pith review of: Variational Bayesian Adaptive Learning of Deep Latent Variables for Acoustic Knowledge Transfer},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JIQ6OWVL}},
  note         = {Machine review of arXiv:2501.15496}
}
read the original abstract

In this work, we propose a novel variational Bayesian adaptive learning approach for cross-domain knowledge transfer to address acoustic mismatches between training and testing conditions, such as recording devices and environmental noise. Different from the traditional Bayesian approaches that impose uncertainties on model parameters risking the curse of dimensionality due to the huge number of parameters, we focus on estimating a manageable number of latent variables in deep neural models. Knowledge learned from a source domain is thus encoded in prior distributions of deep latent variables and optimally combined, in a Bayesian sense, with a small set of adaptation data from a target domain to approximate the corresponding posterior distributions. Two different strategies are proposed and investigated to estimate the posterior distributions: Gaussian mean-field variational inference, and empirical Bayes. These strategies address the presence or absence of parallel data in the source and target domains. Furthermore, structural relationship modeling is investigated to enhance the approximation. We evaluated our proposed approaches on two acoustic adaptation tasks: 1) device adaptation for acoustic scene classification, and 2) noise adaptation for spoken command recognition. Experimental results show that the proposed variational Bayesian adaptive learning approach can obtain good improvements on target domain data, and consistently outperforms state-of-the-art knowledge transfer methods.

Figures

Figures reproduced from arXiv: 2501.15496 by the authors.

Figure 1
Figure 1. Illustration of the deep latent variables based acoustic knowledge transfer framework. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Histograms of sample hidden embedding elements from [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Evaluation results (classification accuracy %) of using different hidden layers by FCNN model for ASC task. Two [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Visualized heatmaps of the intra-class discrepancy [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Feature visualization via T-SNE results of four devices [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

77 extracted references · 55 canonical work pages

  1. [34]

    A variational bayesian approach to learning latent variables for acoustic knowledge transfer,

    H. Hu, S. M. Siniscalchi, C.-H. H. Yang, and C.-H. Lee, “A variational bayesian approach to learning latent variables for acoustic knowledge transfer,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2022, pp. 4041–4045

    work page 2022

  2. [1]

    Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups,

    G. Hinton, L. Deng, D. Yu, G. E. Dahl, A.-r. Mohamed, N. Jaitly, A. Senior, V . Vanhoucke, P. Nguyen, T. N. Sainath et al., “Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups,” IEEE Signal processing magazine , vol. 29, no. 6, pp. 82–97, 2012

    2012

  3. [2]

    Conversational speech transcription using context-dependent deep neural networks,

    F. Seide, G. Li, and D. Yu, “Conversational speech transcription using context-dependent deep neural networks,” in Interspeech, 2011

    work page 2011

  4. [3]

    A regression approach to speech enhancement based on deep neural networks,

    Y . Xu, J. Du, L.-R. Dai, and C.-H. Lee, “A regression approach to speech enhancement based on deep neural networks,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 23, no. 1, pp. 7–19, 2014

    work page 2014

  5. [4]

    Listen, attend and spell: A neural network for large vocabulary conversational speech recognition,

    W. Chan, N. Jaitly, Q. Le, and O. Vinyals, “Listen, attend and spell: A neural network for large vocabulary conversational speech recognition,” in 2016 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, 2016, pp. 4960–4964

    2016

  6. [5]

    Robust speech recognition via large-scale weak super- vision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak super- vision,” in International Conference on Machine Learning . PMLR, 2023, pp. 28 492–28 518

    2023

  7. [6]

    A multi-device dataset for urban acoustic scene classification,

    A. Mesaros, T. Heittola, and T. Virtanen, “A multi-device dataset for urban acoustic scene classification,” arXiv preprint arXiv:1807.09840 , 2018

    arXiv 2018

  8. [7]

    Unsupervised adversarial domain adaptation for acoustic scene classification

    S. Gharib, K. Drossos, E. Cakir, D. Serdyuk, and T. Virtanen, “Unsu- pervised adversarial domain adaptation for acoustic scene classification,” arXiv preprint arXiv:1808.05777 , 2018

    work page Pith review arXiv 2018

Show all 77 references
  1. [8]

    Low- complexity models for acoustic scene classification based on re- ceptive field regularization and frequency damping,

    K. Koutini, F. Henkel, H. Eghbal-zadeh, and G. Widmer, “Low- complexity models for acoustic scene classification based on re- ceptive field regularization and frequency damping,” arXiv preprint arXiv:2011.02955, 2020

    2011 arXiv

  2. [9]

    Novel augmentation schemes for device robust acoustic scene classification

    S. Sonowal and A. Tamse, “Novel augmentation schemes for device robust acoustic scene classification.” in INTERSPEECH 2022, 2022, pp. 4182–4186

    2022

  3. [10]

    Device-robust acoustic scene classification based on two-stage categorization and data augmen- tation,

    H. Hu, C.-H. H. Yang, X. Xia, X. Bai et al. , “Device-robust acoustic scene classification based on two-stage categorization and data augmen- tation,” arXiv preprint arXiv:2007.08389 , 2020

    2007 arXiv

  4. [11]

    Domain generalization with relaxed instance frequency-wise normalization for multi-device acoustic scene classification,

    B. Kim, S. Yang, J. Kim, H. Park, J. Lee, and S. Chang, “Domain generalization with relaxed instance frequency-wise normalization for multi-device acoustic scene classification,” INTERSPEECH 2022, 2022

    2022

  5. [12]

    Device-robust acoustic scene classification via impulse response augmentation,

    T. Morocutti, F. Schmid, K. Koutini, and G. Widmer, “Device-robust acoustic scene classification via impulse response augmentation,” in 2023 31st European Signal Processing Conference (EUSIPCO) . IEEE, 2023, pp. 176–180

    2023

  6. [13]

    Multi-level distance embedding learning for robust acoustic scene classification with unseen devices,

    G. Jiang, Z. Ma, Q. Mao, and J. Zhang, “Multi-level distance embedding learning for robust acoustic scene classification with unseen devices,” Pattern Analysis and Applications, vol. 26, no. 3, pp. 1089–1099, 2023

    2023

  7. [14]

    An empirical investigation of catastrophic forgetting in gradient-based neural networks,

    I. J. Goodfellow, M. Mirza, D. Xiao, A. Courville, and Y . Bengio, “An empirical investigation of catastrophic forgetting in gradient-based neural networks,” arXiv preprint arXiv:1312.6211 , 2013

    2013 arXiv

  8. [15]

    Max- imum a posteriori adaptation of network parameters in deep models,

    Z. Huang, S. M. Siniscalchi, I.-F. Chen, J. Wu, and C.-H. Lee, “Max- imum a posteriori adaptation of network parameters in deep models,” Interspeech, 2015

    2015

  9. [16]

    Overcoming catastrophic forgetting in neural networks,

    J. Kirkpatrick et al. , “Overcoming catastrophic forgetting in neural networks,” Proceedings of the national academy of sciences , vol. 114, no. 13, pp. 3521–3526, 2017

    2017

  10. [17]

    W. B. Powell, Approximate Dynamic Programming: Solving the curses of dimensionality. John Wiley & Sons, 2007, vol. 703

    2007

  11. [18]

    Why and when can deep-but not shallow-networks avoid the curse of dimension- ality: a review,

    T. Poggio, H. Mhaskar, L. Rosasco, B. Miranda, and Q. Liao, “Why and when can deep-but not shallow-networks avoid the curse of dimension- ality: a review,” International Journal of Automation and Computing , vol. 14, no. 5, pp. 503–519, 2017

    2017

  12. [19]

    B. D. Ripley, Pattern recognition and neural networks . Cambridge university press, 1996. 11

    1996

  13. [20]

    On adaptive decision rules and decision parameter adaptation for automatic speech recognition,

    C.-H. Lee and Q. Huo, “On adaptive decision rules and decision parameter adaptation for automatic speech recognition,” Proceedings of the IEEE, vol. 88, no. 8, pp. 1241–1269, 2000

    2000

  14. [21]

    Maximum a posteriori estimation for multivariate gaussian mixture observations of markov chains,

    J.-L. Gauvain and C.-H. Lee, “Maximum a posteriori estimation for multivariate gaussian mixture observations of markov chains,” IEEE Transactions on speech and audio processing, vol. 2, no. 2, pp. 291–298, 1994

    1994

  15. [22]

    Bayesian unsupervised batch and online speaker adaptation of activation function parameters in deep models for automatic speech recognition,

    Z. Huang, S. M. Siniscalchi, and C.-H. Lee, “Bayesian unsupervised batch and online speaker adaptation of activation function parameters in deep models for automatic speech recognition,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 25, no. 1, pp. 64–75, 2017

    2017

  16. [23]

    R. O. Duda, P. E. Hart et al., Pattern classification and scene analysis . Wiley New York, 1973, vol. 3

    1973

  17. [24]

    Joint maximum a posteriori adaptation of transformation and hmm parameters,

    O. Siohan, C. Chesta, and C.-H. Lee, “Joint maximum a posteriori adaptation of transformation and hmm parameters,” IEEE Transactions on Speech and Audio Processing , vol. 9, no. 4, pp. 417–428, 2001

    2001

  18. [25]

    On-line adaptive learning of the continuous density hidden markov model based on approximate recursive bayes estimate,

    Q. Huo and C.-H. Lee, “On-line adaptive learning of the continuous density hidden markov model based on approximate recursive bayes estimate,” IEEE transactions on speech and audio processing , vol. 5, no. 2, pp. 161–172, 1997

    1997

  19. [26]

    A structural bayes approach to speaker adaptation,

    K. Shinoda and C.-H. Lee, “A structural bayes approach to speaker adaptation,” IEEE Transactions on Speech and Audio Processing, vol. 9, no. 3, pp. 276–287, 2001

    2001

  20. [27]

    Hierarchical bayesian combination of plug-in maximum a posteriori decoders in deep neural networks-based speech recognition and speaker adaptation,

    Z. Huang, S. M. Siniscalchi, and C.-H. Lee, “Hierarchical bayesian combination of plug-in maximum a posteriori decoders in deep neural networks-based speech recognition and speaker adaptation,” Pattern Recognition Letters, vol. 98, pp. 1–7, 2017

    2017

  21. [28]

    Probabilistic inference using markov chain monte carlo methods,

    R. M. Neal, “Probabilistic inference using markov chain monte carlo methods,” 1993

    1993

  22. [29]

    P. S. Maybeck, Stochastic models, estimation, and control . Academic press, 1982

    1982

  23. [30]

    Variational algorithms for approximate bayesian inference,

    M. J. Beal, “Variational algorithms for approximate bayesian inference,” Ph.D. dissertation, UCL (University College London), 2003

    2003

  24. [31]

    Variational bayesian inference with stochastic search,

    J. Paisley, D. Blei, and M. Jordan, “Variational bayesian inference with stochastic search,” arXiv preprint arXiv:1206.6430 , 2012

    2012 arXiv

  25. [32]

    Variational bayesian estimation and clustering for speech recognition,

    S. Watanabe, Y . Minami, A. Nakamura, and N. Ueda, “Variational bayesian estimation and clustering for speech recognition,” IEEE Trans- actions on Speech and Audio Processing , vol. 12, no. 4, pp. 365–381, 2004

    2004

  26. [33]

    Auto-encoding variational bayes,

    D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114, 2013

    2013 arXiv

  27. [35]

    Variational information bot- tleneck for effective low-resource audio classification,

    S. Si, J. Wang, H. Sun, J. Wu et al. , “Variational information bot- tleneck for effective low-resource audio classification,” arXiv preprint arXiv:2107.04803, 2021

    2021 arXiv

  28. [36]

    Learning small-size dnn with output-distribution-based criteria,

    J. Li, R. Zhao, J.-T. Huang, and Y . Gong, “Learning small-size dnn with output-distribution-based criteria,” in Interspeech, 2014

    2014

  29. [37]

    Distilling the knowledge in a neural network,

    G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531 , 2015

    2015 arXiv

  30. [38]

    Fitnets: Hints for thin deep nets,

    A. Romero, N. Ballas, S. E. Kahou, A. Chassang, C. Gatta, and Y . Ben- gio, “Fitnets: Hints for thin deep nets,” arXiv preprint arXiv:1412.6550, 2014

    2014 arXiv

  31. [39]

    Paying more attention to attention: Improving the performance of convolutional neural networks via atten- tion transfer,

    S. Zagoruyko and N. Komodakis, “Paying more attention to attention: Improving the performance of convolutional neural networks via atten- tion transfer,” arXiv preprint arXiv:1612.03928 , 2016

    2016 arXiv

  32. [40]

    A comprehensive overhaul of feature distillation,

    B. Heo, J. Kim, S. Yun, H. Park, N. Kwak, and J. Y . Choi, “A comprehensive overhaul of feature distillation,” in CVPR, 2019, pp. 1921–1930

    2019

  33. [42]

    Practical variational inference for neural networks,

    A. Graves, “Practical variational inference for neural networks,” in Advances in neural information processing systems . Citeseer, 2011, pp. 2348–2356

    2011

  34. [43]

    Unifying dis- tillation and privileged information,

    D. Lopez-Paz, L. Bottou, B. Sch ¨olkopf, and V . Vapnik, “Unifying dis- tillation and privileged information,” arXiv preprint arXiv:1511.03643 , 2015

    2015 arXiv

  35. [44]

    Learning using privileged information: similarity control and knowledge transfer

    V . Vapnik and R. Izmailov, “Learning using privileged information: similarity control and knowledge transfer.” J. Mach. Learn. Res., vol. 16, no. 1, pp. 2023–2049, 2015

    2023

  36. [45]

    Large- scale domain adaptation via teacher-student learning,

    J. Li, M. L. Seltzer, X. Wang, R. Zhao, and Y . Gong, “Large- scale domain adaptation via teacher-student learning,” arXiv preprint arXiv:1708.05466, 2017

    2017 arXiv

  37. [46]

    On information and sufficiency,

    S. Kullback and R. A. Leibler, “On information and sufficiency,” The annals of mathematical statistics , vol. 22, no. 1, pp. 79–86, 1951

    1951

  38. [47]

    M ´ezard, G

    M. M ´ezard, G. Parisi, and M. A. Virasoro, Spin glass theory and beyond: An Introduction to the Replica Method and Its Applications . World Scientific Publishing Company, 1987, vol. 9

    1987

  39. [48]

    Ambjørn, B

    J. Ambjørn, B. Durhuus, T. Jonsson, and O. Jonsson, Quantum geom- etry: a statistical field theory approach . Cambridge University Press, 1997

    1997

  40. [49]

    Pattern recognition,

    C. M. Bishop, “Pattern recognition,” Machine learning, vol. 128, no. 9, 2006

    2006

  41. [50]

    Robbins, An empirical Bayes approach to statistics

    H. Robbins, An empirical Bayes approach to statistics . University of California Press, 1964

    1964

  42. [51]

    Fixed-form variational posterior approximation through stochastic linear regression,

    T. Salimans, D. A. Knowles et al. , “Fixed-form variational posterior approximation through stochastic linear regression,” Bayesian Analysis, vol. 8, no. 4, pp. 837–882, 2013

    2013

  43. [52]

    M. H. DeGroot, Optimal statistical decisions . John Wiley & Sons, 1970, vol. 82

    1970

  44. [53]

    Relational teacher student learning with neural label embedding for device adaptation in acoustic scene classification,

    H. Hu, S. M. Siniscalchi, Y . Wang, and C.-H. Lee, “Relational teacher student learning with neural label embedding for device adaptation in acoustic scene classification,” Interspeech, 2020

    2020

  45. [54]

    Relational knowledge distilla- tion,

    W. Park, D. Kim, Y . Lu, and M. Cho, “Relational knowledge distilla- tion,” in CVPR, 2019, pp. 3967–3976

    2019

  46. [55]

    Correlation congruence for knowledge distillation,

    B. Peng, X. Jin, J. Liu, D. Li, Y . Wu, Y . Liu, S. Zhou, and Z. Zhang, “Correlation congruence for knowledge distillation,” in CVPR, 2019, pp. 5007–5016

    2019

  47. [56]

    Approximating the kullback leibler divergence between gaussian mixture models,

    J. R. Hershey and P. A. Olsen, “Approximating the kullback leibler divergence between gaussian mixture models,” inICASSP, vol. 4. IEEE, 2007, pp. IV–317

    2007

  48. [57]

    Robust estimation of a location parameter,

    P. J. Huber, “Robust estimation of a location parameter,” in Break- throughs in statistics . Springer, 1992, pp. 492–518

    1992

  49. [58]

    Acoustic scene classification in dcase 2020 challenge: generalization across devices and low com- plexity solutions,

    T. Heittola, A. Mesaros, and T. Virtanen, “Acoustic scene classification in dcase 2020 challenge: generalization across devices and low com- plexity solutions,” arXiv preprint arXiv:2005.14623 , 2020

    2020 arXiv

  50. [59]

    Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences,

    S. Davis and P. Mermelstein, “Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences,” IEEE transactions on acoustics, speech, and signal processing , vol. 28, no. 4, pp. 357–366, 1980

    1980

  51. [60]

    A two-stage approach to device-robust acoustic scene classification,

    H. Hu, C.-H. H. Yang, X. Xia, X. Bai et al., “A two-stage approach to device-robust acoustic scene classification,” in ICASSP. IEEE, 2021, pp. 845–849

    2021

  52. [61]

    A lottery ticket hypothesis framework for low-complexity device-robust neural acoustic scene clas- sification,

    H. Yen, C.-H. H. Yang, H. Hu, S. M. Siniscalchi, Q. Wang, Y . Wang, X. Xia, Y . Zhao, Y . Wu, Y . Wang et al. , “A lottery ticket hypothesis framework for low-complexity device-robust neural acoustic scene clas- sification,” arXiv preprint arXiv:2107.01461 , 2021

    2021 arXiv

  53. [62]

    Low- complexity acoustic scene classification for multi-device audio: Analysis of dcase 2021 challenge systems,

    I. Mart ´ın-Morat´o, T. Heittola, A. Mesaros, and T. Virtanen, “Low- complexity acoustic scene classification for multi-device audio: Analysis of dcase 2021 challenge systems,” arXiv preprint arXiv:2105.13734 , 2021

    2021 arXiv

  54. [63]

    mixup: Beyond empirical risk minimization,

    H. Zhang, M. Cisse, Y . N. Dauphin, and D. Lopez-Paz, “mixup: Beyond empirical risk minimization,” arXiv preprint arXiv:1710.09412 , 2017

    2017 arXiv

  55. [64]

    Specaugment: A simple data augmentation method for automatic speech recognition,

    D. S. Park, W. Chan, Y . Zhang, C.-C. Chiu, B. Zoph, E. D. Cubuk, and Q. V . Le, “Specaugment: A simple data augmentation method for automatic speech recognition,” arXiv preprint arXiv:1904.08779 , 2019

    1904 arXiv

  56. [65]

    Speech commands: A dataset for limited-vocabulary speech recognition,

    P. Warden, “Speech commands: A dataset for limited-vocabulary speech recognition,” arXiv preprint arXiv:1804.03209 , 2018

    2018 arXiv

  57. [66]

    A neural attention model for speech command recognition,

    D. C. de Andrade, S. Leo, M. L. D. S. Viana, and C. Bernkopf, “A neural attention model for speech command recognition,” arXiv preprint arXiv:1808.08929, 2018

    2018 arXiv

  58. [67]

    Decentralizing feature extraction with quantum convolutional neural network for automatic speech recognition,

    C.-H. H. Yang, J. Qi, S. Y .-C. Chen, P.-Y . Chen, S. M. Siniscalchi, X. Ma, and C.-H. Lee, “Decentralizing feature extraction with quantum convolutional neural network for automatic speech recognition,” in ICASSP. IEEE, 2021, pp. 6523–6527

    2021

  59. [68]

    Hello edge: Keyword spotting on microcontrollers,

    Y . Zhang, N. Suda, L. Lai, and V . Chandra, “Hello edge: Keyword spotting on microcontrollers,” arXiv preprint arXiv:1711.07128 , 2017

    2017 arXiv

  60. [69]

    Single-word speech recognition with convolutional neural networks on raw waveforms,

    P. Jansson, “Single-word speech recognition with convolutional neural networks on raw waveforms,” 2018

    2018

  61. [70]

    Neural Model Reprogramming with Similarity Based Mapping for Low-Resource Spoken Command Recognition,

    H. Yen, P.-J. Ku, C.-H. H. Yang, H. Hu, S. M. Siniscalchi, P.-Y . Chen, and Y . Tsao, “Neural Model Reprogramming with Similarity Based Mapping for Low-Resource Spoken Command Recognition,” in INTERSPEECH 2023, 2023, pp. 3317–3321

    2023

  62. [71]

    L-vector: Neural label embedding for domain adaptation,

    Z. Meng, H. Hu, J. Li, C. Liu, Y . Huang, Y . Gong, and C.-H. Lee, “L-vector: Neural label embedding for domain adaptation,” in ICASSP. IEEE, 2020, pp. 7389–7393

    2020

  63. [72]

    Knowledge transfer via distillation of activation boundaries formed by hidden neurons,

    B. Heo, M. Lee, S. Yun, and J. Y . Choi, “Knowledge transfer via distillation of activation boundaries formed by hidden neurons,” inAAAI, vol. 33, no. 01, 2019, pp. 3779–3787. 12

    2019

  64. [73]

    Variational information distillation for knowledge transfer,

    S. Ahn, S. X. Hu, A. Damianou, N. D. Lawrence, and Z. Dai, “Variational information distillation for knowledge transfer,” in CVPR, 2019, pp. 9163–9171

    2019

  65. [74]

    A gift from knowledge distillation: Fast optimization, network minimization and transfer learning,

    J. Yim, D. Joo, J. Bae, and J. Kim, “A gift from knowledge distillation: Fast optimization, network minimization and transfer learning,” in CVPR, 2017, pp. 4133–4141

    2017

  66. [75]

    Similarity-preserving knowledge distillation,

    F. Tung and G. Mori, “Similarity-preserving knowledge distillation,” in CVPR, 2019, pp. 1365–1374

    2019

  67. [76]

    Learning deep representations with proba- bilistic knowledge transfer,

    N. Passalis and A. Tefas, “Learning deep representations with proba- bilistic knowledge transfer,” in ECCV, 2018, pp. 268–284

    2018

  68. [77]

    Like what you like: Knowledge distill via neuron selectivity transfer,

    Z. Huang and N. Wang, “Like what you like: Knowledge distill via neuron selectivity transfer,” arXiv preprint arXiv:1707.01219 , 2017

    2017 arXiv

  69. [78]

    Visualizing data using t-sne

    L. Van der Maaten and G. Hinton, “Visualizing data using t-sne.” Journal of machine learning research , vol. 9, no. 11, 2008. Hu Hu received the B.S. degree from the Depart- ment of Computer Science and Technology, Shang- hai Jiao Tong University, Shanghai, China, in 2018. He ...

    2008

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.