Pith. sign in

REVIEW 3 major objections 4 minor 62 references

Towards Pre-training an Effective Respiratory Audio Foundation Model

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Pre-training on general audio beats respiratory-only pre-training for respiratory sound analysis, and adding respiratory data on top sets a new OPERA benchmark record.

desk verdict Useful benchmark study with a credible core finding, but the combined-pre-training SOTA is contaminated by overlapping training data and benchmark-driven selection, so it needs a disjoint evaluation before the headline number is trusted. read the letter →

arxiv 2505.15307 v1 pith:W4V7RKFJ submitted 2025-05-21 eess.AS cs.SD

Daisuke Niizumi , Daiki Takeuchi , Masahiro Yasuda , Binh Thien Nguyen , Yasunori Ohishi , Noboru Harada This is my paper · ORCID
classification eess.AScs.SD
keywords respiratoryaudiofoundationmodelpre-trainingSetself-supervisedlearningOPERAbenchmarkfrequency-wisefeatureaggregationcoughclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks what pre-training practice actually yields an effective respiratory audio foundation model, and answers with a broad comparison on the OPERA benchmark. Across 21 audio models, the authors find that models pre-trained on AudioSet, a large general audio collection that includes coughs, breathing, and many other sounds, outperform models pre-trained exclusively on respiratory audio. The recommended recipe is two-stage: start from the self-supervised M2D model trained on AudioSet, then further pre-train it on a mix of AudioSet and respiratory sound data (COUGHVID, HF Lung, and ICBHI). That model, M2D+Resp, reaches an average AUROC of 0.814 on OPERA's seven tasks, beating the previous respiratory-specific state of the art (OPERA-CT, 0.733). The paper also reports that preserving frequency-wise structure when aggregating patch features is central to the gain; mean pooling the features drops average AUROC from 0.782 to 0.712 in ablations.

What carries the argument

The workhorse is the M2D model (Masked Modeling Duo), a self-supervised audio model that learns by reconstructing masked spectrogram patches, with the M2D-X variant used for further pre-training on small datasets by adding AudioSet as background noise. The paper uses M2D as a controlled testbed to ablate pre-training data, training objective, patch resolution, and feature aggregation. The decisive mechanism is that M2D summarizes a spectrogram by concatenating frequency-wise patch features for each time frame, preserving spectral information; switching to mean pooling drops average AUROC from 0.782 to 0.712. The other load-bearing mechanism is two-stage pre-training: beginning from the AudioSet-trained M2D and refining it on a mix of AudioSet samples and respiratory recordings.

What would settle it

Run the same M2D+Resp recipe but evaluate on a respiratory corpus whose recordings were held out of pre-training entirely, or repeat the pre-training after removing COUGHVID, HF Lung, and ICBHI from the training mix and test on those tasks. If the average AUROC advantage of M2D+Resp over the AudioSet-only M2D collapses to roughly zero or reverses, the central claim that combined pre-training helps would be falsified. A second check is to compare mean-pooled versus frequency-wise concatenation on such held-out tasks to see whether the aggregation benefit persists outside OPERA.

Watch

Extended reading notes

Core claim

The central discovery is that, for respiratory audio, the variety and scale of pre-training data outweighs domain specificity. Eight of the nineteen general-audio models evaluated beat OPERA-CT, the best respiratory-only model, and the top general-audio models beat it on every task. The strongest configuration is M2D+Resp: the AudioSet-pre-trained M2D model further pre-trained on a combined AudioSet and respiratory dataset of 400,000 samples, which reaches 0.814 average AUROC on OPERA versus 0.733 for OPERA-CT. Ablations on M2D isolate two deciding factors: the data, with further pre-training on AudioSet plus respiratory sounds helping while respiratory sounds alone do not, and the feature aggregation, with concatenating frequency-wise patch features being far better than mean pooling. Speech-based SSL models underperform because their training signals are built from speech clusters that carry few breath-like acoustic patterns.

Load-bearing premise

The load-bearing premise is that the overlap between the further-pre-training dataset, which contains COUGHVID, HF Lung, and ICBHI recordings including the training splits of benchmark tasks T5-T7, does not inflate the reported gains; if it does, the advantage of combining respiratory data with AudioSet may not hold for unseen respiratory recordings.

Editorial extensions

If this is right

  • Future respiratory foundation models should pre-train on large diverse audio before any respiratory-specific adaptation, since respiratory-only pre-training is a weak base.
  • Two-stage pre-training on AudioSet plus respiratory data is a concrete recipe that raises average AUROC to 0.814, a new OPERA record.
  • Feature aggregation choices matter as much as data: frequency-wise concatenation should be the default for respiratory tasks, not mean pooling.
  • The penultimate layer of the AudioSet model is the best feature source across tasks, so respiratory systems should use deep-layer outputs rather than early or middle layers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported gain from combining AudioSet with respiratory data is measured on benchmark tasks whose training splits overlap the pre-training set (T5-T7); whether the gain survives on fully unseen respiratory recordings is an open question the paper does not answer.
  • The frequency-wise aggregation insight likely transfers to other bioacoustic tasks such as heart-sound or bowel-sound classification, where spectral details carry diagnostic information.
  • A natural extension is to apply the same AudioSet-plus-respiratory two-stage recipe to other strong base models, for example ATST-Frame or CED, to see whether the gain is specific to M2D or general.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper evaluates 21 pre-trained audio foundation models on seven tasks (T5–T11) of the OPERA respiratory audio benchmark under a linear-probing protocol. It reports that models pre-trained on AudioSet outperform the respiratory-specific OPERA-CT and OPERA-GT models, that further pre-training an AudioSet-pretrained M2D on a combination of AudioSet and respiratory sound data ('M2D+Resp') yields a new state of the art (average AUROC 0.814 vs 0.733), and that preserving frequency-wise information in feature aggregation is important. Ablation experiments vary training objective, patch resolution, data composition, and layer choice, and the paper releases evaluation code.

Significance. If the central result holds, the paper provides actionable guidance for pre-training respiratory audio models and a reusable benchmark harness for the community. The study is a well-structured empirical comparison: it reports means and standard deviations over five runs, compares a broad set of external models (ATST-Frame, MSM-MAE, CLAP, etc.), and makes code publicly available. The claim that general AudioSet pre-training is more effective than the two evaluated respiratory-specific models is supported by multiple models not developed by the authors. However, the headline SOTA claim for M2D+Resp is currently weakened by two issues: the further pre-training data overlaps with the training splits of tasks T5–T7, and the 400K configuration was selected by benchmarking on the same test tasks. These issues need to be addressed before the combined-pretraining benefit can be considered established.

major comments (3)
  1. [Table 4, §3.2] The headline result M2D+Resp (average AUROC 0.814) comes from further pre-training on a respiratory set that, as the table's own footnote states, 'contains the data from tasks with ∈Resp' — i.e., the training splits of T5–T7. Consequently, the gains on T5 and T6 (0.627 vs 0.595 and 0.856 vs 0.797) may reflect exposure to the exact benchmark training recordings rather than improved generalization to unseen respiratory audio. Gains on the disjoint tasks T8–T10 do exist, which partially mitigates the concern, but the headline average aggregates the overlapping tasks. Please add a held-out evaluation on disjoint tasks or on disjoint splits of the respiratory datasets, or substantially temper the claim that combining AudioSet with respiratory data 'enhances performance.'
  2. [Table 4, §3.2] The 400K sample count for M2D+Resp was selected by running configurations of 100K, 200K, 300K, 400K, and 500K and choosing the one with the highest average AUROC on the same OPERA benchmark. No validation split or pre-registered selection criterion is used, so the reported 0.814 is the maximum of a configuration sweep and is likely optimistically biased. The same issue appears in Table 2 for the speech SSL models (rows 4–6), where the 'Layer#7' notation indicates a layer chosen by benchmark performance, and in Figure 1, where the penultimate layer is identified as best. Please report the selection procedure explicitly, use a validation split for configuration and layer selection, or present the chosen configuration as exploratory and confirm it on a held-out benchmark.
  3. [Table 1, Table 2] The average AUROC across the seven tasks is strongly influenced by tasks with very small sample sizes and ceiling effects. T7 has only 35 samples in one class and reaches 1.000 for several models (AST, MSM-MAE, M2D, M2D+Resp), and T10 has 129/105 samples. With only five runs, the reported standard deviations are small, but the class imbalance and saturation make per-task differences less informative and the average less representative. Please report per-task confidence intervals and include a sensitivity analysis that excludes or reweights T7 (and possibly T10) to show that the main conclusions are not driven by these tasks.
minor comments (4)
  1. [Section 3] The heading 'Emperical Analysis' should be 'Empirical Analysis.'
  2. [Table 2 caption] The asterisk is explained only in the running text after the table; please add a legend to the caption (e.g., '* indicates better than OPERA-CT').
  3. [Table 4 footnote] The notation '∈Resp' is used without definition in the caption; please define it explicitly, for example, 'Resp denotes the respiratory pre-training set, which includes the training splits of tasks T5–T7.'
  4. [Abstract] The abstract states that AudioSet models 'are more effective than the models specifically pre-trained on respiratory sounds,' but only two such models (OPERA-CT and OPERA-GT) are evaluated; 'the evaluated respiratory-specific models' would be more precise.

Circularity Check

1 steps flagged · score 4.0 of 10

Empirical benchmark study; the AudioSet-vs-respiratory finding is independent, but the M2D+Resp SOTA is partly reduced by pre-training on the T5-T7 training splits before evaluating those tasks.

  1. fitted input called prediction [Section 3.2 (RQ2), Table 4 caption and surrounding text]
    "The respiratory pre-training data contains the data from tasks with ∈Resp. ... Note that the respiratory sound dataset used for the pre-training contains the training data from tasks T5 to T7."

    The reported SOTA model, M2D+Resp, was created by further pre-training on a respiratory set that includes the training splits of benchmark tasks T5-T7. The headline average 0.814 then aggregates the AUROC values for those same tasks. For T5-T7, the feature extractor has already been adapted to the exact unlabeled training recordings whose labels the linear probe subsequently learns, so those three scores are transductive fits rather than predictions for unseen respiratory audio. The paper discloses the overlap, and genuine gains on disjoint tasks T8-T10 do exist, so the reduction is partial rather than total; nevertheless, the magnitude of the combined AudioSet+Resp benefit is not established without a held-out evaluation on disjoint respiratory data.

full rationale

This is an empirical benchmark study, not a formal derivation: there are no equations whose outputs equal their inputs. Self-citation is present (M2D, BYOL-A, MSM-MAE are the authors' models), but it is not load-bearing in the circular sense because the central conclusion that AudioSet pre-training beats respiratory-only pre-training is corroborated by external models such as ATST-Frame and CLAP2023, which also outperform OPERA-CT. The main concrete reduction is the benchmark overlap in the M2D+Resp result: the further pre-training dataset contains the training data of tasks T5-T7, and the reported SOTA averages exactly those tasks, so part of the gain is forced by the pre-training input set overlapping the evaluation tasks. The paper is transparent about this overlap, and gains on disjoint tasks T8-T10 do exist, which mitigates the issue. Model selection on the same benchmark (choosing 400K among several AS+Resp sizes and resolutions) adds optimism but is a statistical selection effect, not equation-level circularity. Overall, the central scientific content is independent, with one partial fitted-input-called-prediction issue.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new mathematical entities or physical postulates. The free parameters are experimental design choices selected on the benchmark itself, and the axioms are assumptions about the validity of the evaluation protocol and the benign effect of data overlap.

free parameters (2)
  • Further pre-training sample count (400K) = 400K (29x augmentation of 14K respiratory samples)
    Selected as the best among 100K/200K/300K/400K/500K on the OPERA benchmark; this is a benchmark-driven choice, not a principled rule.
  • Base model selection (M2D) = M2D
    M2D was chosen for ablations and final SOTA because it had the highest average performance in the initial 21-model comparison; results may depend on this choice.
assumptions (3)
  • domain assumption The OPERA benchmark's linear evaluation protocol is a valid proxy for respiratory audio representation quality.
    The paper's conclusions all rest on this protocol; no evidence is given that linear probe AUROC predicts real-world diagnostic performance.
  • domain assumption Pre-training on the same datasets used for evaluation (including exact training samples) does not materially inflate results.
    The respiratory pre-training data includes COUGHVID and ICBHI, which are also the sources of tasks T5-T7; the paper acknowledges this overlap but does not measure its effect with a disjoint split.
  • domain assumption The five-run mean AUROC is a sufficient statistic for model comparison.
    No significance tests are performed across models; tasks have very different sample sizes and class balances.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Pre-training an Effective Respiratory Audio Foundation Model." pith.science (2026). https://pith.science/paper/W4V7RKFJ

@misc{pith2026250515307,
  author       = {Pith},
  title        = {Pith review of: Towards Pre-training an Effective Respiratory Audio Foundation Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/W4V7RKFJ}},
  note         = {Machine review of arXiv:2505.15307}
}
read the original abstract

Recent advancements in foundation models have sparked interest in respiratory audio foundation models. However, the effectiveness of applying conventional pre-training schemes to datasets that are small-sized and lack diversity has not been sufficiently verified. This study aims to explore better pre-training practices for respiratory sounds by comparing numerous pre-trained audio models. Our investigation reveals that models pre-trained on AudioSet, a general audio dataset, are more effective than the models specifically pre-trained on respiratory sounds. Moreover, combining AudioSet and respiratory sound datasets for further pre-training enhances performance, and preserving the frequency-wise information when aggregating features is vital. Along with more insights found in the experiments, we establish a new state-of-the-art for the OPERA benchmark, contributing to advancing respiratory audio foundation models. Our code is available online at https://github.com/nttcslab/eval-audio-repr/tree/main/plugin/OPERA.

Figures

Figures reproduced from arXiv: 2505.15307 by the authors.

Figure 1
Figure 1. M2D performance by layers. dataset. Note that the respiratory sound dataset used for the pre-training contains the training data from tasks T5 to T7. Further pre-training on data combining AudioSet and res￾piratory sound datasets improves performance. In the exper￾iments, we pre-trained M2D from scratch or further pre-trained it on various datasets. Among various attempts, further pre￾trainings of AS+Resp 100 to 500… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

62 extracted references · 57 canonical work pages

  1. [1]

    These models may serve as a fundamen- tal block for health monitoring and disease diagnosis applica- tions

    Introduction As a trend in non-invasive diagnostic methods using sound [1], respiratory audio foundation models have gained attention, driven by recent advancements in deep learning foundation models [2, 3, 4]. These models may serve as a fundamen- tal block for health monitoring and disease diagnosis applica- tions. Since training foundation models requi...

    work page

  2. [2]

    Experimental Setup We utilized a unified respiratory audio benchmark to evaluate models under the following experimental setup. 2.1. Benchmark We conducted experiments on the publicly available OPERA

    work page

  3. [3]

    HTS-AT [13] 0.577 ±0.002 0.615±0.001 0.765±0.013 0.590±0.010 0.699±0.002 0.778±0.014∗ 0.521±0.031 0.649 Speech SSL models (with the best performing layer used for extracting features)

    work page

  4. [4]

    The seven tasks listed in Table 1 are health condition classification problems, where all are binary classification tasks except T11, which is a five-class classifica- tion

    benchmark and used seven tasks (T5 to T11) for which the data are publicly available. The seven tasks listed in Table 1 are health condition classification problems, where all are binary classification tasks except T11, which is a five-class classifica- tion. The benchmark takes a linear evaluation protocol, where model weights are frozen. A model encodes...

    arXiv 2025

  5. [5]

    AST [11] 0.607 ±0.008∗ 0.739±0.002 1.000±0.000∗ 0.671±0.011 0.833±0.000 0.837±0.007∗ 0.652±0.020∗ 0.763∗

    work page

  6. [6]

    PANNs Cnn14 [12] 0.533±0.006 0.566±0.003 0.606±0.037 0.533±0.014 0.549±0.007 0.447±0.057 0.527±0.022 0.537

    work page

  7. [7]

    wav2vec2Layer#7[14] 0.480 ±0.005 0.634±0.003 0.172±0.013 0.589±0.020 0.606±0.004 0.620±0.021 0.560±0.022 0.523

    work page

  8. [8]

    HuBERTLayer#7[15] 0.558 ±0.002 0.736±0.001 0.644±0.012 0.683±0.004 0.807±0.002 0.689±0.019 0.658±0.018∗ 0.682

    work page

Show all 62 references
  1. [9]

    WavLMLayer#6[16] 0.555 ±0.002 0.700±0.001 0.599±0.016 0.687±0.004∗ 0.771±0.001 0.703±0.020 0.624±0.011 0.663 CLAP models

  2. [10]

    LAION-CLAP [17] 0.549±0.001 0.660±0.001 0.674±0.062 0.531±0.003 0.714±0.002 0.776±0.015∗ 0.584±0.033 0.641

  3. [11]

    CLAP2022[18] 0.599 ±0.007∗ 0.665±0.001 0.933±0.005∗ 0.680±0.009 0.742±0.001 0.697±0.004 0.636±0.045∗ 0.707

  4. [12]

    CLAP2023[19] 0.602 ±0.007∗ 0.779±0.001 0.988±0.004∗ 0.687±0.007∗ 0.866±0.001 0.795±0.012∗ 0.606±0.037 0.760∗ General audio SSL models

  5. [13]

    BYOL-A [20] 0.531 ±0.011 0.702±0.006 0.950±0.031∗ 0.581±0.035 0.807±0.011 0.710±0.047 0.566±0.029 0.693

  6. [14]

    ATST-Clip [21] 0.609±0.009∗ 0.793±0.001 0.977±0.007 0.679±0.010 0.850±0.002 0.828±0.012 0.606±0.029 0.763∗

  7. [15]

    ATST-Frame [21]0.621±0.007∗ 0.801±0.001∗ 0.998±0.001∗ 0.687±0.010∗ 0.908±0.001∗ 0.843±0.006∗ 0.657±0.003∗ 0.788∗

  8. [16]

    AudioMAE [22] 0.554±0.004 0.628±0.001 0.886±0.017∗ 0.549±0.022 0.724±0.001 0.616±0.041 0.510±0.021 0.638

  9. [17]

    BEATs [23] 0.555 ±0.002 0.644±0.001 0.823±0.011 0.631±0.004 0.695±0.003 0.723±0.031∗ 0.623±0.015 0.670

  10. [18]

    MSM-MAE [24] 0.569±0.003 0.781±0.000 1.000±0.000∗ 0.721±0.008∗ 0.879±0.001∗ 0.746±0.009∗ 0.662±0.006∗ 0.765∗

  11. [19]

    M2D [25] 0.595 ±0.008∗ 0.797±0.000∗ 1.000±0.000∗ 0.703±0.024∗ 0.905±0.001∗ 0.756±0.013∗ 0.720±0.012∗ 0.782∗ Ensemble SSL model (CED), large parameter model (Dasheng 1.2B), and audio-visual contrastive SSL model (OpenL3)

  12. [20]

    CED [26] 0.614 ±0.001∗ 0.782±0.001 0.997±0.001∗ 0.713±0.007∗ 0.873±0.001 0.833±0.019∗ 0.597±0.117 0.773∗

  13. [21]

    Dasheng-1.2B [27] 0.582±0.005∗ 0.734±0.002 0.915±0.031∗ 0.662±0.016 0.772±0.002 0.700±0.072 0.660±0.021∗ 0.718

  14. [22]

    OpenL3[28] 0.608 ±0.011∗ 0.754±0.006 0.978±0.007∗ 0.695±0.018∗ 0.845±0.007 0.751±0.027∗ 0.639±0.025∗ 0.753∗ Respiratory audio SSL models

  15. [23]

    OPERA-CT [4] 0.578±0.001 0.795±0.001 0.855±0.012 0.685±0.012 0.874±0.000 0.722±0.016 0.625±0.038 0.733

  16. [24]

    Bold results are better than the models above

    OPERA-GT [4] 0.552±0.003 0.735±0.000 0.741±0.011 0.650±0.005 0.825±0.001 0.703±0.016 0.606±0.015 0.687 Reference from Table 3: M2D further pre-trained on AudioSet + Resipratory sound data. Bold results are better than the models above. M2D+Resp 0.627±0.009∗ 0.856±0.001∗ 1.000±...

  17. [25]

    We also evaluated the layer-wise performance (Figures 1 and 2)

    Emperical Analysis We evaluated 21 audio foundation models on the OPERA benchmark (Table 2) and conducted ablation experiments on the M2D model (Table 3) and on data (Table 4). We also evaluated the layer-wise performance (Figures 1 and 2). 3.1. RQ1 What are the better practic...

    2023

  18. [26]

    For further pre-training experiments, we pre- trained the AudioSet-pre-trained M2D for 50 epochs on each Table 4: Data ablations of M2D on OPERA benchmark

    We used the same setting with M2D and replaced the pre- training dataset. For further pre-training experiments, we pre- trained the AudioSet-pre-trained M2D for 50 epochs on each Table 4: Data ablations of M2D on OPERA benchmark. The respiratory pre-training data contains the ...

  19. [27]

    Conclusion We investigated the better practices for pre-training an effective respiratory audio foundation model by comparing numerous au- dio models under the unified respiratory benchmark OPERA. Experiments provided various insights, such as the effective- ness of pre-traini...

  20. [28]

    Body acoustics for the non-invasive diagnosis of medical conditions,

    J. Cook, M. Umar, F. Khalili, and A. Taebi, “Body acoustics for the non-invasive diagnosis of medical conditions,” Bioengineer- ing, vol. 9, no. 4, 2022

    2022

  21. [29]

    HeAR – health acoustic representations,

    S. Baur, Z. Nabulsi, W.-H. Weng, J. Garrison, L. Blankemeier, S. Fishman, C. Chen, S. Kakarmath, M. Maimbolwa, N. Sanjase, B. Shuma, Y . Matias, G. S. Corrado, S. Patel, S. Shetty, S. Prab- hakara, M. Muyoyeta, and D. Ardila, “HeAR – health acoustic representations,” arXiv pre...

    2024 arXiv

  22. [30]

    Foun- dation models for cardiovascular disease detection via biosignals from digital stethoscopes,

    G. Mathew, D. Barbosa, J. Prince, and S. a. Venkatraman, “Foun- dation models for cardiovascular disease detection via biosignals from digital stethoscopes,” npj Cardiovascular Health, 2024

    2024

  23. [31]

    Towards open respiratory acoustic foundation models: Pretraining and benchmarking,

    Y . Zhang, T. Xia, J. Han, Y . Wu, G. Rizos, Y . Liu, M. Mosuily, J. Chauhan, and C. Mascolo, “Towards open respiratory acoustic foundation models: Pretraining and benchmarking,” in NeurIPS, 2024

    2024

  24. [32]

    Auscultabase: A foundational step towards ai-powered body sound diagnostics,

    P. Wang, Z. Zhao, L. Zhao, M. He, X. Sun, Y . Zhang, K. Sun, Y . Wang, and Y . Wang, “Auscultabase: A foundational step towards ai-powered body sound diagnostics,” arXiv preprint arXiv:2411.07547, 2024

    2024 arXiv

  25. [33]

    The COUGHVID crowdsourcing dataset, a corpus for the study of large-scale cough analysis algorithms,

    L. Orlandic, T. Teijeiro, and D. Atienza, “The COUGHVID crowdsourcing dataset, a corpus for the study of large-scale cough analysis algorithms,” Scientific Data, vol. 8, no. 156, 2021

    2021

  26. [34]

    An open access database for the evaluation of respiratory sound classification algorithms,

    B. M. Rocha, D. Filos, L. Mendes, G. Serbes, S. Ulukaya, Y . P. Kahya, N. Jakovljevic, T. L. Turukalo, I. M. V ogiatzis, E. Peran- toni, E. Kaimakamis, P. Natsiavas, A. Oliveira, C. J ´acome, A. Marques, N. Maglaveras, R. P. Paiva, I. Chouvarda, and P. de Carvalho, “An open ac...

    2019

  27. [35]

    Coswara: A respiratory sounds and symp- toms dataset for remote screening of sars-cov-2 infection,

    D. Bhattacharya et al., “Coswara: A respiratory sounds and symp- toms dataset for remote screening of sars-cov-2 infection,” Scien- tific Data, vol. 10, no. 397, 2023

    2023

  28. [36]

    A dataset of lung sounds recorded from the chest wall using an elec- tronic stethoscope,

    M. Fraiwan, L. Fraiwan, B. Khassawneh, and A. Ibnian, “A dataset of lung sounds recorded from the chest wall using an elec- tronic stethoscope,” Data in Brief, vol. 35, p. 106913, 2021

    2021

  29. [37]

    Mul- timedia respiratory database (respiratorydatabase@tr): Ausculta- tion sounds and chest x-rays,

    G. Altan, Y . Kutlu, Y . Garbi, A. O. Pekmezci, and S. Nural, “Mul- timedia respiratory database (respiratorydatabase@tr): Ausculta- tion sounds and chest x-rays,” Natural and Engineering Sciences, vol. 2, no. 3, p. 59–72, 2017

    2017

  30. [38]

    AST: Audio Spectrogram Transformer,

    Y . Gong, Y .-A. Chung, and J. Glass, “AST: Audio Spectrogram Transformer,” in Interspeech, 2021, pp. 571–575

    2021

  31. [39]

    PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition,

    Q. Kong, Y . Cao, T. Iqbal, Y . Wang, W. Wang, and M. D. Plumb- ley, “PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition,” IEEE/ACM Trans. Audio, Speech, Language Process., vol. 28, pp. 2880–2894, 2020

    2020

  32. [40]

    HTS-AT: A hierarchical token-semantic audio transformer for sound classification and detection,

    K. Chen, X. Du, B. Zhu, Z. Ma, T. Berg-Kirkpatrick, and S. Dub- nov, “HTS-AT: A hierarchical token-semantic audio transformer for sound classification and detection,” inICASSP, 2022, pp. 646– 650

    2022

  33. [41]

    wav2vec 2.0: A framework for self-supervised learning of speech representa- tions,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representa- tions,” in NeurIPS, 2020

    2020

  34. [42]

    HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhut- dinov, and A. Mohamed, “HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units,” IEEE/ACM Trans. Audio, Speech, Language Process. , p. 3451–3460, 2021

    2021

  35. [43]

    WavLM: Large- Scale Self-Supervised Pre-Training for Full Stack Speech Pro- cessing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao, J. Wu, L. Zhou, S. Ren, Y . Qian, Y . Qian, J. Wu, M. Zeng, X. Yu, and F. Wei, “WavLM: Large- Scale Self-Supervised Pre-Training for Full Stack Speech Pro- cessing,” IEEE J. Sel. Top. Si...

    2022

  36. [44]

    Large-Scale Contrastive Language-Audio Pretrain- ing with Feature Fusion and Keyword-to-Caption Augmentation,

    Y . Wu, K. Chen, T. Zhang, Y . Hui, T. Berg-Kirkpatrick, and S. Dubnov, “Large-Scale Contrastive Language-Audio Pretrain- ing with Feature Fusion and Keyword-to-Caption Augmentation,” in ICASSP, 2023

    2023

  37. [45]

    CLAP: Learning Audio Concepts From Natural Language Supervision,

    B. Elizalde, S. Deshmukh, M. Al Ismail, and H. Wang, “CLAP: Learning Audio Concepts From Natural Language Supervision,” in ICASSP, 2023

    2023

  38. [46]

    Natural language su- pervision for general-purpose audio representations,

    B. Elizalde, S. Deshmukh, and H. Wang, “Natural language su- pervision for general-purpose audio representations,” in ICASSP, 2024, pp. 336–340

    2024

  39. [47]

    BYOL for Audio: Exploring Pre-trained General-purpose Audio Representations,

    D. Niizumi, D. Takeuchi, Y . Ohishi, N. Harada, and K. Kashino, “BYOL for Audio: Exploring Pre-trained General-purpose Audio Representations,” IEEE/ACM Trans. Audio, Speech, Language Process., vol. 31, p. 137–151, 2023

    2023

  40. [48]

    Self-Supervised Audio Teacher- Student Transformer for Both Clip-Level and Frame-Level Tasks,

    X. Li, N. Shao, and X. Li, “Self-Supervised Audio Teacher- Student Transformer for Both Clip-Level and Frame-Level Tasks,” IEEE/ACM Trans. Audio, Speech, Language Process. , vol. 32, pp. 1336–1351, 2024

    2024

  41. [49]

    Masked autoencoders that lis- ten,

    P.-Y . Huang, H. Xu, J. Li, A. Baevski, M. Auli, W. Galuba, F. Metze, and C. Feichtenhofer, “Masked autoencoders that lis- ten,” in NeurIPS, 2022

    2022

  42. [50]

    BEATs: Audio Pre-Training with Acoustic Tokenizers,

    S. Chen, Y . Wu, C. Wang, S. Liu, D. Tompkins, Z. Chen, and F. Wei, “BEATs: Audio Pre-Training with Acoustic Tokenizers,” in ICML, 2023

    2023

  43. [51]

    Masked Spectrogram Modeling using Masked Autoencoders for Learning General-purpose Audio Representation,

    D. Niizumi, D. Takeuchi, Y . Ohishi, N. Harada, and K. Kashino, “Masked Spectrogram Modeling using Masked Autoencoders for Learning General-purpose Audio Representation,” in HEAR: Holistic Evaluation of Audio Representations (NeurIPS 2021 Competition), vol. 166, 2022, pp. 1–24

    2021

  44. [52]

    Masked Modeling Duo: Towards a Universal Audio Pre- Training Framework,

    ——, “Masked Modeling Duo: Towards a Universal Audio Pre- Training Framework,” IEEE/ACM Trans. Audio, Speech, Lan- guage Process., vol. 32, pp. 2391–2406, 2024

    2024

  45. [53]

    CED: Con- sistent ensemble distillation for audio tagging,

    H. Dinkel, Y . Wang, Z. Yan, J. Zhang, and Y . Wang, “CED: Con- sistent ensemble distillation for audio tagging,” in ICASSP, 2024

    2024

  46. [54]

    Scaling up masked audio encoder learning for general audio clas- sification,

    H. Dinkel, Z. Yan, Y . Wang, J. Zhang, Y . Wang, and B. Wang, “Scaling up masked audio encoder learning for general audio clas- sification,” in Interspeech, 2024, pp. 547–551

    2024

  47. [55]

    Look, listen and learn more: Design choices for deep audio embeddings,

    J. Cramer, H.-H. Wu, J. Salamon, and J. P. Bello, “Look, listen and learn more: Design choices for deep audio embeddings,” in ICASSP, Brighton, UK, May 2019, pp. 3852––3856

    2019

  48. [56]

    Masked autoencoders are scalable vision learners,

    K. He, X. Chen, S. Xie, Y . Li, P. Doll´ar, and R. Girshick, “Masked autoencoders are scalable vision learners,” in CVPR, June 2022, pp. 16 000–16 009

    2022

  49. [57]

    Lib- rispeech: An asr corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Lib- rispeech: An asr corpus based on public domain audio books,” in ICASSP, 2015, pp. 5206–5210

    2015

  50. [58]

    Audio Set: An ontology and human-labeled dataset for audio events,

    J. F. Gemmeke, D. P. W. Ellis, D. Freedman, A. Jansen, W. Lawrence, R. C. Moore, M. Plakal, and M. Ritter, “Audio Set: An ontology and human-labeled dataset for audio events,” in ICASSP, 2017, pp. 776–780

    2017

  51. [59]

    A progressively expanded database for automated lung sound analysis: An update,

    F.-S. Hsu, S.-R. Huang, C.-W. Huang, Y .-R. Cheng, C.-C. Chen, J. Hsiao, C.-W. Chen, and F. Lai, “A progressively expanded database for automated lung sound analysis: An update,” Applied Sciences, vol. 12, no. 15, 2022

    2022

  52. [60]

    M2D-CLAP: Masked Modeling Duo Meets CLAP for Learning General-purpose Audio-Language Representation,

    D. Niizumi, D. Takeuchi, Y . Ohishi, N. Harada, M. Yasuda, S. Tsubaki, and K. Imoto, “M2D-CLAP: Masked Modeling Duo Meets CLAP for Learning General-purpose Audio-Language Representation,” in Interspeech, 2024, pp. 57–61

    2024

  53. [61]

    Masked Modeling Duo for Speech: Specializing General- Purpose Audio Representation to Speech using Denoising Distil- lation,

    D. Niizumi, D. Takeuchi, Y . Ohishi, N. Harada, and K. Kashino, “Masked Modeling Duo for Speech: Specializing General- Purpose Audio Representation to Speech using Denoising Distil- lation,” in Interspeech, 2023, pp. 1294–1298

    2023

  54. [62]

    Composing General Audio Representation by Fusing Mul- tilayer Features of a Pre-trained Model,

    ——, “Composing General Audio Representation by Fusing Mul- tilayer Features of a Pre-trained Model,” in EUSIPCO, 2022, pp. 200–204

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.