REVIEW 3 major objections 3 minor 25 references
Assessing the Utility of Audio Foundation Models for Heart and Respiratory Sound Analysis
T0 review · 3 major / 3 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read Frozen audio foundation models, used without fine-tuning, reach state-of-the-art-level performance on clean respiratory and heart sound tasks but not on noisy ones.
desk verdict Useful frozen-feature benchmark for medical audio, but the BMD-HS 'SOTA' claim is undermined by unverified baseline comparability and likely patient leakage. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the frozen feature extractor: a pretrained audio encoder with weights locked, feeding a small task-specific network (a multilayer perceptron or a 4-block transformer encoder) that is trained on each task's training set. Because the encoder weights are fixed, task performance is attributed to the representation quality of the foundation model itself. The comparison machinery is alignment: each task uses the same data splits, preprocessing, augmentation, and metrics as the prior state-of-the-art studies it is measured against, so the frozen-model scores are directly comparable to published fine-tuned baselines.
What would settle it
Re-run the BMD-HS evaluation with the dataset authors' original multi-class task definition and original train/test split, using the same frozen models; if AST, BEATs, and M2D no longer exceed the baseline's accuracy and F1, the clean-data heart sound claim fails. A second check: apply a denoiser to ICBHI2017 before frozen M2D feature extraction; the noise-based explanation predicts the score should move toward 61.2.
Extended reading notes
Core claim
The central claim is empirical: with weights frozen, today's general-purpose audio foundation models are already competitive with state-of-the-art fine-tuned results on clean auscultation data, and the quality of their frozen representations largely determines task performance. On SPRS, BEATs and M2D reach or approach the best published fine-tuned scores, and on BMD-HS, AST, BEATs, and M2D exceed the original baseline's accuracy and F1. On ICBHI2017 and CirCor, where recordings are noisy, none of the frozen models reaches the SOTA results, though M2D comes closest on CirCor. The paper further reports that OPERA-CT, a respiratory-sound-specific foundation model, underperforms general audio models on respiratory tasks, and attributes this to its smaller and less diverse pretraining corpus (140K monotonic respiratory sounds versus 2M AudioSet samples).
Load-bearing premise
The load-bearing assumption is that the BMD-HS comparison is valid: the paper converts a multi-disease dataset into a binary murmur-presence/absence task with new stratified splits and treats the resulting scores as comparable to the original baseline, without showing that baseline used the same task definition and splits; if that comparability fails, the clean-data heart sound result is not established.
Editorial extensions
If this is right
- For clean stethoscope or microphone recordings, fixed-weight general-purpose audio encoders can be deployed without per-task fine-tuning and still match the best published results.
- On noisy real-life recordings such as ICBHI2017 and CirCor, frozen features are not enough; fine-tuning or a denoising/target-sound-extraction front-end is needed.
- Pretraining on a large, diverse general audio corpus is more valuable for respiratory tasks than pretraining on a narrow respiratory-only corpus of similar scale.
- Recent self-supervised pretraining methods (M2D, BEATs) provide more transferable representations than earlier ones (AST, BYOL-A) in these medical audio tasks.
- The released evaluation code offers a common benchmark for future audio foundation models to report frozen-feature performance.
Reading between the lines
- Extension: if the clean-versus-noisy pattern holds beyond these four datasets, recording quality, not domain match, is the main predictor of whether an audio foundation model can be used frozen; a universal denoising front-end could be the highest-leverage next component.
- Extension: the BMD-HS result rests on a binary murmur-presence/absence redefinition with new stratified splits; the original baseline was not rerun under those exact conditions, so a strict head-to-head on identical splits is still an open check.
- Extension: OPERA-CT's shortfall should not be read as evidence that domain-specific pretraining is useless at any scale; it only shows that 140K respiratory samples do not beat 2M diverse AudioSet samples with current methods.
- Extension: a testable extension is to add denoising or sound separation before frozen encoding and measure whether ICBHI2017 and CirCor scores close the SOTA gap.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper evaluates five audio foundation models (AST, BYOL-A, M2D, BEATs, and OPERA-CT) as frozen feature extractors across four medical sound tasks: ICBHI2017 and SPRS for respiratory sounds and CirCor and BMD-HS for heart sounds. For each task, a task-specific classifier (MLP or a four-block transformer encoder) is trained on the frozen features, and results are compared with published fine-tuned baselines and SOTA results. The authors report that general-purpose audio models consistently outperform the respiratory-sound-specific OPERA-CT, that frozen models are competitive on clean-data tasks (SPRS, BMD-HS) but not on noisy-data tasks (ICBHI2017, CirCor), and they release evaluation code. The central claim is that frozen general-purpose audio representations are practically useful for clean medical sound recordings.
Significance. If fully supported, the paper would provide useful practical guidance: frozen general-purpose audio encoders can serve as strong feature extractors for clean stethoscope recordings, and domain-specific pretraining on a narrow respiratory corpus is not automatically superior. The paper is strong in its use of public models and checkpoints, its adherence to established evaluation protocols for ICBHI2017, SPRS, and CirCor, and its release of code. The comparison across four tasks with consistent methodology is a valuable contribution. However, the BMD-HS comparison is not controlled for subject identity or task definition, and the abstract's 'achieved SOTA performance' wording overstates the SPRS results. With the BMD-HS issue fixed and the claims scaled back, the paper would still be a useful benchmark for the community.
major comments (3)
- [II-F, Table IV] The BMD-HS comparison is not controlled for subject identity or task definition. The 864 recordings come from only 108 subjects (8 recordings each), and the paper describes randomly stratified splits of 576/288 recordings without stating that subjects are kept disjoint. With recording-level splits, the same subject almost certainly appears in both training and test sets, enabling the classifier to exploit subject-specific acoustic patterns and inflating the reported metrics (e.g., BEATs accuracy 0.952, F1 0.875). In addition, the baseline from [21] is not shown to use the same binary murmur-presence task, the same subjects, or the same evaluation metric; the paper simply asserts comparability. To support the claim that frozen models outperform or achieve SOTA on BMD-HS, the authors should report results with subject-disjoint splits (e.g., group split by subject ID) and verify that the baseline used an identical task and partitioning scheme. As written, Table IV does not substantiate the 'SOTA performance' claim for clean data.
- [Abstract and Table II] The abstract and conclusion state that the models 'achieved SOTA performance on the other tasks with clean data.' This is not supported for SPRS: in Table II, the best frozen model (BEATs) reaches a score of 88.49±0.82, while the SOTA fine-tuned M2D-X scores 89.77±0.16, and M2D (16×4) scores 88.07±0.53. These results are close but below the SOTA value; the text itself says 'comparable.' For BMD-HS, the only comparison is against the dataset paper's baseline, which is not established as a SOTA system. The claim should be rephrased to say that frozen general-purpose features are competitive with, or in some cases exceed, published baselines, but they do not generally surpass current fine-tuned SOTA.
- [III Discussion, II-D, II-F] The paper's central insight—that frozen features work well on 'clean' data but require fine-tuning on 'noisy' data—relies on a qualitative, unmeasured clean/noisy distinction. The authors characterize SPRS and BMD-HS as 'clean' and ICBHI2017 and CirCor as 'noisy' based on dataset descriptions and anecdotal comments, but no quantitative noise metric is provided. This distinction is load-bearing for the explanatory narrative. A quantitative indicator of noise (e.g., estimated SNR, proportion of non-target acoustic events, or recording-device diversity) would strengthen the claim and allow a falsifiable test of the correlation. As it stands, the clean/noisy attribution is a post-hoc interpretation.
minor comments (3)
- [I and II-B] The phrase 'off-the-shelf' may mislead readers, because a task-specific network (MLP or transformer encoder) is trained on the frozen features. Suggest clarifying that 'off-the-shelf' refers to the fixed feature extractor only, and that a trainable downstream classifier is used in all experiments.
- [II-F] Please clarify how the binary murmur-presence label is derived from the multi-disease labels in BMD-HS, and report the class distribution in the training and test splits. This information is necessary for assessing the comparability with the baseline.
- [Table I] The large standard deviation for M2D (16×4) specificity (7.25) indicates instability across the five attempts; consider reporting per-seed results or discussing this variance in the text.
Circularity Check
No circularity: this is an empirical benchmark using public checkpoints and external baselines, with no prediction derived from its own inputs.
full rationale
The paper is a benchmark study: it freezes publicly released audio encoders (AST, BYOL-A, M2D, BEATs, OPERA-CT), trains only a small task head on official training splits, and compares results to previously published baselines and SOTA numbers on ICBHI2017, SPRS, CirCor, and BMD-HS. No model parameter is fit to test labels, and no theoretical quantity is derived from the quantity it is said to predict. The authors' self-citations ([6], [10], [11]) supply the M2D/BYOL-A checkpoints and prior fine-tuned scores; those checkpoints are publicly available and the scores are externally reproducible, so the self-citation is independent evidence rather than a load-bearing circular premise. The BMD-HS comparison is a possible experimental-control concern (task conversion and split comparability with [21]), but that is a validity issue, not circularity: the frozen-model result is measured, not constructed from the baseline. Therefore no circular step is present.
Assumptions & free parameters
assumptions (3)
- domain assumption The official evaluation protocols of the four public datasets, including data splits and metrics, are applied correctly, and the external SOTA and baseline numbers are produced under the same task definitions and splits.
- domain assumption The categorization of SPRS and BMD-HS as clean and ICBHI2017 and CirCor as noisy is valid and explains the observed performance gap.
- domain assumption OPERA-CT is used correctly as a frozen feature extractor, so its underperformance is not an artifact of misconfiguration.
Cite this review
Pith. "Pith review of Assessing the Utility of Audio Foundation Models for Heart and Respiratory Sound Analysis." pith.science (2026). https://pith.science/paper/IVJZUQXK
@misc{pith2026250418004,
author = {Pith},
title = {Pith review of: Assessing the Utility of Audio Foundation Models for Heart and Respiratory Sound Analysis},
year = {2026},
howpublished = {\url{https://pith.science/paper/IVJZUQXK}},
note = {Machine review of arXiv:2504.18004}
}
read the original abstract
Pre-trained deep learning models, known as foundation models, have become essential building blocks in machine learning domains such as natural language processing and image domains. This trend has extended to respiratory and heart sound models, which have demonstrated effectiveness as off-the-shelf feature extractors. However, their evaluation benchmarking has been limited, resulting in incompatibility with state-of-the-art (SOTA) performance, thus hindering proof of their effectiveness. This study investigates the practical effectiveness of off-the-shelf audio foundation models by comparing their performance across four respiratory and heart sound tasks with SOTA fine-tuning results. Experiments show that models struggled on two tasks with noisy data but achieved SOTA performance on the other tasks with clean data. Moreover, general-purpose audio models outperformed a respiratory sound model, highlighting their broader applicability. With gained insights and the released code, we contribute to future research on developing and leveraging foundation models for respiratory and heart sounds.
Figures
Reference graph
Works this paper leans on
-
[21]
S. N. Ali, A. Zahin, S. B. Shuvo, N. B. Nizam, S. I. S. K. Nuhash, S. S. Razin, S. M. S. Sani, F. Rahman, N. B. Nizam, F. B. Azam, R. Hossen, S. Ohab, N. Noor, and T. Hasan, “BUET Multi- disease Heart Sound Dataset: A comprehensive auscultation dataset for developing computer-aided diagnostic systems,” arXiv preprint arXiv:2409.00724, 2024
arXiv 2024
-
[1]
On the opportunities and risks of foundation models,
R. Bommasani et al. , “On the opportunities and risks of foundation models,” arXiv preprint arXiv:2108.07258 , 2021
arXiv 2021
-
[2]
Towards open respiratory acoustic foundation models: Pretraining and benchmarking,
Y . Zhang, T. Xia, J. Han, Y . Wu, G. Rizos, Y . Liu, M. Mosuily, J. Chauhan, and C. Mascolo, “Towards open respiratory acoustic foundation models: Pretraining and benchmarking,” in NeurIPS, 2024
2024
-
[3]
HeAR – health acoustic representations,
S. Baur, Z. Nabulsi, W.-H. Weng, J. Garrison, L. Blankemeier, S. Fish- man, C. Chen, S. Kakarmath, M. Maimbolwa, N. Sanjase, B. Shuma, Y . Matias, G. S. Corrado, S. Patel, S. Shetty, S. Prabhakara, M. Muy- oyeta, and D. Ardila, “HeAR – health acoustic representations,” arXiv preprint arXiv:2403.02522, 2024
arXiv 2024
-
[4]
Foundation models for cardiovascular disease detection via biosignals from digital stethoscopes,
G. Mathew, D. Barbosa, J. Prince, and S. a. Venkatraman, “Foundation models for cardiovascular disease detection via biosignals from digital stethoscopes,” npj Cardiovascular Health , 2024
work page 2024
-
[5]
An open access database for the evaluation of respiratory sound classification algorithms,
B. M. Rocha, D. Filos, L. Mendes, G. Serbes, S. Ulukaya, Y . P. Kahya, N. Jakovljevic, T. L. Turukalo, I. M. V ogiatzis, E. Perantoni, E. Kaimakamis, P. Natsiavas, A. Oliveira, C. J ´acome, A. Marques, N. Maglaveras, R. P. Paiva, I. Chouvarda, and P. de Carvalho, “An open access database for the evaluation of respiratory sound classification algorithms,” ...
work page 2019
-
[6]
Exploring pre-trained general-purpose audio representations for heart murmur detection,
D. Niizumi, D. Takeuchi, Y . Ohishi, N. Harada, and K. Kashino, “Exploring pre-trained general-purpose audio representations for heart murmur detection,” in EMBC, 2024, pp. 1–4
work page 2024
-
[7]
A survey on multimodal large language models,
S. Yin, C. Fu, S. Zhao, K. Li, X. Sun, T. Xu, and E. Chen, “A survey on multimodal large language models,” National Science Review, vol. 11, no. 12, Nov. 2024
work page 2024
Show all 25 references
-
[8]
RespLLM: Unifying audio and text with multimodal llms for generalized respiratory health prediction,
Y . Zhang, T. Xia, A. Saeed, and C. Mascolo, “RespLLM: Unifying audio and text with multimodal llms for generalized respiratory health prediction,” arXiv preprint arXiv:2410.05361 , 2024
2024 arXiv
-
[9]
AST: Audio Spectrogram Transformer,
Y . Gong, Y .-A. Chung, and J. Glass, “AST: Audio Spectrogram Transformer,” in Interspeech, 2021, pp. 571–575
2021
-
[10]
BYOL for Audio: Exploring Pre-trained General-purpose Audio Rep- resentations,
D. Niizumi, D. Takeuchi, Y . Ohishi, N. Harada, and K. Kashino, “BYOL for Audio: Exploring Pre-trained General-purpose Audio Rep- resentations,” IEEE/ACM Trans. Audio, Speech, Language Process. , vol. 31, p. 137–151, 2023
2023
-
[11]
Masked Modeling Duo: Towards a Universal Audio Pre- Training Framework,
——, “Masked Modeling Duo: Towards a Universal Audio Pre- Training Framework,” IEEE/ACM Trans. Audio, Speech, Language Process., vol. 32, pp. 2391–2406, 2024
2024
-
[12]
BEATs: Audio Pre-Training with Acoustic Tokenizers,
S. Chen, Y . Wu, C. Wang, S. Liu, D. Tompkins, Z. Chen, and F. Wei, “BEATs: Audio Pre-Training with Acoustic Tokenizers,” in ICML, 2023
2023
-
[13]
Audio Set: An ontology and human-labeled dataset for audio events,
J. F. Gemmeke, D. P. W. Ellis, D. Freedman, A. Jansen, W. Lawrence, R. C. Moore, M. Plakal, and M. Ritter, “Audio Set: An ontology and human-labeled dataset for audio events,” in ICASSP, 2017, pp. 776– 780
2017
-
[14]
Pretraining Respiratory Sound Rep- resentations using Metadata and Contrastive Learning,
I. Moummad and N. Farrugia, “Pretraining Respiratory Sound Rep- resentations using Metadata and Contrastive Learning,” in WASPAA, 2023, pp. 1–5
2023
-
[15]
Patch-Mix Contrastive Learning with Audio Spectrogram Transformer on Respiratory Sound Classification,
S. Bae, J.-W. Kim, W.-Y . Cho, H. Baek, S. Son, B. Lee, C. Ha, K. Tae, S. Kim, and S.-Y . Yun, “Patch-Mix Contrastive Learning with Audio Spectrogram Transformer on Respiratory Sound Classification,” in Interspeech, 2023, pp. 5436–5440
2023
-
[16]
Bts: Bridging text and sound modalities for metadata-aided respiratory sound classification,
J.-W. Kim, M. Toikkanen, Y . Choi, S.-E. Moon, and H.-Y . Jung, “Bts: Bridging text and sound modalities for metadata-aided respiratory sound classification,” in Interspeech 2024, 2024, pp. 1690–1694
2024
-
[17]
Heart murmur detection from phonocardiogram recordings: The George B. Moody PhysioNet Challenge 2022,
M. A. Reyna, Y . Kiarashi, A. Elola, J. Oliveira, F. Renna, A. Gu, E. A. P. Alday, N. Sadr, A. Sharma, J. Kpodonu, S. Mattos, M. T. Coimbra, R. Sameni, A. B. Rad, and G. D. Clifford, “Heart murmur detection from phonocardiogram recordings: The George B. Moody PhysioNet Challen...
2022
-
[18]
PhysioBank, PhysioToolkit, and PhysioNet: Components of a New Research Resource for Complex Physiologic Signals,
A. L. Goldberger, L. A. N. Amaral, L. Glass, J. M. Hausdorff, P. C. Ivanov, R. G. Mark, J. E. Mietus, G. B. Moody, C.-K. Peng, and H. E. Stanley, “PhysioBank, PhysioToolkit, and PhysioNet: Components of a New Research Resource for Complex Physiologic Signals,” Circulation, vol...
2000
-
[19]
Exploring wav2vec 2.0 model for heart murmur detection,
D. S. Panah, A. Hines, and S. McKeever, “Exploring wav2vec 2.0 model for heart murmur detection,” in EUSIPCO, 2023, pp. 1010– 1014
2023
-
[20]
Detection of Heart Murmurs in Phonocardiograms with Parallel Hidden Semi-Markov Models,
A. McDonald, M. J. Gales, and A. Agarwal, “Detection of Heart Murmurs in Phonocardiograms with Parallel Hidden Semi-Markov Models,” in CinC, vol. 498, 2022, pp. 1–4
2022
-
[22]
Detection of atrial fibril- lation from ppg sensor data using variational mode decomposition,
D. Pachori, R. K. Tripathy, and T. K. Jain, “Detection of atrial fibril- lation from ppg sensor data using variational mode decomposition,” IEEE Sensors Letters , vol. 8, no. 3, pp. 1–4, 2024
2024
-
[23]
Automated human emotion recognition system using tqwt-based eeg subbands,
D. Pachori and T. K. Gandhi, “Automated human emotion recognition system using tqwt-based eeg subbands,” IEEE Sensors Letters, vol. 8, no. 12, pp. 1–4, 2024
2024
-
[24]
SoundBeam: Target sound extraction conditioned on sound-class labels and enrollment clues for increased performance and continuous learning,
M. Delcroix, J. B. V ´azquez, T. Ochiai, K. Kinoshita, Y . Ohishi, and S. Araki, “SoundBeam: Target sound extraction conditioned on sound-class labels and enrollment clues for increased performance and continuous learning,” IEEE/ACM Trans. Audio, Speech, Language Process., vol...
2023
-
[25]
SoundBeam meets M2D: Target sound extraction with audio foundation model,
C. Hernandez-Olivan, M. Delcroix, T. Ochiai, D. Niizumi, N. Tawara, T. Nakatani, and S. Araki, “SoundBeam meets M2D: Target sound extraction with audio foundation model,” in to appear at ICASSP , 2025
2025
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.