{"id":"869fe2cb-8bf6-4227-b4d3-80133cc6bd0b","arxiv_id":"2505.15307","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"General audio pre-training (AudioSet) outperforms respiratory-specific pre-training for respiratory sound tasks, and further pre-training on combined AudioSet plus respiratory data yields the best results on the OPERA benchmark.","lead":"Researchers compared 21 pre-trained audio models on seven respiratory sound tasks and found that models pre-trained on general audio (AudioSet) beat models trained specifically on respiratory sounds. Further pre-training an AudioSet model on a mix of AudioSet and respiratory sounds set a new state of the art on the OPERA benchmark.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The M2D+Resp SOTA is selected on the same OPERA benchmark and pre-training includes T5-T7 training splits, so the combined-pre-training gain needs a held-out evaluation before it is established.","rationale":"The reader's weakest_assumption focuses on the pre-training overlap with T5-T7 training splits, which is a real and explicitly acknowledged issue. However, the more load-bearing problem is the combination of that overlap with benchmark-driven model selection: the final configuration was chosen as the best among many variants on the same benchmark, so the SOTA number is optimistically biased even for tasks without data overlap. The paper is transparent about the overlap, and the gains on T8-T10 provide some evidence that combining AudioSet and respiratory data helps outside the overlapping tasks. Thus the core guidance (AudioSet pre-training beats respiratory-only; preserving frequency-wise features matters) is likely robust, but the precise SOTA margin and the size of the combined-pre-training benefit remain uncertain. The proposed concrete test would settle whether the benefit persists on disjoint tasks and whether the selected configuration is an artifact. The reader's CONDITIONAL verdict is appropriate, so no change is needed; the stress-test adds a sharper emphasis on selection bias.","tokens_in":10725,"tokens_out":6009,"duration_ms":56372,"concrete_test":"Report the OPERA average and the M2D vs M2D+Resp delta using only tasks disjoint from the pre-training data (T8, T9, T10, T11). If the average gain over M2D on these tasks is within noise or negative, the combined-pre-training benefit is not supported for unseen respiratory recordings. Then, freeze all hyperparameters (base model, sample count, input resolution, layer index) before seeing the benchmark and evaluate on a held-out split or an external respiratory dataset; if the frozen configuration no longer matches the reported SOTA, the improvement is a selection artifact.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central SOTA claim rests on M2D+Resp, obtained by further pre-training M2D on AudioSet plus a respiratory set that, per Table 4, 'contains the data from tasks with ∈Resp' — specifically the training splits of COUGHVID (T5, T6) and ICBHI (T7). Linearly probing features from a model that has already been pre-trained on the very training recordings of three of the seven benchmark tasks is not an unbiased estimate of generalization to unseen respiratory audio. In addition, M2D+Resp was selected as the best among seven further-pre-training configurations (Resp only, AS+Resp 100K through 500K) and multiple resolution/layer choices, all evaluated on the same OPERA benchmark. No holdout set was used for model selection, so the reported 0.814 is the maximum over a garden of forking paths and is optimistically biased. Gains on disjoint tasks T8-T10 do exist, which partially mitigates the overlap concern, but the headline average still aggregates the overlapping tasks, and the selection bias applies to all tasks. Without a pre-registered configuration or a held-out evaluation, the magnitude of the combined AudioSet+Resp benefit is not established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper evaluates 21 pre-trained audio foundation models on seven tasks (T5–T11) of the OPERA respiratory audio benchmark under a linear-probing protocol. It reports that models pre-trained on AudioSet outperform the respiratory-specific OPERA-CT and OPERA-GT models, that further pre-training an AudioSet-pretrained M2D on a combination of AudioSet and respiratory sound data ('M2D+Resp') yields a new state of the art (average AUROC 0.814 vs 0.733), and that preserving frequency-wise information in feature aggregation is important. Ablation experiments vary training objective, patch resolution, data composition, and layer choice, and the paper releases evaluation code.","tokens_in":10909,"tokens_out":5500,"duration_ms":49886,"significance":"If the central result holds, the paper provides actionable guidance for pre-training respiratory audio models and a reusable benchmark harness for the community. The study is a well-structured empirical comparison: it reports means and standard deviations over five runs, compares a broad set of external models (ATST-Frame, MSM-MAE, CLAP, etc.), and makes code publicly available. The claim that general AudioSet pre-training is more effective than the two evaluated respiratory-specific models is supported by multiple models not developed by the authors. However, the headline SOTA claim for M2D+Resp is currently weakened by two issues: the further pre-training data overlaps with the training splits of tasks T5–T7, and the 400K configuration was selected by benchmarking on the same test tasks. These issues need to be addressed before the combined-pretraining benefit can be considered established.","major_comments":[{"comment":"The headline result M2D+Resp (average AUROC 0.814) comes from further pre-training on a respiratory set that, as the table's own footnote states, 'contains the data from tasks with ∈Resp' — i.e., the training splits of T5–T7. Consequently, the gains on T5 and T6 (0.627 vs 0.595 and 0.856 vs 0.797) may reflect exposure to the exact benchmark training recordings rather than improved generalization to unseen respiratory audio. Gains on the disjoint tasks T8–T10 do exist, which partially mitigates the concern, but the headline average aggregates the overlapping tasks. Please add a held-out evaluation on disjoint tasks or on disjoint splits of the respiratory datasets, or substantially temper the claim that combining AudioSet with respiratory data 'enhances performance.'","section":"Table 4, §3.2"},{"comment":"The 400K sample count for M2D+Resp was selected by running configurations of 100K, 200K, 300K, 400K, and 500K and choosing the one with the highest average AUROC on the same OPERA benchmark. No validation split or pre-registered selection criterion is used, so the reported 0.814 is the maximum of a configuration sweep and is likely optimistically biased. The same issue appears in Table 2 for the speech SSL models (rows 4–6), where the 'Layer#7' notation indicates a layer chosen by benchmark performance, and in Figure 1, where the penultimate layer is identified as best. Please report the selection procedure explicitly, use a validation split for configuration and layer selection, or present the chosen configuration as exploratory and confirm it on a held-out benchmark.","section":"Table 4, §3.2"},{"comment":"The average AUROC across the seven tasks is strongly influenced by tasks with very small sample sizes and ceiling effects. T7 has only 35 samples in one class and reaches 1.000 for several models (AST, MSM-MAE, M2D, M2D+Resp), and T10 has 129/105 samples. With only five runs, the reported standard deviations are small, but the class imbalance and saturation make per-task differences less informative and the average less representative. Please report per-task confidence intervals and include a sensitivity analysis that excludes or reweights T7 (and possibly T10) to show that the main conclusions are not driven by these tasks.","section":"Table 1, Table 2"}],"minor_comments":[{"comment":"The heading 'Emperical Analysis' should be 'Empirical Analysis.'","section":"Section 3"},{"comment":"The asterisk is explained only in the running text after the table; please add a legend to the caption (e.g., '* indicates better than OPERA-CT').","section":"Table 2 caption"},{"comment":"The notation '∈Resp' is used without definition in the caption; please define it explicitly, for example, 'Resp denotes the respiratory pre-training set, which includes the training splits of tasks T5–T7.'","section":"Table 4 footnote"},{"comment":"The abstract states that AudioSet models 'are more effective than the models specifically pre-trained on respiratory sounds,' but only two such models (OPERA-CT and OPERA-GT) are evaluated; 'the evaluated respiratory-specific models' would be more precise.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"This is a competent empirical study with useful benchmark results and released code. The main barrier is not the comparison among the frozen models in Table 2, which is informative, but the SOTA claim for M2D+Resp, which is compromised by training/benchmark overlap and by selection on the test benchmark. If the authors can provide a disjoint held-out evaluation or clearly reposition the result as exploratory, the paper could be acceptable. The paper fits the journal's scope, and I do not see a fundamental correctness flaw in the central AudioSet-vs-respiratory comparison."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a genuinely useful empirical benchmark study, and the main guidance—that AudioSet pre-training beats respiratory-only pre-training for OPERA—is credible. But the paper's stronger claim, that the combined AudioSet+respiratory further-pre-training recipe sets a new SOTA, is contaminated: the respiratory pre-training set includes the training splits of three of the seven benchmark tasks, and the recipe (M2D, 400K samples) was selected on the same benchmark. I'd send it to review, but the SOTA claim needs a disjoint evaluation before it is taken as established.\n\nThe core comparison is worthwhile. It evaluates 21 models under a fixed linear-probe protocol, reports means and stds over five runs, and releases code. The central result is supported by external models: ATST-Frame, MSM-MAE, and CLAP2023 all beat OPERA-CT, so the \"data scale/diversity beats domain-specific pre-training\" conclusion isn't just the authors' models beating their own baseline. The frequency-wise aggregation ablation is clean: mean-pooling the patch features drops M2D's average from 0.782 to 0.712, with the largest effect on gender and severity tasks where frequency resolution plausibly matters. The layer-wise analysis adds a small but useful data point.\n\nThe soft spots are real. Table 4 explicitly notes that the respiratory pre-training data contains the training data for T5-T7; evaluating a model that has seen those recordings during pre-training is not a clean generalization test. The disjoint tasks T8-T10 do improve, which is genuine evidence for the recipe, but the 0.814 headline pools contaminated and uncontaminated tasks. The choice of 400K samples and of M2D as the base model was made by looking at the same benchmark, so the SOTA number carries selection bias on top of the overlap. There are also no significance tests between close models; some mid-tier differences are within the reported stds. The \"Resp only\" row partially mitigates the concern, but it does not replace a held-out split.\n\nThis paper is for anyone choosing a pre-trained audio encoder for respiratory sound work, and for benchmark designers. It deserves a serious referee. The referee should ask for a contamination-free evaluation (exclude T5-T7 training data from pre-training, or use a disjoint respiratory set) and a fixed model-selection rule. With that, the practical guidance would be credible; as it stands, treat the SOTA number as optimistic.","headline":"Useful benchmark study with a credible core finding, but the combined-pre-training SOTA is contaminated by overlapping training data and benchmark-driven selection, so it needs a disjoint evaluation before the headline number is trusted.","tokens_in":11483,"tokens_out":4892,"would_cite":true,"duration_ms":42099,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Pre-training on general audio beats respiratory-only pre-training for respiratory sound analysis, and adding respiratory data on top sets a new OPERA benchmark record.","keywords":["respiratory audio","foundation model","pre-training","AudioSet","self-supervised learning","OPERA benchmark","frequency-wise feature aggregation","cough classification"],"falsifier":"Run the same M2D+Resp recipe but evaluate on a respiratory corpus whose recordings were held out of pre-training entirely, or repeat the pre-training after removing COUGHVID, HF Lung, and ICBHI from the training mix and test on those tasks. If the average AUROC advantage of M2D+Resp over the AudioSet-only M2D collapses to roughly zero or reverses, the central claim that combined pre-training helps would be falsified. A second check is to compare mean-pooled versus frequency-wise concatenation on such held-out tasks to see whether the aggregation benefit persists outside OPERA.","tokens_in":10486,"feed_emoji":"🫁","tokens_out":7245,"duration_ms":59362,"temperature":0.7,"pith_summary":"This paper asks what pre-training practice actually yields an effective respiratory audio foundation model, and answers with a broad comparison on the OPERA benchmark. Across 21 audio models, the authors find that models pre-trained on AudioSet, a large general audio collection that includes coughs, breathing, and many other sounds, outperform models pre-trained exclusively on respiratory audio. The recommended recipe is two-stage: start from the self-supervised M2D model trained on AudioSet, then further pre-train it on a mix of AudioSet and respiratory sound data (COUGHVID, HF Lung, and ICBHI). That model, M2D+Resp, reaches an average AUROC of 0.814 on OPERA's seven tasks, beating the previous respiratory-specific state of the art (OPERA-CT, 0.733). The paper also reports that preserving frequency-wise structure when aggregating patch features is central to the gain; mean pooling the features drops average AUROC from 0.782 to 0.712 in ablations.","feed_headline":"General audio beats respiratory-only pre-training for lung AI","feed_subtitle":"Add cough and lung recordings to a general-sound model and it reaches a new best on seven clinical tasks.","key_machinery":"The workhorse is the M2D model (Masked Modeling Duo), a self-supervised audio model that learns by reconstructing masked spectrogram patches, with the M2D-X variant used for further pre-training on small datasets by adding AudioSet as background noise. The paper uses M2D as a controlled testbed to ablate pre-training data, training objective, patch resolution, and feature aggregation. The decisive mechanism is that M2D summarizes a spectrogram by concatenating frequency-wise patch features for each time frame, preserving spectral information; switching to mean pooling drops average AUROC from 0.782 to 0.712. The other load-bearing mechanism is two-stage pre-training: beginning from the AudioSet-trained M2D and refining it on a mix of AudioSet samples and respiratory recordings.","core_discovery":"The central discovery is that, for respiratory audio, the variety and scale of pre-training data outweighs domain specificity. Eight of the nineteen general-audio models evaluated beat OPERA-CT, the best respiratory-only model, and the top general-audio models beat it on every task. The strongest configuration is M2D+Resp: the AudioSet-pre-trained M2D model further pre-trained on a combined AudioSet and respiratory dataset of 400,000 samples, which reaches 0.814 average AUROC on OPERA versus 0.733 for OPERA-CT. Ablations on M2D isolate two deciding factors: the data, with further pre-training on AudioSet plus respiratory sounds helping while respiratory sounds alone do not, and the feature aggregation, with concatenating frequency-wise patch features being far better than mean pooling. Speech-based SSL models underperform because their training signals are built from speech clusters that carry few breath-like acoustic patterns.","pith_inferences":["The reported gain from combining AudioSet with respiratory data is measured on benchmark tasks whose training splits overlap the pre-training set (T5-T7); whether the gain survives on fully unseen respiratory recordings is an open question the paper does not answer.","The frequency-wise aggregation insight likely transfers to other bioacoustic tasks such as heart-sound or bowel-sound classification, where spectral details carry diagnostic information.","A natural extension is to apply the same AudioSet-plus-respiratory two-stage recipe to other strong base models, for example ATST-Frame or CED, to see whether the gain is specific to M2D or general."],"forward_implications":["Future respiratory foundation models should pre-train on large diverse audio before any respiratory-specific adaptation, since respiratory-only pre-training is a weak base.","Two-stage pre-training on AudioSet plus respiratory data is a concrete recipe that raises average AUROC to 0.814, a new OPERA record.","Feature aggregation choices matter as much as data: frequency-wise concatenation should be the default for respiratory tasks, not mean pooling.","The penultimate layer of the AudioSet model is the best feature source across tasks, so respiratory systems should use deep-layer outputs rather than early or middle layers."],"supporting_citations":[{"why":"Provides the OPERA benchmark tasks and the OPERA-CT/GT respiratory-only baselines that all comparisons must beat.","marker":"[4]"},{"why":"Supplies the M2D model and M2D-X framework used as the base for ablations and for further pre-training.","marker":"[25]"},{"why":"AudioSet is the large general-audio dataset whose pre-training advantage is the paper's central empirical claim.","marker":"[31]"},{"why":"COUGHVID supplies cough recordings used both as benchmark tasks T5/T6 and as part of the respiratory pre-training mix.","marker":"[6]"},{"why":"HF Lung provides the stethoscope lung-sound recordings used in further pre-training.","marker":"[32]"},{"why":"ICBHI supplies lung sounds used as benchmark task T7 and in the respiratory pre-training dataset.","marker":"[7]"},{"why":"MSM-MAE is cited as another model that preserves frequency-wise structure in feature aggregation, supporting the aggregation finding.","marker":"[24]"},{"why":"Earlier study reporting that preserving frequency-wise information matters for general audio; the paper extends this to respiratory audio.","marker":"[35]"}],"fun_headline_variants":["General audio beats respiratory-only for lung sound AI","Lung AI thrives on general sounds plus cough data","AudioSet pre-training wins for respiratory models","Combining general and lung audio sets new OPERA record","Why general audio beats respiratory-only pre-training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the overlap between the further-pre-training dataset, which contains COUGHVID, HF Lung, and ICBHI recordings including the training splits of benchmark tasks T5-T7, does not inflate the reported gains; if it does, the advantage of combining respiratory data with AudioSet may not hold for unseen respiratory recordings.","fun_headline_variants_meta":{"raw":{"variants":["General audio beats respiratory-only for lung sound AI","Lung AI thrives on general sounds plus cough data","AudioSet pre-training wins for respiratory models","Combining general and lung audio sets new OPERA record","Why general audio beats respiratory-only pre-training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000142,"raw_usage":{"total_tokens":1132,"prompt_tokens":873,"completion_tokens":259,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":489,"completion_tokens_details":{"reasoning_tokens":187}},"tokens_in":489,"tokens_out":259,"duration_ms":2833,"temperature":1.0,"reasoning_tokens":187,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:19:33.812845+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same M2D+Resp recipe but evaluate on a respiratory corpus whose recordings were held out of pre-training entirely, or repeat the pre-training after removing COUGHVID, HF Lung, and ICBHI from the training mix and test on those tasks. If the average AUROC advantage of M2D+Resp over the AudioSet-only M2D collapses to roughly zero or reverses, the central claim that combined pre-training helps would be falsified. A second check is to compare mean-pooled versus frequency-wise concatenation on such held-out tasks to see whether the aggregation benefit persists outside OPERA.","supporting_citations":[{"cited_title":"We also evaluated the layer-wise performance (Figures 1 and 2)","cited_arxiv_id":null,"evidence_quote":"Supplies the M2D model and M2D-X framework used as the base for ablations and for further pre-training."},{"cited_title":"Towards open respiratory acoustic foundation models: Pretraining and benchmarking,","cited_arxiv_id":null,"evidence_quote":"AudioSet is the large general-audio dataset whose pre-training advantage is the paper's central empirical claim."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"COUGHVID supplies cough recordings used both as benchmark tasks T5/T6 and as part of the respiratory pre-training mix."},{"cited_title":"A Generalist Audio Foundation Model for Comprehensive Body Sound Auscultation","cited_arxiv_id":"2411.07547","evidence_quote":"HF Lung provides the stethoscope lung-sound recordings used in further pre-training."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"ICBHI supplies lung sounds used as benchmark task T7 and in the respiratory pre-training dataset."},{"cited_title":"Bold results are better than the models above","cited_arxiv_id":null,"evidence_quote":"MSM-MAE is cited as another model that preserves frequency-wise structure in feature aggregation, supporting the aggregation finding."},{"cited_title":"Coswara: A respiratory sounds and symp- toms dataset for remote screening of sars-cov-2 infection,","cited_arxiv_id":null,"evidence_quote":"Earlier study reporting that preserving frequency-wise information matters for general audio; the paper extends this to respiratory audio."}],"review_version":1}