{"id":"236dd494-cc8c-4dc2-8926-d8a931663b8e","arxiv_id":"2505.06271","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Adding stethoscope type as a third classification task in a multitask model slightly improves respiratory disease diagnosis and lung sound classification on the ICBHI dataset.","lead":"The authors test a multitask deep learning model that classifies lung sounds, diagnoses respiratory disease, and predicts patient metadata (such as stethoscope type) all at once. On a public respiratory sound dataset, adding stethoscope type as an auxiliary task slightly improves both disease diagnosis and lung sound classification over simpler baselines.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported gains depend on an ambiguous run-selection rule and on per-run variance that overlaps the baseline differences; without clarification or code the headline improvement is not established.","rationale":"The paper's central claim is that adding stethoscope metadata as a third task in soft parameter sharing improves both lung sound classification and disease diagnosis. For that claim to hold, the reported means must be honest estimates of typical performance and the differences must be larger than run-to-run noise. The weakest point is exactly the run-selection sentence in Section III.A.3: it can be read as selecting the five runs with the highest Sc, which would make every reported mean a selected maximum rather than a representative average. This is not a matter of disagreement with an external consensus; it is an internal threat to the validity of the comparison. The numerical inconsistencies between the text and Table II, and between the abstract and Table II, reinforce that the experimental record is not transparent enough to rule out selection or reporting bias. I agree with the reader's assessment that this is the load-bearing assumption. The reader's CONDITIONAL verdict is appropriate: the flaws are fixable by releasing per-run results, clarifying the selection protocol, and adding paired significance tests, but until then the headline improvement is not statistically supported. I do not see a separate, stronger objection. The public ICBHI dataset and use of a standard pretrained AST are positive features, and the proposed architecture is clearly described; the issue is specifically with the evidence for the claimed improvement, not with the method's plausibility.","tokens_in":8422,"tokens_out":2941,"duration_ms":30615,"concrete_test":"Request or re-run the per-seed results for each configuration (or obtain the authors' training code and reproduce Table II with ten fixed seeds). Compute paired differences between Soft(Stethoscope) and the best Two-MTL/Soft baseline for both lung sound classification and disease diagnosis, then apply a paired permutation test or report bias-corrected 95% confidence intervals on the differences. If the original protocol selected the five highest-Sc runs from a larger set, repeat the analysis using all runs. The abstract's significance claim is supported only if the paired differences remain significant under the corrected, fully reported protocol.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III.A.3 states that 'every reported value of Sp, Se, and Sc reflects the mean and variance across five independent runs where Sc is highest.' If this means the five retained runs were selected for highest Sc, the reported means are order statistics rather than unbiased estimates, and the comparison against Two-MTL is invalid. Even under a benign reading, the key contrasts in Table II are within one standard deviation: lung Soft(Stethoscope) is 60.21±1.42 versus Two-MTL Soft 59.19±0.93, and disease diagnosis is 90.16±3.19 versus 88.37±2.33. With n=5 per group and no paired significance test, the abstract's claim of 'significant improvements' is unsupported. The concern is compounded by internal inconsistencies: Section III.B.2 reports disease Sc=90.77±1.36 for Soft(Stethoscope), while Table II lists 90.16±3.19, and the abstract's clinical-relevance specificity of 85.83% does not match Table II's 86.23%. These discrepancies indicate that the reported statistics are not a complete, reproducible record, making the central claim depend on data the paper does not supply.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Tri-MTL, a multitask learning framework for respiratory disease diagnosis that jointly optimizes lung sound classification, disease diagnosis, and a metadata classification task on the ICBHI dataset. The authors compare hard and soft parameter-sharing variants against single-task and two-task baselines using a pretrained Audio Spectrogram Transformer. Their headline result is that adding stethoscope metadata under soft parameter sharing improves both lung sound classification (Sc = 60.21 ± 1.42) and disease diagnosis (Sc = 90.16 ± 3.19), relative to two-task soft sharing (Sc = 59.19 ± 0.93 and 88.37 ± 2.33, respectively). The paper also provides t-SNE visualizations to explain why MTL benefits disease diagnosis more than lung sound classification. The central claim is that stethoscope metadata, integrated as a third task, yields significant improvements in both objectives.","tokens_in":8625,"tokens_out":3210,"duration_ms":32780,"significance":"If the reported results are valid, Tri-MTL is a simple and practical extension of existing MTL ideas, and the finding that stethoscope metadata improves both tasks would be clinically relevant and would strengthen the prior evidence base for metadata-aided respiratory sound classification. The paper builds on a public benchmark (ICBHI), uses a standard pretrained backbone, and reports comparisons across several metadata attributes, which are useful for the community. However, the significance is currently tempered by ambiguous run-selection reporting, internal numerical inconsistencies, and the absence of significance testing: the claimed improvements are not established by the evidence as presented. The authors also do not release code, which limits reproducibility of the exact numbers.","major_comments":[{"comment":"The sentence 'every reported value of Sp, Se, and Sc reflects the mean and variance across five independent runs where Sc is highest' is ambiguous and potentially selection-biased. If this means that the five retained runs were selected for having the highest Sc, then the reported means are order statistics rather than unbiased estimates, and all comparisons against Two-MTL and single-task baselines are invalid under the current reporting. If it instead means that five seeds were run and the reported Sc corresponds to the run with highest Sc, this is even more problematic. Please state exactly how the five runs were chosen, report results for all runs or fixed seeds without post hoc selection, and provide per-run breakdowns or code so the reader can verify the statistics.","section":"Section III.A.3"},{"comment":"The text in Section III.B.2 reports disease diagnosis Sc = 90.77 ± 1.36 for Soft (Stethoscope), while Table II lists 90.16 ± 3.19 for the same configuration. This is a direct internal inconsistency. In addition, the abstract's clinical-relevance specificity of 85.83% for disease diagnosis does not match Table II's 86.23% for Soft (Stethoscope). These discrepancies mean that the reported statistics are not a complete and reproducible record, so the reader cannot determine which numbers support the headline claim.","section":"Section III.B.2 and Table II"},{"comment":"The central claim of 'significant improvements' is not supported by the reported variability. For lung sound classification, Soft (Stethoscope) gives 60.21 ± 1.42 versus Two-MTL Soft 59.19 ± 0.93; for disease diagnosis, 90.16 ± 3.19 versus 88.37 ± 2.33. With five runs per condition and no paired significance test, these differences are within plausible sampling noise. Please report paired tests (e.g., paired t-test or bootstrap) across the same seeds, confidence intervals, or effect sizes for the key comparisons in Table II. Without this, the word 'significant' in the abstract is unsupported.","section":"Table II and Abstract"}],"minor_comments":[{"comment":"Equation (3) contains a typographical error: 'LHard = = Σ' should be 'LHard = Σ'.","section":"Equation (3)"},{"comment":"The cross-entropy loss in Equation (4) is written without division by N; the text mentions this omission, but it would be clearer to write the normalized form or explicitly define the notation as an unnormalized sum over the batch.","section":"Equation (4)"},{"comment":"Figure 1(c) includes '× α', but α is not defined anywhere in the text or equations; please either define it or remove it from the figure.","section":"Figure 1 caption"},{"comment":"The text says 'mean and variance' but the tables report mean ± standard deviation; please align the wording with the reported quantities.","section":"Section III.A.3"},{"comment":"The conclusion contains a typo: 'ICHBI respiratory sound dataset' should be 'ICBHI respiratory sound dataset'.","section":"Section IV"}],"recommendation":"major_revision","confidential_remarks":"This is a potentially useful empirical study, but the statistical reporting and internal inconsistencies are load-bearing issues that must be resolved before the headline claim can be accepted. The paper relies heavily on the authors' prior work to motivate the metadata task, which is acceptable but should be positioned carefully. I would recommend major revision rather than rejection because the central idea is defensible and the issues appear fixable with clearer reporting and additional analysis."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper reports a plausible but under-supported result: adding a stethoscope-metadata classification head to a two-task MTL improves both lung sound classification and disease diagnosis on ICBHI. The genuinely new bit is the third task, which builds on the authors' earlier metadata-aided work. That is a small, legitimate extension, and the architecture is simple enough to reproduce in principle. The experiments use a public benchmark, compare hard and soft parameter sharing, and evaluate four metadata types. The t-SNE analysis is a nice, honest attempt to explain why MTL helps diagnosis more than lung sound classification. I believe the direction of the effect: stethoscope type is a known confounder in respiratory sound data, and MTL is a reasonable way to exploit it.\n\nThe soft spots are real, and one is load-bearing. Section III.A.3 says every reported value reflects the mean and variance across five independent runs where Sc is highest. If that means the retained runs were selected for highest Sc, the reported means are order statistics, not unbiased estimates, and the comparison against Two-MTL is invalid. Even under a benign reading, the wording is confusing, and the key contrasts in Table II sit within one standard deviation: lung Soft(Stethoscope) is 60.21±1.42 versus Two-MTL Soft 59.19±0.93; disease diagnosis is 90.16±3.19 versus 88.37±2.33. With n=5 per group and no paired significance test, the abstract's claim of significant improvements is unsupported. There are also internal number inconsistencies: the text reports disease Sc=90.77±1.36 for Soft(Stethoscope) while Table II lists 90.16±3.19, and the abstract's disease specificity of 85.83% does not match Table II's 86.23%. These do not prove fabrication, but they mean the reported statistics are not a clean, reproducible record. No code is provided.\n\nFor all that, the paper is not a waste of time. The idea is sensible, the dataset is public, and the flaws are fixable. But the central empirical claim is not yet established. I would send this to a serious referee, with instructions to require clarification of the run-selection rule, add significance tests, reconcile the numbers, and release code. For someone outside respiratory-audio MTL, it is a minor entry; for someone working in that niche, it is worth a look. I would not cite it as a reliable result until the statistics are straightened out.","headline":"A plausible but under-supported MTL extension; the headline gains hinge on an ambiguous run-selection rule and unresolved number discrepancies.","tokens_in":9200,"tokens_out":2946,"would_cite":false,"duration_ms":27294,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Stethoscope metadata lifts both lung-sound and disease diagnosis scores in a multitask model.","keywords":["multitask learning","respiratory sound classification","disease diagnosis","stethoscope metadata","Audio Spectrogram Transformer","ICBHI dataset","soft parameter sharing","metadata-aided diagnosis"],"falsifier":"Rerun Table II with the same five seeds and report every run, then compare Soft (Stethoscope) against Soft Two-MTL and Single Task using the full distribution: the lung-sound scores overlap within one standard deviation (60.21±1.42 vs 59.55±0.88) and the disease scores also overlap (90.16±3.19 vs 88.37±2.33), so a paired statistical test on all five runs would settle whether the claimed improvement is real.","tokens_in":8200,"feed_emoji":"🩺","tokens_out":4431,"duration_ms":40833,"temperature":0.7,"pith_summary":"The paper tries to show that multitask learning can mirror a clinician's workflow: instead of classifying lung sounds and diseases separately, one model should do both and also predict patient metadata, with stethoscope type as a third task. On the ICBHI respiratory sound benchmark, the proposed Tri-MTL with soft parameter sharing and stethoscope metadata reports the best results: lung sound score 60.21 and disease diagnosis score 90.16, exceeding single-task and two-task baselines. The reason this matters is that it suggests recording-device information carries signal about how lung sounds are captured, and that a shared representation can use that signal to improve diagnosis. The authors frame the contribution as an extension of earlier findings that metadata aids respiratory sound classification, now tested inside a multitask framework.","feed_headline":"Stethoscope type lifts multitask lung-disease scores","feed_subtitle":"Joint training on lung sounds, disease labels, and device metadata beats single-task baselines on ICBHI.","key_machinery":"The machinery is a family of three multitask architectures built on a pretrained Audio Spectrogram Transformer audio encoder. Hard parameter sharing uses one shared encoder with separate classification heads for lung sounds, disease, and metadata, summing the three cross-entropy losses. Soft parameter sharing instead gives each task its own encoder and adds an L2 regularization loss that penalizes differences between corresponding layer weights, balancing task-specific and shared features. Tri-MTL is the version that adds the metadata classifier to either sharing scheme; the pair that carries the reported result is soft sharing with stethoscope-type metadata, where the metadata loss is included alongside the lung and disease losses.","core_discovery":"The central claim is that jointly training lung sound classification, disease diagnosis, and metadata classification improves performance over training either task alone or the two tasks together. The paper finds that the two-task MTL already raises disease diagnosis from 81.51 to 88.37, while lung sound classification barely moves; adding metadata as a third task, specifically the recording stethoscope type under soft parameter sharing, lifts lung sound score to 60.21 and disease diagnosis to 90.16. The authors interpret this as evidence that stethoscope metadata helps the model separate acoustic differences introduced by different recording devices, and that the lung-sound task supplies useful features for diagnosis even though the reverse direction gives less. The result is specific to the ICBHI official 60/40 split and the Audio Spectrogram Transformer backbone.","pith_inferences":["An extension the paper leaves implicit: a model that receives stethoscope type as a conditioning input rather than predicting it as a task might capture the same device-effect information with fewer parameters.","If device-specific acoustic effects are the true driver, the gains should shrink on datasets recorded with a single stethoscope type and grow where multiple devices are mixed; that is a checkable prediction the paper does not run.","The location-metadata results suggest some metadata tasks can hurt diagnosis (Hard Location falls below single-task performance), so a selective metadata-gating mechanism is a natural next step the authors do not explore."],"forward_implications":["If the result is correct, stethoscope type should be treated as a useful input or auxiliary task in respiratory sound models, not as a nuisance variable to be averaged away.","Multitask learning offers a practical route to disease diagnosis from lung sounds: the two-task model already beats the single-task disease baseline by several score points.","Soft parameter sharing, which lets each task keep separate weights while encouraging similarity, appears better suited than hard sharing for this pair of tasks.","The reported numbers become the new comparison point for the ICBHI benchmark under AST-based encoders with the official 60/40 split."],"supporting_citations":[{"why":"Supplies the ICBHI dataset, its official 60/40 split, and the Sp/Se/Sc scoring definitions used in all comparisons.","marker":"[25]"},{"why":"The pretrained Audio Spectrogram Transformer that every encoder in the paper is initialized from and fine-tuned.","marker":"[23]"},{"why":"Prior evidence that stethoscope-guided adaptation improves respiratory sound classification, which Tri-MTL extends.","marker":"[20]"},{"why":"Prior work showing metadata-aided respiratory sound classification, cited as the direct inspiration for the metadata task.","marker":"[21]"},{"why":"Establishes the preprocessing and AST-based training recipe the paper follows.","marker":"[24]"},{"why":"The L2 pairwise parameter-similarity loss used to define soft parameter sharing.","marker":"[29]"},{"why":"The multitask learning formulation underlying the hard parameter sharing baseline.","marker":"[28]"}],"fun_headline_variants":["Stethoscope metadata boosts multitask lung-disease accuracy","Triple MTL with stethoscope type lifts lung sound and diagnosis scores","Stethoscope metadata as third task improves lung disease MTL","Device metadata in multitask learning sharpens lung disease diagnosis","Triple training with stethoscope info bests two-task lung diagnosis"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported claim of improvement depends on the five-run statistics being honestly and completely reported; the paper selects runs where Sc is highest, so if that selection hides run-to-run variability, the gains over baselines could vanish.","fun_headline_variants_meta":{"raw":{"variants":["Stethoscope metadata boosts multitask lung-disease accuracy","Triple MTL with stethoscope type lifts lung sound and diagnosis scores","Stethoscope metadata as third task improves lung disease MTL","Device metadata in multitask learning sharpens lung disease diagnosis","Triple training with stethoscope info bests two-task lung diagnosis"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000575,"raw_usage":{"total_tokens":2670,"prompt_tokens":859,"completion_tokens":1811,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":475,"completion_tokens_details":{"reasoning_tokens":1722}},"tokens_in":475,"tokens_out":1811,"duration_ms":11169,"temperature":1.0,"reasoning_tokens":1722,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:53:26.937353+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun Table II with the same five seeds and report every run, then compare Soft (Stethoscope) against Soft Two-MTL and Single Task using the full distribution: the lung-sound scores overlap within one standard deviation (60.21±1.42 vs 59.55±0.88) and the disease scores also overlap (90.16±3.19 vs 88.37±2.33), so a paired statistical test on all five runs would settle whether the claimed improvement is real.","supporting_citations":[{"cited_title":"A respiratory sound database for the development of automated classification,","cited_arxiv_id":null,"evidence_quote":"Supplies the ICBHI dataset, its official 60/40 split, and the Sp/Se/Sc scoring definitions used in all comparisons."},{"cited_title":"Stethoscope- guided supervised contrastive learning for cross-domain adaptation on respiratory sound classification,","cited_arxiv_id":null,"evidence_quote":"Prior evidence that stethoscope-guided adaptation improves respiratory sound classification, which Tri-MTL extends."},{"cited_title":"Bts: Bridging text and sound modalities for metadata-aided respiratory sound classification,","cited_arxiv_id":null,"evidence_quote":"Prior work showing metadata-aided respiratory sound classification, cited as the direct inspiration for the metadata task."},{"cited_title":"Patch-Mix Contrastive Learning with Audio Spectrogram Transformer on Respiratory Sound Classification,","cited_arxiv_id":null,"evidence_quote":"Establishes the preprocessing and AST-based training recipe the paper follows."},{"cited_title":"Low resource dependency parsing: Cross-lingual parameter sharing in a neural network parser,","cited_arxiv_id":null,"evidence_quote":"The L2 pairwise parameter-similarity loss used to define soft parameter sharing."},{"cited_title":"Multitask learning: A knowledge-based source of induc- tive bias1,","cited_arxiv_id":null,"evidence_quote":"The multitask learning formulation underlying the hard parameter sharing baseline."}],"review_version":1}