{"id":"e5a09e7c-4b01-4732-bdf9-331da7d1eb0a","arxiv_id":"2507.16845","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"Adding MixMatch, Co-Refinement, and Co-Refurbishing to an MFCC+CNN classifier is reported to improve lung sound classification accuracy from 89.1% to 92.9% on the ICHBI respiratory sound dataset.","lead":"This paper tests whether adding three semi-supervised learning modules to a standard MFCC+CNN audio classifier improves lung disease detection from breathing sounds. The authors report accuracy rising from 89.1% to 92.9% on a public respiratory sound dataset, but the evaluation method is not fully described.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section 4 says the model is trained again on the labeled plus validation data before accuracy is reported, and Section 5 says the authors trained and validated on the labeled dataset; with no test split defined, the 92.9% and +3.8% gains are not established as held-out generalization.","rationale":"I read the paper in good faith as an initial empirical proposal whose central claim is a specific accuracy improvement. For that claim to hold, the evaluation must use a held-out test set that is untouched during training and model selection. The paper's own text contradicts this requirement: Section 4 describes a final training pass on the full labeled dataset plus the validation dataset, and Section 5 says the authors trained and validated on the labeled dataset. No independent test set is mentioned anywhere. This is the most load-bearing weakness because it attacks the validity of the headline number itself, not just its magnitude or statistical significance. If the evaluation is compromised, the entire contribution, however plausible, is unsupported. My agreement with the reader is complete: the weakest assumption is evaluation integrity. The proposed concrete test is a single clean experimental protocol that would settle the concern: re-run with a fixed split and report held-out accuracy. If the gap disappears, the paper's central claim fails; if the gap persists, the result becomes credible enough to reconsider. Because the reader already rejected the paper on exactly this basis, my read does not change the verdict. The paper should not be accepted in its current form, though a careful re-evaluation on a true held-out test set could support a much smaller and better-documented claim.","tokens_in":5832,"tokens_out":4878,"duration_ms":54418,"concrete_test":"Re-run the experiment on ICHBI with a strict, pre-registered split: stratify by class and by participant into 70% train, 15% validation, and 15% test; use only train and unlabeled data for all training, select hyperparameters on validation, and evaluate the baseline and the full semi-supervised model on the test set exactly once. Report per-class accuracy, overall accuracy, and 95% confidence intervals. If the add-on test accuracy does not exceed baseline by at least 3.8 points, or if the gap is within noise, the published improvement is an artifact of validation-set training.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that adding MixMatch, Co-Refinement, and Co-Refurbishing to an MFCC+CNN baseline raises six-class ICHBI lung-sound accuracy from 89.1% to 92.9%. That claim is only meaningful if 92.9% is measured on data unused during training. The manuscript does not establish this. Section 4 states: \"Once the loop is completed, the model is trained again, this time using the full labeled dataset and the validation dataset.\" Section 5 states: \"During the evaluation of the complete model, we trained and validated on the labeled dataset.\" No test split, split sizes, or participant separation are reported. If the validation set participates in the final training pass, the reported accuracy measures memorization or transductive fit, not generalization, and the +3.8% improvement could be entirely an artifact. The absence of a held-out test set also makes the per-class precision/recall gains, such as pneumonia recall rising from 0.71 to 1.0, uninterpretable. I am not claiming the result is false; the known semi-supervised modules could plausibly help with limited labels. But the paper's own text prevents a reader from verifying that the headline number is an honest estimate of test accuracy.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes combining an MFCC+CNN baseline with three existing semi-supervised learning modules—MixMatch, Co-Refinement, and Co-Refurbishing—for six-class lung sound classification on the ICHBI dataset. The abstract and conclusion claim that this combination raises accuracy from 89.1% to 92.9% (a 3.8 percentage-point gain), with per-class improvements for COPD, pneumonia, and healthy classes. Section 4 describes the experimental setup and Section 5 reports the evaluation and ablations.","tokens_in":6048,"tokens_out":4006,"duration_ms":41925,"significance":"If the reported accuracy were a valid held-out estimate, the result would provide a useful empirical demonstration that standard SSL modules can benefit lung-sound classification with limited labels. The paper does not propose a new method, but a careful evaluation of three existing modules on a medical audio task could be of interest to the community. The paper also has some strengths: it uses a public dataset (ICHBI) and reports confusion matrices and classification reports. However, the evaluation protocol as described invalidates the headline result, so the current manuscript does not establish its central claim.","major_comments":[{"comment":"The evaluation is circular. The text states that after the semi-supervised loop, \"the model is trained again, this time using the full labeled dataset and the validation dataset,\" and Section 5 states \"we trained and validated on the labeled dataset.\" No held-out test set is defined or used. Consequently, the reported 92.9% accuracy and the +3.8% improvement over baseline are measurements on training data, not estimates of generalization. This is a load-bearing flaw because the central claim of the paper depends entirely on this number.","section":"Section 4, paragraph 1"},{"comment":"The paper never specifies the labeled/unlabeled split, the number of labeled samples per class, or how the unlabeled pool was selected. Without this information, the reader cannot verify that the semi-supervised setting is meaningful or that the claimed per-class gains (e.g., pneumonia recall from 0.71 to 1.0) are not artifacts of an unbalanced or inadvertently informative split.","section":"Sections 4-5"},{"comment":"The ablation results (89.7% without Co-Refinement, 90.7% without Co-Refurbishing) are reported as point estimates with no measure of variance, no repeated runs, and no significance test. Given the very small sample sizes in some classes (13-16 total recordings), these differences are within plausible random variation, so the claim that the combination \"leads to an improvement\" is not supported.","section":"Section 5, ablation paragraph"}],"minor_comments":[{"comment":"The phrase \"feature insufficient labeled data\" appears to be a typo; it should likely read \"feature complexity and insufficient labeled data.\"","section":"Abstract"},{"comment":"The claimed \"increase of 3.8%\" should be stated as \"3.8 percentage points\" to avoid ambiguity.","section":"Abstract and Conclusion"},{"comment":"The CNN training hyperparameters (learning rate, batch size, number of epochs, optimizer) are not reported, nor is the MFCC configuration (e.g., number of filters, window length) beyond the coefficient count; this hinders reproducibility.","section":"Section 3.1"},{"comment":"The baseline accuracy (89.1%) appears only in the Conclusion; reporting it in Section 5 alongside the baseline classification report would improve clarity.","section":"Section 5"},{"comment":"The figures are referenced without explanation of what is plotted (e.g., normalized or raw values); the captions are also minimal, making them difficult to interpret independently.","section":"Figures 4-7"}],"recommendation":"reject","confidential_remarks":"This manuscript is a short conference-style paper with a fundamental evaluation flaw: the reported accuracy is computed after training on the validation set, so the headline result is not an estimate of generalization. In addition, essential experimental details (data split, hyperparameters, code) are missing. The paper's contribution is limited to applying existing semi-supervised modules to a public dataset, and the current evidence does not support the claimed improvement. A major revision would require rerunning all experiments with a proper train/validation/test split and reporting uncertainty estimates, which is beyond the scope of a minor revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here is the short version: the central claim is not established. The paper's own text says the final model is trained again using the full labeled dataset and the validation dataset, and later says they 'trained and validated on the labeled dataset.' No test split is defined, so the reported 92.9% (and the +3.8% over the 89.1% baseline) cannot be read as held-out accuracy. If the validation set participated in training, the number measures transductive fit at best.\n\nWhat the paper does offer: a clear, honest application of three existing semi-supervised modules (MixMatch, Co-Refinement, Co-Refurbishing) to an MFCC+CNN baseline on the ICHBI dataset, with per-class results and simple ablations. The authors correctly note that prior semi-supervised lung sound classifiers exist (Lang et al.), so they are not claiming a first application. The specific combination and the reported improvements are new empirical results.\n\nThe soft spots are substantial. Beyond the evaluation problem, there are no split sizes, hyperparameters, error bars, or code, so the experiment is not reproducible. The per-class gains (e.g., pneumonia recall from 0.71 to 1.0) are uninterpretable if the evaluation set was used in training. And the novelty is modest, since graph-based semi-supervised methods for lung sounds already appear in the cited literature.\n\nWho this is for: researchers testing semi-supervised modules for respiratory audio might find the combination plausible and worth trying, but the current manuscript does not give them a trustworthy result. The right path is to return the paper to the authors with a request to rerun the evaluation on a genuinely held-out test split, report exact split sizes and hyperparameters, and share code. If a corrected run confirms the trend, a much smaller claim could be credible. As it stands, I would desk reject rather than send it to peer review.","headline":"The evaluation is circular: the text says the final model is trained on the validation set, so the headline 92.9% is not an honest estimate of test accuracy.","tokens_in":6635,"tokens_out":3862,"would_cite":false,"duration_ms":39136,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Semi-supervised learning modules added to an MFCC+CNN model raise six-class lung-sound classification accuracy on an open respiratory sound dataset from 89.1% to 92.9%.","keywords":["lung sounds","semi-supervised learning","MFCC","CNN","Mix-Match","Co-Refinement","Co-Refurbishing","respiratory sound classification"],"falsifier":"Retrain the identical MFCC+CNN and semi-supervised pipeline with a strict split where the test recordings are withheld until after training, then compare the held-out accuracy to the reported 92.9% and 89.1% baselines.","tokens_in":5543,"feed_emoji":"🫁","tokens_out":6059,"duration_ms":57868,"temperature":0.7,"pith_summary":"The paper sets out to show that semi-supervised learning can improve lung-sound disease classification without requiring more manual annotation. It starts from an MFCC+CNN classifier and adds three semi-supervised modules—Mix-Match, Co-Refinement, and Co-Refurbishing—that make use of unlabeled recordings during training. On a 920-recording respiratory sound dataset with six diagnostic classes, the combined model reaches 92.9% accuracy, up from the 89.1% baseline. The intended takeaway is that unlabeled audio can substitute for scarce labeled data, with the largest gains appearing in classes that had enough samples to learn from.","feed_headline":"Semi-supervised modules lift lung-sound accuracy to 92.9%","feed_subtitle":"MFCC+CNN gains 3.8 points on six respiratory classes by adding Mix-Match, Co-Refinement, and Co-Refurbishing.","key_machinery":"The pipeline consists of an MFCC feature front end, a four-layer 2x2-filter CNN classifier with six output nodes, and three semi-supervised modules applied each epoch. Mix-Match blends labeled and unlabeled samples through Mix-Up augmentation and sharpened pseudo-labels; Co-Refinement trains the model on its own predictions for unlabeled data merged with labeled data; Co-Refurbishing mixes unlabeled predictions with a subset of labeled data to update model parameters. The open-access respiratory sound dataset used in the study supplies 920 recordings from 126 participants across six respiratory conditions, and the semi-supervised modules are what carry the claimed gain by turning unlabeled recordings into training signal.","core_discovery":"The central claim is that the combination of Mix-Match, Co-Refinement, and Co-Refurbishing with an MFCC+CNN backbone improves six-class respiratory sound classification from 89.1% to 92.9% accuracy. The paper reports that pneumonia precision and recall improve from 0.36 and 0.71 to 0.78 and 1.00, and that COPD, the most frequent class, reaches precision 0.97 and recall 1.00. Removing Co-Refinement drops accuracy to 89.7%, and removing Co-Refurbishing drops it to 90.7%, so the authors argue both modules contribute to the final gain. Classes with very few samples, bronchiectasis and bronchiolitis, improve less, which the paper attributes to the small number of training examples available for them.","pith_inferences":["My inference: the 3.8-point gain should be treated with caution until the model is evaluated on a test split that never participates in training, because the paper's own wording suggests the validation set was used during training.","My inference: the same module stack could be tested on other respiratory sound benchmarks or on spectrogram features besides MFCC to see whether the gain transfers.","My inference: the poor results for bronchiectasis and bronchiolitis suggest that class-balanced sampling or synthetic augmentation, not just semi-supervision, would be needed to improve the smallest classes."],"forward_implications":["If the gain is real, semi-supervised training can lift lung-sound classification by roughly four accuracy points without new labeled recordings.","The per-class results suggest that classes with enough training samples, such as COPD, pneumonia, and healthy, benefit most, while very small classes remain hard to improve.","The reported sensitivity to removing either Co-Refinement or Co-Refurbishing implies that the two modules are complementary rather than redundant.","A working semi-supervised pipeline would lower the annotation burden for building lung-disease screening tools from stethoscope recordings."],"supporting_citations":[{"why":"supplies the Mix-Match label-guessing and Mix-Up augmentation procedure used as the first training module each epoch.","marker":"[2]"},{"why":"provides the MFCC feature extraction approach used for the baseline front end.","marker":"[3]"},{"why":"supplies the CNN architecture used as the classification backbone.","marker":"[5]"},{"why":"introduces the co-refinement and co-refurbishing training ideas that the paper adapts into its two additional modules.","marker":"[10]"},{"why":"provides the open-access respiratory sound dataset used for training and evaluation.","marker":"[11]"},{"why":"contributes the SELFIE refurbishing strategy that underlies Co-Refurbishing.","marker":"[12]"}],"fun_headline_variants":["Semi-supervised modules lift lung-sound accuracy to 92.9%","Lung disease diagnosis accuracy hits 92.9% via semi-supervision","MFCC+CNN with semi-supervision reaches 92.9% for lung sounds","Semi-supervised learning lifts lung diagnosis to 92.9% accuracy","Semi-supervised modules boost lung-sound diagnosis to 92.9%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the 92.9% accuracy is measured on recordings the model never trained on, since the paper says the model is trained again on the full labeled and validation sets before evaluation.","fun_headline_variants_meta":{"raw":{"variants":["Semi-supervised modules lift lung-sound accuracy to 92.9%","Lung disease diagnosis accuracy hits 92.9% via semi-supervision","MFCC+CNN with semi-supervision reaches 92.9% for lung sounds","Semi-supervised learning lifts lung diagnosis to 92.9% accuracy","Semi-supervised modules boost lung-sound diagnosis to 92.9%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000667,"raw_usage":{"total_tokens":2995,"prompt_tokens":849,"completion_tokens":2146,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":465,"completion_tokens_details":{"reasoning_tokens":2039}},"tokens_in":465,"tokens_out":2146,"duration_ms":15746,"temperature":1.0,"reasoning_tokens":2039,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T15:41:08.733648+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the identical MFCC+CNN and semi-supervised pipeline with a strict split where the test recordings are withheld until after training, then compare the held-out accuracy to the reported 92.9% and 89.1% baselines.","supporting_citations":[{"cited_title":"Mixmatch: A ho listic approach to semi-supervised learning","cited_arxiv_id":null,"evidence_quote":"supplies the Mix-Match label-guessing and Mix-Up augmentation procedure used as the first training module each epoch."},{"cited_title":"High accuracy classification of COVID -19 coughs using Mel-frequency cepstral coefficients and a convoluti onal neural network with a use case for smart home devices","cited_arxiv_id":null,"evidence_quote":"provides the MFCC feature extraction approach used for the baseline front end."},{"cited_title":"CNN architectur es for large-scale audio classification","cited_arxiv_id":null,"evidence_quote":"supplies the CNN architecture used as the classification backbone."},{"cited_title":"Improving unsupervised image clustering with robust learning","cited_arxiv_id":null,"evidence_quote":"introduces the co-refinement and co-refurbishing training ideas that the paper adapts into its two additional modules."},{"cited_title":"An open access database for the evaluation of respiratory sound classification algorithms","cited_arxiv_id":null,"evidence_quote":"provides the open-access respiratory sound dataset used for training and evaluation."},{"cited_title":"Selfie: Refurbishing unclean samples for robu st deep learning","cited_arxiv_id":null,"evidence_quote":"contributes the SELFIE refurbishing strategy that underlies Co-Refurbishing."}],"review_version":1}