{"id":"e6d60a7e-ede4-4f3f-9c2b-f5f66c5be07b","arxiv_id":"2412.01996","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Using band-specific spectral features and a mean/std long-term representation, the system reports 92.71% sensitivity and 88.58% specificity for cough detection, but only in the low-noise protocol part.","lead":"This paper describes a smartphone-audio cough detector that splits sound into five frequency bands, selects 29 robust audio features, and classifies 300 ms segments with a support vector machine. The reported accuracy is about 90 percent, but the headline numbers come from the quietest test scenario and the dataset is small and private.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Feature selection in §III-B/IV-A may use test-set labels before cross-validation, potentially inflating Table IV/V; nested selection is needed to validate the claimed superiority.","rationale":"The reader's weakest_assumption focuses on manual annotation reliability, which is a valid concern. However, the most load-bearing weakness is the feature-selection protocol: §III-B and §IV-A describe supervised Relieff selection on random subsets of observations before the cross-validation partitions are defined, and the text only claims that long-term features from train/test sets were not used. This leaves open the possibility that short-term frames from test blocks were included in feature selection, leaking test labels into model construction. That would directly inflate the headline metrics and the comparison with state-of-the-art methods, undermining the central claim. The proposed nested-selection test is concrete and would settle the issue: if results hold under proper nesting, the concern is resolved; if not, the paper's conclusion is not supported. I am not moving the verdict because the reader already assigned CONDITIONAL, and this concern reinforces rather than overturns that assessment; the paper is plausible but the evidence as presented is not fully reliable.","tokens_in":15255,"tokens_out":4131,"duration_ms":43308,"concrete_test":"Rerun the complete pipeline with feature selection nested inside each of the five block-defined folds: for each fold, run the §III-B Relieff and combination procedure using only the training blocks, choose 29 features, then evaluate on the held-out block. Repeat for the three protocol parts and for the state-of-the-art comparisons in Table V. If the re-estimated SEN/SPE/AUC drop materially (e.g., more than 2–3 percentage points) or the ordering against [17] and [18] changes, the claimed advantage is partly an artifact of leakage. The authors should also release the fold and selection-subsample membership masks to confirm no overlap.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the proposed AvgSD pipeline outperforms prior cough detectors, supported by Table IV/V and the abstract's 92.71% sensitivity, 88.58% specificity, 90.69% AUC. The evaluation protocol has a potential leakage in feature selection. In §III-B, Relieff is applied to 10% of observations per protocol part, with no statement that these observations are restricted to training folds. In §IV-A, the process is repeated five times on disjoint random subsets totaling 50% of observations, again before the block-wise 5-fold partitions described in §III-D are applied. The paper's only safeguard is the sentence 'no long-term features from the train or test sets in each of the 5 folds in Fig. 7 was employed for selection' (end of §IV-A). That statement addresses long-term features, not the short-term frames used by Relieff. Since Relieff is supervised, if any randomly selected short-term frame belongs to a block or patient later used for testing, the 29 chosen features are selected with knowledge of test labels. This can overstate SEN/SPE/AUC and, more importantly, bias the comparison with [15], [17], and [18], all of which are evaluated using the same leaked feature set. The reported 'outperform[ing] so far proposed methods' conclusion is therefore not reliable unless feature selection is provably nested inside each training fold.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a two-stage machine-hearing system for cough detection from audio. Short-term spectral features are computed in five predefined frequency bands and combined with standard audio descriptors; a supervised Relieff-based procedure selects 29 features; a high-level representation is obtained either by mean/standard deviation (AvgSD) of 300-ms long-term frames or by supervised bag-of-audio-words; classification is performed by a polynomial-kernel SVM. The system is evaluated on 13 patients recorded under three noise scenarios, with per-part models, a leave-one-patient-out cross-validation, and a final ensemble. The abstract reports 92.71% sensitivity, 88.58% specificity, and 90.69% AUC for the first (low-noise) part, and the paper claims the approach outperforms state-of-the-art methods including a CNN.","tokens_in":15659,"tokens_out":4507,"duration_ms":46451,"significance":"If the reported results are valid, the paper would demonstrate that a lightweight handcrafted-feature pipeline with a high-level AvgSD representation can match or beat deep-learning cough detectors on noisy real-life recordings, which is practically relevant for smartphone-based continuous monitoring. The study has clear strengths: evaluation in three protocol-defined noise environments, leave-one-patient-out cross-validation, statistical comparison via McNemar's test, and an explicit acknowledgment of the small dataset as a limitation. However, the central claim of superiority is currently supported only by the per-part models, and the evaluation protocol has a potentially serious feature-selection leakage problem that directly affects the headline numbers. The significance of the contribution is therefore conditional on the evaluation being corrected.","major_comments":[{"comment":"The feature selection procedure appears to use test labels before the cross-validation splits are formed. Section III-B selects 10% of observations per protocol part with Relieff, and Section IV-A repeats this five times on disjoint subsets, totaling 50% of the observations. The block-wise five-fold partitions described in Section III-D are then applied afterward. The only safeguard stated at the end of Section IV-A is that 'no long-term features from the train or test sets in each of the 5 folds in Fig. 7 was employed for selection,' which addresses long-term features only, not the short-term frames to which Relieff was applied. Since Relieff is supervised, any randomly selected short-term frame belonging to a later test block or test patient can influence the chosen 29 features, inflating the results in Table IV and Table V and biasing the comparison with [15], [17], and [18]. The authors must either prove that no observation used in selection ever appears in a test fold, or rerun the whole pipeline with feature selection nested inside each training fold and report the resulting SEN/SPE/AUC.","section":"III-B, IV-A, III-D"},{"comment":"The abstract's headline numbers (92.71% sensitivity, 88.58% specificity, 90.69% AUC) are taken from the first part of the protocol in Table IV, which is the low-noise scenario, not from the final deployed system. The final system, whether the single model in Fig. 10(b) or the majority-voting ensemble in Fig. 10(a), achieves sensitivity in the 90% range but specificity around 80%. The conclusion that the system 'outperform[s] so far proposed methods in terms of cough detection' is therefore not supported by the final-system evaluation. The paper should either report the final-system comparison against the state-of-the-art methods, or clearly qualify the headline claim as applying to the per-part models and not to the system as a whole.","section":"Abstract, Table IV, Fig. 10, Conclusions"},{"comment":"The manual annotation procedure is described only as 'Audio files were manually annotated on a time-frame basis' with majority-class assignment for mixed frames. No inter-annotator reliability, annotation protocol details, or confidence measures are reported. Since these labels define both the training targets and the ground truth for evaluation, label noise or systematic bias could directly affect the reported sensitivity and specificity, regardless of classifier performance. The authors should provide inter-annotator agreement statistics or, at minimum, a sensitivity analysis showing that the main conclusions are robust to plausible annotation errors.","section":"II"},{"comment":"The abstract and conclusions state an AUC value of 90.69%, but Tables IV and V report only SEN, SPE, and ACC, and no numeric AUC values are given for any model or comparison. ROC curves are shown in Fig. 8, but the AUC value in the abstract cannot be traced to a specific table entry. The authors should report AUC values (with confidence intervals) for each part and for the state-of-the-art methods, and state explicitly which model the abstract's AUC refers to.","section":"IV-B, Table IV, Table V, Fig. 8"}],"minor_comments":[{"comment":"The phrase '75 ms frames with 19s overlap' in Section III-D appears to contain a typo: the overlap should be 19 ms, as confirmed by the calculation (75−19)·4+75 = 299 ms.","section":"III-D"},{"comment":"The fifth frequency band is defined inconsistently as [2, 5.5125] kHz in the text and [2, 5.5125) kHz in the band-index list; the intended inclusion of the upper endpoint should be clarified.","section":"III-A1"},{"comment":"Equation (14) for the spectral Renyi entropy is missing parentheses: as written it appears as log(ΣPSD)^q rather than the intended log(ΣPSD^q)/(1−q).","section":"III-A1, Eq. (14)"},{"comment":"The manuscript contains several typographical errors, including 'contaning', 'conciousness', and '?rst' in Section IV-B, and would benefit from a careful proofreading pass.","section":"Whole manuscript"},{"comment":"No code or data availability statement is provided, which limits the reproducibility of the reported results; a public release of the feature extraction and evaluation scripts would strengthen the paper.","section":"II, IV"}],"recommendation":"major_revision","confidential_remarks":"The manuscript header indicates that this is the final accepted version of a paper already published in IEEE Transactions on Biomedical Engineering, vol. 66, no. 8, August 2019, with a DOI. If this is being considered as a new submission, the prior publication should be disclosed and the novelty framing reconsidered. The main technical concern in my report, the feature-selection leakage, is fixable by rerunning the evaluation with nested selection, but the reported headline numbers may change materially."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What should you know: this is a solid engineering paper from 2019, now on arXiv, with a plausible method and a real evaluation weakness that matters for the headline claim. The band-specific computation of 12 spectral features plus mean/std over 300 ms frames is a genuine combination not present in the cited prior work, and the authors compare fairly against three recent detectors on a 13-patient database covering three noise conditions. If the numbers are trustworthy, the system beats a CNN and two handcrafted-feature baselines, which is a useful result for phone-based cough monitoring.\n\nCredit where due: the pipeline is clearly described; the selection process is detailed enough to spot the issue; the leave-one-patient-out experiments are the right generalization check; and McNemar tests on the main comparisons are a step up from just reporting means. The paper also acknowledges the small population and notes that the state-of-the-art methods were originally evaluated on easier train/test splits.\n\nNow the soft spot, and it is a real one. Section III-B runs Relieff on 10% of observations per protocol part, and IV-A repeats this five times on disjoint subsets covering 50% of observations, then picks 29 features. The only safeguard stated is that no long-term features from the train or test sets of the Fig. 7 folds were used for selection. But Relieff runs on short-term frames, and the statement does not say those short-term frames came only from training blocks or training patients. If any selected short-term frame belongs to a block or patient later used for testing, the feature ranking is supervised by test labels and the Table IV/V comparison is biased in favor of the proposed system. The stress-test note is correct on this point. The fix is standard: nest feature selection inside each training fold and report per-fold and per-patient intervals. Also, the abstract quotes only the Part 1 numbers (92.71 SEN, 88.58 SPE); the final system's specificity is closer to 80%, so the abstract is misleading. Minor: no inter-annotator reliability for the frame labels, and no code/data, so the numbers cannot be independently checked.\n\nBottom line: if the leakage is real, the quantitative edge over prior work is not established, but the method itself is coherent and the band-specific features could still be valuable. The paper is worth a serious referee: an editor should send it to review, and the review should push for nested feature selection and honest headline numbers. I'd bring it to a reading group as an example of a clearly written applied ML paper with an evaluation pitfall.","headline":"A coherent applied cough-detection paper with a genuine method contribution, but the evaluation has a feature-selection leakage that undercuts the headline comparison until fixed.","tokens_in":16088,"tokens_out":1531,"would_cite":false,"duration_ms":15238,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Simple audio features beat deep learning on cough detection","keywords":["cough detection","machine hearing","audio classification","spectral features","frequency bands","support vector machine","feature selection","noise robustness"],"falsifier":"Re-annotate the same thirteen hours of recordings with at least one additional independent annotator using the same frame-based rule, then measure inter-annotator agreement; if agreement is low, or if retraining and testing the pipeline on each annotator's labels yields substantially different sensitivity/specificity from the reported 92.71% and 88.58%, the central claim is not settled.","tokens_in":15026,"feed_emoji":"🫁","tokens_out":5743,"duration_ms":48256,"temperature":0.7,"pith_summary":"This paper claims that a lightweight machine hearing system can detect coughs in noisy real-life audio more reliably than previously proposed methods, including a convolutional neural network. The system computes a battery of spectral features separately in five frequency bands, selects 29 of them that stay informative across three noise conditions, and summarizes short-term features by mean and standard deviation over 300 ms windows before feeding an SVM. On recordings from thirteen patients in three simulated real-life scenarios, it reports 92.71% sensitivity, 88.58% specificity, and a 90.69% area under the ROC curve. If correct, the finding matters because it suggests accurate, real-time cough monitoring can run on a smartphone without the computational cost of deep learning.","feed_headline":"Simple audio features beat deep learning on cough detection","feed_subtitle":"A 29-feature SVM hits 92.7% sensitivity in noisy real-life recordings, pointing to low-power smartphone monitoring.","key_machinery":"The machinery is the combination of five predefined frequency bands with a mean/standard-deviation summary (AvgSD) over long-term frames. Each band [0,0.5), [0.5,1), [1,1.5), [1.5,2), [2,5.5125] kHz acts as a separate channel in which spectral centroid, bandwidth, crest factor, flatness, flux, roll-off, f50/f90 ratio, peak entropy, Renyi entropy, kurtosis, skewness, and relative power are computed from a Welch PSD; computing these descriptors per band localizes the impact of colored ambient noise to the affected bands. The 300 ms long-term window matches the average cough duration and, by concatenating feature-wise means and standard deviations, compresses the short-term stream into a compact vector that an SVM can separate robustly.","core_discovery":"The central claim is that cough detection in noisy environments is better served by band-specific short-term spectral features with a high-level mean-and-standard-deviation representation than by the end-to-end deep networks and global-spectrum feature sets used in prior work. The authors construct 117 short-term features—twelve spectral descriptors computed independently in each of five bands covering [0, 5.5125] kHz, plus spectral entropy and standard audio descriptors—then reduce them by Relieff selection combined across three protocol parts to a 29-dimensional set. Long-term frames of about 300 ms are formed from five overlapping 75 ms frames, and the per-feature mean and standard deviation constitute the high-level representation that feeds a polynomial-kernel SVM. In block-wise cross-validation, this AvgSD pipeline yields the reported figures in all three noise scenarios and outperforms the ensemble-subband method of You et al., the CNN of Amoh and Odame, and a moment-based cepstrogram method, with McNemar's test indicating significance for sensitivity and specificity.","pith_inferences":["Inference: the same band-specific plus mean/std recipe could transfer to other impulsive bioacoustic events with sparse spectra, such as sneezes, snoring, or throat clearing, by re-running the Relieff selection on the new target labels.","Inference: because the paper shows the selected features are largely robust across its three noise parts, one testable extension is to evaluate the fixed 29-feature set on additive noise at SNRs below -6 dB or on unseen recording devices, where the band-localization argument predicts graceful degradation.","Inference: the authors do not quantify how much of the gain comes from the band decomposition versus the AvgSD representation; an ablation that computes the same features over the full spectrum with and without mean/std, or over the five bands with short-term classification, would isolate the contribution of each design choice.","Inference: the clinical rationale implies that secondary measures—cough rate, cough intensity, dry versus wet cough—could be derived from the same high-level features, but the paper does not demonstrate this; a follow-up study could test whether the 29 features separate cough subtypes."],"forward_implications":["A smartphone-based cough monitor could run continuously in real time because the final feature set has only 29 dimensions and the classifier is a simple SVM with a polynomial kernel.","The band-specific feature computation appears to make the detector robust across noise levels from low (speech/reading) to high (patient movement plus TV or radio), so a single model can be deployed without first classifying the acoustic environment.","Leave-one-patient-out experiments suggest the system generalizes to new patients in terms of sensitivity and specificity mostly above 80%, with one patient (ID 6) consistently lower.","The comparison results imply that a handcrafted feature pipeline can outperform a CNN for cough detection when the positive class is small and the negative class is diverse, which may guide architecture choices in other audio event detection tasks.","Training three noise-specific models and combining them by majority voting increases specificity relative to a single model, at the cost of some sensitivity; both configurations keep sensitivity in the 90% range."],"supporting_citations":[{"why":"Supplies the average 300 ms cough duration and the average periodogram showing peaks near 500 Hz and 1000–1500 Hz that motivate the five frequency bands.","marker":"[12]"},{"why":"The ensemble-subband cough detector used as a state-of-the-art baseline that the proposed system outperforms.","marker":"[15]"},{"why":"The CNN-based cough segmentation system used as the deep-learning baseline in the comparison.","marker":"[17]"},{"why":"The authors' prior moment-theory-based cough detector, another baseline, which achieves higher specificity but lower sensitivity than the proposed method.","marker":"[18]"},{"why":"The Welch PSD estimation used to compute band-specific spectral features and credited with lower variance.","marker":"[23]"},{"why":"Source of the short-term audio feature definitions and the AvgSD high-level representation baseline.","marker":"[24]"},{"why":"Maximum-likelihood intrinsic-dimension estimator used to set the number of features to select (25–30).","marker":"[31]"},{"why":"Relieff algorithm used for supervised feature selection before the cross-part combination procedure.","marker":"[32]"}],"fun_headline_variants":["Band-specific audio features win for cough detection in noise","SVM with 29 features beats deep learning for cough detection","Robust cough detection via band-specific spectral features","High-level audio representation outperforms deep nets on coughs","Cough spotting: simple features top deep learning in noisy audio"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation rests on the assumption that the manually annotated frame labels in the thirteen-patient database are correct and consistent; if the labels are noisy or biased, both training and evaluation would be affected, and the reported sensitivity and specificity could be misleading.","fun_headline_variants_meta":{"raw":{"variants":["Band-specific audio features win for cough detection in noise","SVM with 29 features beats deep learning for cough detection","Robust cough detection via band-specific spectral features","High-level audio representation outperforms deep nets on coughs","Cough spotting: simple features top deep learning in noisy audio"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000495,"raw_usage":{"total_tokens":2503,"prompt_tokens":1096,"completion_tokens":1407,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":712,"completion_tokens_details":{"reasoning_tokens":1337}},"tokens_in":712,"tokens_out":1407,"duration_ms":8305,"temperature":1.0,"reasoning_tokens":1337,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T23:56:14.299918+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-annotate the same thirteen hours of recordings with at least one additional independent annotator using the same frame-based rule, then measure inter-annotator agreement; if agreement is low, or if retraining and testing the pipeline on each annotator's labels yields substantially different sensitivity/specificity from the reported 92.71% and 88.58%, the central claim is not settled.","supporting_citations":[{"cited_title":"Ambulatory methods for recording cough,","cited_arxiv_id":null,"evidence_quote":"Supplies the average 300 ms cough duration and the average periodogram showing peaks near 500 Hz and 1000–1500 Hz that motivate the five frequency bands."},{"cited_title":"Cough detection by ensembling multiple frequency subband features,","cited_arxiv_id":null,"evidence_quote":"The ensemble-subband cough detector used as a state-of-the-art baseline that the proposed system outperforms."},{"cited_title":"Deep neural networks for identifying cough sounds,","cited_arxiv_id":null,"evidence_quote":"The CNN-based cough segmentation system used as the deep-learning baseline in the comparison."},{"cited_title":"Robust detection of audio-cough events using local hu moments,","cited_arxiv_id":null,"evidence_quote":"The authors' prior moment-theory-based cough detector, another baseline, which achieves higher specificity but lower sensitivity than the proposed method."},{"cited_title":"Haykin, Communication Systems, 5th ed","cited_arxiv_id":null,"evidence_quote":"The Welch PSD estimation used to compute band-specific spectral features and credited with lower variance."},{"cited_title":"Chapter 4 - audio features,","cited_arxiv_id":null,"evidence_quote":"Source of the short-term audio feature definitions and the AvgSD high-level representation baseline."},{"cited_title":"Maximum likelihood estimation of intrinsic dimension,","cited_arxiv_id":null,"evidence_quote":"Maximum-likelihood intrinsic-dimension estimator used to set the number of features to select (25–30)."},{"cited_title":"Theoretical and empirical anal- ysis of relieff and rrelieff,","cited_arxiv_id":null,"evidence_quote":"Relieff algorithm used for supervised feature selection before the cross-part combination procedure."}],"review_version":1}