{"id":"5f9bbd89-9e76-4092-81b5-e5305d7331de","arxiv_id":"2607.16220","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"PCEN (0.915) and multi-resolution (0.916) spectrogram front-ends slightly outperform plain log-mel (0.910) on PhysioNet 2016 modified accuracy, with ~0.95 sensitivity across all three.","lead":"This study compares three ways of turning heart-sound recordings into spectrogram images for the same small neural network on the public PhysioNet 2016 dataset. All three catch about 95% of abnormal clips, and the fancier front-ends (PCEN, multi-resolution) beat plain log-mel by roughly half a point on the official accuracy metric — a gap within the seed-to-seed noise the authors describe.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Single-seed design makes the reported front-end ordering statistically unsupported: §9 admits 1–2 point seed shifts vs 0.5–0.6 point deltas in Table 1, so the 'edges out' claim needs multi-seed replication.","rationale":"The paper is an honest, controlled comparison: fixing architecture, optimizer, epochs, class weights, and seed is the right way to isolate front-end effects, and the recording-level split avoids leakage. The main finding—all front-ends achieve sensitivity ~0.95 with modest specificity differences—is plausible and consistent with prior work. However, the headline ordering is supported by a single run per front-end. The paper's own §9 admits that seed-to-seed MAcc variation in the smaller architecture is 1–2 points, dwarfing the 0.5–0.6 point gaps in Table 1. Since no confidence intervals or repeated runs are provided, the \"edges out\" language in the abstract and the choice of multi-resolution as \"best\" in §6.4 are not statistically supported. The reader's CONDITIONAL verdict already captures this: the required condition is multi-seed replication or an explicit demotion of the ordering. Our stress test finds no additional internal inconsistency; the Grad-CAM claims are appropriately cautious, and the class-weighting explanation for high sensitivity is transparent. The one thing that would change the verdict is if a multi-seed replication showed the ordering is consistently positive; absent that, the current conditional acceptance is appropriate. We therefore recommend no change to the reader's verdict.","tokens_in":9634,"tokens_out":3763,"duration_ms":39141,"concrete_test":"Run the Question 1 main-architecture experiment for all three front-ends with the same fixed recording-level split but 10 different random seeds (varying only the global seed). For each seed, compute the paired differences Δ_PCEN = MAcc_PCEN − MAcc_vanilla and Δ_MR = MAcc_MR − MAcc_vanilla on the same test clips. Report the mean and 95% bootstrap confidence interval of these paired differences. If the CI includes zero or the sign flips across seeds, the 'edges out' claim is not supported and the front-ends should be reported as statistically indistinguishable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that PCEN and multi-resolution reliably outperform vanilla log-mel on the PhysioNet modified accuracy (0.915/0.916 vs 0.910, Table 1)—rests on a single random seed. The paper itself states in §9 that \"re-running the smaller-architecture experiment shifted modified accuracy by one to two points between seeds.\" That seed-level variation (0.01–0.02) is 2–4× the 0.005–0.006 MAcc gaps in Table 1. Because each front-end was trained only once, the observed ordering is one draw from a noisy distribution; without repeated-seed runs there is no evidence that the ordering is stable. The abstract's \"edge out\" language and the §6.4 conclusion that multi-resolution is \"best-performing\" overstate what a single run can establish. A secondary concern is that test metrics treat 1,984 clips as independent although they come from 486 recordings (§3); correlated clips inflate the effective test size, so the sensitivity ≈0.95 and the Table 1 differences have unknown confidence intervals. The paper honestly flags the seed issue in §9, but the abstract and conclusions present the ranking as stronger than the evidence supports.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports a controlled comparison of three spectrogram front-ends — plain log-mel, PCEN, and multi-resolution log-mel — for binary abnormal/normal heart-sound classification on the PhysioNet/CinC 2016 dataset. A fixed compact CNN, optimizer, training schedule, and random seed are used across conditions; only the input representation changes. The central claim is that PCEN and multi-resolution log-mel modestly outperform plain log-mel on the official modified-accuracy metric (0.915 and 0.916 vs 0.910), mainly by reducing false positives on normal clips, and that this advantage persists and even widens when the network is made smaller (§6, §7). Grad-CAM analysis is used to argue that the models attend to low-frequency S1/S2 regions. The paper is written as an independent, non-affiliated project and contains many informal passages and typos.","tokens_in":9866,"tokens_out":2955,"duration_ms":34196,"significance":"If the reported effect were statistically established, the paper would provide a useful empirical data point: for a fixed architecture and training protocol, the spectral front-end can influence specificity without sacrificing abnormal-class sensitivity, and the effect is larger when model capacity is reduced. The design has genuine strengths: a recording-level stratified split (preventing clip-level leakage), a held-out test set chosen by validation F1, identical training conditions across front-ends, and explicit reporting of confusion matrices and the official PhysioNet metric. The authors also honestly flag the run-to-run variance caveat in §9. However, the headline deltas in Table 1 are smaller than the seed-to-seed variation the paper itself reports, which undermines the 'edges out' and 'best-performing' conclusions as currently stated. The clip-level evaluation on correlated clips further overstates the precision of the reported metrics.","major_comments":[{"comment":"The central claim that PCEN and multi-resolution 'edge out' plain log-mel (0.915/0.916 vs 0.910) rests on a single random seed. The paper itself states in §9 that 're-running the smaller-architecture experiment shifted modified accuracy by one to two points between seeds.' That seed-level variation (0.01–0.02) is 2–4 times the size of the Table 1 differences (0.005–0.006), so the observed ordering is one draw from a noisy distribution. The abstract and §6.4 present the ranking as reliable ('consistently beat', 'best-performing'), which overstates the evidence. The authors should either run multiple seeds and report means/variances or substantially temper the claims to 'tendency' as §9 suggests.","section":"§9 and Table 1"},{"comment":"The test evaluation treats 1,984 clips as independent samples, but these clips come from only 486 recordings and clips from the same recording are strongly correlated. This inflates the effective test size behind sensitivity ≈0.95 and the reported accuracy differences. The paper should use recording-level clustered inference (e.g., cluster bootstrap or aggregating predictions per recording) or at least report confidence intervals that account for this correlation. Without this, the small front-end differences in Table 1 have unknown statistical precision even beyond the seed issue.","section":"§3 and Table 1"},{"comment":"The statement that multi-resolution 'performs better than the Potes et al first place winner of the PhysioNet 2016 challenge' (§6.4) is not supported by the experiment design. Potes et al. reported on the challenge's hidden test set under the challenge protocol, whereas this paper uses a different split, a different subset of recordings, and a different model-selection rule. Such cross-protocol comparisons are not meaningful and should be removed or heavily qualified.","section":"§6.4 and §7"},{"comment":"The Grad-CAM 'low-frequency S1/S2 focus' interpretation is partly by construction: the mel input is band-limited to 20–800 Hz, so the model cannot attend to anything above 800 Hz. Calling the low-frequency attention evidence that 'the network is learning a stable acoustic pattern tied to cardiac function rather than exploiting noise' (§9) is therefore weaker than claimed. To support this, the authors would need to show, for example, that attention localization differs meaningfully from what a model trained on shuffled labels or on a noise-only input would produce, or compare against a baseline with a wider frequency band.","section":"§8 and §9"}],"minor_comments":[{"comment":"The manuscript contains numerous typos and grammatical errors that impede readability: 'thsi', 'classifer', 'abonral', 'teh', 'purposelfully', 'Mutli-resolution', 'Recoridngs', 'Compariosn', 'indciate', 'heratbeast', 'archiecture', 'arcitecture', etc. A thorough proofread and language edit is needed.","section":"Throughout"},{"comment":"The abstract's informal tone ('Heart disease kills a lot of people', 'Turns out') is not appropriate for a journal submission. The background section also contains sentence fragments and misspellings that should be corrected.","section":"Abstract and §1"},{"comment":"PCEN and Grad-CAM are introduced without citations. The authors should cite the original PCEN paper (Lostanlen et al., 2019) and Grad-CAM (Selvaraju et al., 2017) so readers can verify the implementations.","section":"§5 and References"},{"comment":"The text says 'Both PCEN and the Mutli-resolution improve on the regular vanilla baseline accross all the major metrics, including F1, specificty and the modified Physionet Accruacy'. But Table 1 shows PCEN has sensitivity 0.951 vs vanilla 0.953, i.e., not all metrics improve for both front-ends. Please correct this overstatement.","section":"§6.2 and Table 1"},{"comment":"The note from the authors ('I didn't have enoggh compute credits...') is written in first person and breaks the formal register; it should be moved to a limitation paragraph or appendix, and the spelling corrected.","section":"§7"},{"comment":"The Grad-CAM formula uses 'y^c' and 'A^k_{i,j}' without defining 'c' as the class index. A one-sentence definition would improve clarity.","section":"§8.1"}],"recommendation":"major_revision","confidential_remarks":"The paper's empirical design is well controlled in the ways that matter (recording-level split, identical training recipe), but the headline conclusion currently overreaches the statistical evidence. The seed-variance admission in §9 is the key problem: the reported front-end ordering is within the noise band the authors themselves acknowledge. I would ask for multi-seed runs or a clustered bootstrap analysis, and for removal of the unsupported comparison to the PhysioNet challenge winner, before this can be accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper before sending it to a referee: it is a genuinely controlled comparison of three spectrogram front-ends for a fixed heart-sound CNN, and it is honest about its own fragility. The stated deltas, though, are smaller than the paper's own reported run-to-run variance, so the \"edges out\" language in the abstract oversells the finding.\n\nWhat is actually new and worth credit: the three-way comparison on PhysioNet 2016 with architecture, optimizer, epochs, and seed held fixed is not in the three cited prior works. The recording-level split is correct and prevents clip-level leakage. The inverse-frequency class weighting is transparent, and the small-architecture experiment in Question 2 is a nice extra: the vanilla front-end degrades much more when the network shrinks, which is internally consistent with the idea that adaptive front-ends carry useful structure. The Grad-CAM section is appropriately worded as interpretation, not proof, and the paper flags that the low-frequency focus is partly a consequence of the 20–800 Hz input band. No circular reasoning here; evaluation is on a held-out split, and the high sensitivity is traced to class weighting.\n\nThe soft spots, in order of severity. First, the entire ranking in Table 1 rests on one random seed. Section 9 says re-running the smaller-architecture experiment shifted modified accuracy by one to two points between seeds, which is two to four times the 0.005–0.006 gaps in Table 1. So the only defensible claim is that all three front-ends perform similarly, with a possible small specificity advantage for PCEN and multi-resolution. The abstract and conclusion should say that, or the authors should run multiple seeds and report error bars. Second, test metrics treat 1,984 clips as independent even though they come from 486 recordings. Clips from the same abnormal recording are correlated, which inflates the effective test size behind the sensitivity and specificity numbers. Third, the claim in §6.4 that multi-resolution \"performs better than the Potes et al first place winner\" compares against a different test protocol without justification; that sentence should go. Minor: the notebook is not released, despite the emphasis on the implementation; and the text has many typos and grammar slips, which a copyedit would fix but which don't affect the experiments.\n\nWho is this for? Builders of heart-sound screening classifiers who want a rough sense of whether PCEN or multi-resolution stacks are worth trying on their own data. It is a modest, useful data point, not a breakthrough. It deserves a serious referee: the design is sound, the limitation is clearly fixable, and the conclusions are overstated rather than false. A referee should ask for multi-seed averages with confidence intervals or a demoted ordering claim, and for clip-level clustering in the test metrics. That is major-revision material, not a desk reject.","headline":"A cleanly run, honestly caveated front-end comparison whose headline ordering is not statistically supported by its own single-seed design.","tokens_in":10531,"tokens_out":1480,"would_cite":false,"duration_ms":17997,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Choosing how a heart-sound recording is turned into a spectrogram changes how well a fixed neural network finds abnormal heartbeats, with adaptive front-ends beating the plain baseline on the benchmark's official metric.","keywords":["heart-sound classification","spectrogram front-end","PCEN","multi-resolution spectrogram","log-mel spectrogram","convolutional neural network","Grad-CAM","modified accuracy"],"falsifier":"Run the same three front-ends over at least five random seeds with the same splits; if PCEN and multi-resolution do not maintain a positive average margin over vanilla log-mel — or if the margin disappears when evaluation is aggregated per recording instead of per clip — the claim that the front-end alone causes the improvement is not supported.","tokens_in":9342,"feed_emoji":"🫀","tokens_out":7128,"duration_ms":67498,"temperature":0.7,"pith_summary":"This paper tries to isolate one variable in heart-sound screening: the audio representation fed to a convolutional network. With the same network, optimizer, epochs, class weighting, and random seed, it compares a plain log-mel spectrogram against two adaptive front-ends — PCEN, which normalizes each frequency channel against its own recent background, and a multi-resolution stack that feeds three different window sizes as three input channels. On the standard 2016 heart-sound benchmark, all three catch roughly 95% of abnormal clips, but the adaptive front-ends reduce false alarms on normal clips and raise the official modified-accuracy metric from 0.910 to 0.915–0.916. The advantage widens when the network is made smaller, and Grad-CAM attention lands on the low-frequency S1/S2 heart-sound band. A sympathetic reading takes the result to be that representation quality is a real, modest lever that matters most when model capacity is limited.","feed_headline":"Two adaptive audio transforms beat plain spectrograms for heart sounds","feed_subtitle":"Same network, same training: the input image alone lifts the benchmark score from 0.910 to 0.916.","key_machinery":"The key mechanism is the controlled comparison itself: one compact CNN trained three times, once per front-end, with the only difference being the input image. The three front-ends are a vanilla log-mel spectrogram (static decibel compression applied identically to every frequency bin), PCEN (per-channel energy normalization, an automatic gain control that suppresses each channel's slow background and emphasizes locally loud transients such as murmurs), and a multi-resolution log-mel stack (three FFT window sizes — 256, 512, and 1024 samples — stacked as three input channels, trading time resolution against frequency resolution). Because the downstream model and training recipe are identical","core_discovery":"The central claim is that the front-end alone moves the outcome: holding every training condition fixed, PCEN and multi-resolution log-mel both outscore vanilla log-mel on the benchmark's modified-accuracy metric (0.915 and 0.916 versus 0.910), and the gain comes almost entirely from specificity — false alarms on normal clips drop from 197 to 179 and 182 out of 1,984, while sensitivity stays near 0.95. The same pattern is stronger in a smaller two-block network, where the two adaptive front-ends lose only about two points of modified accuracy but the plain baseline loses more than eight. Averaged Grad-CAM maps concentrate on the low-frequency S1/S2 band, with abnormal clips drawing more atte","pith_inferences":["The authors report that rerunning the smaller architecture changed scores by one to two points between seeds — two to four times the 0.5-point gaps in the main table — so a natural next step is a multi-seed replication; if the average margin does not stay positive, the front-end ordering should be read as a tendency, not a ranking.","An implication the authors leave implicit is that PCEN's advantage on the smaller network suggests a capacity-for-representation trade: for a fixed accuracy budget, spending computation on the front-end may be cheaper than adding model parameters.","Because clips from the same recording are correlated, reporting per-recording majority-vote accuracy would clarify whether the apparent 95% sensitivity reflects independent evidence or a handful of recordings.","The multi-resolution stack's broad attention pattern hints that it may be less sensitive to stethoscope placement and patient-specific timing than single-window front-ends; a cross-dataset test would check that."],"forward_implications":["If the front-end alone shifts the decision boundary, representation design is a viable substitute for architecture or hyperparameter search in low-data audio tasks.","The false-alarm reduction means an adaptive front-end can lower the cost of follow-up screening without sacrificing abnormal-case detection.","If the smaller-network result generalizes, an informative representation can let a much smaller model match a larger one, which matters for edge devices and battery-powered stethoscopes.","Because attention concentrates in the S1/S2 band, the same pipeline may be a foundation for finer-grained classification of specific heart conditions rather than just normal-versus-abnormal."],"fun_headline_variants":["Front-end swap lifts heart-sound CNN score from 0.910 to 0.916","Adaptive spectrograms edge out plain logmel for heart-sound CNNs","Same CNN, better heart-sound detection via smarter input images","PCEN and multi-res spectrograms beat logmel in heart-sound test","How you picture heart sounds matters: adaptive beats standard"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The ordering of the three front-ends is treated as stable even though the paper reports that rerunning the smaller-network experiment shifts modified accuracy by 0.01–0.02 between seeds, two to four times the 0.005 gaps that separate the front-ends, and test clips from the same recording are counted as independent samples.","fun_headline_variants_meta":{"raw":{"variants":["Front-end swap lifts heart-sound CNN score from 0.910 to 0.916","Adaptive spectrograms edge out plain logmel for heart-sound CNNs","Same CNN, better heart-sound detection via smarter input images","PCEN and multi-res spectrograms beat logmel in heart-sound test","How you picture heart sounds matters: adaptive beats standard"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000286,"raw_usage":{"total_tokens":1548,"prompt_tokens":802,"completion_tokens":746,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":546,"completion_tokens_details":{"reasoning_tokens":657}},"tokens_in":546,"tokens_out":746,"duration_ms":6741,"temperature":1.0,"reasoning_tokens":657,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T11:22:27.702035+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same three front-ends over at least five random seeds with the same splits; if PCEN and multi-resolution do not maintain a positive average margin over vanilla log-mel — or if the margin disappears when evaluation is aggregated per recording instead of per clip — the claim that the front-end alone causes the improvement is not supported.","supporting_citations":[],"review_version":1}