{"id":"f32aed21-0bb3-4683-91c4-66385fa4b5dc","arxiv_id":"2505.08681","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A Mamba-based model with note-guided pitch decoding and confidence-based consistency training improves singing melody extraction accuracy and inference efficiency on three public datasets.","lead":"This paper describes a computer model that automatically finds the sung melody line in music recordings, using a neural network design that is faster than the transformer models commonly used for this task. The model also learns from unlabeled music and predicts musical notes before refining the pitch estimate, which the authors show improves accuracy on standard datasets.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"MedleyDB training/test overlap is not ruled out; if any of the 35 training tracks appear among the 12 test tracks, the reported MedleyDB gains are invalid.","rationale":"The reader's weakest_assumption is the note-f0 decoder's unspecified note-to-bin mapping. That is a legitimate reproducibility concern, but it is about an internal module's explanation and the transferability of the claimed gain; even if the mapping is arbitrary, the final accuracy numbers could still hold. The train/test overlap on MedleyDB, by contrast, goes directly to the validity of one of the three datasets in the 'highest results in general' claim. The paper's own setup creates the risk: training and testing both draw from MedleyDB, and no disjointness is stated. Because the code is released, this concern is quickly and definitively settled. If overlap is found, the MedleyDB row and the 'in general' wording collapse; if no overlap is found, the paper should still disclose the split to make the evaluation reproducible. The existing CONDITIONAL verdict already asks for additional experimental clarity, so my concern reinforces the need for a specific split disclosure rather than changing the overall verdict.","tokens_in":7400,"tokens_out":6510,"duration_ms":64460,"concrete_test":"Download the released code from https://github.com/Tinkle01/SpectMamba, inspect the data loading script, and enumerate the 35 MedleyDB training filenames and the 12 MedleyDB test filenames; compute the intersection. If the intersection is nonempty, rerun the MedleyDB evaluation with a disjoint split (or drop MedleyDB from the claim) and update Table 2(c). If the intersection is empty, report the split mechanism (e.g., random seed) in the paper and additionally run a per-track bootstrap to estimate the uncertainty of the 4.75 RPA gap.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III-A states that 35 music tracks from MedleyDB are used as training data and that 12 tracks from MedleyDB are used for testing, but it never states that these sets are disjoint. This matters because the central claim of 'highest results in general' is supported by Table 2(c), where SpectMamba beats S2Former by 1.27 OA and 4.75 RPA. If the 12 test tracks are a subset of the 35 training tracks, the model has already seen the test tracks and the MedleyDB comparison is circular. The paper also gives no split procedure, so a reader cannot verify the independence of the MedleyDB evaluation. Even if the split is in fact disjoint, the missing disclosure prevents reproduction of the experimental setup. This is distinct from the note-f0 decoder issue: an underspecified mapping affects interpretability, whereas an overlapping split invalidates a data point in the headline comparison. The same concern does not apply to MIR-1K because the test sets are ADC2004/MIREX 05, not MIR-1K.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SpectMamba, a Mamba-based network for semi-supervised singing melody extraction. The model uses CFP spectrogram patches, a bidirectional state-space encoder with linear complexity, a coarse-to-fine note-f0 decoder that uses note predictions to guide f0 prediction, and a confidence binary regularization (CBR) module that applies consistency supervision between weakly and strongly augmented unlabeled samples using an adaptive top-k confidence split. Experiments on ADC2004, MIREX 05, and MedleyDB report state-of-the-art results, with ablations showing contributions from the note-f0 decoder and CBR, and an efficiency claim of 2x inference speed and 1/10 GPU memory versus S2Former.","tokens_in":7647,"tokens_out":7444,"duration_ms":72239,"significance":"If the results hold, the paper makes a useful contribution to singing melody extraction: it is, to my knowledge, the first Mamba-based SME model, and the linear-complexity encoder directly addresses the quadratic cost of transformer baselines. The note-f0 coarse-to-fine idea is a plausible way to inject musical prior knowledge, and the CBR module is a principled extension of consistency regularization to the dense prediction setting. The paper includes an ablation study that directionally supports both novel components, and it provides a GitHub link to code. However, several specification gaps and evaluation weaknesses currently prevent the contributions from being fully verified.","major_comments":[{"comment":"Section III-A states that 35 MedleyDB tracks are used for training and 12 MedleyDB tracks are used for testing, but it never states that these two sets are disjoint or provides a split procedure. If any of the 12 test tracks are contained in the 35 training tracks, the results in Table II(c) (e.g., SpectMamba OA 72.62 vs. S2Former 71.35) would be invalid because the model would have already seen the test labels. Even if the split is actually disjoint, the missing disclosure prevents verification and reproduction of the experimental setup. The authors must explicitly state that the training and test subsets of MedleyDB are disjoint, provide the exact track lists or split indices, and if any overlap exists, recompute the MedleyDB results on a truly held-out set.","section":"Section III-A / Table II(c)"},{"comment":"The note-f0 decoder relies on a fixed mapping from V note classes to F f0 bins, but the paper never specifies V, how note classes are defined, how the repetition operation in hat(P_note) assigns each f0 bin to a note class, or how Q_note is derived from Q_f0. With 60 bins per octave and a 31-1250 Hz range, a single note class may cover many bins whose frequency content varies; if the mapping is coarse or misaligned, Eq. (8) can suppress correct f0 classes and the reported gains from note supervision would not transfer to other settings. This mapping is required for reproducibility and to assess whether the note prior is actually valid. Please specify the mapping (e.g., a MIDI-note-to-CFP-bin table), the number of note classes V, and the exact procedure used to obtain Q_note from Q_f0.","section":"Section II-C / Eq. (8)"},{"comment":"The CBR algorithm is underspecified. Equation (10) uses k as a fixed number of positive classes in the computation of p_t, but the following sentence says 'adjusting k until the cumulative value exceeds the global top-k confidence,' which implies k is not fixed but is determined per sample. The EMA update for the global confidence mu_t is never written out; the paper only defines p_t and states that EMA is used. This is not a complete algorithm, and it is load-bearing because CBR is the central semi-supervised contribution. Please provide the explicit update rule for mu_t (including the decay schedule and initialization), define the positive-set selection rule unambiguously (whether k is a fixed integer or a per-sample variable), and report the values of k, mu_0, and the EMA decay hyperparameter.","section":"Section II-D / Eq. (10)"},{"comment":"The evaluation uses only 12, 9, and 12 test tracks for ADC2004, MIREX 05, and MedleyDB, respectively, and reports single runs without standard deviations, confidence intervals, or significance tests. The OA improvements over S2Former are 2.24, 1.85, and 1.27 percentage points; with such small test sets, these differences may not be statistically distinguishable from noise. Please report per-track results, bootstrap confidence intervals, or paired significance tests, and if multiple runs are performed, report means and standard deviations. This is necessary to support the claim in Section III-D that the proposed model 'achieves the highest results in general.'","section":"Section III-D / Tables I and II"},{"comment":"The paper claims semi-supervised singing melody extraction, but all three baselines (FTANet, TONet, S2Former) are supervised models; there is no semi-supervised baseline in the comparison. The contribution of CBR is evaluated only through an internal ablation (SpectMamba_B in Table I). To support the semi-supervised claim, the authors should compare against existing semi-supervised SME methods, such as the cited HKDSME [1] and MCSSME [3], trained under the same labeled/unlabeled data setup. Without such a comparison, it remains unclear whether the reported gains over S2Former come from the semi-supervised component or from the supervised architecture, and the claimed advantage in low-label regimes is not directly demonstrated.","section":"Section III-D / Table II"}],"minor_comments":[{"comment":"The caption contains typos: 'SPECTMAMBAN STANDS FOR THE PROPOSED MODEL WITHOUT NOTE-F0 DECODER AND SPECTMAMBAB STANDS FOR THE PROPOSE MOEDEL WITHOUT CBR MODULE' should be corrected to 'SpectMamba_N stands for the proposed model without the note-f0 decoder and SpectMamba_B stands for the proposed model without the CBR module.'","section":"Table I caption"},{"comment":"The notation CE(a_i^s, a_i^w) is ambiguous because both arguments are probability distributions; cross-entropy is normally defined between a predicted distribution and a target distribution or label. Please define the consistency loss precisely, e.g., as H(a_i^w, a_i^s) = -sum a_i^w log a_i^s, or state whether a symmetrized objective is used.","section":"Eqs. (11)-(12)"},{"comment":"The text says that for the backward direction the 1-D convolution and SSM are applied in reverse order, but Eq. (4) writes y_back = SSM_back(Conv_back(...)), which is the same order as the forward direction in Eq. (3). Please clarify the intended order or correct the equation.","section":"Eqs. (3)-(4)"},{"comment":"The title of reference [17] reads 'A scalable sarse Transformer model'; 'sarse' appears to be a typo for 'sparse'.","section":"Reference [17]"},{"comment":"Figure 2 reports inference time and GPU memory usage, but the experimental conditions are not specified: no hardware, batch size, input audio length, number of averaged runs, or measurement methodology is given. Please provide these details so the 2x speed and 1/10 memory claims can be reproduced and interpreted.","section":"Figure 2"},{"comment":"The description of the unlabeled data is minimal: 1000 tracks from FMA are used, but it is not stated whether these are full tracks or excerpts, whether they are processed with the same CFP front-end, or how they are split into weak/strong augmentation pairs. A sentence on the preprocessing and usage of the unlabeled data would improve reproducibility.","section":"Section III-A"}],"recommendation":"major_revision","confidential_remarks":"The most serious concern is the MedleyDB training/test overlap, which is load-bearing for one of the three table entries in the headline comparison. Although the paper includes a code link, the note-to-f0 mapping and the CBR algorithm are underspecified to the point of preventing independent reimplementation, so I recommend asking for a revised manuscript that addresses all major comments. I also note that the paper's references include two prior semi-supervised SME works by the same group (HKDSME and MCSSME), and their absence from the comparisons is conspicuous; this should be resolved in revision. If the MedleyDB split is found to be overlapping, the paper should be reconsidered for rejection, but at this stage the available evidence suggests a major revision is appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper proposes SpectMamba for semi-supervised singing melody extraction: a Vision Mamba encoder, a note-conditioned f0 decoder, and a confidence binary regularization (CBR) module adapted from AllMatch. The combination is new for this task, and the ablations on ADC2004 and MIREX 05 show each component adds a point or two of overall accuracy. That is an honest engineering contribution, not a breakthrough. The efficiency claim (2x faster, 1/10 memory versus S2Former) is directionally plausible given Mamba's linear complexity, but the paper gives no measurement protocol, so treat it as an assertion.\n\nThe soft spots are real. The most serious is the MedleyDB evaluation: the paper says 35 tracks are used for training and 12 for testing but never states the sets are disjoint. If any test track appears in training, the headline gain of 1.27 OA on MedleyDB is invalid. This is easily fixable with one sentence, but without that sentence the comparison is not reproducible. Second, the paper compares only against supervised baselines (FTANet, TONet, S2Former). Since the whole contribution is semi-supervised, the lack of a semi-supervised baseline (e.g., the same model trained without unlabeled data, or an existing semi-supervised SME method) makes it impossible to attribute the gains to the CBR module rather than the Mamba encoder. Third, test sets are tiny (9-12 tracks) and no significance tests or confidence intervals are reported; the claimed 1-2% gains could plausibly be noise. Fourth, the CBR algorithm is underspecified: Eq. (10) uses k before the adaptive k is defined, and the EMA update rule for mu_t is not stated. A reader cannot reimplement it from the text. The note-f0 decoder also never specifies the mapping from note classes to CFP frequency bins, which is a moderate interpretability gap but not a fatal one given the ablation.\n\nThe paper shows clear thinking and the components are standard, so I don't see circularity or self-citation problems. The stress-test note about MedleyDB overlap is the one issue that could sink a headline result; it must be checked before publication.\n\nWho is this for? Researchers in MIR who want a fast, semi-supervised melody extraction model and are willing to re-run the experiments with proper controls. It deserves a serious referee, but the referee should demand the split disclosure, semi-supervised baselines, and a fully specified CBR algorithm. I would not cite it in my own work until those are fixed.","headline":"Plausible semi-supervised SME method with a useful combination of existing ideas, but the undisclosed MedleyDB split and missing semi-supervised baselines keep the headline claims from being substantiated.","tokens_in":8169,"tokens_out":1523,"would_cite":false,"duration_ms":15864,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SpectMamba, a Mamba-based network with note-f0 decoding and confidence binary regularization, claims top accuracy on three singing melody extraction benchmarks while running twice as fast.","keywords":["singing melody extraction","Mamba","semi-supervised learning","note-f0 decoder","confidence binary regularization","state space model","fundamental frequency estimation","polyphonic music"],"falsifier":"Retrain SpectMamba with the note replication in Eq. (8) replaced by a constant attention mask (no note information) while keeping everything else identical: if overall accuracy stays within measurement noise on ADC2004 and MIREX 05, the note-f0 decoder is not load-bearing. Alternatively, perturb the note-to-bin assignment by half a bin and measure the change in overall accuracy.","tokens_in":7226,"feed_emoji":"🎵","tokens_out":5564,"duration_ms":46691,"temperature":0.7,"pith_summary":"The paper is trying to establish that singing melody extraction can be done more accurately and far more efficiently by replacing the transformer encoder with a bidirectional state-space (Mamba) model and by supervising pitch with musical notes instead of only raw frequencies. On three standard benchmarks it reports overall accuracy gains of 2.24%, 1.85%, and 1.27% over the S2Former transformer baseline, along with roughly twice the inference speed and one tenth the memory use. A second claim is that unlabeled audio can be exploited through confidence binary regularization, which forces weak and strong augmentations of the same clip to agree on the most confident pitch and note classes. If these claims hold, melody extraction becomes cheaper to deploy and less dependent on large labeled datasets.","feed_headline":"Mamba melody extractor tops transformer by 2.24% at 2x speed","feed_subtitle":"Semi-supervised model also cuts memory use to a tenth on three benchmarks.","key_machinery":"The load-bearing mechanism is the note-f0 decoder, which realizes a coarse-to-fine prior: it expands the note prediction $\\hat{P}_{\\mathrm{note}}$ by replicating each note class across all f0 classes that belong to that note, applies softmax to form attention weights, and multiplies them onto the raw f0 prediction before a final MLP: $\\hat{P}_{f0} = \\mathrm{MLP}(\\mathrm{Softmax}(\\hat{P}_{\\mathrm{note}}) \\cdot P_{f0})$. This makes the f0 estimate concentrate inside the predicted note's frequency band. Around it sit the SpectMamba encoder, which patches the spectrogram and processes it with a bidirectional state-space model in both time directions, and the confidence binary regularization module, which uses an EMA-tracked global top-k confidence to split each unlabeled prediction into positive and negative parts and enforce weak-strong consistency with cross-entropy.","core_discovery":"The paper's central claim is that a vision-Mamba encoder, a note-f0 decoder, and a confidence binary regularization module jointly yield the highest singing melody extraction accuracy among the models compared. The note-f0 decoder predicts a note sequence and a frequency sequence in parallel, then uses the note prediction as an attention mask on the f0 prediction so that the model first commits to the musical note and then refines the fundamental frequency inside that note's frequency range. The confidence binary regularization module divides each unlabeled sample's predictions into a positive part (top-k confident classes) and a negative part (the rest), with the boundary set by an EMA-updated global confidence, and enforces consistency between weakly and strongly augmented versions. On ADC2004, MIREX 05, and MedleyDB the paper reports higher overall accuracy than FTANet, TONet, and S2Former, and it attributes the accuracy gain to the note supervision and the efficiency gain to the linear-complexity Mamba encoder.","pith_inferences":["If the fixed note-to-bin mapping is why the note-f0 decoder works, the decoder should degrade gracefully as the frequency resolution per note decreases; one testable extension is to train variants with 30 or 120 bins per octave and compare overall accuracy.","The note-f0 decoder's masking idea could also serve as a differentiable pitch-range prior in source separation or instrument transcription, not just melody extraction.","A stronger claim implicit in the ablation is that note supervision and CBR contribute additively; a direct test would be to remove both simultaneously and see whether the drops compound or interact."],"forward_implications":["Melody extraction at 2x inference speed and 1/10 memory use makes real-time or on-device singing melody extraction practical at the same accuracy level as transformer models.","Note-level supervision gives a principled way to inject musical structure, so the approach could transfer to other pitch-tracking tasks where notes are defined.","Confidence binary regularization lets unlabeled music contribute to training, reducing the annotation burden for melody extraction systems.","The reported gains are consistent across three datasets, suggesting the method generalizes beyond a single corpus."],"supporting_citations":[{"why":"Supplies the bidirectional state-space (Mamba) encoder that gives linear-complexity processing of spectrogram patches.","marker":"[9]"},{"why":"Provides the confidence-based consistency idea that the CBR module adapts for unlabeled music.","marker":"[33]"},{"why":"S2Former is the transformer baseline that SpectMamba claims to beat on accuracy, speed, and memory.","marker":"[17]"},{"why":"TONet is the tone-octave supervised baseline; it motivates the note-based supervision and serves as a comparison.","marker":"[18]"},{"why":"Supplies the CFP input representation, non-melody frame convention, and the labeled data split practice.","marker":"[13]"},{"why":"MIR-1K provides labeled training tracks for the supervised part.","marker":"[10]"},{"why":"MedleyDB provides both labeled training tracks and one of the test sets.","marker":"[11]"},{"why":"FMA supplies the unlabeled tracks used to train the CBR module.","marker":"[12]"},{"why":"mir_eval computes the evaluation metrics (VR, VFA, RPA, RCA, OA) used to report results.","marker":"[14]"}],"fun_headline_variants":["SpectMamba: linear-complexity melody extraction beats transformer baselines","Semi-supervised Mamba melody extractor: 2x speed, 10x memory, higher accuracy","Note-aware decoding boosts Mamba-based semi-supervised melody extraction","Confidence binary regularization unlocks unlabeled data for Mamba melody extraction"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The note-f0 decoder assumes a fixed, known mapping from musical note classes to CFP frequency bins, and that replicating a note prediction across all bins belonging to that note is a valid attention prior; the paper does not specify this mapping.","fun_headline_variants_meta":{"raw":{"variants":["SpectMamba: linear-complexity melody extraction beats transformer baselines","Semi-supervised Mamba melody extractor: 2x speed, 10x memory, higher accuracy","Note-aware decoding boosts Mamba-based semi-supervised melody extraction","Confidence binary regularization unlocks unlabeled data for Mamba melody extraction"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000447,"raw_usage":{"total_tokens":2265,"prompt_tokens":963,"completion_tokens":1302,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":579,"completion_tokens_details":{"reasoning_tokens":1218}},"tokens_in":579,"tokens_out":1302,"duration_ms":11443,"temperature":1.0,"reasoning_tokens":1218,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:48:32.445388+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain SpectMamba with the note replication in Eq. (8) replaced by a constant attention mask (no note information) while keeping everything else identical: if overall accuracy stays within measurement noise on ADC2004 and MIREX 05, the note-f0 decoder is not load-bearing. Alternatively, perturb the note-to-bin assignment by half a bin and measure the change in overall accuracy.","supporting_citations":[{"cited_title":"A scalable sarse Transformer model for singing melody extraction,","cited_arxiv_id":null,"evidence_quote":"S2Former is the transformer baseline that SpectMamba claims to beat on accuracy, speed, and memory."},{"cited_title":"TONet: Tone-Octave Network for Singing Melody Extraction from Polyphonic Music,","cited_arxiv_id":null,"evidence_quote":"TONet is the tone-octave supervised baseline; it motivates the note-based supervision and serves as a comparison."},{"cited_title":"A streamlined encoder/decoder architecture for melody extraction,","cited_arxiv_id":null,"evidence_quote":"Supplies the CFP input representation, non-melody frame convention, and the labeled data split practice."},{"cited_title":"On the improvement of singing voice separation for monaural recordings using the MIR-1K dataset,","cited_arxiv_id":null,"evidence_quote":"MIR-1K provides labeled training tracks for the supervised part."},{"cited_title":"Medleydb: A multitrack dataset for annotation-intensive MIR research,","cited_arxiv_id":null,"evidence_quote":"MedleyDB provides both labeled training tracks and one of the test sets."},{"cited_title":"FMA: A Dataset for Music Analysis,","cited_arxiv_id":null,"evidence_quote":"FMA supplies the unlabeled tracks used to train the CBR module."},{"cited_title":"mir eval: A transparent implementation of common mir metrics,","cited_arxiv_id":null,"evidence_quote":"mir_eval computes the evaluation metrics (VR, VFA, RPA, RCA, OA) used to report results."}],"review_version":1}