{"id":"68dd2508-d168-4683-bd0f-b757a3f21fae","arxiv_id":"2501.12907","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"S-KEY learns 24 major and minor keys from audio with a self-supervised pseudo-labeling task and matches supervised key estimation accuracy without human labels.","lead":"The paper presents S-KEY, a self-supervised system that learns to identify both the key and the major or minor mode of a song from raw audio, without expert annotations. It reports accuracy on par with supervised state-of-the-art systems on four public music datasets, suggesting annotation-free key detection could scale to very large catalogs.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The benchmark claim rests on training on a proprietary 1M-song Deezer corpus and testing on public datasets, but the paper reports no overlap check between training and evaluation; if evaluation tracks are in the training corpus, the \"matches supervised SOTA\" numbers are invalid.","rationale":"The reader's weakest assumption is the correctness of qmax in Section II-C. I agree that qmax is a mechanism-level postulate, but the empirical benchmark could still be valid even if some pseudo-labels are wrong. The more decisive point is that the headline claim depends on separating the proprietary training corpus from the public evaluation corpora. The paper gives no evidence of such separation, and the proprietary nature of the training data makes the claim unauditable. The calibration step is transparent and inherited from STONE, so it weakens the 'no human annotation' wording but does not invalidate the benchmark numbers; data leakage, by contrast, would invalidate the central numerical claim. I therefore disagree with the reader's choice of weakest assumption, while agreeing that a conditional verdict is appropriate. The suggested fingerprint overlap test is a single, concrete check that would settle whether leakage is real, and if it is not, the central claim would stand as stated.","tokens_in":9273,"tokens_out":11907,"duration_ms":135721,"concrete_test":"Run an exact or approximate audio fingerprint match (e.g., Chromaprint/audfprint or decoded-audio hashes) between the Deezer training set and each evaluation dataset, and report the number and proportion of overlapping tracks. If any evaluation track appears in training, retrain S-KEY with those tracks removed and recompute Tables I and II; alternatively, train S-KEY on a public large corpus and check whether the reported MIREX scores are reproduced.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that S-KEY is the first SSL method matching supervised SOTA—rests entirely on the numbers in Tables I and II. Section III-A describes training S-KEY on a proprietary 1M-song Deezer corpus, while Section III-D lists evaluation sets (FMAKv2, GTZAN, GiantSteps, SWD) that are public recordings. The paper never reports any deduplication, fingerprinting, or overlap check between the training corpus and these evaluation sets. Because Deezer's catalog is large and includes commercial EDM (GiantSteps) and popular-music recordings (GTZAN), evaluation tracks may plausibly appear in training. Even without labels, SSL can benefit from seeing the same audio during training, so even modest overlap can inflate reported MIREX scores. This is more load-bearing than the qmax pseudo-label assumption: if qmax is occasionally wrong, the method still yields testable results; if training/eval overlap exists, the headline comparison with madmom is not valid evidence that SSL replaces supervision. Since the training corpus is not public, the reader cannot audit this from the paper. This missing support must be supplied before \"matches supervised SOTA\" can be accepted.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces S-KEY, a self-supervised method for 24-class major/minor key estimation from audio. It extends STONE's CPSD-based pitch-equivariant pretraining with an auxiliary mode-classification task: a pseudo-label nu is computed by comparing the CQT energy at the predicted root qmax(theta|x) with the energy a minor third below, and the network is trained with binary cross-entropy against this pseudo-label together with a batch-level prior that roughly half the songs are major. S-KEY is trained on 60k or 1M songs from the Deezer catalog and evaluated on FMAKv2, GTZAN, GiantSteps, and SWD. The authors report MIREX scores that match or exceed the supervised state of the art (madmom) and conclude that S-KEY is the first SSL method to match supervised deep learning for 24-key tonality estimation.","tokens_in":9562,"tokens_out":5589,"duration_ms":59991,"significance":"If the claims hold, this is a meaningful advance: it would show that large-scale unlabeled audio can substitute for expert annotation in full-fledged key and mode estimation, while keeping the same parameter budget as STONE. The paper's strengths include a release of training and inference code, evaluation on four public benchmarks with a breakdown into MIREX, KSEA, and mode accuracy, and the nu-STONE ablation, which usefully isolates the contribution of the learned component over the rule-based heuristic. The main caveats are evidentiary: the headline comparisons rest on single-run point estimates and on a proprietary training corpus with no reported overlap control, and the pseudo-labeling mechanism is partly self-referential. These concerns do not diminish the interest of the idea but do affect whether the central claim is currently established.","major_comments":[{"comment":"The training/evaluation separation is not audited. S-KEY is trained on 60k or 1M songs from the Deezer catalog (Section III-A) and evaluated on public recordings in FMAKv2, GTZAN, GiantSteps, and SWD (Section III-D), but the paper does not report any fingerprinting, deduplication, or overlap check between the training corpus and those evaluation sets. Because several evaluation sets consist of commercial or popular-music recordings that could plausibly appear in a large commercial catalog, even modest overlap could inflate the reported MIREX scores in Table II and would invalidate the claim that S-KEY matches supervised SOTA. This missing control is load-bearing for the headline result and must be supplied, or the claim should be restricted to a protocol that rules out overlap.","section":"III-A, III-D, Table II"},{"comment":"The pseudo-labeling premise is self-referential and is stated as a postulate rather than tested. Equation (5) defines nu(theta|x,c) through qmax(theta|x) from Equation (3), and Equation (6) trains mu_theta toward nu; hence the network is trained toward targets it generated itself. The paper acknowledges the cold-start issue and introduces the batch prior in Equation (7), but it does not quantify how often qmax is correct, how sensitive the final result is to errors in qmax, or how much the pseudo-labels change during training. The claim that a known key signature plus the CQT energy rule reaches 79.4% mode accuracy is also stated without a derivation or confidence interval. Please report (a) agreement between qmax and available key annotations on the evaluation sets, (b) an ablation using an oracle qmax, and (c) statistics on pseudo-label stability over training.","section":"II-C, Equations (3)-(6)"},{"comment":"All reported scores are single-run point estimates with no error bars, number of seeds, or significance tests. The central numerical claims rest on margins that are within plausible sampling noise: S-KEY (60k) trails madmom by 1.0 MIREX point on FMAKv2 (72.1 vs 73.1), S-KEY (1M) leads by 0.1 point (73.2 vs 73.1), and the GTZAN lead is 6.5 points but comes from a single evaluation run. Without variance estimates, the words \"matches\" and \"outperforms\" are not established for these small margins. Please report per-seed means and standard deviations, confidence intervals obtained by bootstrapping over songs, or per-song score distributions.","section":"IV-A, IV-B, Tables I and II"},{"comment":"The abstract's claim \"requiring no human annotation\" is not fully supported as written. Section III-B describes creating two synthetic samples, one in C major and one in A minor, to calibrate the two output channels; this requires human music-theoretic knowledge to specify the correct labels for those samples. Although this is different from annotating a large training corpus, it is a form of supervision, and the paper should state this explicitly and explain why the calibration step does not count as annotation for the purpose of the headline claim.","section":"III-B and Abstract"}],"minor_comments":[{"comment":"The text says the hyperparameters are lambda_BCE and lambda_avg, but the BCE term in Equation (8) is multiplied by lambda_S-KEY; please harmonize the notation.","section":"II-F, Equation (8)"},{"comment":"The word \"unlabled\" in \"analyzing an unlabled corpus of 1M songs\" should be \"unlabeled.\"","section":"IV-D"},{"comment":"The sentence \"24-STONE, as a first proposition toward self-supervised key signature and mode estimator, underperforms by 15%\" is vague; specify whether this is 15 percentage points on MIREX score and relative to which baseline.","section":"Section I"},{"comment":"The abstract says S-KEY matches the supervised SOTA on FMAKv2 and GTZAN, but on FMAKv2 the 60k model trails madmom by 1.0 point and the match comes from the 1M model; please state that the match is obtained after scaling to 1M songs.","section":"Abstract and Section I"},{"comment":"The note that the reported madmom GiantSteps score differs from the original paper (74.6%) is helpful, but it would be even more useful to state the exact madmom version and parameters used for all datasets, not only for GiantSteps.","section":"Table II note"}],"recommendation":"major_revision","confidential_remarks":"The proprietary 60k/1M Deezer training corpus makes independent verification impossible, which is why the missing overlap check is the most serious issue. I would not require the authors to release the corpus, but they should provide an overlap statistic or demonstrate that the public evaluation tracks have not been seen during training. The falsifiability of the central claim is also weakened by single-run results; the authors should at least provide bootstrap confidence intervals over songs and ideally multiple seeds. The paper's scope fits a signal-processing or MIR journal, and the methodological idea is sufficiently novel to warrant a revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nHere's my take on the S-KEY paper. The short version: it does what it says on the tin, and it is the first SSL method to match supervised SOTA on 24-key estimation. That is a genuine result. The method is a clever extension of their own STONE work: the auxiliary pseudo-labeling via transposition-invariant chroma features is novel, breaks the relative-key ambiguity, and is well motivated. The code is public, the evaluation spans four datasets with genre breakdowns, and the empirical story is coherent. I would send this to a serious referee.\n\nThe main soft spot is real: no overlap check between the proprietary 1M-song Deezer training corpus and the public evaluation sets (FMAKv2, GTZAN, GiantSteps, SWD). For an SSL paper making a headline claim about matching supervised SOTA, seeing the same audio during training can inflate scores even without labels. The paper neither reports deduplication nor discusses the risk. That is not necessarily fatal, but it must be addressed before the claim is fully trusted.\n\nSecond, the results are single-run, without error bars or multiple seeds. The 60k-to-1M improvement on FMAKv2 is only 1.1 points (72.1 to 73.2), and the gap to madmom at 1M is 0.1 point. That is inside plausible sampling noise. For a claim of parity, error bars matter.\n\nThird, the calibration step uses two labeled synthetic samples (C major and A minor). The paper describes the pipeline as “no human annotation,” which is slightly misleading. It is minor, but worth naming.\n\nFourth, the qmax postulate in Section II-C is stated as a postulate, and the paper does not directly verify it. The 79.4% accuracy quoted for the chroma-energy heuristic also appears without derivation or details. These are weaknesses, but the empirical results give indirect support.\n\nIn balance, the central argument holds up. The method is sound, the contribution is clear, and the limitations (24-key taxonomy, genre weaknesses) are acknowledged. I would want the overlap check and error bars before accepting, but I would not desk-reject it. Send it to review; it deserves referee time.","headline":"First SSL method to match supervised key estimation, but the headline claim needs an overlap check and error bars before it fully stands.","tokens_in":10078,"tokens_out":2456,"would_cite":false,"duration_ms":26130,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Self-supervised key detection now matches supervised models","keywords":["music key estimation","self-supervised learning","tonality","chroma features","pseudo-labels","music information retrieval","circle of fifths","mode classification"],"falsifier":"Train S-KEY on a corpus with known root annotations and compare qmax against the true root on songs where the CPSD loss is low; if qmax frequently selects the dominant or another scale degree instead of the tonic, the pseudo-labeling mechanism fails and the mode accuracy would drop, which would be visible as a large gap between S-KEY and a supervised model on mode-only accuracy.","tokens_in":9033,"feed_emoji":"🎵","tokens_out":2801,"duration_ms":28648,"temperature":0.7,"pith_summary":"This paper claims that a self-supervised model, S-KEY, can estimate the full 24-key taxonomy (12 key signatures times major/minor mode) from raw audio with accuracy on par with the best supervised system, and that it is the first SSL method to do so. The approach extends the STONE architecture with an auxiliary pretext task that produces pseudo-labels for mode by comparing chroma energy at the estimated root with energy a minor third below it. Trained on 60,000 songs, S-KEY reaches a MIREX score of 72.1% on FMAKv2; scaling to one million unlabeled songs raises the score to 73.2%, matching the supervised state of the art at 73.1%. If right, this means expert musical annotations are not necessary for high-accuracy tonality estimation, and large unlabeled music corpora can substitute for them.","feed_headline":"Self-supervised key detection matches supervised accuracy","feed_subtitle":"S-KEY learns 24 major/minor keys from unlabeled audio and hits 73.2% on FMAKv2, on par with supervised madmom.","key_machinery":"The load-bearing mechanism is a structured output with two components: a pitch-equivariant vector \\(\\lambda_{\\$\\theta$,A,c}\\) over the 12 pitch classes and a pitch-invariant vector \\(\\mu_{\\$\\theta$,A,c}\\) over the two modes. A cross-power spectral density (CPSD) loss with \\(\\omega=7\\) enforces equivariance to transposition along the circle of fifths, and a pseudo-labeling function \\(\\nu(\\$\\theta$|x,c)\\) compares chroma energy at \\(q_{\\max}\\) with energy a minor third below to generate a binary major/minor target. A batch-wise average loss \\(L_{\\text{avg}}\\) steadies training by assuming roughly half major and half minor songs per batch, preventing cold-start collapse.","core_discovery":"The central claim is that a self-supervised pretext task can replace human annotation for full 24-key tonality estimation. S-KEY combines the existing CPSD-based key-signature equivariance loss with a pseudo-labeling rule that decides major versus minor by comparing the pitch-class energy at the predicted root (qmax) against the energy three semitones below it, i.e., the root of the relative minor. Jointly optimizing key-signature equivariance and this pseudo-labeled mode objective yields a model that matches the supervised madmom network on FMAKv2, GTZAN, GiantSteps, and Schubert Winterreise, while using the same parameter budget as STONE and no labeled data.","pith_inferences":["The same pseudo-labeling principle could be extended to other musical modes (e.g., blues or modal harmony) by choosing different interval comparisons below the root estimate, as the paper itself suggests.","Because S-KEY extracts a root estimate qmax on short segments, the architecture could be adapted to track key modulations over time within a single piece, a task the paper does not evaluate.","The reliance on a roughly balanced major/minor batch prior suggests the method might degrade on corpora with very skewed mode distributions, which could be tested by training on genre-specific collections.","The circle-of-fifths structure visible in the learned embeddings hints that the pretext task implicitly organizes harmonic relations; this could be probed as a general unsupervised representation for harmonic analysis beyond key classification."],"forward_implications":["Self-supervised pretraining on unlabeled music can replace supervised training for tonality estimation, removing the need for expert key annotations.","Scaling the training corpus from 60k to 1M songs improves S-KEY on three of four evaluation datasets, showing that SSL benefits from larger unlabeled corpora.","The pseudo-labeling rule transfers to new datasets and genres without fine-tuning, and S-KEY outperforms supervised baselines on metal, jazz, and reggae.","Since S-KEY uses the same parameter budget as STONE, the additional mode discrimination comes from the pretext task design, not from added capacity."],"supporting_citations":[{"why":"Provides the STONE architecture, CPSD loss, and the 24-STONE baseline that S-KEY extends and compares against.","marker":"[16]"},{"why":"Defines the supervised state-of-the-art model (madmom) that S-KEY aims to match or outperform.","marker":"[17]"},{"why":"Supplies the template-matching Krumhansl baseline with no learning, used as a non-ML comparison.","marker":"[18]"},{"why":"Introduces the FMAKv2 dataset, the primary benchmark for MIREX score comparison.","marker":"[20]"},{"why":"Provides the GTZAN dataset with unique-key annotations used for cross-dataset evaluation.","marker":"[22]"},{"why":"Supplies the GiantSteps EDM key dataset used for evaluation.","marker":"[23]"},{"why":"Contributes the Schubert Winterreise dataset of classical pieces used for evaluation.","marker":"[24]"},{"why":"Defines the MIREX score metric via mir_eval that weights predictions by tonal proximity.","marker":"[25]"}],"fun_headline_variants":["Self-supervised key detection rivals supervised training","S-KEY learns major/minor keys without human annotations","No labels, same accuracy: self-supervised key finding","Self-supervised model matches supervised tonality accuracy","Unlabeled audio: self-supervised key detection on par with labels"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that when the CPSD loss is low and a song is in a major key, the most-active pitch class qmax is the song's actual root note; if qmax points to the wrong scale degree, the pseudo-label for major versus minor is wrong.","fun_headline_variants_meta":{"raw":{"variants":["Self-supervised key detection rivals supervised training","S-KEY learns major/minor keys without human annotations","No labels, same accuracy: self-supervised key finding","Self-supervised model matches supervised tonality accuracy","Unlabeled audio: self-supervised key detection on par with labels"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000178,"raw_usage":{"total_tokens":1241,"prompt_tokens":833,"completion_tokens":408,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":449,"completion_tokens_details":{"reasoning_tokens":330}},"tokens_in":449,"tokens_out":408,"duration_ms":4922,"temperature":1.0,"reasoning_tokens":330,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T16:38:09.850706+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train S-KEY on a corpus with known root annotations and compare qmax against the true root on songs where the CPSD loss is low; if qmax frequently selects the dominant or another scale degree instead of the tonic, the pseudo-labeling mechanism fails and the mode accuracy would drop, which would be visible as a large gap between S-KEY and a supervised model on mode-only accuracy.","supporting_citations":[{"cited_title":"Stone: Self-supervised tonality estimator,","cited_arxiv_id":null,"evidence_quote":"Provides the STONE architecture, CPSD loss, and the 24-STONE baseline that S-KEY extends and compares against."},{"cited_title":"Genre-agnostic key clas- sification with convolutional neural networks,","cited_arxiv_id":null,"evidence_quote":"Defines the supervised state-of-the-art model (madmom) that S-KEY aims to match or outperform."},{"cited_title":"Krumhansl, Cognitive foundations of musical pitch , Oxford University Press, 2001","cited_arxiv_id":null,"evidence_quote":"Supplies the template-matching Krumhansl baseline with no learning, used as a non-ML comparison."},{"cited_title":"Fmak: A dataset of key and mode annotations for the free music archive–extended abstract,","cited_arxiv_id":null,"evidence_quote":"Introduces the FMAKv2 dataset, the primary benchmark for MIREX score comparison."},{"cited_title":"Genre-specific key profiles,","cited_arxiv_id":null,"evidence_quote":"Provides the GTZAN dataset with unique-key annotations used for cross-dataset evaluation."},{"cited_title":"Giantsteps key dataset,","cited_arxiv_id":null,"evidence_quote":"Supplies the GiantSteps EDM key dataset used for evaluation."},{"cited_title":"Schubert winterreise dataset: A multimodal scenario for music analysis,","cited_arxiv_id":null,"evidence_quote":"Contributes the Schubert Winterreise dataset of classical pieces used for evaluation."},{"cited_title":"mir eval: A Transparent Implementation of Common MIR Metrics.,","cited_arxiv_id":null,"evidence_quote":"Defines the MIREX score metric via mir_eval that weights predictions by tonal proximity."}],"review_version":1}