Pith. sign in

REVIEW 5 major objections 5 minor 40 references

The paper claims that training a chord-recognition student on pseudo-labels from a pre-trained teacher, then fine-tuning on ground truth with selective knowledge distillation, lets the student surpass both the teacher and a fully supervised

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 21:31 UTC pith:GTDHTVB5

load-bearing objection A practical two-stage pseudo-labeling + KD pipeline for chord recognition with a plausible but under-supported claim that the student beats the teacher; the missing dedup check against the test set is the real worry. the 5 major comments →

arxiv 2602.19778 v5 pith:GTDHTVB5 submitted 2026-02-23 cs.SD cs.IRcs.LGcs.MM

Enhancing Automatic Chord Recognition via Pseudo-Labeling and Knowledge Distillation

Nghia Phan , Rong Jin , Gang Liu , Xiao Dong This is my paper
classification cs.SD cs.IRcs.LGcs.MM
keywords automatic chord recognitionpseudo-labelingknowledge distillationsemi-supervised learningcontinual learningrare chord qualitymusic information retrievalselective knowledge distillation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to show that automatic chord recognition does not need large amounts of hand-aligned labels if a good pre-trained model is available. To establish this, the authors build a two-stage pipeline: first, a pre-trained teacher (the BTC model) pseudo-labels over 1,000 hours of unlabeled audio, and a student is trained solely on those labels; second, the student is fine-tuned on ground-truth labels while selective knowledge distillation stabilizes the adaptation. The main result is that the best student surpasses both the fully supervised baseline and the teacher itself on all seven standard evaluation metrics, with the largest gains on rare chord qualities (overall average chord quality accuracy rises from 29.0% to 39.5%, while Dim7 recognition goes from 0% to 45.6%). The stakes: if this holds, chord recognition models can be trained on a fraction of the manually curated labels, and open-weight models can serve as practical teachers for specialized audio understanding tasks.

Core claim

On its own terms, the paper claims that a knowledge-distilled student can outperform its own teacher when the teacher is used only to generate pseudo-labels for unlabeled audio, and the result transfers across architectures. Stage 1 gives the BTC student over 99% of the teacher's score and a compact transformer student (2E1D) about 97%; Stage 2, with ground-truth fine-tuning and selective knowledge distillation, takes the BTC student to 83.03% root accuracy versus the teacher's 81.95% and the supervised baseline's 81.52%, and surpasses them on every one of the seven metrics. The most striking numbers are on rare chord qualities: Dim7 accuracy jumps from 0% (supervised) to 45.6%, and the clas

What carries the argument

The load-bearing mechanism is the two-stage pipeline with selective knowledge distillation. Stage 1 uses a pre-trained teacher (BTC) to produce frame-wise pseudo-labels (teacher argmax, no confidence filtering) for over 1,000 hours of diverse unlabeled audio, and trains a student on those labels until convergence; three pooled corpora are chosen so that the 12 chord roots are nearly uniform (98.4% uniformity), removing the need for pitch-shifting augmentation. Stage 2 initializes from the Stage-1 student and continues training on ground-truth labels while adding a KD loss to the cross-entropy, weighted by an asymmetric confidence function that discards low-confidence teacher predictions (bel

Load-bearing premise

All headline improvements are measured against a teacher model whose training data is not disclosed; if the teacher saw the test songs, or if the 1,000+ hours of 'unlabeled' audio include the test tracks, the student's lead over the teacher could be an artifact of data leakage.

What would settle it

Check whether any of the 120 test songs appear in the pseudo-label pool or in the teacher's training corpus. If the student's margin over the teacher on the seven metrics shrinks or inverts after removing such tracks from both the unlabeled pool and the teacher's evaluation, the leakage explanation is confirmed; if it persists on a disjoint subset, the transfer claim survives.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Labeled-data requirements drop: with only 210 labeled songs (50% of the training set), the BTC student already beats the teacher on most metrics, and with the full 420 songs it beats the teacher on all seven.
  • Rare chords become realistic: chord qualities effectively absent from supervised training (Dim7 at 0%) reach 45.6% under the pipeline, and ACQA improves by 10.5 points.
  • KD as a regularizer transfers across architecture: the wider, purely transformer 2E1D student needs stronger KD (α=0.5) and still improves over its supervised baseline, while the deeper BTC peaks at α=0.3.
  • Noise robustness: when annotations are misaligned, without KD both models degrade severely (the wider model collapses), while with selective KD they recover most of the gap.
  • Natural root coverage replaces pitch shifting: pseudo-labels from diverse unlabeled corpora produce near-uniform chord-root coverage (98.4% uniformity), so the pipeline avoids pitch-shift artifacts and augmentation cost.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The recipe generalizes beyond chords: any MIR task with expensive labels (key estimation, beat tracking) could borrow a strong open-weight teacher, pseudo-label a large unlabeled pool, then fine-tune with selective KD; the paper hints at this but does not test it.
  • The 'student surpasses teacher' result is sensitive to the teacher's own training data: if the teacher was trained on songs that appear in the 120-song test set, part of the margin is leakage rather than transfer; a clean check is to rerun with a fully documented teacher.
  • Selective KD's asymmetric weighting implies that overconfident teacher predictions are often wrong for rare chords; this could be tested by ablating the overconfidence cap and measuring per-class ACQA.
  • Because the two stages are decoupled, the pipeline offers a practical timeline: deploy with pseudo-label training first, then incorporate human labels as they arrive, making it suitable for continual label acquisition in production systems.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes a two-stage pipeline for automatic chord recognition (ACR). Stage 1 uses a pre-trained BTC model as a fixed teacher to pseudo-label more than 1,000 hours of unlabeled audio (FMA, MAESTRO, DALI) and trains student models (BTC and a new compact 2E1D) on those pseudo-labels. Stage 2 continually fine-tunes the student on ground-truth labels while applying a selective knowledge-distillation regularizer from the same teacher. Experiments on a 420/60/120 split of Isophonics/McGill/RWC/USPop report that, after Stage 2, the BTC student surpasses both the supervised-learning baseline and the teacher on all seven mir_eval metrics, with the largest gains on rare chord qualities such as Dim, Dim7, and Aug. The paper also provides an ablation showing KD protects against noisy-label fine-tuning.

Significance. If the reported results hold under controlled conditions, the paper makes a useful practical contribution: it shows that pseudo-label pretraining on large unlabeled corpora can reduce labeled-data requirements in ACR and improve rare-chord recognition, and that KD can serve as a forgetting regularizer during continual fine-tuning. The paper is explicit about using a fixed open-weight teacher, which is a realistic setting, and it evaluates with standard mir_eval metrics. The cross-architecture validation with 2E1D and the open-sourced web application are additional strengths. However, the empirical claims currently rest on a single split, a single training run, and an unverified separation between the unlabeled and test corpora, which puts the central 'student surpasses teacher' claim at risk.

major comments (5)
  1. [§4.1, §5.3] No deduplication is reported between the unlabeled corpora (especially DALI) and the 120-song test set. DALI consists of YouTube audio for full-length commercial recordings, and the Isophonics/McGill/RWC/USPop test sets contain commercial pop/rock tracks, so a test track may well appear in the unlabeled pool. In Stage 1 the student is trained on teacher pseudo-labels for those exact test tracks, giving it transductive knowledge the frozen teacher never receives. This alone could explain part of the 1.1–3.2 point margin in Table 3. The paper must either demonstrate deduplication by artist/title metadata and/or audio fingerprinting, or retrain the pipeline after removing any overlapping tracks.
  2. [§5.1, §5.3, Tables 1 and 3] All numbers come from one 420/60/120 split and one training run. The key differences are small: BTC(CL) full root 83.03 vs teacher 81.95 and vs BTC(SL) 81.52, and several metrics differ by ~1–2 points. Without confidence intervals, multiple seeds, or statistical significance tests, these differences are within plausible run-to-run and split-to-split noise, especially with only 120 test songs. The central claim that the student 'surpasses' the teacher requires proper uncertainty quantification.
  3. [§5.3, Table 3] The comparison to Bortolozzo et al. [5] and Li et al. [7] is not controlled. The table reports metrics from the original papers evaluated on different test sets, while the text implies a direct comparison ('Prior work... results weaker'). Re-running these methods under the same test protocol, or clearly labeling the rows as 'as reported in the original paper, not directly comparable,' is necessary before drawing conclusions about pseudo-label quality.
  4. [§4.2, §5.3] The teacher is treated as a fixed external baseline, but its pre-training data are never stated. If the BTC teacher was trained on some of the labeled songs used here, the teacher's pseudo-labels on those songs (if they also appear in the unlabeled pool) would carry additional label information, further confounding the 'student surpasses teacher' claim. The authors should identify the teacher checkpoint and its training corpus, or explicitly state that this information is unavailable and explain how the interpretation changes.
  5. [§3.2.4, §4.2, Table 2] The selective-KD thresholds θmin, θmax, K, the KD weight α, and temperature τ are all set empirically. The ablation in Table 2 varies only α and only under one noisy-label condition; the sensitivity of the final Stage-2 results to τ, θmin, θmax, and K is not reported. Since these hyperparameters directly control the regularizer that the paper claims is essential, the default choices need sensitivity analysis or a principled justification.
minor comments (5)
  1. [§4.2] The text says 'We reserve 10% of the pseudo-labeled data for validation and 10% as a held-out test set,' but all reported metrics in Section 5 are evaluated on the 120-song labeled test set. The role of the pseudo-labeled held-out set needs clarification.
  2. [§3.2.4, Eq. (8)] For K=1, w(c) approaches 0 as c→1; with the chosen K=0.8 this is nonnegative, but the nonnegativity constraint is not stated. Add a brief constraint or remark.
  3. [Figure 3 caption] The caption contains an incomplete phrase 'Data C:maj' and is otherwise hard to parse. Please rewrite for clarity.
  4. [Table 3 footnote] The footnote 'We report only the metrics available in the original papers with our test set' is self-contradictory if the original papers used different test sets. Please state precisely how these numbers were obtained.
  5. [References] Reference [19] is formatted incorrectly ('L. S. Helong Zhou'); the author list should match the original paper. Also, several arXiv references lack version/accession dates.

Circularity Check

0 steps flagged

No significant circularity: the central result is evaluated against held-out ground-truth labels and is not forced by construction.

full rationale

The paper's central claim—that after Stage 2 the BTC student surpasses the pre-trained teacher on all seven mir_eval metrics (Table 3)—is evaluated against ground-truth chord labels on a held-out 120-song test set, not against the teacher's outputs or any quantity the training procedure forces. Stage 1 pseudo-labels are generated by the teacher, but the Stage 1 result is itself measured on ground truth and is explicitly reported as approaching the teacher (e.g., 'the best BTC student reaches over 99% of teacher performance'), not as identical to it; no equation defines the student's test metric as the teacher's output. Stage 2 fine-tunes on ground-truth labels (Eq. 4) with a KD regularizer, and the KD term is a training loss, not the evaluation target. Table 1's 'frame-wise vs. teacher' columns are explicitly labeled as agreement with teacher predictions and are not used as evidence of external performance. The paper also does not cite its own authors for any load-bearing claim, and no uniqueness theorem or imported ansatz is used. The remaining concerns—possible overlap between the unlabeled corpora (e.g., DALI YouTube tracks) and the labeled test set, and the absence of reported deduplication—are data-hygiene/external-validity issues, not definitional circularity, because the reported metric values are not forced by construction or by a self-referential definition.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

No new physical entities are introduced. The main free parameters are distillation and selective-KD hyperparameters selected from the teacher's confidence distribution and ablations. The load-bearing assumptions are the teacher's independence from the test set and the transferability of unfiltered pseudo-labels.

free parameters (4)
  • KD weight α = 0.3 (finals); 0.5 for 2E1D in noisy-label ablation
    Controls the balance between teacher soft targets and ground-truth hard labels; chosen from the ablation in Table 2 (§4.2).
  • KD temperature τ = 3.0
    Explicitly 'empirically selected as the optimal value in all settings' (§4.2).
  • Selective KD thresholds θmin/θmax and factor K = 0.1 / 0.9 / 0.8
    Set as 'robust defaults based on the observed teacher-confidence distribution' (§3.2.4).
  • Inference smoothing kernel width k and overlap ratio r = not reported
    The smoothing kernel and sliding-window overlap are defined via σ=k/6 and r, but the chosen values are not given (§3.2.5); they affect final frame-level metrics.
axioms (4)
  • domain assumption The pre-trained BTC teacher has not been trained on the labeled test songs.
    §4.1 defines the test set from Isophonics/McGill/RWC/USPop; §3.2 uses the teacher from [1] without stating its training corpus. If overlap exists, 'surpasses teacher' is contaminated.
  • domain assumption Frame-wise argmax pseudo-labels without confidence filtering are useful training targets that transfer to ground-truth chord recognition.
    Eqs. (1)-(2) convert 100% of teacher predictions into hard labels; if the teacher's outputs are too noisy or vocabulary-incompatible, the Stage-1 result reduces to simple mimicry.
  • domain assumption The large unlabeled corpora's near-uniform key distribution substitutes for pitch-shifting augmentation.
    §3.2.1 argues the natural root coverage in FMA/DALI/MAESTRO (98.4% uniformity) removes the need for augmentation; this is an empirical property of the chosen corpora, not a guaranteed general fact.
  • standard math Standard softmax/KL-divergence arithmetic and backpropagation assumptions hold.
    Eqs. (3)-(7) rely on conventional softmax, KL divergence, and gradient rules; no non-standard math is used.

pith-pipeline@v1.3.0-alltime-deepseek · 13403 in / 11251 out tokens · 107529 ms · 2026-08-02T21:31:15.408579+00:00 · methodology

0 comments
read the original abstract

Automatic Chord Recognition (ACR) is constrained by the scarcity of aligned chord annotations, which are costly to acquire. At the same time, open-weight pre-trained models are more accessible than their proprietary training data. In this work, we present a two-stage training pipeline that leverages pre-trained models together with unlabeled audio. The proposed method decouples training into two stages. In the first stage, we use the pre-trained BTC model as a teacher to generate pseudo-labels for over 1,000 hours of diverse unlabeled audio and train a student model solely on these pseudo-labels. In the second stage, the student is continually trained on ground-truth labels as they become available. To prevent catastrophic forgetting of the representations learned in the first stage, we apply selective knowledge distillation (KD) from the teacher as a regularizer. In our experiments, two models (BTC, 2E1D) were used as students. In Stage 1, using only pseudo-labels, the BTC student achieves about 99% of the teacher's performance, while the 2E1D model achieves about 97% of the teacher's performance across seven standard mir_eval metrics. After continual training with labeled data in Stage 2, the resulting BTC student model consistently surpasses both the traditional supervised learning baseline and the original pre-trained teacher model across all metrics. The resulting 2E1D student model also outperforms the supervised baseline and approaches teacher-level performance, with both models demonstrating substantial gains on rare chord qualities.

Figures

Figures reproduced from arXiv: 2602.19778 by Gang Liu, Nghia Phan, Rong Jin, Xiao Dong.

Figure 1
Figure 1. Figure 1: Duration-weighted chord root distribution across pseudo￾labeled datasets. The dashed line indicates uniform distribution (8.33%). Pitch classes are well-represented with 98.4% uniformity. 3.2.1. Data Augmentation vs. Natural Chord Root Coverage Supervised chord recognition systems typically require pitch-shifting augmentation to address severe chord root imbalance in labeled datasets. Studies show that wid… view at source ↗
Figure 2
Figure 2. Figure 2: Constant-Q Transform (CQT) comparison revealing pitch-shifting artifacts. Top row: original and pitch-shifted (−5 semitones) spectrograms. Bottom row: artifact intensity maps for ±5 semitones, computed by realigning shifted CQT bins to compensate for the intended frequency shift. Annotations indicate spectral spreading (low-frequency energy diffusion) and spectral artifacts (high-frequency noise) introduce… view at source ↗
Figure 3
Figure 3. Figure 3: Illustration of the proposed two-stage training pipeline. Details of audio data are described in Section 4.1. Note: The resulting Student model CL from stage 2 can be continually trained when additional labeled data is available. A pre-trained model is used as a teacher to infer frame-wise pseudo￾labels (PL) on the unlabeled audio for each track, producing paired data (Spectrogram, PL). A student model is … view at source ↗
Figure 4
Figure 4. Figure 4: , 2E1D adopts a wider design: (1) a frequency encoder that groups CQT bins into spectral clusters and applies self-attention 2Model: https://github.com/ptnghia-j/ChordMini Normalization Multi-head Self-attention Q = K Feed forward 𝐍𝐨𝐫𝐦𝐚𝐥𝐢𝐳𝐚𝐭𝐢𝐨𝐧 Linear Layer × 𝐿 Multi-head Self-attention Q K V Feed forward Block Dropout Layer Normalization Dropout Linear Layer Layer Normalization × 𝐿 Multi-head Self-attenti… view at source ↗
Figure 5
Figure 5. Figure 5: Evaluation loss of BTC model during continual training with different KD weights. Higher α values provide stronger regu￾larization, mitigating performance degradation from noisy labels. Model α Root Thirds Triads 7ths Tetrads Majmin MIREX Seg 2E1D 0 52.47 50.70 49.88 42.52 41.00 51.12 50.99 66.72 0.1 55.85 54.16 53.41 47.68 45.78 54.88 54.07 67.70 0.3 66.05 63.89 62.79 55.78 53.39 64.70 63.87 73.32 0.5 74.… view at source ↗
Figure 6
Figure 6. Figure 6: visualizes this effect. Major (45.5%) and Minor (22.3%) chords dominate the training distribution, while rare qualities (Dim: 1.8%, Dim7: 0.4%, Aug: 0.3%) constitute less than 3% combined. The SL baseline achieves 0% on Dim7 due to insufficient examples, while our pipeline leverages teacher pretraining to achieve 45.6%, a shift from failure to reasonable recognition. Comparison to Prior Work [PITH_FULL_IM… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 8 linked inside Pith

  1. [1]

    INTRODUCTION Automatic Chord Recognition (ACR) is a fundamental task in Mu- sic Information Retrieval (MIR) that aims to identify the harmonic content of audio recordings by outputting a sequence of chord la- bels. While large-scale labeled datasets are readily available for many machine learning domains, ACR faces significant data con- straints: publicly...

    2026

  2. [2]

    RELATED WORK Pseudo-labeling, where a trained model generates labels for un- labeled data, has become a cornerstone of semi-supervised learn- ing [10]. The Noisy Student framework [ 11] demonstrated that iteratively training larger student models on pseudo-labeled data with noise injection can surpass teacher performance, establishing a paradigm for lever...

    Pith/arXiv arXiv 2026

  3. [3]

    METHODOLOGY 3.1. Problem Formulation Let Dl ={(x i, yi)}Nl i=1 denote a small labeled dataset where xi ∈R T×F represents time–frequency features (e.g., Constant- Q Transform) with T frames and F frequency bins, and yi ∈ {1,2, . . . , C}T are frame-wise chord labels over a vocabulary of size C. Let Du ={x j}Nu j=1 represent large-scale unlabeled datasets w...

  4. [4]

    50%” and “full

    EXPERIMENTS 4.1. Datasets and Preprocessing Unlabeled Datasets.We use three large-scale unlabeled datasets for pseudo-label generation, totaling over 1,000 hours of audio. The Free Music Archive [27] (DF M A) provides over 100,000 short- form tracks (∼30s) with extensive genre diversity. The MAESTRO dataset [28] ( DM AEST RO) contributes over 200 hours of...

    2048

  5. [5]

    All reported metrics are evaluated on the held-out test set (120 songs) from the clean labeled dataset described in Section 4.1

    RESULTS In this section, we present results for our two-stage training method across the metrics described in Section 4.3, including an ablation study on the KD regularization against noisy labels. All reported metrics are evaluated on the held-out test set (120 songs) from the clean labeled dataset described in Section 4.1. 5.1. Stage 1: Training with ps...

  6. [6]

    We show that students trained solely on pseudo-labels can approach teacher-level performance across seven mir_eval metrics

    CONCLUSION Since model weights are often more readily available than pro- prietary training data, we present a practical training strategy for the ACR problem that leverages open-weight pre-trained models when high-quality labels are scarce. We show that students trained solely on pseudo-labels can approach teacher-level performance across seven mir_eval ...

  7. [7]

    A Bi- directional Transformer for Musical Chord Recognition,

    J. Park, K. Choi, S. Jeon, D. Kim, and J. Park, “A Bi- directional Transformer for Musical Chord Recognition,” in Proc. Int. Soc. Music Inf. Retrieval Conf. (ISMIR), Delft, The Netherlands, 2019, pp. 620–627

    2019

  8. [8]

    Four Timely Insights on Automatic Chord Estimation,

    E. J. Humphrey and J. P. Bello, “Four Timely Insights on Automatic Chord Estimation,” inProc. Int. Soc. Music Inf. Retrieval Conf. (ISMIR), Málaga, Spain, Oct. 2015, pp. 673– 679

    2015

  9. [9]

    Towards Automatic Extraction of Harmony Infor- mation from Music Signals,

    C. Harte, “Towards Automatic Extraction of Harmony Infor- mation from Music Signals,” Ph.D. dissertation, Queen Mary, University of London, Aug. 2010

    2010

  10. [10]

    20 Years of Automatic Chord Recognition from Audio,

    J. Pauwels, K. O’Hanlon, E. Gómez, and M. B. Sandler, “20 Years of Automatic Chord Recognition from Audio,” in Proc. Int. Soc. Music Inf. Retrieval Conf. (ISMIR), Delft, The Netherlands, Nov. 2019, pp. 54–63. 7

    2019

  11. [11]

    Improving the Classification of Rare Chords With Unlabeled Data,

    M. Bortolozzo, R. Schramm, and C. R. Jung, “Improving the Classification of Rare Chords With Unlabeled Data,” inIEEE Int. Conf. Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2021, pp. 3390–3394

    2021

  12. [12]

    Catastrophic Forgetting in Connectionist Net- works,

    R. M. French, “Catastrophic Forgetting in Connectionist Net- works,”Trends in Cognitive Sciences, vol. 3, no. 4, pp. 128– 135, Apr. 1999

    1999

  13. [13]

    Large-V ocabulary Chord Recognition Based on Contrastive Learning and Noisy Stu- dent,

    C. Li, J. Jiang, Y . Li, and L. Tian, “Large-V ocabulary Chord Recognition Based on Contrastive Learning and Noisy Stu- dent,”IEEE Transactions on Consumer Electronics, vol. 71, no. 2, pp. 3695–3706, Jul. 2024, early access published July 2024; print issue May 2025

    2024

  14. [14]

    Attention Is All You Need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention Is All You Need,” inAdvances in Neural Information Processing Systems (NeurIPS), vol. 30, 2017, pp. 5998–6008

    2017

  15. [15]

    mir_eval: A transparent imple- mentation of common MIR metrics,

    C. Raffel, B. McFee, E. J. Humphrey, J. Salamon, O. Nieto, D. Liang, and D. P. W. Ellis, “mir_eval: A transparent imple- mentation of common MIR metrics,” inProc. Int. Soc. Music Inf. Retrieval Conf. (ISMIR), 2014, pp. 367–372

    2014

  16. [16]

    Pseudo-label: The simple and efficient semi- supervised learning method for deep neural networks,

    D.-H. Lee, “Pseudo-label: The simple and efficient semi- supervised learning method for deep neural networks,” in ICML Workshop on Challenges in Representation Learning, 2013

    2013

  17. [17]

    Self-training with Noisy Student improves ImageNet classification,

    Q. Xie, M.-T. Luong, E. Hovy, and Q. V . Le, “Self-training with Noisy Student improves ImageNet classification,” in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recog- nition (CVPR), 2020, pp. 10 687–10 698

    2020

  18. [18]

    FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence,

    K. Sohn, D. Berthelot, N. Carlini, Z. Zhang, H. Zhang, C. A. Raffel, E. D. Cubuk, A. Kurakin, and C.-L. Li, “FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence,” inAdvances in Neural Information Processing Systems (NeurIPS), vol. 33, 2020, pp. 596–608

    2020

  19. [19]

    Mean Teachers are Better Role Models: Weight-Averaged Consistency Targets Improve Semi-Supervised Learning Results,

    A. Tarvainen and H. Valpola, “Mean Teachers are Better Role Models: Weight-Averaged Consistency Targets Improve Semi-Supervised Learning Results,” inAdvances in Neural Information Processing Systems (NeurIPS), vol. 30, 2017, pp. 1195–1204

    2017

  20. [20]

    Meta Pseudo La- bels,

    H. Pham, Z. Dai, Q. Xie, and Q. V . Le, “Meta Pseudo La- bels,” inProc. IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR), 2021, pp. 11 557–11 568

    2021

  21. [21]

    Semi-Supervised Piano Transcrip- tion Using Pseudo-Labeling Techniques,

    S. Strahl and M. Müller, “Semi-Supervised Piano Transcrip- tion Using Pseudo-Labeling Techniques,” inProc. Int. Soc. Music Inf. Retrieval Conf. (ISMIR), 2024

    2024

  22. [22]

    Scaling up Musical Information Retrieval Training with Semi-supervised Learning,

    Y .-N. Hung, J.-C. Wang, M. Won, and D. Le, “Scaling up Musical Information Retrieval Training with Semi-supervised Learning,”arXiv preprint arXiv:2310.01353, 2023. [Online]. Available: https://arxiv.org/abs/2310.01353

    Pith/arXiv arXiv 2023

  23. [23]

    Continuous Soft Pseudo-Labeling in ASR,

    T. Likhomanenko, R. Collobert, N. Jaitly, and S. Bengio, “Continuous Soft Pseudo-Labeling in ASR,” arXiv preprint arXiv:2211.06007, Nov. 2022, version v2

    Pith/arXiv arXiv 2022

  24. [24]

    Distilling the Knowledge in a Neural Network,

    G. Hinton, O. Vinyals, and J. Dean, “Distilling the Knowledge in a Neural Network,” arXiv preprint arXiv:1503.02531, Mar. 2015

    Pith/arXiv arXiv 2015

  25. [25]

    Rethinking soft labels for knowledge distillation: A bias–variance tradeoff perspective,

    L. S. Helong Zhou, “Rethinking soft labels for knowledge distillation: A bias–variance tradeoff perspective,”Proceed- ings of International Conference on Learning Representations (ICLR), 2021

    2021

  26. [26]

    Learning From Biased Soft Labels,

    H. Yuan, N. Xu, Y . Shi, X. Geng, and Y . Rui, “Learning From Biased Soft Labels,” arXiv preprint arXiv:2302.08155, Feb. 2023

    Pith/arXiv arXiv 2023

  27. [27]

    A Note on Knowledge Distillation Loss Function for Object Classification,

    D. Chen, “A Note on Knowledge Distillation Loss Function for Object Classification,” arXiv preprint arXiv:2109.06458, Sep. 2023, version v3

    Pith/arXiv arXiv 2023

  28. [28]

    A com- prehensive survey on knowledge distillation,

    A. M. Mansourian, R. Ahmadi, M. Ghafouri, A. M. Babaei, E. B. Golezani, Z. yasamani ghamchi, V . Ramezanian, A. Taherian, K. Dinashi, A. Miri, and S. Kasaei, “A com- prehensive survey on knowledge distillation,”Transactions on Machine Learning Research, 2025

    2025

  29. [29]

    Learning without Forgetting,

    Z. Li and D. Hoiem, “Learning without Forgetting,”IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 12, pp. 2935– 2947, Dec. 2018

    2018

  30. [30]

    Three Scenarios for Continual Learning,

    G. M. van de Ven and A. S. Tolias, “Three Scenarios for Continual Learning,”arXiv preprint arXiv:1904.07734, 2019

    Pith/arXiv arXiv 1904

  31. [31]

    A Review of Time-Scale Mod- ification of Music Signals,

    J. Driedger and M. Müller, “A Review of Time-Scale Mod- ification of Music Signals,”Applied Sciences, vol. 6, no. 2, p. 57, 2016

    2016

  32. [32]

    Automatic Chord Recognition with Fully Con- volutional Neural Networks,

    H. H. Fard, “Automatic Chord Recognition with Fully Con- volutional Neural Networks,” Master’s thesis, Technische Universität Berlin, Sep. 2020

    2020

  33. [33]

    FMA: A Dataset for Music Analysis,

    M. Defferrard, K. Benzi, P. Vandergheynst, and X. Bresson, “FMA: A Dataset for Music Analysis,” inProc. Int. Soc. Music Inf. Retrieval Conf. (ISMIR), 2017, pp. 316–323. [Online]. Available: https://arxiv.org/abs/1612.01840

    Pith/arXiv arXiv 2017

  34. [34]

    En- abling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset,

    C. Hawthorne, A. Stasyuk, A. Roberts, I. Simon, C.-Z. A. Huang, S. Dieleman, E. Elsen, J. Engel, and D. Eck, “En- abling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset,” inProc. Int. Conf. Learning Repre- sentations (ICLR), 2019

    2019

  35. [35]

    DALI: A Large Dataset of Synchronized Audio, Lyrics and Notes, Automatically Created Using Teacher-Student Ma- chine Learning Paradigm,

    G. Meseguer-Brocal, A. Cohen-Hadria, and G. Peeters, “DALI: A Large Dataset of Synchronized Audio, Lyrics and Notes, Automatically Created Using Teacher-Student Ma- chine Learning Paradigm,” inProc. Int. Soc. Music Inf. Re- trieval Conf. (ISMIR), Paris, France, 2018

    2018

  36. [36]

    Omras2 Metadata Project 2009,

    M. Mauch, C. Cannam, M. Davies, S. Dixon, C. Harte, S. Kolozali, D. Tidhar, and M. Sandler, “Omras2 Metadata Project 2009,” inProc. Int. Soc. Music Inf. Retrieval Conf. (ISMIR), Kobe, Japan, Oct. 2009

    2009

  37. [37]

    An Expert Ground Truth Set for Audio Chord Recognition and Music Analysis,

    J. A. Burgoyne, J. Wild, and I. Fujinaga, “An Expert Ground Truth Set for Audio Chord Recognition and Music Analysis,” inProc. Int. Soc. Music Inf. Retrieval Conf. (ISMIR), Miami, Florida, USA, Oct. 2011, pp. 633–638

    2011

  38. [38]

    RWC Music Database: Popular, Classical and Jazz Music Databases,

    M. Goto, H. Hashiguchi, T. Nishimura, and R. Oka, “RWC Music Database: Popular, Classical and Jazz Music Databases,” inProc. Int. Soc. Music Inf. Retrieval Conf. (IS- MIR), Paris, France, Oct. 2002, pp. 287–288

    2002

  39. [39]

    Structured Training for Large- V ocabulary Chord Recognition,

    B. McFee and J. P. Bello, “Structured Training for Large- V ocabulary Chord Recognition,” inProc. Int. Soc. Music Inf. Retrieval Conf. (ISMIR), Suzhou, China, Oct. 2017, pp. 188– 194

    2017

  40. [40]

    Constant-Q Transform Tool- box for Music Processing,

    C. Schörkhuber and A. Klapuri, “Constant-Q Transform Tool- box for Music Processing,” inProc. Sound and Music Com- puting Conf. (SMC), Barcelona, Spain, Jul. 2010, pp. 3–64. 8

    2010