Pith. sign in

REVIEW 3 major objections 5 minor 34 references

Is MixIT Really Unsuitable for Correlated Sources? Exploring MixIT for Unsupervised Pre-training in Music Source Separation

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read MixIT pre-training on unlabeled music improves supervised music source separation across model sizes.

desk verdict A useful, mostly solid empirical result with one real confound: validation-guided channel selection may be carrying part of the reported gains, so the title question remains open. read the letter →

arxiv 2505.07631 v1 pith:4SGMNLAN submitted 2025-05-12 eess.AS

Kohei Saijo , Yoshiaki Bando This is my paper · ORCID
classification eess.AS
keywords musicsourceseparationMixITunsupervisedpre-trainingmixtureinvarianttrainingunlabeleddataMUSDB18FreeArchiveband-splitTF-Locoformer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Music source separation normally needs isolated stems, which are costly to obtain, and the standard labeled dataset MUSDB18 has only about 150 tracks. This paper asks whether MixIT, an unsupervised training method that separates mixtures of mixtures without labels, can pre-train a separation model on unlabeled in-the-wild music so that later supervised fine-tuning starts from a better place. The authors claim it can: across three model sizes, fine-tuning a MixIT-pretrained band-split TF-Locoformer on MUSDB18 beats training from scratch, with average uSDR gains above 0.5 dB for the medium and large models. They also argue that the common belief that MixIT fails on music because musical sources are too correlated is untested, and that the real difficulty is the ill-posed definition of stems, an ambiguity that supervised fine-tuning can resolve. If this is right, unlabeled music data can substitute for expensive stem labels in the pre-training stage.

What carries the argument

The central mechanism is mixture-invariant training (MixIT), a loss that trains a separator on a mixture of two unlabeled mixtures: the model must assign each output to one of the two input mixtures so that remixing reconstructs them, and the minimum over binary assignment matrices is the training loss. An efficient assignment solver (argmin over real matrices followed by projection to binary) is used instead of exhaustive search. The paper wraps this objective around the band-split TF-Locoformer, an encoder-separator-decoder separation model, first pre-training on unlabeled FMA music with 12 output channels and then selecting 4 channels for supervised fine-tuning on MUSDB18. The MixIT loss is what forces the unlabeled pre-training to learn source structure without any stem labels; the subsequent fine-tuning resolves the stem-definition ambiguity that MixIT cannot know.

What would settle it

Repeat the fine-tuning pipeline on a MixIT-pretrained model whose four output channels are chosen at random instead of by validation ground truth; if the improvement over from-scratch training disappears, the central claim is not supported.

Watch

Extended reading notes

Core claim

The paper's central claim is that MixIT-based pre-training on large-scale unlabeled music improves supervised music source separation, and that the gains are attributable to MixIT learning source structure rather than to the assumption that it should fail because of high inter-source correlation. Concretely, after pre-training band-split TF-Locoformer models on the Free Music Archive with MixIT and fine-tuning on MUSDB18-HQ, every model size beats its from-scratch counterpart: for the medium model average cSDR rises from 9.04 to 9.46 dB and average uSDR from 8.75 to 9.30 dB, and for the large model average uSDR rises from 9.21 to 9.72 dB. Supporting evidence comes from freezing the separator and fine-tuning only the encoder and decoder, where the pre-trained model still separates far better than a randomly initialized one, and from data-scale experiments where larger unlabeled corpora give further improvements. The authors interpret these results as showing that MixIT's earlier poor showing in music reflects the ill-posed nature of stem definitions (for example, the 'other' stem is a group of instruments), not an intrinsic failure on correlated sources.

Load-bearing premise

The comparison depends on the premise that MixIT pre-training, not the extra step of choosing which output channels to keep using ground-truth validation data, causes the improvement; an ablation with randomly chosen channels would test whether that premise holds.

Editorial extensions

If this is right

  • If the claim is correct, unlabeled in-the-wild music can replace expensive stem labels during pre-training, lowering the cost of building music source separation systems.
  • Pre-training effectively increases model capacity: the MixIT-pretrained small model outperforms the from-scratch medium model, and the pretrained medium model reaches the level of the from-scratch large model on average uSDR.
  • Freezing the separator and fine-tuning only the encoder and decoder still yields strong separation, showing the MixIT pre-trained separator has learned source structure rather than only task-agnostic features.
  • Scaling unlabeled pre-training data improves fine-tuned performance across all FMA subset sizes, although the gains become modest beyond the large subset.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If ill-posed stem definitions are the real obstacle, MixIT pre-training should transfer more cleanly to separation tasks with semantically clear single-instrument stems; evaluating on a per-instrument labeled corpus would test that implication directly.
  • A random-channel-selection ablation would isolate whether MixIT pre-training itself, rather than the supervised selection of 4 of 12 output channels on validation ground truth, produces the reported gains.
  • The small gains from a tenfold increase in pre-training data suggest model capacity is the current bottleneck; repeating the data-scale curve with a larger separator should show steeper returns.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper revisits the common belief that mixture-invariant training (MixIT) is unsuitable for music source separation due to high inter-source correlation. The authors pre-train a band-split TF-Locoformer (BS-Locoformer) on unlabeled Free Music Archive data with MixIT, using 12 output channels, then fine-tune on MUSDB18-HQ after selecting 4 output channels with MUSDB validation ground truth. They report that the MixIT-pretrained models outperform from-scratch training across small, medium, and large model sizes on cSDR and uSDR (Table 1), that fine-tuning only the encoder/decoder yields much better results than without pre-training (Table 2), and that larger pre-training corpora give modest gains (Table 3). The paper proposes that MixIT's difficulty in MSS is due to ill-posed stem definitions rather than source correlation.

Significance. If the reported gains are attributable to MixIT pre-training itself, the result is valuable: it demonstrates that large unlabeled in-the-wild music collections can substitute for expensive stem labels at the pre-training stage. The study has several strengths: it evaluates three model sizes, two metrics, per-stem results, four pre-training data scales, and includes a frozen-separator fine-tuning experiment that directly probes the quality of the MixIT-learned representation. The writing is clear and the central hypothesis is explicit. However, the comparison in Table 1 is confounded by a validation-guided output-channel selection step that the from-scratch baselines do not receive, and the paper does not directly test its correlation-versus-ill-posedness hypothesis; these issues prevent the central claim from being fully supported as stated.

major comments (3)
  1. [Section 4.3] Section 4.3, steps 1-4: The pre-trained pipeline selects 4 of 12 output channels using MUSDB validation ground truth, which gives the MixIT-pretrained model a supervised channel-selection step that the from-scratch models in Table 1 do not receive. Since this selection is one of C(12,4)=495 choices and is informed by the validation labels, the reported gains (e.g., +0.42 dB cSDR and +0.55 dB uSDR for M2 vs M1) may be partly due to this selection rather than to the MixIT objective. The authors should provide an ablation that fine-tunes the pre-trained model with randomly selected (or otherwise unsupervised-selected) output channels; without it, the causal claim that MixIT pre-training improves over scratch is not isolated.
  2. [Title, Abstract, Section 3.2] The paper frames the study as addressing whether MixIT is unsuitable for correlated sources, attributing its limited performance to ill-posed stem definitions rather than high inter-source correlation. However, no experiment varies the degree of inter-source correlation or the stem definition ambiguity; the argument in Section 3.2 is qualitative, and the pre-training results are equally consistent with both hypotheses. The claims should be reworded to focus on the demonstrated pre-training benefit, or supplemented with a direct test (e.g., measuring stem correlations in the training data and correlating them with MixIT behavior).
  3. [Tables 1-3] All results are from a single training run with no repeated seeds, error bars, or statistical significance tests. Some differences are small (e.g., in Table 3, the Large and Medium pre-training differ by 0.01 dB in average uSDR), and the margins in Table 1 are modest relative to typical run-to-run variability. The authors should report results across at least two or three seeds for the main comparisons, or provide a clear justification for why a single run is sufficient.
minor comments (5)
  1. [Section 4.4] The claim that pre-training 'enables the model to achieve performance comparable to that of a larger model trained from scratch' is imprecise; for S2 vs M1 the average cSDR differs by 0.24 dB and for M2 vs L1 by 0.12 dB, which may not be 'comparable' depending on the reader's threshold.
  2. [Section 3.1, Eq. (2)] The loss function L is not defined at first use; the thresholded SNR loss is introduced later in Eq. (4). Consider defining it at the earlier point.
  3. [References] Reference [24] is listed as 'Anonymous' and appears to be an unfinished citation; it should be completed or removed.
  4. [Section 1] The manuscript promises to release code and model weights upon acceptance, but no code is available during review. Given the empirical nature of the claims, releasing code or providing a reproducibility appendix would strengthen the submission.
  5. [Table 1] The caption states that grey rows are not directly comparable, but the grey shading is not visible in the text; ensure the formatting is clear in the camera-ready version.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the MixIT pre-training claim is an empirical benchmark comparison with no derivation that reduces to its inputs.

full rationale

The paper's central claim is empirical: fine-tuning a MixIT-pretrained BS-Locoformer on MUSDB18-HQ outperforms the same model trained from scratch. The MixIT loss in Eq. (2) is defined solely from unlabeled mixture-of-mixtures inputs, with no use of MUSDB stem ground truth during pre-training, and the reported cSDR/uSDR values in Tables 1-3 are measured benchmark results rather than quantities entailed by any fitted parameter. The fine-tuning stage uses a standard supervised SNR loss on labeled stems, and the comparison against from-scratch training is a direct experimental contrast. The validation-based channel selection in Section 4.3, which uses MUSDB validation ground truth to choose 4 of the 12 pre-trained output channels, is a legitimate methodological concern about leakage or a confounded comparison, but it is not circularity: the test-set scores are not forced by the selection procedure by construction, and the claim could in principle fail if random channel selection were used. Self-citations such as [25]-[27] are architecture references and are not load-bearing for the empirical finding that MixIT pre-training helps. No equation is defined in terms of the result it is supposed to predict, and no fitted input is renamed as a prediction. Therefore no significant circularity is present.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard MixIT assumptions and dataset-domain assumptions, plus one paper-specific assumption about channel selection. There are no invented entities and no constants fitted to the benchmark.

assumptions (5)
  • domain assumption MixIT's mixture-of-mixtures reconstruction with optimal binary assignment learns to separate independent sources.
    Section 3.1 to 3.2: the entire pre-training relies on this property of MixIT.
  • domain assumption FMA audio contains enough separable instrument sources for MixIT to learn useful representations.
    Section 4.1: FMA large is used as unlabeled pre-training data.
  • domain assumption MUSDB18-HQ ground-truth stems, evaluation splits, and cSDR/uSDR metrics reliably measure separation quality.
    Section 4.1 and 4.3 describe the use of these data and metrics.
  • ad hoc to paper Selecting 4 of 12 output channels using MUSDB validation ground truth does not unfairly bias the comparison with from-scratch training.
    Section 4.3 channel selection steps 1 to 4; there is no random-selection ablation.
  • domain assumption All from-scratch and fine-tuned runs use otherwise identical optimization settings, so differences are attributable to pre-training.
    Section 4.3 states the same optimizer and schedule are used unless otherwise noted.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Is MixIT Really Unsuitable for Correlated Sources? Exploring MixIT for Unsupervised Pre-training in Music Source Separation." pith.science (2026). https://pith.science/paper/4SGMNLAN

@misc{pith2026250507631,
  author       = {Pith},
  title        = {Pith review of: Is MixIT Really Unsuitable for Correlated Sources? Exploring MixIT for Unsupervised Pre-training in Music Source Separation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4SGMNLAN}},
  note         = {Machine review of arXiv:2505.07631}
}
read the original abstract

In music source separation (MSS), obtaining isolated sources or stems is highly costly, making pre-training on unlabeled data a promising approach. Although source-agnostic unsupervised learning like mixture-invariant training (MixIT) has been explored in general sound separation, they have been largely overlooked in MSS due to its implicit assumption of source independence. We hypothesize, however, that the difficulty of applying MixIT to MSS arises from the ill-posed nature of MSS itself, where stem definitions are application-dependent and models lack explicit knowledge of what should or should not be separated, rather than from high inter-source correlation. While MixIT does not assume any source model and struggles with such ambiguities, our preliminary experiments show that it can still separate instruments to some extent, suggesting its potential for unsupervised pre-training. Motivated by these insights, this study investigates MixIT-based pre-training for MSS. We first pre-train a model on in-the-wild, unlabeled data from the Free Music Archive using MixIT, and then fine-tune it on MUSDB18 with supervision. Using the band-split TF-Locoformer, one of the state-of-the-art MSS models, we demonstrate that MixIT-based pre-training improves the performance over training from scratch.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 30 canonical work pages

  1. [1]

    Open-unmix-a reference implementation for music source separation,

    F.-R. St ¨oter, S. Uhlich, A. Liutkus, and Y . Mitsufuji, “Open-unmix-a reference implementation for music source separation,” Journal of Open Source Software, vol. 4, no. 41, p. 1667, 2019

    work page 2019

  2. [2]

    Spleeter: a fast and efficient music source separation tool with pre-trained models,

    R. Hennequin, A. Khlif, F. V oituret, and M. Moussallam, “Spleeter: a fast and efficient music source separation tool with pre-trained models,” Journal of Open Source Software , vol. 5, no. 50, p. 2154, 2020

    work page 2020

  3. [3]

    Demucs: Deep extractor for music sources with extra unlabeled data remixed,

    A. D´efossez, N. Usunier, L. Bottou, and F. Bach, “Demucs: Deep extractor for music sources with extra unlabeled data remixed,” arXiv preprint arXiv:1909.01174, 2019

    arXiv 1909

  4. [4]

    Music source separation with band-split rnn,

    Y . Luo and J. Yu, “Music source separation with band-split rnn,” IEEE/ACM Trans. Audio, Speech, Lang. Process. , vol. 31, pp. 1893– 1901, 2023

    work page 1901

  5. [5]

    Music source separation with band-split rope transformer,

    W.-T. Lu, J.-C. Wang, Q. Kong, and Y .-N. Hung, “Music source separation with band-split rope transformer,” in Proc. ICASSP, 2024, pp. 481–485

    work page 2024

  6. [6]

    MUSDB18-HQ - an uncompressed version of MUSDB18,

    Z. Rafii, A. Liutkus, F.-R. St ¨oter, S. I. Mimilakis, and R. Bittner, “MUSDB18-HQ - an uncompressed version of MUSDB18,” Dec. 2019. [Online]. Available: https://doi.org/10.5281/zenodo.3338373

    doi:10.5281/zenodo.3338373 2019

  7. [7]

    Pac-hubert: Self-supervised music source separation via primitive auditory clustering and hidden-unit bert,

    K. Chen, G. Wichern, F. G. Germain, and J. Le Roux, “Pac-hubert: Self-supervised music source separation via primitive auditory clustering and hidden-unit bert,” in Proc. ICASSP SASB . IEEE, 2023, pp. 1–5

    work page 2023

  8. [8]

    Hubert: Self-supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,” IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 29, pp. 3451–3460, 2021

    work page 2021

Show all 34 references
  1. [9]

    Repeating pattern extraction technique (repet): A simple method for music/voice separation,

    Z. Rafii and B. Pardo, “Repeating pattern extraction technique (repet): A simple method for music/voice separation,” IEEE Trans. Audio, Speech, Lang. Process., vol. 21, no. 1, pp. 73–84, 2012

    2012

  2. [10]

    Melody extraction from polyphonic music signals using pitch contour characteristics,

    J. Salamon and E. G ´omez, “Melody extraction from polyphonic music signals using pitch contour characteristics,” IEEE Trans. Audio, Speech, Lang. Process., vol. 20, no. 6, pp. 1759–1770, 2012

    2012

  3. [11]

    Remix-cycle-consistent learning on adversarially learned separator for accurate and stable unsupervised speech separation,

    K. Saijo and T. Ogawa, “Remix-cycle-consistent learning on adversarially learned separator for accurate and stable unsupervised speech separation,” in Proc. ICASSP, 2022

    2022

  4. [12]

    Remixit: Continual self-training of speech enhancement models via bootstrapped remixing,

    E. Tzinis, Y . Adi, V . K. Ithapu, B. Xu, P. Smaragdis, and A. Kumar, “Remixit: Continual self-training of speech enhancement models via bootstrapped remixing,” IEEE Journal of Selected Topics in Signal Processing, 2022

    2022

  5. [13]

    Self-remixing: Unsupervised speech separation via separation and remixing,

    K. Saijo and T. Ogawa, “Self-remixing: Unsupervised speech separation via separation and remixing,” in Proc. ICASSP, 2023, pp. 1–5

    2023

  6. [14]

    Unsupervised sound separation using mixture invariant training,

    S. Wisdom, E. Tzinis, H. Erdogan, R. Weiss, K. Wilson, and J. Hershey, “Unsupervised sound separation using mixture invariant training,” Proc. NeurIPS, vol. 33, pp. 3846–3857, 2020

    2020

  7. [15]

    Source separation by steering pretrained music models,

    E. Manilow, P. O’Reilly, P. Seetharaman, and B. Pardo, “Source separation by steering pretrained music models,” in Proc. ICASSP. IEEE, 2022, pp. 126–130

    2022

  8. [16]

    Unsupervised music source separation using differentiable parametric source models,

    K. Schulze-Forster, G. Richard, L. Kelley, C. S. Doire, and R. Badeau, “Unsupervised music source separation using differentiable parametric source models,” IEEE/ACM Trans. Audio, Speech, Lang. Process. , vol. 31, pp. 1276–1289, 2023

    2023

  9. [17]

    Semi- supervised singing voice separation with noisy self-training,

    Z. Wang, R. Giri, U. Isik, J.-M. Valin, and A. Krishnaswamy, “Semi- supervised singing voice separation with noisy self-training,” in Proc. ICASSP. IEEE, 2021, pp. 31–35

    2021

  10. [18]

    Unsupervised blind source separation with variational auto-encoders,

    J. Neri, R. Badeau, and P. Depalle, “Unsupervised blind source separation with variational auto-encoders,” in Proc, EUSIPCO. IEEE, 2021, pp. 311–315

    2021

  11. [19]

    Bootstrapping deep music separation from primitive auditory grouping principles,

    P. Seetharaman, G. Wichern, J. Le Roux, and B. Pardo, “Bootstrapping deep music separation from primitive auditory grouping principles,” arXiv preprint arXiv:1910.11133, 2019

    1910 arXiv

  12. [20]

    Sparse, efficient, and semantic mixture invariant training: Taming in- the-wild unsupervised sound separation,

    S. Wisdom, A. Jansen, R. J. Weiss, H. Erdogan, and J. R. Hershey, “Sparse, efficient, and semantic mixture invariant training: Taming in- the-wild unsupervised sound separation,” in Proc. WASPAA, 2021, pp. 51–55

    2021

  13. [21]

    Teacher-student mixit for unsupervised and semi-supervised speech separation,

    J. Zhang, C. Zoril ˘a, R. Doddipatla, and J. Barker, “Teacher-student mixit for unsupervised and semi-supervised speech separation,” in Proc. Interspeech, 2021, pp. 3495–3499

    2021

  14. [22]

    Remixing-based Unsupervised Source Separation from Scratch,

    K. Saijo and T. Ogawa, “Remixing-based Unsupervised Source Separation from Scratch,” in Proc. Interspeech, 2023, pp. 1678–1682

    2023

  15. [23]

    FMA: A dataset for music analysis,

    M. Defferrard, K. Benzi, P. Vandergheynst, and X. Bresson, “FMA: A dataset for music analysis,” arXiv preprint arXiv:1612.01840 , 2016

    2016 arXiv

  16. [24]

    Interspeech 2025 URGENT speech enhancement challenge,

    Anonymous, “Interspeech 2025 URGENT speech enhancement challenge,” in Submitted to Interspeech 2025 , 2025

    2025

  17. [25]

    Task- aware unified source separation,

    K. Saijo, J. Ebbers, F. G. Germain, G. Wichern, and J. Le Roux, “Task- aware unified source separation,” in Proc. ICASSP, 2025, pp. 1–5

    2025

  18. [26]

    A comparative study on positional encoding for time-frequency domain dual-path transformer-based source separation models,

    K. Saijo and T. Ogawa, “A comparative study on positional encoding for time-frequency domain dual-path transformer-based source separation models,” arXiv preprint arXiv:2504.19605 , 2025

    2025 arXiv

  19. [27]

    Tf- locoformer: Transformer with local modeling by convolution for speech separation and enhancement,

    K. Saijo, G. Wichern, F. G. Germain, Z. Pan, and J. Le Roux, “Tf- locoformer: Transformer with local modeling by convolution for speech separation and enhancement,” in Proc. IWAENC, 2024, pp. 205–209

    2024

  20. [28]

    Improving music source separation with simo stereo band-split rnn,

    Y . Luo and R. Gu, “Improving music source separation with simo stereo band-split rnn,” in Proc. ICASSP, 2024, pp. 426–430

    2024

  21. [29]

    A two-stage band-split mamba-2 network for music separation,

    J. Bai, Y . Fang, J. Wang, and X. Zhang, “A two-stage band-split mamba-2 network for music separation,” arXiv preprint arXiv:2409.06245 , 2024

    2024 arXiv

  22. [30]

    Decoupled weight decay regularization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in Proc. ICLR, 2018

    2018

  23. [31]

    Flashattention: Fast and memory-efficient exact attention with io-awareness,

    T. Dao, D. Fu, S. Ermon, A. Rudra, and C. R ´e, “Flashattention: Fast and memory-efficient exact attention with io-awareness,” Proc. NeurIPS, vol. 35, pp. 16 344–16 359, 2022

    2022

  24. [32]

    What’s all the fuss about free universal sound separation data?

    S. Wisdom, H. Erdogan, D. P. Ellis, R. Serizel, N. Turpault, E. Fonseca, J. Salamon, P. Seetharaman, and J. R. Hershey, “What’s all the fuss about free universal sound separation data?” in Proc. ICASSP, 2021, pp. 186–190

    2021

  25. [33]

    Music demixing challenge 2021,

    Y . Mitsufuji, G. Fabbro, S. Uhlich, F.-R. St ¨oter, A. D ´efossez, M. Kim, W. Choi, C.-Y . Yu, and K.-W. Cheuk, “Music demixing challenge 2021,” Frontiers in Signal Processing , vol. 1, p. 808395, 2022

    2021

  26. [34]

    The 2018 signal separation evaluation campaign,

    F.-R. St¨oter, A. Liutkus, and N. Ito, “The 2018 signal separation evaluation campaign,” in Latent V ariable Analysis and Signal Separation: 14th International Conference, LVA/ICA 2018, Guildford, UK, July 2–5, 2018, Proceedings 14. Springer, 2018, pp. 293–305

    2018

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.