pith. sign in

arxiv: 2604.08415 · v1 · submitted 2026-04-09 · 📡 eess.AS

Ring Mixing with Auxiliary Signal-to-Consistency-Error Ratio Loss for Unsupervised Denoising in Speech Separation

Matthew Maciejewski , Samuele Cornell This is my paper

Pith reviewed 2026-05-10 17:09 UTC · model grok-4.3

classification 📡 eess.AS
keywords speech separationunsupervised denoisingring mixingconsistency lossWHAM! benchmarkVoxCelebnoisy mixturesauxiliary loss
0
0 comments X

The pith

Ring mixing paired with a consistency penalty lets speech separation models denoise using only noisy recordings.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Noisy speech separation systems are normally trained on clean synthetic mixtures, which limits how well they handle real recordings full of background noise. When models are instead trained on noisy mixtures, the loss function's symmetry causes them to retain the background noise in the separated outputs because noise cannot be distinguished from speech on a per-mixture basis. The paper introduces ring mixing, a batch construction where each source appears in two different mixtures, together with an auxiliary Signal-to-Consistency-Error Ratio loss that penalizes differences between the two estimates of the same source. This breaks the symmetry and rewards the model for removing components that are inconsistent across the two mixtures. On a WHAM!-based benchmark the approach reduces residual noise by up to half and also succeeds when trained on naturally noisy speech from VoxCeleb.

Core claim

The central claim is that ring mixing combined with the SCER auxiliary loss overcomes the symmetry problem in unsupervised speech separation. By forcing each source estimate to be consistent across two ring-mixed mixtures while penalizing inconsistency, the method incentivizes removal of background noise that is not shared across the mixtures, allowing models trained solely on noisy data to produce estimates with substantially less residual noise.

What carries the argument

Ring mixing, a batch strategy in which each source participates in two mixtures, together with the Signal-to-Consistency-Error Ratio (SCER) auxiliary loss that penalizes inconsistency between the two resulting estimates of that source.

If this is right

  • Training can shift from fully synthetic clean mixtures to in-domain noisy recordings without clean references.
  • Residual noise in separated outputs drops by up to half on WHAM!-style benchmarks.
  • Models become usable on naturally noisy corpora such as VoxCeleb.
  • The same batch construction and consistency penalty can be applied to other separation tasks where noise is the dominant inconsistent element.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The symmetry-breaking idea may generalize to unsupervised source separation in other modalities where shared content across re-mixed examples can be isolated from variable interference.
  • It raises the question of whether similar consistency penalties could replace explicit denoising stages in end-to-end audio pipelines.
  • If the inconsistency signal reliably isolates noise, the method could reduce the need for expensive clean-data collection in low-resource acoustic environments.

Load-bearing premise

Penalizing inconsistency between estimates of the same source across ring-mixed batches will specifically remove background noise rather than other artifacts or trivial solutions such as silence or averaging.

What would settle it

A controlled test on mixtures with known clean references that measures whether the method removes only the non-shared background noise or instead distorts the target speech or suppresses consistent noise components.

Figures

Figures reproduced from arXiv: 2604.08415 by Matthew Maciejewski, Samuele Cornell.

Figure 1
Figure 1. Figure 1: A 6-mixture batch with normal and ring mixing. • ℓλ=k = ℓλ=1−k, i.e. is symmetric about λ = 0.5. • For ||n1||2 = ||n2||2 , the minimum is at λ = 0.5. • If ||n1||2 or ||n2||2 is 0, the two minima are at λ ∈ {0, 1}. This is enough to support a useful characterization of the func￾tion: If the amount of noise in each recording is roughly the same, the optimal value of λ is 0.5. As the total noise starts to bec… view at source ↗
Figure 2
Figure 2. Figure 2: Validation metrics over the first 100 k training steps on 10 dB WHAM!+, comparing SCER systems at various α values to the baseline systems trained with s noisy and s clean supervision. mixture clean SCER α = 2.0 [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Example 10 dB validation-set spectrograms of an in￾put mixture as well as outputs from the clean-supervised system and the noisy-supervised system using SCER with α = 2.0. SCER loss seem to have any effect (good or bad) on suppres￾sion of interfering speech. In the occupancy metrics for noise, we again see that in all cases occ.nother and occ.nself are nearly identical (more evidence of noise inseparabilit… view at source ↗
read the original abstract

Noisy speech separation systems are typically trained on fully-synthetic mixtures, limiting generalization to real-world scenarios. Though training on mixtures of in-domain (thus often noisy) speech is possible, we show that this leads to undesirable optima where mixture noise is retained in the estimates, due to the inseparability of the background noises and the loss function's symmetry. To address this, we propose ring mixing, a batch strategy of using each source in two mixtures, alongside a new Signal-to-Consistency-Error Ratio (SCER) auxiliary loss penalizing inconsistent estimates of the same source from different mixtures, breaking symmetry and incentivizing denoising. On a WHAM!-based benchmark, our method can reduce residual noise by upwards of half, effectively learning to denoise from only noisy recordings. This opens the door to training more generalizable systems using in-the-wild data, which we demonstrate via systems trained using naturally-noisy speech from VoxCeleb.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper claims that standard symmetric losses on noisy speech mixtures lead to retention of background noise due to inseparability of noise components. To address this, it introduces ring mixing (placing each source in two mixtures within a batch) and an auxiliary Signal-to-Consistency-Error Ratio (SCER) loss that penalizes inconsistency between estimates of the same source from different mixtures. This is said to break symmetry and incentivize denoising. On a WHAM!-based benchmark the method reportedly reduces residual noise by upwards of half, enabling unsupervised training on noisy recordings; the approach is also demonstrated on naturally noisy VoxCeleb data.

Significance. If the central empirical claim holds and the mechanism is shown to specifically remove noise rather than other artifacts, the work would enable training speech separation models on in-the-wild noisy data without clean references, improving real-world generalization. The combination of a simple batch strategy with an auxiliary consistency loss is a potentially lightweight contribution to unsupervised audio source separation.

major comments (2)
  1. [Abstract and §4] Abstract and §4 (Experiments): the claim that residual noise is reduced 'by upwards of half' is presented without any reported baselines, error bars, ablation studies, or statistical details. This makes it impossible to assess whether the improvement is attributable to ring mixing + SCER versus other factors, undermining evaluation of the central claim.
  2. [§3.2] §3.2 (SCER loss definition): the argument that penalizing inconsistency between ring-mixed estimates specifically removes background noise (rather than converging to consistent but partially noisy outputs or averaged artifacts) is not isolated. No analysis or controlled experiment demonstrates that the loss symmetry is the dominant cause of noise retention in the baseline or that SCER uniquely drives denoising over trivial consistent solutions.
minor comments (1)
  1. [Abstract] Notation for SCER and ring mixing should be introduced with a brief inline definition in the abstract or introduction for readers unfamiliar with the terms.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We are grateful to the referee for their insightful comments, which have helped us improve the clarity and rigor of our presentation. We address each major comment below.

read point-by-point responses

  1. Referee: [Abstract and §4] Abstract and §4 (Experiments): the claim that residual noise is reduced 'by upwards of half' is presented without any reported baselines, error bars, ablation studies, or statistical details. This makes it impossible to assess whether the improvement is attributable to ring mixing + SCER versus other factors, undermining evaluation of the central claim.

    Authors: We fully agree that the central empirical claim requires stronger supporting evidence to allow proper evaluation. Accordingly, in the revised manuscript we have expanded §4 with a new table presenting results against the standard symmetric loss baseline, error bars computed over multiple training runs, ablation studies isolating the contributions of ring mixing and the SCER loss, and statistical tests confirming the significance of the observed improvements. These additions demonstrate that the reported noise reduction is attributable to the proposed techniques rather than other factors. revision: yes

  2. Referee: [§3.2] §3.2 (SCER loss definition): the argument that penalizing inconsistency between ring-mixed estimates specifically removes background noise (rather than converging to consistent but partially noisy outputs or averaged artifacts) is not isolated. No analysis or controlled experiment demonstrates that the loss symmetry is the dominant cause of noise retention in the baseline or that SCER uniquely drives denoising over trivial consistent solutions.

    Authors: We acknowledge the need for a more explicit isolation of the mechanism. Section 3.2 provides the theoretical reasoning for why symmetric losses retain noise and how the SCER loss breaks this symmetry to promote denoising. To strengthen this, the revised version includes a controlled experiment comparing the full SCER loss to a baseline consistency penalty (without the signal-to-consistency ratio), along with qualitative examples of the separated signals. This shows that SCER drives denoising beyond what a trivial consistent but noisy solution would achieve, and that the symmetry in the baseline is indeed a key factor in noise retention. revision: yes

Circularity Check

0 steps flagged

No significant circularity; auxiliary loss introduced independently

full rationale

The paper presents ring mixing as a batch construction and SCER as a new auxiliary loss term that penalizes inconsistency between source estimates from different mixtures. This is motivated by an observed empirical tendency for standard symmetric losses to retain noise, but the loss itself is not defined in terms of fitted parameters, prior self-citations, or the target denoising outcome. No equations reduce the proposed method to its inputs by construction, and the benchmark result (noise reduction on WHAM!) is reported as an empirical outcome rather than a tautological prediction. The derivation chain is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Central claim rests on the domain assumption that loss symmetry is the primary driver of noise retention and that consistency enforcement will selectively remove noise; no free parameters or invented entities are described in the abstract.

axioms (1)
  • domain assumption Training on noisy mixtures leads to undesirable optima where mixture noise is retained due to inseparability of background noises and loss symmetry.
    Explicitly stated in the abstract as the motivation for the new method.

pith-pipeline@v0.9.0 · 5465 in / 1160 out tokens · 50601 ms · 2026-05-10T17:09:42.649626+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

30 extracted references · 30 canonical work pages · 1 internal anchor

  1. [1]

    Introduction Speech separation is the task of producing individual wave- forms for each talker in a recording where multiple people have spoken at the same time. With the advent of deep learning, the performance of speech separation systems has improved dras- tically, with many systems achieving Scale-Invariant Signal-to- Distortion-Ratio (SI-SDR) [1] imp...

    work page

  2. [2]

    Ring Mixing with Auxiliary Signal-to-Consistency-Error Ratio Loss for Unsupervised Denoising in Speech Separation

    Proposed Method 2.1. Problem Formulation In the most basic of conventional supervised speech separation systems, the problem is formulated as estimating two speech signals from their mixture. Audio recordings generally hold the superposition principle, meaning the natural mixture of two audio signals can be approximated by simply adding the two waveforms ...

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  3. [3]

    noisy” and “clean

    Experimental Setup 3.1. Datasets The primary dataset we used for our experiments (which we refer to as WHAM!+) is the dataset described by Maciejewski et al. [9]. It consists of the WHAM! [4] dataset (wsj0-2mix [3] with added noise), where each mixture is assigned anadditional WHAM! noise recording, such that each source in the mixture gets its own noise ...

    work page

  4. [4]

    In all cases, including SCER on mixtures of noisy speech improves SI-SDRi, by 1.2 −1.9 dB, closing about half the gap to the ideal, clean-speech supervision

    Results and Discussion The results of our initial experiments are in Table 1, in which we trained and evaluated systems on three separate noise levels of WHAM!+: 20 dB for low, 0 dB for high, and 10 dB for a roughly “typical” amount of noise. In all cases, including SCER on mixtures of noisy speech improves SI-SDRi, by 1.2 −1.9 dB, closing about half the ...

    work page

  5. [5]

    Conclusion In this work, we have demonstrated that using SI-SDR loss while training speech separation systems using mixtures of naturally-noisy speech results in an undesirable optimum, a po- tential contributor to the limited successes of separation in prac- tical environments, where in-domain speech training data often includes noise. To address this, w...

    work page

  6. [6]

    SDR – half-baked or well done?

    J. Le Roux, S. Wisdom, H. Erdogan, and J. R. Hershey, “SDR – half-baked or well done?” inProc. ICASSP, 2019

    work page 2019

  7. [7]

    TF-GridNet: Intergrating full- and sub-band modeling for speech separation,

    Z.-Q. Wang, S. Cornell, S. Choi, Y . Lee, B.-Y . Kim, and S. Watan- abe, “TF-GridNet: Intergrating full- and sub-band modeling for speech separation,”IEEE/ACM Trans. Audio, Speech, Lang. Pro- cess., vol. 31, 2023

    work page 2023

  8. [8]

    Deep clus- tering: Discriminative embeddings for segmentation and separa- tion,

    J. R. Hershey, Z. Chen, J. Le Roux, and S. Watanabe, “Deep clus- tering: Discriminative embeddings for segmentation and separa- tion,” inProc. ICASSP, 2016

    work page 2016

  9. [9]

    WHAM!: Extending speech separation to noisy environments,

    G. Wichern, J. Antognini, M. Flynn, L. R. Zhu, E. McQuinn, D. Crow, E. Manilow, and J. Le Roux, “WHAM!: Extending speech separation to noisy environments,” inProc. Interspeech, 2019

    work page 2019

  10. [10]

    WHAMR!: Noisy and reverberant single-channel speech sepa- ration,

    M. Maciejewski, G. Wichern, E. McQuinn, and J. Le Roux, “WHAMR!: Noisy and reverberant single-channel speech sepa- ration,” inProc. ICASSP, 2020

    work page 2020

  11. [11]

    SMS-WSJ: Database, performance measures, and baseline recipe for multi-channel source separation and recognition,

    L. Drude, J. Heitkaemper, C. Boeddeker, and R. Haeb-Umbach, “SMS-WSJ: Database, performance measures, and baseline recipe for multi-channel source separation and recognition,” 2019, arXiv:1910.13934v1

    work page arXiv 2019

  12. [12]

    LibriMix: An open-source dataset for generalizable speech separation

    J. Cosentino, M. Pariente, S. Cornell, A. Deleforge, and E. Vin- cent, “LibriMix: An open-source dataset for generalizable speech separation,” 2020, arXiv:2005.11262v1

    work page arXiv 2020

  13. [13]

    Do ImageNet classifiers generalize to ImageNet?

    B. Recht, R. Roelofs, L. Schmidt, and V . Shankar, “Do ImageNet classifiers generalize to ImageNet?” inProc. ICML, 2019

    work page 2019

  14. [14]

    A dilemma of ground truth in noisy speech separation and an ap- proach to lessen the impact of imperfect training data,

    M. Maciejewski, J. Shi, S. Watanabe, and S. Khudanpur, “A dilemma of ground truth in noisy speech separation and an ap- proach to lessen the impact of imperfect training data,”Comput. Speech Lang., vol. 77, 2023

    work page 2023

  15. [15]

    Unsupervised sound separation using mixture in- variant training,

    S. Wisdom, E. Tzinis, H. Erdogan, R. J. Weiss, K. Wilson, and J. R. Hershey, “Unsupervised sound separation using mixture in- variant training,” inProc. NeurIPS, 2020

    work page 2020

  16. [16]

    Recent trends in distant conversational speech recognition: A review of chime-7 and 8 dasr challenges,

    S. Cornell, C. Boeddeker, T. Park, H. Huang, D. Raj, M. Wies- ner, Y . Masuyama, X. Chang, Z.-Q. Wang, S. Squartini, P. Garcia, and S. Watanabe, “Recent trends in distant conversational speech recognition: A review of chime-7 and 8 dasr challenges,”Comput. Speech Lang., 2025

    work page 2025

  17. [17]

    Teacher- student MixIT for unsupervised and semi-supervised speech sep- aration,

    J. Zhang, C. Zoril ˘a, R. Doddipatla, and J. Barker, “Teacher- student MixIT for unsupervised and semi-supervised speech sep- aration,” inProc. Interspeech, 2021

    work page 2021

  18. [18]

    MixCycle: Unsupervised speech sep- aration via cyclic mixture permutation invariant training,

    E. Karamatlı and S. Kırbız, “MixCycle: Unsupervised speech sep- aration via cyclic mixture permutation invariant training,”IEEE Signal Process. Lett., vol. 29, 2022

    work page 2022

  19. [19]

    Remixing-based unsupervised source separation from scratch,

    K. Saijo and T. Ogawa, “Remixing-based unsupervised source separation from scratch,” inProc. Interspeech, 2023

    work page 2023

  20. [20]

    Noisy- target training: A training strategy for DNN-based speech en- hancement without clean speech,

    T. Fujimura, Y . Koizumi, K. Yatabe, and R. Miyazaki, “Noisy- target training: A training strategy for DNN-based speech en- hancement without clean speech,” inProc. EUSIPCO, 2021

    work page 2021

  21. [21]

    RemixIT: Continual self-training of speech enhancement models via bootstrapped remixing,

    E. Tzinis, Y . Adi, V . K. Ithapu, B. Xu, P. Smaragdis, and A. Ku- mar, “RemixIT: Continual self-training of speech enhancement models via bootstrapped remixing,”IEEE J. Sel. Top. Signal Pro- cess., vol. 16, no. 6, 2022

    work page 2022

  22. [22]

    Unsupervised speech enhance- ment using optimal transport and speech presence probability,

    W. Jiang, K. Yu, and F. Wen, “Unsupervised speech enhance- ment using optimal transport and speech presence probability,” IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 32, 2024

    work page 2024

  23. [23]

    Reverberation as supervision for speech separation,

    R. Aralikatti, C. Boeddeker, G. Wichern, A. Subramanian, and J. Le Roux, “Reverberation as supervision for speech separation,” inProc. ICASSP, 2023

    work page 2023

  24. [24]

    UNSSOR: Unsupervised neural speech separation by leveraging over-determined training mix- tures,

    Z.-Q. Wang and S. Watanabe, “UNSSOR: Unsupervised neural speech separation by leveraging over-determined training mix- tures,” inProc. NeurIPS, 2023

    work page 2023

  25. [25]

    Enhanced reverberation as supervision for unsupervised speech separation,

    K. Saijo, G. Wichern, F. G. Germain, Z. Pan, and J. Le Roux, “Enhanced reverberation as supervision for unsupervised speech separation,” inProc. Interspeech, 2024

    work page 2024

  26. [26]

    Neural fast full-rank spatial covariance analysis for blind source separa- tion,

    Y . Bando, Y . Masuyama, A. A. Nugraha, and K. Yoshii, “Neural fast full-rank spatial covariance analysis for blind source separa- tion,” inProc. EUSIPCO, 2023

    work page 2023

  27. [27]

    V oxCeleb: A large- scale speaker identification dataset,

    A. Nagrani, J. S. Chung, and A. Zisserman, “V oxCeleb: A large- scale speaker identification dataset,” inProc. Interspeech, 2017

    work page 2017

  28. [28]

    V oxCeleb2: Deep speaker recognition,

    J. S. Chung, A. Nagrani, and A. Zisserman, “V oxCeleb2: Deep speaker recognition,” inProc. Interspeech, 2018

    work page 2018

  29. [29]

    TF-GridNet: Making time-frequency domain models great again for monaural speaker separation,

    Z.-Q. Wang, S. Cornell, S. Choi, Y . Lee, B.-Y . Kim, and S. Watan- abe, “TF-GridNet: Making time-frequency domain models great again for monaural speaker separation,” inProc. ICASSP, 2023

    work page 2023

  30. [30]

    Declaration Generative AI tools and technologies were not used in the preparation of this manuscript

    work page