Deep Bayesian Unsupervised Source Separation Based on a Complex Gaussian Mixture Model

Kazuyoshi Yoshii; Yoko Sasaki; Yoshiaki Bando

REVIEW 5 major objections 6 minor 37 references

Deep Bayesian Unsupervised Source Separation Based on a Complex Gaussian Mixture Model

T0 review · 5 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper shows that a neural source separator can be trained with no clean reference signals, using only multichannel mixtures and a complex Gaussian mixture model objective.

desk verdict The core idea is a genuine step forward, but Table 1 compares K=2 against K=6, which makes the headline claim shaky as reported. read the letter →

arxiv 1908.11307 v1 pith:PXTCPZIN submitted 2019-08-29 cs.SD cs.LGeess.ASstat.ML

Yoshiaki Bando , Yoko Sasaki , Kazuyoshi Yoshii This is my paper · ORCID

classification cs.SDcs.LGeess.ASstat.ML

keywords unsupervisedsourceseparationcomplexGaussianmixturemodeldeepBayesianlearningfrequencypermutationambiguitydirection-of-arrivalestimationmultichannelEMalgorithmamortizedvariationalinferencemonaural

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a neural source separator can be trained with no clean reference signals at all, using only multichannel mixtures. The idea is to treat a complex Gaussian mixture model (cGMM) as the objective: a separation network and a localization network are trained jointly to maximize an evidence lower bound whose latent variables are time-frequency masks and directions of arrival. Because the two networks are trained together, the frequency permutation ambiguity that normally plagues frequency-wise spatial models is resolved inside the same objective, with no extra alignment step. The trained network can separate a monaural mixture by itself and can also initialize a multichannel EM algorithm; in simulated speech mixtures this initialization gave higher SDR than a conventional directional initialization and than a 4-channel independent vector analysis baseline.

What carries the argument

The engine of the method is the complex Gaussian mixture model with two categorical latent variables: $z_{tfk}$ selects the active source at each time-frequency bin, and $w_{kd}$ assigns each source a direction of arrival from a fixed set of $D=72$ candidate angles. An observed multichannel spectrogram is modeled as a mixture of zero-mean complex Gaussians with spatial covariance matrices $H_{fd}$, on which an inverse Wishart prior with anechoic template SCMs $G_{fd}$ is placed. The training objective is the evidence lower bound (Eq. 15) of this generative model, computed with neural approximations $q_g(z)$ and $q_h(w)$. Maximizing this ELBO by SGD trains both networks jointly; the same bound drives the EM algorithm that the pre-trained network initializes.

What would settle it

Run the same unsupervised training on mixtures with strong reverberation (RT60 above 0.6 s) or with sources whose directions differ by less than 20 degrees, and compare EM-cGMM initialized by the trained network against the conventional directional initialization; if the proposed initialization no longer improves SDR, or if the jointly trained networks fail to separate, the claim that the template-based ELBO carries the training would be weakened. A direct check on real recordings would also test whether the anechoic template assumption breaks outside simulated rooms.

Watch

Extended reading notes

Core claim

The central claim is that optimizing the ELBO of a cGMM whose latent variables are TF masks and DoAs is a viable unsupervised training objective for deep separation. From random weights, the separation network $g_{tfk}$ and the localization network $h_{kd}$ can be trained by stochastic gradient ascent on this ELBO, using only mixture recordings and the known geometry of the array. The frequency permutation ambiguity is resolved by the localization network's role in selecting a consistent DoA for each source across frequencies, so no external permutation solver is needed. The trained separation network doubles as an initialization for EM-cGMM; with this initialization, EM-cGMM reaches $10.6 \pm 4.2$ dB SDR on the test set, versus $9.7 \pm 5.0$ dB for the conventional initialization and $9.9 \pm 4.4$ dB for AuxIVA+.

Load-bearing premise

The method assumes that anechoic planewave template spatial covariance matrices, computed from the array geometry, are accurate enough proxies for the true reverberant transfer functions of the room, even though the paper notes the templates and simulated room impulse responses are much different.

Editorial extensions

If this is right

Unsupervised training of separation networks becomes possible for domains where clean sources are unavailable, such as daily-life audio events, as long as multichannel mixtures and array geometry are available.
The frequency permutation ambiguity is resolved by the DoA latent variable within a single objective, removing the need for separate permutation-alignment post-processing.
A pre-trained separation network can initialize EM-cGMM, improving SDR over the conventional directional initialization, especially for sources with close directions (DoA difference under 60 degrees).
The same network supports monaural separation after training, since the separation network takes only a single-channel log-magnitude spectrogram as input.
Because DoAs are estimated along with masks, the framework has a route to handling an unknown number of sources, e.g. through a non-parametric Bayesian extension.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the fixed anechoic templates were replaced with SCMs estimated from the mixture itself during training, the mismatch the authors note could shrink and the monaural separation network might improve; this is a natural extension the paper mentions as future work.
The same ELBO objective could be tested with microphone arrays of different geometry or with more than two sources, provided the localization network's candidate-direction set is adjusted; nothing in the derivation restricts it to the 4-channel circular array used here.
The DoA outputs of the localization network could themselves serve as a source-counting signal, so the framework may extend to recordings with a variable number of sources without knowing $K$ in advance.
On real recordings, template-mismatch effects are likely to matter more than in simulated rooms, so evaluating with measured array impulse responses would be a sharper test of the method's practical value.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The first thing you should know: the central comparison in Table 1 may not be fair. The proposed EM-cGMM initialized with the network runs with K=2, because the localization network has 2 filters and the mixtures contain two speakers. The conventional initialization from Otsuka et al. is explicitly run with K=6 ("We set the number of source classes K=6 for this method"). That is not an apples-to-apples comparison. EM-cGMM needs a fixed K, and nobody explains how a K=6 run is reduced to two output sources. If the baseline is over-parameterized while the proposed method already knows the answer, the reported 10.6 vs 9.7 dB may partly reflect that mismatch, not a better initialization. This is a load-bearing soft spot, not a detail.

The paper does have a real new idea: jointly training mask and DoA networks by maximizing the ELBO of a cGMM, using direction-of-arrival information to resolve frequency permutation instead of mask correlation. That genuinely goes beyond Drude et al. [15], which needs the source count and aligns masks across frequencies. The amortized variational inference framing is coherent, and the math in Section 3 mostly checks out.

Where else does it weaken? There is no comparison against Drude et al. [15], the closest prior method, which is a strange omission when you claim to improve on the initialization approach. There is also no ablation isolating the DoA network's role; the improvement could come from a better mask estimate rather than from resolving permutation. And with 10.6±4.2 vs 9.7±5.0, the 0.9 dB average gain is not backed by any significance test, so I would not treat it as solid evidence alone.

The template-versus-RIR mismatch is acknowledged and affects both initializers, so I do not see that as the main threat. The K mismatch is the more direct threat to the central claim.

Overall, the idea is worth engaging with. The paper is for anyone working on unsupervised multichannel separation, and for people interested in amortized variational inference for audio. It deserves peer review, but the K mismatch and the missing comparison to Drude et al. should be fixed before acceptance.

Recommendation: send it out, with the expectation of major revision.

Referee Report

5 major / 6 minor

Summary. The paper proposes an unsupervised method for training neural source separation from multichannel mixtures only, based on a complex Gaussian mixture model (cGMM) with time-frequency masks and direction-of-arrival (DoA) variables as latent variables. A separation network and a localization network are jointly trained by maximizing an evidence lower bound (ELBO) of the cGMM, with spectral powers and spatial covariance matrices fixed to a global average and anechoic template steering vectors. The trained separation network is then used to initialize an EM algorithm for the cGMM at test time. Experiments on simulated WSJ0-mix mixtures report an average SDR of 10.6 ± 4.2 dB for the EM-cGMM initialized with the proposed network, versus 9.7 ± 5.0 dB for a conventional directional initialization and 9.9 ± 4.4 dB for AuxIVA+.

Significance. If the comparison is fair, the paper makes a valuable contribution by showing that amortized variational inference can replace hand-crafted initialization for cGMM-based source separation, and that unsupervised neural training can improve over a conventional multichannel method. The derivation of the ELBO and the EM updates is generally clear, and the paper addresses a known weakness of multichannel methods, namely sensitivity to initialization. The main concern is the comparability of the baseline in Table 1, where the number of latent sources K differs between the proposed and conventional initializations, and the absence of a direct comparison with the closest prior work, Drude et al. [15]. The paper also explicitly acknowledges that the anechoic template steering vectors differ from the reverberant room impulse responses used to generate the data, which is a limitation of the training model. With these gaps addressed, the findings would be a solid contribution to unsupervised multichannel source separation.

major comments (5)

[Section 4.2, Table 1] The central comparison in Table 1 is not controlled for the number of latent sources K. The proposed EM-cGMM runs with K=2, as indicated by '2 (=K)' in the localization network description in Section 4.2, matching the two speakers in the WSJ0-mix dataset. The conventional initialization of (19)–(20) is run with K=6, as stated in Section 4.2. Since the EM-cGMM updates in (16)–(17) depend on K through the mask and DoA variables, the reported 10.6 dB versus 9.7 dB advantage may reflect a different model order rather than a better initialization. Please re-run the baseline with K=2 and the proposed method with K=6, or otherwise justify that the comparison isolates the initialization alone.
[Section 1, Section 4.3] No experimental comparison is provided with Drude et al. [15], which is cited as the closest prior method that trains a network by optimizing a cGMM likelihood and initializes a multichannel algorithm with the network output. Since the abstract claims that the proposed method outperforms a conventional initialization method, a quantitative comparison with [15], or at least a discussion of the differences in experimental setup and expected performance, is needed to support this claim.
[Section 4.3, Table 1] The paper reports only averages and standard deviations for the SDR results. With 3,000 test mixtures, paired significance testing is feasible and should be reported, especially for the differences between EM-cGMM with gtfk (10.6 dB) and EM-cGMM with (19)–(20) (9.7 dB), and between EM-cGMM with gtfk and AuxIVA+ (9.9 dB). Standard deviations of 4–5 dB make it unclear whether these differences are statistically reliable without such tests.
[Section 3.4, Section 4.3] The role of the localization network hkd in the final result is not isolated. The EM-cGMM initialization in Section 3.4 uses the separation network gtfk for TF masks and the closed-form expression (18) for DoAs, not the output of hkd. An ablation that trains with only gtfk and the conventional DoA initialization, or with hkd ablated, would clarify whether the reported improvement comes from joint training of both networks or mainly from the separation network.
[Section 4.2, Eq. (15)] The paper acknowledges in Section 4.2 that the planewave template steering vectors bfd and the simulated room impulse responses are 'much different'. Because the training objective in Eq. (15) fixes Hfd to the template Gfd and λ to a global average, the separation network is trained under a spatial model that does not match the reverberant test conditions. The paper mentions this limitation in Section 5, but does not quantify its impact. Please discuss or experimentally bound the effect of this mismatch, for example by comparing against a variant that estimates Hfd during training.

minor comments (6)

[Section 3.4] In the sentence 'Since the localization network gtfk can potentially overfit...', the symbol should be hkd, not gtfk, because gtfk refers to the separation network.
[Section 3.3, Eq. (16)] In the denominator of Eq. (16), the sum is written as '∑K K=1' but should use a dummy index k (e.g., ∑K k'=1 or ∑K k=1 with a renamed index).
[Table 1, Table 2] The notation 'A VI-cGMM' appears with a space in Tables 1 and 2; it should likely be 'AVI-cGMM' or 'AuxVI-cGMM' for consistency with the text.
[Section 3.3, Eq. (14)] The input feature ωkd depends on the current TF mask estimates ẑtfk, which creates a moving target during training. It would be helpful to state whether gradients flow through ẑtfk into the localization network or whether ẑtfk is treated as a constant.
[Section 4.2, Section 4.3] The text does not specify how the SDR is computed (e.g., global SDR or scale-invariant SDR) beyond citing [29], nor whether the EM-cGMM is run with the same number of random restarts for both initializations. Please clarify.
[Section 3.4, Eq. (18)] The choice to initialize DoAs with Eq. (18) instead of using the localization network output is clear from the text, but no quantitative comparison is given between these two DoA initialization choices. A short experiment would help justify this design decision.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: unsupervised ELBO training and EM-cGMM evaluation share a model, but the network only provides an initialization and no fitted target or self-citation is back-propagated into the test result.

full rationale

The derivation chain is self-contained. The training objective is the ELBO of the cGMM in Eq. (8)/(15), maximized with respect to the separation and localization networks on unlabeled mixtures while fixing lambda_tfk and H_fd to simple values (Sec. 3.3). No labeled source, no fitted constant, and no target SDR enters the training loss; the test-time EM-cGMM in Sec. 3.4 optimizes the same model with its own E-step (16)-(17) and M-step (9)-(11). The network output is used only to initialize the TF masks, and the DoA initialization in Eq. (18) is also a data-dependent initialization, not a fitted prediction. The headline Table 1 comparison is therefore a comparison of two initializers for the same EM algorithm, not a prediction that is equivalent to its inputs by construction. The K=6 versus K=2 initialization difference flagged by the skeptic is a possible experimental confound, but it is not circularity: the reported SDR values are not algebraically forced by any prior fit. There are no load-bearing self-citations; references [9] and [24] are external prior works on LDA/cGMM inference, and no uniqueness theorem from the present authors is invoked. The acknowledged mismatch between template SCMs G_fd and simulated RIRs (Sec. 4.2) is a modeling limitation, not a circular step, because the EM algorithm can update H_fd from the observed mixture via Eq. (9). Accordingly, no specific circular reduction can be exhibited.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard cGMM assumptions plus two ad hoc choices: the fixed anechoic template SCMs and the fixed power normalization during network training. No new physical entities are introduced. Several hyperparameters (K, nu, epsilon, D) are hand-set and influence the results but are not fitted to external targets.

free parameters (4)

Number of latent sources K = 2
The model requires K in advance; experiments fix K=2 for two-speaker mixtures, and the paper notes extension to unknown K is future work (Sections 3.1, 4.2, 5).
Inverse Wishart degrees of freedom nu = M+5 = 9
Hand-chosen hyperparameter in Eq. (6); it controls the MAP update of Hfd in Eq. (9).
Diagonal loading epsilon = 1.0e-2
Hand-chosen constant added to the template SCM Gfd to ensure positive definiteness, stated in Section 4.2.
Number of direction candidates D = 72
Directions sampled every 5 degrees on the horizontal plane, a design choice setting DoA resolution (Section 3.1).

assumptions (5)

domain assumption Time-frequency sparseness: each TF bin is exclusively generated by one source (Eq. 1, sum_k ztfk = 1).
Standard cGMM assumption; enables mask-based separation but is an approximation for speech mixtures.
domain assumption Source spectrograms are zero-mean complex Gaussian distributed (Eq. 4).
The cGMM likelihood model; speech is not truly Gaussian, but this is a widely used model in the field.
ad hoc to paper The template steering vectors bfd, computed under a planewave anechoic assumption, approximate the true array transfer functions in reverberant rooms.
Network training fixes Hfd to Gfd; the authors note in Section 4.2 that the templates and the simulated RIRs are 'much different', so the training signal depends on this approximation.
standard math Mean-field variational approximation q(Z)q(W), Eq. (7).
Standard variational inference approximation, also used in the EM algorithm for this model.
ad hoc to paper During network training, the power spectra λtfk and spatial covariances Hfd are fixed to the global average λtilde and the template Gfd.
Fixed for numerical stability; the simplified ELBO (Eq. 15) is then optimized rather than the full model likelihood.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Deep Bayesian Unsupervised Source Separation Based on a Complex Gaussian Mixture Model." pith.science (2026). https://pith.science/paper/PXTCPZIN

@misc{pith2026190811307,
  author       = {Pith},
  title        = {Pith review of: Deep Bayesian Unsupervised Source Separation Based on a Complex Gaussian Mixture Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PXTCPZIN}},
  note         = {Machine review of arXiv:1908.11307}
}

read the original abstract

This paper presents an unsupervised method that trains neural source separation by using only multichannel mixture signals. Conventional neural separation methods require a lot of supervised data to achieve excellent performance. Although multichannel methods based on spatial information can work without such training data, they are often sensitive to parameter initialization and degraded with the sources located close to each other. The proposed method uses a cost function based on a spatial model called a complex Gaussian mixture model (cGMM). This model has the time-frequency (TF) masks and direction of arrivals (DoAs) of sources as latent variables and is used for training separation and localization networks that respectively estimate these variables. This joint training solves the frequency permutation ambiguity of the spatial model in a unified deep Bayesian framework. In addition, the pre-trained network can be used not only for conducting monaural separation but also for efficiently initializing a multichannel separation algorithm. Experimental results with simulated speech mixtures showed that our method outperformed a conventional initialization method.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 35 canonical work pages

[15]

Real-time independent vector analysis for con- volutive blind source separation,

T. Kim, “Real-time independent vector analysis for con- volutive blind source separation,” IEEE Trans. on Cir- cuits and Systems I: Regular Papers , vol. 57, no. 7, pp. 1431–1438, 2010

work page 2010
[1]

Deep Bayesian Unsupervised Source Separation Based on a Complex Gaussian Mixture Model

INTRODUCTION Deep neural networks (DNNs) have demonstrated excellent performance in source separation tasks, such as speech sep- aration [1–3] and music separation [4, 5]. Permutation in- variant training (PIT), for example, trains a DNN to output time-frequency (TF) masks for corresponding sources. Such a method requires a large number of clean source si...

work page Pith review arXiv 1908
[2]

RELA TED WORK This section overviews cGMM-based TF clustering and then introduces unsupervised neural source separation. 2.1. Complex Gaussian mixture models A popular approach to separating a multichannel mixture sig- nal is to mask each TF bin [9–11,19,20]. This mask is conven- tionally estimated by clustering hand-crafted features at each TF bin [19, 2...

work page
[3]

This training is based on the LDA model [9, 24], which has TF masks and DoAs of sources as latent variables

DEEP BA YESIAN SOURCE SEPARA TION The proposed method trains separation and localization net- works by using only multichannel mixture signals and re- solves the frequency permutation ambiguity in a uniﬁed framework. This training is based on the LDA model [9, 24], which has TF masks and DoAs of sources as latent variables. The objective function is deriv...

work page
[4]

predict TF masks ˆztfk and DoAs ˆwkd withgtfk andhkd for each mixture recording in a mini-batch,

work page
[5]

update model parameters Θ ={π,φ} with (11), and

work page
[6]

calculate L and update the network parameters by using a stochastic gradient descent (SGD) method. The ELBOL can be calculated as follows: L =− T,F,K,D∑ t,f,k,d=1 ˆztfk ˆwkd ( log|Gfd| + 1 ˜λ xH tfG−1 fd xtf ) + T,F,K∑ t,f,k=1 ˆztfk log πtk ˆztfk + K,D∑ k,d=1 ˆwkd log φd ˆwkd + const.. (15) The loss value for a mini-batch is a sum of this local ELBO norma...

work page
[7]

EXPERIMENTAL EV ALUA TION We conducted an evaluation with speech mixture signals gen- erated by using simulated room impulse responses (RIRs). 4.1. Dataset The mixture signals used in this evaluation were generated by convolving RIRs to source signals in the WSJ0-mix dataset [1], which is widely used for neural speech sepa- ration [1–3]. Each of the mixtu...

work page

Show all 37 references

[8]

The proposed method trains separation and localization networks by using a cost function based on a cGMM that has the TF masks and DoAs as latent variables

CONCLUSION We presented an unsupervised method that trains neural source separation by using only multichannel mixture sig- nals. The proposed method trains separation and localization networks by using a cost function based on a cGMM that has the TF masks and DoAs as latent v...
[9]

Deep clustering: Discriminative embeddings for segmentation and separation,

J. R. Hershey, et al., “Deep clustering: Discriminative embeddings for segmentation and separation,” in IEEE ICASSP, 2016, pp. 31–35

2016
[10]

Multitalker speech separation with utterance-level permutation invariant training of deep recurrent neural networks,

M. Kolbæk, et al., “Multitalker speech separation with utterance-level permutation invariant training of deep recurrent neural networks,” IEEE/ACM TASLP, vol. 25, no. 10, pp. 1901–1913, 2017

1901
[11]

Deep attractor networks for speaker re-identiﬁcation and blind source separation,

L. Drude, et al., “Deep attractor networks for speaker re-identiﬁcation and blind source separation,” in IEEE ICASSP, 2018, pp. 11–15

2018
[12]

Singing voice separation with deep U-Net convolutional networks,

A. Jansson, et al., “Singing voice separation with deep U-Net convolutional networks,” in ISMIR, 2017, pp. 745–751

2017
[13]

Deep clustering and conventional net- works for music separation: Stronger together,

Y . Luo, et al., “Deep clustering and conventional net- works for music separation: Stronger together,” inIEEE ICASSP, 2017, pp. 61–65

2017
[14]

Multichannel nonnegative matrix fac- torization in convolutive mixtures for audio source sepa- ration,

A. Ozerov et al., “Multichannel nonnegative matrix fac- torization in convolutive mixtures for audio source sepa- ration,” IEEE/ACM TASLP, vol. 18, no. 3, pp. 550–563, 2010

2010
[16]

Stable and fast update rules for independent vector analysis based on auxiliary function technique,

N. Ono, “Stable and fast update rules for independent vector analysis based on auxiliary function technique,” in IEEE WASPAA, 2011, pp. 189–192

2011
[17]

Bayesian nonparametrics for micro- phone array processing,

T. Otsuka, et al., “Bayesian nonparametrics for micro- phone array processing,” IEEE/ACM TASLP , vol. 22, no. 2, pp. 493–504, 2014

2014
[18]

Robust MVDR beamforming using time-frequency masks for online/ofﬂine ASR in noise,

T. Higuchi, et al., “Robust MVDR beamforming using time-frequency masks for online/ofﬂine ASR in noise,” in IEEE ICASSP, 2016, pp. 5210–5214

2016
[19]

Permutation-free CGMM: Complex Gaussian mixture model with inverse Wishart mixture model based spatial prior for permutation-free source separation and source counting,

J. Azcarreta, et al., “Permutation-free CGMM: Complex Gaussian mixture model with inverse Wishart mixture model based spatial prior for permutation-free source separation and source counting,” inIEEE ICASSP, 2018, pp. 51–55

2018
[20]

Unsupervised training of a deep clus- tering model for multichannel blind source separation,

L. Drude, et al., “Unsupervised training of a deep clus- tering model for multichannel blind source separation,” in IEEE ICASSP, 2019, pp. 695–699

2019
[21]

Bootstrapping single-channel source separation via unsupervised spatial clustering on stereo mixtures,

P. Seetharaman, et al., “Bootstrapping single-channel source separation via unsupervised spatial clustering on stereo mixtures,” in IEEE ICASSP, 2019, pp. 356–360

2019
[22]

Unsupervised deep clustering for source separation: Direct learning from mixtures using spatial information,

E. Tzinis, et al., “Unsupervised deep clustering for source separation: Direct learning from mixtures using spatial information,” inIEEE ICASSP, 2019, pp. 81–85

2019
[23]

Unsupervised training of neural mask- based beamforming,

L. Drude, et al., “Unsupervised training of neural mask- based beamforming,” arXiv preprint arXiv:1904.01578 (accepted to Interspeech 2019), 2019

1904 arXiv
[24]

Measuring dependence of bin- wise separated signals for permutation alignment in frequency-domain BSS,

H. Sawada, et al., “Measuring dependence of bin- wise separated signals for permutation alignment in frequency-domain BSS,” in IEEE ISCAS , 2007, pp. 3247–3250

2007
[25]

Auto-encoding variational Bayes,

D. P. Kingma et al., “Auto-encoding variational Bayes,” arXiv preprint arXiv:1312.6114, 2013

2013 arXiv
[26]

Collapsed variational Dirichlet pro- cess mixture models

K. Kurihara, et al., “Collapsed variational Dirichlet pro- cess mixture models.” in IJCAI, vol. 7, 2007, pp. 2796– 2801

2007
[27]

Blind sparse source separation for unknown number of sources using Gaussian mixture model ﬁtting with Dirichlet prior,

S. Araki, et al., “Blind sparse source separation for unknown number of sources using Gaussian mixture model ﬁtting with Dirichlet prior,” in IEEE ICASSP , 2009, pp. 33–36

2009
[28]

An em algorithm for localizing multiple sound sources in reverberant environments,

M. I. Mandel, et al., “An em algorithm for localizing multiple sound sources in reverberant environments,” in NIPS, 2007, pp. 953–960

2007
[29]

Complex angular central gaussian mix- ture model for directional statistics in mask-based mi- crophone array signal processing,

N. Ito, et al., “Complex angular central gaussian mix- ture model for directional statistics in mask-based mi- crophone array signal processing,” in EUSIPCO, 2016, pp. 1153–1157

2016
[30]

Audio-visual scene analysis with self- supervised multisensory features,

A. Owens et al., “Audio-visual scene analysis with self- supervised multisensory features,” in ECCV, 2018, pp. 1–18

2018
[31]

Self-supervised audio-visual co-segmentation,

A. Rouditchenko, et al., “Self-supervised audio-visual co-segmentation,” in IEEE ICASSP , 2019, pp. 2357– 2361

2019
[32]

Bayesian uniﬁcation of sound source localization and separation with permutation resolu- tion,

T. Otsuka, et al., “Bayesian uniﬁcation of sound source localization and separation with permutation resolu- tion,” in AAAI, 2012, pp. 2038–2045

2012
[33]

Black box variational inference,

R. Ranganath, et al., “Black box variational inference,” in AISTATS, 2014, pp. 814–822

2014
[34]

Image method for efﬁciently simu- lating small-room acoustics,

J. B. Allen et al., “Image method for efﬁciently simu- lating small-room acoustics,” JASA, vol. 65, no. 4, pp. 943–950, 1979

1979
[35]

Adam: A method for stochastic optimization,

D. P. Kingma et al., “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014

2014 arXiv
[36]

Relaxation of rank-1 spatial con- straint in overdetermined blind source separation,

D. Kitamura, et al., “Relaxation of rank-1 spatial con- straint in overdetermined blind source separation,” in EUSIPCO, 2015, pp. 1261–1265

2015
[37]

Performance measurement in blind audio source separation,

E. Vincent, et al., “Performance measurement in blind audio source separation,” IEEE TASLP, vol. 14, no. 4, pp. 1462–1469, 2006

2006

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [15]

Real-time independent vector analysis for con- volutive blind source separation,

T. Kim, “Real-time independent vector analysis for con- volutive blind source separation,” IEEE Trans. on Cir- cuits and Systems I: Regular Papers , vol. 57, no. 7, pp. 1431–1438, 2010

work page 2010

[2] [1]

Deep Bayesian Unsupervised Source Separation Based on a Complex Gaussian Mixture Model

INTRODUCTION Deep neural networks (DNNs) have demonstrated excellent performance in source separation tasks, such as speech sep- aration [1–3] and music separation [4, 5]. Permutation in- variant training (PIT), for example, trains a DNN to output time-frequency (TF) masks for corresponding sources. Such a method requires a large number of clean source si...

work page Pith review arXiv 1908

[3] [2]

RELA TED WORK This section overviews cGMM-based TF clustering and then introduces unsupervised neural source separation. 2.1. Complex Gaussian mixture models A popular approach to separating a multichannel mixture sig- nal is to mask each TF bin [9–11,19,20]. This mask is conven- tionally estimated by clustering hand-crafted features at each TF bin [19, 2...

work page

[4] [3]

This training is based on the LDA model [9, 24], which has TF masks and DoAs of sources as latent variables

DEEP BA YESIAN SOURCE SEPARA TION The proposed method trains separation and localization net- works by using only multichannel mixture signals and re- solves the frequency permutation ambiguity in a uniﬁed framework. This training is based on the LDA model [9, 24], which has TF masks and DoAs of sources as latent variables. The objective function is deriv...

work page

[5] [4]

predict TF masks ˆztfk and DoAs ˆwkd withgtfk andhkd for each mixture recording in a mini-batch,

work page

[6] [5]

update model parameters Θ ={π,φ} with (11), and

work page

[7] [6]

calculate L and update the network parameters by using a stochastic gradient descent (SGD) method. The ELBOL can be calculated as follows: L =− T,F,K,D∑ t,f,k,d=1 ˆztfk ˆwkd ( log|Gfd| + 1 ˜λ xH tfG−1 fd xtf ) + T,F,K∑ t,f,k=1 ˆztfk log πtk ˆztfk + K,D∑ k,d=1 ˆwkd log φd ˆwkd + const.. (15) The loss value for a mini-batch is a sum of this local ELBO norma...

work page

[8] [7]

EXPERIMENTAL EV ALUA TION We conducted an evaluation with speech mixture signals gen- erated by using simulated room impulse responses (RIRs). 4.1. Dataset The mixture signals used in this evaluation were generated by convolving RIRs to source signals in the WSJ0-mix dataset [1], which is widely used for neural speech sepa- ration [1–3]. Each of the mixtu...

work page