Incremental Binarization On Recurrent Neural Networks For Single-Channel Source Separation

Minje Kim; Mrinmoy Maity; Sunwoo Kim

REVIEW 3 major objections 4 minor 34 references

Incremental Binarization On Recurrent Neural Networks For Single-Channel Source Separation

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Incremental binarization lets a fully bitwise GRU outperform a real-valued feedforward network on single-channel source separation.

desk verdict Useful empirical result for binary GRU source separation, but the incremental schedule's benefit is untested without a one-shot binarization control. read the letter →

arxiv 1908.08898 v1 pith:2UQUYVYO submitted 2019-08-23 eess.AS cs.LGcs.SDeess.SP

Sunwoo Kim , Mrinmoy Maity , Minje Kim This is my paper · ORCID

classification eess.AScs.LGcs.SDeess.SP

keywords incrementalbinarizationbitwiseGRUsingle-channelsourceseparationspeechenhancementmodelcompressionbinaryneuralnetworksrecurrentgatedunits

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to show that a recurrent network can be pushed to fully binary weights and binary activations—no floating-point multiplications in the feedforward pass—without losing its quality advantage over feedforward networks on single-channel speech separation. The proposed Bitwise GRU (BGRU) binarizes the three weight sets inside each GRU cell and the gate and candidate-state activations, replacing real-valued products with XNOR and popcount operations. To keep quantization from destroying performance, the authors pretrain a real-valued GRU and then gradually raise the fraction of binarized parameters through a random Bernoulli mask, from 10% to 100%. At full binarization the single-layer BGRU reaches 11.76 dB SDR on the test set, above the 10.57 dB of a real-valued two-layer FCN and the 9.82 dB of a two-layer BNN, with fewer parameters. If correct, the result means cheap bitwise recurrence can substitute for larger real-valued networks, trading about 4.4 dB of SDR relative to the unquantized GRU for a large cut in computation.

What carries the argument

The load-bearing mechanism is the masked interpolated feedforward. Each weight matrix is replaced by $\hat{W} = (\bar{\varphi}(W) \odot B) \odot C + \varphi(W) \odot (1 - C)$, where $\bar{\varphi}$ is the sign function, $B$ is a scaled-sparsity mask that zeros the smallest-magnitude weights (chosen so that a fraction $\rho = 0.8$ survive per layer) and scales the survivors by their average absolute value $\mu$, and $C$ is a random Bernoulli mask that selects the fraction $\pi$ of elements to binarize at the current training stage. The same mask blends bitwise activations (sign and hard step) with their real-valued counterparts for the gates and candidate hidden states. Training runs in two rounds: first a plain GRU with tanh-compressed weights, then the BGRU with the masks, incrementing $\pi$ by 0.1 and shrinking the learning rate at each step; gradients of the non-differentiable activations are replaced by gradients of their smooth counterparts, and only the real-valued weights are updated.

What would settle it

Train the same BGRU with identical pretrained weights, data, and total epochs, but set $\pi = 1.0$ from the start (no incremental schedule). If the final SDR is equal to or higher than 11.76 dB, the gradual schedule is not the source of the result. A second check: compare the incremental schedule against a single round of continued training at $\pi = 0.1$ followed by a final jump to $\pi = 1.0$.

Watch

Extended reading notes

Core claim

The central claim is that incremental binarization makes a fully binary GRU network practical for single-channel source separation. Starting from a pretrained real-valued GRU, the authors randomly select a growing fraction $\pi$ of the weights, gates, and candidate hidden units to replace with bipolar binary versions while the remaining elements stay real-valued, until $\pi = 1.0$. The binarized forward pass then uses only sign and hard-step activations, XNOR-based products, and popcounts, while the real-valued weights are retained behind the scenes for gradient updates. On a TIMIT-based noisy-speech separation test set, the fully binarized single-layer BGRU with 1024 units achieves 11.76 dB SDR and 0.874 STOI, outperforming a two-layer real-valued FCN at 10.57 dB and a two-layer BNN at 9.82 dB despite having fewer layers and weights. The authors interpret this as evidence that the recurrent structure, not numerical precision, is what carries separation quality, and that gradual binarization is what protects that structure during quantization.

Load-bearing premise

The claim that gradually increasing the binarization fraction $\pi$ is what preserves quality is not tested against simply training the fully binarized network from the pretrained weights for the same number of epochs; the reported benefit could come from the extra training rather than from the gradual schedule.

Editorial extensions

If this is right

A fully binarized single GRU layer can beat a two-layer real-valued FCN on this task, so bitwise recurrence is not merely a compression trick; it can also win on separation quality.
With all forward operators reduced to XNOR, popcount, and sign, the trained model runs with no floating-point multiplications at inference, which suits low-power or FPGA deployment.
The monotone drop in SDR as $\pi$ increases from 0.1 to 1.0 (16.12 dB down to 11.76 dB) gives a predictable complexity-quality tradeoff: halting binarization at any intermediate $\pi$ yields a known operating point.
The two-round recipe—tanh-compressed pretraining followed by masked fine-tuning—carries BNN-style training practice into recurrent architectures, which have several interacting weight matrices per cell.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper does not compare incremental binarization against directly binarizing the pretrained network at $\pi = 1.0$ with the same total training, so the specific contribution of the gradual schedule remains untested; a matched direct-binarization run would settle whether the gain comes from the schedule or merely from more training.
Because each GRU cell has three interacting weight sets, the incremental schedule spreads quantization across gates; the same masked interpolation should transfer to LSTM or attention-based recurrent cells, where the number of gates is similar or larger.
The scaling factor $\mu$ is fixed from the pretrained weight magnitudes and folded into the binary weights after training; learning $\mu$ per layer during the incremental schedule is a natural extension the authors do not explore and could recover some of the 4.4 dB gap to the real-valued GRU.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

The result worth knowing: gradual binarization of GRU cells works for single-channel separation, and a fully binary one-layer BGRU lands at 11.76 dB SDR, above a 2048-wide two-layer BNN (9.82) and a real-valued FCN (10.57). If you are in embedded speech processing, that is a genuinely useful data point.

The new bit is the architecture/training combo: a GRU cell with binary activations, a scaled sparsity mask B, a Bernoulli mask C for gradual binarization, and a two-round pretrain-then-binaries schedule. The equations are clear and the method is reproducible. The paper also honestly reports the performance curve as pi goes from 0.1 to 1.0, including the gradual degradation.

The soft spots are where the central mechanistic claim overreaches. The paper attributes the final result to the incremental schedule 'minimizing potential loss,' but there is no control where the same pretrained network is binarized directly at pi=1.0 (or at a fixed high pi) and trained for comparable iterations. Equations (7) and (8) make that control trivial. Without it, the extra training, learning-rate annealing, and early stopping at pi=1.0 could all be doing the work; the schedule itself is untested. This is the paper's main weakness and it is a real one.

Second, the evaluation is uncalibrated. Table 1 has single runs, no error bars, no repeated seeds, no significance tests. The rho=0.8 choice and the pi schedule are selected after looking at results on the same test set, and the early stopping at pi=1.0 is chosen from reported test SDR. With margins of only 1-2 dB, selection effects could change the conclusion. Minor point: the novelty is incremental rather than radical; trained ternary quantization is cited but not deeply discussed, and there is no comparison to other gradual quantization schedules.

If the claim is narrowed to 'this particular training recipe yields a competitive fully binary GRU on this task,' I believe it. If the claim is 'the incremental schedule is responsible for the gain,' it is not yet established. One control experiment and repeated runs would fix it.

Bottom line: worth a serious referee. It is a solid engineering contribution with one missing comparison that bears directly on the paper's own explanation. I would ask the authors to add the one-shot binarization control and some repeated runs before accepting.

Best.

Referee Report

3 major / 4 minor

Summary. The paper proposes a Bitwise Gated Recurrent Unit (BGRU) for single-channel source separation. A real-valued GRU is first pretrained with tanh-compressed weights, and the network is then gradually converted to a fully binarized feedforward process by increasing a Bernoulli parameter π from 0.1 to 1.0, using sparsity and scaling masks. Experiments on TIMIT utterances mixed with non-stationary noise report SDR and STOI for the BGRU at each π level, and the fully binarized single-layer BGRU is reported to outperform a real-valued FCN and a BNN of larger topology. The main empirical claim is that incremental binarization yields 11.76 dB SDR at π=1.0.

Significance. Single-channel source separation with low-precision recurrent networks is practically relevant, and the BGRU formulation is clearly specified with bitwise feedforward equations and an explicit training schedule. If the empirical claim is robust, it would be a useful demonstration that a fully binarized recurrent architecture can beat larger fully connected baselines. The paper also makes a useful distinction between sparsity masking and gradual Bernoulli masking, and the equations in Section 2 make the method straightforward to reproduce. However, the experimental section currently lacks the control and statistical grounding needed to support the causal claim that the incremental schedule itself is responsible for the reported performance.

major comments (3)

[Section 2.3.2 and Section 3.2, Table 1] The paper's central claim is that incremental binarization minimizes quantization loss, but no experiment compares the proposed schedule with direct binarization at π=1.0 from the same pretrained weights. Because the fully binarized case is defined by C=1 in Eqs. (7) and (8), such a control is implementable, and it is needed to determine whether the 11.76 dB SDR at π=1.0 is due to the gradual schedule or simply to additional training with the same optimizer, learning-rate schedule, and early stopping. Without this control, the title claim about incremental binarization is not tested.
[Section 3.1 and Table 1] All SDR and STOI results are single numbers without error bars, repeated runs, or significance tests, and the test set is the same one used to select ρ=0.8 and the number of epochs. The reported 1–2 dB margins over FCN and BNN may therefore reflect selection effects rather than robust differences, and the paper should report means and standard deviations over repeated runs or speaker subsamples and clearly separate any hyperparameter selection or early stopping from the final test evaluation.
[Section 3.2 and Figure 1] The early-stopping rule at π=1.0 is described as stopping because performance degrades after 100 epochs, but the paper does not state what criterion is used (test SDR, validation SDR, or training SDR) or how the stopping point is selected. If the reported 11.76 dB is the best test-set value across epochs, the comparison is optimistic for the proposed method, and the statement that the BGRU outperforms baselines by 1–2 dB is not calibrated against this selection effect.

minor comments (4)

[Abstract and Section 1] There are typographical errors such as 'ne twork' in the abstract and 'Recurr ent' in Section 1 that should be corrected.
[Section 2.2.3, Eq. (5)] The text says the cutoff value β and scaling parameter μ are found jointly, but the equations define S from β and then μ from S; it would be clearer to state that β is determined by sorting the absolute weight values and μ is then computed as the mean absolute value of the selected weights.
[Section 3.1] The sentence 'Minibatch size is set as 10 for 10 mixed utterances constructed from 1 clean signal mixed with the 10 noise signals' is ambiguous about whether each minibatch contains all 10 noise conditions for one clean utterance or a random sample, so please clarify.
[Figures 1(a) and 1(b)] The figure labels for individual π values are difficult to read in the typeset version, and the final number of epochs used per π value would be better reported as a short table to improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the paper makes empirical performance claims, not derivations, and no prediction reduces to a fitted input by construction.

full rationale

The paper's central contribution is an empirically trained BGRU whose SDR/STOI numbers are measured outcomes, not quantities derived from the model equations. The binarization mask definitions in Eqs. (5)-(8) and (11) are construction recipes, and the reported π=1.0 result (11.76 dB SDR) is a training outcome rather than a consequence of those equations. The only potentially circular-looking elements are the authors' self-citations to prior BNN work ([16], [26]) for the two-round training scheme and QaD inputs; those are background techniques, not load-bearing uniqueness claims and not equivalent to the present paper's result. The abstract's statement that incremental binarization is intended 'to minimize the potential loss' is a motivation, not a derived claim. The skeptic's concern—that no one-shot binarization control was run—is an experimental-validity weakness, not circularity, because the gradual schedule is not shown to equal its own benchmark by definition. No evidence in the manuscript shows Eq. (7) reducing to Eq. (8), or any fitted parameter being renamed as a prediction. Accordingly, the circularity score is 0.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The BGRU pipeline contains several hand-tuned knobs: ρ, the π schedule, learning-rate reduction, and the early-stopping point. The central SDR comparison is an empirical result of this pipeline, so the free-parameter count is moderate. The axioms are standard deep-learning approximations from cited prior work, not new physics.

free parameters (4)

ρ (sparsity parameter) = 0.8
Reported as 'we found ρ=0.8 to perform well', likely tuned on the development set or the same task; this controls the amount of weight sparsification before binarization.
π schedule (binarization ratio) = 0.1 to 1.0 in steps of 0.1
The schedule defines how many parameters are binarized at each stage. The number of epochs per π (1000, except 100 at π=1.0) is also a hand-picked setting not ablated.
Learning-rate reduction per π = not specified in the paper
The authors state learning rates are reduced at each π increase, but do not give values or schedule, so the final performance depends on an undisclosed hyperparameter.
Early-stopping criterion at π=1.0 = 100 epochs
Training is stopped after 100 epochs because performance began to degrade; this selection is based on the reported test-set SDR, which can bias the result.

assumptions (3)

domain assumption Straight-through estimator for binary activations
In Section 2.3.2, the derivatives of sign functions are replaced by tanh/sigmoid derivatives; this approximation is standard but unproven, and training behavior depends on it.
domain assumption φ-compressed weights improve regularization and transfer
The first round wraps weights in tanh (Section 2.3.1). The paper assumes this prepares the weights for binarization; no ablation supports it.
domain assumption 4-bit QaD input quantization preserves enough information
Inputs are binarized with QaD from prior work; the paper does not analyze sensitivity of the final SDR to this quantization.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Incremental Binarization On Recurrent Neural Networks For Single-Channel Source Separation." pith.science (2026). https://pith.science/paper/2UQUYVYO

@misc{pith2026190808898,
  author       = {Pith},
  title        = {Pith review of: Incremental Binarization On Recurrent Neural Networks For Single-Channel Source Separation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2UQUYVYO}},
  note         = {Machine review of arXiv:1908.08898}
}

read the original abstract

This paper proposes a Bitwise Gated Recurrent Unit (BGRU) network for the single-channel source separation task. Recurrent Neural Networks (RNN) require several sets of weights within its cells, which significantly increases the computational cost compared to the fully-connected networks. To mitigate this increased computation, we focus on the GRU cells and quantize the feedforward procedure with binarized values and bitwise operations. The BGRU network is trained in two stages. The real-valued weights are pretrained and transferred to the bitwise network, which are then incrementally binarized to minimize the potential loss that can occur from a sudden introduction of quantization. As the proposed binarization technique turns only a few randomly chosen parameters into their binary versions, it gives the network training procedure a chance to gently adapt to the partly quantized version of the network. It eventually achieves the full binarization by incrementally increasing the amount of binarization over the iterations. Our experiments show that the proposed BGRU method produces source separation results greater than that of a real-valued fully connected network, with 11-12 dB mean Signal-to-Distortion Ratio (SDR). A fully binarized BGRU still outperforms a Bitwise Neural Network (BNN) by 1-2 dB even with less number of layers.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 27 canonical work pages

[1]

INTRODUCTION Neural network-based approaches to source separation task s have been becoming more prevalent [1, 2, 3]. Fully connected deep neu- ral networks (DNN) have shown to be capable of learning compl ex mapping functions from a large set of noisy signals and their corre- sponding ideal binary mask (IBM) target outputs [4, 5, 6]. Re current neural ne...

work page
[2]

Background: Bitwise Neural Networks Binarization has been explored as a method of network compression

BITWISE GATED RECURRENT UNITS (BGRU) 2.1. Background: Bitwise Neural Networks Binarization has been explored as a method of network compression. BinaryConnect [17], binarized neural networks [19], train ed ternary quantization [25], and Bitwise Neural Networks (BNN) [26] h ave implemented a binarized or ternarized neural network in bip olar bi- naries (wi...

work page
[3]

Experimental Setups For the experiment, we randomly subsample 12 speakers for tr ain- ing and 4 speakers for testing from the TIMIT corpus

EXPERIMENTS 3.1. Experimental Setups For the experiment, we randomly subsample 12 speakers for tr ain- ing and 4 speakers for testing from the TIMIT corpus. For both subsamples, we select half of the speakers as male and the oth er half as female. There are 10 short utterances per speaker rec orded with a 16kHz sampling rate. Each utterances are mixed wit...

work page
[4]

The training is done in two roun ds, ﬁrst in a weight compressed network and then in an incrementa lly bitwise version with the same topology

CONCLUSION In this paper, we proposed an incremental binarization proc edure to binarize a RNN with GRU cells. The training is done in two roun ds, ﬁrst in a weight compressed network and then in an incrementa lly bitwise version with the same topology. The pretrained weig hts of the ﬁrst round are used to initialize the weights of the bitwi se net- work....

work page
[5]

An experimental st udy on speech enhancement based on deep neural networks,

Y . Xu, J. Du, L.-R. Dai, and C.-H. Lee, “An experimental st udy on speech enhancement based on deep neural networks,” IEEE Signal processing letters, vol. 21, no. 1, pp. 65–68, 2014

work page 2014
[6]

Joint optimization of masks and deep recurrent neural net- works for monaural source separation,

P . Huang, M. Kim, M. Hasegawa-Johnson, and P . Smaragdis, “Joint optimization of masks and deep recurrent neural net- works for monaural source separation,” IEEE/ACM Transac- tions on Audio, Speech, and Language Processing , vol. 23, no. 12, pp. 2136–2147, 2015

work page 2015
[7]

Multichannel au- dio source separation with deep neural networks.,

A. A. Nugraha, A. Liutkus, and E. Vincent, “Multichannel au- dio source separation with deep neural networks.,” IEEE/ACM Trans. Audio, Speech & Language Processing , vol. 24, no. 9, pp. 1652–1664, 2016

work page 2016
[8]

Towards scaling up classiﬁcation- based speech separation,

Y . Wang and D. Wang, “Towards scaling up classiﬁcation- based speech separation,” IEEE Transactions on Audio, Speech, and Language Processing , vol. 21, no. 7, pp. 1381– 1390, 2013

2013

Show all 34 references

[9]

Deep NMF for speech separation,

J. Le Roux, J. R. Hershey, and F. Weninger, “Deep NMF for speech separation,” in Acoustics, Speech and Signal Process- ing (ICASSP), 2015 IEEE International Conference on . IEEE, 2015, pp. 66–70

2015
[10]

Deep neural net- works for single channel source separation,

E. M. Grais, M. U. Sen, and H. Erdogan, “Deep neural net- works for single channel source separation,” in Acoustics, Speech and Signal Processing (ICASSP), 2014 IEEE Interna- tional Conference on. IEEE, 2014, pp. 3734–3738

2014
[11]

Phase-sensitive and recognition-boosted speech separat ion using deep recurrent neural networks,

H. Erdogan, J. R. Hershey, S. Watanabe, and J. Le Roux, “Phase-sensitive and recognition-boosted speech separat ion using deep recurrent neural networks,” in Acoustics, Speech and Signal Processing (ICASSP), 2015 IEEE International Conference on. IEEE, 2015, pp. 708–712

2015
[12]

Speech enhancement with LSTM recurrent neural networks and its application to noise - robust ASR,

F. Weninger, H. Erdogan, S. Watanabe, E. Vincent, J. Le Ro ux, J. R. Hershey, and B. Schuller, “Speech enhancement with LSTM recurrent neural networks and its application to noise - robust ASR,” in International Conference on Latent V ariable Analysis and Signal Separation . S...

2015
[13]

Discriminatively trained recurrent neural networks for s ingle- channel speech separation,

F. Weninger, J. R. Hershey, J. Le Roux, and B. Schuller, “Discriminatively trained recurrent neural networks for s ingle- channel speech separation,” in Proceedings 2nd IEEE Global Conference on Signal and Information Processing, GlobalSI P , Machine Learning Applications in S...

2014
[14]

Single-channel multi-speaker separation using deep clus ter- ing,

Y . Isik, J. Le Roux, Z. Chen, S. Watanabe, and J. R. Hershe y, “Single-channel multi-speaker separation using deep clus ter- ing,” arXiv preprint arXiv:1607.02173, 2016

2016 arXiv
[15]

Spe ech enhancement and recognition using multi-task learning of l ong short-term memory recurrent neural networks,

Z. Chen, S. Watanabe, H. Erdogan, and J. R. Hershey, “Spe ech enhancement and recognition using multi-task learning of l ong short-term memory recurrent neural networks,” in Sixteenth Annual Conference of the International Speech Communica- tion Association, 2015

2015
[16]

Learning long-t erm dependencies with gradient descent is difﬁcult,

Y . Bengio, P . Simard, and P . Frasconi, “Learning long-t erm dependencies with gradient descent is difﬁcult,” IEEE trans- actions on neural networks , vol. 5, no. 2, pp. 157–166, 1994

1994
[17]

An efﬁcient gradient-based al- gorithm for on-line training of recurrent network trajecto ries,

R. J. Williams and J. Peng, “An efﬁcient gradient-based al- gorithm for on-line training of recurrent network trajecto ries,” Neural computation, vol. 2, no. 4, pp. 490–501, 1990

1990
[18]

Sutskever, Training recurrent neural networks , University of Toronto Toronto, Ontario, Canada, 2013

I. Sutskever, Training recurrent neural networks , University of Toronto Toronto, Ontario, Canada, 2013

2013
[19]

Learning phrase representations using RNN encoder-decoder for statistica l ma- chine translation,

K. Cho, B. V an Merri¨ enboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y . Bengio, “Learning phrase representations using RNN encoder-decoder for statistica l ma- chine translation,” arXiv preprint arXiv:1406.1078, 2014

2014 arXiv
[20]

Bitwise neural networks for ef ﬁ- cient single-channel source separation,

M. Kim and P . Smaragdis, “Bitwise neural networks for ef ﬁ- cient single-channel source separation,” in 2018 IEEE Interna- tional Conference on Acoustics, Speech and Signal Processi ng (ICASSP). IEEE, 2018, pp. 701–705

2018
[21]

BinaryCon- nect: Training deep neural networks with binary weights dur - ing propagations,

M. Courbariaux, Y . Bengio, and J. P . David, “BinaryCon- nect: Training deep neural networks with binary weights dur - ing propagations,” in Advances in neural information process- ing systems, 2015, pp. 3123–3131

2015
[22]

Recurrent neural networks with limited nu- merical precision,

Joachim Ott, Zhouhan Lin, Ying Zhang, Shih-Chii Liu, an d Y oshua Bengio, “Recurrent neural networks with limited nu- merical precision,” arXiv preprint arXiv:1608.06902, 2016

2016 arXiv
[23]

Binarized neural networks,

I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y . Bengio, “Binarized neural networks,” in Advances in neural information processing systems, 2016, pp. 4107–4115

2016
[24]

Xn or- net: Imagenet classiﬁcation using binary convolutional ne u- ral networks,

M. Rastegari, V . Ordonez, J. Redmon, and A. Farhadi, “Xn or- net: Imagenet classiﬁcation using binary convolutional ne u- ral networks,” in European Conference on Computer Vision . Springer, 2016, pp. 525–542

2016
[25]

Analysis of high-performance ﬂoating-point arithmetic on fpgas,

G. Govindu, L. Zhuo, S. Choi, and V . Prasanna, “Analysis of high-performance ﬂoating-point arithmetic on fpgas,” in null. IEEE, 2004, p. 149b

2004
[26]

Embedded ﬂoating-point units in fpgas,

M. J. Beauchamp, S. Hauck, K. D. Underwood, and K. S. Hem- mert, “Embedded ﬂoating-point units in fpgas,” in Proceed- ings of the 2006 ACM/SIGDA 14th international symposium on Field programmable gate arrays. ACM, 2006, pp. 12–20

2006
[27]

Fixed-point feedforward deep neu ral network design using weights+ 1, 0, and- 1,

K. Hwang and W. Sung, “Fixed-point feedforward deep neu ral network design using weights+ 1, 0, and- 1,” in Signal Pro- cessing Systems (SiPS), 2014 IEEE W orkshop on. IEEE, 2014, pp. 1–6

2014
[28]

Training de ep neural networks with low precision multiplications,

M. Courbariaux, Y . Bengio, and J. P . David, “Training de ep neural networks with low precision multiplications,” arXiv preprint arXiv:1412.7024, 2014

2014 arXiv
[29]

Trained ternary quantization,

C. Zhu, S. Han, H. Mao, and W. J. Dally, “Trained ternary quantization,” arXiv preprint arXiv:1612.01064, 2016

2016 arXiv
[30]

Bitwise neural networks,

M. Kim and P . Smaragdis, “Bitwise neural networks,” in Inter- national Conference on Machine Learning (ICML) W orkshop on Resource-Efﬁcient Machine Learning , Jul 2015

2015
[31]

Least squares quantization in PCM,

S. Lloyd, “Least squares quantization in PCM,” IEEE transac- tions on information theory , vol. 28, no. 2, pp. 129–137, 1982

1982
[32]

Online PLCA for real-time semi-supervised source separation,

Z. Duan, G. J. Mysore, and P . Smaragdis, “Online PLCA for real-time semi-supervised source separation,” in International Conference on Latent V ariable Analysis and Signal Separation. Springer, 2012, pp. 34–41

2012
[33]

Performanc e mea- surement in blind audio source separation,

E. Vincent, R. Gribonval, and C. F´ evotte, “Performanc e mea- surement in blind audio source separation,” IEEE transactions on audio, speech, and language processing , vol. 14, no. 4, pp. 1462–1469, 2006

2006
[34]

A short-time objective intelligibility measure for time-fr equency weighted noisy speech,

C. H. Taal, R. C. Hendriks, R. Heusdens, and J. Jensen, “A short-time objective intelligibility measure for time-fr equency weighted noisy speech,” in Acoustics Speech and Signal Pro- cessing (ICASSP), 2010 IEEE International Conference on . IEEE, 2010, pp. 4214–4217

2010

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

INTRODUCTION Neural network-based approaches to source separation task s have been becoming more prevalent [1, 2, 3]. Fully connected deep neu- ral networks (DNN) have shown to be capable of learning compl ex mapping functions from a large set of noisy signals and their corre- sponding ideal binary mask (IBM) target outputs [4, 5, 6]. Re current neural ne...

work page

[2] [2]

Background: Bitwise Neural Networks Binarization has been explored as a method of network compression

BITWISE GATED RECURRENT UNITS (BGRU) 2.1. Background: Bitwise Neural Networks Binarization has been explored as a method of network compression. BinaryConnect [17], binarized neural networks [19], train ed ternary quantization [25], and Bitwise Neural Networks (BNN) [26] h ave implemented a binarized or ternarized neural network in bip olar bi- naries (wi...

work page

[3] [3]

Experimental Setups For the experiment, we randomly subsample 12 speakers for tr ain- ing and 4 speakers for testing from the TIMIT corpus

EXPERIMENTS 3.1. Experimental Setups For the experiment, we randomly subsample 12 speakers for tr ain- ing and 4 speakers for testing from the TIMIT corpus. For both subsamples, we select half of the speakers as male and the oth er half as female. There are 10 short utterances per speaker rec orded with a 16kHz sampling rate. Each utterances are mixed wit...

work page

[4] [4]

The training is done in two roun ds, ﬁrst in a weight compressed network and then in an incrementa lly bitwise version with the same topology

CONCLUSION In this paper, we proposed an incremental binarization proc edure to binarize a RNN with GRU cells. The training is done in two roun ds, ﬁrst in a weight compressed network and then in an incrementa lly bitwise version with the same topology. The pretrained weig hts of the ﬁrst round are used to initialize the weights of the bitwi se net- work....

work page

[5] [5]

An experimental st udy on speech enhancement based on deep neural networks,

Y . Xu, J. Du, L.-R. Dai, and C.-H. Lee, “An experimental st udy on speech enhancement based on deep neural networks,” IEEE Signal processing letters, vol. 21, no. 1, pp. 65–68, 2014

work page 2014

[6] [6]

Joint optimization of masks and deep recurrent neural net- works for monaural source separation,

P . Huang, M. Kim, M. Hasegawa-Johnson, and P . Smaragdis, “Joint optimization of masks and deep recurrent neural net- works for monaural source separation,” IEEE/ACM Transac- tions on Audio, Speech, and Language Processing , vol. 23, no. 12, pp. 2136–2147, 2015

work page 2015

[7] [7]

Multichannel au- dio source separation with deep neural networks.,

A. A. Nugraha, A. Liutkus, and E. Vincent, “Multichannel au- dio source separation with deep neural networks.,” IEEE/ACM Trans. Audio, Speech & Language Processing , vol. 24, no. 9, pp. 1652–1664, 2016

work page 2016

[8] [8]

Towards scaling up classiﬁcation- based speech separation,

Y . Wang and D. Wang, “Towards scaling up classiﬁcation- based speech separation,” IEEE Transactions on Audio, Speech, and Language Processing , vol. 21, no. 7, pp. 1381– 1390, 2013

2013