Salient Speech Representations Based on Cloned Networks

Felicia S. C. Lim; Jan Skoglund; Michael Chinen; W. Bastiaan Kleijn

REVIEW 4 major objections 4 minor 38 references

Salient Speech Representations Based on Cloned Networks

T0 review · 4 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Clone-based training extracts a 12-dimensional salient speech feature set that outperforms PCA as WaveNet conditioning, especially under noise.

desk verdict A clean formulation of clone-based salient features with a sensible toy experiment, but the paper's central listening-test claim is unverifiable as written because the figure is absent and no numeric scores are reported. read the letter →

arxiv 1908.07045 v1 pith:EYQGZ4B3 submitted 2019-08-19 eess.AS cs.SD

W. Bastiaan Kleijn , Felicia S. C. Lim , Michael Chinen , Jan Skoglund This is my paper · ORCID

classification eess.AScs.SD

keywords salientfeaturesclone-basedtrainingSiamesenetworksmaximummeandiscrepancyspeechrepresentationlearningWaveNetconditioningenhancementcoding

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces a definition of salient features: a feature is salient if it is shared by all signals a designer declares equivalent. To find such features without labels, the authors train several copies (clones) of one encoder on different equivalent signals and drive their outputs to coincide, while also encouraging independent components and, optionally, reconstructing a clean target. The claim is that this weak, qualitative supervision is enough: in a listening test, 12 clone-learned features conditioned a WaveNet speech synthesizer more naturally than 12 PCA features, with the gap largest under additive noise. If true, the method offers a way to build compact, distortion-robust speech representations for coding and enhancement without hand-engineered features.

What carries the argument

The load-bearing mechanism is clone-based training: $Q$ copies of the same encoder network, with identical weights, each receive one member of an equivalence set, such as clean speech plus 32 noisy versions at 0 to 10 dB signal-to-noise ratio. The objective (4) combines three terms: a squared-error term (5) forcing the clones' feature vectors to agree; a maximum mean discrepancy (MMD) term (6)-(7) pushing the feature distribution toward an iid Laplacian, which encourages independence and a prescribed variance; and an optional decoder term mapping the shared features to a clean target. During training a small Gaussian perturbation is added to the encoder output to enforce smoothness, then removed at inference.

What would settle it

Take the trained clone encoder and test it on noise types and signal-to-noise ratios outside the training set, such as babble at negative signal-to-noise ratios or music at high signal-to-noise ratios; if clone-conditioned WaveNet stops beating the PCA-conditioned baseline there, or if features shared across arbitrary random distortions are just as effective as features shared across the 0 to 10 dB noisy set, the saliency claim collapses because equivalence, not noise robustness, is doing the work.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that an encoder trained by the clone objective of equations (4) through (7) extracts a 12-dimensional feature sequence that is invariant across a designer-chosen equivalence set of noisy versions of an utterance, and that this sequence carries enough phonetic, speaker, and prosodic information to condition WaveNet. Including a decoder that maps the shared features to the clean log-mel spectrogram substantially reduces short-phoneme errors, and the dual-window input variant brings noisy-condition naturalness close to the clean-input case. The authors conclude that clone-based training defines saliency qualitatively and produces a representation that is inherently robust to distortion, significantly outperforming a PCA reference in listening tests, particularly under noisy conditions.

Load-bearing premise

The load-bearing premise is that 32 noisy variants of the same utterance at 0 to 10 dB signal-to-noise ratio, plus the clean version, define the equivalence relation that matches what listeners care about, so that forcing clones to share features across these variants yields perceptually salient features rather than merely noise-invariant ones.

Editorial extensions

If this is right

The clone objective turns a designer's choice of equivalent signals into a training curriculum, so the same encoder architecture can be redirected to new notions of saliency by changing the clone input set.
With the decoder term active, the shared features suppress short-phoneme errors enough that noisy-input synthesis approaches clean-input quality, which is the property a speech-enhancement front end needs.
Because only 12 features per frame are needed, the representation is a natural fit for low-rate speech coding when paired with a generative decoder.
The improvements over PCA emerge most clearly under noisy conditions, indicating the clone objective is selecting distortion-invariant information rather than simply compressing the average spectrum.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

This suggests the same machinery could extract controllable factors of speech by choosing equivalence sets for speaker identity, emotion, or room acoustics, yielding voice-conversion or style-transfer features the paper does not itself build.
A natural next experiment is to quantize the 12-dimensional features and measure bit-rate versus quality, since the paper demonstrates conditioning quality but does not address quantization or entropy coding.
One could also test whether the independence term, not just the shared-features term, is what drives the gains by ablating $\lambda_f$ on the real speech task; the paper only varies the decoder and windowing.
If saliency is defined by the designer's equivalence set, then the method's ceiling is set by how well that set captures listener judgment, so systematic comparison of different equivalence sets would map the method's limits.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The thing to know: this is a conference-length paper with a genuinely new training recipe for extracting salient features, but its headline claim rests on a listening-test figure that is not present in the manuscript, and no scores or significance tests are given. Read it as a method proposal with a promising toy result, not as an established performance result.

What is new and what it does well: the paper defines salient features as features shared across a designer-chosen set of equivalent signals, then trains Q encoder clones with tied weights to map those equivalent signals to the same feature vector. It adds an MMD term to encourage independence and an optional decoder to reconstruct a target. Applying this to WaveNet conditioning for robust speech coding is a sensible, practical application. The objective in equations (4)-(7) is clearly stated. The toy experiment is genuinely informative: it shows the clone objective can recover and disentangle two nonlinearly mixed formant variables, even when the desired MMD distribution is mismatched. The use of 32 noisy clones plus a clean target for the decoder is a reasonable design.

The soft spots, in proportion: the load-bearing weakness is Section 3.3.2. The text says the clone-based systems 'significantly outperformed the reference system,' but the cited Figure 3 is not in the provided manuscript, and there are no MUSHRA scores, confidence intervals, or significance tests. That is not a minor omission. The entire real-world claim depends on those numbers, and without them a reader cannot tell whether SalientS-dw-noisy actually beats PCA12 or just sounds different. The baseline is PCA, not another learned representation, which is acceptable for a first demonstration but limits how strongly the result can be stated. Also, the equivalence set of clean plus 0-10 dB noisy versions is a defensible choice, but it only yields perceptually salient features if listeners actually care about the shared structure; the missing listening-test data are the only evidence for that link. The Ds term does make the shared-feature outcome partly tautological, but the useful claim is the full pipeline with the decoder, so that is a minor concern. No code or trained models are provided, which compounds the reproducibility problem.

This paper is for people working on speech representation learning, robust coding, and generative conditioning. It deserves a serious referee, but only with the explicit request that the authors supply the listening-test figure, per-condition scores, and significance analysis. I would not cite it as a demonstrated performance result until that evidence appears.

Referee Report

4 major / 4 minor

Summary. The paper introduces a definition of salient features as features shared across signals declared equivalent by a system designer, and proposes a clone-based training procedure in which identical encoder networks (Siamese-style clones) are trained on different equivalent inputs to produce identical feature vectors. The objective combines three terms: a shared-feature similarity term (Eq. 5), an independence/prior term implemented with MMD (Eqs. 6-7), and an optional decoder term that reconstructs a target signal. The authors present a toy experiment on synthetic formant-like data and a real-world experiment in which 12 clone-extracted features are used as WaveNet conditioning for speech coding/enhancement. They report that the clone-based features significantly outperform a PCA-based reference in a MUSHRA-like listening test.

Significance. If the empirical claims are substantiated, the paper makes a useful conceptual contribution: it turns the notion of salient features into an operational objective that allows a designer to specify equivalence classes, and it demonstrates a concrete application to speech conditioning. The mathematical formulation is clear, the use of MMD for independence is well grounded, and the toy experiment provides a plausible proof of concept. However, the central real-world claim currently rests on a listening-test figure that is absent from the manuscript and on no reported numeric scores, confidence intervals, or significance tests; the paper also compares against only a linear PCA baseline. These gaps prevent the reader from verifying the main claim.

major comments (4)

[Section 3.3.2, Fig. 3] The claim that clone-based systems 'significantly outperformed the reference system' is supported only by reference to Fig. 3, which is not included in the manuscript and is not accompanied by any numeric MUSHRA-like scores, per-condition means, confidence intervals, or significance-test results. Because this is the only real-world evidence for the central claim, the claim is not verifiable or reproducible from the manuscript as submitted. Please add a results table or figure with exact scores, variability measures, and a statistical comparison against PCA12.
[Section 3.3.1, Section 3.3.2] The only baseline is PCA12, a linear feature extractor. Without a comparison to a learned nonlinear representation (e.g., an autoencoder or VAE trained on the same inputs) or to direct WaveNet conditioning on the log-mel features, the reported improvement could be due to nonlinearity rather than to the clone-based saliency objective. Please add at least one learned baseline to isolate the contribution of the shared-feature objective.
[Section 2.2, Eq. (5)] The similarity term Ds directly penalizes differences between clone outputs, so the fact that equivalent signals are mapped to similar features is largely a consequence of the objective by construction. The substantive claim is that these shared features are perceptually salient, which must be established by the listening test. Since the listening-test results are not quantitatively reported, the link between the objective and perceptual saliency is currently unsupported.
[Section 3.2.2, Fig. 2] The toy experiment is presented only as a 'typical visual result' with no quantitative measure of disentanglement, smoothness, or injectivity. The text states that the extracted features are disentangled and that the formant structure is captured, but without a numerical evaluation (e.g., correlation or mutual information between ground-truth and extracted factors) the strength of this demonstration is limited. Please provide quantitative support or clearly label the toy result as illustrative.

minor comments (4)

[Section 2.1] The text contains typos such as 'distentangled' and 'speecph'; please proofread the manuscript.
[Section 1, Section 2.2] The reference list and text use inconsistent capitalization and formatting for terms such as 'Kulback-Leibler' (should be 'Kullback-Leibler') and 'hightlight'; please correct these.
[Section 3.1] The phrase 'The output uses as criterion an 2-norm error measure' should be 'an ℓ2-norm error measure' or 'a 2-norm error measure' for grammatical correctness.
[Section 3.1] The hyperparameters λf = 1, λd = 18, the number of clones Q = 32, the feature dimension 12, and the σϵ schedule are given without any sensitivity analysis; a sentence discussing robustness to these choices would strengthen the paper.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the clone objective is an operationalization of the paper's own definition of saliency, and the main empirical claim rests on an independent listening test.

full rationale

The paper stipulatively defines salient features as features shared across designer-supplied equivalent signals (Section 1), and the clone objective (4)-(5) directly trains encoders to output identical features across clones. This is an implementation of the definition, not a prediction derived from it; the paper never claims to prove that the trained features are salient as an empirical discovery. The load-bearing claim is that clone-based features 'significantly outperformed the reference system' in a MUSHRA-like listening test against PCA features (Section 3.3.2). That is an external evaluation of usefulness, not an algebraic consequence of the training objective, and it does not reduce to the Eq. (5) similarity term. No parameter is fitted to a subset of data and then renamed as a prediction; the toy experiment checks recovery of known ground-truth shared formants, which is a controlled sanity test with independent ground truth. The only self-citations (e.g., [18] for the perceptual irrelevance of waveform information, and [9] for variational autoencoder variants) are motivational or background and not load-bearing for the clone method. One reporting issue, unrelated to circularity, is that Figure 3 (listening test results) is not present in the provided manuscript and no numeric scores, confidence intervals, or significance tests are reported, so the claimed 'significant' improvement over PCA cannot be verified from the text; this is a reproducibility shortfall, not a circular derivation.

Assumptions & free parameters 8 free parameters · 5 assumptions · 0 invented entities

No new physical entities are introduced. The free parameters are training hyperparameters, the feature dimensionality, the MMD target distribution, and the unspecified kernel. The main domain assumption is that noise-corrupted versions of the same utterance are equivalent signals, which carries the saliency definition into training.

free parameters (8)

lambda_f (independence weight) = 1
Weight of the MMD term in objective (4); chosen by hand, no sweep reported.
lambda_d (decoder weight) = 18
Weight of the decoder term in objective (4); chosen by hand, no sweep reported.
sigma_epsilon noise schedule = start 0.2, decay exponent 0.98 per 1000 steps
Stochastic encoder noise in equation (3); schedule chosen manually.
Number of salient features = 12
Dimensionality of the feature vector z; designer-chosen output size of the last layer.
Number of clones Q = 32
Number of equivalent signals per training instance; hand-chosen.
Desired feature distribution for MMD = iid Laplacian
MMD target distribution in equation (6); chosen anticipating sparse features, no kernel specified.
MMD kernel k(.,.) = not specified
Kernel choice matters for the empirical MMD estimator (7) but is never named in the paper.
Toy formant variance bounds b_Gamma1 and b_Gamma2 = 0.01 and 0.005
Toy data parameters in equation (8); chosen to demonstrate disentanglement, not fitted.

assumptions (5)

standard math The MMD estimator (7) is a valid unbiased estimator of squared MMD for the chosen RKHS kernel.
Invoked in Section 2.2 to measure the distance between the feature distribution and the desired Laplacian distribution.
domain assumption Noisy and clean versions of the same utterance are equivalent signals for the system designer.
Section 3.3.1 defines equivalence sets as 32 noise-corrupted copies plus the clean utterance; this is the main carrier of the saliency definition.
domain assumption The encoder and decoder mappings f_phi and g_phi are surjective, allowing the feature vector to take an arbitrary prescribed distribution.
Section 2.1 uses surjectivity to justify enforcing the desired feature distribution and independence.
domain assumption Matching the feature distribution to an iid Laplacian via MMD produces independent or disentangled features.
Section 2.2 chooses the Laplacian target to encourage sparsity and independence; the toy experiment notes the desired distribution is mismatched yet still works.
domain assumption A 12-dimensional conditioning vector is sufficient for WaveNet to generate natural-sounding, noise-free speech.
Section 3.3 uses 12 features as conditioning for WaveNet; no capacity analysis is given.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Salient Speech Representations Based on Cloned Networks." pith.science (2026). https://pith.science/paper/EYQGZ4B3

@misc{pith2026190807045,
  author       = {Pith},
  title        = {Pith review of: Salient Speech Representations Based on Cloned Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EYQGZ4B3}},
  note         = {Machine review of arXiv:1908.07045}
}

read the original abstract

We define salient features as features that are shared by signals that are defined as being equivalent by a system designer. The definition allows the designer to contribute qualitative information. We aim to find salient features that are useful as conditioning for generative networks. We extract salient features by jointly training a set of clones of an encoder network. Each network clone receives as input a different signal from a set of equivalent signals. The objective function encourages the network clones to map their input into a set of features that is identical across the clones. It additionally encourages feature independence and, optionally, reconstruction of a desired target signal by a decoder. As an application, we train a system that extracts a time-sequence of feature vectors of speech and uses it as a conditioning of a WaveNet generative system, facilitating both coding and enhancement.

Figures

Figures reproduced from arXiv: 1908.07045 by the authors.

**Figure 1.** Clone-based training setup. Red arrows hightlight when only the first clone is used. The decoders are optional. measure of spectral amplitude for a frequency region where the signal-to-noise ratio is high. Note that this process naturally leads to disentanglement. Various approaches can be used to encourage the features to be independent, and have a given variance, some requiring a desired distribution. Examples are… view at source ↗

**Figure 3.** shows the result of the listening test. Clone-based learning without a decoder with a single window (SalientU-sw) provides natural speech quality with good speaker identity but with fairly frequent errors for phonemes of short duration. As expected, the number of errors is lower for the clean (SalientUsw-clean) than for noisy input signals. Learning with a decoder with a single window (SalientS-sw) reduces the e… view at source ↗

**Figure 2.** Ground-truth features (blue) and extracted salient features (green) for the toy experiment. Colored lines connect nodes that were neighbors in the ground truth, and black lines connect ground truth to extracted features. 3.3. WaveNet experiments The WaveNet [23] algorithm requires conditioning to produce natural-sounding speech. Our goal was to extract 12 salient features that can be used as conditioning to produce… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 24 canonical work pages

[1]

Salient Speech Representations Based on Cloned Networks

Introduction In speech processing, as well as other applications, it is valuable to ﬁnd a meaningful representation that summarizes the salient attributes of a signal. We deﬁne a salient feature set as a fea- ture set that is shared by signals that are judged to be equivalent by a user. This deﬁnition allows the user to contribute qualita- tive knowledge ...

work page Pith review arXiv 1908
[2]

We then describe clone-based training proce- dures for the method in 2.2

Extracting Salient Features In this section we ﬁrst motivate and describe the basic extraction network in 2.1. We then describe clone-based training proce- dures for the method in 2.2. We denote random variables (RVs) with upper case and realizations with lower case font. 2.1. The encoder network Our objective is to extract salient representations that ca...

work page
[3]

Signals that are equivalent result in (almost) identical features

work page
[4]

Different regions of feature space can be identiﬁed with different signal attributes, render- ing the representation meaningful

The map is smooth. Different regions of feature space can be identiﬁed with different signal attributes, render- ing the representation meaningful

work page
[5]

Ideally they are distentangled: the system then discovers a ’natural’ set of independent features corre- sponding to ground-truth factors [10]

The components have a prescribed variance and are inde- pendent. Ideally they are distentangled: the system then discovers a ’natural’ set of independent features corre- sponding to ground-truth factors [10]. The ﬁrst desired attribute requires a surjective mapping from the speech signal to a salient feature set. We show it can be obtained by training clo...

work page
[6]

The shared features are found by encouraging the de- terministic mapping fφ to result in outputs that are maximally similar for all clones, despite their different inputs

(5) An alternative is the 1-norm and we can add cross terms for all clones. The shared features are found by encouraging the de- terministic mapping fφ to result in outputs that are maximally similar for all clones, despite their different inputs. The method preferably selects features describing information components that are shared between the clone in...

work page
[7]

Experimental Results This section provides results for a toy experiment and real- world data. 3.1. System setup The same basic conﬁguration is used for the toy and real-world experiments. We ﬁrst describe the real-world setup and then note the difference with the toy experiments. For the real-world system the encoder network is a stack of two LSTM layers ...

work page
[8]

From ex- periments with a toy example, we conclude that clone-based training can be used to disentangle formants from a signal

Conclusion We showed that clone-based training allows saliency to be de- ﬁned in a qualitative manner by a system designer. From ex- periments with a toy example, we conclude that clone-based training can be used to disentangle formants from a signal. In a real-world application, the addition of a decoder improved the performance of the clone-based featur...

work page

Show all 38 references

[9]

Infogan: Interpretable representation learning by in- formation maximizing generative adversarial nets,

X. Chen, Y . Duan, R. Houthooft, J. Schulman, I. Sutskever, and P. Abbeel, “Infogan: Interpretable representation learning by in- formation maximizing generative adversarial nets,” in Advances in Neural Information Processing Systems, 2016, pp. 2172–2180

2016
[10]

Auto-encoding variational Bayes,

D. P. Kingma and M. Welling, “Auto-encoding variational Bayes,” arXiv preprint arXiv:1312.6114, 2013

2013 arXiv
[11]

Stochastic back- propagation and approximate inference in deep generative mod- els,

D. J. Rezende, S. Mohamed, and D. Wierstra, “Stochastic back- propagation and approximate inference in deep generative mod- els,”arXiv preprint arXiv:1401.4082, 2014

2014 arXiv
[12]

Variational lossy au- toencoder,

X. Chen, D. P. Kingma, T. Salimans, Y . Duan, P. Dhariwal, J. Schulman, I. Sutskever, and P. Abbeel, “Variational lossy au- toencoder,”arXiv preprint arXiv:1611.02731, 2016

2016 arXiv
[13]

InfoV AE: Information maximiz- ing variational autoencoders,

S. Zhao, J. Song, and S. Ermon, “InfoV AE: Information maximiz- ing variational autoencoders,” arXiv preprint arXiv:1706.02262 , 2017

2017 arXiv
[14]

An information-theoretic analysis of deep latent- variable models,

A. A. Alemi, B. Poole, I. Fischer, J. V . Dillon, R. A. Saurous, and K. Murphy, “An information-theoretic analysis of deep latent- variable models,”arXiv preprint arXiv:1711.00464, 2017

2017 arXiv
[15]

The mutual autoencoder: Control- ling information in latent code representations,

M. Phuong, M. Welling, N. Kushman, R. Tomioka, and S. Nowozin, “The mutual autoencoder: Control- ling information in latent code representations,” 2017, https://openreview.net/forum?id=HkbmWqxCZ

2017
[16]

Is maximum likelihood useful for representation learning?

F. Huszar, “Is maximum likelihood useful for representation learning?” 2017, ttp://www.inference.vc/maximum-likelihood- for-representation-learning-2/

2017
[17]

Bounded information rate vari- ational autoencoders,

D. Braithwaite and W. B. Kleijn, “Bounded information rate vari- ational autoencoders,”arXiv preprint arXiv:1807.07306, 2018

2018 arXiv
[18]

Representation learning: A review and new perspectives,

Y . Bengio, A. Courville, and P. Vincent, “Representation learning: A review and new perspectives,” IEEE transactions on pattern analysis and machine intelligence, vol. 35, no. 8, pp. 1798–1828, 2013

2013
[19]

The variational fair autoencoder,

C. Louizos, K. Swersky, Y . Li, M. Welling, and R. Zemel, “The variational fair autoencoder,” arXiv preprint arXiv:1511.00830 , 2015

2015 arXiv
[20]

β-V AE: Learning basic visual concepts with a constrained variational framework,

I. Higgins, L. Matthey, A. Pal, C. Burgess, X. Glorot, M. Botvinick, S. Mohamed, and A. Lerchner, “β-V AE: Learning basic visual concepts with a constrained variational framework,” in International Conference on Learning Representations, 2017

2017
[21]

Understanding disentangling in β- V AE,

C. P. Burgess, I. Higgins, A. Pal, L. Matthey, N. Watters, G. Des- jardins, and A. Lerchner, “Understanding disentangling in β- V AE,”arXiv preprint arXiv:1804.03599, 2018

2018 arXiv
[22]

Structured disen- tangled representations,

B. Esmaeili, H. Wu, S. Jain, A. Bozkurt, N. Siddharth, B. Paige, D. H. Brooks, J. Dy, and J.-W. van de Meent, “Structured disen- tangled representations,”stat, vol. 1050, p. 29, 2018

2018
[23]

Disentangling by factorising,

H. Kim and A. Mnih, “Disentangling by factorising,” arXiv preprint arXiv:1802.05983, 2018

2018 arXiv
[24]

The information bottle- neck method,

N. Tishby, F. C. Pereira, and W. Bialek, “The information bottle- neck method,”arXiv preprint physics/0004057, 2000

2000 arXiv
[25]

Deep varia- tional information bottleneck,

A. A. Alemi, I. Fischer, J. V . Dillon, and K. Murphy, “Deep varia- tional information bottleneck,” arXiv preprint arXiv:1612.00410, 2016

2016 arXiv
[26]

WaveNet based low rate speech cod- ing,

W. B. Kleijn, F. S. Lim, A. Luebs, J. Skoglund, F. Stimberg, Q. Wang, and T. C. Walters, “WaveNet based low rate speech cod- ing,”arXiv preprint arXiv:1712.01120, 2017

2017 arXiv
[27]

Extraction of relevant speech fea- tures using the information bottleneck method,

R. M. Hecht and N. Tishby, “Extraction of relevant speech fea- tures using the information bottleneck method,” in Ninth Eu- ropean Conference on Speech Communication and Technology , 2005

2005
[28]

Sig- nature veriﬁcation using a

J. Bromley, I. Guyon, Y . LeCun, E. S¨ackinger, and R. Shah, “Sig- nature veriﬁcation using a ”Siamese” time delay neural network,” in Advances in Neural Information Processing Systems, 1994, pp. 737–744

1994
[29]

Deep metric learning using triplet net- work,

E. Hoffer and N. Ailon, “Deep metric learning using triplet net- work,” in International Workshop on Similarity-Based Pattern Recognition. Springer, 2015, pp. 84–92

2015
[30]

Deep convolutional acoustic word embeddings using word-pair side information,

H. Kamper, W. Wang, and K. Livescu, “Deep convolutional acoustic word embeddings using word-pair side information,” in Acoustics, Speech and Signal Processing (ICASSP), 2016 IEEE International Conference on. IEEE, 2016, pp. 4950–4954

2016
[31]

WaveNet: A Generative Model for Raw Audio,

A. van den Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, and K. Kavukcuoglu, “WaveNet: A Generative Model for Raw Audio,” ArXiv e-prints, Sep. 2016

2016
[32]

A kernel method for the two-sample-problem,

A. Gretton, K. M. Borgwardt, M. Rasch, B. Sch ¨olkopf, and A. J. Smola, “A kernel method for the two-sample-problem,” in Ad- vances in neural information processing systems, 2007, pp. 513– 520

2007
[33]

A kernel two-sample test,

A. Gretton, K. M. Borgwardt, M. J. Rasch, B. Sch ¨olkopf, and A. Smola, “A kernel two-sample test,”Journal of Machine Learn- ing Research, vol. 13, no. Mar, pp. 723–773, 2012

2012
[34]

Wasserstein generative adversarial networks,

M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein generative adversarial networks,” in International Conference on Machine Learning, 2017, pp. 214–223

2017
[35]

Learning with a Wasserstein loss,

C. Frogner, C. Zhang, H. Mobahi, M. Araya, and T. A. Poggio, “Learning with a Wasserstein loss,” in Advances in Neural Infor- mation Processing Systems, 2015, pp. 2053–2061

2015
[36]

Long short-term memory,

S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation, vol. 9, no. 8, pp. 1735–1780, 1997

1997
[37]

Adam: A method for stochastic opti- mization,

D. P. Kingma and J. Ba, “Adam: A method for stochastic opti- mization,”arXiv preprint arXiv:1412.6980, 2014

2014 arXiv
[38]

The design for the wall street journal- based csr corpus,

D. B. Paul and J. M. Baker, “The design for the wall street journal- based csr corpus,” inProceedings of the workshop on Speech and Natural Language. Association for Computational Linguistics, 1992, pp. 357–362

1992

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Salient Speech Representations Based on Cloned Networks

Introduction In speech processing, as well as other applications, it is valuable to ﬁnd a meaningful representation that summarizes the salient attributes of a signal. We deﬁne a salient feature set as a fea- ture set that is shared by signals that are judged to be equivalent by a user. This deﬁnition allows the user to contribute qualita- tive knowledge ...

work page Pith review arXiv 1908

[2] [2]

We then describe clone-based training proce- dures for the method in 2.2

Extracting Salient Features In this section we ﬁrst motivate and describe the basic extraction network in 2.1. We then describe clone-based training proce- dures for the method in 2.2. We denote random variables (RVs) with upper case and realizations with lower case font. 2.1. The encoder network Our objective is to extract salient representations that ca...

work page

[3] [3]

Signals that are equivalent result in (almost) identical features

work page

[4] [4]

Different regions of feature space can be identiﬁed with different signal attributes, render- ing the representation meaningful

The map is smooth. Different regions of feature space can be identiﬁed with different signal attributes, render- ing the representation meaningful

work page

[5] [5]

Ideally they are distentangled: the system then discovers a ’natural’ set of independent features corre- sponding to ground-truth factors [10]

The components have a prescribed variance and are inde- pendent. Ideally they are distentangled: the system then discovers a ’natural’ set of independent features corre- sponding to ground-truth factors [10]. The ﬁrst desired attribute requires a surjective mapping from the speech signal to a salient feature set. We show it can be obtained by training clo...

work page

[6] [6]

The shared features are found by encouraging the de- terministic mapping fφ to result in outputs that are maximally similar for all clones, despite their different inputs

(5) An alternative is the 1-norm and we can add cross terms for all clones. The shared features are found by encouraging the de- terministic mapping fφ to result in outputs that are maximally similar for all clones, despite their different inputs. The method preferably selects features describing information components that are shared between the clone in...

work page

[7] [7]

Experimental Results This section provides results for a toy experiment and real- world data. 3.1. System setup The same basic conﬁguration is used for the toy and real-world experiments. We ﬁrst describe the real-world setup and then note the difference with the toy experiments. For the real-world system the encoder network is a stack of two LSTM layers ...

work page

[8] [8]

From ex- periments with a toy example, we conclude that clone-based training can be used to disentangle formants from a signal

Conclusion We showed that clone-based training allows saliency to be de- ﬁned in a qualitative manner by a system designer. From ex- periments with a toy example, we conclude that clone-based training can be used to disentangle formants from a signal. In a real-world application, the addition of a decoder improved the performance of the clone-based featur...

work page