Pith. sign in

REVIEW 4 major objections 4 minor 28 references

Factorized RVQ-GAN For Disentangled Speech Tokenization

T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper introduces Hierarchical Audio Codec (HAC), which factorizes a neural codec's bottleneck into disentangled acoustic, phonetic, and lexical token sets, and argues these can coexist in one model without sacrificing reconstruction…

desk verdict A genuinely new lexical token stream in a codec, with solid but indirect disentanglement evidence and an overstated abstract. read the letter →

arxiv 2506.15456 v1 pith:FXTFWQJW submitted 2025-06-18 eess.AS cs.CLcs.SD

Sameer Khurana , Dominik Klement , Antoine Laurent , Dominik Bobos , Juraj Novosad , Peter Gazdik , Ellen Zhang , Zili Huang
show 8 more authors
Amir Hussein Ricard Marxer Yoshiki Masuyama Ryo Aihara Chiori Hori Francois G. Germain Gordon Wichern Jonathan Le Roux
This is my paper · ORCID
classification eess.AScs.CLcs.SD
keywords neuralaudiocodecspeechtokenizationresidualvectorquantizationknowledgedistillationdisentangledrepresentationphonetictokenslexicalmultilingual
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces Hierarchical Audio Codec (HAC), a neural speech codec that factorizes its discrete token bottleneck into three levels: acoustic, phonetic, and lexical. It trains the phonetic and lexical branches by distillation from self-supervised speech and text encoders, leaving the acoustic residual vector-quantization branch to carry the remaining detail. The claim is that the resulting token sets are disentangled: one aligns with phonemes, another with words, while the acoustic tokens preserve natural-sounding speech. This matters because a single codec could then supply discrete units for both speech generation and language understanding without stitching token streams from separate models.

What carries the argument

The load-bearing object is the factorized VQ bottleneck: three quantizers in parallel, one residual RVQ for acoustics and two plain VQ layers for phonetic and lexical content, whose outputs are summed before decoding. The mechanism that separates the levels is the pair of knowledge-distillation losses, which pull the phonetic and lexical codebook embeddings toward HuBERT and LaBSE (or SAMU) representations; the acoustic branch is left free to explain everything else. Transformer encoders before the phonetic and lexical VQ layers, plus the low-dimensional code lookup procedure from the underlying codec architecture, support the separation.

What would settle it

Take a trained HAC and measure word-detection F1 or phoneme-level mutual information on the acoustic token stream. If acoustic tokens predict words or phonemes nearly as well as the lexical or phonetic streams, the levels have not separated and the central claim fails.

Watch

Extended reading notes

Core claim

HAC's central discovery is that an additive bottleneck decomposition $Z_Q = Z_{Q_p} + Z_{Q_a} + Z_{Q_l}$ in a residual vector-quantization generative adversarial network, trained with two knowledge-distillation losses, produces token sets that isolate linguistic levels. The phonetic branch matches averaged HuBERT embeddings, the lexical branch matches LaBSE text embeddings of forced-aligned transcripts, and the acoustic branch, left unconstrained, learns the residual spectral detail. In the reported evaluations, phonetic tokens show phoneme alignment through ABX and phoneme-normalized mutual information, lexical tokens act as word detectors, and acoustic tokens show little lexical content while reconstruction quality remains comparable to a codec trained only for reconstruction.

Load-bearing premise

The central assumption is that training two of the three parallel quantizers to imitate speech and text encoders keeps the three token streams from overlapping, leaving the untrained acoustic stream with only the leftover acoustic detail.

Editorial extensions

If this is right

  • A single trained HAC can emit token streams for three abstraction levels from one forward pass, removing the need to merge or align tokens from separate codecs.
  • Downstream systems can choose token type by task: acoustic tokens for high-fidelity synthesis, phonetic tokens for phoneme-related structure, and lexical tokens for semantic and syntactic content.
  • Because SAMU can replace LaBSE as the lexical teacher, lexical tokens can be learned without text transcripts at inference, extending the approach to languages without forced alignments.
  • The additive residual formulation means the acoustic branch can be trained largely unsupervised while distillation guides the other branches, potentially easing extension to new languages or new linguistic levels.
  • Multilingual training results suggest the disentanglement transfers across languages, so one codec could serve speech understanding and generation for many languages.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An untested but direct consequence of the additive decomposition is ablatability: setting one branch to zero should remove exactly one linguistic level from the reconstruction, leaving the other levels mostly intact.
  • The three-level tokenization could serve as a controllable interface for generation, such as swapping lexical tokens to change content while keeping acoustic tokens to preserve voice, which the paper does not demonstrate.
  • The same distillation-into-a-factorized-bottleneck recipe could be applied to other teacher signals, such as prosody or speaker embeddings, to create further token dimensions within one codec.
  • A reader could directly estimate mutual information between the phonetic and lexical token streams; near-zero mutual information would confirm strong disentanglement, and the paper does not report such a measurement.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces Hierarchical Audio Codec (HAC), a neural speech codec with a factorized bottleneck intended to produce three disentangled token streams: acoustic, phonetic, and lexical. HAC is trained with a GAN-based reconstruction objective plus two knowledge-distillation losses, one from HuBERT for phonetic structure and one from LaBSE/SAMU for lexical information. Experiments compare HAC with SpeechTokenizer and Descript Audio Codec variants on ABX discrimination, phoneme normalized mutual information (PNMI), word-detector F1, and reconstruction metrics on English and multilingual data. The central claim is that HAC yields token sets that align with phonemes and words, preserve speech naturalness, and outperform single-level baselines in disentanglement and reconstruction.

Significance. If substantiated, the HAC architecture would be a useful contribution: a single codec that supplies acoustic, phonetic, and lexical discrete units could serve speech language modeling, speech-to-speech translation, and spoken understanding. The paper's strengths include evaluation with external phoneme and word labels rather than teacher embeddings, which grounds the main results independently of the authors' own teacher model; the LaBSE-based variant reproduces the lexical effect without relying on SAMU; and the evaluation spans multiple languages. However, the central claim of full disentanglement is not yet supported: no explicit mechanism enforces separation among the three token streams, and several quantitative comparisons contradict the abstract's overstatement. These issues are fixable with additional diagnostics and revised claims, but they are load-bearing for the paper's contribution.

major comments (4)
  1. [Section 2 (architecture and training objective)] The factorization Z_Q = Z_Qp + Z_Qa + Z_Ql is presented without any explicit constraint enforcing information separation among the three codebook streams. RVQ_acoust is applied directly to Z, not to a residual after removing Z_Qp and Z_Ql, and the KD losses only align the phonetic and lexical branches to their respective teachers. Nothing in the objective prevents the acoustic codebook from encoding phonetic or lexical content. The layer-wise PNMI and word-F1 results show that the phonetic and lexical branches match their teachers, but they do not establish that the acoustic branch is linguistically empty or that the three token sets are informationally disjoint, which is precisely the paper's central claim.
  2. [Abstract, Table 1, Figures 2-3] The abstract states that HAC 'outperforms single-level baselines in both disentanglement and reconstruction quality,' but the reported numbers do not support this for the DAC baselines. In Table 1, DAC has lower Mel-D (0.55 vs. 0.58), lower STFT-D (1.34 vs. 1.37), higher SI-SDR (7.82 vs. 7.42), and higher ViSQOL (4.50 vs. 4.34) than HAC. In Figures 2 and 3, DAC models achieve lower ABX error and higher PNMI than HAC models. The abstract should be revised to accurately reflect that HAC trails DAC on several metrics and outperforms the ST baseline specifically, and that the claimed advantage is in lexical-word detection rather than in reconstruction or phoneme-level disentanglement.
  3. [Figure 4] The word-detector F1 comparison is confounded by codebook size. HAC's lexical VQ layer has a 14-bit codebook with 16,384 entries, while the DAC and ST baselines use 10-bit codebooks with 1,024 entries. Reporting the absolute number of tokens that exceed an F1 threshold therefore gives HAC a structural advantage. The authors should report the fraction of codebook entries that act as word detectors, or compare against a baseline with a matched codebook size, to support the claim that HAC's lexical branch is genuinely more lexically specialized rather than merely larger.
  4. [Section 3, Figure 5, Table 1] There is an inconsistency in model naming that makes the central results hard to interpret. Section 3 defines HAC-14-SAMU-HuB-T-en and HAC-14-LaBSE-HuB-T-en, but Figure 5 and Table 1 refer to an undefined model 'HAC-10-SAMU-HuB-T-en'. It is unclear whether this is a typo or a distinct configuration, and the reader cannot tell which model is actually being evaluated in the layer-wise PNMI and reconstruction comparisons. This should be clarified.
minor comments (4)
  1. [Figures 2-3 and Table 1] No confidence intervals or significance tests are reported for any of the quantitative comparisons, and several differences are small (e.g., Mel-D 0.55 vs. 0.58). Adding error bars or significance statements would strengthen the reliability of the claims.
  2. [Figure 6] The word-F1 analysis for the acoustic tokens shows that virtually no acoustic codebook entries act as word detectors, but no analogous phoneme-level analysis (e.g., PNMI of the acoustic tokens) is reported. Adding acoustic-token PNMI would directly address the concern that the acoustic branch may still carry phonetic information.
  3. [Section 3, word-detector evaluation] The word-detector evaluation is performed on English words only, while the multilingual HAC models are trained on 16 languages. The multilingual lexical claim would be better supported by at least a small multilingual word-detector or lexical-alignment evaluation.
  4. [Section 2, Eq. (3)-(4)] The KD losses average teacher representations over all layers, which is a coarse aggregation. The choice of layer averaging is not discussed, and a comparison of different teacher layer choices would help establish the robustness of the phonetic and lexical alignments.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: KD losses are training objectives, and the disentanglement claims are checked against external phoneme/word labels rather than the teacher embeddings themselves.

full rationale

The paper's central claim is that HAC's factorized bottleneck yields acoustic, phonetic, and lexical token sets. The phonetic and lexical branches are explicitly trained with KD losses to align to HuBERT and LaBSE/SAMU embeddings (Section 2), so observing that those branches correlate with phonemes and words is partly a check that the distillation objectives were optimized. However, the evaluations in Figures 3-6 use external phoneme labels, word labels, and ABX triples, not the teacher embeddings themselves, so the linguistic content is independently grounded. The acoustic branch receives no KD loss and is not constrained by any orthogonality term; its lack of word-detector tokens (Figure 6) is an empirical outcome rather than a fitted input. The SAMU teacher is cited from co-authored prior work, but the LaBSE-based HAC variant reproduces the lexical effect, so the self-citation is not load-bearing. The absence of an explicit orthogonality or mutual-information penalty is a potential architectural limitation, but that is a correctness/robustness concern, not circularity: the paper does not define disentanglement as 'trained by the KD objective' and then re-report the objective. No equation in the paper reduces a predicted quantity to a fitted parameter or to a self-cited uniqueness theorem.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on external teacher encoders, an additive codebook combination asserted without derivation, and several hand-chosen capacities and loss weights. No new physical or theoretical entities are introduced. The evaluation metrics use external phoneme and word labels, so the disentanglement result is not defined into existence by the teachers.

free parameters (3)
  • KD loss weighting coefficients
    The total objective in Section 2 lists L_KD-Phn and L_KD-Lex but never gives their scalar weights; these weights determine how strongly the codebook entries are pushed toward linguistic targets versus reconstruction, and are hand-chosen.
  • Teacher feature averaging mode = Avg over all layers
    The KD targets are averaged over all HuBERT and LaBSE layers following [12]; this choice is not ablated and fixes the reference signal the codebooks must match.
  • Codebook capacities = P=L=16384, A=1024, N=7
    The phonetic and lexical codebooks are 14-bit and the acoustic RVQ has seven 10-bit layers; these hand-selected sizes set the information balance between the three token streams and are not swept in any ablation.
assumptions (4)
  • domain assumption HuBERT embeddings carry phoneme-level structure and LaBSE/SAMU embeddings carry lexical semantics, so cosine-similarity distillation transfers these properties to discrete codebooks.
    Section 2 defines the KD losses against these pretrained teachers; the paper does not test whether matching these targets is sufficient to induce the intended structure in discrete tokens.
  • ad hoc to paper The decoder input Z_Q = Z_Qp + Z_Qa + Z_Ql is a valid additive factorization in which the three codebook streams remain informationally separable.
    The addition is asserted in Section 2 without orthogonality or interference arguments; the disentanglement claim depends on this decomposition.
  • domain assumption Forced alignments are accurate enough to give frame-level lexical supervision.
    LaBSE-based lexical KD uses force-aligned transcripts; alignment errors would corrupt the lexical token targets. The SAMU variant avoids transcripts but is itself a distilled speech encoder.
  • domain assumption ABX, PNMI, and word-detector F1 are sufficient evidence of disentanglement.
    These are standard evaluation metrics, but they measure pairwise associations only and do not establish that the acoustic branch is free of linguistic content or that the streams are independent.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Factorized RVQ-GAN For Disentangled Speech Tokenization." pith.science (2026). https://pith.science/paper/FXTFWQJW

@misc{pith2026250615456,
  author       = {Pith},
  title        = {Pith review of: Factorized RVQ-GAN For Disentangled Speech Tokenization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FXTFWQJW}},
  note         = {Machine review of arXiv:2506.15456}
}
read the original abstract

We propose Hierarchical Audio Codec (HAC), a unified neural speech codec that factorizes its bottleneck into three linguistic levels-acoustic, phonetic, and lexical-within a single model. HAC leverages two knowledge distillation objectives: one from a pre-trained speech encoder (HuBERT) for phoneme-level structure, and another from a text-based encoder (LaBSE) for lexical cues. Experiments on English and multilingual data show that HAC's factorized bottleneck yields disentangled token sets: one aligns with phonemes, while another captures word-level semantics. Quantitative evaluations confirm that HAC tokens preserve naturalness and provide interpretable linguistic information, outperforming single-level baselines in both disentanglement and reconstruction quality. These findings underscore HAC's potential as a unified discrete speech representation, bridging acoustic detail and lexical meaning for downstream speech generation and understanding tasks.

Figures

Figures reproduced from arXiv: 2506.15456 by the authors.

Figure 1
Figure 1. Diagram of our proposed Hierarchical Audio Codec (HAC). HAC encodes the input speech signal x into a multi￾level set of disentangled discrete tokens (lexical Ql, phonetic Qp, and acoustic Qa) capturing distinct aspects of the audio. speaker variation. Nonetheless, because A-NSCs focus on fine￾grained acoustic details, they often lack coherent linguistic and grammatical structure. Prominent examples of A-NSCs include… view at source ↗
Figure 2
Figure 2. ABX error rate across different models and scenarios [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Phoneme Normalized Mutual Information (PNMI) across different models and languages. netic VQ layer, allowing the VQ layer to capture richer con￾text before quantization. DAC-14-mHuB-T-en: This version uses multilingual HuBERT (mHuBERT) [18] as the teacher for phonetic KD loss. It follows the same architecture as DAC-14- HuB-T-en but replaces HuBERT-Base with a multilingual vari￾ant. DAC-14-mHuB-T-L16: A multilingual… view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Layer-wise Phoneme Normalized Mutual Information for different models. in DAC-14-HuB-T-en further reduces ABX error. HAC mod￾els show slightly higher ABX errors than DAC possibly be￾cause juggling both phoneme-level (LKD-Phn) and lexical-level (LKD-Lex) losses introduc…

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 24 canonical work pages

  1. [1]

    hj/iAePKhU/wEzYUdFoprleUHDo=

    Introduction Neural speech codecs (NSCs) are a family of neural network ar- chitectures that convert speech signals into discrete token repre- sentations [1–4]. These discrete tokens can then be leveraged in various downstream tasks, ranging from spoken language mod- eling [5] and speech-to-speech translation [6] to text-to-speech synthesis [7] and speech...

    work page

  2. [2]

    Hierarchical Audio Codec (HAC) HAC, illustrated in Fig. 1, consists of a down-sampling CNN encoder (ENC), two transformer encoders (TrfENC p,l), a fac- torized bottleneck consisting of three VQ modules (VQphn, RVQacoust, andVQlex), and an up-sampling CNN decoder (DEC). HAC is trained using tuples(x,y ali), wherex∈R T is a speech utterance andy ali is its ...

    work page Pith review arXiv 2025

  3. [3]

    These objectives are described in detail in [3]

    an adversarial loss to encourage natural-sounding outputs, and 3) codebook learning losses to update the codebook entries. These objectives are described in detail in [3]. To ensure that each token set encodes the intended type of information, we introduce knowledge distillation (KD) losses on the phonetic and lexical bottlenecks: 1)L KD-Phn encouragesQ p...

    work page

  4. [4]

    For evaluation, we use forced-aligned test sets from Lib- riSpeech and Multilingual LibriSpeech (MLS) [16]

    Experiments We train English-language models on the LibriSpeech dataset [14], consisting of 960 hours of transcribed English speech, and multilingual models on the V oxPopuli dataset [15], which in- cludes 1.7K hours of transcribed speech across 16 languages. For evaluation, we use forced-aligned test sets from Lib- riSpeech and Multilingual LibriSpeech (...

    work page

  5. [5]

    Like Encodec, ST updates its codebooks via exponential moving average (EMA) and period- ically re-initializes them to maximize utilization

    (an earlier RVQ-GAN for audio compression) by adding the Phoneme level KD loss to Encodec’s original generative and codebook learning objectives. Like Encodec, ST updates its codebooks via exponential moving average (EMA) and period- ically re-initializes them to maximize utilization. To have a fair comparison with other models, we train the baseline usin...

    work page

  6. [6]

    Conclusions This paper introduced Hierarchical Audio Codec (HAC), a fac- torized RVQ-GAN framework that unifies acoustic, phonetic, and lexical token sets within a single model. Through dedi- cated knowledge distillation losses from speech-focused (Hu- BERT) and text-based (LaBSE) encoders, HAC learns comple- mentary token groups at different levels of li...

    work page 2024

  7. [7]

    SoundStream: An end-to-end neural audio codec,

    N. Zeghidour, A. Luebs, A. Omran, J. Skoglund, and M. Tagliasacchi, “SoundStream: An end-to-end neural audio codec,”IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 30, pp. 495–507, 2021

    2021

  8. [8]

    High fidelity neural audio compression,

    A. D ´efossez, J. Copet, G. Synnaeve, and Y . Adi, “High fidelity neural audio compression,”TMLR, 2023

    2023

Show all 28 references
  1. [9]

    High-fidelity audio compression with improved RVQGAN,

    R. Kumar, P. Seetharaman, A. Luebs, I. Kumar, and K. Kumar, “High-fidelity audio compression with improved RVQGAN,” in Proc. NeurIPS, 2023

    2023

  2. [10]

    ESPnet-Codec: Com- prehensive training and evaluation of neural codecs for audio, mu- sic, and speech,

    J. Shi, J. Tian, Y . Wu, J.-W. Jung, J. Q. Yip, Y . Masuyama, W. Chen, Y . Wu, Y . Tang, M. Baaliet al., “ESPnet-Codec: Com- prehensive training and evaluation of neural codecs for audio, mu- sic, and speech,” inProc. SLT, 2024, pp. 562–569

    2024

  3. [11]

    AudioLM: a language modeling approach to audio gener- ation,

    Z. Borsos, R. Marinier, D. Vincent, E. Kharitonov, O. Pietquin, M. Sharifi, D. Roblek, O. Teboul, D. Grangier, M. Tagliasacchi et al., “AudioLM: a language modeling approach to audio gener- ation,”IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 31, pp. 2523–2533, 2023

    2023

  4. [12]

    Direct speech-to-speech translation with discrete units,

    A. Lee, P.-J. Chen, C. Wang, J. Gu, S. Popuri, X. Ma, A. Polyak, Y . Adi, Q. He, Y . Tanget al., “Direct speech-to-speech translation with discrete units,” inProc. ACL, May 2022, pp. 3327–3339

    2022

  5. [13]

    Neural codec language models are zero-shot text to speech synthesizers,

    S. Chen, C. Wang, Y . Wu, Z. Zhang, L. Zhou, S. Liu, Z. Chen, Y . Liu, H. Wang, J. Liet al., “Neural codec language models are zero-shot text to speech synthesizers,”IEEE Trans. Audio, Speech, Lang. Process., vol. 33, pp. 705–718, 2025

    2025

  6. [14]

    HuBERT: Self-supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhut- dinov, and A. Mohamed, “HuBERT: Self-supervised speech representation learning by masked prediction of hidden units,” IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 29, pp. 3451 – 3460, 2021

    2021

  7. [15]

    Exploring speech recognition, translation, and understanding with discrete speech units: A com- parative study,

    X. Chang, B. Yan, K. Choi, J.-W. Jung, Y . Lu, S. Maiti, R. Sharma, J. Shi, J. Tian, S. Watanabeet al., “Exploring speech recognition, translation, and understanding with discrete speech units: A com- parative study,” inProc. ICASSP, 2024, pp. 11 481–11 485

    2024

  8. [16]

    On generative spoken language modeling from raw audio,

    K. Lakhotia, E. Kharitonov, W.-N. Hsu, Y . Adi, A. Polyak, B. Bolte, T.-A. Nguyen, J. Copet, A. Baevski, A. Mohamed et al., “On generative spoken language modeling from raw audio,” TACL, vol. 9, pp. 1336–1354, 2021

    2021

  9. [17]

    SpeechTok- enizer: Unified speech tokenizer for speech large language mod- els,

    X. Zhang, D. Zhang, S. Li, Y . Zhou, and X. Qiu, “SpeechTok- enizer: Unified speech tokenizer for speech large language mod- els,” inProc. ICLR, 2024

    2024

  10. [18]

    Language-agnostic BERT sentence embedding,

    F. Feng, Y . Yang, D. Cer, N. Arivazhagan, and W. Wang, “Language-agnostic BERT sentence embedding,” inProc. ACL, May 2022, pp. 878–891

    2022

  11. [19]

    wav2vec 2.0: A framework for self-supervised learning of speech representa- tions,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representa- tions,”Proc. NeurIPS, vol. 33, pp. 12 449–12 460, 2020

    2020

  12. [20]

    Lib- rispeech: An asr corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Lib- rispeech: An asr corpus based on public domain audio books,” inProc. ICASSP, 2015, pp. 5206–5210

    2015

  13. [21]

    V oxPopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation,

    C. Wang, M. Riviere, A. Lee, A. Wu, C. Talnikar, D. Haz- iza, M. Williamson, J. Pino, and E. Dupoux, “V oxPopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation,” inProc. ACL, Aug. 2021, pp. 993–1003

    2021

  14. [22]

    MLS: A large-scale multilingual dataset for speech research,

    V . Pratap, Q. Xu, A. Sriram, G. Synnaeve, and R. Collobert, “MLS: A large-scale multilingual dataset for speech research,” in Proc. Interspeech, 2020, pp. 2757–2761

    2020

  15. [23]

    Montreal Forced Aligner: Trainable text-speech align- ment using Kaldi,

    M. McAuliffe, M. Socolof, S. Mihuc, M. Wagner, and M. Son- deregger, “Montreal Forced Aligner: Trainable text-speech align- ment using Kaldi,” inProc. Interspeech, 2017, pp. 498–502

    2017

  16. [24]

    mHuBERT-147: A compact multilingual HuBERT model,

    M. Z. Boito, V . Iyer, N. Lagos, L. Besacier, and I. Calapode- scu, “mHuBERT-147: A compact multilingual HuBERT model,” arXiv preprint arXiv:2406.06371, 2024

    2024 arXiv

  17. [25]

    SAMU-XLSR: Semantically-aligned multimodal utterance-level cross-lingual speech representation,

    S. Khurana, A. Laurent, and J. Glass, “SAMU-XLSR: Semantically-aligned multimodal utterance-level cross-lingual speech representation,”IEEE J. Sel. Top. Signal Process., vol. 16, no. 6, pp. 1493–1504, 2022

    2022

  18. [26]

    Abx-discriminability measures and applications,

    T. Schatz, “Abx-discriminability measures and applications,” Ph.D. dissertation, Universit´e Paris 6 (UPMC), Sep. 2016

    2016

  19. [27]

    The zero resource speech challenge 2021: Spoken language modelling,

    E. Dunbar, M. Bernard, N. Hamilakis, T. A. Nguyen, M. De Seyssel, P. Roz ´e, M. Rivi `ere, E. Kharitonov, and E. Dupoux, “The zero resource speech challenge 2021: Spoken language modelling,”arXiv preprint arXiv:2104.14700, 2021

    2021 arXiv

  20. [28]

    Learning hierarchical dis- crete linguistic units from visually-grounded speech,

    D. Harwath, W.-N. Hsu, and J. Glass, “Learning hierarchical dis- crete linguistic units from visually-grounded speech,” inProc. ICLR, 2020

    2020

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.