Pith. sign in

REVIEW 3 major objections 3 minor 39 references

Entropy-based grouping shrinks 50 Hz speech tokens to 15 Hz and still improves ASR and translation.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Predictive entropy from a token-level speech language model finds merge boundaries, producing compressed semantic tokens that keep ASR and translation accuracy at 15 Hz while lowering latency.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Entropy-based token grouping is a sensible new idea, but the headline claim is unproven because the dense baseline uses a different encoder. the 3 major comments →

arxiv 2509.00503 v1 pith:BWW4FPF2 submitted 2025-08-30 cs.CL eess.AS

Entropy-based Coarse and Compressed Semantic Speech Representation Learning

classification cs.CL eess.AS
keywords semantic speech tokensentropy-based compressionHuBERTtoken aggregationautomatic speech recognitionspeech-to-text translationvoice conversiondiscrete speech representation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the 25–50 tokens per second produced by speech tokenizers like HuBERT are finer than semantic tasks need, and that the redundancy can be removed by merging tokens into variable-length groups using the predictive entropy of a small speech language model. The authors train a lightweight LM on HuBERT discrete units, mark a group boundary wherever next-token entropy exceeds a threshold, and aggregate each group with cross-attention. At a moderate compression to 15 tokens per second, the compressed representations improve Word Error Rate on LibriSpeech from 6.2 to 5.6 and EN-CN translation BLEU from 28.2 to 31.5 relative to dense 50 Hz tokens, while cutting ASR decoding latency from 172 to 120 ms. The entropy threshold acts as a dial: coarser 7 Hz tokens still beat uniform fixed-length pooling at 12.5 Hz, though generation tasks like voice conversion need finer granularity. If right, tokenized speech for understanding tasks can be made roughly three times cheaper without paying an accuracy penalty.

Core claim

The paper claims that the next-token entropy of a lightweight autoregressive LM trained on HuBERT discrete units tells you where to cut a speech token stream, and that merging tokens inside those cuts yields compressed semantic representations that are at least as good as the densely tokenized original for understanding tasks. On LibriSpeech ASR the 15 Hz compressed tokens lower WER from 6.2% to 5.6% and CER from 3.3% to 2.9% against the 50 Hz HuBERT baseline; on CVSS-C EN-CN translation BLEU rises from 28.2 to 31.5. The 7 Hz setting still beats fixed-length max pooling at 12.5 Hz, and decoding latency falls from 172 ms to 120 ms at 15 Hz and 89 ms at 7 Hz. The paper's own forced-alignment a

What carries the argument

The central mechanism is entropy-based dynamic aggregation, driven by a lightweight autoregressive LM over HuBERT k-means tokens (K=500), followed by a Cross-Attentive Local Encoder (CALE) that fuses tokens inside each group with a learned query. The per-token conditional entropy H(ui) = -Σ p(v|u<i) log p(v|u<i) carries the argument: a global threshold θg selects boundaries; the same threshold controls granularity, with θg ≈ 7.8 giving 24 Hz tokens, 8.5 giving 15 Hz, and 9.7 giving 7 Hz. CALE's group-level cross-attention recovers information that simple max, average, or attention pooling loses, as the paper's ablations show.

Load-bearing premise

The load-bearing premise is that low predictive entropy in the HuBERT-token language model marks stretches that can be merged without losing task-relevant linguistic content; this is validated only by the paper's own forced-alignment analysis, not by a controlled test.

What would settle it

Compare entropy-defined 15 Hz groups against randomly constructed 15 Hz groups on the same ASR and translation test sets; if random grouping closes the WER/BLEU gap, boundary selection is not the cause. A second check: run the same threshold on a non-English corpus; if the 15 Hz advantage disappears, the claim depends on English phonotactics rather than on entropy as a general signal.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • At 15 Hz, entropy-compressed tokens outperform dense 50 Hz HuBERT tokens on ASR, with WER dropping from 6.2 to 5.6 and CER from 3.3 to 2.9.
  • Speech-to-text translation improves across EN-CN, EN-FR, and EN-ES, with EN-CN BLEU rising from 28.2 to 31.5 at 15 Hz.
  • Decoding latency drops roughly 30% at 15 Hz and about 48% at 7 Hz relative to the 50 Hz baseline.
  • Entropy-guided compression preserves more information than fixed-length pooling at the same or lower token rates, especially under strong compression.
  • The compression ratio is controllable through one threshold, allowing different granularities for different tasks; voice conversion prefers finer tokens than recognition tasks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The entropy LM is trained on English MLS speech; on tonal, agglutinative, or code-switched speech, the same threshold may not mark recoverable semantic units, and the 15 Hz advantage could shrink or reverse.
  • A randomized-boundary ablation at the same 15 Hz token rate would separate the benefit of adaptive boundary selection from the mere benefit of shorter sequences.
  • The same entropy mechanism might compress other discrete speech token streams, but the paper's own results suggest acoustic tokens lose quality under coarse merging, so a different signal would be needed for generation-heavy tasks.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper proposes an entropy-based dynamic aggregation framework for compressing HuBERT semantic tokens. A lightweight autoregressive LM is trained on discrete HuBERT units; next-token predictive entropy is used to find boundaries, adjacent tokens are merged into variable-length groups, and a cross-attentive local encoder aggregates each group into a single representation. The method is evaluated on ASR, speech-to-text translation, and voice conversion at three compression rates (≈24, 15, and 7 Hz). The authors claim the compressed representations perform on par with or better than dense token sequences, with the 15 Hz setting giving the best ASR/ST results and lower decoding latency.

Significance. If the central claim is validated, the work addresses a real limitation of current semantic speech tokenizers—their high token rate and redundant phonetic detail—and offers a controllable efficiency/performance trade-off. The manuscript includes useful ablations (C.1, C.4, C.5), a robustness check with K=2000 clusters (C.2), and an honest statement of limitations for generation tasks. However, as it stands, the central empirical claim is not established: the main comparison changes both the token rate and the encoder architecture, and the threshold sweep is internally selected. The contributions are potentially valuable, but the evidence base needs substantial strengthening before publication.

major comments (3)
  1. [§4.2, Tables 3–4, App. C.1] The headline comparison is confounded. §4.2 states that raw semantic units use a standard Conformer encoder, while compressed coarse semantic representations use a lightweight local encoder. Thus Table 3's HuBERT 50 Hz (WER 6.2) vs. Entropy-guided 15 Hz (WER 5.6) and Table 4's corresponding BLEU gains change both token rate and encoder architecture simultaneously. The C.1 ablation shows the CALE module itself contributes substantially (boundary + attention pooling reaches 6.1 WER vs. 5.6 with CALE). The improvement attributed to entropy-based compression could therefore be partly or wholly an architecture effect. Please provide a same-architecture baseline—e.g., feed original 50 Hz tokens through the same local encoder, or run the entropy-grouped sequence through the same Conformer—and ideally a fixed-length grouping baseline with the same CALE architecture at the same token rate.
  2. [§5.1, App. C.5] The optimality of the 15 Hz setting is partly a selection artifact. The entropy thresholds θg are empirically tuned to hit target token rates, and C.5 states the authors internally tested a 6–30 Hz range and selected 24, 15, and 7 Hz as representative settings. The conclusion that moderate compression is optimal is therefore not an out-of-sample prediction. Moreover, no error bars or significance tests are reported; differences such as WER 5.6 vs. 5.9 (Table 3) or BLEU 31.5 vs. 29.8 (Table 4) are not shown to be reliable. Please report the full sweep or a held-out threshold-selection protocol, and include variance or significance information for at least the ASR and ST comparisons.
  3. [Tables 1, 3, and 13 (§6.1, App. C.5)] The reported ASR result for the 7 Hz entropy-guided setting is internally inconsistent. Table 1 lists 'Coarse-grained 7 Hz' with WER 6.8, and Table 13 in Appendix C.5 also lists 7 Hz (M1) with WER 6.8, but Table 3 reports 'Entropy-guided (high, 7Hz)' with WER 10.5. This discrepancy is large and materially changes the interpretation of how degradation behaves under compression. Please clarify which value is correct and reconcile the tables. The same inconsistency should be checked for the VC results across Table 1 and Table 5.
minor comments (3)
  1. [Abstract, Table 5] The abstract's claim that compressed representations perform 'on par with or better than dense token sequences' is too strong for the VC results. In Table 5, HuBERT (original, 50 Hz) gives Q-MOS 3.89 and Entropy-guided 15 Hz gives 3.85, while deduplicated 26 Hz gives 4.12. The paper's own limitation section acknowledges degradation in generation tasks. Please qualify the abstract accordingly.
  2. [§4.2 vs. App. B vs. App. C.5] The entropy-threshold values are reported inconsistently. Section 4.2 gives θg = 9.7 for the 7 Hz setting; Appendix B gives θg = 9.7, 8.5, 7.8 for 7/15/24 Hz; Appendix C.5 gives θg = 0.97, 0.85, 0.78 for the same settings. This suggests a decimal-point error and must be corrected for reproducibility.
  3. [Table 11, App. C.3] The boundary-alignment analysis is descriptive. The 7 Hz setting achieves the highest word-boundary alignment (89.7%) but the worst ASR in Table 3 (WER 10.5, or 6.8 if Table 1 is correct). The paper should explain why high word-boundary alignment does not translate into better recognition performance, since this bears on the semantic-validity assumption in §3.3.

Circularity Check

0 steps flagged

No significant circularity: the entropy threshold is an explicitly empirical hyperparameter, and the central comparisons are measured results rather than predictions derived from the model.

full rationale

The paper does not derive its conclusions from its inputs by construction. The claimed chain is: HuBERT tokens -> entropy LM -> entropy thresholds -> group boundaries -> cross-attentive local encoder -> downstream metrics. The coupling of thresholds to target token rates is disclosed ('This global threshold θg is empirically determined'; App. C.5: 'we ... selected 24Hz, 15Hz, and 7Hz as representative settings'), and the paper does not present the resulting 15 Hz optimum as a first-principles prediction. Selecting representative settings after an internal sweep is a reporting/selection concern, not a circular derivation: the comparisons against HuBERT-original, deduplicated, and fixed-length pooling are genuine empirical tests, and the C.1 ablation independently separates the CALE module from boundary selection. The self-citations (Zuo et al. 2025, Ji et al. 2024) appear only in background/related-work and are not load-bearing. The main validity caveat—raw tokens (50 Hz) use a Conformer encoder while compressed tokens use the CALE encoder (Sec. 4.2)—is a confound that weakens attribution of the gains to entropy compression, but it is a correctness/experimental-design issue, not equivalence of output to input. Hence no circularity step meets the quoted-evidence bar.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The central claim rests on a pretrained HuBERT/k-means substrate, a learned entropy LM, and thresholds that are tuned rather than derived. There are no new physical or architectural entities beyond the group representation; the main hidden cost is empirical tuning of theta_g and theta_r and the choice of target rates.

free parameters (3)
  • Global entropy threshold theta_g = Section 4.2 reports 9.7 at 7 Hz; Appendix B reports 7.8/8.5/9.7 for 24/15/7 Hz; Appendix C.5 reports 0.78/0.85/0.97 for
    Chosen empirically to hit target token rates and task performance; no formula or independent estimate is provided.
  • Relative entropy threshold theta_r = 0.62 to 0.71 across settings in Appendix C.5 (M2/M3 variants).
    Alternative boundary criterion, also empirically selected; the main system uses theta_g only.
  • Representative token rates (24/15/7 Hz) = 24, 15, and 7 tokens per second.
    Selected after an internal sweep across 6-30 Hz (Appendix C.5), so the evaluation granularities are post hoc choices.
axioms (4)
  • domain assumption HuBERT features quantized with k-means are a valid discrete semantic token representation that preserves content needed for ASR, ST, and VC.
    Invoked in Section 3.2 without independent validation in this paper; relies on prior SSL results.
  • ad hoc to paper Predictive next-token entropy under the trained language model indicates meaningful, mergeable segment boundaries.
    Core mechanism of Section 3.3; only self-reported MFA boundary alignment in Appendix C.3 supports it.
  • ad hoc to paper A boundary is found when H(ui) exceeds a global threshold theta_g or the jump H(ui)-H(ui-1) exceeds theta_r.
    This rule is arbitrary and is not derived from any model of speech; the threshold values are tuned.
  • domain assumption The cross-attention local encoder can recover information lost during token merging.
    Needed for compressed representations to be useful in Section 3.4; supported only by the paper's own ablation.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Entropy-based Coarse and Compressed Semantic Speech Representation Learning." pith.science (2026). https://pith.science/paper/BWW4FPF2

@misc{pith2026250900503,
  author       = {Pith},
  title        = {Pith review of: Entropy-based Coarse and Compressed Semantic Speech Representation Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BWW4FPF2}},
  note         = {Machine review of arXiv:2509.00503}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Discrete speech representation learning has recently attracted increasing interest in both acoustic and semantic modeling. Existing approaches typically encode 16 kHz waveforms into discrete tokens at a rate of 25 or 50 tokens per second. However, given that speech generally conveys only 2 to 5 words per second, such fine-grained tokenization introduces redundancy and hinders efficiency in downstream training and inference. Moreover, semantic speech representations at this frequency primarily capture phonetic-level information, while semantic understanding may not require such detailed token-level resolution. To address these limitations, we propose an entropy-based dynamic aggregation framework for learning compressed semantic speech representations. A speech language model is first pre-trained via next-token prediction on large-scale unlabeled data to capture frequent token patterns. Predictive entropy is then used to adaptively determine aggregation boundaries, followed by a cross-attention module that fuses information within each segment. By adjusting the entropy threshold, the granularity and compression ratio of the representations can be flexibly controlled. Experiments on ASR, speech-to-text translation, and voice conversion tasks demonstrate that the compressed representations perform on par with or better than dense token sequences, demonstrating the effectiveness of the proposed approach.

Figures

Figures reproduced from arXiv: 2509.00503 by Guangyan Zhang, Jialong Zuo, Jingyu Li, Minghui Fang, Shengpeng Ji, Xiaoqi Jiao, Yiwen Guo, Zhou Zhao.

Figure 1
Figure 1. Figure 1: Illustration of the proposed entropy-based semantic token compression framework. Figure (a) shows [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

39 extracted references · 10 canonical work pages · 1 internal anchor

  1. [1]

    Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in neural information processing systems, 33:12449--12460

  2. [2]

    Zal \'a n Borsos, Rapha \"e l Marinier, Damien Vincent, Eugene Kharitonov, Olivier Pietquin, Matt Sharifi, Dominik Roblek, Olivier Teboul, David Grangier, Marco Tagliasacchi, and 1 others. 2023. Audiolm: a language modeling approach to audio generation. IEEE/ACM transactions on audio, speech, and language processing, 31:2523--2533

  3. [3]

    Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, and 1 others. 2022. Wavlm: Large-scale self-supervised pre-training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing, 16(6):1505--1518

  4. [4]

    Cheol Jun Cho, Abdelrahman Mohamed, Alan W Black, and Gopala K Anumanchipalli. 2024. Self-supervised models of speech infer universal articulatory kinematics. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 12061--12065. IEEE

  5. [5]

    Cheol Jun Cho, Peter Wu, Abdelrahman Mohamed, and Gopala K Anumanchipalli. 2023. Evidence of vocal tract articulation in self-supervised learning of speech. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1--5. IEEE

  6. [6]

    Kwanghee Choi, Ankita Pasad, Tomohiko Nakamura, Satoru Fukayama, Karen Livescu, and Shinji Watanabe. 2024. Self-supervised speech representations are more phonetic than semantic. arXiv preprint arXiv:2406.08619

  7. [7]

    Yu-An Chung, Yu Zhang, Wei Han, Chung-Cheng Chiu, James Qin, Ruoming Pang, and Yonghui Wu. 2021. W2v-bert: Combining contrastive learning and masked language modeling for self-supervised speech pre-training. In 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pages 244--250. IEEE

  8. [8]

    Alexandre D \'e fossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi. 2022. High fidelity neural audio compression. arXiv preprint arXiv:2210.13438

  9. [9]

    Zhihao Du, Qian Chen, Shiliang Zhang, Kai Hu, Heng Lu, Yexin Yang, Hangrui Hu, Siqi Zheng, Yue Gu, Ziyang Ma, and 1 others. 2024. Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens. arXiv preprint arXiv:2407.05407

  10. [10]

    Anmol Gulati, James Qin, Chung-Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Zhang, Yonghui Wu, and Ruoming Pang. 2020. https://arxiv.org/abs/2005.08100 Conformer: Convolution-augmented transformer for speech recognition . Preprint, arXiv:2005.08100

  11. [11]

    Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. 2021. https://arxiv.org/abs/2106.07447 Hubert: Self-supervised speech representation learning by masked prediction of hidden units . Preprint, arXiv:2106.07447

  12. [12]

    Shengpeng Ji, Ziyue Jiang, Wen Wang, Yifu Chen, Minghui Fang, Jialong Zuo, Qian Yang, Xize Cheng, Zehan Wang, Ruiqi Li, and 1 others. 2024. Wavtokenizer: an efficient acoustic discrete codec tokenizer for audio language modeling. arXiv preprint arXiv:2408.16532

  13. [13]

    Ye Jia, Michelle Tadmor Ramanovich, Quan Wang, and Heiga Zen. 2022. Cvss corpus and massively multilingual speech-to-speech translation. arXiv preprint arXiv:2201.03713

  14. [14]

    Zeqian Ju, Yuancheng Wang, Kai Shen, Xu Tan, Detai Xin, Dongchao Yang, Yanqing Liu, Yichong Leng, Kaitao Song, Siliang Tang, and 1 others. 2024. Naturalspeech 3: Zero-shot speech synthesis with factorized codec and diffusion models. arXiv preprint arXiv:2403.03100

  15. [15]

    Heeseung Kim, Sungwon Kim, Jiheum Yeom, and Sungroh Yoon. 2023. Unitspeech: Speaker-adaptive speech synthesis with untranscribed data. arXiv preprint arXiv:2306.16083

  16. [16]

    Jungil Kong, Jaehyeon Kim, and Jaekyoung Bae. 2020. https://arxiv.org/abs/2010.05646 Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis . Preprint, arXiv:2010.05646

  17. [17]

    Rithesh Kumar, Prem Seetharaman, Alejandro Luebs, Ishaan Kumar, and Kundan Kumar. 2023. High-fidelity audio compression with improved rvqgan. Advances in Neural Information Processing Systems, 36:27980--27993

  18. [18]

    Songting Liu. 2024. Zero-shot voice conversion with diffusion transformers. arXiv preprint arXiv:2411.09943

  19. [19]

    Tu Anh Nguyen, Benjamin Muller, Bokai Yu, Marta R Costa-Jussa, Maha Elbayad, Sravya Popuri, Christophe Ropers, Paul-Ambroise Duquenne, Robin Algayres, Ruslan Mavlyutov, and 1 others. 2025. Spirit-lm: Interleaved spoken and written language model. Transactions of the Association for Computational Linguistics, 13:30--52

  20. [20]

    Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. 2015. Librispeech: an asr corpus based on public domain audio books. In 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 5206--5210. IEEE

  21. [21]

    Vineel Pratap, Qiantong Xu, Anuroop Sriram, Gabriel Synnaeve, and Ronan Collobert. 2020. Mls: A large-scale multilingual dataset for speech research. arXiv preprint arXiv:2012.03411

  22. [22]

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2022. https://arxiv.org/abs/2212.04356 Robust speech recognition via large-scale weak supervision . Preprint, arXiv:2212.04356

  23. [23]

    Takaaki Saeki, Detai Xin, Wataru Nakata, Tomoki Koriyama, Shinnosuke Takamichi, and Hiroshi Saruwatari. 2022. Utmos: Utokyo-sarulab system for voicemos challenge 2022. arXiv preprint arXiv:2204.02152

  24. [24]

    Steffen Schneider, Alexei Baevski, Ronan Collobert, and Michael Auli. 2019. wav2vec: Unsupervised pre-training for speech recognition. arXiv preprint arXiv:1904.05862

  25. [25]

    Amitay Sicherman and Yossi Adi. 2023. Analysing discrete self supervised speech representation for spoken language modeling. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1--5. IEEE

  26. [26]

    Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Yunfeng Liu. 2023. https://arxiv.org/abs/2104.09864 Roformer: Enhanced transformer with rotary position embedding . Preprint, arXiv:2104.09864

  27. [27]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. https://arxiv.org/abs/2302.13971 Llama: Open and efficient foundation language models . Preprint, arXiv:2302.13971

  28. [28]

    Aaron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. 2018. https://arxiv.org/abs/1711.00937 Neural discrete representation learning . Preprint, arXiv:1711.00937

  29. [29]

    Changhan Wang, Anne Wu, Jiatao Gu, and Juan Pino. 2021. Covost 2 and massively multilingual speech translation. In Interspeech, volume 2021, pages 2247--2251

  30. [30]

    Yuancheng Wang, Haoyue Zhan, Liwei Liu, Ruihong Zeng, Haotian Guo, Jiachen Zheng, Qiang Zhang, Xueyao Zhang, Shunsi Zhang, and Zhizheng Wu. 2024. Maskgct: Zero-shot text-to-speech with masked generative codec transformer. arXiv preprint arXiv:2409.00750

  31. [31]

    Zhifei Xie and Changqiao Wu. 2024 a . Mini-omni: Language models can hear, talk while thinking in streaming. arXiv preprint arXiv:2408.16725

  32. [32]

    Zhifei Xie and Changqiao Wu. 2024 b . Mini-omni2: Towards open-source gpt-4o with vision, speech and duplex capabilities. arXiv preprint arXiv:2410.11190

  33. [33]

    Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, and 1 others. 2025. Qwen2. 5-omni technical report. arXiv preprint arXiv:2503.20215

  34. [34]

    Yifan Yang, Feiyu Shen, Chenpeng Du, Ziyang Ma, Kai Yu, Daniel Povey, and Xie Chen. 2023. https://arxiv.org/abs/2309.07377 Towards universal speech discrete tokens: A case study for asr and tts . Preprint, arXiv:2309.07377

  35. [35]

    Xin Zhang, Dong Zhang, Shimin Li, Yaqian Zhou, and Xipeng Qiu. 2023. Speechtokenizer: Unified speech tokenizer for speech large language models. arXiv preprint arXiv:2308.16692

  36. [36]

    Xueyao Zhang, Xiaohui Zhang, Kainan Peng, Zhenyu Tang, Vimal Manohar, Yingru Liu, Jeff Hwang, Dangna Li, Yuhao Wang, Julian Chan, and 1 others. 2025. Vevo: Controllable zero-shot voice imitation with self-supervised disentanglement. arXiv preprint arXiv:2502.07243

  37. [37]

    Jialong Zuo, Shengpeng Ji, Minghui Fang, Ziyue Jiang, Xize Cheng, Qian Yang, Wenrui Liu, Guangyan Zhang, Zehai Tu, Yiwen Guo, and 1 others. 2025. Enhancing expressive voice conversion with discrete pitch-conditioned flow matching model. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1--5. IEEE

  38. [38]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  39. [39]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.