Pith. sign in

REVIEW 3 major objections 7 minor 38 references

A 9B bilingual SpeechLM can match or beat larger peers on English and Korean speech tasks while keeping text QA, then extend to full-duplex turn-taking.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 08:21 UTC pith:XTRJ3YW5

load-bearing objection Solid open 9B bilingual SpeechLM with a real recipe and Korean benchmarks; the duplex half of the claim is softer because of synthetic data and an internal FDB evaluator. the 3 major comments →

arxiv 2605.23912 v1 pith:XTRJ3YW5 submitted 2026-04-08 cs.CL cs.AIcs.SD

Raon-Speech Technical Report

Beomsoo Kim , Changho Choi , Dohyun Kim , Dongki Lee , Ethan Ewer , Eunchong Kim , Gyeongman Kim , Haechan Kim
show 18 more authors
Hyeonghwan Kim Inkyu Park Jihun Yun Jihwan Moon Jiyun Kim Joonghyun Bae Junhyuck Kim Minkyu Kim Sehun Lee Seungjun Chung Sungwoo Cho Dongmin Park Dongwon Kim Hara Kang Jonghyun Lee Keon Lee Kangwook Lee Jaewoong Cho
This is my paper
classification cs.CL cs.AIcs.SD
keywords speech language modelfull-duplex dialogueEnglish-Korean bilingualknowledge distillationpreference optimizationturn-takingSpeechLMopen-source checkpoints
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This technical report claims that a pre-trained text LLM can be turned into a strong English–Korean speech language model without erasing its text ability, and then extended into a full-duplex conversational system that listens and speaks at the same time. Raon-Speech does this by adding speech understanding and generation modules, training on 1.38 million hours of curated speech and text through alignment, knowledge-distillation pre-training, and preference-based post-training. Across 42 English and Korean benchmarks it reports the strongest overall speech-centric profile among eight similarly sized recent audio foundation models, with especially clear gains on spoken question answering, speech understanding, and generated-speech intelligibility, while still leading text QA suites such as MMLU-Pro. Raon-SpeechChat continues from that base on 119K hours of time-aligned dialogue, using a causal encoder, interleaved user-speech/assistant-text/assistant-speech tokens, and explicit when-to-speak state tokens, and shows its clearest strengths on Full-Duplex-Bench v1.0 turn-taking and interruption handling. The authors also release three new Korean spoken benchmarks and open-source the checkpoints, pipeline, and demo, arguing that practical bilingual real-time spoken interaction is now within reach of open ~9B models.

Core claim

A staged recipe—module alignment, end-to-end pre-training with on-policy knowledge distillation from text-conditioned and backbone teachers, then multi-task preference optimization—transforms a pre-trained LLM into a 9B SpeechLM that both understands and generates English and Korean speech while preserving text QA; continuing with a causal encoder and interleaved full-duplex sequences yields competitive real-time turn-taking and interruption behavior on FDB.

What carries the argument

Three-stage SpeechLM recipe plus full-duplex sequence design: freeze-then-unfreeze alignment of speech modules, CE + on-policy KL distillation, SimPO post-training; then causal streaming encoder, word-aligned interleaved user-speech/assistant-text/assistant-speech tokens, and SIL/BOW/BC state tokens that separate when to speak from what to say, with text lookahead.

Load-bearing premise

That scores from the authors' internal Full-Duplex-Bench evaluator and a mostly synthetic, time-aligned dialogue corpus will still hold up under real human duplex conversation and the official public evaluation scripts.

What would settle it

Re-run the same checkpoints on the public FDB v1.0/v1.5 reference scripts with their original thresholds and judges, and on live human full-duplex sessions with natural interruptions; if turn-taking and interruption metrics fall below the strongest open baselines, the duplex claim does not transfer.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The manuscript presents Raon-Speech, a 9B-parameter English–Korean SpeechLM built by adapting Qwen3-VL-8B-Instruct with speech understanding and Mimi-codec generation modules, and Raon-SpeechChat, a full-duplex extension with a causal encoder, word-level interleaved user-speech/assistant-text/assistant-speech sequences, and SIL/BOW/BC state tokens. Training uses 1.38M hours for Raon-Speech (alignment → end-to-end pre-training with on-policy KD → SimPO post-training) and 119K hours of time-aligned dialogue for Raon-SpeechChat (causal adaptation → duplex pre-training → two-stage fine-tuning). Across Tables 4–5 and appendices, Raon-Speech is compared to eight similarly sized audio foundation models on ASR, speech generation, SpokenQA, speech understanding, and TextQA (including newly released KVoiceBench, KOpenAudioBench, and KMMAU). Raon-SpeechChat is evaluated on Full-Duplex-Bench v1.0/v1.5/v2.0 (Table 6). The paper claims the strongest speech-centric profile for Raon-Speech while preserving text QA, and clearest strengths for Raon-SpeechChat on FDB v1.0 turn-taking and interruption behaviors, with full open-sourcing of checkpoints, pipelines, and a demo.

Significance. If the empirical claims hold under standard evaluation protocols, this is a substantial systems contribution to bilingual SpeechLMs and full-duplex spoken dialogue under 10B parameters. Strengths that should be credited include: (i) a carefully staged recipe that jointly adds speech understanding/generation while retaining strong MMLU-Pro/MMLU-Redux and Korean text QA; (ii) release of three Korean speech benchmarks tailored to culture and language; (iii) open-sourcing of all checkpoints, training/inference code, and an interactive demo, which is rare at this scale and directly enables reproduction and follow-on work; (iv) explicit architectural choices for duplex control (SIL/BOW/BC, text lookahead, causal streaming encoder) with detailed data and hyperparameter documentation (Tables 2–3, Appendices B–D). The work is of clear interest to the speech–language modeling community even if some duplex evaluation details need tightening.

major comments (3)
  1. Table 6 and Appendix G: The abstract and §5.2 claim that Raon-SpeechChat shows its “clearest strengths” on FDB v1.0 turn-taking and interruption-sensitive behaviors. Appendix G states that the offline evaluator differs from the public FDB v1.0/v1.5 reference scripts in load-bearing ways (pause takeover: 1.5 s / 5-word vs public 1.0 s / 3-word; ASR-refined anchors and 0.5 s post-anchor margin; GPT-5.2 vs the public judge models). Without a side-by-side re-score under the public scripts (or an official leaderboard submission), the headline TOR/Freq/Judge numbers cannot be treated as comparable to prior FDB reports. Please either re-evaluate all models under the public scripts and report both, or substantially qualify the FDB v1.0 claim and any ranking language that depends on these metrics.
  2. §4.2 and Appendix D: Full-duplex training uses 119K hours of which 106.33K (~89%) are synthetic, with backchannels/interruptions/overlap placed by annotated roles, rule-based timing, a backchannel prediction model, and barge-in truncation at random word boundaries. Table 6’s interruption and turn-taking gains may therefore partly reflect training–evaluation alignment to synthetic interaction patterns rather than transfer to real human duplex timing. A real-only (or real-heavy) ablation, or at minimum a held-out real-conversation duplex evaluation with the same metrics, is needed to support the claim of “natural real-time conversation” beyond synthetic FDB slices. If such an ablation is infeasible, the abstract/§5.2 claims should be scoped to the synthetic-heavy training regime and offline FDB protocol actually used.
  3. Table 4 (Speech Generation rows) and the “strongest overall profile on speech-centric tasks” claim (§1, abstract): Kimi-Audio and Audio Flamingo 3 report “–” for LibriSpeech-c and Seed generation, so the intelligibility SOTA is established only against the subset of baselines that emit speech. Please either obtain generation numbers for those models, restrict the generation comparison explicitly to models with reported WER/UTMOS, or rephrase the overall speech-centric ranking so it does not imply a complete head-to-head on generation for all eight baselines. Relatedly, UTMOS is often second-tier while WER/CER is best; the text already notes this, but the abstract’s unqualified “top-performing” framing should match the more nuanced body (intelligibility vs perceptual naturalness).
minor comments (7)
  1. §3.1 knowledge distillation: On-policy self-distillation with a text-conditioned teacher is reasonable, but a short note on whether teacher and student share the same decoding temperature/top-p (and any filtering of degenerate trajectories) would help readers assess the mild circularity risk.
  2. §5.1 SpokenQA: GPT-5.4 (and GPT-5.2 in Appendix G) as judges limits long-term reproducibility. Please pin model versions/dates and release the exact judge prompts (beyond “VoiceBench prompt / translated”) or report inter-judge agreement on a subset.
  3. Figure 1 and Figure 5: Zero-to-max normalization per axis can visually inflate small absolute gaps. Consider adding raw-score small multiples or a table of normalized values in the appendix so radar plots are not the only comparative view.
  4. Table 2 / §3.2: Loss weights on PAD/SIL (0.75/0.5 then SIL 0.25) and 50× CE on BC are important free parameters; a one-paragraph sensitivity note (even qualitative) would strengthen confidence that duplex behavior is not brittle to these choices.
  5. Appendix E: AlpacaEval appears in both VoiceBench and OpenAudioBench aggregates (and likewise KAlpacaEval). State explicitly whether double-counting affects the suite averages or is only for readability of the two official groupings.
  6. Typos/clarity: “sumsumsumsum” in Figure 2 caption layout; “Librispeech-c Seed” label stacking in Figure 1; occasional missing spaces in compound names. Also fix “Raon-Speech successfully transforms…” repetition between abstract and §1 for concision.
  7. §7 Future work mentions vision and agentic extensions; a brief limitation paragraph on latency/throughput of the 9–9.8B stack under true streaming (chunk sizes, RTF) would help practitioners more than the current high-level future-work list.

Circularity Check

0 steps flagged

No circular derivation: claims are empirical leaderboard comparisons on external (or jointly evaluated) benchmarks, not identities forced by fitted constants or self-citation theorems.

full rationale

Raon-Speech / Raon-SpeechChat is a systems/training report. Its load-bearing claims are comparative scores on ASR, TTS intelligibility, SpokenQA suites, MMAU/KMMAU, MMLU-Pro/Redux, and Full-Duplex-Bench against named external models. There is no first-principles derivation chain, uniqueness theorem, or fitted scale that is then re-presented as a prediction of a closely related quantity. Self-distillation (teacher = same model on text transcripts) and SimPO online rejects are training objectives; success is still measured on held-out public or multi-model-evaluated benchmarks, so the reported SOTA profile is not true by construction. Synthetic full-duplex data and Appendix G’s internal FDB evaluator differences raise transfer/reproducibility risk, not circularity: they do not make TOR/Freq/Judge algebraically equal to training inputs. New Korean suites (KVoiceBench, KOpenAudioBench, KMMAU) are author-built but scored for all baselines under the same protocol, so relative ranking is not a self-definitional identity. No self-citation is used as a uniqueness or ansatz load-bearing step that forbids alternatives. Score 0 is appropriate.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 2 invented entities

This is an empirical systems paper. Load-bearing content is architectural choices, large curated/synthetic corpora, and many optimization hyperparameters rather than mathematical axioms. Free parameters are the training schedule and loss weights that define the final model; invented entities are the control tokens and interleaving scheme that make full-duplex training well-defined.

free parameters (4)
  • Stage-wise learning rates, warmups, batch sizes, sequence lengths
    Table 2 lists many hand-chosen schedules (e.g., 1.5e-4→1.5e-5, 60k pretrain steps, SimPO 1e-6). Final quality depends on these choices.
  • Loss weights on PAD/SIL and 50× CE on BC
    Full-duplex stages down-weight PAD/SIL and up-weight BC to fix label imbalance; these scalars directly shape turn-taking behavior.
  • 10% mix of original Raon-Speech data during duplex pre-training
    Ad-hoc retention mix chosen to reduce forgetting; not derived from a principle.
  • RMSNorm adaptor init scale 0.02; speaker-embedding dropout 0.2; 16/32 RVQ depths
    Initialization and codec depth choices that stabilize alignment and real-time quality.
axioms (4)
  • domain assumption A frozen strong text LLM backbone plus aligned speech modules can acquire speech understanding/generation without catastrophic text forgetting when TextQA and KD are mixed in.
    Stated goal of Sections 2–3; standard multimodal-LLM assumption, not proved here.
  • domain assumption Mimi RVQ semantic+acoustic tokens at 12.5 Hz plus residual code prediction are an adequate discrete speech interface for both half- and full-duplex generation.
    Architecture Section 2; inherits from Moshi/Mimi literature.
  • ad hoc to paper Word-level interleaved user-speech / assistant-text / assistant-speech with SIL/BOW/BC and one-frame text lookahead is a sufficient sequence model of simultaneous listening and speaking.
    Section 2.2 design choice that defines Raon-SpeechChat training targets.
  • domain assumption LLM-as-judge and Whisper/in-house ASR transcriptions are faithful enough proxies for spoken-QA quality and generation intelligibility.
    Evaluation Section 5; standard but imperfect measurement assumption.
invented entities (2)
  • SIL / BOW / BC special tokens for listening vs speaking vs backchannel no independent evidence
    purpose: Separate when-to-speak from what-to-say and control backchannel frequency in full-duplex generation.
    Introduced in Section 2.2 as explicit state modeling beyond Moshi’s single PAD token; independent evidence is only behavioral FDB metrics, not external physics.
  • KVoiceBench, KOpenAudioBench, KMMAU Korean speech benchmarks independent evidence
    purpose: Fill missing Korean spoken-QA and speech-understanding evaluation.
    Constructed by translation + TTS and capability questions from Korean corpora; useful artifacts but author-constructed, so not fully independent of the paper’s evaluation narrative.

pith-pipeline@v1.1.0-grok45 · 31783 in / 3590 out tokens · 48939 ms · 2026-07-13T08:21:00.726972+00:00 · methodology

0 comments
read the original abstract

We present Raon-Speech, a top-performing 9B-parameter speech language model (SpeechLM) for English and Korean speech understanding, answering, and generation, and Raon-SpeechChat, a high-performing full-duplex extension for natural real-time conversation. Raon-Speech successfully transforms a pre-trained LLM into a SpeechLM that both understands and generates speech while preserving strong text capabilities. It trains on 1.38M hours of highly curated English and Korean speech and text datasets with the following training stages: (1) speech modules alignment, (2) end-to-end SpeechLM pre-training with knowledge distillation, and (3) multi-task preference optimization-based post-training. Across 42 English and Korean speech and text benchmarks, Raon-Speech establishes the strongest overall profile on speech-centric tasks in our comparison against eight similarly sized recent audio foundation models, including Qwen2.5-Omni and Fun-Audio-Chat, while preserving strong text question answering performance. Building upon it, Raon-SpeechChat enables natural full-duplex conversation by continual training on 119K hours of time-aligned real and synthetic dialogue data. It proceeds through three complementary training stages: (1) causal encoder adaptation, (2) full-duplex pre-training, (3) full-duplex fine-tuning for voice and role-control. On multiple full-duplex benchmarks, Raon-SpeechChat shows its clearest strengths on the turn-taking and interruption-sensitive behaviors covered by FDB v1.0, and remains competitive across the broader full-duplex evaluation suite. We open-source all model checkpoints, the training and inference pipeline, and an interactive demo.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

38 extracted references · 1 canonical work pages

  1. [1]

    URL https://arxiv.org/abs/2406.02430. S. Arora, K.-W. Chang, C.-M. Chien, Y. Peng, H. Wu, Y. Adi, E. Dupoux, H.-Y. Lee, K. Livescu, and S. Watanabe. On the landscape of spoken language models: A comprehensive survey.arXiv preprint arXiv:2504.08528,

    Pith/arXiv arXiv

  2. [2]

    K. Baba, W. Nakata, Y. Saito, and H. Saruwatari. The t05 system for the VoiceMOS Challenge 2024: Transfer learning from deep image classifier to naturalness MOS prediction of high-quality synthetic speech. InIEEE Spoken Language Technology Workshop (SLT), pages 818–824,

    2024

  3. [3]

    doi: 10.1109/SLT61566.2024.10832315. S. Bai, Y. Cai, R. Chen, K. Chen, X. Chen, Z. Cheng, L. Deng, W. Ding, C. Gao, C. Ge, W. Ge, Z. Guo, Q. Huang, J. Huang, F. Huang, B. Hui, S. Jiang, Z. Li, M. Li, M. Li, K. Li, Z. Lin, J. Lin, X. Liu, J. Liu, C. Liu, Y. Liu, D. Liu, S. Liu, D. Lu, R. Luo, C. Lv, R. Men, L. Meng, X. Ren, X. Ren, S. Song, Y. Sun, J. Tang...

    doi:10.1109/slt61566.2024.10832315 2024

  4. [4]

    URL https://arxiv.org/abs/2511.21631. J.-U. Bang, S. Yun, S.-H. Kim, M.-Y. Choi, M.-K. Lee, Y.-J. Kim, D.-H. Kim, J. Park, Y.-J. Lee, and S.-H. Kim. Ksponspeech: Korean spontaneous speech corpus for automatic speech recognition.Applied Sciences, 10(19):6936,

    Pith/arXiv arXiv

  5. [5]

    Chang, E.-P

    K.-W. Chang, E.-P. Hu, C.-Y. Kuan, W. Ren, W.-C. Chen, G.-T. Lin, Y. Tsao, S.-H. Sun, H.-y. Lee, and J. Glass. Game-time: Evaluating temporal dynamics in spoken language models.arXiv preprint arXiv:2509.26388,

    Pith/arXiv arXiv

  6. [6]

    Y. Chen, X. Yue, C. Zhang, X. Gao, R. T. Tan, and H. Li. Voicebench: Benchmarking llm-based voice assistants.arXiv preprint arXiv:2410.17196,

    Pith/arXiv arXiv

  7. [7]

    Y. Chu, J. Xu, Q. Yang, H. Wei, X. Wei, Z. Guo, Y. Leng, Y. Lv, J. He, J. Lin, et al. Qwen2-audio technical report.arXiv preprint arXiv:2407.10759,

    Pith/arXiv arXiv

  8. [8]

    URL https: //arxiv.org/abs/2205.12446. N. Dawalatabad, M. Ravanelli, F. Grondin, J. Thienpondt, B. Desplanques, and H. Na. Ecapa-tdnn embeddings for speaker diarization.arXiv preprint arXiv:2104.01466,

    Pith/arXiv arXiv

  9. [9]

    Défossez, J

    A. Défossez, J. Copet, G. Synnaeve, and Y. Adi. High fidelity neural audio compression.arXiv preprint arXiv:2210.13438,

    Pith/arXiv arXiv

  10. [10]

    Défossez, L

    A. Défossez, L. Mazaré, M. Orsini, A. Royer, P. Pérez, H. Jégou, E. Grave, and N. Zeghidour. Moshi: a speech-text foundation model for real-time dialogue.arXiv preprint arXiv:2410.00037,

    Pith/arXiv arXiv

  11. [11]

    Desplanques, J

    B. Desplanques, J. Thienpondt, and K. Demuynck. Ecapa-tdnn: Emphasized channel attention, propaga- tion and aggregation in tdnn based speaker verification.arXiv preprint arXiv:2005.07143,

    Pith/arXiv arXiv 2005

  12. [12]

    16 Raon-Speech Technical Report D. Ding, Z. Ju, Y. Leng, S. Liu, T. Liu, Z. Shang, K. Shen, W. Song, X. Tan, H. Tang, et al. Kimi-audio technical report.arXiv preprint arXiv:2504.18425,

    Pith/arXiv arXiv

  13. [13]

    Z. Du, C. Gao, Y. Wang, F. Yu, T. Zhao, H. Wang, X. Lv, H. Wang, C. Ni, X. Shi, et al. Cosyvoice 3: Towards in-the-wild speech generation via scaling-up and post-training.arXiv preprint arXiv:2505.17589,

    Pith/arXiv arXiv

  14. [14]

    Dubey, A

    H. Dubey, A. Aazami, V. Gopal, B. Naderi, S. Braun, R. Cutler, A. Ju, M. Zohourian, M. Tang, M. Golestaneh, et al. Icassp 2023 deep noise suppression challenge.IEEE Open Journal of Signal Processing, 5:725–737,

    2023

  15. [15]

    URLhttps://arxiv.org/abs/2406.04127. A. Goel, S. Ghosh, J. Kim, S. Kumar, Z. Kong, S.-g. Lee, C.-H. H. Yang, R. Duraiswami, D. Manocha, R. Valle, et al. Audio flamingo 3: Advancing audio intelligence with fully open large audio language models.arXiv preprint arXiv:2507.08128,

    Pith/arXiv arXiv

  16. [16]

    URLhttps://arxiv.org/abs/2507.08924. H. Hu, X. Zhu, T. He, D. Guo, B. Zhang, X. Wang, Z. Guo, Z. Jiang, H. Hao, Z. Guo, et al. Qwen3-tts technical report.arXiv preprint arXiv:2601.15621, 2026a. J. Hu, D. Zhu, X. Luo, D. Zhang, S. He, Y. Lei, H. Zheng, S. Feng, J. He, Y. Sun, et al. Cord: Bridging the audio-text reasoning gap via weighted on-policy cross-m...

    Pith/arXiv arXiv

  17. [17]

    URLhttps://arxiv.org/abs/2508.13992. T. Li and D. Khashabi. Simplemix: Frustratingly simple mixing of off-and on-policy data in language model preference learning.arXiv preprint arXiv:2505.02363,

    Pith/arXiv arXiv

  18. [18]

    T. Li, J. Liu, T. Zhang, Y. Fang, D. Pan, M. Wang, Z. Liang, Z. Li, M. Lin, G. Dong, et al. Baichuan-audio: A unified framework for end-to-end speech interaction.arXiv preprint arXiv:2502.17239,

    Pith/arXiv arXiv

  19. [19]

    Lin, S.-Y

    G.-T. Lin, S.-Y. S. Kuan, J. Shi, K.-W. Chang, S. Arora, S. Watanabe, and H. yi Lee. Full-duplex-bench- v2: A multi-turn evaluation framework for duplex dialogue systems with an automated examiner, 2025a. URLhttps://arxiv.org/abs/2510.07838. G.-T. Lin, S.-Y. S. Kuan, Q. Wang, J. Lian, T. Li, S. Watanabe, and H. yi Lee. Full-duplex-bench v1.5: Evaluating o...

    Pith/arXiv arXiv

  20. [20]

    Loshchilov and F

    17 Raon-Speech Technical Report I. Loshchilov and F. Hutter. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101,

    Pith/arXiv arXiv

  21. [21]

    Official model card accessed on 2026-03-31. V. Panayotov, G. Chen, D. Povey, and S. Khudanpur. Librispeech: an asr corpus based on public domain audio books. In2015 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 5206–5210. IEEE,

    2026

  22. [22]

    URLhttps://arxiv.org/abs/2212.04356. A. Reece, G. Cooney, P. Bull, C. Chung, B. Dawson, C. Fitzpatrick, T. Glazer, D. Knox, A. Liebscher, and S. Marin. The candor corpus: Insights from a large multimodal dataset of naturalistic conversation. Science Advances, 9(13):eadf3197,

    Pith/arXiv arXiv

  23. [23]

    URL https://doi.org/10

    doi: 10.1126/sciadv.adf3197. URL https://doi.org/10. 1126/sciadv.adf3197. R. Roy, J. Raiman, S. gil Lee, T.-D. Ene, R. Kirby, S. Kim, J. Kim, and B. Catanzaro. Personaplex: Voice and role control for full duplex conversational speech models,

    work page doi:10.1126/sciadv.adf3197

  24. [24]

    URLhttps://arxiv.org/ abs/2602.06053. S. Sakshi, U. Tyagi, S. Kumar, A. Seth, R. Selvakumar, O. Nieto, R. Duraiswami, S. Ghosh, and D. Manocha. Mmau: A massive multi-task audio understanding and reasoning benchmark.arXiv preprint arXiv:2410.19168,

    arXiv

  25. [25]

    X. Shi, X. Wang, Z. Guo, Y. Wang, P. Zhang, X. Zhang, Z. Guo, H. Hao, Y. Xi, B. Yang, J. Xu, J. Zhou, and J. Lin. Qwen3-asr technical report.arXiv preprint arXiv:2601.21337,

    Pith/arXiv arXiv

  26. [26]

    Singh, A

    A. Singh, A. Fry, A. Perelman, A. Tart, A. Ganesh, A. El-Kishky, A. McLaughlin, A. Low, A. Ostrow, A. Ananthram, et al. Openai gpt-5 system card.arXiv preprint arXiv:2601.03267,

    Pith/arXiv arXiv

  27. [27]

    W. Tong, H. Guo, D. Ran, J. Chen, J. Lu, K. Wang, K. Li, X. Zhu, J. Li, K. Li, et al. Interactiveomni: A unified omni-modal model for audio-visual multi-turn dialogue.arXiv preprint arXiv:2510.13747,

    arXiv

  28. [28]

    Fun-audio-chat technical report.arXiv preprint arXiv:2512.20156,

    Tongyi. Fun-audio-chat technical report.arXiv preprint arXiv:2512.20156,

    arXiv

  29. [29]

    Umair, V

    M. Umair, V. Sarathy, and J. Ruiter. Large language models know what to say but not when to speak. InFindings of the Association for Computational Linguistics: EMNLP 2024, pages 15503–15514, Miami, Florida, USA, nov

    2024

  30. [30]

    doi: 10.18653/v1/2024

    Association for Computational Linguistics. doi: 10.18653/v1/2024. findings-emnlp.909. URLhttps://aclanthology.org/2024.findings-emnlp.909/. E. Wang, Q. Li, Z. Tang, and Y. Jia. Cross-modal knowledge distillation for speech large language models. arXiv preprint arXiv:2509.14930,

    doi:10.18653/v1/2024 2024

  31. [31]

    X. Wang, Y. Li, C. Fu, Y. Shen, L. Xie, K. Li, X. Sun, and L. Ma. Freeze-omni: A smart and low latency speech-to-speech dialogue model with frozen llm, 2024a. URLhttps://arxiv.org/abs/2411.00774. Y. Wang, X. Ma, G. Zhang, Y. Ni, A. Chandra, S. Guo, W. Ren, A. Arulraj, X. He, Z. Jiang, T. Li, M. Ku, K. Wang, A. Zhuang, R. Fan, X. Yue, and W. Chen. Mmlu-pro...

    Pith/arXiv arXiv

  32. [32]

    18 Raon-Speech Technical Report J. Xu, Z. Guo, H. Hu, Y. Chu, X. Wang, J. He, Y. Wang, X. Shi, T. He, X. Zhu, et al. Qwen3-omni technical report.arXiv preprint arXiv:2509.17765,

    Pith/arXiv arXiv

  33. [33]

    A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388,

    Pith/arXiv arXiv

  34. [34]

    Z. Yao, L. Guo, X. Yang, W. Kang, F. Kuang, Y. Yang, Z. Jin, L. Lin, and D. Povey. Zipformer: A faster and better encoder for automatic speech recognition.arXiv preprint arXiv:2310.11230,

    Pith/arXiv arXiv

  35. [35]

    cc/paper_files/paper/2019/file/1e8a19426224ca89e83cef47f1e7f53b-Paper.pdf

    URLhttps://proceedings.neurips. cc/paper_files/paper/2019/file/1e8a19426224ca89e83cef47f1e7f53b-Paper.pdf. B. Zhang, C. Guo, G. Yang, H. Yu, H. Zhang, H. Lei, J. Mai, J. Yan, K. Yang, M. Yang, et al. Minimax-speech: Intrinsic zero-shot text-to-speech with a learnable speaker encoder.arXiv preprint arXiv:2505.07916, 2025a. Q. Zhang, L. Cheng, C. Deng, Q. C...

    Pith/arXiv arXiv 2019

  36. [36]

    Detailed Model Configuration Table 8 summarizes the detailed architectural configurations of Raon-Speech and Raon-SpeechChat

    Encoder Architecture Hidden 1024, 24 layers, FFN 4096 (GELU) Output 2048 Hidden 1280, 32 layers, FFN 5120 (SiLU) Output 5120 (1280×4 frame stacking) Attention Pattern Non-causal, full-context Causal SW A (15 s window) Feature Extractor Conv downsampling Downsample hidden 480 Mel spectrogram followed by patch embedding Speech Input Adaptor 2-layer MLP, 204...

    2048

  37. [37]

    AlpacaEval appears in both blocks because we keep the benchmark groupings used for the main aggregate rows

    The VoiceBench block covers OpenBookQA, MMSU, BBH, SD-QA, AlpacaEval, CommonEval, WildVoice, IFEval, and AdvBench, whereas the OpenAudioBench block covers AlpacaEval, LlamaQ, TriviaQA, and WebQ. AlpacaEval appears in both blocks because we keep the benchmark groupings used for the main aggregate rows. The Average rows reproduce the aggregate scores report...

    arXiv

  38. [38]

    22 Raon-Speech Technical Report Table 10Detailed Korean spoken question answering results for Raon-Speech. The upper block unpacks the aggregate KVoiceBench row and the lower block unpacks the aggregate KOpenAudioBench row from the main table.Boldand underline indicate the best and the second-best performance, respectively. Benchmark Raon Qwen2.5 Audio St...

    arXiv