Pith. sign in

REVIEW 4 major objections 4 minor 2 cited by

Advancing Singlish Understanding: Bridging the Gap with Datasets and Multimodal Models

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper introduces the largest standardized spoken Singlish corpus and a multitask audio-LLM that reports 10–30% gains over prior systems.

desk verdict The MNSC dataset is a real contribution; the SingAudioLLM SOTA claim does not survive its own tables. read the letter →

arxiv 2501.01034 v2 pith:PWHWGMBY submitted 2025-01-02 cs.CL cs.SDeess.AS

Bin Wang , Xunlong Zou , Shuo Sun , Wenyu Zhang , Yingxu He , Zhuohan Liu , Chengwei Wei , Nancy F. Chen
show 1 more author
AiTi Aw
This is my paper · ORCID
classification cs.CLcs.SDeess.AS
keywords Singlishspokenlanguageunderstandingaudiolargemodelsmultimodalspeechrecognitionquestionansweringdialoguesummarizationparalinguisticanalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Singlish is a widely spoken creole in Singapore, but its spoken form has lacked the standardized, multitask resources that modern speech models need. This paper tries to close that gap by releasing MNSC, a cleaned and re-split version of the largest available Singlish speech corpus, with training data for speech recognition, spoken question answering, dialogue summarization, and paralinguistic question answering, plus human-verified test sets. It also proposes SingAudioLLM, a single audio-language model trained jointly on all four tasks. The paper reports that SingAudioLLM outperforms prior audio-LLMs and a cascade baseline by 10–30% on these benchmarks, and that the corpus helps even a standard ASR model adapt to Singlish. If the claims hold, researchers gain a reproducible benchmark and a reference model for a low-resource creole that has so far been limited mostly to ASR.

What carries the argument

The central object is MNSC, the Multitask National Speech Corpus: standardized splits over the National Speech Corpus, with synthetic training data for SQA and SDS created by prompting an LLM with transcripts, and human-annotated test sets. The central mechanism is the fusion pipeline of SingAudioLLM: a Whisper encoder turns audio into features, a Conv-1D adapter downsamples and projects them into the text-token space, and the Gemma2-9B LLM decoder, tuned with LoRA, reads the audio tokens alongside a task instruction and generates the answer. The paper argues that jointly training this pipeline across ASR, SQA, SDS, and PQA lets the model share acoustic and semantic information, while keeping the decoder's instruction-following ability intact.

What would settle it

The quickest test would be to take the released MNSC test sets and compute confidence intervals on the reported scores; with 100 items per subtask, if the 10–30% margins overlap between SingAudioLLM and the next-best model, the claimed lead is not established. A stronger test would be an independent, larger set of Singlish clips (e.g., 500 per task) with human-verified labels, scored by the same models: if the ordering or margin changes, the central claim fails.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that spoken Singlish understanding can be lifted from a single-task ASR problem to a four-task multimodal benchmark, and that an end-to-end audio LLM trained jointly on those tasks is the right way to attack it. To do this, the paper constructs MNSC by taking the National Speech Corpus, cleaning alignment errors, standardizing train/test splits, adding LLM-synthesized question-answer pairs and dialogue summaries for training, and building human-annotated test sets for SQA and SDS. SingAudioLLM uses a Whisper-large-v3 encoder, a Conv-1D adapter that compresses audio features, and a Gemma2-9B-Instruct decoder trained with LoRA; the same model answers prompts for ASR, SQA, SDS, and PQA. On MNSC it reports the best scores in all four task groups, and it claims a 10–30% improvement over other AudioLLMs and a Whisper-to-LLM cascade, with the largest gaps in paralinguistic tasks such as accent identification.

Load-bearing premise

The benchmark's value and the model's reported lead rest on machine-generated QA and summary training data being a faithful stand-in for real spoken comprehension, and on 100 human-checked test items per subtask being enough to separate models.

Editorial extensions

If this is right

  • The MNSC splits give the community a common yardstick for Singlish spoken ASR, SQA, SDS, and PQA, replacing the previous situation where benchmark splits did not exist.
  • A single jointly trained audio-LLM can match or beat a cascade that transcribes first and then runs an LLM, at least on the MNSC test sets, so end-to-end fusion is a viable design for creole speech understanding.
  • Fine-tuning Whisper on MNSC-ASR data cuts Singlish word error rates relative to the off-the-shelf Whisper and transfers to the SEAME code-switching dataset, indicating the corpus has value beyond the proposed model.
  • Decoder size matters for reasoning-heavy tasks such as QA and summarization, but not much for ASR, while encoder strength matters across tasks; this points to where future Singlish speech models should spend parameters.
  • The release of datasets, model, and code turns Singlish from a language with only ASR resources into a testbed for multilingual and code-switched multimodal understanding.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the synthetic training recipe transfers, the same pipeline—clean an existing speech corpus, let an LLM generate QA and summaries from transcripts, and verify a small test set by humans—could be applied to other under-resourced creoles and dialects, not just Singlish.
  • The PQA results are suggestive rather than conclusive: accent labels come from speaker metadata, so a model might be exploiting lexical cues rather than acoustic ones; a follow-up with de-lexicalized or masked audio would separate those.
  • Because SQA and SDS test sets contain only 100 samples per subtask, the 10–30% margins may not be statistically stable; reporting confidence intervals or expanding the test set would turn a promising result into a solid benchmark.
  • The observation that a task-specific fine-tuned Whisper beats the multitask model on ASR hints at a specialization–generalization trade-off; routing or an expert mixture could let one model keep both the multitask strengths and the ASR edge.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces MNSC, a multitask spoken Singlish benchmark derived from the National Speech Corpus, with standardized splits and human-annotated test sets for ASR, spoken question answering (SQA), spoken dialogue summarization (SDS), and paralinguistic question answering (PQA). The authors also propose SingAudioLLM, a fusion-style AudioLLM that combines a Whisper encoder, an adaptor, and an LLM decoder trained jointly on all four tasks. They report strong in-domain results, claim a 10-30% improvement over prior AudioLLMs and cascaded baselines, and release the dataset, code, and model. The main empirical evidence is in Tables 2 and 3, with component analyses in Figure 3.

Significance. If the benchmark is sound and the reported comparisons are statistically reliable, MNSC would be a valuable resource for a genuinely underserved language, and the release of standardized splits, human-annotated test sets, and code would lower the barrier for future work. The paper also has useful component analyses (encoder size, adaptor choice, decoder size) that speak to practical design questions for low-resource AudioLLMs. However, the central performance claim is not yet supported: the SQA/SDS results rest on 100-sample test sets with undisclosed model-as-judge scoring and no error bars, and several of the paper's own tables show baseline models beating SingAudioLLM on specific tasks. The dataset contribution is promising; the state-of-the-art claim needs substantially stronger evidence.

major comments (4)
  1. [§3.2, Table 2] The SQA/SDS test sets contain only n=100 human-annotated samples per subtask, and the paper reports no error bars, confidence intervals, or paired significance tests. For an accuracy score with n=100, the standard error can reach roughly 5 percentage points, so most of the reported margins are within plausible noise: SingAudioLLM is behind the cascade on MNSC-SQA PART 4 (51.8 vs 53.8), tied on MNSC-SQA PART 6 (64.0 vs 64.0), and leads by only 1 point on MNSC-SDS PART 5 (50.0 vs 49.0). The abstract's claim of outperforming prior models by 10-30% is not supportable from Table 2 without confidence intervals or paired tests.
  2. [§5, evaluation protocol] The evaluation section states that metrics use 'WER or model-as-judge scores' but never names the judge model, the prompt template, or the scoring protocol, and no agreement with human ratings is reported. This is load-bearing for the SQA/SDS results because the training targets were synthesized by LLAMA-3.1-70B (§3.2); if the same model family is used as the judge, the comparison may be biased toward outputs that resemble that model's style. Please disclose the judge, report human-model agreement, and provide an error analysis on the 100-sample test sets.
  3. [Tables 2 and 3, §5] The blanket 'state-of-the-art' claim is not supported by the paper's own evidence. In Table 3, FT-Whisper achieves lower WER than SingAudioLLM on MNSC-ASR Parts 2, 3, and 4, and on SEAME-Dev-SGE and SEAME-Dev-MAN; in Table 2, the cascade model beats SingAudioLLM on MNSC-SQA Part 4 and ties on Part 6. The claim should be restricted to the settings where it actually holds (for example, certain AudioLLM comparisons on MNSC), or the authors should provide an aggregate result with uncertainty that justifies the headline.
  4. [§3.1 and Limitations] The dataset construction caps dialogue duration at 30 seconds, excludes recordings that 'could not be perfectly aligned,' and the Limitations section admits that a substantial portion of the source corpus was excluded. Since no statistics or sensitivity analyses are reported for these exclusions, readers cannot tell whether the MNSC benchmark and the resulting model rankings are representative of the full NSC or are artifacts of the curation thresholds. Please report the amount and characteristics of excluded data and, if feasible, test robustness to the 30-second cap and the alignment threshold.
minor comments (4)
  1. [Abstract, §3.1, Table 1] There are typos and inconsistencies: 'models's' in the Abstract, 'Coprus' in §3.1, 'Pralinguistic' in Table 1, and 'NMSC' instead of 'MNSC' in the Table 3 caption. Please proofread the manuscript.
  2. [Appendix A, Table 4] The header 'MNSC-SQS-PART 3-6' should be 'MNSC-SQA-PART 3-6', and the appendix inconsistently uses 'Sing-AudioLLM' instead of 'SingAudioLLM'. Please unify the naming.
  3. [Figure 3] The legend and axes do not clearly separate the three studies (encoder size, adaptor, decoder); please mark the different conditions consistently and add confidence intervals or at least multiple-seed markers to support the qualitative claims about performance differences.
  4. [Appendix B] Appendix B is titled 'Hardward' instead of 'Hardware'. In addition, the paper does not state how many random seeds were used for LoRA training or for decoding, which is needed for reproducibility of the numerical comparisons.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the MNSC benchmark is grounded in external NSC audio and human annotations, and the SingAudioLLM comparison is an independent empirical evaluation.

full rationale

The paper's two central contributions, the MNSC dataset and the SingAudioLLM model, are not defined in terms of each other. MNSC is constructed from the externally collected National Speech Corpus (Koh et al., 2019), with standardized splits, metadata-derived PQA labels, LLM-synthesized SQA/SDS training targets, and human-annotated test sets. SingAudioLLM is a standard fusion architecture trained on those datasets and evaluated against external baselines such as Qwen2-Audio-Instruct, WavLLM, SALMONN, and a Whisper-Llama cascade, as well as on zero-shot hold-out sets including SEAME and public-speech QA. No equation or construction makes the model's scores depend on the benchmark definition: ASR uses WER with reference transcripts, PQA uses objective metadata labels, and the SQA/SDS test answers are human-written rather than generated by the model's own teacher. The only potential concern is that the 'model-as-judge' metric is not named and the SQA/SDS test sets are small (100 samples per subtask), but without identifying the judge there is no demonstrated reduction of the claimed result to its own inputs. Self-citations such as AudioBench, MeraLion-AudioLLM, and MoWE-Audio appear in related work or as sources of hold-out evaluation sets, but they are not load-bearing for the state-of-the-art claim. Accordingly, no circular step is identified.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new physical or formal entities are introduced. The list captures the design choices and background assumptions the empirical claims rest on. The most consequential are the reliance on LLM-synthesized training targets and the small human test sets.

free parameters (3)
  • Maximum dialogue duration = 30 seconds
    Dialogues are capped at 30 seconds, a design choice that simplifies processing but may truncate longer conversations and affect SQA/SDS content.
  • Adaptor scaling factor s = 3 (MLP-500), 15 (MLP-100)
    The Rescale-MLP adaptor uses scaling factors chosen by hand to control the number of audio tokens fed to the LLM; ablations compare them.
  • Data exclusion threshold = Not quantified
    Recordings with alignment noise or inappropriate annotations were excluded based on an unspecified confidence criterion, which shapes the corpus composition.
assumptions (4)
  • domain assumption NSC orthographic transcripts are accurate enough to serve as ASR labels and as the basis for generating QA and summary targets.
    The entire dataset construction in Section 3 relies on NSC transcriptions being correct, despite the paper noting misaligned annotations in the source corpus.
  • domain assumption LLM-generated question-answer pairs and summaries from transcripts are valid training targets for spoken comprehension tasks.
    Section 3.2 uses LLAMA-3.1-70B to synthesize SQA and SDS training data, assuming the synthetic outputs reflect the true task distribution.
  • domain assumption Whisper encoder features, after full fine-tuning, transfer to Singlish accents, code-switching, and local terms.
    Section 4.1 selects Whisper because of its multilingual ASR, and Section 4.4 fine-tunes it fully; the model's performance depends on this transfer.
  • ad hoc to paper A 100-sample human-annotated test set per SQA/SDS subtask is sufficient to rank models reliably.
    Section 3.2 describes 100 test samples per subtask; with such small samples, reported percentage differences may be statistical noise.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Advancing Singlish Understanding: Bridging the Gap with Datasets and Multimodal Models." pith.science (2026). https://pith.science/paper/PWHWGMBY

@misc{pith2026250101034,
  author       = {Pith},
  title        = {Pith review of: Advancing Singlish Understanding: Bridging the Gap with Datasets and Multimodal Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PWHWGMBY}},
  note         = {Machine review of arXiv:2501.01034}
}
read the original abstract

Singlish, a Creole language rooted in English, is a key focus in linguistic research within multilingual and multicultural contexts. However, its spoken form remains underexplored, limiting insights into its linguistic structure and applications. To address this gap, we standardize and annotate the largest spoken Singlish corpus, introducing the Multitask National Speech Corpus (MNSC). These datasets support diverse tasks, including Automatic Speech Recognition (ASR), Spoken Question Answering (SQA), Spoken Dialogue Summarization (SDS), and Paralinguistic Question Answering (PQA). We release standardized splits and a human-verified test set to facilitate further research. Additionally, we propose SingAudioLLM, a multi-task multimodal model leveraging multimodal large language models to handle these tasks concurrently. Experiments reveal our models adaptability to Singlish context, achieving state-of-the-art performance and outperforming prior models by 10-30% in comparison with other AudioLLMs and cascaded solutions.

Figures

Figures reproduced from arXiv: 2501.01034 by the authors.

Figure 1
Figure 1. A demonstration of our target multi-task learning datasets and end-to-end multimodal models for [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The architecture of SingAudioLLM incorporates a modality adaptor that processes audio features extracted [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. We conducted a study to assess the effectiveness of different encoder sizes, adaptor types, decoder sizes, [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. FastSLM: Hierarchical Temporal Abstraction for Efficient Long-Form Speech Adaptation

    eess.AS 2026-01 conditional novelty 5.0 of 10

    A hierarchical Q-Former compresses speech to about 1.67 tokens/sec, enabling hour-long audio processing with near-linear memory scaling and competitive benchmark scores.

  2. Incorporating Contextual Paralinguistic Understanding in Large Speech-Language Models

    cs.CL 2025-08 conditional novelty 5.0 of 10

    Training a speech-LLM on question-answer pairs generated with both discrete and continuous emotion labels improves its contextual emotion reasoning as scored by an LLM judge.

Reference graph

Works this paper leans on

45 extracted references · 18 canonical work pages · cited by 2 Pith papers

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Ashley Boo, Junwen Lee, and Ying-Ying Tan. 2023. Particle stacking in singlish--new data from the national speech corpus. Lingua, 287:103513

    work page 2023

  4. [4]

    Wenda Chen, Y Tan, E Chng, and Haizhou Li. 2010. The development of a singapore english call resource. Oriental COCOSDA, Nepal

    work page 2010

  5. [5]

    Yunfei Chu, Jin Xu, Qian Yang, Haojie Wei, Xipin Wei, Zhifang Guo, Yichong Leng, Yuanjun Lv, Jinzheng He, Junyang Lin, et al. 2024. Qwen2-audio technical report. arXiv preprint arXiv:2407.10759

    arXiv 2024

  6. [6]

    Yunfei Chu, Jin Xu, Xiaohuan Zhou, Qian Yang, Shiliang Zhang, Zhijie Yan, Chang Zhou, and Jingren Zhou. 2023. Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models. arXiv preprint arXiv:2311.07919

    arXiv 2023

  7. [7]

    Wenqian Cui, Dianzhi Yu, Xiaoqi Jiao, Ziqiao Meng, Guangyan Zhang, Qichao Wang, Yiwen Guo, and Irwin King. 2024. Recent advances in speech language models: A survey. arXiv preprint arXiv:2410.03751

    arXiv 2024

  8. [8]

    Alexandre D \'e fossez, Laurent Mazar \'e , Manu Orsini, Am \'e lie Royer, Patrick P \'e rez, Herv \'e J \'e gou, Edouard Grave, and Neil Zeghidour. 2024. Moshi: a speech-text foundation model for real-time dialogue. arXiv preprint arXiv:2410.00037

    arXiv 2024

Show all 45 references
  1. [9]

    David Deterding. 2007. Singapore English. Edinburgh University Press

    2007

  2. [10]

    David Deterding and Ee Ling Low. 2001. The nie corpus of spoken singapore english (niecsse). SAAL Quarterly, 56(1):2--5

    2001

  3. [11]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  4. [12]

    Stefan Elfwing, Eiji Uchibe, and Kenji Doya. 2018. Sigmoid-weighted linear units for neural network function approximation in reinforcement learning. Neural networks, 107:3--11

    2018

  5. [13]

    Robbie BH Goh. 2016. The anatomy of singlish: globalisation, multiculturalism and the construction of the ‘local’in singapore. Journal of Multilingual and Multicultural Development, 37(8):748--758

    2016

  6. [14]

    Yingxu He, Zhuohan Liu, Shuo Sun, Bin Wang, Wenyu Zhang, Xunlong Zou, Nancy F Chen, and Ai Ti Aw. 2024. Meralion-audiollm: Technical report. arXiv preprint arXiv:2412.09818

    2024 arXiv

  7. [15]

    Khaled Hechmi, Trung Ngo Trong, Ville Hautam \"a ki, and Tomi Kinnunen. 2021. Voxceleb enrichment for age and gender recognition. In 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pages 687--693. IEEE

    2021

  8. [16]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685

    2021 arXiv

  9. [17]

    Shujie Hu, Long Zhou, Shujie Liu, Sanyuan Chen, Hongkun Hao, Jing Pan, Xunying Liu, Jinyu Li, Sunit Sivasankaran, Linquan Liu, et al. 2024. Wavllm: Towards robust and adaptive speech large language model. arXiv preprint arXiv:2404.00656

    2024 arXiv

  10. [18]

    Shengpeng Ji, Yifu Chen, Minghui Fang, Jialong Zuo, Jingyu Lu, Hanting Wang, Ziyue Jiang, Long Zhou, Shujie Liu, Xize Cheng, et al. 2024. Wavchat: A survey of spoken dialogue models. arXiv preprint arXiv:2411.13577

    2024 arXiv

  11. [19]

    Heeseung Kim, Soonshin Seo, Kyeongseok Jeong, Ohsung Kwon, Soyoon Kim, Jungwhan Kim, Jaehong Lee, Eunwoo Song, Myungwoo Oh, Jung-Woo Ha, et al. 2024. Paralinguistics-aware speech-empowered large language models for natural conversation. In The Thirty-eighth Annual Conference o...

    2024

  12. [20]

    Suyoun Kim, Takaaki Hori, and Shinji Watanabe. 2017. Joint ctc-attention based end-to-end speech recognition using multi-task learning. In 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 4835--4839. IEEE

    2017

  13. [21]

    Jia Xin Koh, Aqilah Mislan, Kevin Khoo, Brian Ang, Wilson Ang, Charmaine Ng, and YY Tan. 2019. Building the singapore english national speech corpus. Malay, 20(25.0):19--3

    2019

  14. [22]

    Junwen Lee. 2022. An analysis of colloquial singapore english lah and its interpretation across speech acts. Languages, 7(3):203

    2022

  15. [23]

    Jiachen Li, Ali Hassani, Steven Walton, and Humphrey Shi. 2023 a . Convmlp: Hierarchical convolutional mlps for vision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6307--6316

    2023

  16. [24]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023 b . Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pages 19730--19742. PMLR

    2023

  17. [25]

    Yadong Li, Haoze Sun, Mingan Lin, Tianpeng Li, Guosheng Dong, Tao Zhang, Bowen Ding, Wei Song, Zhenglin Cheng, Yuqi Huo, et al. 2024. Baichuan-omni technical report. arXiv preprint arXiv:2410.08565

    2024

  18. [26]

    Lisa Lim. 2011. Tone in singlish: Substrate features from sinitic and malay. Substrate Features in Creole Languages, pages 271--288

    2011

  19. [27]

    Ke-Han Lu, Zhehuai Chen, Szu-Wei Fu, Chao-Han Huck Yang, Jagadeesh Balam, Boris Ginsburg, Yu-Chiang Frank Wang, and Hung-yi Lee. 2024. Developing instruction-following speech language model without speech instruction-tuning data. arXiv preprint arXiv:2409.20007

    2024 arXiv

  20. [28]

    Dau-Cheng Lyu, Tien Ping Tan, Engsiong Chng, and Haizhou Li. 2010. Seame: a mandarin-english code-switching speech corpus in south-east asia. In Interspeech, volume 10, pages 1986--1989

    2010

  21. [29]

    Tu Anh Nguyen, Benjamin Muller, Bokai Yu, Marta R Costa-Jussa, Maha Elbayad, Sravya Popuri, Paul-Ambroise Duquenne, Robin Algayres, Ruslan Mavlyutov, Itai Gat, et al. 2024. Spirit-lm: Interleaved spoken and written language model. arXiv preprint arXiv:2402.05755

    2024 arXiv

  22. [30]

    Jing Pan, Jian Wu, Yashesh Gaur, Sunit Sivasankaran, Zhuo Chen, Shujie Liu, and Jinyu Li. 2023. Cosmic: Data efficient instruction-tuning for speech in-context learning. arXiv preprint arXiv:2311.02248

    2023 arXiv

  23. [31]

    Jing Peng, Yucheng Wang, Yu Xi, Xv Li, and Kai Yu. 2024. A survey on speech large language models. arXiv preprint arXiv:2410.18908

    2024

  24. [32]

    Daniel Povey, Arnab Ghoshal, Gilles Boulianne, Lukas Burget, Ondrej Glembek, Nagendra Goel, Mirko Hannemann, Petr Motlicek, Yanmin Qian, Petr Schwarz, et al. 2011. The kaldi speech recognition toolkit. In IEEE 2011 workshop on automatic speech recognition and understanding. IE...

    2011

  25. [33]

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In International Conference on Machine Learning, pages 28492--28518. PMLR

    2023

  26. [34]

    Paul K Rubenstein, Chulayuth Asawaroengchai, Duc Dung Nguyen, Ankur Bapna, Zal \'a n Borsos, F \'e lix de Chaumont Quitry, Peter Chen, Dalia El Badawy, Wei Han, Eugene Kharitonov, et al. 2023. Audiopalm: A large language model that can speak and listen. arXiv preprint arXiv:2306.12925

    2023 arXiv

  27. [35]

    Ying-Ying Tan. 2019. Spontaneous speech elicitation for large speech corpus in multilingual singapore. IPSS

    2019

  28. [36]

    Changli Tang, Wenyi Yu, Guangzhi Sun, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun MA, and Chao Zhang. 2024. SALMONN : Towards generic hearing abilities for large language models. In International Conference on Learning Representations (ICLR)

    2024

  29. [37]

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. 2023. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805

    2023 arXiv

  30. [38]

    Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivi \`e re, Mihir Sanjay Kale, Juliette Love, et al. 2024. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295

    2024 arXiv

  31. [39]

    Roberto Togneri and Daniel Pullella. 2011. An overview of speaker identification: Accuracy and robustness issues. IEEE circuits and systems magazine, 11(2):23--61

    2011

  32. [40]

    Bin Wang, Zhengyuan Liu, and Nancy Chen. 2023 a . https://doi.org/10.18653/v1/2023.emnlp-main.474 Instructive dialogue summarization with query aggregations . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7630--7653, Singapore...

    2023 doi

  33. [41]

    Bin Wang, Xunlong Zou, Geyu Lin, Shuo Sun, Zhuohan Liu, Wenyu Zhang, Zhengyuan Liu, AiTi Aw, and Nancy F Chen. 2024. Audiobench: A universal benchmark for audio large language models. arXiv preprint arXiv:2406.16020

    2024 arXiv

  34. [42]

    Smith, Daniel Khashabi, and Hannaneh Hajishirzi

    Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023 b . https://doi.org/10.18653/v1/2023.acl-long.754 Self-instruct: Aligning language models with self-generated instructions . In Proceedings of the 61st Annual ...

    2023 doi

  35. [43]

    Shinji Watanabe, Takaaki Hori, Shigeki Karita, Tomoki Hayashi, Jiro Nishitoba, Yuya Unno, Nelson Enrique Yalta Soplin , Jahn Heymann, Matthew Wiesner, Nanxin Chen, Adithya Renduchintala, and Tsubasa Ochiai. 2018. https://doi.org/10.21437/Interspeech.2018-1456 ESPnet : End-to-e...

    2018 doi

  36. [44]

    Chih-Kai Yang, Yu-Kuan Fu, Chen-An Li, Yi-Cheng Lin, Yu-Xiang Lin, Wei-Chih Chen, Ho Lam Chung, Chun-Yi Kuan, Wei-Ping Huang, Ke-Han Lu, et al. 2024. Building a taiwanese mandarin spoken language model: A first attempt. arXiv preprint arXiv:2411.07111

    2024 arXiv

  37. [45]

    Wenyu Zhang, Shuo Sun, Bin Wang, Xunlong Zou, Zhuohan Liu, Yingxu He, Geyu Lin, Nancy F Chen, and Ai Ti Aw. 2025. Mowe-audio: Multitask audiollms with mixture of weak encoders. ICASSP 2025

    2025

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.