pith. machine review for the scientific record. sign in

arxiv: 2604.06327 · v1 · submitted 2026-04-07 · 💻 cs.SD · cs.AI

Recognition: 2 theorem links

· Lean Theorem

A Novel Automatic Framework for Speaker Drift Detection in Synthesized Speech

Jia-Hong Huang , Seulgi Kim , Yi Chieh Liu , Yixian Shen , Hongyi Zhu , Prayag Tiwari , Stevan Rudinac , Evangelos Kanoulas
Authors on Pith no claims yet

Pith reviewed 2026-05-10 18:21 UTC · model grok-4.3

classification 💻 cs.SD cs.AI
keywords speaker driftsynthesized speechtext-to-speechcosine similaritylarge language modelsbinary classificationspeech embeddingsbenchmark dataset
0
0 comments X

The pith

Speaker drift in synthesized speech is detected automatically by computing cosine similarities across overlapping segments and prompting large language models to classify consistency.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes the first automatic framework for identifying speaker drift, a gradual shift in perceived speaker identity within a single utterance of text-to-speech output. This matters for maintaining coherence in long-form or interactive synthetic speech applications where drift currently goes unnoticed. Detection is cast as a binary classification task on utterance-level speaker consistency. Cosine similarity is measured between speaker embeddings from overlapping segments, and the resulting values are structured as input for large language models to decide whether drift occurred. A new benchmark dataset with human-validated annotations allows evaluation, and experiments confirm that the pipeline functions with multiple state-of-the-art models while speaker embeddings show geometric clustering on the unit sphere.

Core claim

Speaker drift detection is formulated as a binary classification task over utterance-level speaker consistency. Cosine similarity is computed across overlapping segments of synthesized speech, and these structured representations are supplied to large language models to assess whether drift is present. Theoretical guarantees are supplied for the cosine-based detection step, and speaker embeddings are shown to exhibit meaningful geometric clustering on the unit sphere. A high-quality synthetic benchmark with human-validated drift annotations supports systematic evaluation, and experiments demonstrate the viability of the overall embedding-to-reasoning pipeline across several large language模型.

What carries the argument

The embedding-to-reasoning pipeline that extracts speaker embeddings from overlapping segments, computes their cosine similarities, structures the results, and prompts large language models to classify the presence of drift.

If this is right

  • Long-form synthesized speech can be scanned automatically for internal consistency without requiring human listeners.
  • Geometric clustering of embeddings on the unit sphere supplies a foundation for further signal-analysis approaches to drift.
  • The pipeline can be inserted into existing text-to-speech systems as a post-generation quality check.
  • Standardized benchmarks now exist for comparing alternative drift detection methods.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same segment-wise comparison plus reasoning structure could be tested on detecting other gradual inconsistencies such as prosody or emotion drift.
  • Replacing the large language model step with a smaller classifier trained on the benchmark might enable real-time drift monitoring during synthesis.
  • The observed unit-sphere clustering raises the possibility that drift thresholds could be derived directly from embedding geometry without external models.

Load-bearing premise

Cosine similarity between speaker embeddings from overlapping segments reliably tracks human perception of gradual shifts in speaker identity.

What would settle it

Run the framework on the human-annotated benchmark and measure whether its binary classifications match the human drift labels at a rate significantly above random guessing; systematic mismatch would falsify the detection claim.

read the original abstract

Recent diffusion-based text-to-speech (TTS) models achieve high naturalness and expressiveness, yet often suffer from speaker drift, a subtle, gradual shift in perceived speaker identity within a single utterance. This underexplored phenomenon undermines the coherence of synthetic speech, especially in long-form or interactive settings. We introduce the first automatic framework for detecting speaker drift by formulating it as a binary classification task over utterance-level speaker consistency. Our method computes cosine similarity across overlapping segments of synthesized speech and prompts large language models (LLMs) with structured representations to assess drift. We provide theoretical guarantees for cosine-based drift detection and demonstrate that speaker embeddings exhibit meaningful geometric clustering on the unit sphere. To support evaluation, we construct a high-quality synthetic benchmark with human-validated speaker drift annotations. Experiments with multiple state-of-the-art LLMs confirm the viability of this embedding-to-reasoning pipeline. Our work establishes speaker drift as a standalone research problem and bridges geometric signal analysis with LLM-based perceptual reasoning in modern TTS.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The paper introduces the first automatic framework for detecting speaker drift in synthesized speech from diffusion-based TTS models. It formulates speaker drift detection as a binary classification task over utterance-level speaker consistency, computes cosine similarity across overlapping segments of speaker embeddings, and prompts LLMs with structured representations (e.g., similarity matrices) to classify drift. The manuscript claims theoretical guarantees for the cosine-based approach, demonstrates geometric clustering of embeddings on the unit sphere, constructs a human-validated synthetic benchmark, and reports experiments with multiple state-of-the-art LLMs confirming the viability of the embedding-to-reasoning pipeline.

Significance. If the central claims hold after addressing the gaps, this would be a moderately significant contribution by establishing speaker drift as a distinct research problem in TTS and by proposing a hybrid geometric-LLM pipeline that could improve coherence in long-form synthetic speech. The benchmark dataset construction is a positive step, but the overall impact is limited by the current lack of detailed validation and supporting analysis for the key assumptions.

major comments (3)
  1. [Abstract and theoretical analysis section] Abstract and theoretical analysis section: The manuscript asserts 'theoretical guarantees for cosine-based drift detection' without providing any derivation details, formal proofs, error bounds, or analysis of assumptions (e.g., how overlap affects the similarity metric). This is load-bearing for the central claim of a reliable automatic framework.
  2. [Benchmark construction and evaluation section] Benchmark construction and evaluation section: Human validation of the synthetic dataset is mentioned but without process specifics, inter-annotator agreement, or any reported correlation metrics between cosine similarity scores and human drift annotations. This leaves unverified the key assumption that cosine similarity on overlapping segments serves as a reliable proxy for human-perceived gradual speaker drift.
  3. [Experiments section] Experiments section: No benchmark statistics, error analysis, ablation studies on pipeline components (cosine vs. LLM), or quantitative results on how well the geometric scores align with annotations are provided, despite claims of experimental validation with multiple LLMs. This undermines assessment of the method's robustness.
minor comments (2)
  1. [Method section] The description of 'structured representations' passed to the LLMs would benefit from a concrete example or figure to clarify the input format.
  2. [Method section] Notation for speaker embeddings and segment overlap could be defined more explicitly at first use to improve readability.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for their constructive and detailed feedback. We address each major comment below and will revise the manuscript to incorporate the suggested improvements for greater rigor and transparency.

read point-by-point responses

  1. Referee: [Abstract and theoretical analysis section] Abstract and theoretical analysis section: The manuscript asserts 'theoretical guarantees for cosine-based drift detection' without providing any derivation details, formal proofs, error bounds, or analysis of assumptions (e.g., how overlap affects the similarity metric). This is load-bearing for the central claim of a reliable automatic framework.

    Authors: We agree that the theoretical analysis section requires expansion to fully substantiate the claims. The current manuscript provides an overview of the geometric clustering on the unit sphere and the motivation for cosine similarity but omits detailed derivations. In the revised version, we will add a dedicated subsection containing formal proofs for the cosine-based drift detection, including the underlying assumptions, error bounds, and an analysis of how segment overlap influences the similarity metric. This will strengthen the load-bearing theoretical component of the framework. revision: yes

  2. Referee: [Benchmark construction and evaluation section] Benchmark construction and evaluation section: Human validation of the synthetic dataset is mentioned but without process specifics, inter-annotator agreement, or any reported correlation metrics between cosine similarity scores and human drift annotations. This leaves unverified the key assumption that cosine similarity on overlapping segments serves as a reliable proxy for human-perceived gradual speaker drift.

    Authors: We acknowledge that additional details on the human validation process are necessary to verify the proxy assumption. The revised manuscript will include specifics on the annotation protocol (e.g., guidelines provided to annotators, number of annotators involved), inter-annotator agreement metrics (such as Fleiss' kappa), and correlation analyses (e.g., Pearson or Spearman coefficients) between the cosine similarity scores and the human drift annotations. These additions will directly address the verification of the key assumption. revision: yes

  3. Referee: [Experiments section] Experiments section: No benchmark statistics, error analysis, ablation studies on pipeline components (cosine vs. LLM), or quantitative results on how well the geometric scores align with annotations are provided, despite claims of experimental validation with multiple LLMs. This undermines assessment of the method's robustness.

    Authors: We agree that the experiments section would benefit from more comprehensive reporting to demonstrate robustness. In the revision, we will add benchmark statistics (including dataset size and drift distribution), error analysis, ablation studies isolating the cosine similarity component versus the full LLM pipeline, and quantitative metrics evaluating the alignment between geometric scores and human annotations. These enhancements will provide a clearer assessment of the method's performance and reliability across the tested LLMs. revision: yes

Circularity Check

0 steps flagged

No significant circularity; derivation uses external metrics and independent validation

full rationale

The paper formulates speaker drift detection as binary classification using cosine similarity on speaker embeddings from overlapping segments, followed by LLM prompting on structured representations. These components rely on standard, externally defined tools (cosine similarity, pre-trained embeddings, LLMs) rather than self-defined quantities or fitted parameters renamed as predictions. The claimed theoretical guarantees for cosine-based detection and geometric clustering demonstration do not reduce to the inputs by construction, as they are presented as properties on the unit sphere. The human-validated benchmark provides separate evaluation data. No self-citation chains, uniqueness theorems from authors, or ansatz smuggling appear load-bearing. The central pipeline remains self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The framework rests on the domain assumption that speaker embeddings form geometrically meaningful clusters on the unit sphere and that cosine similarity serves as a proxy for perceptual consistency; no free parameters or invented entities are explicitly introduced in the abstract.

axioms (1)
  • domain assumption Speaker embeddings exhibit meaningful geometric clustering on the unit sphere
    Stated as demonstrated in the abstract; used to support cosine-based detection

pith-pipeline@v0.9.0 · 5496 in / 1288 out tokens · 39482 ms · 2026-05-10T18:21:01.690004+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

48 extracted references · 9 canonical work pages · 2 internal anchors

  1. [1]

    INTRODUCTION Recent progress in text-to-speech (TTS) synthesis, particu- larly with diffusion-based models, has significantly enhanced the naturalness, expressiveness, and controllability of gen- erated speech [1–19]. These models can synthesize long- form utterances with high perceptual fidelity, supporting ap- plications such as personalized virtual ass...

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  2. [2]

    no drift

    DATASET CONSTRUCTION 2.1 Overview To systematically study the speaker drift phenomenon in synthetic speech, we construct a benchmark dataset designed for binary classification, determining whether a speaker’s identity remains consistent or shifts within a given utterance. Reflecting real-world scenarios where speaker consistency is crucial, the dataset fe...

  3. [3]

    We for- malize the task as a binary classification problem

    METHOD 3.1 Problem Formulation Given a speech utterancex(t), divided into three contigu- ous segments(s 1, s2, s3)of equal duration, the goal is to de- tect whether the speaker identity remains consistent through- out or experiences a drift at one or more boundaries. We for- malize the task as a binary classification problem. Let the labely∈ {0,1}, wherey...

  4. [4]

    Each sample is 9 to 40 seconds long

    EXPERIMENTS 4.1 Experimental Setup Dataset and Evaluation.We use the dataset described in Section III, consisting of 128 samples, 64 with speaker drift and 64 without, synthesized from 384 high-quality utterances by different speakers and verified by human annotators. Each sample is 9 to 40 seconds long. For evaluation, we report ac- curacy and F1 score. ...

  5. [5]

    Our method bridges low-level acoustic embeddings with high-level evaluation and is supported by a new bench- mark dataset with human-verified annotations

    CONCLUSION AND FUTURE WORK In this work, we introduced the first automated framework for detecting speaker drift in diffusion-based TTS, leveraging cosine similarity as a theoretically grounded proxy for vocal identity consistency and prompting LLMs for perceptual rea- soning. Our method bridges low-level acoustic embeddings with high-level evaluation and...

  6. [6]

    Tacotron: Towards end-to-end speech synthesis,

    Yuxuan Wang, RJ Skerry-Ryan, Daisy Stanton, Yonghui Wu, Ron J Weiss, Navdeep Jaitly, Zongheng Yang, Ying Xiao, Zhifeng Chen, Samy Bengio, et al., “Tacotron: Towards end-to-end speech synthesis,”arXiv preprint arXiv:1703.10135, 2017

    work page arXiv 2017

  7. [7]

    WaveNet: A Generative Model for Raw Audio

    Aaron Van Den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, Koray Kavukcuoglu, et al., “Wavenet: A generative model for raw audio,” arXiv preprint arXiv:1609.03499, vol. 12, 2016

    work page internal anchor Pith review arXiv 2016

  8. [8]

    Autoregressive speech synthesis without vector quantization

    Lingwei Meng, Long Zhou, Shujie Liu, Sanyuan Chen, Bing Han, Shujie Hu, Yanqing Liu, Jinyu Li, Sheng Zhao, Xixin Wu, et al., “Autoregressive speech syn- thesis without vector quantization,”arXiv preprint arXiv:2407.08551, 2024

    work page arXiv 2024

  9. [9]

    V oicebox: Text-guided multilingual universal speech generation at scale,

    Matthew Le, Apoorv Vyas, Bowen Shi, Brian Kar- rer, Leda Sari, Rashel Moritz, Mary Williamson, Vimal Manohar, Yossi Adi, Jay Mahadeokar, et al., “V oicebox: Text-guided multilingual universal speech generation at scale,”Advances in neural information processing sys- tems, vol. 36, pp. 14005–14034, 2023

    2023

  10. [10]

    Naturalspeech 2: Latent diffusion models are natural and zero-shot speech and singing synthesizers.arXiv preprint arXiv:2304.09116, 2023

    Kai Shen, Zeqian Ju, Xu Tan, Yanqing Liu, Yichong Leng, Lei He, Tao Qin, Sheng Zhao, and Jiang Bian, “Naturalspeech 2: Latent diffusion models are natural and zero-shot speech and singing synthesizers,”arXiv preprint arXiv:2304.09116, 2023

    work page arXiv 2023

  11. [11]

    Natural tts synthesis by conditioning wavenet on mel spectrogram predictions,

    Jonathan Shen, Ruoming Pang, Ron J Weiss, Mike Schuster, Navdeep Jaitly, Zongheng Yang, Zhifeng Chen, Yu Zhang, Yuxuan Wang, Rj Skerrv-Ryan, et al., “Natural tts synthesis by conditioning wavenet on mel spectrogram predictions,” in2018 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, 2018, pp. 4779–4783

    2018

  12. [12]

    A novel evaluation framework for image2text gener- ation,

    Jia-Hong Huang, Hongyi Zhu, Yixian Shen, Stevan Rudinac, Alessio M. Pacces, and Evangelos Kanoulas, “A novel evaluation framework for image2text gener- ation,” inInternational ACM SIGIR Conference on Research and Development in Information Retrieval, LLM4Eval Workshop, 2024

    2024

  13. [13]

    Fastspeech: Fast, robust and controllable text to speech,

    Yi Ren, Yangjun Ruan, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, and Tie-Yan Liu, “Fastspeech: Fast, robust and controllable text to speech,”Advances in neural information processing systems, vol. 32, 2019

    2019

  14. [14]

    Diff-tts: A denoising diffusion model for text-to-speech.arXiv preprint arXiv:2104.01409, 2021

    Myeonghun Jeong, Hyeongju Kim, Sung Jun Cheon, Byoung Jin Choi, and Nam Soo Kim, “Diff-tts: A denoising diffusion model for text-to-speech,”arXiv preprint arXiv:2104.01409, 2021

    work page arXiv 2021

  15. [15]

    Grad-tts: A diffusion probabilistic model for text-to-speech,

    Vadim Popov, Ivan V ovk, Vladimir Gogoryan, Tasnima Sadekova, and Mikhail Kudinov, “Grad-tts: A diffusion probabilistic model for text-to-speech,” inICML, 2021, pp. 8599–8608

    2021

  16. [16]

    Schrodinger bridges beat diffusion models on text-to-speech synthesis,

    Zehua Chen, Guande He, Kaiwen Zheng, Xu Tan, and Jun Zhu, “Schrodinger bridges beat diffusion models on text-to-speech synthesis,”arXiv preprint arXiv:2312.03491, 2023

    work page arXiv 2023

  17. [17]

    Image2text2image: A novel framework for label-free evaluation of image- to-text generation with text-to-image diffusion models,

    Jia-Hong Huang, Hongyi Zhu, Yixian Shen, Stevan Rudinac, and Evangelos Kanoulas, “Image2text2image: A novel framework for label-free evaluation of image- to-text generation with text-to-image diffusion models,” inInternational Conference on Multimedia Modeling. Springer, 2025, pp. 413–427

    2025

  18. [18]

    Towards fine-grained citation evaluation in generated text: A comparative analysis of faithfulness metrics,

    Weijia Zhang, Mohammad Aliannejadi, Yifei Yuan, Ji- ahuan Pei, Jia-Hong Huang, and Evangelos Kanoulas, “Towards fine-grained citation evaluation in generated text: A comparative analysis of faithfulness metrics,” in Proceedings of the 17th International Natural Language Generation Conference, 2024, pp. 427–439

    2024

  19. [19]

    Sample-efficient diffusion for text-to-speech synthesis,

    Justin Lovelace, Soham Ray, Kwangyoun Kim, Kil- ian Q Weinberger, and Felix Wu, “Sample-efficient diffusion for text-to-speech synthesis,”arXiv preprint arXiv:2409.03717, 2024

    work page arXiv 2024

  20. [20]

    BASE TTS: lessons from building a billion- parameter text-to-speech model on 100k hours of data

    Mateusz Lajszczak, Guillermo C ´ambara, Yang Li, Fatih Beyhan, Arent Van Korlaar, Fan Yang, Arnaud Joly, ´Alvaro Mart´ın-Cortinas, Ammar Abbas, Adam Michal- ski, et al., “Base tts: Lessons from building a billion- parameter text-to-speech model on 100k hours of data,” arXiv preprint arXiv:2402.08093, 2024

    work page arXiv 2024

  21. [21]

    Gradient weight- normalized low-rank projection for efficient llm train- ing,

    Jia-Hong Huang, Yixian Shen, Hongyi Zhu, Stevan Rudinac, and Evangelos Kanoulas, “Gradient weight- normalized low-rank projection for efficient llm train- ing,” inProceedings of the AAAI Conference on Artifi- cial Intelligence, 2025, vol. 39, pp. 24123–24131

    2025

  22. [22]

    Matcha-tts: A fast tts archi- tecture with conditional flow matching,

    Shivam Mehta, Ruibo Tu, Jonas Beskow, ´Eva Sz´ekely, and Gustav Eje Henter, “Matcha-tts: A fast tts archi- tecture with conditional flow matching,” inICASSP 2024-2024 IEEE International Conference on Acous- tics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 11341–11345

    2024

  23. [23]

    E1 tts: Simple and fast non- autoregressive tts,

    Zhijun Liu, Shuai Wang, Pengcheng Zhu, Mengxiao Bi, and Haizhou Li, “E1 tts: Simple and fast non- autoregressive tts,” inICASSP 2025-2025 IEEE Inter- national Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2025, pp. 1–5

    2025

  24. [24]

    Continuous- token diffusion for speaker-referenced tts in multimodal llms,

    Xinlu He, Swayambhu Nath Ray, Harish Mallidi, Jia-Hong Huang, Ashwin Bellur, Chander Chandak, M Maruf, and Venkatesh Ravichandran, “Continuous- token diffusion for speaker-referenced tts in multimodal llms,”The Thirty-ninth Annual Conference on Neu- ral Information Processing Systems (NeurIPS) Work- shop on Structured Probabilistic Inference & Genera- tiv...

    2025

  25. [25]

    Query-based video summarization with pseudo label supervision,

    Jia-Hong Huang, Luka Murn, Marta Mrak, and Mar- cel Worring, “Query-based video summarization with pseudo label supervision,” in2023 IEEE International Conference on Image Processing (ICIP). IEEE, 2023, pp. 1430–1434

    2023

  26. [26]

    Enhancing interactive image re- trieval with query rewriting using large language mod- els and vision language models,

    Hongyi Zhu, Jia-Hong Huang, Stevan Rudinac, and Evangelos Kanoulas, “Enhancing interactive image re- trieval with query rewriting using large language mod- els and vision language models,” inProceedings of the 2024 International Conference on Multimedia Re- trieval, 2024, pp. 978–987

    2024

  27. [27]

    Interactive image retrieval meets query rewriting with large language and vision language models,

    Hongyi Zhu, Jia-Hong Huang, Yixian Shen, Stevan Rudinac, and Evangelos Kanoulas, “Interactive image retrieval meets query rewriting with large language and vision language models,”ACM Transactions on Mul- timedia Computing, Communications and Applications, vol. 21, no. 10, pp. 1–23, 2025

    2025

  28. [28]

    Multi-modal video summarization,

    Jia-Hong Huang, “Multi-modal video summarization,” inProceedings of the 2024 International Conference on Multimedia Retrieval, 2024, pp. 1214–1218

    2024

  29. [29]

    Reasoning beyond points: A visual introspective ap- proach for few-shot 3d segmentation,

    Changshuo Wang, Shuting He, Xiang Fang, Zhijian Hu, Jia-Hong Huang, Yixian Shen, and Prayag Tiwari, “Reasoning beyond points: A visual introspective ap- proach for few-shot 3d segmentation,” inThe Thirty- ninth Annual Conference on Neural Information Pro- cessing Systems, 2025

    2025

  30. [30]

    Macp: Mini- mal yet mighty adaptation via hierarchical cosine pro- jection,

    Yixian Shen, Qi Bi, Jia-Hong Huang, Hongyi Zhu, Andy D Pimentel, and Anuj Pathania, “Macp: Mini- mal yet mighty adaptation via hierarchical cosine pro- jection,” inProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2025, pp. 20602–20618

    2025

  31. [31]

    Synthesiz- ing new retinal symptom images by multiple genera- tive models,

    Yi-Chieh Liu, Hao-Hsiang Yang, C-H Huck Yang, Jia- Hong Huang, Meng Tian, Hiromasa Morikawa, Yi- Chang James Tsai, and Jesper Tegner, “Synthesiz- ing new retinal symptom images by multiple genera- tive models,” inAsian Conference on Computer Vision. Springer, 2018, pp. 235–250

    2018

  32. [32]

    Query- controllable video summarization,

    Jia-Hong Huang and Marcel Worring, “Query- controllable video summarization,” inProceedings of the International Conference on Multimedia Retrieval, 2020, pp. 242–250

    2020

  33. [33]

    Ssh: Sparse spectrum adaptation via discrete hartley transforma- tion,

    Yixian Shen, Qi Bi, Jia-Hong Huang, Hongyi Zhu, Andy D Pimentel, and Anuj Pathania, “Ssh: Sparse spectrum adaptation via discrete hartley transforma- tion,” inProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Tech- nologies (Volume 1: Long Papers), 2025, pp. 10400– 10415

    2025

  34. [34]

    Gpt2mvs: Generative pre-trained transformer-2 for multi-modal video summarization,

    Jia-Hong Huang, Luka Murn, Marta Mrak, and Mar- cel Worring, “Gpt2mvs: Generative pre-trained transformer-2 for multi-modal video summarization,” in Proceedings of the International Conference on Multi- media Retrieval, 2021, pp. 580–589

    2021

  35. [35]

    The dawn of quantum natural language processing,

    Riccardo Di Sipio, Jia-Hong Huang, Samuel Yen-Chi Chen, Stefano Mangini, and Marcel Worring, “The dawn of quantum natural language processing,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 8612–8616

    2022

  36. [36]

    Beyond relevant documents: A knowledge-intensive approach for query-focused summarization using large language models,

    Weijia Zhang, Jia-Hong Huang, Svitlana Vakulenko, Yumo Xu, Thilina Rajapakse, and Evangelos Kanoulas, “Beyond relevant documents: A knowledge-intensive approach for query-focused summarization using large language models,” inInternational Conference on Pat- tern Recognition. Springer, 2024, pp. 89–104

    2024

  37. [37]

    Causal video summarizer for video exploration,

    Jia-Hong Huang, Chao-Han Huck Yang, Pin-Yu Chen, Andrew Brown, and Marcel Worring, “Causal video summarizer for video exploration,” in2022 IEEE Inter- national Conference on Multimedia and Expo (ICME). IEEE, 2022, pp. 1–6

    2022

  38. [38]

    Robust speaker change detection,

    Jitendra Ajmera, Iain McCowan, and Herv ´e Bourlard, “Robust speaker change detection,”IEEE signal pro- cessing letters, vol. 11, no. 8, pp. 649–651, 2004

    2004

  39. [39]

    Fully supervised speaker di- arization,

    Aonan Zhang, Quan Wang, Zhenyao Zhu, John Pais- ley, and Chong Wang, “Fully supervised speaker di- arization,” inICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019, pp. 6301–6305

    2019

  40. [40]

    Turn-to-diarize: Online speaker diarization constrained by transformer transducer speaker turn detection,

    Wei Xia, Han Lu, Quan Wang, Anshuman Tripathi, Yil- ing Huang, Ignacio Lopez Moreno, and Hasim Sak, “Turn-to-diarize: Online speaker diarization constrained by transformer transducer speaker turn detection,” in ICASSP, 2022, pp. 8077–8081

    2022

  41. [41]

    Gpt-4 technical report,

    OpenAI, “Gpt-4 technical report,” 2024

    2024

  42. [42]

    Gemini: A family of highly capable multimodal models,

    Gemini Team, “Gemini: A family of highly capable multimodal models,” 2024

    2024

  43. [43]

    Claude (language model),

    Anthropic, “Claude (language model),”https: //www.anthropic.com/claude/sonnet, 2025, Accessed: 2025-05-15

    2025

  44. [44]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,

    DeepSeek-AI, “Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,” 2025

    2025

  45. [45]

    Qwen technical re- port,

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Zhenru Zhang, Chang Zhou, Jingren Zhou, Xi- aohuan Zhou, and Tianhang Zhu, “Qwen technical re- port,” 2023

    2023

  46. [46]

    wav2vec 2.0: A framework for self-supervised learning of speech representations,

    Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” NeurIPS, vol. 33, pp. 12449–12460, 2020

    2020

  47. [47]

    Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences,

    Steven Davis and Paul Mermelstein, “Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences,”IEEE Transactions on Acoustics, Speech, and Signal Process- ing, vol. 28, no. 4, pp. 357–366, 1980

    1980

  48. [48]

    Robust speech recognition via large-scale weak supervision,

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brock- man, Christine McLeavey, and Ilya Sutskever, “Robust speech recognition via large-scale weak supervision,” in ICML, 2023, pp. 28492–28518

    2023