Pith. sign in

REVIEW 4 major objections 6 minor 34 references

AURA: Agent for Understanding, Reasoning, and Automated Tool Use in Voice-Driven Tasks

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An open-source speech assistant that reasons and calls real tools matches GPT-4o on a spoken QA benchmark.

desk verdict A useful open-source voice-agent system, but the headline VoiceBench comparison is confounded by web search the baselines don't get. read the letter →

arxiv 2506.23049 v1 pith:NAVCG2HS submitted 2025-06-29 cs.AI cs.CLcs.SDeess.AS

Leander Melroy Maben , Gayathri Ganesh Lakshmy , Srijith Radhakrishnan , Siddhant Arora , Shinji Watanabe This is my paper
classification cs.AIcs.CLcs.SDeess.AS
keywords speechagentsReActreasoningtool-augmentedLLMsspokendialoguesystemscascadedpipelineVoiceBenchOpenBookQAspeech-to-speechassistant
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AURA is a fully voice-based assistant assembled from open-weight components: speech recognition, a large language model, speech synthesis, and a small set of real tools (calendar, contacts, email, web search). The paper's central claim is that this is the first open-source speech-to-speech agent that combines reasoning with tool use, letting users complete compound goals in multi-turn dialogue. The evidence includes a 92.75% accuracy score on VoiceBench's OpenBookQA task, which the authors say beats all open-weight systems and nearly matches GPT-4o at 92.97%, and a 90% success rate across 30 human-evaluated tasks. If correct, the result shows that practical spoken task-oriented agents do not require closed models; they can be built from openly available parts and a modular control loop.

What carries the argument

The load-bearing object is the ReAct-style agent loop: the LLM output is structured as a Thought, an Action type, and a Payload, the controller executes the action (chat, calendar booking, web search, contact lookup, or email send), and the resulting observation is written back into a state that tracks action-observation history, conversation history, and a structured dialog state. That loop lets a pure text LLM drive a spoken interface, and lets tool access be extended by declaring new action classes with natural-language prompts. A second mechanism is the explicit prompt that forces at least one web search before answering, which the paper shows improves OpenBookQA accuracy. Dialog state tracking is done in the same prompt-based style, extracting structured state with LLM inference rather than fine-tuning.

What would settle it

Run AURA on the same VoiceBench OpenBookQA split with the web-search tool disabled. If accuracy falls to 87.69% (the no-search number reported) or to the level of the open-weight baselines, then the 92.75% result comes from tool-grounded retrieval rather than from the agent's reasoning, and the claimed edge over other open-weight systems disappears when all systems answer from memory alone.

Watch

Extended reading notes

Core claim

On its own terms, the paper reports that AURA closes most of the gap between open-weight and closed-weight speech assistants on a spoken question-answering benchmark, while being the first open system of its kind with real tool use in a speech-to-speech loop. The architecture is a cascaded pipeline in which Whisper-v3-large or OWSM transcribes speech, LLaMA3.3-70B plans in a ReAct-style thought-action-observation cycle, a controller executes actions against external APIs, and a TTS module speaks the reply. The authors report 92.75% accuracy on OpenBookQA with Whisper, 4.39 on AlpacaEval, 28.76 joint goal accuracy on SpokenWOZ (a three-point gain over the prior best baseline), and 90% task success with satisfaction ratings above four on average in a 30-task human evaluation. They also report that requiring at least one web search before answering raises OpenBookQA accuracy from 87.69% to 89.23%. The discovery is that an open, modular, tool-augmented cascade can approach the performance of a much larger closed system on a spoken reasoning task.

Load-bearing premise

The comparison presumes that AURA's tool access does not give it an unfair advantage over the baselines; if web search retrieves the answers to OpenBookQA questions while the comparison systems are scored without tools, the headline accuracy reflects retrieval plus a cascaded pipeline rather than the agent's own reasoning.

Editorial extensions

If this is right

  • Open-weight components are enough to build a spoken agent that handles compound, multi-turn goals such as booking a calendar slot and then sending email.
  • New tools can be added by describing them in natural language and implementing an action class, without retraining any model.
  • Forcing the agent to perform at least one web search before answering improves multiple-choice accuracy from 87.69% to 89.23%, indicating that tool-grounded retrieval helps spoken QA.
  • The gap to closed systems like GPT-4o is small on OpenBookQA (92.75% vs 92.97%), suggesting the open-weights route is competitive on this benchmark.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The comparison may not be strictly fair: if the leaderboard baselines were run without tool access while AURA used web search, the accuracy gap measures tool-based retrieval rather than the agent's reasoning. A controlled run with tools disabled for AURA would settle this.
  • The human evaluation is small (30 tasks) and was rated by a co-author; independent raters and a larger task set would be needed to confirm the 90% success claim.
  • Because the design is modular, its performance should track the newest open LLMs; the specific numbers are less important than the finding that a cascaded agent-plus-tools architecture can rival closed systems on spoken reasoning tasks.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces AURA, an open-source cascaded speech-to-speech agent that combines a ReAct-style controller with real-world tools (calendar, email, contacts, web search) and a LLaMA-3.3-70B backbone. The main empirical claims are: (1) on VoiceBench OpenBookQA, AURA with Whisper-v3-large reaches 92.75%, outperforming all open-weight systems and nearing GPT-4o (92.97%), while the OWSM configuration reaches 89.23%; (2) on VoiceBench AlpacaEval, AURA scores 4.39, competitive with other open-weight systems; (3) a human evaluation on 30 self-designed tasks reports a 90% success rate; and (4) on SpokenWOZ, prompt-based DST achieves 28.76 JGA, surpassing the best reported baseline of 25.65. The paper also claims to be the first open-source speech-to-speech task-oriented agent with integrated reasoning and tool use.

Significance. If the benchmark comparisons were clean, AURA would be a useful open-source reference system for speech-based tool-augmented agents: it uses entirely open-weight components, has a modular design, and the code and demo are publicly linked. The architecture itself is straightforward and reproducible. However, the central benchmark claim is presently undermined by an unstated tool-use condition in Table I: the OWSM row exactly matches the forced-web-search accuracy reported in the ablation, and no no-tool Whisper result is reported, so the 'surpassing all open-weight systems' claim may only hold in a retrieval-augmented setting not shared by the baselines. The human evaluation is scored by a co-author on self-designed tasks, which further weakens the real-world claim. These issues are fixable with additional experiments, and the paper's positive contribution - a modular, open-source speech agent with tool use - remains of interest.

major comments (4)
  1. [Section III-A, Table I] The headline OpenBookQA comparison is confounded by tool use. The OWSM row in Table I reports 89.23%, which exactly matches the forced-web-search condition described in Section III-A, while the no-tool OWSM accuracy is stated as 87.69%. No no-tool Whisper-v3-large result is reported, so the 92.75% value may also include mandatory web search, whereas every baseline in Table I (GPT-4o-Audio, GPT-4o plus Whisper, Moshi, Mini-Omni2, Kimi-Audio) is a standard VoiceBench entry without external retrieval. Please (i) report the Whisper-v3-large result without web search, (ii) state explicitly for each row whether web search was enabled, and (iii) either compare all systems under identical tool-access conditions or reframe the 92.75% result as retrieval-augmented and soften the 'surpassing all open-weight systems' claim.
  2. [Section III-B] The reported 90% success rate rests on ratings by a co-author on 30 self-designed tasks. The paper does not describe the task specifications, the success criteria, or the scoring rubric beyond a 1-5 satisfaction scale, and no independent evaluation or inter-annotator agreement is reported. This is a serious validity threat to the claim of robust real-world task execution. Please provide a detailed evaluation protocol, use at least one independent rater blind to the system, and report per-task definitions of success and rating agreement.
  3. [Section III-A] The accuracy differences (87.69%, 89.23%, 92.75%) are reported without confidence intervals, the number of test items, or the number of evaluation runs. OpenBookQA contains 500 test questions, so a 1.5-point difference corresponds to roughly seven questions; without variance estimates it is unclear whether the improvements from web search and from the Whisper ASR are statistically significant. Please report the evaluation set size, run-to-run variability, and confidence intervals or significance tests.
  4. [Section I] The claim that AURA is the 'first open-source, speech-to-speech task-oriented agent' that combines reasoning and tool use is not adequately substantiated relative to prior work. The paper cites ReSpAct [18] as an inspiration, but ReSpAct's own title ('Harmonizing Reasoning, Speaking, and Acting') suggests a closely related spoken conversational agent; the paper does not explain the specific differences in spoken interaction, tool use, or open-source release. Please provide a systematic comparison with prior spoken agents that combine reasoning and acting, or soften the 'first' claim to avoid an unsupported novelty assertion.
minor comments (6)
  1. [Abstract and Section III-B] The abstract and introduction state 'over 30 compound tasks,' while Section III-B and Table II report exactly 30 tasks (6 easy, 12 medium, 12 hard); please align the numbers.
  2. [Section II-A] The ASR component is described as 'Whisper-v3-larger' but elsewhere and in Table I it is 'Whisper-v3-large'; please use consistent naming.
  3. [References] Reference [27] is titled 'GPT-4o: OpenAI's new multimodal flagship model' but the URL points to the GPT-4o-mini announcement, and references [21] and [27] contain placeholder access dates 'YYYY-MM-DD' that should be filled in.
  4. [Table I] The table formatting makes it difficult to distinguish the OpenBookQA and AlpacaEval blocks, and the 'C'/'O' and 'E2E' labels are not explained in the caption or text; please clarify the layout and define closed/open and end-to-end.
  5. [Section II-B] The web search action returns 'top-3 results per source,' but the paper does not specify which web search API or search engine is used; for reproducibility, please identify the search backend or provide a brief description in addition to the GitHub link.
  6. [Section III-C] The SpokenWOZ evaluation does not state which ASR is used or whether the transcriptions come from the benchmark's own speech recognition; please specify the input condition so the DST result can be reproduced.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: reported results are external benchmark measurements with disclosed ablations; self-citations are component reuse, not load-bearing arguments.

full rationale

AURA's central claims are empirical benchmark outcomes rather than deductions, so there is no derivation chain in which a conclusion is assumed in its premises. The OpenBookQA web-search ablation in Section III-A reports two measured configurations (87.69% without the forced-search prompt and 89.23% with it); the 89.23% OWSM row in Table I is the disclosed tool-using condition, not a fit disguised as a prediction, and the Whisper-based 92.75% is a separate measured result. The comparison of a retrieval-augmented agent to non-retrieval baselines is a benchmark-fairness concern, not circularity, because the tool use is the system under test rather than an input that presupposes the output. Citations to ESPnet-SDS [1] and OWSM [22] are reuse of the authors' own open-source components; neither is invoked as a theorem that forces the paper's conclusions, and both are available as code, so they do not constitute load-bearing self-citation. The SpokenWOZ DST evaluation uses an external benchmark with ground-truth states, and the human evaluation is a co-author-rated study, which is a validity and bias concern rather than a circular reasoning issue. No equation in the paper defines its output in terms of its input, and no fitted parameter is renamed as a prediction.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central claims rest on external benchmarks and on assumptions about the value of the cascaded/ReAct design; no new theoretical entities are introduced, but the paper assumes that the selected open components and evaluation settings reflect meaningful real-world spoken tool use.

free parameters (2)
  • web_search_result_limit = 3
    Section II-B.3: Web search returns top-3 results per source; this is a hand-chosen limit that determines what evidence the LLM sees.
  • forced_web_search_prompt = at least one Web search before answering
    Section III-A: Prompting at least one web search raises OpenBookQA accuracy from 87.69% to 89.23%; this prompt choice is an ad hoc tuning decision on the benchmark.
assumptions (5)
  • domain assumption Cascaded speech pipelines outperform end-to-end models on current voice benchmarks.
    Invoked in Section I to motivate the cascaded architecture, citing VoiceBench results.
  • domain assumption ReAct-style interleaving of reasoning and action improves multi-step task performance.
    Invoked in Section I to justify the agent loop, citing Yao et al. [15].
  • domain assumption LLaMA-3.3-Instruct-70B and Whisper-v3-large are representative open-weight components whose outputs transfer to spoken tool use.
    The paper selects these models without comparative ablations; the scores are conditional on this stack.
  • domain assumption VoiceBench QA tasks and spoken OpenBookQA measure tool-augmented agent capability.
    Used in Section III-A to claim competitive or superior performance; the benchmark protocol is assumed comparable across systems.
  • domain assumption A co-author's ratings on 30 self-designed tasks estimate real-world success.
    Used in Section III-B; no independent evaluation or pre-registered protocol is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AURA: Agent for Understanding, Reasoning, and Automated Tool Use in Voice-Driven Tasks." pith.science (2026). https://pith.science/paper/NAVCG2HS

@misc{pith2026250623049,
  author       = {Pith},
  title        = {Pith review of: AURA: Agent for Understanding, Reasoning, and Automated Tool Use in Voice-Driven Tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NAVCG2HS}},
  note         = {Machine review of arXiv:2506.23049}
}
read the original abstract

Despite advances in language and speech technologies, no open-source system enables full speech-to-speech, multi-turn dialogue with integrated tool use and agentic reasoning. We introduce AURA (Agent for Understanding, Reasoning, and Automated Tool Use), the first open-source, speech-native assistant capable of completing complex, goal-driven tasks through dynamic tool invocation and multi-turn conversation. AURA combines open-weight ASR, TTS, and LLMs in a cascaded pipeline and supports tools such as calendar booking, contact lookup, web search, and email. Its modular design allows easy integration of new tools using natural language prompts and action classes. On VoiceBench, AURA scores 92.75% on OpenBookQA-outperforming all open-weight systems and nearing GPT-4o-and 4.39 on AlpacaEval, competitive with other open-weight systems. Human evaluation shows 90% task success on complex, multi-turn speech tasks.

Figures

Figures reproduced from arXiv: 2506.23049 by the authors.

Figure 1
Figure 1. 1Code: https://github.com/Sentientia/Aura Demo: https://www.youtube.com/watch?v=cb7w0GVwwF0 arXiv:2506.23049v1 [cs.AI] 29 Jun 2025 [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 1
Figure 1. User Interface for AURA II. SYSTEM DESIGN AND METHOD We implement AURA using a cascaded architecture com￾prising four main components: (i) User Interface (UI), (ii) Dialog Processing Unit (DPU), (iii) LLM Server, and (iv) External APIs [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Software Architecture for AURA E. Security and Privacy Safeguards Following initial authentication (via Google login for access to email, calendar, and contacts), an access token is generated and stored locally, never uploaded to any external server or database. To prevent unintended communication, the system enforces a configurable whitelist, ensuring emails are only sent to approved recipients. III. EXPERIMENTS AN… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

34 extracted references · 12 canonical work pages

  1. [18]

    Available: https://arxiv.org/abs/2501.07246

    [Online]. Available: https://arxiv.org/abs/2501.07246

    arXiv

  2. [1]

    Espnet-sds: Unified toolkit and demo for spoken dialogue systems,

    S. Arora, Y . Peng, J. Shi, J. Tian, W. Chen, S. Bharadwaj, H. Futami, Y . Kashiwagi, E. Tsunoo, S. Shimizu, V . Srivastav, and S. Watanabe, “Espnet-sds: Unified toolkit and demo for spoken dialogue systems,”

    work page

  3. [2]

    Speechgpt: Empowering large language models with intrinsic cross-modal conversational abilities,

    D. Zhang, S. Li, X. Zhang, J. Zhan, P. Wang, Y . Zhou, and X. Qiu, “Speechgpt: Empowering large language models with intrinsic cross-modal conversational abilities,” 2023. [Online]. Available: https://arxiv.org/abs/2305.11000

    arXiv 2023

  4. [3]

    Baichuan- audio: A unified framework for end-to-end speech interaction,

    T. Li, J. Liu, T. Zhang, Y . Fang, D. Pan, M. Wang, Z. Liang, Z. Li, M. Lin, G. Dong, J. Xu, H. Sun, Z. Zhou, and W. Chen, “Baichuan- audio: A unified framework for end-to-end speech interaction,” 2025. [Online]. Available: https://arxiv.org/abs/2502.17239

    arXiv 2025

  5. [5]

    Mini-omni: Language models can hear, talk while thinking in streaming,

    Z. Xie and C. Wu, “Mini-omni: Language models can hear, talk while thinking in streaming,” 2024. [Online]. Available: https: //arxiv.org/abs/2408.16725

    arXiv 2024

  6. [6]

    Openomni: Advancing open-source omnimodal large language models with progressive multimodal alignment and real-time self-aware emotional speech synthesis,

    R. Luo, T.-E. Lin, H. Zhang, Y . Wu, X. Liu, M. Yang, Y . Li, L. Chen, J. Li, L. Zhang, Y . Chen, X. Xia, H. Alinejad-Rokny, and F. Huang, “Openomni: Advancing open-source omnimodal large language models with progressive multimodal alignment and real-time self-aware emotional speech synthesis,” 2025. [Online]. Available: https://arxiv.org/abs/2501.04561

    arXiv 2025

  7. [7]

    V oicebench: Benchmarking llm-based voice assistants,

    Y . Chen, X. Yue, C. Zhang, X. Gao, R. T. Tan, and H. Li, “V oicebench: Benchmarking llm-based voice assistants,” 2024. [Online]. Available: https://arxiv.org/abs/2410.17196

    arXiv 2024

  8. [8]

    Multiwoz–a large-scale multi-domain wizard- of-oz dataset for task-oriented dialogue modelling,

    P. Budzianowski, T.-H. Wen, B.-H. Tseng, I. Casanueva, S. Ultes, O. Ra- madan, and M. Ga ˇsi´c, “Multiwoz–a large-scale multi-domain wizard- of-oz dataset for task-oriented dialogue modelling,” arXiv preprint arXiv:1810.00278, 2018

    arXiv

Show all 34 references
  1. [9]

    Spokenwoz: A large-scale speech-text benchmark for spoken task-oriented dialogue agents,

    S. Si, W. Ma, H. Gao, Y . Wu, T.-E. Lin, Y . Dai, H. Li, R. Yan, F. Huang, and Y . Li, “Spokenwoz: A large-scale speech-text benchmark for spoken task-oriented dialogue agents,” 2024. [Online]. Available: https://arxiv.org/abs/2305.13040

    2024 arXiv

  2. [10]

    Training language models to follow instructions with human feedback,

    L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe, “Training language models to follow instructions...

    2022 arXiv

  3. [11]

    Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face,

    Y . Shen, K. Song, X. Tan, D. Li, W. Lu, and Y . Zhuang, “Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face,” 2023. [Online]. Available: https://arxiv.org/abs/2303.17580

    2023 arXiv

  4. [12]

    Api-bank: A comprehensive benchmark for tool-augmented llms,

    M. Li, Y . Zhao, B. Yu, F. Song, H. Li, H. Yu, Z. Li, F. Huang, and Y . Li, “Api-bank: A comprehensive benchmark for tool-augmented llms,” 2023. [Online]. Available: https://arxiv.org/abs/2304.08244

    2023 arXiv

  5. [13]

    Tooldial: Multi-turn dialogue generation method for tool-augmented language models,

    J. Shim, G. Seo, C. Lim, and Y . Jo, “Tooldial: Multi-turn dialogue generation method for tool-augmented language models,” 2025. [Online]. Available: https://arxiv.org/abs/2503.00564

    2025 arXiv

  6. [14]

    Rethinking task-oriented dialogue systems: From complex modularity to zero-shot autonomous agent,

    H.-D. Xu, X.-L. Mao, P. Yang, F. Sun, and H. Huang, “Rethinking task-oriented dialogue systems: From complex modularity to zero-shot autonomous agent,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , L.-W. Ku...

    2024

  7. [15]

    React: Synergizing reasoning and acting in language models,

    S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y . Cao, “React: Synergizing reasoning and acting in language models,” in International Conference on Learning Representations (ICLR) , 2023

    2023

  8. [16]

    Audio-cot: Exploring chain-of-thought reasoning in large audio language model,

    Z. Ma, Z. Chen, Y . Wang, E. S. Chng, and X. Chen, “Audio-cot: Exploring chain-of-thought reasoning in large audio language model,”

  9. [17]

    Audio-reasoner: Improving reasoning capability in large audio language models,

    Z. Xie, M. Lin, Z. Liu, P. Wu, S. Yan, and C. Miao, “Audio-reasoner: Improving reasoning capability in large audio language models,” 2025. [Online]. Available: https://arxiv.org/abs/2503.02318

    2025

  10. [19]

    Can a suit of armor conduct electricity? a new dataset for open book question answering,

    T. Mihaylov, P. Clark, T. Khot, and A. Sabharwal, “Can a suit of armor conduct electricity? a new dataset for open book question answering,” in EMNLP, 2018

    2018

  11. [20]

    Respact: Harmonizing reasoning, speaking, and acting towards building large language model-based conversational ai agents,

    V . Dongre, X. Yang, E. C. Acikgoz, S. Dey, G. Tur, and D. Hakkani- T¨ur, “Respact: Harmonizing reasoning, speaking, and acting towards building large language model-based conversational ai agents,” 2025. [Online]. Available: https://arxiv.org/abs/2411.00927

    2025 arXiv

  12. [21]

    Gpt-4o: Openai’s new multimodal flagship model,

    OpenAI, “Gpt-4o: Openai’s new multimodal flagship model,” https:// platform.openai.com/docs/models/gpt-4o, 2024, accessed: YYYY-MM- DD

    2024

  13. [22]

    Robust speech recognition via large-scale weak supervision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” 2022. [Online]. Available: https://arxiv.org/abs/2212.04356

    2022 arXiv

  14. [23]

    Espnet-TTS: Unified, reproducible, and integratable open source end-to-end text-to-speech toolkit,

    T. Hayashi, R. Yamamoto, K. Inoue, T. Yoshimura, S. Watanabe, T. Toda, K. Takeda, Y . Zhang, and X. Tan, “Espnet-TTS: Unified, reproducible, and integratable open source end-to-end text-to-speech toolkit,” in Proceedings of IEEE International Conference on Acoustics, Speech an...

    2020

  15. [24]

    Owsm v3.1: Better and faster open whisper-style speech models based on e-branchformer,

    Y . Peng, J. Tian, W. Chen, S. Arora, B. Yan, Y . Sudo, M. Shakeel, K. Choi, J. Shi, X. Chang, J. weon Jung, and S. Watanabe, “Owsm v3.1: Better and faster open whisper-style speech models based on e-branchformer,” 2024. [Online]. Available: https://arxiv.org/abs/2401. 16658

    2024

  16. [25]

    Efficient memory management for large language model serving with pagedattention,

    W. Kwon, Z. Li, S. Zhuang, Y . Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, and I. Stoica, “Efficient memory management for large language model serving with pagedattention,” 2023. [Online]. Available: https://arxiv.org/abs/2309.06180

    2023 arXiv

  17. [26]

    The llama 3 herd of models,

    A. G. et al., “The llama 3 herd of models,” 2024. [Online]. Available: https://arxiv.org/abs/2407.21783

    2024 arXiv

  18. [27]

    Gpt-4o: Openai’s new multimodal flagship model,

    OpenAI, “Gpt-4o: Openai’s new multimodal flagship model,” https: //openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/, 2024, accessed: YYYY-MM-DD

    2024

  19. [28]

    Alpacaeval: An automatic evaluator of instruction-following models,

    X. Li, T. Zhang, Y . Dubois, R. Taori, I. Gulrajani, C. Guestrin, P. Liang, and T. B. Hashimoto, “Alpacaeval: An automatic evaluator of instruction-following models,” https://github.com/tatsu-lab/alpaca eval, 5 2023

    2023

  20. [29]

    Moshi: a speech-text foundation model for real-time dialogue,

    A. D ´efossez, L. Mazar ´e, M. Orsini, A. Royer, P. P ´erez, H. J ´egou, E. Grave, and N. Zeghidour, “Moshi: a speech-text foundation model for real-time dialogue,” Tech. Rep., 2024. [Online]. Available: https://arxiv.org/abs/2410.00037

    2024 arXiv

  21. [30]

    Gpt-4o: Openai’s new multimodal flagship model,

    ——, “Gpt-4o: Openai’s new multimodal flagship model,” https://platform.openai.com/docs/models/gpt-4o-audio-preview, 2024, accessed: YYYY-MM-DD

    2024

  22. [31]

    Kimi-audio technical report,

    KimiTeam, D. Ding, Z. Ju, Y . Leng, S. Liu, T. Liu, Z. Shang, K. Shen, W. Song, X. Tan, H. Tang, Z. Wang, C. Wei, Y . Xin, X. Xu, J. Yu, Y . Zhang, X. Zhou, Y . Charles, J. Chen, Y . Chen, Y . Du, W. He, Z. Hu, G. Lai, Q. Li, Y . Liu, W. Sun, J. Wang, Y . Wang, Y . Wu, Y . Wu,...

    2025 arXiv

  23. [32]

    Mini-omni2: Towards open-source gpt-4o with vision, speech and duplex capabilities,

    Z. Xie and C. Wu, “Mini-omni2: Towards open-source gpt-4o with vision, speech and duplex capabilities,” 2024. [Online]. Available: https://arxiv.org/abs/2410.11190

    2024 arXiv

  24. [33]

    Parakeet-tdt-0.6b-v2,

    NVIDIA, “Parakeet-tdt-0.6b-v2,” https://huggingface.co/nvidia/ parakeet-tdt-0.6b-v2, 2024, accessed: 2025-06-24

    2024

  25. [34]

    Qwen3 technical report,

    Q. Team, “Qwen3 technical report,” 2025. [Online]. Available: https://arxiv.org/abs/2505.09388

    2025 arXiv

  26. [2025]

    Available: https://arxiv.org/abs/2503.08533

    [Online]. Available: https://arxiv.org/abs/2503.08533

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.