Spoken Function Calling: A New Perspective on Spoken Language Understanding for Large Audio Language Models

Changfeng Gao; Xiangang Li; Xie Chen; Yuezhang Peng; Yuxin Liu; Zhifu Gao

REVIEW 3 major objections 6 minor 51 references

Spoken Function Calling: A New Perspective on Spoken Language Understanding for Large Audio Language Models

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Spoken Function Calling—formulating user requests as executable API calls with explicit schemas—outperforms traditional intent-and-slot spoken language understanding for text and audio models, and a post-trained 7B model surpasses the…

desk verdict Useful benchmark and post-training recipe, but the core SFC-vs-SLU comparison likely inflated by an unfair SLU baseline. read the letter →

arxiv 2608.05126 v1 pith:AM2DHZG5 submitted 2026-08-05 cs.CL cs.MM

Yuezhang Peng , Yuxin Liu , Changfeng Gao , Zhifu Gao , Xiangang Li , Xie Chen This is my paper

classification cs.CLcs.MM

keywords SpokenLanguageUnderstandingFunctionCallingLargeAudioModelsIn-ContextLearningReinforcementSFC-BenchTask-OrientedDialogue

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes treating spoken language understanding (SLU)—parsing what a user asks a voice assistant to do—as a function-calling problem rather than a problem of classifying intents and filling slots. It constructs SFC-Bench, a synthetic benchmark of about 7,000 spoken queries paired with structured function calls, and reports that on the Level-1 subset the function-calling formulation beats the traditional intent-and-slot formulation by 5 to 18 points in overall accuracy for both text and audio models. The paper then post-trains a 7B audio-language model with group-relative policy optimization and a decomposed reward, reaching 74.15% overall accuracy on the in-distribution test set, above the roughly 68–69% of the strongest closed-source audio models it compares against. A sympathetic reader should care because, if the formulation is right, voice assistants can move away from rigid closed-set schemas toward open-domain tool use driven by in-context learning.

What carries the argument

The load-bearing object is the spoken function definition: a named tool with a fixed JSON-like schema (for example, add_schedule(content, time, location)) that serves as the output contract. The paper follows the OpenFunction protocol, a schema format for executable tool definitions, and supplements the input with an explicit environment context to resolve speech omissions, assigning NAN to missing parameters so downstream systems can trigger completion. In addition, the RL recipe decomposes the exact-match reward into three dense sub-rewards—function name match, parameter-key Jaccard similarity, and parameter-value normalized edit distance—to provide learning signal early in training. The combination of schema as prior, environment context as deictic anchor, and decomposed reward is what carries the claimed advantage.

What would settle it

Re-run the Level-1 comparison of Table 3 using SLU ground truth produced independently from native SLU annotations (for example, the original labels of SLURP or MAC-SLU) while keeping the same utterances, functions, and models; if the SFC advantage over SLU falls below 5 points or reverses on those labels, the claimed advantage is an artifact of constructing SLU labels by converting function calls, whereas if the advantage persists, the SFC formulation itself carries the gain.

Watch

Extended reading notes

Core claim

The paper's central claim is that Spoken Function Calling (SFC)—mapping a spoken query, an environmental context, and a set of function definitions to one or more executable API calls—is a better semantic-understanding target for LLMs and large audio-language models than the traditional intent-and-slot frame. The authors argue that fixed function schemas jointly constrain the function name and each parameter value, removing the ambiguous slot-name boundaries that hurt SLU when, for instance, the same time value could be labeled 'time' or 'start_time'. On the Level-1 subset of their SFC-Bench, SFC improves overall accuracy by 5 to 18 points over traditional SLU across text and audio models under both in-context learning and supervised fine-tuning, with the largest gains in parameter-value extraction. They further claim that RL post-training with a fine-grained reward—separate rewards for function-name match, parameter-key Jaccard similarity, and parameter-value edit distance—lets a 7B model surpass the strongest closed-source audio models on SFC-Bench while retaining ASR and audio-reasoning performance.

Load-bearing premise

The load-bearing premise is that the intent-and-slot SLU baseline is a fair operationalization of traditional SLU; the comparison is only fair if converting the SFC ground truth back into SLU frames did not create artificial slot-name ambiguities (such as 'time' versus 'start_time') that a real SLU dataset would not present.

Editorial extensions

If this is right

If SFC is the better target, voice-assistant training data can be authored as executable tool calls with schemas, which is directly consumable by downstream APIs and tool orchestration systems.
Open-domain semantics no longer requires per-domain supervised fine-tuning: models can interpret a fresh function schema at inference time through in-context learning, which is what the paper's out-of-distribution split tests.
The NAN convention makes missing information explicit, allowing systems to detect hallucinated values (such as inventing a concrete date for 'this Friday') and route to clarification instead of executing an incorrect action.
RL post-training with fine-grained rewards transfers beyond the benchmark, improving text-based function calling, ASR, and audio reasoning relative to the base model, whereas supervised fine-tuning alone did not.
High-difficulty multi-intent and multi-turn SFC remains unsolved: even the best audited closed models drop to about 30% overall accuracy at Level 3-2, so this benchmark can track progress on long-horizon semantic completion.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If this result transfers to naturally collected human speech, product teams may stop maintaining intent ontologies and instead maintain tool schemas, because schema-bound extraction appears to give higher accuracy per annotation effort.
A testable extension would be to build an SFC-style benchmark directly from existing SLU datasets' native annotations rather than by converting SFC labels back into SLU frames, which would isolate whether the 5–18 point gain comes from the formulation itself or from label construction.
The fine-grained reward decomposition is a general recipe for any structured-output RL task, not only speech, and could be evaluated on text-only structured-output benchmarks as a control.
SFC-Bench uses synthetic TTS with strict speaker isolation, so real speech with disfluencies, code-switching, and background noise may compress the reported margins; that gap is not measured in the paper.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this paper has genuinely useful new artifacts—SFC-Bench, a synthesized spoken function-calling dataset with 300 tools and speaker-isolated splits, plus a fine-grained reward for RL post-training. The 5–18 point SFC-vs-SLU gain in Table 3, though, is less convincing than it looks, because the SLU baseline may have been set up to lose.

What is new: applying the function-calling paradigm to end-to-end SLU, building a multi-agent synthetic data pipeline, and showing a 7B LALM post-trained with GRPO and a decomposed reward outperforms GPT-4o-Audio and Gemini-2.5-Pro on their benchmark. The ablations on reward type, rollout count, and precision are welcome; the speaker isolation between train/test is a real methodological plus.

Where it gets soft. The stress-test note lands. In Section 5.1, the SLU baseline is derived from the same SFC annotations, and Table 2 shows the model failing on 'time' vs 'start_time'—a direct consequence of offering a flat slot pool without per-intent constraints. Real SLU datasets (ATIS, SNIPS, SLURP, MTOP) define slot schemas per intent, so a model can know that add_schedule expects time, content, location. By withholding that structure from the SLU prompt, the comparison may be testing prompt format rather than semantic extraction ability. The paper's own claim—that 'rules for intent classification and slot filling are often decoupled'—is true of many old datasets, but it is not a fair representation of current SLU practice. If the authors fixed the baseline to use per-intent slot lists, the gap would very likely shrink. I don't think the core claim collapses—structured schemas are genuinely helpful—but the magnitude is overstated.

Two smaller things. The OOD set is admitted to be less complex than the ID set, which undercuts the generality claim. And the headline SpokenFC-7B result compares a fine-tuned model against zero-shot closed models; that's a useful data point but not an apples-to-apples comparison.

Who this is for: people building spoken task-oriented agents and LALM post-training pipelines. The dataset and reward will be reused. It deserves serious peer review—the bias concern is addressable and the contributions are real. I'd ask for a revised baseline and a re-analysis of Table 3 before accepting.

Referee Report

3 major / 6 minor

Summary. This paper proposes Spoken Function Calling (SFC), a reformulation of SLU in which a spoken user query is mapped to executable function calls with explicit schemas rather than to intent-and-slot frames. The authors construct SFC-Bench, a synthetic benchmark of 300 spoken functions derived from existing SLU datasets, with single-intent, multi-intent, multi-turn, and OOD splits. They compare SFC with a traditional SLU formulation on Level 1 of the benchmark across Qwen3-8B, Qwen2.5-7B, and Qwen2.5-Omni-7B under ICL and SFT, reporting 5–18 point overall accuracy gains for SFC. They then benchmark LLMs, ASR+LLM pipelines, and LALMs on SFC-Bench, and post-train Qwen2.5-Omni-7B with GRPO and a fine-grained reward, obtaining SpokenFC-7B with 74.15% ID overall accuracy, which they report as surpassing GPT-4o-Audio and Gemini-2.5-Pro.

Significance. If the results are reproducible, the paper makes a useful contribution: the SFC format couples intent and slot prediction through per-function parameter lists, which is a natural fit for tool-augmented agents and for ICL in open-domain settings. The dataset construction addresses a real gap (speech-centric function calling benchmarks), and the post-training study with reward decomposition and rollout scaling is informative. The paper includes several strengths: it ships code and data, uses speaker-isolated splits, provides an OOD split, and ablates the main design choices. However, the central empirical claim depends on a comparison whose SLU baseline is not fully specified and whose numbers are not internally consistent with the benchmark table; both issues must be resolved before the contribution can be assessed.

major comments (3)

[§5.1.1 vs §5.2.1, Tables 3 and 5] The Level 1 SFC results for the same models disagree between Table 3 and Table 5. For Qwen3-8B, Table 3 reports Slot 91.83 / Overall 85.70 for ICL SFC, while Table 5 reports Slot 92.99 / Overall 87.69. For Qwen2.5-Omni-7B, Table 3 reports Intent 93.02 / Slot 79.69 / Overall 63.86 for speech input, while Table 5 reports 93.46 / 81.67 / 65.19 for the LALM row. Since Table 3 is the evidence for the paper's central 'SFC outperforms SLU' claim, these discrepancies must be reconciled; if the two tables use different test subsets or scoring functions, this must be stated explicitly.
[§5.1, Figure 1, Table 2] The construction of the SLU baseline is not described. Figure 1 and Table 2 suggest that the SLU condition uses a flat, global pool of slot names (including both 'time' and 'start_time') without per-intent slot constraints, and that SLU ground truth is obtained by converting SFC function calls into intent-slot frames. This asymmetrically disadvantages SLU, because SFC's function definitions provide exactly the per-intent parameter constraints that the SLU prompt lacks. The authors should report the exact SLU prompt and annotation procedure and rerun the comparison with per-intent slot schemas or on established SLU benchmarks (e.g., ATIS, SNIPS, SLURP) to support the claimed 5–18 point advantage.
[§5.3, Table 7] The comparison between SpokenFC-7B and the closed-source models is not controlled: SpokenFC-7B was post-trained on the SFC-Bench training split (the ID functions), whereas GPT-4o-Audio and Gemini-2.5-Pro are evaluated zero-shot. The paper should state this asymmetry in the text and, ideally, include a zero-shot or few-shot evaluation of the base Qwen2.5-Omni-7B on the same test sets for a like-for-like comparison; as written, 'surpasses GPT-4o-Audio' conflates the effect of post-training with the effect of having seen the task format.

minor comments (6)

[§4.2] The prompts used for the Query, Label, Speech, and Validator agents are not included; providing them is necessary for reproducibility.
[Table 5] The rows labeled 'ASR+LLM Qwen2.5-Omni' are confusing: Qwen2.5-Omni is an LALM, and it is unclear whether this row uses it as an ASR front-end or as an LLM decoder.
[§3.2, Eq. (12)] Specify the behavior when the ground-truth key set K is empty; the current notation leaves the division by |K| undefined.
[Figure 1] Several rendered strings are corrupted (e.g., 'Ame', 'locaAon', 'denAst'); please fix the figure generation.
[References] Reference [24] is cited for BFCL but points to the Gorilla paper; the bibliography entry should be corrected.
[§3.1] The 'OpenFunction protocol' is introduced but never defined or referenced; either provide a definition or remove the term.

Circularity Check

2 steps flagged · score 6.0 of 10

The SFC-vs-SLU comparison is partially built into the benchmark: SLU labels are the SFC calls re-rendered as intent-slot frames over a global slot pool, so the reported 5-18 point advantage is partly an artifact of format construction.

self definitional [Section 5.1 / Table 2 / Figure 1 (SLU Context)]
"SLU Prediction: "intent": "add_schedule", "slots": { "content": "dentist appointment", "start_time": "2024-07-20 15:00:00", "location": "Paris" } ... Ground Truth: "intent": "add_schedule", "slots": { "content": "dentist appointment", "time": "2024-07-20 15:00:00", "location": "Paris" }"

The SLU ground truth shown is simply the SFC function call add_schedule(content=..., time=..., location=...) relabeled as intent+slots. The SLU prompt supplies a single global slot pool containing both 'time' and 'start_time', while the SFC schema for add_schedule exposes only the parameter 'time'. The model's 'start_time' error is therefore manufactured by the authors' choice of SLU prompt format; SFC's success follows from the per-function parameter list rather than from superior semantic extraction. The claimed 5-18 point gain is partly an artifact of this asymmetric formatting.
other [Sections 4.1-4.2 and 5.1]
"Firstly, we aggregated a comprehensive set of scenarios from widely utilized SLU benchmarks. Secondly, we leveraged Gemini-2.5 Pro to transform these scenarios into standardized function definitions. ... We initially conducted comparative experiments using the Level 1 subset of SFC-Bench. These experiments involved both ICL and SFT ... compare the performance of models when processing identical semantic datasets under traditional SLU and our proposed SFC perspective."

The benchmark's native labels are function calls generated by the Label Agent from the SFC function pool; traditional SLU is then evaluated on the same Level-1 subset, with its intent-slot frames recoverable only by projecting the SFC calls. Thus the 'identical semantic datasets' are identical only in the surface queries and environment, not in the annotation ontology. The comparison is between SFC's native representation and a derived SLU representation, so the perspective being promoted defines the evaluation format. This makes the headline 'SFC outperforms SLU' partially true by construction, though the use of real SLU scenarios as the source and external closed models as zero-shot evaluators provides some independent grounding.

full rationale

The paper's central comparative claim, that SFC outperforms traditional SLU by 5-18 points (Section 5.1.1, Table 3), is evaluated on SFC-Bench, a dataset the authors constructed with function-call labels as the native format. The SLU baseline is tested on the same subset, but its intent-slot ground truth is visibly the SFC call converted into intent and slot fields (Table 2), and the SLU prompt offers a global pool of slot names including overlapping candidates such as 'time' and 'start_time'. This gives SFC a structural advantage by construction: each SFC function carries an explicit parameter list, while the SLU prompt does not constrain which slots apply to which intent. The specific failure shown in Table 2, where the model predicts 'start_time' instead of 'time', is a direct consequence of that prompt design rather than an independent measure of semantic understanding. However, this is partial circularity, not total circularity. The SFC-Bench functions are derived from established SLU datasets, the post-trained SpokenFC-7B is also checked on external benchmarks (CommonVoice, MMSU, API-Bank), and closed-source models are evaluated zero-shot, so the post-training and general-capability claims have independent content. The self-citation to MAC-SLU [25] is not load-bearing for the main argument. The score is 6 because the headline SFC-versus-SLU advantage reduces substantially by construction, while the benchmark study and post-training results remain externally grounded.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No constants are fit to data; the empirical claims depend on the synthetic data pipeline, the TTS proxy for real speech, and the hand-designed reward decomposition. The main burden is that the benchmark is constructed in-house, so the comparative claims rely on the representativeness of that construction. No new physical or conceptual entities are introduced; SFC is a task formulation, not an entity.

free parameters (3)

Fine-grained reward weights = 1/3, 1/3, 1/3
Equal weights for function-name, key-matching, and value-accuracy rewards in Eq. 9; chosen by hand, not learned, and affect post-training behavior.
Number of RL rollouts N = 32
Selected for the final SpokenFC-7B after ablating N=8,16,32,64; affects performance and is a training choice, not fitted to an external dataset.
ID/OOD function split = 232 in-distribution, 68 out-of-distribution
The split into ID and OOD functions is a design choice; the authors acknowledge the OOD set is slightly lower in complexity, which is a potential confound for the generalization claim.

assumptions (4)

domain assumption Synthetic multi-agent generation produces valid and representative task-oriented spoken queries and labels.
The dataset is generated by GPT-OSS-120B agents with no human validation on the training set; quality of synthetic data is assumed. Section 4.2.
domain assumption TTS speech from IndexTTS-2 with LibriSpeech speaker profiles is a sufficient proxy for real user speech in evaluating LALMs.
All speech inputs are synthesized; real-speech robustness is inferred from this proxy. Section 4.2.
ad hoc to paper Fine-grained reward decomposition with equal weights and normalized edit distance is a valid training signal for SFC.
The reward in Eqs. 9-12 is introduced for this task; its guidance toward true task success is not independently verified.
domain assumption Zero-shot closed-source models provide a fair comparison baseline for a fine-tuned open model.
SpokenFC-7B is post-trained on the benchmark's training split while GPT-4o-Audio and Gemini are evaluated zero-shot; the paper implies this comparison is informative. Section 5.3.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Spoken Function Calling: A New Perspective on Spoken Language Understanding for Large Audio Language Models." pith.science (2026). https://pith.science/paper/AM2DHZG5

@misc{pith2026260805126,
  author       = {Pith},
  title        = {Pith review of: Spoken Function Calling: A New Perspective on Spoken Language Understanding for Large Audio Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AM2DHZG5}},
  note         = {Machine review of arXiv:2608.05126}
}

read the original abstract

Spoken Language Understanding (SLU) is the core component of task-oriented dialogue systems and a pivotal link in achieving seamless human-agent interaction. While traditional SLU can effectively extract user semantics for closed-set tasks after in-domain supervised fine-tuning, it faces significant challenges in leveraging in-context learning for open-domain tasks due to its ambiguous rule definitions. This work proposes Spoken Function Calling (SFC), a novel semantic understanding perspective that optimizes semantic understanding with structured rule definitions, to evolve beyond traditional closed-set SLU. Specifically, we curate and extend a suite of spoken functions based on traditional SLU datasets, construct a multi-agent system to synthesize the SFC-Bench dataset, evaluate the performance of Large Language Models (LLMs) and Large Audio Language Models (LALMs), and enhance the SFC capabilities of LALMs through post-training. Experiments demonstrate that SFC outperforms traditional SLU, substantially enhancing the semantic extraction accuracy for LLMs and LALMs.

Figures

Figures reproduced from arXiv: 2608.05126 by the authors.

**Figure 2.** Example of Multi-Intent and Multi-Turn Spoken Function Calling tasks. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

**Figure 3.** Multi-Agent System for Data Construction. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: The domain distribution (left) and detailed statistics (right) of average question lengths (tokens), answer lengths [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

**Figure 5.** Figure 5: Ablation study on reward types. 8 16 32 64 Number of Rollouts (N) 60 70 80 Accuracy (%) SFC ID SFC OOD API-Bank MMSU CV15-EN 8.2 8.3 8.4 8.5 ASR WER (%) [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

51 extracted references · 28 canonical work pages

[1]

Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, et al. 2025. gpt-oss-120b & gpt-oss-20b model card.arXiv preprint arXiv:2508.10925(2025)

arXiv 2025
[2]

Rosana Ardila, Megan Branson, Kelly Davis, Michael Kohler, Josh Meyer, Michael Henretty, Reuben Morais, Lindsay Saunders, Francis Tyers, and Gregor Weber

work page
[3]

Emanuele Bastianelli, Andrea Vanzo, Pawel Swietojanski, and Verena Rieser. 2020. SLURP: A spoken language understanding resource package.In Proc. EMNLP (2020), 7252–7262

work page 2020
[4]

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners.In Proc. NeurIPS33 (2020), 1877–1901

work page 2020
[5]

Pawel Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gasic. 2018. MultiWOZ - A Large- Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling. InProc. EMNLP. Association for Computational Linguistics, 5016–5026

work page 2018
[6]

Mark Chen. 2021. Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374(2021)

arXiv 2021
[7]

Alice Coucke, Alaa Saade, Adrien Ball, Théodore Bluche, Alexandre Caulier, David Leroy, Clément Doumouro, Thibault Gisselbrecht, Francesco Caltagirone, Thibaut Lavril, et al. 2018. Snips Voice Platform: an embedded spoken language understanding system for private-by-design voice interfaces.arXiv preprint arXiv:1805.10190(2018)

arXiv 2018
[8]

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models.arXiv e-prints(2024), arXiv–2407

2024

Show all 51 references

[9]

Mutian He and Philip N Garner. 2023. Can ChatGPT detect intent? evaluating large language models for spoken language understanding.In Proc. InterSpeech (2023), 1109–1113

2023
[10]

Charles T Hemphill, John J Godfrey, and George R Doddington. 1990. The ATIS spoken language systems pilot corpus. InSpeech and Natural Language: Proceedings of a Workshop Held at Hidden Valley, Pennsylvania, June 24-27, 1990

1990
[11]

Xinyi Hou, Yanjie Zhao, Shenao Wang, and Haoyu Wang. 2025. Model Context Protocol (MCP): Landscape, security threats, and future research directions.arXiv preprint arXiv:2503.23278(2025)

2025 arXiv
[12]

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. 2022. Lora: Low-rank adaptation of large language models.In Proc. ICLR1, 2 (2022), 3

2022
[13]

Ziwei Ji, Tiezheng Yu, Yan Xu, Nayeon Lee, Etsuko Ishii, and Pascale Fung
[14]

Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models.arXiv preprint arXiv:2001.08361(2020)

2020 arXiv
[15]

Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proc. SOSP. 611–626

2023
[16]

Haoran Li, Abhinav Arora, Shuohui Chen, Anchit Gupta, Sonal Gupta, and Yashar Mehdad. 2021. MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmark. InProc. EACL. Association for Computational Linguistics, 2950–2962

2021
[17]

Mohan Li, Cong-Thanh Do, Simon Keizer, Youmna Farag, Svetlana Stoyanchev, and Rama Doddipatla. 2024. WHISMA: A Speech-LLM to perform zero-shot spoken language understanding. InProc. SLT. IEEE, 1115–1122

2024
[18]

Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. 2023. API-Bank: A comprehensive benchmark for tool-augmented LLMs.arXiv preprint arXiv:2304.08244(2023)

2023 arXiv
[19]

Weiwen Liu, Xu Huang, Xingshan Zeng, Xinlong Hao, Shuai Yu, Dexun Li, Shuai Wang, Weinan Gan, Zhengying Liu, Yuanqing Yu, et al. 2024. ToolACE: Winning the points of llm function calling.arXiv preprint arXiv:2409.00920(2024)

2024 arXiv
[20]

Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. 2024. MathVista: Evaluating mathematical reasoning of foundation models in visual contexts.In Proc. ICLR(2024)

2024
[21]

Loren Lugosch, Mirco Ravanelli, Patrick Ignoto, Vikrant Singh Tomar, and Yoshua Bengio. 2019. Speech model pre-training for end-to-end spoken language under- standing.In Proc. InterSpeech(2019), 7458–7462

2019
[22]

Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. 2015. Librispeech: an asr corpus based on public domain audio books. InProc. ICASSP. IEEE, 5206–5210

2015
[23]

Bhargavi Paranjape, Scott Lundberg, Sameer Singh, Hannaneh Hajishirzi, Luke Zettlemoyer, and Marco Tulio Ribeiro. 2023. Art: Automatic multi-step reasoning and tool-use for large language models.arXiv preprint arXiv:2303.09014(2023)

2023 arXiv
[24]

Shishir G Patil, Tianjun Zhang, Xin Wang, and Joseph E Gonzalez. 2024. Gorilla: Large language model connected with massive apis.In Proc. NeurIPS37 (2024), 126544–126565

2024
[25]

Yuezhang Peng, Chonghao Cai, Ziang Liu, Shuai Fan, Sheng Jiang, Hua Xu, Yuxin Liu, Qiguang Chen, Kele Xu, Yao Li, et al. 2025. MAC-SLU: Multi-Intent Automotive Cabin Spoken Language Understanding Benchmark.arXiv preprint arXiv:2512.01603(2025)

2025
[26]

Penghui Qi, Zichen Liu, Xiangxin Zhou, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. 2025. Defeating the training-inference mismatch via fp16.arXiv preprint arXiv:2510.26788(2025)

2025
[27]

Cheng Qian, Emre Can Acikgoz, Qi He, Hongru Wang, Xiusi Chen, Dilek Hakkani- Tür, Gokhan Tur, and Heng Ji. 2025. ToolRL: Reward is all tool learning needs.In Proc. NeurIPS(2025)

2025
[28]

Libo Qin, Tailu Liu, Wanxiang Che, Bingbing Kang, Sendong Zhao, and Ting Liu
[29]

Libo Qin, Xiao Xu, Wanxiang Che, and Ting Liu. 2020. AGIF: An adaptive graph- interactive framework for joint multiple intent detection and slot filling.arXiv preprint arXiv:2004.10087(2020)

2020 arXiv
[30]

Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. 2023. ToolLLM: Facilitating large language models to master 16000+ real-world apis.arXiv preprint arXiv:2307.16789(2023)

2023 arXiv
[31]

Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. InProc. ICML. PMLR, 28492–28518

2023
[32]

Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools.In Proc. NeurIPS 36 (2023), 68539–68551

2023
[33]

John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov
[34]

Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. 2024. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300(2024)

2024 arXiv
[35]

2011.Spoken language understanding: Systems for extracting semantic information from speech

Gokhan Tur and Renato De Mori. 2011.Spoken language understanding: Systems for extracting semantic information from speech. John Wiley & Sons

2011
[36]

Dingdong Wang, Jincenzi Wu, Junan Li, Dongchao Yang, Xueyuan Chen, Tianhua Zhang, and Helen Meng. 2025. MMSU: A Massive Multi-task Spoken Language Understanding and Reasoning Benchmark.arXiv preprint arXiv:2506.04779(2025)

2025 arXiv
[37]

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models.In Proc. NeurIPS35 (2022), 24824–24837

2022
[38]

Williams, Antoine Raux, and Matthew Henderson

J. Williams, Antoine Raux, and Matthew Henderson. 2016. The Dialog State Tracking Challenge Series: A Review.Dialogue Discourse7 (2016), 4–33

2016
[39]

Williams, Antoine Raux, Deepak Ramachandran, and Alan W

J. Williams, Antoine Raux, Deepak Ramachandran, and Alan W. Black. 2013. The Dialog State Tracking Challenge. InProc. SIGDIAL

2013
[40]

Boyong Wu, Chao Yan, Chen Hu, Cheng Yi, Chengli Feng, Fei Tian, Feiyu Shen, Gang Yu, Haoyang Zhang, Jingbei Li, et al. 2025. Step-Audio 2 Technical Report. arXiv preprint arXiv:2507.16632(2025)

2025 arXiv
[41]

Jin Xu, Zhifang Guo, Hangrui Hu, Yunfei Chu, Xiong Wang, Jinzheng He, Yuxuan Wang, Xian Shi, Ting He, Xinfa Zhu, et al. 2025. Qwen3-omni technical report. arXiv preprint arXiv:2509.17765(2025)

2025 arXiv
[42]

An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025. Qwen3 technical report.arXiv preprint arXiv:2505.09388(2025)

2025 arXiv
[43]

Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2022. ReAct: Synergizing reasoning and acting in language models. InProc. ICLR

2022
[44]

Yuze Zhao, Jintao Huang, Jinghan Hu, Xingjun Wang, Yunlin Mao, Daoze Zhang, Zeyinzi Jiang, Zhikai Wu, Baole Ai, Ang Wang, et al. 2025. SWIFT: a scalable lightweight infrastructure for fine-tuning. InProc. AAAI, Vol. 39. 29733–29735

2025
[45]

Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. Llamafactory: Unified efficient fine-tuning of 100+ language models.arXiv preprint arXiv:2403.13372(2024)

2024 arXiv
[46]

Siyi Zhou, Yiquan Zhou, Yi He, Xun Zhou, Jinchao Wang, Wei Deng, and Jingchen Shu. 2025. IndexTTS2: A Breakthrough in Emotionally Expressive and Duration-Controlled Auto-Regressive Zero-Shot Text-to-Speech.arXiv preprint arXiv:2506.21619(2025)

2025 arXiv
[47]

Yuchen Zhuang, Yue Yu, Kuan Wang, Haotian Sun, and Chao Zhang. 2023. ToolQA: A dataset for llm question answering with external tools.In Proc. NeurIPS36 (2023), 50117–50143

2023
[2017]

Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347 (2017)

2017 arXiv
[2020]

Common Voice: A massively-multilingual speech corpus. InProc. LREC. 4218–4222
[2021]

A co-interactive transformer for joint slot filling and intent detection. In Proc. ICASSP. IEEE, 8193–8197
[2023]

Towards mitigating LLM hallucination via self reflection. InProc. EMNLP. 1827–1843

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, et al. 2025. gpt-oss-120b & gpt-oss-20b model card.arXiv preprint arXiv:2508.10925(2025)

arXiv 2025

[2] [2]

Rosana Ardila, Megan Branson, Kelly Davis, Michael Kohler, Josh Meyer, Michael Henretty, Reuben Morais, Lindsay Saunders, Francis Tyers, and Gregor Weber

work page

[3] [3]

Emanuele Bastianelli, Andrea Vanzo, Pawel Swietojanski, and Verena Rieser. 2020. SLURP: A spoken language understanding resource package.In Proc. EMNLP (2020), 7252–7262

work page 2020

[4] [4]

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners.In Proc. NeurIPS33 (2020), 1877–1901

work page 2020

[5] [5]

Pawel Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gasic. 2018. MultiWOZ - A Large- Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling. InProc. EMNLP. Association for Computational Linguistics, 5016–5026

work page 2018

[6] [6]

Mark Chen. 2021. Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374(2021)

arXiv 2021

[7] [7]

Alice Coucke, Alaa Saade, Adrien Ball, Théodore Bluche, Alexandre Caulier, David Leroy, Clément Doumouro, Thibault Gisselbrecht, Francesco Caltagirone, Thibaut Lavril, et al. 2018. Snips Voice Platform: an embedded spoken language understanding system for private-by-design voice interfaces.arXiv preprint arXiv:1805.10190(2018)

arXiv 2018

[8] [8]

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models.arXiv e-prints(2024), arXiv–2407

2024