Pith. sign in

REVIEW 4 major objections 3 minor 3 cited by

BLAB: Brutally Long Audio Bench

T0 review · 4 major / 3 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read BLAB, an 833-hour benchmark of 15-minute-to-2-hour audio, finds that current audio language models, including Gemini 2.0 Pro and GPT-4o, score near floor on localization, counting, duration, and emotion tasks, with accuracy falling as…

desk verdict BLAB is a genuinely useful first hour-scale audio benchmark, but its headline claims overstate the evaluation scope and one of its key metrics is partly an output-capacity artifact. read the letter →

arxiv 2505.03054 v2 pith:TGFHWS6X submitted 2025-05-05 cs.AI cs.CLcs.SDeess.AS

Orevaoghene Ahia , Martijn Bartelds , Kabir Ahuja , Hila Gonen , Valentin Hofmann , Siddhant Arora , Shuyue Stella Li , Vishal Puttagunta
show 8 more authors
Mofetoluwa Adeyemi Charishma Buchireddy Ben Walls Noah Bennett Shinji Watanabe Noah A. Smith Yulia Tsvetkov Sachin Kumar
This is my paper · ORCID
classification cs.AIcs.CLcs.SDeess.AS
keywords BLABlong-formaudiounderstandinglanguagemodelsbenchmarklocalizationdurationestimationspeakercountingemotionreasoning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

BLAB is a benchmark built to ask whether audio language models understand speech the way people encounter it: in long, messy, hour-long recordings rather than 30-second snippets. The paper's central claim is that they do not. Across eight tasks on 833+ hours of permissively licensed audio averaging 51 minutes, every evaluated model, including Gemini 2.0 Pro and GPT-4o, performs at or near floor level when asked to localize words, advertisements, or named entities, count speakers, estimate event or total durations, and rank emotional valence and arousal. The same models do much better on 30-second excerpts, so the paper attributes the failure primarily to audio length and to a reliance on the text prompt rather than on the acoustic signal.

What carries the argument

The load-bearing instrument is BLAB itself: 833+ hours of Creative Commons YouTube audio split into eight tasks across four categories, localization, counting, emotion, and duration, with roughly 200 question-audio pairs per task and an average audio length of 51 minutes. Annotation pipelines combine WhisperX forced alignment with human checking for word and advertisement localization, Claude-3-extracted entity spans mapped back to WhisperX timestamps for named-entity localization, and human-AI collaboration for emotion ranking. Evaluation uses word-level or frame-level F1 for localization and exact-match accuracy for counting, duration, and emotion, plus a 30-second BLAB-MINI subset that isolates the effect of audio length and noise-placement experiments that test whether models actually listen to the signal.

What would settle it

Take a random subset of BLAB's named-entity localization items, have independent human annotators mark every entity span directly in the audio, and recompute frame-level F1 against both the human labels and the Claude-3/WhisperX labels; if models score substantially higher against human labels, or if the machine labels contain many spans humans cannot hear, then the reported NE-localization difficulty is partly an artifact of label generation rather than model limitation.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that state-of-the-art audio language models have no reliable long-form audio understanding. On BLAB's localization tasks, the best reported word-level F1 is below 2%, and frame-level F1 for advertisement or named-entity localization stays below 5%; exact-match accuracy is below 9% for speaker number estimation, below 5% for event duration, and below 4% for whole-file duration. The models can sometimes detect that an event exists, with Gemini finding about 27% of ground-truth named entities, but they cannot place those events in time. On emotion reasoning, the best exact match is 64.29%, and valence ranking is near chance. The paper also shows that performance collapses with duration: truncating the same tasks to 30 seconds raises Gemini's word-localization F1 from about 1 to about 30 and its entity-localization F1 from roughly 2 to nearly 50, while replacing long audio with silence or Gaussian noise does not make models reliably answer zero.

Load-bearing premise

The benchmark's named-entity localization ground truth is the weakest link because Claude-3, not a human, decided what counts as an entity, and if its extracted spans are incomplete or misaligned with WhisperX timestamps, models that actually detect those entities in the audio will be scored as wrong, making BLAB look harder than it is.

Editorial extensions

If this is right

  • A model that performs well on BLAB would, by construction, handle hour-scale audio localization, speaker counting, duration estimation, and emotion reasoning, so BLAB sets a concrete target for long-context audio training beyond short-clip benchmarks.
  • BLAB-MINI results imply that 30-second or 5-minute audio evaluations substantially overstate real-world capability, since Gemini's word-localization F1 drops from about 30 to about 1 when audio grows from 30 seconds to 30-plus minutes.
  • The noise-replacement and clip-placement analyses imply that long audio LMs can answer from the prompt alone and lose track of information in the middle of the context, so input ordering and prompt design matter as much as the acoustic signal.
  • Zero-shot chain-of-thought prompting raises event-duration accuracy only from about 4% to about 7% and speaker-count accuracy from about 8% to about 11%, so the identified bottleneck is auditory perception rather than high-level reasoning.
  • Most open-weight audio LMs cannot even ingest the long-form inputs BLAB requires, which means the open ecosystem is currently unevaluable on the very capability the benchmark targets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not publish a no-audio prompt-only baseline, so its finding that models lean on prompts implies a testable extension: run BLAB questions with the audio replaced by silence and score residual accuracy, since any model that still answers above chance is effectively gambling on the prompt.
  • Because speaker-count ground truth is a range in 60% of cases, scoring models against individual annotators rather than the consensus range could reveal whether model errors concentrate where humans disagree, a sign of inherent ambiguity rather than model failure.
  • Gemini's degraded performance when a clean clip sits in the middle of noisy audio suggests an architectural probe: compare full-context listening with a chunk-then-aggregate pipeline on the same BLAB items to see whether the bottleneck is long-range attention or auditory perception.
  • If the benchmark's difficulty is confirmed, cascade systems built from transcription, forced alignment, and text search may beat end-to-end audio LMs on word and entity localization even though BLAB was designed to resist cascades; a direct head-to-head comparison would settle that.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. This paper introduces BLAB, a benchmark of 833+ hours of long-form audio (samples from 15 minutes to 2 hours) with eight tasks across four categories: localization, counting, emotion, and duration estimation. The authors evaluate Gemini 2.0 Flash and Gemini 2.0 Pro on the full-length BLAB tasks and six audio LMs on a 30-second subset called BLAB-MINI. They report uniformly low performance on long-form tasks, with localization F1 scores near zero, exact-match accuracies below 10% on counting and duration tasks, and better but still limited performance on emotion reasoning. The paper includes analyses of duration effects, chain-of-thought prompting, and robustness to silence and Gaussian noise. The central empirical claim is that current audio LMs, including Gemini 2.0 Pro and GPT-4o, struggle with long-form audio and that performance declines with increasing duration.

Significance. If validated, BLAB fills a real gap in audio evaluation: most existing benchmarks cap audio at 30 seconds to 5 minutes, and timestamp localization over hour-scale audio is largely absent from prior benchmarks. The dataset construction from permissively licensed YouTube sources, the human-assisted filtering pipeline, and the planned public release of data, code, and model generations are concrete strengths. The paper also makes falsifiable observations—near-zero localization scores, systematic underestimation of speaker counts, duration overestimation, and degraded robustness on long noisy inputs—that are useful for future work on long-context audio models. The significance is conditional, however, on resolving the evaluation-coverage, metric-confounding, and annotation-validation issues detailed below.

major comments (4)
  1. [Sec. 4.1, Fig. 3(a), Table 2] The paper itself discloses that Gemini's 8,096-token output context limits generation to roughly 261 word timestamps per sample, against ~10,500 ground-truth words. Under the word-level F1 metric, a model that emits exactly the first 261 correct timestamps has recall 261/10,500 ≈ 2.5% and F1 ≈ 4.9%, which is only slightly above the reported 1.12 (Flash) and 0.19 (Pro). The sharp improvement on BLAB-MINI (word F1 30.22 and 8.61) is therefore partly a decoding-capacity artifact because 30-second clips contain only ~200–300 words and fit inside the output budget; the comparison does not cleanly measure duration effects. I ask the authors to either evaluate word localization on a matched output budget (e.g., only the first 261 ground-truth words), report recall@K or precision conditioned on emitted tokens, or otherwise separate output-capacity limits from long-context understanding.
  2. [Abstract; Sec. 3.1 and Table 3] The paper claims to evaluate six audio LMs on BLAB, but full-length BLAB results (Table 2) are only for Gemini 2.0 Flash and Gemini 2.0 Pro. Qwen2-Audio, Audio Flamingo 2, Phi-4-Multimodal, and GPT-4o were evaluated only on BLAB-MINI, the ≤30-second subset. Thus the statement that 'all of them, including advanced models such as Gemini 2.0 Pro and GPT-4o, struggle with the tasks in BLAB' does not follow from the reported experiments. Please revise the abstract and conclusion to distinguish full-length evaluation (Gemini only) from short-clip evaluation (six models), or extend the evaluation to the additional models on representative long-form samples within their input limits.
  3. [Sec. 2.1, Named Entity Localization; Limitations] Ground-truth entity spans for NE localization are produced by feeding WhisperX transcripts to Claude-3 and mapping extracted spans back to timestamps, with no described human verification step, unlike advertisement localization, where an annotator reviewed all predicted segments and removed false positives. Because the Limitations section acknowledges that WhisperX is not robust to overlapping speech, incomplete or misaligned entity spans would depress frame-level F1 and inflate the benchmark's difficulty. I request a validation study of the NE extraction step (e.g., manual review of a sample with correction rates, or inter-annotator agreement), and reporting of how many predicted spans were corrected.
  4. [Sec. 2.3, Table 1, Sec. 3.2] The size of the emotion tasks is internally inconsistent. Table 1's caption says each task has 200 question-audio-answer pairs, and Section 3.2 says emotion tasks result in 200 unique questions, but Section 2.3 reports 44 questions for Emotion Reasoning and 156 verified segments for Valence & Arousal Ranking. The reported emotion accuracies (e.g., 64.29% for Emotion Reasoning on Gemini 2.0 Pro) are therefore based on a much smaller sample than the benchmark's headline size, and no confidence intervals are given. Please correct the inconsistency and report uncertainty or per-sample variance for these small tasks.
minor comments (3)
  1. [Sec. 3.2, Table 6] In the Valence & Arousal Ranking prompt, the option list contains two entries labeled '(D)'—'Very Unpleasant' and 'Unpleasant'—and skips a unique label for one of them; fix the option letters.
  2. [Sec. 4.1] The sentence 'Gemini 2.0 flash achieves a precision score of 24.37 indicating that the model predictions are correct approximately 24.58% of the time' contains two different numbers for the same precision value; clarify which is correct.
  3. [Sec. 5, robustness] The sentence 'Our goal is to measure the models ability to model's ability to disregard background noise and focus on meaningful content' contains a duplicated phrase; rewrite it.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: BLAB is an empirical benchmark, and its evaluation results are independent measurements rather than derived consequences of its construction.

full rationale

BLAB is a benchmark construction and evaluation study, not a derivation chain. The central claims—that audio LMs perform poorly on long-form tasks and that performance declines with duration—are empirical findings obtained by running models on held-out benchmark data. The benchmark's ground-truth annotations are generated by independent procedures (forced alignment, human verification, LLM-assisted extraction with human review) that do not incorporate the evaluated models' outputs or any fitted parameters. The comparison between BLAB and BLAB-MINI is an experimental contrast, and although the word-localization results may be partly influenced by Gemini's output-context limit, that is a measurement confound or validity concern, not circularity: the paper does not define the benchmark in terms of the models' predictions, nor does it fit a parameter and then rename that fit as a prediction. The NE-localization labels generated by Claude-3 without full human verification raise annotation-quality concerns, but again this does not make the evaluation circular; the labels are external ground truth, not derived from the tested systems. Self-citations in the paper are limited to contextual references to prior benchmarks and do not carry a load-bearing argument. No equation or construction step equates an input with an output, and no uniqueness theorem or ansatz is imported from the authors' prior work to force a conclusion. The paper is therefore self-contained as a measurement study, and no significant circularity is present.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The central claim is an empirical evaluation, so there are no free parameters or invented entities. The main assumptions concern the quality of automatically generated annotations and the permissibility of the data.

assumptions (3)
  • domain assumption WhisperX forced alignment provides sufficiently accurate word-level timestamps for building localization ground truth.
    Used to generate word and NE localization labels; only 1% of word timestamps required correction, but no correction is reported for the entity-to-timestamp mapping.
  • domain assumption Claude-3's extracted named entity spans are accurate enough to serve as ground truth.
    Section 2.1: entity spans are extracted by Claude-3 from WhisperX transcripts and mapped to timestamps without explicit human verification.
  • domain assumption Creative Commons licensing of the YouTube sources permits redistribution and model evaluation.
    Stated in Ethics Statement, but no license verification details are provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BLAB: Brutally Long Audio Bench." pith.science (2026). https://pith.science/paper/TGFHWS6X

@misc{pith2026250503054,
  author       = {Pith},
  title        = {Pith review of: BLAB: Brutally Long Audio Bench},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TGFHWS6X}},
  note         = {Machine review of arXiv:2505.03054}
}
read the original abstract

Developing large audio language models (LMs) capable of understanding diverse spoken interactions is essential for accommodating the multimodal nature of human communication and can increase the accessibility of language technologies across different user populations. Recent work on audio LMs has primarily evaluated their performance on short audio segments, typically under 30 seconds, with limited exploration of long-form conversational speech segments that more closely reflect natural user interactions with these models. We introduce Brutally Long Audio Bench (BLAB), a challenging long-form audio benchmark that evaluates audio LMs on localization, duration estimation, emotion, and counting tasks using audio segments averaging 51 minutes in length. BLAB consists of 833+ hours of diverse, full-length audio clips, each paired with human-annotated, text-based natural language questions and answers. Our audio data were collected from permissively licensed sources and underwent a human-assisted filtering process to ensure task compliance. We evaluate six open-source and proprietary audio LMs on BLAB and find that all of them, including advanced models such as Gemini 2.0 Pro and GPT-4o, struggle with the tasks in BLAB. Our comprehensive analysis reveals key insights into the trade-offs between task difficulty and audio duration. In general, we find that audio LMs struggle with long-form speech, with performance declining as duration increases. They perform poorly on localization, temporal reasoning, counting, and struggle to understand non-phonemic information, relying more on prompts than audio content. BLAB serves as a challenging evaluation framework to develop audio LMs with robust long-form audio understanding capabilities.

Figures

Figures reproduced from arXiv: 2505.03054 by the authors.

Figure 1
Figure 1. Overview of BLAB, designed to test true multimodal understanding abilities of audio LMs. It contains eight distinct audio tasks across four categories, namely localization, counting, emotion, and duration estimation. †All images are designed by Freepik . temporal dimension is particularly important for understanding long-form content, where events unfold sequentially rather than as isolated units. To address this ga… view at source ↗
Figure 2
Figure 2. Comparison of predicted and ground truth values for speaker number estimation [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Comparison of long audio and short audio results across Gemini Models [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Performance comparison when the original audio input is replaced with silence or Gaussian noise. Since the entire input is noisy, the ground truth label is 0 We follow Sakshi et al. (2024) in this analysis and start by comparing the original model’s prediction for spea…
Figure 5
Figure 5. Figure 5: Placing a 30-second clean audio clip at different points within a long, noisy audio [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. VoxZip: Semantic-Anchored Temporal KV Cache Compression for Long-Context Audio Inference

    cs.AI 2026-08 conditional novelty 6.0 of 10

    A train-free, ASR-anchored two-stage KV cache compression method preserves over 90% of uncompressed performance on long-context audio inference at 20x compression.

  2. Auto-AEG: Scalable Data Construction for Open-Vocabulary Audio Event Grounding

    cs.SD 2026-07 conditional novelty 6.0 of 10

    Automatically constructed synthetic exact-GT data plus multi-model pseudo-labels with interval-aware GRPO rewards improve LALM open-vocabulary audio event grounding on AEGBench and DESED.

  3. X$^3$-OPD: Distilling Reasoning into Large Audio-Language Models via On-Policy Alignment

    cs.LG 2026-07 conditional novelty 5.0 of 10

    X3-OPD improves audio-grounded reasoning by training the audio student on its own rollouts with token-level teacher feedback, using a three-tier paired text-audio corpus.

Reference graph

Works this paper leans on

51 extracted references · 16 canonical work pages · cited by 3 Pith papers

  1. [1]

    The claude 3 model family: Opus, sonnet, haiku

    AI Anthropic. The claude 3 model family: Opus, sonnet, haiku. Claude-3 Model Card, 1: 0 1, 2024

    2024

  2. [2]

    wav2vec 2.0: a framework for self-supervised learning of speech representations

    Alexei Baevski, Henry Zhou, Abdelrahman Mohamed, and Michael Auli. wav2vec 2.0: a framework for self-supervised learning of speech representations. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS '20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546

    2020

  3. [3]

    Longbench: A bilingual, multitask benchmark for long context understanding

    Yushi Bai, Xin Lv, Jiajie Zhang, Hong Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. Longbench: A bilingual, multitask benchmark for long context understanding. Annual Meeting of the Association for Computational Linguistics, 2023

    work page 2023

  4. [4]

    Whisperx: Time-accurate speech transcription of long-form audio

    Max Bain, Jaesung Huh, Tengda Han, and Andrew Zisserman. Whisperx: Time-accurate speech transcription of long-form audio. INTERSPEECH 2023, 2023

    2023

  5. [5]

    Which of these best describes multiple choice evaluation with llms? a) forced b) flawed c) fixable d) all of the above, 2025

    Nishant Balepur, Rachel Rudinger, and Jordan Lee Boyd-Graber. Which of these best describes multiple choice evaluation with llms? a) forced b) flawed c) fixable d) all of the above, 2025. URL https://arxiv.org/abs/2502.14127

    arXiv 2025

  6. [6]

    pyannote.audio 2.1 speaker diarization pipeline: principle, benchmark, and recipe

    Hervé Bredin. pyannote.audio 2.1 speaker diarization pipeline: principle, benchmark, and recipe . In Proc. INTERSPEECH 2023, 2023

    2023

  7. [7]

    Train long and test long:leveraging full document contexts in speech processing

    William Chen, Takatomo Kano, Atsunori Ogawa, Marc Delcroix, and Shinji Watanabe. Train long and test long:leveraging full document contexts in speech processing. ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.\ 13066--13070, 2024. URL https://api.semanticscholar.org/CorpusID:268585517

    work page 2024

  8. [8]

    Qwen2-audio technical report

    Yunfei Chu, Jin Xu, Qian Yang, Haojie Wei, Xipin Wei, Zhifang Guo, Yichong Leng, Yuanjun Lv, Jinzheng He, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen2-audio technical report. arXiv preprint arXiv:2407.10759, 2024

    arXiv 2024

Show all 51 references
  1. [9]

    Speechbert: An audio-and-text jointly learned language model for end-to-end spoken question answering

    Yung-Sung Chuang, Chi-Liang Liu, Hung yi Lee, and Lin shan Lee. Speechbert: An audio-and-text jointly learned language model for end-to-end spoken question answering. In Interspeech 2020, pp.\ 4168--4172, 2020. doi:10.21437/Interspeech.2020-1570

    2020 doi

  2. [10]

    Drossos, Samuel Lipping, and Tuomas Virtanen

    K. Drossos, Samuel Lipping, and Tuomas Virtanen. Clotho: an audio captioning dataset. IEEE International Conference on Acoustics, Speech, and Signal Processing, 2019

    2019

  3. [11]

    Gemini Team , Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, Soroosh Mariooryad, Yifan Ding, Xinyang Geng, Fred Alcober, Roy Frostig, Mark Omernick, Lexi Walker, Cosmin Paduraru, Christina Sorokin, ...

    2024 arXiv

  4. [12]

    Sakshi, Oriol Nieto, R

    Sreyan Ghosh, Ashish Seth, Sonal Kumar, Utkarsh Tyagi, Chandra Kiran Reddy Evuru, S Ramaneswaran, S. Sakshi, Oriol Nieto, R. Duraiswami, and Dinesh Manocha. Compa: Addressing the gap in compositional reasoning in audio-language models. International Conference on Learning Repr...

    2023

  5. [13]

    GAMA : A large audio-language model with advanced audio understanding and complex reasoning abilities

    Sreyan Ghosh, Sonal Kumar, Ashish Seth, Chandra Kiran Reddy Evuru, Utkarsh Tyagi, S Sakshi, Oriol Nieto, Ramani Duraiswami, and Dinesh Manocha. GAMA : A large audio-language model with advanced audio understanding and complex reasoning abilities. In Yaser Al-Onaizan, Mohit Ban...

    2024

  6. [14]

    Gama: A large audio-language model with advanced audio understanding and complex reasoning abilities, 2024 b

    Sreyan Ghosh, Sonal Kumar, Ashish Seth, Chandra Kiran Reddy Evuru, Utkarsh Tyagi, S Sakshi, Oriol Nieto, Ramani Duraiswami, and Dinesh Manocha. Gama: A large audio-language model with advanced audio understanding and complex reasoning abilities, 2024 b . URL https://arxiv.org/...

    2024 arXiv

  7. [15]

    Audio flamingo 2: An audio-language model with long-audio understanding and expert reasoning abilities, 2025

    Sreyan Ghosh, Zhifeng Kong, Sonal Kumar, S Sakshi, Jaehyeon Kim, Wei Ping, Rafael Valle, Dinesh Manocha, and Bryan Catanzaro. Audio flamingo 2: An audio-language model with long-audio understanding and expert reasoning abilities, 2025. URL https://arxiv.org/abs/2503.03983

    2025 arXiv

  8. [16]

    Liu, Hongyin Luo, Leonid Karlinsky, and James Glass

    Yuan Gong, Alexander H. Liu, Hongyin Luo, Leonid Karlinsky, and James Glass. Joint audio and speech understanding. In 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pp.\ 1--8, 2023. doi:10.1109/ASRU57964.2023.10389742

    2023

  9. [17]

    Dynamic-superb: Towards a dynamic, collaborative, and comprehensive instruction-tuning benchmark for speech

    Chien-yu Huang, Ke-Han Lu, Shi Wang, Chi-Yuan Hsiao, Chun-Yi Kuan, Haibin Wu, Siddhant Arora, Kai-Wei Chang, Jiatong Shi, Yifan Peng, Roshan Sharma, Shinji Watanabe, Bhiksha Ramakrishnan, Shady Shehata, and Hung yi Lee. Dynamic-superb: Towards a dynamic, collaborative, and com...

    2023

  10. [18]

    Chien-yu Huang, Wei-Chih Chen, Shu wen Yang, Andy T. Liu, Chen-An Li, Yu-Xiang Lin, Wei-Cheng Tseng, Anuj Diwan, Yi-Jen Shih, Jiatong Shi, William Chen, Xuanjun Chen, Chi-Yuan Hsiao, Puyuan Peng, Shih-Heng Wang, Chun-Yi Kuan, Ke-Han Lu, Kai-Wei Chang, Chih-Kai Yang, Fabian Rit...

    2024 arXiv

  11. [19]

    Audiocaps: Generating captions for audios in the wild

    Chris Dongjoo Kim, Byeongchang Kim, Hyunmin Lee, and Gunhee Kim. Audiocaps: Generating captions for audios in the wild. North American Chapter of the Association for Computational Linguistics, 2019

    2019

  12. [20]

    A. S. Koepke, Andreea-Maria Oncescu, João F. Henriques, Zeynep Akata, and Samuel Albanie. Audio retrieval with natural language queries: A benchmark study. IEEE transactions on multimedia, 2021

    2021

  13. [21]

    Large language models are zero-shot reasoners

    Takeshi Kojima, Shixiang (Shane) Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp.\ 2...

    2022

  14. [22]

    Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang

    Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12: 0 157--173, 2024. doi:10.1162/tacl_a_00638...

    2024 doi

  15. [23]

    S. R. Livingstone and F. Russo. The ryerson audio-visual database of emotional speech and song (ravdess): A dynamic, multimodal set of facial and vocal expressions in north american english. PLoS ONE, 2018

    2018

  16. [24]

    Lotfian and C

    R. Lotfian and C. Busso. Building naturalistic emotionally balanced speech corpus by retrieving emotional speech from existing podcast recordings. IEEE Transactions on Affective Computing, 10 0 (4): 0 471--483, October-December 2019. doi:10.1109/TAFFC.2017.2736999

    2019

  17. [25]

    Martínez, Georgios N

    Héctor P. Martínez, Georgios N. Yannakakis, and John Hallam. Don’t classify ratings of affect; rank them! IEEE Transactions on Affective Computing, 5 0 (3): 0 314--326, 2014. doi:10.1109/TAFFC.2014.2352268

    2014

  18. [26]

    Phi-4-mini technical report: Compact yet powerful multimodal language models via mixture-of-loras, 2025

    Microsoft, :, Abdelrahman Abouelenin, Atabak Ashfaq, Adam Atkinson, Hany Awadalla, Nguyen Bach, Jianmin Bao, Alon Benhaim, Martin Cai, Vishrav Chaudhary, Congcong Chen, Dong Chen, Dongdong Chen, Junkun Chen, Weizhu Chen, Yen-Chun Chen, Yi ling Chen, Qi Dai, Xiyang Dai, Ruchao ...

    2025 arXiv

  19. [27]

    Needle in the haystack for memory based large language models

    Elliot Nelson, Georgios Kollias, Payel Das, Subhajit Chaudhury, and Soham Dan. Needle in the haystack for memory based large language models. arXiv preprint arXiv:2407.01437, 2024

    2024 arXiv

  20. [28]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff ...

    2024 arXiv

  21. [29]

    Plausibly problematic questions in multiple-choice benchmarks for commonsense reasoning

    Shramay Palta, Nishant Balepur, Peter Rankel, Sarah Wiegreffe, Marine Carpuat, and Rachel Rudinger. Plausibly problematic questions in multiple-choice benchmarks for commonsense reasoning. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Findings of the Association...

    2024 doi

  22. [30]

    Khudanpur

    Vassil Panayotov, Guoguo Chen, Daniel Povey, and S. Khudanpur. Librispeech: An asr corpus based on public domain audio books. IEEE International Conference on Acoustics, Speech, and Signal Processing, 2015

    2015

  23. [31]

    Powerset multi-class cross entropy loss for neural speaker diarization

    Alexis Plaquet and Hervé Bredin. Powerset multi-class cross entropy loss for neural speaker diarization . In Proc. INTERSPEECH 2023, 2023

    2023

  24. [32]

    Robust speech recognition via large-scale weak supervision, 2022

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision, 2022. URL https://arxiv.org/abs/2212.04356

    2022 arXiv

  25. [33]

    James A. Russell. A circumplex model of affect. Journal of Personality and Social Psychology, 39 0 (6): 0 1161--1178, 1980. doi:10.1037/h0077714

    1980 doi

  26. [34]

    Sainath, Ruoming Pang, David Rybach, Basi Garc \'i a, and Trevor Strohman

    Tara N. Sainath, Ruoming Pang, David Rybach, Basi Garc \'i a, and Trevor Strohman. Emitting word timings with end-to-end models. In Interspeech, 2020. URL https://api.semanticscholar.org/CorpusID:226200377

    2020

  27. [35]

    Mmau: A massive multi-task audio understanding and reasoning benchmark, 2024

    S Sakshi, Utkarsh Tyagi, Sonal Kumar, Ashish Seth, Ramaneswaran Selvakumar, Oriol Nieto, Ramani Duraiswami, Sreyan Ghosh, and Dinesh Manocha. Mmau: A massive multi-task audio understanding and reasoning benchmark, 2024. URL https://arxiv.org/abs/2410.19168

    2024 arXiv

  28. [36]

    Narayanan

    Vidhyasaharan Sethu, Emily Mower Provost, Julien Epps, Carlos Busso, Nicholas Cummins, and Shrikanth S. Narayanan. The ambiguous world of emotion representation. ArXiv, abs/1909.00360, 2019. URL https://api.semanticscholar.org/CorpusID:202540864

    1909 arXiv

  29. [37]

    ML-SUPERB 2.0: Benchmarking Multilingual Speech Models Across Modeling Constraints, Languages, and Datasets

    Jiatong Shi, Shih-Heng Wang, William Chen, Martijn Bartelds, Vanya Bannihatti Kumar , Jinchuan Tian, Xuankai Chang, Dan Jurafsky, Karen Livescu, Hung yi Lee, and Shinji Watanabe. ML-SUPERB 2.0: Benchmarking Multilingual Speech Models Across Modeling Constraints, Languages, and...

    2024 doi

  30. [38]

    SLUE phase-2: A benchmark suite of diverse spoken language understanding tasks

    Suwon Shon, Siddhant Arora, Chyi-Jiunn Lin, Ankita Pasad, Felix Wu, Roshan Sharma, Wei-Lun Wu, Hung-yi Lee, Karen Livescu, and Shinji Watanabe. SLUE phase-2: A benchmark suite of diverse spoken language understanding tasks. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazak...

    2023

  31. [39]

    SALMONN : Towards generic hearing abilities for large language models

    Changli Tang, Wenyi Yu, Guangzhi Sun, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun MA, and Chao Zhang. SALMONN : Towards generic hearing abilities for large language models. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/...

    2024

  32. [40]

    Long range arena: A benchmark for efficient transformers

    Yi Tay, Mostafa Dehghani, Samira Abnar, Yikang Shen, Dara Bahri, Philip Pham, Jinfeng Rao, Liu Yang, Sebastian Ruder, and Donald Metzler. Long range arena: A benchmark for efficient transformers. arXiv preprint arXiv:2011.04006, 2020

    2011 arXiv

  33. [41]

    Turian, Jordie Shier, H

    Joseph P. Turian, Jordie Shier, H. Khan, B. Raj, Björn Schuller, C. Steinmetz, C. Malloy, G. Tzanetakis, Gissel Velarde, K. McNally, Max Henry, Nicolas Pinto, Camille Noufi, Christian Clough, Dorien Herremans, Eduardo Fonseca, Jesse Engel, J. Salamon, P. Esling, Pranay Manocha...

    2022

  34. [42]

    Dawn of the transformer era in speech emotion recognition: Closing the valence gap

    Johannes Wagner, Andreas Triantafyllopoulos, Hagen Wierstorf, Maximilian Schmitt, Felix Burkhardt, Florian Eyben, and Bj \"o rn W Schuller. Dawn of the transformer era in speech emotion recognition: Closing the valence gap. IEEE Transactions on Pattern Analysis and Machine Int...

    2023

  35. [43]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neu...

    2022

  36. [44]

    AIR -bench: Benchmarking large audio-language models via generative comprehension

    Qian Yang, Jin Xu, Wenrui Liu, Yunfei Chu, Ziyue Jiang, Xiaohuan Zhou, Yichong Leng, Yuanjun Lv, Zhou Zhao, Chang Zhou, and Jingren Zhou. AIR -bench: Benchmarking large audio-language models via generative comprehension. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.),...

    1979 doi

  37. [45]

    Lin, Andy T

    Shu-Wen Yang, Po-Han Chi, Yung-Sung Chuang, Cheng-I Lai, Kushal Lakhotia, Yist Y. Lin, Andy T. Liu, Jiatong Shi, Xuankai Chang, Guan-Ting Lin, Tzu hsien Huang, W. Tseng, Ko tik Lee, Da-Rong Liu, Zili Huang, Shuyan Dong, Shang-Wen Li, Shinji Watanabe, Abdel rahman Mohamed, and ...

    2021

  38. [46]

    Yannakakis, R

    G.N. Yannakakis, R. Cowie, and C. Busso. The ordinal nature of emotions. In International Conference on Affective Computing and Intelligent Interaction (ACII 2017), pp.\ 248--255, San Antonio, TX, USA, October 2017. doi:10.1109/ACII.2017.8273608

    2017

  39. [47]

    Yannakakis, R

    G.N. Yannakakis, R. Cowie, and C. Busso. The ordinal nature of emotions: An emerging approach. IEEE Transactions on Affective Computing, 12 0 (1): 0 16--35, January-March 2021. doi:10.1109/TAFFC.2018.2879512

    2021

  40. [48]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  41. [49]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  42. [50]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  43. [51]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

    2025

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.