Pith. sign in

REVIEW 3 major objections 3 minor 39 references

A single self-supervised audio encoder, combined with a simple layer-fusion adapter, can match or beat specialised supervised and multi-encoder designs in audio large language models, while using far less supervised training data.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A single frozen self-supervised audio encoder plus a multi-layer fusion adapter achieves comparable or better audio understanding than supervised multi-encoder systems, with explicit training required for multimodal in-context learning.

T0 review reviewed 2026-08-01 challenge →

load-bearing objection A genuinely useful ALLM architecture study with a credible controlled ablation, but the SOTA claim rests on cross-paper numbers and one metric switch, so treat the headline margins as provisional. the 3 major comments →

arxiv 2607.17079 v1 pith:KSFYIE5M submitted 2026-07-19 eess.AS

SALMONN-2: Advancing General-Purpose Hearing Abilities with Self-Supervised Representations

Xiaoyu Yang , Xuenan Xu , Wenyi Yu , Siyin Wang , Changli Tang , Terumi Chiba , Siyuan Hou , Ziyang Zhang
show 5 more authors
Wen Wu Baoxiang Li Guangzhi Sun Chao Zhang Philip Woodland
This is my paper
classification eess.AS
keywords audio large language modelsself-supervised learningmulti-layer feature fusionmultimodal in-context learningcontextual biasingaudio understanding benchmarksSPEARtimestamp injection
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SALMONN-2 sets out to prove that a general-purpose self-supervised audio encoder can serve as the sole perception module for a capable audio large language model (ALLM). The paper shows that a frozen SSL encoder (SPEAR) plus a newly proposed multi-layer feature fusion adapter matches or outperforms specialised supervised encoders and dual-encoder architectures across speech, audio, music, and paralinguistic tasks. On three general audio understanding benchmarks (MMAU-Pro, MMAR, MMSU) the resulting 9B model achieves state-of-the-art results among similar-scale open-weight ALLMs while using under 20K hours of instruction-tuning data. The paper also establishes that multimodal in-context learning for contextual speech recognition does not emerge naturally from standard audio-text instruction tuning, but can be instilled through targeted contextual biasing training. These findings matter because they suggest that large-scale supervised audio-text data may not be the only or best route to general-purpose audio intelligence.

Core claim

SALMONN-2 demonstrates that a single general-purpose SSL audio encoder, SPEAR, combined with a multi-layer feature fusion (MLF) adapter, is a more effective and data-efficient foundation for an ALLM than specialised supervised encoders or dual-encoder setups. The MLF adapter concatenates layer-normalised hidden states from all 13 encoder layers, down-projects them, and then projects the fused frame sequence into the LLM embedding space, preserving hierarchical acoustic information that final-layer representations discard. The resulting model, with a 9B or 30B Qwen LLM adapted via LoRA, outperforms comparable open-weight ALLMs on MMAU-Pro, MMAR, and MMSU. It also gains extended audio analysis

What carries the argument

The multi-layer feature fusion (MLF) adapter is the central mechanism that lets a single SSL encoder replace specialised multi-encoder designs. It takes the hidden states from all 13 SPEAR layers, applies layer normalisation to each, concatenates them along the feature dimension, learns a down-projection to 1280 dimensions, groups frames to reduce the 50 Hz rate to 10 Hz, and then maps the fused sequence through a two-layer MLP into the LLM embedding space. Timestamp tokens are interleaved among audio embeddings using existing LLM vocabulary embeddings, and multimodal context is formatted as audio-text pairs. Together, these pieces let the frozen SSL encoder's rich hierarchical representatio

Load-bearing premise

The state-of-the-art benchmark claim depends on comparing SALMONN-2's numbers with baseline scores that were obtained under different prompting, decoding, and metric-calibration settings (official reports or heuristic prompts), and the AAC metric was changed to SentenceBERT similarity, so the paper's margins on MMAR and MMSU (as small as 0.1–0.3 points) could disappear under a strictly controlled re-evaluation.

What would settle it

Run SALMONN-2 and the main baselines (e.g., MOSS-Audio, Audio Flamingo 3, Kimi-Audio) under identical evaluation conditions: same prompt template for each task, same decoding parameters, and the same SentenceBERT metric for AAC and the same official metric harnesses for MMAU-Pro, MMAR, and MMSU. If MOSS-Audio or another baseline then matches or beats SALMONN-2 on MMAR and MMSU within the reported margins, or if using n-gram metrics flips the AudioCaps/Clotho ordering, the paper's central claims of state-of-the-art performance and general superiority of the SSL+MLF design are called into questi

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If a single frozen SSL encoder plus a lightweight adapter can match or beat specialised multi-encoder ALLMs, future ALLM systems can simplify their architecture and reduce the need for large volumes of supervised audio-text paired data.
  • The MLF adapter's consistent gains over final-layer and weighted-sum baselines indicate that SSL encoders encode task-relevant information across all layers, and that preserving layer-wise information before projection is generally beneficial for downstream audio-language tasks.
  • The finding that multimodal in-context learning does not emerge spontaneously, but can be trained with targeted biased-context data, provides a practical recipe for teaching ALLMs to exploit rare-word pronunciations and contextual cues.
  • Timestamp injection using textual token embeddings improves not only sound event detection but also general audio QA benchmarks, suggesting that explicit temporal grounding is a cheap and broadly useful addition for ALLMs.
  • Scaling the LLM backbone from 8B to 30B-A3B yields larger gains on reasoning-heavy benchmarks than on canonical recognition tasks, implying that audio understanding limits are increasingly set by language reasoning once the perception front-end is strong.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Other multimodal LLMs beyond audio—such as vision-language or video-language models built on frozen SSL or masked autoencoder backbones—may similarly benefit from a layer-fusion adapter rather than relying on final-layer features, a hypothesis the paper does not test but its design directly suggests.
  • The MICL result raises a caution about claims of emergent 'in-context learning' in other audio-language models: what looks like contextual understanding could be task-specific memorisation unless the model is explicitly trained on distractor-heavy context, as SALMONN-2 is.
  • If the data-efficiency result holds under controlled re-evaluation, it implies that the industry-wide race to collect millions of hours of transcribed or captioned audio may be partly overkill; a well-frozen SSL front-end could lower the barrier to entry for building general-purpose audio assistants.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper presents SALMONN-2, an audio LLM that replaces supervised or dual audio encoders with a single frozen self-supervised encoder (SPEAR), feeds the LLM through a multi-layer feature fusion (MLF) adapter that concatenates all encoder layers, injects textual timestamp tokens, and adds explicit multimodal in-context learning (MICL) training for contextual ASR. The model is trained in two stages on 18.2k hours of instruction data and evaluated on canonical tasks (ASR, AAC, MC, ER, En2Zh, OSR, SV), three ALLM benchmarks (MMAU-Pro, MMAR, MMSU), contextual ASR with text and audio biasing lists, and extended analysis tasks (SED, spoofing, SQA). The claims are that a single SSL encoder is comparable or superior to supervised/multi-encoder designs, that the proposed MLF adapter improves over final-layer and weighted-sum pooling, and that SALMONN-2 reaches state-of-the-art among similar-scale open-weight models while using under 20K hours of instruction-tuning data.

Significance. If the benchmark claims hold, the work is significant for ALLM design: it provides evidence that a large supervised audio front-end is not required, that cheap adapter modifications can exploit SSL hierarchy, and that MICL can be instilled by targeted training rather than being expected to emerge. The paper has concrete strengths: controlled ablations with a common reduced training budget and frozen encoders, transparent reporting of heuristic prompts for some baselines, and open-source code and model weights. The main weakness is that the externally validated headline numbers rely on cross-paper comparisons with a changed AAC metric and small margins, so the SOTA/efficiency conclusions are not yet supported at the same standard as the internal ablations.

major comments (3)
  1. [V-B, Tables V–VI] The headline SOTA claim (MMAU-Pro 58.5, MMAR 64.5, MMSU 69.5) is based on cross-paper numbers that are not produced under a shared protocol. Table VI notes that training-hour values are 'approximate and may not be strictly comparable,' and Table V states that some baselines were run with 'heuristically optimised prompts.' On MMAR the margin over MOSS-Audio is 0.1 point. Prompt template, decoding settings, and answer parsing can move such scores by more than this. Please release the exact evaluation harness (prompts, decoding parameters, parsing rules) and either re-run all baselines under it or provide a sensitivity analysis across reasonable prompt variants; if sampling is used, report variance over repeated runs.
  2. [IV-E and Table V] The AAC columns in Table V are not comparable as presented. Section IV-E changes the metric to SentenceBERT similarity, while many baseline numbers are taken from official reports/technical reports that historically report n-gram-based AAC metrics. Unless every baseline in AudioCaps/Clotho was re-evaluated with the same SBERT similarity, the 'best on AudioCaps' claim is an artifact of metric mismatch. Please state which metric was used for each cell and re-evaluate all baselines with the identical metric, or remove the cross-model AAC comparison.
  3. [III-B and V-E (Table IX)] The paper's general claim about 'a general-purpose SSL encoder' is supported by experiments with a single SSL model, SPEAR, which is also developed by the authors. Table IX compares SPEAR with supervised and dual-encoder alternatives, but does not include any other SSL encoder as a single encoder with the same MLF adapter. As written, the title/abstract generalize beyond the evidence. Either add an independent SSL encoder with comparable domain coverage to the controlled ablation, or reframe the conclusions to 'SPEAR, a unified SSL encoder' throughout.
minor comments (3)
  1. [Table V] Typos in table headers: 'CoV oST2' should be 'CoVoST2' and 'V oxCeleb1' should be 'VoxCeleb1'. Also standardize 'AF-NEXT' vs 'AF-Next' between the caption and the table body.
  2. [Table VII] The row for MICL-trained SALMONN-2 appears to have one fewer WER/B-WER/U-WER triple than the table has evaluation columns. Add an em-dash for the no-biasing condition or clarify why it is omitted.
  3. [IV-C] The timestamp granularity g is set to 2, but no unit is stated. Specify that it is 2 seconds.

Circularity Check

0 steps flagged

No significant circularity: headline results are external benchmark evaluations and no claim reduces by construction to its inputs.

full rationale

No step in the paper's derivation chain is circular by the standards of this pass. The headline results (Tables V, VI, VII, VIII) are measured on held-out external benchmarks; no benchmark score is used as a training target or fused into model parameters. The proposed MLF adapter (Eqs. 2-6) is a deterministic architecture whose outputs are not defined in terms of the downstream metrics it is claimed to improve. The encoder choice, SPEAR [7], is a self-citation with author overlap, but the paper does not rely on [7] alone: SPEAR is re-evaluated in this paper's own ablations (Table IX), and its performance is an empirical result rather than an imported uniqueness or ansatz. Comparisons against SALMONN, HoliAntiSpoof, and QualiSpeech are likewise evaluations of prior or specialized systems, not derivations that presuppose the conclusion. The MICL/contextual-ASR experiments train with explicit biasing data and test on held-out GigaSpeech; the observation that an untrained variant fails is a controlled ablation, not a tautology. The table notes about approximate training hours, heuristically optimized prompts, and the SBERT AAC metric are validity/comparability caveats and thus correctness risks, but they do not show that any result reduces by construction to its inputs. No fitted parameter is renamed as a prediction, and no uniqueness theorem from the authors' prior work is invoked to force a design choice. The paper is self-contained enough for a circularity finding of zero.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The paper's claims are empirical; the ledger captures the assumptions that make the benchmark comparisons meaningful and the generalization from SPEAR to SSL encoders valid.

free parameters (4)
  • Frame grouping size q = 5
    Chosen to reduce audio token rate from 50 Hz to 10 Hz; not swept.
  • Timestamp granularity g = 2 seconds
    Chosen for timestamp injection; k=20 tokens per interval.
  • LoRA rank = 64 (8B) / 64 (30B-A3B)
    Adapter capacity for LLM tuning; set by hand.
  • Distractor ratio and dropout probabilities in contextual ASR = 25–50 distractors, 0.1 word dropout, 0.2 audio dropout
    Hyperparameters for MICL training; chosen without systematic ablation.
axioms (4)
  • domain assumption Benchmark scores on MMAU-Pro, MMAR, MMSU measure general-purpose audio understanding and transfer across domains.
    The SOTA claim rests on these benchmarks being valid and representative.
  • domain assumption Reported baseline results from other papers and the authors' heuristic-prompt evaluations are directly comparable to the authors' own evaluation.
    Cross-paper score comparisons can differ due to prompts, decoders, and metrics (e.g., SBERT vs n-gram).
  • domain assumption Training and evaluation sets do not overlap in a way that inflates SALMONN-2's performance.
    Training includes AudioSet-strong and GigaSpeech/SPGISpeech; evaluation uses DESED and GigaSpeech test; no contamination analysis is provided.
  • ad hoc to paper SPEAR's pre-training without any label data yields representations that are representative of general-purpose SSL encoders.
    Only one SSL encoder is tested; the conclusion is phrased generally.

reviewed 2026-08-01 · how reviews work

0 comments
Cite this review

Pith. "Pith review of SALMONN-2: Advancing General-Purpose Hearing Abilities with Self-Supervised Representations." pith.science (2026). https://pith.science/paper/KSFYIE5M

@misc{pith2026260717079,
  author       = {Pith},
  title        = {Pith review of: SALMONN-2: Advancing General-Purpose Hearing Abilities with Self-Supervised Representations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KSFYIE5M}},
  note         = {Machine review of arXiv:2607.17079}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Recent audio large language models (ALLMs) are typically built upon audio encoders trained with large amounts of supervised data. Since self-supervised learning (SSL) audio encoder models are known to learn general-purpose and transferable representations, we investigate whether general-purpose SSL audio representations can serve as an effective foundation for ALLMs. We present SALMONN-2, an ALLM built upon a unified SSL encoder. To better exploit the hierarchical representations learned by SSL encoders, we propose a multi-layer feature fusion (MLF) adapter that aggregates information from all encoder layers before projecting them into the language model. Beyond conventional audio understanding tasks, we further explore multimodal in-context learning (MICL) in ALLMs and study how this capability can be acquired through contextual biasing training. Experimental results show that a general-purpose SSL encoder achieves performance comparable to, or better than, specialised supervised audio encoders while providing a more balanced capability across speech, audio, music and paralinguistic tasks. SALMONN-2 further achieves state-of-the-art performance among comparable-scale open-weight models on ALLM understanding benchmarks, obtaining the best results on MMAU-Pro, MMAR and MMSU. We also show that MICL does not emerge naturally in ALLMs, but can be effectively acquired through targeted contextual biasing training.

Figures

Figures reproduced from arXiv: 2607.17079 by Baoxiang Li, Changli Tang, Chao Zhang, Guangzhi Sun, Philip Woodland, Siyin Wang, Siyuan Hou, Terumi Chiba, Wen Wu, Wenyi Yu, Xiaoyu Yang, Xuenan Xu, Ziyang Zhang.

Figure 1
Figure 1. Figure 1: Performance of SALMONN-2 and existing open-weight ALLMs. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The architecture of SALMONN-2. The unified SSL audio encoder (SPEAR) transforms the input waveform into layer-wise representations, which [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

39 extracted references · 10 linked inside Pith

  1. [1]

    SALMONN: Towards Generic Hearing Abilities for Large Language Models,

    C. Tang, W. Yu, G. Sun, X. Chen, T. Tan, W. Li, L. Lu, Z. MA, and C. Zhang, “SALMONN: Towards Generic Hearing Abilities for Large Language Models,” inProc. ICLR, Vienna, 2024

    2024

  2. [2]

    Qwen2-Audio Technical Report,

    Y . Chu, J. Xu, Q. Yang, H. Wei, X. Wei, Z. Guo, Y . Leng, Y . Lv, J. He, J. Linet al., “Qwen2-Audio Technical Report,”arXiv preprint arXiv:2407.10759, 2024

    Pith/arXiv arXiv 2024

  3. [3]

    MiMo-Audio: Audio Language Models are Few-Shot Learners,

    D. Zhang, G. Wang, J. Xue, K. Fang, L. Zhao, R. Ma, S. Ren, S. Liu, T. Guo, W. Zhuanget al., “MiMo-Audio: Audio Language Models are Few-Shot Learners,”arXiv preprint arXiv:2512.23808, 2025

    arXiv 2025

  4. [4]

    Step-Audio-R1.5 Technical Report,

    Y . Zhang, X. T. Zhang, D. Liu, F. Tian, Y . Deng, J. Chen, Q. Lin, H. Zhang, Y . Li, J. Gonget al., “Step-Audio-R1.5 Technical Report,” arXiv preprint arXiv:2604.25719, 2026

    Pith/arXiv arXiv 2026

  5. [5]

    Qwen2.5-Omni Technical Report,

    J. Xu, Z. Guo, J. He, H. Hu, T. He, S. Bai, K. Chen, J. Wang, Y . Fan, K. Danget al., “Qwen2.5-Omni Technical Report,”arXiv preprint arXiv:2503.20215, 2025

    Pith/arXiv arXiv 2025

  6. [6]

    Kimi-Audio Technical Report,

    D. Ding, Z. Ju, Y . Leng, S. Liu, T. Liu, Z. Shang, K. Shen, W. Song, X. Tan, H. Tanget al., “Kimi-Audio Technical Report,”arXiv preprint arXiv:2504.18425, 2025

    Pith/arXiv arXiv 2025

  7. [7]

    SPEAR: A Unified SSL Framework for Learning Speech and Audio Representations,

    X. Yang, Y . Yang, Z. Jin, Z. Cui, W. Wu, B. Li, C. Zhang, and P. Woodland, “SPEAR: A Unified SSL Framework for Learning Speech and Audio Representations,” inProc. ICML, Seoul, 2026

    2026

  8. [8]

    Qwen-Audio: Advancing Universal Audio Understand- ing via Unified Large-Scale Audio-Language Models,

    Y . Chu, J. Xu, X. Zhou, Q. Yang, S. Zhang, Z. Yan, C. Zhou, and J. Zhou, “Qwen-Audio: Advancing Universal Audio Understand- ing via Unified Large-Scale Audio-Language Models,”arXiv preprint arXiv:2311.07919, 2023

    Pith/arXiv arXiv 2023

  9. [9]

    MidashengLM: Efficient Audio Understanding with General Audio Captions,

    H. Dinkel, G. Li, J. Liu, J. Luan, Y . Niu, X. Sun, T. Wang, Q. Xiao, J. Zhang, and J. Zhou, “MidashengLM: Efficient Audio Understanding with General Audio Captions,”arXiv preprint arXiv:2508.03983, 2025

    arXiv 2025

  10. [10]

    Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Models,

    S. Ghosh, A. Goel, J. Kim, S. Kumar, Z. Kong, S.-g. Lee, C.-H. Yang, R. Duraiswami, D. Manocha, R. Valleet al., “Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Models,” inProc. NeurIPS, San Diego, 2026

    2026

  11. [11]

    MOSS-Audio Technical Report,

    C. Yang, C. Yu, H. Chen, J. Zhu, J. Chen, K. Chen, W. Wang, Y . Wang, Y . Jiang, Y . Jianget al., “MOSS-Audio Technical Report,”arXiv preprint arXiv:2606.01802, 2026

    Pith/arXiv arXiv 2026

  12. [12]

    HuBERT: Self-Supervised Speech Representation Learn- ing by Masked Prediction of Hidden Units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “HuBERT: Self-Supervised Speech Representation Learn- ing by Masked Prediction of Hidden Units,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 3451–3460, 2021

    2021

  13. [13]

    WavLM: Large-Scale Self-Supervised Pre- Training for Full Stack Speech Processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiaoet al., “WavLM: Large-Scale Self-Supervised Pre- Training for Full Stack Speech Processing,”IEEE Journal of Selected Topics in Signal Processing, vol. 16, no. 6, pp. 1505–1518, 2022

    2022

  14. [14]

    Masked Autoencoders that Listen,

    P.-Y . Huang, H. Xu, J. Li, A. Baevski, M. Auli, W. Galuba, F. Metze, and C. Feichtenhofer, “Masked Autoencoders that Listen,” inProc. NeurIPS, New Orleans, 2022

    2022

  15. [15]

    BEATs: Audio Pre-Training with Acoustic Tokenizers,

    S. Chen, Y . Wu, C. Wang, S. Liu, D. Tompkins, Z. Chen, and F. Wei, “BEATs: Audio Pre-Training with Acoustic Tokenizers,” inProc. ICML, Hawaii, 2023

    2023

  16. [16]

    MERT: Acoustic Music Understanding Model with Large-Scale Self-Supervised Training,

    Y . Li, R. Yuan, G. Zhang, Y . Ma, X. Chen, H. Yin, C. Xiao, C. Lin, A. Ragni, E. Benetoset al., “MERT: Acoustic Music Understanding Model with Large-Scale Self-Supervised Training,” inProc. ICLR, Vienna, 2024

    2024

  17. [17]

    MuQ: Self-Supervised Music Representation Learning with Mel Residual Vector Quantization,

    H. Zhu, Y . Zhou, H. Chen, J. Yu, Z. Ma, R. Gu, Y . Luo, W. Tan, and X. Chen, “MuQ: Self-Supervised Music Representation Learning with Mel Residual Vector Quantization,”IEEE Transactions on Audio, Speech and Language Processing, vol. 33, pp. 3653–3664, 2025

    2025

  18. [18]

    USAD: Universal Speech and Audio Representation via Distillation,

    H.-J. Chang, S. Bhati, J. Glass, and A. H. Liu, “USAD: Universal Speech and Audio Representation via Distillation,” inProc. ASRU, Hawaii, 2025

    2025

  19. [19]

    Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities,

    Z. Kong, A. Goel, R. Badlani, W. Ping, R. Valle, and B. Catanzaro, “Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities,” inProc. ICML, Vienna, 2024

    2024

  20. [20]

    A Large-Scale Evaluation of Speech Foundation Models,

    S.-w. Yang, H.-J. Chang, Z. Huang, A. T. Liu, C.-I. Lai, H. Wu, J. Shi, X. Chang, H.-S. Tsai, W.-C. Huang, T.-h. Feng, P.-H. Chi, Y . Y . Lin, Y .-S. Chuang, T.-H. Huang, W.-C. Tseng, K. Lakhotia, S.-W. Li, A. Mohamed, S. Watanabe, and H.-y. Lee, “A Large-Scale Evaluation of Speech Foundation Models,”IEEE/ACM Transactions on Audio, Speech, and Language Pr...

    2024

  21. [21]

    SpotSound: Enhancing Large Audio-Language Models with Fine-Grained Temporal Grounding,

    L. Sun, X. Zhou, Z. Li, Y . Zhang, Y . Wang, and W. Xie, “SpotSound: Enhancing Large Audio-Language Models with Fine-Grained Temporal Grounding,”arXiv preprint arXiv:2604.13023, 2026

    Pith/arXiv arXiv 2026

  22. [22]

    Towards Fine-grained Temporal Perception: Post-Training Large Audio-Language Models with Audio-Side Time Prompt,

    Y . Shi, P. Cai, J. Liu, Q. Gu, N. Jiang, L. Dai, I. McLoughlin, and Y . Song, “Towards Fine-grained Temporal Perception: Post-Training Large Audio-Language Models with Audio-Side Time Prompt,”arXiv preprint arXiv:2604.13715, 2026

    Pith/arXiv arXiv 2026

  23. [23]

    Qwen3.5-Omni Technical Report,

    Qwen Team, “Qwen3.5-Omni Technical Report,”arXiv preprint arXiv:2604.15804, 2026

    Pith/arXiv arXiv 2026

  24. [24]

    HoliAntiSpoof: Audio LLM for Holistic Speech Anti-Spoofing,

    X. Xu, Y . Ren, L. Liu, W. Wu, B. Li, C. Lu, S. Wang, and C. Zhang, “HoliAntiSpoof: Audio LLM for Holistic Speech Anti-Spoofing,”arXiv preprint arXiv:2602.04535, 2026

    arXiv 2026

  25. [25]

    The Benefit of Temporally-Strong Labels in Audio Event Classification,

    S. Hershey, D. P. Ellis, E. Fonseca, A. Jansen, C. Liu, R. C. Moore, and M. Plakal, “The Benefit of Temporally-Strong Labels in Audio Event Classification,” inProc. ICASSP, Toronto, 2021

    2021

  26. [26]

    FineLAP: Taming Heterogeneous Supervision for Fine-grained Language-Audio Pretraining,

    X. Li, X. Xu, Z. Ma, W. Chen, H. He, Q. Kong, and X. Chen, “FineLAP: Taming Heterogeneous Supervision for Fine-grained Language-Audio Pretraining,” inProc. ACL, San Diego, 2026

    2026

  27. [27]

    PicoAudio2: Temporal Controllable Text-to-Audio Generation with Natural Language Description,

    Z. Zheng, Z. Xie, X. Xu, W. Wu, C. Zhang, and M. Wu, “PicoAudio2: Temporal Controllable Text-to-Audio Generation with Natural Language Description,” inProc. ICASSP, Barcelona, 2026

    2026

  28. [28]

    Qualispeech: A Speech Quality As- sessment Dataset with Natural Language Reasoning and Descriptions,

    S. Wang, W. Yu, X. Chen, X. Tian, J. Zhang, L. Lu, Y . Tsao, J. Ya- magishi, Y . Wang, and C. Zhang, “Qualispeech: A Speech Quality As- sessment Dataset with Natural Language Reasoning and Descriptions,” inProc. ACL, Vienna, 2025

    2025

  29. [29]

    ZipFormer: A Faster and Better Encoder for Automatic Speech Recognition,

    Z. Yao, L. Guo, X. Yang, W. Kang, F. Kuang, Y . Yang, Z. Jin, L. Lin, and D. Povey, “ZipFormer: A Faster and Better Encoder for Automatic Speech Recognition,” inProc. ICLR, Vienna, 2024

    2024

  30. [30]

    GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audio,

    G. Chen, S. Chai, G. Wang, J. Duet al., “GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audio,” inProc. Interspeech, Brno, 2021

    2021

  31. [31]

    SPGISpeech: 5,000 Hours of Transcribed Financial Audio for Fully Formatted End-to-End Speech Recognition,

    P. K. O’Neill, V . Lavrukhin, S. Majumdar, V . Noroozi, Y . Zhang, O. Kuchaiev, J. Balam, Y . Dovzhenko, K. Freyberg, M. D. Shulman et al., “SPGISpeech: 5,000 Hours of Transcribed Financial Audio for Fully Formatted End-to-End Speech Recognition,” inProc. Interspeech, Brno, 2021

    2021

  32. [32]

    OmniV oice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models,

    H. Zhu, L. Ye, W. Kang, Z. Yao, L. Guo, F. Kuang, Z. Han, W. Zhuang, L. Lin, and D. Povey, “OmniV oice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models,” inProc. Interspeech, Sydney, 2026

    2026

  33. [33]

    Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks,

    N. Reimers and I. Gurevych, “Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks,” inProc. EMNLP, Hong Kong, 2019

    2019

  34. [34]

    MMAU-Pro: A Challenging and Comprehensive Benchmark for Holistic Evaluation of Audio General Intelligence,

    S. Kumar, ˇS. Sedl ´aˇcek, V . Lokegaonkar, F. L ´opez, W. Yu, N. Anand, H. Ryu, L. Chen, M. Pli ˇcka, M. Hlav ´aˇceket al., “MMAU-Pro: A Challenging and Comprehensive Benchmark for Holistic Evaluation of Audio General Intelligence,” inProc. AAAI, Singapore, 2026

    2026

  35. [35]

    MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mix,

    Z. Ma, Y . Ma, Y . Zhu, C. Yang, Y .-W. Chao, R. Xu, W. Chen, Y . Chen, Z. Chen, J. Conget al., “MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mix,” inProc. NeurIPS, San Diego, 2025

    2025

  36. [36]

    MMSU: A Massive Multi-Task Spoken Language Understanding and Reasoning Benchmark,

    D. Wang, J. Li, J. Wu, D. Yang, X. Chen, T. Zhang, and H. Meng, “MMSU: A Massive Multi-Task Spoken Language Understanding and Reasoning Benchmark,” inProc. ICLR, Rio de Janeiro, 2026

    2026

  37. [37]

    Sound Event Detection in Domestic Environments with Weakly Labeled Data and Soundscape Synthesis,

    N. Turpault, R. Serizel, A. P. Shah, and J. Salamon, “Sound Event Detection in Domestic Environments with Weakly Labeled Data and Soundscape Synthesis,” inProc. DCASE Workshop, New York, 2019

    2019

  38. [38]

    PartialEdit: Identifying Partial Deepfakes in the Era of Neural Speech Editing,

    Y . Zhang, B. Tian, L. Zhang, and Z. Duan, “PartialEdit: Identifying Partial Deepfakes in the Era of Neural Speech Editing,” inProc. Interspeech, Rotterdam, 2025

    2025

  39. [39]

    Audio Flamingo Next: Next-Generation Open Audio-Language Models for Speech, Sound, and Music,

    S. Ghosh, A. Goel, K. Jayakumar, L. Koroshinadze, N. Anand, Z. Kong, S. Gururani, S.-g. Lee, J. Kim, A. Aljafariet al., “Audio Flamingo Next: Next-Generation Open Audio-Language Models for Speech, Sound, and Music,”arXiv preprint arXiv:2604.10905, 2026

    Pith/arXiv arXiv 2026

This paper was first reviewed by deepseek-v4-flash on August 1, 2026.