Pith. sign in

REVIEW 3 major objections 4 minor 2 cited by

State-Space Large Audio Language Models

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Replacing both the audio encoder and the language model with state-space models yields an audio language model that matches transformer-based rivals while using about 40% of the parameters.

desk verdict A clean state-space swap of LTU that performs competitively, but the under-specified cosine-similarity evaluation leaves the headline claim shaky. read the letter →

arxiv 2411.15685 v1 pith:5NKNT24Z submitted 2024-11-24 eess.AS cs.AI

Saurabhchand Bhati , Yuan Gong , Leonid Karlinsky , Hilde Kuehne , Rogerio Feris , James Glass This is my paper · ORCID
classification eess.AScs.AI
keywords State-spacemodelsLargeaudiolanguagereasoningDASSMambaLow-rankadaptationclassificationcaptioning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether the two heavy transformer components of a large audio language model (LALM) — the audio perception encoder and the language model itself — can be swapped for state-space models without losing accuracy. It builds a hybrid model pairing the distilled audio state-space encoder DASS with the transformer LLaMA, and a full state-space LALM pairing DASS with a 2.8B state-space LLM. Across eight audio classification benchmarks and two captioning benchmarks, both models match or slightly exceed the transformer-based LTU baseline while using roughly 40% of the total parameters. The paper's claim is that state-space LALMs are a viable efficient alternative for close-ended audio reasoning tasks, retaining competitive performance with far lower memory and time overhead.

What carries the argument

The load-bearing pieces are DASS, a distilled audio state-space encoder trained on AudioSet-2M whose teacher is the transformer AST; the state-space LLM-2.8B trained on the Pile; and low-rank LoRA adapters that keep trainable parameters small. The audio is passed as a 1024x128 spectrogram into DASS, producing a 32x4x768 feature map that is downsampled and projected to the LLM embedding size. Training uses next-token prediction on audio-question-answer triples from OpenAQA, and evaluation scores classification by cosine similarity between the text-encoder embedding of the model output and each class label, and captioning by SPICE.

What would settle it

Re-evaluate the same models on the same benchmarks using forced-choice label generation without a text-encoder similarity step, or with human raters judging the outputs; if transformer LALMs then clearly outperform ssLALMs on close-ended accuracy, the paper's competitive claim collapses.

Watch

Extended reading notes

Core claim

The central discovery is that state-space models can carry both ends of an audio-language system: a state-space audio encoder distilled from a transformer teacher, together with a state-space LLM, is enough to reach the accuracy of transformer-based LALMs on close-ended tasks. The paper reports that the full state-space ssLALM with 2.8B total parameters achieves a classification average of 51.4 (medium) versus LTU's 50.3, and a captioning average of 14.7–15.1 versus LTU's 14.5, with only 43–62M trainable parameters via LoRA. This is the first demonstration that a purely state-space LALM is competitive, suggesting quadratic attention is not necessary for audio-grounded question answering when the model is trained on the OpenAQA dataset.

Load-bearing premise

The comparison stands or falls on whether cosine similarity between the model's output text and the class-label text is a fair measure of audio reasoning, a protocol that could favor particular phrasings.

Editorial extensions

If this is right

  • State-space LALMs can be trained faster and with less memory: the paper reports batch size can rise from 4 to 16 on the same GPUs, cutting training time from roughly three days to under two.
  • Total parameter count falls from 6.8B (hybrid with LLaMA) to 2.8B (ssLALM) with no loss in average close-ended performance, implying a feasible path to on-device audio assistants.
  • Because the state-space LLM used here is not instruction-tuned, the competitive results suggest instruction tuning may be less critical when a strong audio encoder provides grounding.
  • Removing the quadratic attention bottleneck in both modules positions the same architecture for long-form audio inputs where transformer LALMs hit memory limits.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the cosine-similarity evaluation were replaced by direct label generation or human judgement, the ranking of ssLALM versus transformer LALMs could shift; the paper reports no such check.
  • The state-space LLM is 2.8B while LLaMA is 7B, so a matched-size comparison with a 7B state-space LLM would isolate the architecture effect from the scale effect — something the authors list as future work.
  • The competitive result may depend on label-phrase overlap between the OpenAQA training set and the cosine-similarity scoring protocol, which could inadvertently reward certain output phrasings.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes the first state-space-based large audio language model (LALM). The audio encoder is DASS, a distilled audio state-space model, and the language model is either a frozen LLaMA/Vicuna or a frozen state-space LLM (Mamba-2.8B), with LoRA adapters and a trainable downsampling projection. Models are trained on the OpenAQA dataset with next-token prediction. Evaluation covers eight audio classification benchmarks and two audio captioning benchmarks, reporting that the state-space models are competitive with transformer-based LALMs such as LTU while using fewer total parameters. The central claim is that both the audio perception and language generation components of a LALM can be replaced by state-space models without a major accuracy loss.

Significance. If validated, this result is useful: it demonstrates that state-space components can substitute for transformers in LALMs, with potential gains in memory and time efficiency for long audio and resource-constrained deployment. The paper is explicit about training details, LoRA configurations, and the evaluation protocol, and it provides a comparison table across 10 benchmarks. However, the validation is currently incomplete because the evaluation protocol is not fully controlled and the reported differences are small relative to the absence of error bars or significance tests. The strength of the central claim therefore depends on additional experiments that the authors can reasonably perform.

major comments (3)
  1. [§III.E, Table I] The classification protocol computes cosine similarity between a text-encoder embedding of the LALM output and embeddings of class labels, but the text encoder is never specified and no control is provided for output phrasing or length. The state-space LLM used in ssLALM is not instruction-tuned, while the transformer baselines (LTU, SALMONN, Pengi) use instruction-tuned LLaMA-7B; instruction-tuned models tend to produce longer, more conversational outputs, and a cosine-similarity scorer can systematically favor terse, label-like outputs. This confound could inflate the ssLALM results relative to the baselines. Please specify the text encoder, report results under an alternative scoring scheme (for example, exact-label matching or length-normalized embeddings), and, if possible, rerun the transformer baselines under the identical protocol with identical prompts and decoding settings.
  2. [Table I] No error bars, multiple-seed results, or significance tests are reported, and many of the headline differences are very small: the classification averages are 50.2 and 49.8 for the hybrid models, 50.0 and 51.4 for the ssLALMs, and 50.3 for LTU. Differences of this size cannot be distinguished from run-to-run or decoding stochasticity. Please provide variance estimates (for example, across at least three decoding runs or training seeds), or at least report per-item bootstrap intervals for the aggregate averages. This is load-bearing because the central claim is that the state-space models are 'competitive,' not merely that they are in the same broad range.
  3. [§III.E and Table I footnotes] For SALMONN, Pengi, and AudioGPT, the results are taken from the GAMA paper, and the footnotes indicate differences in evaluation conditions (for example, ZS- weak zero-shot, and a higher sampling rate for some methods). If the evaluation protocol, prompt template, text encoder, or decoding settings used to produce those numbers differ from those used for the proposed models, the comparison is not controlled. The authors should either rerun all baselines with the same evaluation pipeline or explicitly state which numbers were produced under which conditions and discuss the limitations. The mAP columns marked with the dagger are also interpreted with a post-hoc explanation that LALMs under-predict non-prominent classes; this is a reasonable hypothesis, but it should not be used to discount unfavorable multi-label results without supporting evidence.
minor comments (4)
  1. [Abstract and Section II] The abstract says 'space-based LALM' where 'state-space-based LALM' is intended, and Section II contains the typo 'wed-scale data' instead of 'web-scale data.'
  2. [Table I] The aggregate columns 'Classif. Avg.' and 'Cap. Avg.' appear to be simple arithmetic means over datasets of very different difficulty and metric types (accuracy, F1, mAP, SPICE). Please clarify the averaging procedure and consider reporting per-dataset results with a defined aggregation rule.
  3. [§III.E] The decoding settings (temperature=0.1, top-k=500, top-p=0.95, repetition penalty=1.1) are given for the proposed models, but it is not stated whether the same settings were used for all baselines in Table I. Please state this explicitly.
  4. [§III.C] For the state-space LLM, the LoRA adapters are said to be added to 'the input projection layers of the state-space block'; specifying the exact matrices (for example, the in-projection of the Mamba block) and the rank/alpha choices for both LLMs would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the central claim is an empirical comparison against external benchmarks using a frozen pretrained state-space audio encoder and independently reported baselines.

full rationale

The paper's central claim is that a state-space audio encoder (DASS) combined with a state-space LLM performs competitively with transformer-based LALMs. This is established by Table I, which reports accuracies and captioning metrics obtained by training only the projection and LoRA adapter parameters on the OpenAQA dataset and evaluating on external benchmarks (ESC-50, DCASE, VS, TUT, BJO, VGG, FSD, AudioSet, AudioCaps, Clotho). No equation in the paper defines these reported numbers in terms of the training objective or pretraining data in a way that would make the evaluation tautological. The only self-citations are: (i) DASS [19] is used as a frozen pretrained audio encoder; its weights are not tuned to the target tasks, and its contribution is externally measurable through the reported LALM performance, so the citation is not load-bearing in a circular sense. (ii) LTU [9] (overlapping authors) supplies the training pipeline and baseline numbers; those numbers are previously published external results, not quantities fitted in this paper. The cosine-similarity evaluation protocol (Section III.E) is shared across all compared models and is a methodological choice, not a definitional link between the paper's inputs and outputs. Therefore no prediction reduces to a fitted parameter or to a self-citation chain; any concern about the evaluation protocol being sensitive to output phrasing is a correctness or validity risk, not circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or conceptual entities. It relies on existing pretrained models (DASS, Mamba) and standard evaluation assumptions. The free parameters are conventional hyperparameters, not fitted to benchmark outcomes in a way that would bias the central comparison.

free parameters (3)
  • LoRA rank r = 8
    Chosen for both LLaMA and Mamba adapters; not tuned on the target benchmarks.
  • LoRA alpha = 16
    Scaling factor for LoRA adapters, set by hand.
  • Downsampling kernel and stride = 3x3 kernel, stride 2
    Used to reduce the DASS feature map from 32x4x768 to fit the LLM input dimension.
assumptions (4)
  • domain assumption DASS encoder pretrained on AudioSet-2M provides transferable audio representations when frozen.
    The architecture relies on DASS as a fixed feature extractor; this comes from the authors' prior work [19].
  • domain assumption State-space LLM (Mamba 2.8B) is a capable base for instruction following after LoRA tuning, despite not being instruction-tuned.
    Used as the frozen backbone for the state-space LALM; no instruction-tuned SSM was available.
  • domain assumption Cosine similarity between text-encoder embeddings of LALM output and class labels is a valid measure of classification accuracy.
    Evaluation protocol in Section III.E, following the LTU pipeline; if invalid, the comparison is unreliable.
  • domain assumption OpenAQA training data is sufficient to align audio and text for the evaluated tasks.
    Training data from LTU [9]; no ablation on data size or composition is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of State-Space Large Audio Language Models." pith.science (2026). https://pith.science/paper/5NKNT24Z

@misc{pith2026241115685,
  author       = {Pith},
  title        = {Pith review of: State-Space Large Audio Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5NKNT24Z}},
  note         = {Machine review of arXiv:2411.15685}
}
read the original abstract

Large Audio Language Models (LALM) combine the audio perception models and the Large Language Models (LLM) and show a remarkable ability to reason about the input audio, infer the meaning, and understand the intent. However, these systems rely on Transformers which scale quadratically with the input sequence lengths which poses computational challenges in deploying these systems in memory and time-constrained scenarios. Recently, the state-space models (SSMs) have emerged as an alternative to transformer networks. While there have been successful attempts to replace transformer-based audio perception models with state-space ones, state-space-based LALMs remain unexplored. First, we begin by replacing the transformer-based audio perception module and then replace the transformer-based LLM and propose the first state-space-based LALM. Experimental results demonstrate that space-based LALM despite having a significantly lower number of parameters performs competitively with transformer-based LALMs on close-ended tasks on a variety of datasets.

Figures

Figures reproduced from arXiv: 2411.15685 by the authors.

Figure 1
Figure 1. Overview of the proposed State-space Large Audio [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Towards Reliable Large Audio Language Model

    cs.SD 2025-05 conditional novelty 6.0 of 10

    Training a large audio language model to say 'I don't know' on one audio type (speech, music, or sound) makes it more likely to refuse uncertain questions on the other types.

  2. SAM: A Mamba-2 State-Space Audio-Language Model

    cs.SD 2025-09 conditional novelty 4.0 of 10

    A 2.7B-parameter Mamba-2 based audio captioning model (MAC) matches or surpasses larger transformer-based audio-language models on several zero-shot classification and captioning benchmarks, with additional design-spa...

Reference graph

Works this paper leans on

38 extracted references · 8 canonical work pages · cited by 2 Pith papers

  1. [1]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    Jacob Devlin, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805 , 2018

    arXiv 2018

  2. [2]

    Explor- ing the limits of transfer learning with a unified text-to-text transformer,

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu, “Explor- ing the limits of transfer learning with a unified text-to-text transformer,” Journal of machine learning research, vol. 21, no. 140, pp. 1–67, 2020

    2020

  3. [3]

    Language models are few-shot learners,

    Tom B Brown, “Language models are few-shot learners,” arXiv preprint arXiv:2005.14165, 2020

    arXiv 2005

  4. [4]

    Training language models to follow instructions with human feedback,

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wain- wright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al., “Training language models to follow instructions with human feedback,” Advances in neural information processing systems, vol. 35, pp. 27730–27744, 2022

    work page 2022

  5. [5]

    Opt: Open pre-trained transformer language models,

    Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al., “Opt: Open pre-trained transformer language models,” arXiv preprint arXiv:2205.01068, 2022

    arXiv 2022

  6. [6]

    A survey of large language models,

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al., “A survey of large language models,” arXiv preprint arXiv:2303.18223, 2023

    arXiv 2023

  7. [7]

    Pengi: An audio language model for audio tasks,

    Soham Deshmukh, Benjamin Elizalde, Rita Singh, and Huaming Wang, “Pengi: An audio language model for audio tasks,” Advances in Neural Information Processing Systems , vol. 36, pp. 18090–18108, 2023

    work page 2023

  8. [8]

    Salmonn: Towards generic hearing abilities for large language models,

    Changli Tang, Wenyi Yu, Guangzhi Sun, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun Ma, and Chao Zhang, “Salmonn: Towards generic hearing abilities for large language models,” arXiv preprint arXiv:2310.13289, 2023

    arXiv 2023

Show all 38 references
  1. [9]

    Listen, think, and understand,

    Yuan Gong, Hongyin Luo, Alexander H Liu, Leonid Karlinsky, and James Glass, “Listen, think, and understand,” arXiv preprint arXiv:2305.10790, 2023

    2023 arXiv

  2. [10]

    Joint audio and speech understanding,

    Yuan Gong, Alexander H Liu, Hongyin Luo, Leonid Karlinsky, and James Glass, “Joint audio and speech understanding,” in 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) . IEEE, 2023, pp. 1–8

    2023

  3. [11]

    Audiogpt: Understanding and generating speech, music, sound, and talking head,

    Rongjie Huang, Mingze Li, Dongchao Yang, Jiatong Shi, Xuankai Chang, Zhenhui Ye, Yuning Wu, Zhiqing Hong, Jiawei Huang, Jinglin Liu, et al., “Audiogpt: Understanding and generating speech, music, sound, and talking head,” in Proceedings of the AAAI Conference on Artificial Int...

    2024

  4. [12]

    Gama: A large audio-language model with advanced audio understanding and complex reasoning abilities,

    Sreyan Ghosh, Sonal Kumar, Ashish Seth, Chandra Kiran Reddy Evuru, Utkarsh Tyagi, S Sakshi, Oriol Nieto, Ramani Duraiswami, and Dinesh Manocha, “Gama: A large audio-language model with advanced audio understanding and complex reasoning abilities,” arXiv preprint arXiv:2406.11768, 2024

    2024 arXiv

  5. [13]

    Attention is all you need,

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin, “Attention is all you need,” Advances in neural information processing systems , vol. 30, 2017

    2017

  6. [14]

    Linformer: Self-attention with linear complexity,

    Sinong Wang, Belinda Z Li, Madian Khabsa, Han Fang, and Hao Ma, “Linformer: Self-attention with linear complexity,” arXiv preprint arXiv:2006.04768, 2020

    2006 arXiv

  7. [15]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in Proceedings of the IEEE/CVF international conference on computer vision , 2021, pp. 10012–10022

    2021

  8. [16]

    Efficiently model- ing long sequences with structured state spaces,

    Albert Gu, Karan Goel, and Christopher R ´e, “Efficiently model- ing long sequences with structured state spaces,” arXiv preprint arXiv:2111.00396, 2021

    2021 arXiv

  9. [17]

    Mamba: Linear-time sequence modeling with selective state spaces,

    Albert Gu and Tri Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” arXiv preprint arXiv:2312.00752 , 2023

    2023 arXiv

  10. [18]

    Vmamba: Visual state space model,

    Yue Liu, Yunjie Tian, Yuzhong Zhao, Hongtian Yu, Lingxi Xie, Yaowei Wang, Qixiang Ye, and Yunfan Liu, “Vmamba: Visual state space model,” arXiv preprint arXiv:2401.10166 , 2024

    2024 arXiv

  11. [19]

    Dass: Distilled audio state space models are stronger and more duration-scalable learners,

    Saurabhchand Bhati, Yuan Gong, Leonid Karlinsky, Hilde Kuehne, Rogerio Feris, and James Glass, “Dass: Distilled audio state space models are stronger and more duration-scalable learners,” arXiv preprint arXiv:2407.04082, 2024

    2024 arXiv

  12. [20]

    Long range language modeling via gated state spaces,

    Harsh Mehta, Ankit Gupta, Ashok Cutkosky, and Behnam Neyshabur, “Long range language modeling via gated state spaces,” arXiv preprint arXiv:2206.13947, 2022

    2022 arXiv

  13. [21]

    Vision mamba: Efficient visual represen- tation learning with bidirectional state space model,

    Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang, “Vision mamba: Efficient visual represen- tation learning with bidirectional state space model,” arXiv preprint arXiv:2401.09417, 2024

    2024 arXiv

  14. [22]

    Audio mamba: Bidirectional state space model for audio representation learning,

    Mehmet Hamza Erol, Arda Senocak, Jiu Feng, and Joon Son Chung, “Audio mamba: Bidirectional state space model for audio representation learning,” arXiv e-prints, pp. arXiv–2406, 2024

    2024

  15. [23]

    Audio mamba: Pretrained audio state space model for audio tagging,

    Jiaju Lin and Haoxuan Hu, “Audio mamba: Pretrained audio state space model for audio tagging,” arXiv preprint arXiv:2405.13636 , 2024

    2024 arXiv

  16. [24]

    Ssamba: Self-supervised audio representation learning with mamba state space model,

    Siavash Shams, Sukru Samet Dindar, Xilin Jiang, and Nima Mesgarani, “Ssamba: Self-supervised audio representation learning with mamba state space model,” arXiv preprint arXiv:2405.11831 , 2024

    2024 arXiv

  17. [25]

    Instruction tuning for large language models: A survey,

    Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, et al., “Instruction tuning for large language models: A survey,” arXiv preprint arXiv:2308.10792, 2023

    2023

  18. [26]

    Hts-at: A hierarchical token-semantic audio transformer for sound classification and detection,

    Ke Chen, Xingjian Du, Bilei Zhu, Zejun Ma, Taylor Berg-Kirkpatrick, and Shlomo Dubnov, “Hts-at: A hierarchical token-semantic audio transformer for sound classification and detection,” in ICASSP 2022- 2022 IEEE International Conference on Acoustics, Speech and Signal Processin...

    2022

  19. [27]

    Language models are unsupervised multitask learners,

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al., “Language models are unsupervised multitask learners,” OpenAI blog, vol. 1, no. 8, pp. 9, 2019

    2019

  20. [28]

    Robust speech recognition via large- scale weak supervision,

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever, “Robust speech recognition via large- scale weak supervision,” in International conference on machine learning. PMLR, 2023, pp. 28492–28518

    2023

  21. [29]

    Conv-tasnet: Surpassing ideal time– frequency magnitude masking for speech separation,

    Yi Luo and Nima Mesgarani, “Conv-tasnet: Surpassing ideal time– frequency magnitude masking for speech separation,” IEEE/ACM transactions on audio, speech, and language processing , vol. 27, no. 8, pp. 1256–1266, 2019

    2019

  22. [30]

    Beats: Audio pre-training with acoustic tokenizers,

    Sanyuan Chen, Yu Wu, Chengyi Wang, Shujie Liu, Daniel Tompkins, Zhuo Chen, and Furu Wei, “Beats: Audio pre-training with acoustic tokenizers,” arXiv preprint arXiv:2212.09058 , 2022

    2022 arXiv

  23. [31]

    Ast: Audio spectrogram transformer,

    Yuan Gong, Yu-An Chung, and James Glass, “Ast: Audio spectrogram transformer,” arXiv preprint arXiv:2104.01778 , 2021

    2021 arXiv

  24. [32]

    Au- dioclip: Extending clip to image, text and audio,

    Andrey Guzhov, Federico Raue, J ¨orn Hees, and Andreas Dengel, “Au- dioclip: Extending clip to image, text and audio,” in ICASSP 2022- 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 976–980

    2022

  25. [33]

    Large-scale contrastive language- audio pretraining with feature fusion and keyword-to-caption augmen- tation,

    Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Taylor Berg- Kirkpatrick, and Shlomo Dubnov, “Large-scale contrastive language- audio pretraining with feature fusion and keyword-to-caption augmen- tation,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech an...

    2023

  26. [34]

    Clap learning audio concepts from natural language supervision,

    Benjamin Elizalde, Soham Deshmukh, Mahmoud Al Ismail, and Huam- ing Wang, “Clap learning audio concepts from natural language supervision,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2023, pp. 1–5

    2023

  27. [35]

    Audio set: An ontology and human-labeled dataset for audio events,

    Jort F Gemmeke, Daniel PW Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, R Channing Moore, Manoj Plakal, and Marvin Ritter, “Audio set: An ontology and human-labeled dataset for audio events,” in 2017 IEEE international conference on acoustics, speech and signal processing...

    2017

  28. [36]

    Llama: Open and efficient foundation language models,

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie- Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi `ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al., “Llama: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971 , 2023

    2023 arXiv

  29. [37]

    Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality,

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al., “Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality,” See https://vicuna. lmsys. org (accessed 14 April 2023)...

    2023

  30. [38]

    Lora: Low-rank adaptation of large language models,

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen, “Lora: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.