Pith. sign in

REVIEW 4 major objections 6 minor 46 references

SAM: A Mamba-2 State-Space Audio-Language Model

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A 2.7B-parameter Mamba-2 audio-language model matches or beats larger 7B transformer models on audio understanding benchmarks.

desk verdict Worth a serious look, but the flagship claim is undercut by an underspecified CLAP evaluation and an abstract that does not match the paper's own tables. read the letter →

arxiv 2509.15680 v3 pith:WHBFWEQ5 submitted 2025-09-19 cs.SD eess.AS

Taehan Lee , Jaehan Jung , Hyukjun Lee This is my paper · ORCID
classification cs.SDeess.AS
keywords audio-languagemodelstate-spaceMamba-2audiocaptioningzero-shotclassificationmultimodalLLMrepresentationanalysisLoRA
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a Mamba-2 state-space backbone can power an audio-language model that matches or beats much larger transformer-based models on zero-shot audio classification and audio captioning, at 2.7B parameters versus 7B baselines. Its evidence comes from a systematic design-space exploration: jointly finetuning the audio encoder, raising the LoRA rank, and using compact audio token representations. The paper also claims that despite SSMs' linear-time scaling, they profit more from short, information-dense audio token sequences than from long uncompressed ones, and that instruction-following supervision substantially improves reasoning, raising MMAU-Sound accuracy from 22.8 to 56.8. If right, this would make state-space models a practical, cheaper alternative for audio-language systems.

What carries the argument

The load-bearing object is the Mamba-2 backbone, a state-space model that processes sequences in linear time via a selective scan implemented through structured state-space duality. The argument runs on three connected mechanisms: the EAT audio encoder and whether it is finetuned; the connector, which either compresses 512 tokens to 64 high-dimensional tokens by concatenation or keeps all tokens in time-major or frequency-major layouts; and two representation-level diagnostics—effective rank and pairwise cosine similarity of audio tokens—used to explain why larger models and shorter token sequences work better.

What would settle it

Re-run the zero-shot classification benchmarks with a standard protocol—for example, a linear probe on EAT features, or top-1 matching of class-label CLAP embeddings to the audio—and check whether MAC-2.7B still ranks above LTU-7B, GAMA-7B, and ssLALM-2.8B. If the ranking changes, the central parity claim fails. Alternatively, evaluate the same model on a long-form audio QA benchmark with minutes-long clips to test the linear-time scaling benefit.

Watch

Extended reading notes

Core claim

The central claim is that a 2.7B-parameter Mamba-2 language model, paired with an EAT audio encoder and a two-layer MLP connector, performs on par with or better than 7B transformer-based audio-language models such as LTU-7B and GAMA-7B across zero-shot audio classification and captioning benchmarks. The supporting analysis contends that joint finetuning of the audio encoder is essential—freezing it costs several accuracy points—and that initializing the encoder from class-level supervised AudioSet finetuning transfers better than masked-modeling-only pretraining. The paper further claims that for SSMs, compressing 512 audio tokens into 64 concatenated tokens outperforms feeding all 520–576 tokens, because the recurrent state must carry information across each token update; this pressure shows up as higher token-level cosine similarity in smaller models. Finally, adding instruction-following supervision is reported to improve sound-reasoning accuracy from 22.8 to 56.8 on MMAU-Sound.

Load-bearing premise

The evaluation of zero-shot classification benchmarks relies on the CLAP encoder measuring similarity between generated captions and ground-truth labels, and if that protocol does not reproduce standard classifier accuracy, the headline comparison against the 7B transformer baselines is not established.

Editorial extensions

If this is right

  • A 2.7B Mamba-2 audio-language model can match 7B transformer baselines on zero-shot classification and captioning, making state-space backbones a viable scaling path with fewer parameters.
  • Jointly finetuning the audio encoder is required; freezing it degrades performance across model sizes and connector types.
  • For SSM audio-language models, compact audio tokens (64) beat full-length 520–576 token sequences, so linear-time complexity does not make long token sequences free.
  • Raising the LoRA rank from 8 to 256 substantially improves accuracy, particularly for the smallest 130M model.
  • Adding instruction-following supervision can sharply improve sound-reasoning ability, as measured by MMAU-Sound.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the linear-time advantage carries over to longer audio, the same recipe should extend to minute- or hour-long recordings where transformer attention becomes impractical; this is the natural next stress test.
  • The evidence that full-length tokens hurt suggests that a learned compression connector (e.g., token merging or a query-based bottleneck) might push SSM audio-language models further by combining compression with semantic selection.
  • Because the classification evaluation relies on CLAP similarity between generated captions and class labels, the parity claim against 7B transformers should be re-checked with a standard classifier head; the authors' own protocol is the main unvalidated link.
  • The representation-level findings (effective rank growth with model size, cosine-similarity pressure with sequence length) likely transfer to other SSM-based multimodal models, predicting that smaller SSMs will compress visual tokens in the same way.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces MAC, a Mamba-2-based audio-language model (ALM) that combines an EAT audio encoder with Mamba-2 language models of sizes 130M, 780M, and 2.7B, using an MLP connector and LoRA finetuning. The authors report zero-shot classification results on eight audio benchmarks and captioning results on AudioCaps and Clotho, claiming that MAC-2.7B matches or surpasses larger transformer-based ALMs such as LTU-7B and GAMA-7B. They also present ablations on whether to finetune the audio encoder, how to initialize it, connector designs that vary token length and dimensionality, and analyses of effective rank and token similarity as functions of model size.

Significance. If the reported results are valid, this is a useful contribution: it is among the first studies of Mamba-2 as an ALM backbone, and the ablations on audio-encoder finetuning, initialization, and connector design provide practical guidance for building SSM-based audio-language models. The eRank and token-similarity analyses are a thoughtful attempt to understand why SSM backbones behave differently from transformers. The paper also reports consistent gains from increasing LoRA rank and model size, and it compares against strong baselines. However, the central empirical claim is currently supported only by an underspecified CLAP-similarity evaluation protocol and by comparisons to baselines whose training data and budgets are not matched; the abstract also reports numbers that do not appear in Table 1. These issues must be resolved before the headline conclusions can be accepted.

major comments (4)
  1. [Section 3, Evaluation method and Table 1] The classification evaluation is defined by a single sentence: 'we used the CLAP encoder to measure the similarity between the generated captions and the ground-truth labels.' This is insufficient to support the classification columns of Table 1. The paper must specify how class labels are converted to text (e.g., prompt templates, whether labels are verbatim), how per-clip similarities are aggregated into accuracy or mAP, and how multi-label datasets such as AudioSet (527 classes) are handled—mAP requires per-class scores and a ranking or threshold rule, not one caption-to-label similarity. It is also unclear whether the baseline numbers for LTU-7B, GAMA-7B, and ssLALM were computed with the same CLAP-based protocol or were taken from papers that used a different evaluation, which would make the comparisons inequitable.
  2. [Section 3, Model training and Table 1] The claimed superiority over larger transformer ALMs is not yet established because training conditions are not matched. The text says the models are trained on OpenAQA with the LTU curriculum, batch size 256, and two learning rates, but the baseline models were trained on datasets that 'include the OpenAQA dataset'—different data mixtures, numbers of steps, and hardware budgets. Without a controlled comparison (same data, same number of updates, or at least a clear statement of what differs), the gains in Table 1 could be due to data or compute rather than to Mamba-2. The authors should either provide matched-training comparisons or soften the 'superior performance' claim accordingly.
  3. [Abstract and title versus Section 3/Table 1] There is a direct inconsistency between the abstract/title and the body. The abstract presents 'SAM-2.7B' as achieving 21.1 mAP on AudioSet and 17.6 SPICE on AudioCaps, and the title announces 'SAM: A Mamba-2 State-Space Audio-Language Model.' The body, however, names the model MAC and reports in Table 1 (E6) 19.5 mAP on AudioSet and 16.8 SPICE on AudioCaps. No row in Table 1 matches the abstract's numbers. The authors must reconcile the naming and verify that all reported numbers correspond to the same configuration, because as written the reader cannot tell which numbers are authoritative.
  4. [Section 4, eRank analysis] The representation-level analysis is presented as evidence for the claim that larger LLMs produce richer audio representations, but the link to the performance numbers is descriptive rather than quantitative. Table 3 reports eRank values for a few configurations and states that larger LLMs increase eRank, but no correlation or regression is given, and the eRank values for MAC-780M (19.45/19.83/18.80) and MAC-2.7B (19.78) are close, while their classification averages differ substantially (50.6 vs. 53.9). The authors should either qualify the claim as a hypothesis or provide a more direct analysis. Additionally, Equation (5), which defines the covariance matrix used for eRank, is garbled in the manuscript and should be rewritten.
minor comments (6)
  1. [Equation (5)] Equation (5) is typeset incorrectly: the covariance expression is missing parentheses and the outer-product structure is unclear. It should be written as a standard sample covariance matrix (1/N) Σ (h_i − h̄)(h_i − h̄)ᵀ.
  2. [Table 1, footnote symbols] The table uses dagger symbols (†) for FSD and AudioSet, but the footnote text defining 'weak zero-shot' (ZS-) does not explain the daggers; the distinction between ZS- and ZS needs a clear definition and a list of which datasets fall into each category.
  3. [Section 3, Evaluation method] The 'Classif. Avg.' column in Table 1 is not defined; the paper should state whether it is a simple mean, a weighted mean, or another aggregation of the eight classification metrics.
  4. [Section 4, Token similarity] In Table 4, the Audio-Flamingo 2 (AF-2) rows report model sizes 500M, 1.5B, and 3B, but the text does not specify how the AF-2 token-similarity values were obtained—whether from public checkpoints, from the authors' own runs, or from the original paper—nor how many audio clips were used for the average.
  5. [Section 4, Figure 2] The text references 'Figure 2' for the state update distance comparison, but the figure content is not included in the manuscript text supplied for review.
  6. [Section 3, Model training] The paper should cite a public source or give a fuller description of the OpenAQA dataset, since it is central to the training setup and the comparisons.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's claims rest on empirical measurements, not on fitted parameters or self-citation chains.

full rationale

The paper is an empirical design-space study. It does not claim to derive a prediction from first principles, nor does it fit a model parameter to the target benchmarks and then rename that fit as a prediction. The central comparison in Table 1 is produced by training MAC models on OpenAQA and evaluating them zero-shot with an underspecified CLAP-similarity protocol, which is a measurement-validity concern rather than a circularity concern. The abstract's SAM-2.7B numbers (21.1 mAP AudioSet, 17.6 SPICE AudioCaps) do not match any row in Table 1, which is an internal-consistency problem, but inconsistency is not identity or reduction to inputs. The representation-level analyses (eRank, token-wise cosine similarity) are descriptive and do not enter the main performance comparison. No uniqueness theorem is invoked, and no load-bearing argument reduces to a self-citation. Therefore, under the requirement that circularity be demonstrated by quoting a specific reduction, no circular step is found.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new theoretical entities; the main assumptions are about the validity of the evaluation method and the comparability of baselines. Hyperparameters such as LoRA rank and connector choice are selected empirically and are listed as free parameters.

free parameters (3)
  • LoRA rank r = 256 (vs 8)
    The paper found r=256 substantially improves performance; the rank is a hand-chosen hyperparameter that the headline results rely on.
  • Connector design = (a) concatenation and compression
    Best performance selected across (a), (b), (c); the choice is empirical and made after seeing results.
  • Learning rates = 1e-3 and 1e-4
    Selected from prior LTU work; not justified beyond that.
assumptions (3)
  • domain assumption CLAP cosine similarity between generated captions and class labels is a valid zero-shot classification score
    The entire classification evaluation rests on this premise; no evidence or prior validation is cited for this specific protocol.
  • domain assumption Baselines trained on datasets that include OpenAQA are directly comparable
    Training sets may differ in composition; the paper does not control for exact data or training budget.
  • domain assumption EAT features finetuned on AudioSet transfer to captioning
    Mostly validated by ablations, but the initial choice relies on prior claims.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SAM: A Mamba-2 State-Space Audio-Language Model." pith.science (2026). https://pith.science/paper/WHBFWEQ5

@misc{pith2026250915680,
  author       = {Pith},
  title        = {Pith review of: SAM: A Mamba-2 State-Space Audio-Language Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WHBFWEQ5}},
  note         = {Machine review of arXiv:2509.15680}
}
read the original abstract

We present SAM, a State-space Audio-language Model that integrates an audio encoder with a Mamba-2 backbone. SAM-2.7B achieves 21.1 mAP on AudioSet and 17.6 SPICE on AudioCaps, matching or surpassing larger 7B transformer-based models with fewer parameters. We further provide the first systematic, representation-level analysis of how SSMs interact with audio encoder outputs: (1) joint audio encoder finetuning is essential, supported by accuracy gains and observed adaptation of token representation rank and similarity across different SSM sizes; (2) despite linear scaling, SSMs benefit more from compact, information-rich audio token representations than from excessively long token sequences; and (3) incorporating instruction-following supervision substantially improves reasoning ability, boosting MMAU-Sound accuracy from 22.8 to 56.8. Through comprehensive experiments and analysis, we establish practical design principles for SSMs as strong, scalable backbones for audio-language models.

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 43 canonical work pages

  1. [1]

    Prior works have improved their ability in various ways, including large-scale QA datasets, curriculum- learning strategy, and advanced connector architectures

    INTRODUCTION In recent years, audio language models (ALMs) that combine Transformer [1] based language models [2, 3, 4] with audio encoders [5, 6, 7, 8, 9, 10] have achieved notable results in au- dio reasoning tasks. Prior works have improved their ability in various ways, including large-scale QA datasets, curriculum- learning strategy, and advanced con...

    work page

  2. [2]

    SAM: A Mamba-2 State-Space Audio-Language Model

    MODEL ARCHITECTURE Our model follows a standard multimodal LLM design (Fig. 1), comprising an audio encoderfa, a text encoderft, a connector fc, and a Mamba-2 LLM. The audio encoder yields audio tokensHa =f a(x) from mel-spectrogramx, which are then mapped by the connector to the audio embeddings Ea = fc(Ha). The text encoder maps the prompt p and caption...

    work page Pith review arXiv 2025

  3. [3]

    Write an audio caption describing the sound

    EXPERIMENT Model training.We trained our model on the OpenAQA dataset and adopted the LTU curriculum learning strategy [5] with the same batch size of 256 and learning rates of 10−3 and 10−4. We applied LoRA [ 23] adapters toin projand out projlayers in Mamba-2 blocks with α= 2r , based on its effectiveness over alternatives (e.g., prefix-tuning) [24, 25]...

    work page

  4. [4]

    ABLA TION STUDIES Trainable audio encoder: Stability vs. Plasticity (Ablation I).Some multimodal LLM studies freeze the vision/audio 2As music understanding benchmarks (e.g., GTZAN) are not included, GAMA’s known strength on those tasks is not fully reflected in Table 1. encoder [7, 13, 15], motivated not only by reducing computa- tional cost but also by ...

    work page

  5. [5]

    For future work, we plan to investigate the effects of advanced connector designs that enable token mixing and Mamba-Transformer hybrid architectures in audio language modeling

    CONCLUSION In this work, we analyze different designs of Mamba-2-based audio-language models. For future work, we plan to investigate the effects of advanced connector designs that enable token mixing and Mamba-Transformer hybrid architectures in audio language modeling

    work page

  6. [6]

    GAMA: A large audio-language model with advanced audio understanding and complex reasoning abilities,

    Sreyan Ghosh et al., “GAMA: A large audio-language model with advanced audio understanding and complex reasoning abilities,” inProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. Nov. 2024, pp. 6288– 6313, Association for Computational Linguistics

    work page 2024

  7. [7]

    Attention is all you need,

    Ashish Vaswani et al., “Attention is all you need,” inAdvances in Neural Information Processing Systems, 2017, vol. 30

    2017

  8. [8]

    Llama 2: Open foundation and fine-tuned chat models,

    Hugo Touvron et al., “Llama 2: Open foundation and fine-tuned chat models,” 2023

    work page 2023

Show all 46 references
  1. [9]

    Opt-iml: Scaling language model instruc- tion meta learning through the lens of generalization,

    Srinivasan Iyer et al., “Opt-iml: Scaling language model instruc- tion meta learning through the lens of generalization,” 2023

    2023

  2. [10]

    Qwen2.5 technical report,

    An Yang et al., “Qwen2.5 technical report,” 2025

    2025

  3. [11]

    Listen, think, and understand,

    Yuan Gong et al., “Listen, think, and understand,” inThe Twelfth International Conference on Learning Representations, 2024

    2024

  4. [12]

    Transformers are ssms: generalized models and efficient algorithms through structured state space duality,

    Tri Dao et al., “Transformers are ssms: generalized models and efficient algorithms through structured state space duality,” in Proceedings of the 41st International Conference on Machine Learning. 2024, ICML’24, JMLR.org

    2024

  5. [13]

    SALMONN: Towards generic hearing abil- ities for large language models,

    Changli Tang et al., “SALMONN: Towards generic hearing abil- ities for large language models,” inThe Twelfth International Conference on Learning Representations, 2024

    2024

  6. [14]

    Audio flamingo 2: An audio-language model with long-audio understanding and expert reasoning abil- ities,

    Sreyan Ghosh et al., “Audio flamingo 2: An audio-language model with long-audio understanding and expert reasoning abil- ities,” inForty-second International Conference on Machine Learning, 2025

    2025

  7. [15]

    Qwen2-audio technical report,

    Yunfei Chu et al., “Qwen2-audio technical report,” 2024

    2024

  8. [16]

    Mellow: a small audio language model for reasoning,

    Soham Deshmukh et al., “Mellow: a small audio language model for reasoning,” 2025

    2025

  9. [17]

    Mamba: Linear-time sequence modeling with selective state spaces,

    Albert Gu et al., “Mamba: Linear-time sequence modeling with selective state spaces,”arXiv preprint arXiv:2312.00752, 2023

    2023 arXiv

  10. [18]

    Eat: Self-supervised pre-training with efficient audio transformer,

    Wenxi Chen et al., “Eat: Self-supervised pre-training with efficient audio transformer,” inProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI- 24, 8 2024, pp. 3807–3815

    2024

  11. [19]

    Ml-mamba: Efficient multi-modal large language model utilizing mamba-2,

    Wenjun Huang et al., “Ml-mamba: Efficient multi-modal large language model utilizing mamba-2,” 2024

    2024

  12. [20]

    VL-Mamba: Exploring state space models for multimodal learning,

    Yanyuan Qiao et al., “VL-Mamba: Exploring state space models for multimodal learning,” inProceedings of The 4th NeurIPS Efficient Natural Language and Speech Processing Workshop. 14 Dec 2024, vol. 262 ofProceedings of Machine Learning Research, pp. 102–113, PMLR

    2024

  13. [21]

    Shaking up VLMs: Comparing transformers and structured state space models for vision & language modeling,

    Georgios Pantazopoulos et al., “Shaking up VLMs: Comparing transformers and structured state space models for vision & language modeling,” inProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. Nov. 2024, pp. 14318–14337, Association for Comp...

    2024

  14. [22]

    State-space large audio language models,

    Saurabhchand Bhati et al., “State-space large audio language models,”arXiv preprint arXiv:2411.15685, 2024

    2024 arXiv

  15. [23]

    On the parameterization and initialization of diagonal state space models,

    Albert Gu et al., “On the parameterization and initialization of diagonal state space models,” inProceedings of the 36th International Conference on Neural Information Processing Systems, 2022, NIPS ’22’

    2022

  16. [24]

    MambaPEFT: Exploring parameter-efficient fine-tuning for mamba,

    Masakazu Yoshimura et al., “MambaPEFT: Exploring parameter-efficient fine-tuning for mamba,” inThe Thirteenth International Conference on Learning Representations, 2025

    2025

  17. [25]

    Audio set: An ontology and human- labeled dataset for audio events,

    Gemmeke Jort F et al., “Audio set: An ontology and human- labeled dataset for audio events,” inIEEE International Con- ference on Acoustics, Speech and Signal Processing (ICASSP), 2017, pp. 776–780

    2017

  18. [26]

    Slam-aac: Enhancing audio captioning with paraphrasing augmentation and clap-refine through llms,

    Wenxi Chen et al., “Slam-aac: Enhancing audio captioning with paraphrasing augmentation and clap-refine through llms,” inIEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2025, pp. 1–5

    2025

  19. [27]

    Sjtu-thu automated audio captioning system for dcase 2024,

    Wenxi Chen et al., “Sjtu-thu automated audio captioning system for dcase 2024,” Tech. Rep. 2, DCASE2024 Challenge, May 2024

    2024

  20. [28]

    VMamba: Visual state space model,

    Yue Liu et al., “VMamba: Visual state space model,” in The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  21. [29]

    LoRA: Low-rank adaptation of large language models,

    Edward J Hu et al., “LoRA: Low-rank adaptation of large language models,” inInternational Conference on Learning Representations, 2022

    2022

  22. [30]

    Vggsound: A large-scale audio-visual dataset,

    Honglie Chen et al., “Vggsound: A large-scale audio-visual dataset,” inIEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020, pp. 721–725

    2020

  23. [31]

    Parameter-efficient fine-tuning of state space models,

    Kevin Galim et al., “Parameter-efficient fine-tuning of state space models,” inForty-second International Conference on Machine Learning, 2025

    2025

  24. [32]

    Flashattention-2: Faster attention with better par- allelism and work partitioning,

    Tri Dao, “Flashattention-2: Faster attention with better par- allelism and work partitioning,” inThe Twelfth International Conference on Learning Representations, 2024

    2024

  25. [33]

    Esc: Dataset for environmental sound classifi- cation,

    Karol J. Piczak, “Esc: Dataset for environmental sound classifi- cation,” 2015

    2015

  26. [34]

    Sound event detection of weakly labelled data with cnn-transformer and automatic threshold optimiza- tion,

    Qiuqiang Kong et al., “Sound event detection of weakly labelled data with cnn-transformer and automatic threshold optimiza- tion,”IEEE/ACM Trans. Audio, Speech and Lang. Proc., vol. 28, pp. 2450–2460, Aug. 2020

    2020

  27. [35]

    V ocalsound: A dataset for improving human vocal sounds recognition,

    Yuan Gong et al., “V ocalsound: A dataset for improving human vocal sounds recognition,” inIEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022, pp. 151–155

    2022

  28. [36]

    BLIP-2: Bootstrapping language-image pre- training with frozen image encoders and large language models,

    Junnan Li et al., “BLIP-2: Bootstrapping language-image pre- training with frozen image encoders and large language models,” inProceedings of the 40th International Conference on Machine Learning. 2023, vol. 202, pp. 19730–19742, PMLR

    2023

  29. [37]

    Fsd50k: An open dataset of human-labeled sound events,

    Fonseca et al., “Fsd50k: An open dataset of human-labeled sound events,”IEEE/ACM Trans. Audio, Speech and Lang. Proc., vol. 30, pp. 829–852, Dec. 2021

    2021

  30. [38]

    Spice: Semantic propositional image caption evaluation,

    Anderson et al., “Spice: Semantic propositional image caption evaluation,” inComputer Vision – ECCV 2016, 2016, pp. 382– 398

    2016

  31. [39]

    AudioCaps: Generating captions for audios in the wild,

    Chris Dongjoo Kim et al., “AudioCaps: Generating captions for audios in the wild,” inProceedings of the 2019 Conference of the North American Chapter of the Association for Compu- tational Linguistics: Human Language Technologies, Volume

    2019

  32. [40]

    119–132, Association for Computational Lin- guistics

    June 2019, pp. 119–132, Association for Computational Lin- guistics

    2019

  33. [41]

    Clotho: an audio captioning dataset,

    Drossos et al., “Clotho: an audio captioning dataset,” inIEEE International Conference on Acoustics, Speech and Signal Pro- cessing (ICASSP), 2020, pp. 736–740

    2020

  34. [42]

    Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmen- tation,

    Yusong Wu et al., “Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmen- tation,” inIEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP, 2023

    2023

  35. [44]

    The effective rank: A measure of effective dimensionality,

    Olivier Roy et al., “The effective rank: A measure of effective dimensionality,” in2007 15th European Signal Processing Conference, 2007, pp. 606–610

    2007

  36. [45]

    Open-world objectness modeling unifies novel object detection,

    Shan Zhang et al., “Open-world objectness modeling unifies novel object detection,” inProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR), June 2025, pp. 30332–30342

    2025

  37. [46]

    Diff-erank: A novel rank-based metric for evaluating large language models,

    Lai Wei et al., “Diff-erank: A novel rank-based metric for evaluating large language models,” inAdvances in Neural In- formation Processing Systems, 2024, vol. 37, pp. 39501–39521

    2024

  38. [6144]

    Prior works on ALMs of- ten use mean pooling to reduce the length of audio tokens, mitigating the quadratic computational cost of self-attention

    Finally, this sequence is fed into the MLP, resulting in a final length of |Ea|= 64 . Prior works on ALMs of- ten use mean pooling to reduce the length of audio tokens, mitigating the quadratic computational cost of self-attention. By contrast, Mamba requires O(TN 2) training ...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.