Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

AudioLens: A Closer Look at Auditory Attribute Perception of Large Audio-Language Models

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read In failing audio-language models, auditory attribute information peaks mid-network and then degrades before the final prediction.

desk verdict First real internal look at auditory attribute perception in LALMs, with a plausible but partly artifact-prone story and a preliminary improvement; deserves referee time with requested revisions. read the letter →

arxiv 2506.05140 v2 pith:EJLCXXSQ submitted 2025-06-05 cs.CL cs.AIcs.SDeess.AS

Chih-Kai Yang , Neo Ho , Yi-Jyun Lee , Hung-yi Lee This is my paper
classification cs.CLcs.AIcs.SDeess.AS
keywords largeaudio-languagemodelsinterpretabilityLogitLensauditoryattributeperceptionlayer-wiseinformationtoken-levelanalysismulti-hopreasoningtraining-freemodelimprovement
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show how large audio-language models (LALMs) represent and resolve auditory attributes such as speaker gender, spoken language, emotion, and animal identity. Projecting intermediate hidden states through the model's unembedding matrix, the paper tracks the probability assigned to the correct attribute at every layer and token position in three LALMs. It finds that attribute information is not monotonically accumulated: for correctly predicted samples it rises with depth, while for wrongly predicted samples it peaks in the middle layers and then degrades, and earlier resolution of the attribute predicts higher accuracy. The paper also finds that the models rely on directly querying the audio input at prediction time rather than on information aggregated at the token that names the attribute. On this basis, it injects an earlier information-rich layer's representation into a layer five steps deeper and improves animal recognition accuracy in one model by 16.3% relative without any training.

What carries the argument

The machinery is the Logit Lens readout: at each layer $\ell$ and token position $i$, the hidden state $h^\ell_i$ is projected through the unembedding matrix $W_U$ and softmax to give a vocabulary distribution $p^\ell_i = \mathrm{softmax}(W_U h^\ell_i)$, whose probability on the attribute label measures how much attribute information that layer carries. The layer-wise information score $I^\ell_i$ is the average indicator that the argmax over attribute labels equals the ground truth, and the critical layer is the contribution-weighted average of layer indices. This readout is what turns hidden representations into a per-layer accuracy curve, letting the paper locate where attributes are resolved and where information is lost.

What would settle it

Train a linear probe on the same hidden states at every layer and compare its attribute accuracy with the Logit Lens scores: if the probe accuracy rises monotonically with depth even on samples the model predicts wrongly, the reported mid-network peak and decay is an artifact of the readout, not a property of the information in the residual stream.

Watch

Extended reading notes

Core claim

The central discovery is that auditory attribute information in LALMs follows two opposite trajectories: it accumulates with depth when recognition will succeed, but for failures it peaks mid-network and is then degraded by later layers. The paper's layer-wise information score, defined as whether the ground-truth attribute has the highest probability after projecting a hidden state through the unembedding matrix, shows that the layer at which the attribute is resolved correlates negatively with accuracy. A token-position analysis adds that the attribute-mentioning token does not hold enough information for prediction; when the audio inputs are masked during self-attention at the final token, information scores drop sharply, implying the model queries the audio directly. The paper ties this to the known weakness of LALMs in multi-hop reasoning, since consolidation of relevant information at mention positions is not happening. Finally, the paper demonstrates a training-free intervention: adding a scaled earlier-layer representation to a five-layers-deeper representation improves DeSTA2's animal recognition accuracy by 16.3% relative.

Load-bearing premise

The load-bearing premise is that projecting a layer's hidden state through the unembedding matrix gives a faithful readout of the attribute information that layer actually encodes; if that projection is misleading for audio-language models, the layer scores, critical layers, and the enrichment method built on them lose their foundation.

Editorial extensions

If this is right

  • Recognition failures in LALMs are not simply missing information; later layers actively degrade attribute information that earlier layers had encoded.
  • Earlier attribute resolution is predictive of accuracy, so techniques that consolidate attribute information in shallower layers should help downstream reasoning.
  • Attribute-mentioning positions do not store the information needed for prediction, which offers a mechanistic explanation for poor multi-hop reasoning in LALMs.
  • Training-free enrichment of deeper representations with selected earlier layers can improve attribute accuracy, so interpretability-guided interventions can serve as a cheap alternative to retraining.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the Logit Lens readout is trustworthy, the same layer-wise curves could be used as an online monitor that flags when a model is about to misrecognize an attribute, since failures are preceded by a mid-network peak followed by decay.
  • The enrichment recipe may transfer to other modalities, such as vision-language models, wherever attribute information is resolved early and then degraded; this is a testable extension rather than a result the paper reports.
  • A stronger test of the causal story would be to train or fine-tune a model to deposit attribute information at the mention position and check whether multi-hop reasoning improves, something the paper does not attempt.
  • The reported 16.3% improvement is on one model and one attribute; whether the intervention generalizes across attributes and models would indicate how universal the decreasing dynamic actually is.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper applies the Logit Lens (Eq. 1) to intermediate hidden states of three large audio-language models (DeSTA2, Qwen-Audio-Chat, Qwen2-Audio-Instruct) and defines a layer-wise information score I^ℓ_i (Eq. 2) over four auditory attributes and three prompt formats. It reports that attribute information does not accumulate monotonically with depth; that for incorrectly predicted samples the information peaks mid-network and then declines; that earlier critical layers correlate with higher accuracy; that LALMs rely on directly querying audio inputs rather than aggregating attribute information at attribute-mentioning token positions; and that adding an earlier information-rich layer's representation five layers deeper improves DeSTA2 animal recognition by 16.3% relative at λ = 1.25 without training. The central contribution is an internal-mechanism analysis of LALMs, with a training-free application derived from the analysis.

Significance. If the Logit Lens readout is reliable, this is the first detailed layer-wise study of how LALMs encode and resolve auditory attributes, and the enrichment experiment provides a concrete, training-free demonstration that the analysis can inform model improvement. The paper's strengths are its systematic coverage of three models, four attributes, and three prompt formats; the clear formal definitions of the information score and critical layer; the explicit comparison of token positions with and without audio masking; and a falsifiable enrichment result. However, the central interpretive claims currently rest on an unvalidated readout, and several of the quantitative conclusions are weakened by definitional or statistical issues. The significance is therefore contingent on additional validation, but the direction is valuable and worth pursuing.

major comments (4)
  1. [§IV-A, Eq. (1)] The vocabulary projection in Eq. (1) is used as a faithful readout of every intermediate layer, but no validation is provided for LALMs. These models inject tokens from a separately pretrained audio encoder into the residual stream, and the LM head in DeSTA2/Qwen normally operates on a final normalized representation; projecting raw h^ℓ_i through W_U omits that normalization, so even the claim in §IV-A that p^L_i equals the next-token distribution may fail at face value. All downstream quantities (I^ℓ_i, critical layers, the token-position comparisons, and the enriched-layer choice in §VI-E) are deterministic functions of this readout. I request a sanity check: for example, compare Eq. (1) against a tuned-lens affine map or a linear probe trained on held-out attributes, and verify whether the curves in Figs. 2–4 are stable after applying the final RMSNorm/LayerNorm before W_U. The Limitations section mentions alternative Logit Lens variants but does not report such a check.
  2. [§VI-B, Eq. (2), Fig. 3] The correct/incorrect split is defined by the final-layer argmax at the last token, so I^L_{-1} is 1 for every correct sample and 0 for every incorrect sample by construction. The final-layer gap between the green and red curves in Fig. 3 is therefore a definitional artifact, not evidence that deeper layers degrade information for wrong predictions. The peak-and-decline shape before the final layer is empirical and worth reporting, but the two-curve comparison as presented overstates the degradation claim. Please re-run the analysis with a split that does not use the same readout at the same position (for example, a held-out or ground-truth-based split), or explicitly exclude the final layer and show that the decline is not driven by the definitional endpoint.
  3. [§VI-C, Table III] The correlation analysis uses only 12 points per model (3 prompts × 4 attributes). The correlations are significant only for DeSTA2; for Qwen and Qwen2 the full-data p-values are 0.183 and 0.106. The "Excluding Gender" rows are post-hoc exclusions motivated by the same data, and while they become significant, this does not license the general claim that earlier resolution yields higher accuracy. Report a preregistered or combined analysis (for example, a mixed-effects model using all model/prompt/attribute points) or add more independent attribute/prompt conditions; if the gender exclusion is principled, justify it independently of Table III.
  4. [§VI-E, Fig. 5] The enrichment experiment selects the scaling factor λ by sweeping over the 400-sample test set and reports the best λ = 1.25 as the 16.3% improvement. This is test-set selection; the 100-sample probing set is used only to select the layer l̄, not λ. Please choose λ on the probing/validation set and report test accuracy at that fixed λ, and state whether the random-layer baseline's λ was selected under the same protocol. As written, the improvement claim is inflated and would not generalize.
minor comments (5)
  1. [§IV-B, Eq. (2)] The definition of p^ℓ_i(y) for attribute labels that may be multi-token (for example, "rooster") is not specified; if only the first token's probability is used, this could bias the information score. Please clarify the tokenization handling for all attribute labels.
  2. [Figs. 2–4] The layer-wise information scores are reported without confidence intervals or error bars; with 500 samples per attribute, bootstrap intervals would help assess whether the observed peaks and drops are meaningful rather than noise.
  3. [§IV-C and §VI-E] The threshold α = 0.2 in Eq. (3) and the five-layer gap in Eq. (5) are selected ad hoc; a short sensitivity analysis would increase confidence in the critical-layer estimates and the enrichment result.
  4. [Fig. 3 axis labels] The axis labels such as "I 1 (Correct Predictions)" appear to have lost their superscript and subscript formatting; please fix the typography so the figure matches Eq. (2).
  5. [§VI-A, Fig. 2(d)] The animal track for DeSTA2 shows layers below the random baseline, which the paper attributes to limited training on animal sounds; please support this explanation with evidence about the model's training data or an auxiliary analysis, rather than leaving it as a conjecture.

Circularity Check

1 steps flagged · score 4.0 of 10

One definitional artifact in the correct/wrong split; otherwise the layer-wise and enrichment analyses are self-contained.

  1. self definitional [Sec. VI-B (RQ2), Eq. (2) and the definition of correct/wrong predictions]
    "A correct prediction means the ground-truth label has the highest next-token probability at the last token position (i.e., “is”) where LALMs are signaled to make predictions. Accordingly, the model’s prediction accuracy equals its I L −1 ... For each model and attribute, we split the dataset into correctly and incorrectly predicted subsets and compute the layer-wise information score I ℓ −1 separately for each subset."

    Because Eq. (2) defines I^ℓ_i as the indicator that the ground-truth label equals the argmax of the Logit-Lens distribution at layer ℓ, and because the correct/incorrect split is defined by the argmax at the final layer L, the final-layer score I^L_{-1} is identically 1 for the correct subset and 0 for the incorrect subset. The large final-layer gap between the green and red curves in Fig. 3 is therefore not an empirical discovery: it holds by construction. The pre-final peak-and-decline trajectory is empirical, but the paper's statement that for incorrect predictions information 'sharply declines' at the output is partly a restatement of the subset definition rather than an independent observation about layer-wise information dynamics.

full rationale

The core analysis is a genuine empirical study rather than a circular derivation: Logit-Lens scores are obtained from frozen LALMs, critical layers are a derived statistic, the audio-masking intervention in RQ4 is a causal manipulation, and the enrichment experiment in RQ5 selects layers on a disjoint 100-sample probing set and evaluates on a 400-sample test set. The only circular step is the RQ2 correct/wrong comparison: because 'correct' is defined by the final-layer argmax and I^L is defined as the accuracy of that same argmax, the final-layer values of the two subsets are 1 and 0 by construction, which partially inflates the 'decreases with depth when recognition fails' narrative at the last layer. The pre-final peak-and-decline shape, the RQ3 correlation, the RQ4 token-position and masking findings, and the RQ5 improvement do not reduce to the method's inputs. The unvalidated faithfulness of the Logit-Lens projection in Eq. (1) is a correctness/robustness risk, not a circularity. Self-citations (e.g., the SAKURA benchmark) supply data and context but are not load-bearing for the derivation.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The analysis rests on the Logit Lens assumption, several hand-set thresholds (α=0.2), heuristic layer gaps (5), and a scaling factor (λ=1.25) tuned on the test set. No new physical or conceptual entities are introduced.

free parameters (3)
  • α (threshold) = 0.2
    Hand-chosen threshold in Eq. (3) to filter layers whose information score barely exceeds chance; affects the critical layer computation.
  • λ (scaling factor) = 1.25
    Scale in Eq. (5) selected from Fig. 5 as the value with best test accuracy, i.e., tuned on the test set. The 16.3% relative improvement depends on this choice.
  • Layer gap = 5
    Heuristic number of layers deeper to enrich (Sec VI-E); no search or justification beyond 'too small/too large' considerations.
assumptions (4)
  • domain assumption Logit Lens projection through W_U is a valid readout of attribute information at intermediate layers of LALMs.
    Invoked in Sec IV-A, Eq (1). The paper transfers this assumption from LLM interpretability studies without validating it for audio-language models, where the residual stream mixes audio encoder outputs.
  • domain assumption The probability that the ground-truth attribute token is the argmax among attribute labels in the projected distribution measures attribute resolution.
    Defined in Sec IV-B, Eq (2). Assumes that attribute knowledge is reducible to the relative probability mass of the attribute token in the vocabulary.
  • domain assumption The final token position ('is') is the position where attribute resolution must be complete.
    Sec V-A: 'we focus on hidden representations at the final token'. This presumes the model resolves attributes at the prediction point, which may not hold for MCQA formats where options are listed.
  • domain assumption Masking audio input positions in self-attention at the last token yields a valid counterfactual for direct audio querying.
    Sec VI-D: 'we mask auditory inputs during self-attention at the last token'. Assumes attention masking isolates the effect of direct audio querying and does not perturb other pathways.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AudioLens: A Closer Look at Auditory Attribute Perception of Large Audio-Language Models." pith.science (2026). https://pith.science/paper/EJLCXXSQ

@misc{pith2026250605140,
  author       = {Pith},
  title        = {Pith review of: AudioLens: A Closer Look at Auditory Attribute Perception of Large Audio-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EJLCXXSQ}},
  note         = {Machine review of arXiv:2506.05140}
}
read the original abstract

Understanding the internal mechanisms of large audio-language models (LALMs) is crucial for interpreting their behavior and improving performance. This work presents the first in-depth analysis of how LALMs internally perceive and recognize auditory attributes. By applying vocabulary projection on three state-of-the-art LALMs, we track how attribute information evolves across layers and token positions. We find that attribute information generally decreases with layer depth when recognition fails, and that resolving attributes at earlier layers correlates with better accuracy. Moreover, LALMs heavily rely on querying auditory inputs for predicting attributes instead of aggregating necessary information in hidden states at attribute-mentioning positions. Based on our findings, we demonstrate a method to enhance LALMs. Our results offer insights into auditory attribute processing, paving the way for future improvements.

Figures

Figures reproduced from arXiv: 2506.05140 by the authors.

Figure 1
Figure 1. Illustration of Logit Lens and our method for investigating the internal [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Layer-wise information scores at the last token position of P1, P2, and P3, computed across all layers for three LALMs and four auditory attributes. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Layer-wise information scores for three LALMs and four auditory attributes at the final token under P3 prompts. Green lines and left y-axis show [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Layer-wise information scores for three LALMs and four auditory attributes at the final token (i.e., the token “is”), the penultimate token (e.g., the [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Accuracy (%) of enriching the deeper layer using layers selected by [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Probing Low-Level Acoustic Attribute Encoding in CLAP Audio Embeddings

    eess.AS 2026-07 accept novelty 6.0 of 10

    RT60, LUFS, and relative pitch are approximately linearly recoverable from frozen CLAP embeddings across noise, speech, and music, while spectral centroid needs non-linear probes; the pattern largely generalizes to ot...

Reference graph

Works this paper leans on

71 extracted references · 53 canonical work pages · cited by 1 Pith paper

  1. [1]

    A survey of large language models,

    W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y . Hou, Y . Min, B. Zhang, J. Zhang, Z. Dong et al. , “A survey of large language models,” arXiv preprint arXiv:2303.18223, 2023

    arXiv 2023

  2. [2]

    The llama 3 herd of models,

    A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan et al. , “The llama 3 herd of models,” arXiv preprint arXiv:2407.21783 , 2024

    arXiv 2024

  3. [3]

    Gpt-4o system card,

    A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda, A. Hayes, A. Radford et al. , “Gpt-4o system card,” arXiv preprint arXiv:2410.21276 , 2024

    arXiv 2024

  4. [4]

    Listen, think, and understand,

    Y . Gong, H. Luo, A. H. Liu, L. Karlinsky, and J. Glass, “Listen, think, and understand,” in International Conference on Learning Representa- tions, 2024

    work page 2024

  5. [5]

    Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models,

    Y . Chu, J. Xu, X. Zhou, Q. Yang, S. Zhang, Z. Yan, C. Zhou, and J. Zhou, “Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models,” arXiv preprint arXiv:2311.07919 , 2023

    arXiv 2023

  6. [6]

    Qwen2-audio technical report,

    Y . Chu, J. Xu, Q. Yang, H. Wei, X. Wei, Z. Guo, Y . Leng, Y . Lv, J. He, J. Lin et al. , “Qwen2-audio technical report,” arXiv preprint arXiv:2407.10759, 2024

    arXiv 2024

  7. [7]

    Developing instruction-following speech language model without speech instruction-tuning data,

    K.-H. Lu, Z. Chen, S.-W. Fu, C.-H. H. Yang, J. Balam, B. Ginsburg, Y .- C. F. Wang, and H.-y. Lee, “Developing instruction-following speech language model without speech instruction-tuning data,” in ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2025, pp. 1–5

    work page 2025

  8. [8]

    Gama: A large audio- language model with advanced audio understanding and complex rea- soning abilities,

    S. Ghosh, S. Kumar, A. Seth, C. K. R. Evuru, U. Tyagi, S. Sakshi, O. Nieto, R. Duraiswami, and D. Manocha, “Gama: A large audio- language model with advanced audio understanding and complex rea- soning abilities,” in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , 2024, pp. 6288–6313

    work page 2024

Show all 71 references
  1. [9]

    Desta: Enhancing speech language models through descriptive speech-text alignment,

    K.-H. Lu, Z. Chen, S.-W. Fu, H. Huang, B. Ginsburg, Y .-C. F. Wang, and H.-y. Lee, “Desta: Enhancing speech language models through descriptive speech-text alignment,” in Proc. Interspeech 2024 , 2024, pp. 4159–4163

    2024

  2. [10]

    Speech- copilot: Leveraging large language models for speech processing via task decomposition, modularization, and program generation,

    C.-Y . Kuan, C.-K. Yang, W.-P. Huang, K.-H. Lu, and H.-y. Lee, “Speech- copilot: Leveraging large language models for speech processing via task decomposition, modularization, and program generation,” in 2024 IEEE Spoken Language Technology Workshop (SLT) . IEEE, 2024, pp. 1060– 1067

    2024

  3. [11]

    Joint audio and speech understanding,

    Y . Gong, A. H. Liu, H. Luo, L. Karlinsky, and J. Glass, “Joint audio and speech understanding,” in 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) . IEEE, 2023, pp. 1–8

    2023

  4. [12]

    SALMONN: Towards generic hearing abilities for large language models,

    C. Tang, W. Yu, G. Sun, X. Chen, T. Tan, W. Li, L. Lu, Z. MA, and C. Zhang, “SALMONN: Towards generic hearing abilities for large language models,” in The Twelfth International Conference on Learning Representations , 2024. [Online]. Available: https://openreview.net/forum?id=...

    2024

  5. [13]

    Blsp-emo: Towards empathetic large speech-language models,

    C. Wang, M. Liao, Z. Huang, J. Wu, C. Zong, and J. Zhang, “Blsp-emo: Towards empathetic large speech-language models,” in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024, pp. 19 186–19 199

    2024

  6. [14]

    Wavllm: Towards robust and adaptive speech large language model,

    S. Hu, L. Zhou, S. Liu, S. Chen, L. Meng, H. Hao, J. Pan, X. Liu, J. Li, S. Sivasankaran et al. , “Wavllm: Towards robust and adaptive speech large language model,” in Findings of the Association for Computational Linguistics: EMNLP 2024 , 2024, pp. 4552–4572

    2024

  7. [15]

    Dynamic-SUPERB phase-2: A collaboratively expanding benchmark for measuring the capabilities of spoken language models with 180 tasks,

    C.-y. Huang, W.-C. Chen, S. wen Yang, A. T. Liu, C.-A. Li, Y .-X. Lin, W.-C. Tseng, A. Diwan, Y .-J. Shih, J. Shi, W. Chen, X. Chen, C.-Y . Hsiao, P. Peng, S.-H. Wang, C.-Y . Kuan, K.-H. Lu, K.-W. Chang, C.-K. Yang, F. A. R. Gutierrez, H. Kuan-Po, S. Arora, Y .-K. Lin, C. M. T...

    2025

  8. [16]

    AIR-bench: Benchmarking large audio-language models via generative comprehension,

    Q. Yang, J. Xu, W. Liu, Y . Chu, Z. Jiang, X. Zhou, Y . Leng, Y . Lv, Z. Zhao, C. Zhou, and J. Zhou, “AIR-bench: Benchmarking large audio-language models via generative comprehension,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (...

    2024

  9. [17]

    MMAU: A massive multi- task audio understanding and reasoning benchmark,

    S. Sakshi, U. Tyagi, S. Kumar, A. Seth, R. Selvakumar, O. Nieto, R. Duraiswami, S. Ghosh, and D. Manocha, “MMAU: A massive multi- task audio understanding and reasoning benchmark,” in The Thirteenth International Conference on Learning Representations , 2025. [Online]. Availab...

    2025

  10. [18]

    SD-eval: A benchmark dataset for spoken dialogue understanding beyond words,

    J. Ao, Y . Wang, X. Tian, D. Chen, J. Zhang, L. Lu, Y . Wang, H. Li, and Z. Wu, “SD-eval: A benchmark dataset for spoken dialogue understanding beyond words,” in The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track , 2024. [Online]...

    2024

  11. [19]

    Audiobench: A universal benchmark for audio large language models,

    B. Wang, X. Zou, G. Lin, S. Sun, Z. Liu, W. Zhang, Z. Liu, A. Aw, and N. F. Chen, “Audiobench: A universal benchmark for audio large language models,” NAACL, 2025

    2025

  12. [20]

    Sakura: On the multi- hop reasoning of large audio-language models based on speech and audio information,

    C.-K. Yang, N. Ho, Y .-T. Piao, and H.-y. Lee, “Sakura: On the multi- hop reasoning of large audio-language models based on speech and audio information,” Interspeech 2025 , 2025

    2025

  13. [21]

    Towards holistic evaluation of large audio-language models: A comprehensive survey,

    C.-K. Yang, N. S. Ho, and H.-y. Lee, “Towards holistic evaluation of large audio-language models: A comprehensive survey,” arXiv preprint arXiv:2505.15957, 2025

    2025 arXiv

  14. [22]

    A preliminary exploration with gpt-4o voice mode,

    Y .-X. Lin, C.-K. Yang, W.-C. Chen, C.-A. Li, C.-y. Huang, X. Chen, and H.-y. Lee, “A preliminary exploration with gpt-4o voice mode,” arXiv preprint arXiv:2502.09940, 2025

    2025 arXiv

  15. [23]

    Language-specific neurons: The key to multilingual ca- pabilities in large language models,

    T. Tang, W. Luo, H. Huang, D. Zhang, X. Wang, W. X. Zhao, F. Wei, and J.-R. Wen, “Language-specific neurons: The key to multilingual ca- pabilities in large language models,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (V olume 1:...

    2024

  16. [24]

    Unveiling a core linguistic region in large language models,

    J. Zhao, Z. Zhang, Y . Ma, Q. Zhang, T. Gui, L. Gao, and X. Huang, “Unveiling a core linguistic region in large language models,” arXiv preprint arXiv:2310.14928, 2023

    2023 arXiv

  17. [25]

    Do large language models latently perform multi-hop reasoning?

    S. Yang, E. Gribovskaya, N. Kassner, M. Geva, and S. Riedel, “Do large language models latently perform multi-hop reasoning?” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Papers) , 2024, pp. 10 210–10 229

    2024

  18. [26]

    Hopping too late: Exploring the limitations of large language models on multi-hop queries,

    E. Biran, D. Gottesman, S. Yang, M. Geva, and A. Globerson, “Hopping too late: Exploring the limitations of large language models on multi-hop queries,” in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , 2024, pp. 14 113–14 130

    2024

  19. [27]

    Back attention: Understanding and enhancing multi-hop reasoning in large language models,

    Z. Yu, Y . Belinkov, and S. Ananiadou, “Back attention: Understanding and enhancing multi-hop reasoning in large language models,” arXiv preprint arXiv:2502.10835, 2025

    2025 arXiv

  20. [28]

    Knowledge neurons in pretrained transformers,

    D. Dai, L. Dong, Y . Hao, Z. Sui, B. Chang, and F. Wei, “Knowledge neurons in pretrained transformers,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Papers), 2022, pp. 8493–8502

    2022

  21. [29]

    Neuron-level knowledge attribution in large language models,

    Z. Yu and S. Ananiadou, “Neuron-level knowledge attribution in large language models,” in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , 2024, pp. 3267–3280

    2024

  22. [30]

    Listen and speak fairly: a study on semantic gender bias in speech integrated large language models,

    Y .-C. Lin, T.-Q. Lin, C.-K. Yang, K.-H. Lu, W.-C. Chen, C.-Y . Kuan, and H.-y. Lee, “Listen and speak fairly: a study on semantic gender bias in speech integrated large language models,” in 2024 IEEE Spoken Language Technology Workshop (SLT) . IEEE, 2024, pp. 439–446

    2024

  23. [31]

    The curse of multi-modalities: Evaluating hallucinations of large multimodal models across language, visual, and audio,

    S. Leng, Y . Xing, Z. Cheng, Y . Zhou, H. Zhang, X. Li, D. Zhao, S. Lu, C. Miao, and L. Bing, “The curse of multi-modalities: Evaluating hallucinations of large multimodal models across language, visual, and audio,” arXiv preprint arXiv:2410.12787 , 2024

    arXiv

  24. [32]

    Can large audio-language models truly hear? tackling hallucinations with multi-task assessment and stepwise audio reasoning,

    C.-Y . Kuan and H.-y. Lee, “Can large audio-language models truly hear? tackling hallucinations with multi-task assessment and stepwise audio reasoning,” in ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2025, pp. 1–5

    2025

  25. [33]

    Interpreting GPT: the logit lens,

    nostalgebraist, “Interpreting GPT: the logit lens,” https://www.lesswrong. com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens, Au- gust 2020

    2020

  26. [34]

    Transformer feed- forward layers build predictions by promoting concepts in the vocabulary space,

    M. Geva, A. Caciularu, K. Wang, and Y . Goldberg, “Transformer feed- forward layers build predictions by promoting concepts in the vocabulary space,” in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , 2022, pp. 30–45

    2022

  27. [35]

    Jump to conclusions: Short-cutting transformers with linear transformations,

    A. Y . Din, T. Karidi, L. Choshen, and M. Geva, “Jump to conclusions: Short-cutting transformers with linear transformations,” in Proceedings of the 2024 Joint International Conference on Computational Linguis- tics, Language Resources and Evaluation (LREC-COLING 2024) , 2024,...

    2024

  28. [36]

    Eliciting latent predictions from trans- formers with the tuned lens,

    N. Belrose, Z. Furman, L. Smith, D. Halawi, I. Ostrovsky, L. McKinney, S. Biderman, and J. Steinhardt, “Eliciting latent predictions from trans- formers with the tuned lens,” arXiv preprint arXiv:2303.08112 , 2023

    2023 arXiv

  29. [37]

    Superb: Speech processing universal performance benchmark,

    S.-w. Yang, P.-H. Chi, Y .-S. Chuang, C.-I. J. Lai, K. Lakhotia, Y . Y . Lin, A. T. Liu, J. Shi, X. Chang, G.-T. Lin et al., “Superb: Speech processing universal performance benchmark,” in Proc. Interspeech 2021, 2021, pp. 1194–1198

    2021

  30. [38]

    Superb-sg: Enhanced speech processing universal performance benchmark for semantic and generative capabilities,

    H.-S. Tsai, H.-J. Chang, W.-C. Huang, Z. Huang, K. Lakhotia, S.-w. Yang, S. Dong, A. Liu, C.-I. Lai, J. Shi et al. , “Superb-sg: Enhanced speech processing universal performance benchmark for semantic and generative capabilities,” in Proceedings of the 60th Annual Meeting of t...

    2022

  31. [39]

    Hear: Holistic evaluation of audio representations,

    J. Turian, J. Shier, H. R. Khan, B. Raj, B. W. Schuller, C. J. Steinmetz, C. Malloy, G. Tzanetakis, G. Velarde, K. McNally et al., “Hear: Holistic evaluation of audio representations,” in NeurIPS 2021 Competitions and Demonstrations Track. PMLR, 2022, pp. 125–145

    2021

  32. [40]

    Marble: Music audio representation benchmark for universal evaluation,

    R. Yuan, Y . Ma, Y . Li, G. Zhang, X. Chen, H. Yin, Y . Liu, J. Huang, Z. Tian, B. Deng et al., “Marble: Music audio representation benchmark for universal evaluation,” Advances in Neural Information Processing Systems, vol. 36, pp. 39 626–39 647, 2023

    2023

  33. [41]

    The zero resource speech benchmark 2021: Metrics and baselines for unsupervised spoken lan- guage modeling,

    T. A. Nguyen, M. de Seyssel, P. Roz ´e, M. Rivi `ere, E. Kharitonov, A. Baevski, E. Dunbar, and E. Dupoux, “The zero resource speech benchmark 2021: Metrics and baselines for unsupervised spoken lan- guage modeling,” in NeuRIPS Workshop on Self-Supervised Learning for Speech a...

    2021

  34. [42]

    Zero re- source code-switched speech benchmark using speech utterance pairs for multiple spoken languages,

    K.-P. Huang, C.-K. Yang, Y .-K. Fu, E. Dunbar, and H.-Y . Lee, “Zero re- source code-switched speech benchmark using speech utterance pairs for multiple spoken languages,” in ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) ,...

    2024

  35. [43]

    Yang, K.-P

    C.-K. Yang, K.-P. Huang, K.-H. Lu, C.-Y . Kuan, C.-Y . Hsiao, and H.-Y . Lee, “Investigating zero-shot generalizability on mandarin-english code- switched asr and speech-to-text translation of recent foundation models with self-supervision and weak supervision,” in 2024 IEEE I...

    2024

  36. [44]

    Ml-superb: Multi- lingual speech universal performance benchmark,

    J. Shi, D. Berrebbi, W. Chen, E.-P. Hu, W.-P. Huang, H.-L. Chung, X. Chang, S.-W. Li, A. Mohamed, H.-y. Lee et al. , “Ml-superb: Multi- lingual speech universal performance benchmark,” in Proc. Interspeech 2023, 2023, pp. 884–888

    2023

  37. [45]

    Hubert: Self-supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,” IEEE/ACM transactions on audio, speech, and language processing , vol. 29, pp. 3451–3460, 2021

    2021

  38. [46]

    Distilhubert: Speech represen- tation learning by layer-wise distillation of hidden-unit bert,

    H.-J. Chang, S.-w. Yang, and H.-y. Lee, “Distilhubert: Speech represen- tation learning by layer-wise distillation of hidden-unit bert,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2022, pp. 7087–7091

    2022

  39. [47]

    wav2vec: Unsu- pervised pre-training for speech recognition,

    S. Schneider, A. Baevski, R. Collobert, and M. Auli, “wav2vec: Unsu- pervised pre-training for speech recognition,” in Proc. Interspeech 2019, 2019, pp. 3465–3469

    2019

  40. [48]

    wav2vec 2.0: A framework for self-supervised learning of speech representations,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” Advances in neural information processing systems , vol. 33, pp. 12 449– 12 460, 2020

    2020

  41. [49]

    Un- supervised cross-lingual representation learning for speech recognition,

    A. Conneau, A. Baevski, R. Collobert, A. Mohamed, and M. Auli, “Un- supervised cross-lingual representation learning for speech recognition,” in Interspeech 2021 , 2021, pp. 2426–2430

    2021

  42. [50]

    Layer-wise analysis of a self-supervised speech representation model,

    A. Pasad, J.-C. Chou, and K. Livescu, “Layer-wise analysis of a self-supervised speech representation model,” in 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE, 2021, pp. 914–921

    2021

  43. [51]

    Comparative layer-wise analysis of self-supervised speech models,

    A. Pasad, B. Shi, and K. Livescu, “Comparative layer-wise analysis of self-supervised speech models,” in ICASSP 2023-2023 IEEE Interna- tional Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2023, pp. 1–5

    2023

  44. [52]

    What do self- supervised speech models know about words?

    A. Pasad, C.-M. Chien, S. Settle, and K. Livescu, “What do self- supervised speech models know about words?” Transactions of the Association for Computational Linguistics , vol. 12, pp. 372–391, 2024

    2024

  45. [53]

    Self-supervised speech representations are more phonetic than semantic,

    K. Choi, A. Pasad, T. Nakamura, S. Fukayama, K. Livescu, and S. Watanabe, “Self-supervised speech representations are more phonetic than semantic,” in Proc. Interspeech 2024 , 2024, pp. 4578–4582

    2024

  46. [54]

    Property neurons in self-supervised speech transformers,

    T.-Q. Lin, G.-T. Lin, H.-y. Lee, and H. Tang, “Property neurons in self-supervised speech transformers,” in 2024 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2024, pp. 401–408

    2024

  47. [55]

    And: Audio network dissection for interpreting deep acoustic models,

    T.-Y . Wu, Y .-X. Lin, and T.-W. Weng, “And: Audio network dissection for interpreting deep acoustic models,” in International Conference on Machine Learning . PMLR, 2024, pp. 53 656–53 680

    2024

  48. [56]

    Do prompts really prompt? exploring the prompt understanding capability of whisper,

    C.-K. Yang, K.-P. Huang, and H.-y. Lee, “Do prompts really prompt? exploring the prompt understanding capability of whisper,” in2024 IEEE Spoken Language Technology Workshop (SLT) . IEEE, 2024, pp. 1–8

    2024

  49. [57]

    Hey asr system! why aren’t you more inclusive? automatic speech recognition systems’ bias and proposed bias mitigation techniques. a literature review,

    M. K. Ngueajio and G. Washington, “Hey asr system! why aren’t you more inclusive? automatic speech recognition systems’ bias and proposed bias mitigation techniques. a literature review,” inInternational conference on human-computer interaction . Springer, 2022, pp. 421– 440

    2022

  50. [58]

    Emo-bias: A large scale evaluation of social bias on speech emotion recognition,

    Y .-C. Lin, H. Wu, H.-C. Chou, C.-C. Lee, and H.-y. Lee, “Emo-bias: A large scale evaluation of social bias on speech emotion recognition,” in Proc. Interspeech 2024 , 2024, pp. 4633–4637

    2024

  51. [59]

    Attention is not only a weight: Analyzing transformers with vector norms,

    G. Kobayashi, T. Kuribayashi, S. Yokoi, and K. Inui, “Attention is not only a weight: Analyzing transformers with vector norms,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , 2020, pp. 7057–7075

    2020

  52. [60]

    An investigation of neuron activation as a unified lens to explain chain-of-thought eliciting arithmetic reasoning of llms,

    D. Rai and Z. Yao, “An investigation of neuron activation as a unified lens to explain chain-of-thought eliciting arithmetic reasoning of llms,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Papers) , 2024, pp. 7174– 7193

    2024

  53. [61]

    Understanding and enhancing safety mechanisms of LLMs via safety-specific neuron,

    Y . Zhao, W. Zhang, Y . Xie, A. Goyal, K. Kawaguchi, and M. Shieh, “Understanding and enhancing safety mechanisms of LLMs via safety-specific neuron,” in The Thirteenth International Conference on Learning Representations , 2025. [Online]. Available: https://openreview.net/for...

    2025

  54. [62]

    Patchscopes: a unifying framework for inspecting hidden representa- tions of language models,

    A. Ghandeharioun, A. Caciularu, A. Pearce, L. Dixon, and M. Geva, “Patchscopes: a unifying framework for inspecting hidden representa- tions of language models,” in Proceedings of the 41st International Conference on Machine Learning , 2024, pp. 15 466–15 490

    2024

  55. [63]

    Probing classifiers: Promises, shortcomings, and ad- vances,

    Y . Belinkov, “Probing classifiers: Promises, shortcomings, and ad- vances,” Computational Linguistics , vol. 48, no. 1, pp. 207–219, 2022

    2022

  56. [64]

    Locating and editing factual associations in gpt,

    K. Meng, D. Bau, A. Andonian, and Y . Belinkov, “Locating and editing factual associations in gpt,” Advances in neural information processing systems, vol. 35, pp. 17 359–17 372, 2022

    2022

  57. [65]

    Language models implement simple word2vec-style vector arithmetic,

    J. Merullo, C. Eickhoff, and E. Pavlick, “Language models implement simple word2vec-style vector arithmetic,” in Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (V olume 1: Long Pape...

    2024

  58. [66]

    Dissecting recall of factual associations in auto-regressive language models,

    M. Geva, J. Bastings, K. Filippova, and A. Globerson, “Dissecting recall of factual associations in auto-regressive language models,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, pp. 12 216–12 235

    2023

  59. [67]

    Logitlens4llms: Extending logit lens analysis to modern large language models,

    Z. Wang, “Logitlens4llms: Extending logit lens analysis to modern large language models,” arXiv preprint arXiv:2503.11667 , 2025

    2025 arXiv

  60. [68]

    Interpreting and editing vision-language representations to mitigate hallucinations,

    N. Jiang, A. Kachinthaya, S. Petryk, and Y . Gandelsman, “Interpreting and editing vision-language representations to mitigate hallucinations,” in The Thirteenth International Conference on Learning Representations , 2025

    2025

  61. [69]

    Towards interpreting visual information processing in vision-language models,

    C. Neo, L. Ong, P. Torr, M. Geva, D. Krueger, and F. Barez, “Towards interpreting visual information processing in vision-language models,” arXiv preprint arXiv:2410.07149 , 2024

    2024 arXiv

  62. [70]

    Mmneuron: Discovering neuron-level domain-specific interpretation in multimodal large language model,

    J. Huo, Y . Yan, B. Hu, Y . Yue, and X. Hu, “Mmneuron: Discovering neuron-level domain-specific interpretation in multimodal large language model,” in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , 2024, pp. 6801–6816

    2024

  63. [71]

    Measuring massive multitask language understanding,

    D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt, “Measuring massive multitask language understanding,” in International Conference on Learning Representations , 2021. [Online]. Available: https://openreview.net/forum?id=d7KBjmI3GmQ

    2021

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.