Pith. sign in

REVIEW 2 major objections 5 minor 39 references

Separating Decision-Rule Misalignment from Readout-Coverage Limitations in Speech Language Models

T0 review · 2 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Diagnostic ladder separates decision-rule from readout-coverage failures in speech emotion answers.

desk verdict A well-executed diagnostic framework that separates decision-rule from readout-coverage gaps in speech LMs; the L* approximation is a real caveat but the authors handle it honestly, and the paper deserves a serious referee. read the letter →

arxiv 2608.06409 v1 pith:M5IA6LBC submitted 2026-08-03 cs.CL cs.AI

Linkai Peng , Baorian Nuchged This is my paper · ORCID
classification cs.CLcs.AI
keywords speechlanguagemodelsparalinguisticevaluationemotionrecognitiondiagnosticladderlogitcorrectionreadout-coveragegapprobingclassifiersactivationpatching
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a speech language model's poor answer on a prompted emotion question does not mean the emotion information never reached the model. It builds a generation-aligned ladder with four rungs — the emitted answer, the option-logit choice, an optimized affine reader of those logits, and a linear reader of the hidden state at the same answer token — so that successive accuracy differences decompose the total gap exactly into an endpoint term, a decision-rule gap, and a readout-coverage gap. Across five systems and two emotion corpora, both explanatory gaps are positive in every one of the ten conditions, and full-state decoding beats generation by 27.8 accuracy points on average. A label-free logit correction improves the emitted answer in all ten conditions, while readout-external directions that remain decodable rarely change the emitted answer when replaced, separating information availability from behavioral use.

What carries the argument

The central object is the generation-aligned diagnostic ladder, anchored at one answer token. Let $x_t$ be the post-normalization hidden state, $W$ the output head, and $z_t = W x_t$ the full-vocabulary logits; the option-contrast vector $c_t \in \mathbb{R}^3$ contains the three option-logit differences relative to a reference option. The ladder compares greedy generation over the full vocabulary, the argmax over the four option logits, an affine reader fitted on $c_t$, and an affine reader fitted on $x_t$, all scored on the same rows. The exact identity $A_{\mathrm{state}}-A_{\mathrm{gen}} = (A_{\mathrm{opt}}-A_{\mathrm{gen}})+(A_{\mathrm{aff}}-A_{\mathrm{opt}})+(A_{\mathrm{state}}-A_{\mathrm{aff}})$ carries the argument, splitting the total gap into endpoint, decision-rule, and readout-coverage terms. To localize the coverage gap, $x_t$ is decomposed as $x_t = P_{V_3}x_t + P_{V_3^\perp}x_t$, where $V_3$ is the row space of the option-contrast matrix $C$ (the answer-readout space); decoding from $V_3$ is compared with supervised and random subspaces of $V_4^\perp \subset V_3^\perp$ at a training-selected layer $L^*$. Minimal-pair activation replacement, $h_r + P_S(h_d - h_r)$, tests whether the selected readout-external directions actually steer the emitted answer.

What would settle it

Repeat the ladder with the full-state reader restricted to a three-dimensional projection of the hidden state chosen without emotion labels, such as the top three principal components of the training states. If its accuracy no longer exceeds the affine reader over the three option-logit contrasts, the readout-coverage gap would be exposed as an artifact of the state reader's larger input space rather than a property of the model's readout geometry.

Watch

Extended reading notes

Core claim

The paper's central discovery is that the gap between what a speech language model emits and what its answer-position hidden state supports has a precise internal structure. At the first answer token, the model's hidden state $x_t$ is passed through the output head $W$ to produce full-vocabulary logits; restricting those logits to the four prompted options gives the native option choice. The paper defines four accuracy levels on identical rows — $A_{\mathrm{gen}}$ (greedy generation), $A_{\mathrm{opt}}$ (option-logit argmax), $A_{\mathrm{aff}}$ (a fitted affine reader over the three option-logit contrasts), and $A_{\mathrm{state}}$ (a regularized affine reader over the full state) — and shows by telescoping that $A_{\mathrm{state}}-A_{\mathrm{gen}}$ equals the sum of an endpoint term, a decision-rule gap $A_{\mathrm{aff}}-A_{\mathrm{opt}}$, and a readout-coverage gap $A_{\mathrm{state}}-A_{\mathrm{aff}}$. In all ten system–corpus conditions both explanatory gaps are positive and their confidence intervals exclude zero, with $A_{\mathrm{state}}$ exceeding $A_{\mathrm{gen}}$ by 27.8 points on average. The readout-coverage gap is not pure readout geometry: the paper treats it as a performance gap and leans on rank-matched subspace comparisons. A label-free logit correction improves the generated answer in every condition, while replacing selected readout-external directions rarely changes the answer, so the paper's sharper claim is that emotion information can be available at the answer position without being used.

Load-bearing premise

The whole readout-coverage story rests on the assumption that the intermediate layer chosen for inspection reflects the model's real answer pathway; if that layer does not line up with the final output head, directions called 'readout-external' are mislabeled, and the weak causal effects no longer show that the model has emotion information it fails to use.

Editorial extensions

If this is right

  • A single generation-accuracy score understates what a speech model knows: in every evaluated condition, the answer-position state supports substantially better emotion decoding than the model's emitted answer.
  • A positive decision-rule gap means logit-side fixes — calibrating option preferences or fitting a better rule over existing contrasts — can recover accuracy without changing hidden representations.
  • A positive readout-coverage gap means reweighting option logits is not enough; improving answers requires changing how hidden-state information reaches the output, such as a readout adapter or auxiliary supervision.
  • Evaluation of paralinguistic language models should report generated answers, option logits, and hidden-state decodability at the same answer event, with speaker-disjoint splits and dimension-matched controls.
  • Probing results alone are not mechanism evidence: matched replacement shows that decodable readout-external information can have only limited causal influence on the answer.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension: the same three-view ladder can be applied to any single-token prompted classification task; the ratio of the two gaps may predict whether logit calibration or readout adaptation will help.
  • An implicit consequence: if readout-external emotion directions rarely change the answer, then auxiliary supervision that pushes emotion information into the option-token span should shrink the coverage gap; donor-following rates would rise if that routing story is right.
  • A stress test of the correction: estimate the option offsets on one corpus and apply them to another; if the gains vanish, the decision-rule gap is dominated by corpus-specific option priors rather than a stable system bias.
  • An editorial caution: the acoustic controls cover twenty measured descriptors and loudness, not every signal property; resynthesis-based controls could settle whether the residual readout-external decodability is genuinely emotion content or additional unmeasured cues.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper introduces a generation-aligned diagnostic ladder for speech language models, comparing four levels at the same first-answer-token event: generated accuracy (A_gen), the option-logit argmax (A_opt), an affine reader over option-logit contrasts (A_aff), and a regularized affine reader over the full hidden state (A_state). Because the levels are nested, the differences telescope into an endpoint-validity term, a decision-rule gap, and a readout-coverage gap (Eq. 3). Across five systems and two emotion corpora (ten conditions), the paper reports A_state - A_gen = 0.278 on average, with both explanatory gaps positive in every condition. A label-free logit correction improves generation in all ten conditions. In rank-matched subspace analyses at an intermediate layer L*, emotion information outside the native option-readout span generalizes to held-out speakers and survives acoustic controls, but minimal-pair replacement of the selected readout-external directions rarely changes the emitted answer. The authors conclude that decision-rule misalignment and readout-coverage limitations are separate, and that decodable information need not be causally used.

Significance. The diagnostic ladder is simple and the telescoping identity is exact; the empirical protocol is unusually careful, with speaker-disjoint evaluation, paired speaker-clustered bootstrap intervals, matched-budget refits, delete-one-speaker jackknives, endpoint audits, perturbation-magnitude diagnostics, and a batch-calibration procedure whose predictions were fixed before the generation pass. The paper also states its limitations explicitly. If the L*-based availability/use analysis is anchored to the final answer position, the contribution would be a useful, falsifiable decomposition that separates behavioral failures from representational ones. The main weakness is that the availability/use claims are established in L* geometry rather than at the answer-position readout, which weakens the central availability-use distinction. With added final-layer validation, this would be a strong paper.

major comments (2)
  1. [III-D and V-D] The availability/use analysis is carried out at an intermediate layer L* using the final head's option-row geometry, while the coverage gap Δ_coverage is defined at the final answer position. The paper acknowledges (Section III-D) that at L*, V3 is "a reference aligned with the final readout, not an exact decomposition of the final logits," but Section V-D nonetheless concludes that "readout-external emotion information has limited causal access at the answer." Because later attention, MLP, and residual blocks can mix a V4⊥ component into the final option logits, a direction that is readout-external in the L* geometry need not be readout-external in the geometry that determines the emitted answer. The one-condition propagation analysis (Supplementary S7-F) shows that in Qwen2-Audio×CREMA-D an injected layer-20 external component reaches the final logits, but no analogous evidence is provided for the other nine conditions. I ask for either (a) a repeat of the key decodability and minimal-pair causal analyses at the final answer-position state, where the V3/V4 decomposition is exact, or (b) a direct validation that the L* projection behaves like the final readout for these subspaces (e.g., compare V4⊥ decodability and injected-component propagation at L* and at the final layer across all conditions). Without this, the readout-external label and the null causal effects are not anchored to the answer-position readout.
  2. [III-D and Table I] The readout-coverage gap is a final-layer performance gap between a three-contrast reader and a full-state reader, and the paper is careful to call it a performance gap. However, the localization of that gap to information "outside the native readout" is based on rank-matched subspace decoding at L*, not at the answer position. If the final-layer V3⊥ complement contains little decodable emotion information, the large Δ_coverage values in Table I could be dominated by the full-state reader's larger input space rather than by information that the native readout fails to expose. The matched-budget refit (Table S6) rules out regularization differences but not the input-space dimension. I would like to see the final-layer analogue: held-out decoding from P_{V3} and P_{V3⊥} at the answer position, with matched ranks and budgets, so that Δ_coverage is directly connected to the availability claim.
minor comments (5)
  1. [Section III-C, Eq. (4)] The definition of the offset b_k is visually ambiguous: the centering term should be written as (1/4) Σ_{j=1}^4 log p̂_j, and the text should state explicitly that the sum runs over the four prompted options.
  2. [Section IV-B and Fig. 4] The causal intervention uses one fixed rand4 draw as the control arm; because individual random subspace draws vary in the decoding analyses (Table S9), a sensitivity check over several rand4 draws would help establish that the small readout-external effects are not an artifact of the particular random control.
  3. [Section V-C and Fig. 4(a)] The continuous chance line at 0.25 in Fig. 4(a) should be accompanied by the empirical majority-class rates (25.94% for CREMA-D and 25.01% for VESUS, Table S2) so the reader can interpret performance relative to the actual class distribution.
  4. [Section III-D] The term "readout-external" is used both for the exact final-position decomposition in Eq. (6) and for the L*-diagnostic geometry; introducing a distinct label for the latter would avoid treating the L* projection as exact.
  5. [References] Reference [4] (Kimi-Audio technical report) lacks a stable identifier, and reference [38] should include the full Interspeech citation details; both are easy to resolve.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the ladder is an exact telescoping identity over measured accuracies, and the readout-external availability and causal-use results are held-out empirical comparisons with acknowledged approximations.

full rationale

The paper's central decomposition (Eq. 3) is an algebraic identity over four measured accuracies, so it does not derive a prediction from a fit. A_aff and A_state are fitted on speaker-disjoint held-out folds and the gaps are reported with uncertainty; the capacity difference between the contrast reader and the full-state reader is explicitly acknowledged, and the readout-coverage gap is treated as a performance gap rather than a geometric claim. The readout-external analysis uses a supervised subspace selected on training speakers, evaluated on held-out speakers against same-rank random subspaces; the paper explicitly notes the search advantage and avoids ranking native vs. external spaces by raw accuracy. The causal replacements are separate interventions, not fits, and the weak donor-following effects are reported as measured outcomes with paired random-arm controls. The logit correction is transductive (estimated on the evaluation batch), which the paper discloses, but it is used as a behavioral intervention, not as a prediction from the fitted readers. The intermediate-layer L* analysis is an acknowledged approximation ('a reference aligned with the final readout, not an exact decomposition of the final logits'), and the conclusions are scoped to the selected subspace and tested intervention; an approximation is not a circular definition. No load-bearing step reduces to its own inputs, and no self-citation chain is used to justify the central claims.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The paper does not introduce a new theory with free parameters; its measurements depend on standard probing, calibration, and intervention toolkits. The free parameters listed are the data-dependent choices (layer, offsets, ranks, descriptor panels) that shape the empirical findings.

free parameters (5)
  • L* (intervention layer) = Varies per condition, e.g., 28 for Qwen2-Audio x CREMA-D
    Selected on the training split as the layer with highest logit-lens accuracy; all readout-external subspace analyses and interventions are anchored at this layer.
  • Label-free correction offsets b_k = Four offsets per condition and prompt variant, estimated from the unlabeled evaluation batch
    Computed from marginal option probabilities on the target batch; the correction gain and its interpretation as recovering part of the decision-rule gap depend on these values.
  • Rank choices (3 for decoding, 4 for intervention) = 3 and 4
    The number of dimensions for the readout-external subspace and intervention; these ranks are chosen to match V3 and V4 and directly affect the null intervention results.
  • Acoustic descriptor panels (10 and 20 descriptors) = Ten base descriptors and twenty extended descriptors, e.g., F0 statistics, RMS energy, jitter, shimmer, HNR
    The acoustic controls' strength depends on this hand-chosen descriptor set; unmeasured cues could explain residual decodability.
  • Probe regularization penalties = C=0.5 for held-out decodability probes, C=0.01 for subspace construction; A_aff penalty via nested CV
    These hyperparameters affect A_aff and A_state and thus the measured gaps; the paper includes a matched-budget robustness check.
assumptions (6)
  • domain assumption Linear (affine) probes are a meaningful measure of information availability in the hidden state.
    The paper defines the readout-coverage gap and the readout-external availability in terms of affine classifiers; nonlinear encodings could be missed. The authors limit claims to 'linearly accessible' information, but the headline 27.8-point gap still rests on this assumption.
  • domain assumption The answer-position hidden state x_t is the appropriate locus for the state-to-answer interface.
    The ladder analyzes only the final answer token's pre-readout state; losses earlier in the audio pathway or in other layers are out of scope by design.
  • domain assumption The option-token row spans V3 and V4 of the output head define the model's native answer readout.
    This is exact at the final logits but only approximate at the intermediate layer L*, where V3 is used as a reference aligned with the final readout.
  • domain assumption The random subspace baseline rand3 is a valid control for the informativeness of a same-rank subspace that was not selected using labels.
    Random 3D subspaces in the V4 complement are averaged over 20 draws; if the state's variance were concentrated in a few non-readout directions, the baseline could be inflated.
  • domain assumption Speaker-disjoint splits prevent label leakage between reader fitting and evaluation.
    All readers and subspace selections are confined to training speakers and evaluated on held-out speakers; this is standard but assumes sufficient speaker diversity, especially in VESUS with 10 speakers.
  • domain assumption The measured acoustic descriptors and loudness equalization capture the main surface confounds.
    The acoustic controls do not cover every property of the speech signal; the authors acknowledge that unmeasured acoustic cues may still contribute.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Separating Decision-Rule Misalignment from Readout-Coverage Limitations in Speech Language Models." pith.science (2026). https://pith.science/paper/M5IA6LBC

@misc{pith2026260806409,
  author       = {Pith},
  title        = {Pith review of: Separating Decision-Rule Misalignment from Readout-Coverage Limitations in Speech Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/M5IA6LBC}},
  note         = {Machine review of arXiv:2608.06409}
}
read the original abstract

Speech language models are increasingly evaluated on paralinguistic tasks by the accuracy of prompted answers, but answer accuracy combines failures at different stages of the audio-to-answer computation. We introduce a generation-aligned diagnostic ladder that compares the emitted answer, the option logits, an affine readout of those logits, and a linear readout of the hidden state at the same answer token. Successive differences separate endpoint, decision-rule, and readout-coverage gaps. Across five systems and two emotion corpora, state decoding exceeds generation by 27.8 accuracy points on average, and both the decision-rule and readout-coverage gaps are positive in all ten conditions. A label-free logit correction improves generated accuracy in every condition, showing that part of the decision-rule gap is actionable. In rank-matched comparisons, emotion information outside the native readout generalizes to held-out speakers and survives controls for measured acoustic descriptors, but replacing the selected readout-external directions usually has little effect on emitted answers. These results distinguish information availability from behavioral use and localize performance losses across the decision rule and the state-to-answer readout.

Figures

Figures reproduced from arXiv: 2608.06409 by the authors.

Figure 1
Figure 1. Overview of the generation-aligned diagnostic ladder. Left: audio and prompt tokens are processed by the encoder, projector, and language model, and the output head W maps the answer-position state xt to full-vocabulary logits. Four readouts score the same answer event: greedy generation over the full vocabulary (Agen), the post-hoc argmax over the four option logits (Aopt), a learned affine reader on the option-log… view at source ↗
Figure 2
Figure 2. Which gap dominates differs by condition. Each point is one model– corpus condition, with color denoting the system and shape the corpus. The dashed line marks equal gaps and the shading separates the two regimes: points in the blue region above the line lose more at the readout (coverage￾dominant), and points in the tan region below it lose more at the decision rule (decision-dominant). Exact values and intervals a… view at source ↗
Figure 3
Figure 3. A label-free logit correction, measured on the emitted answer. Gains are grouped by system, with a solid CREMA-D bar and a hatched VESUS bar. Bars show only the generated-accuracy gain over the uncorrected baseline; baseline and corrected accuracies, together with transmission and parseability diagnostics, are in Supplementary Table S7. Robustness analyses reproduce the gaps with nonlinear logit-side decoding, match… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Availability does not imply effective use. (a) Held-out decoding accuracy at L∗ for the native readout space V3, a label-free random subspace rand3 ⊂ V ⊥ 4 ⊂ V ⊥ 3 averaged over 20 draws, and the supervised space Sdecoding selected in the same complement; the dashed li…
Figure 5
Figure 5. Figure 5: Depth profile of causal access in Qwen2-Audio × CREMA￾D. Donor-following effects of matched rank-four minimal-pair replacement at eight depths, with speaker-clustered 95% intervals; hollow markers mark intervals containing zero, and the vertical axis is symmetric-logar…
Figure 6
Figure 6. Figure 6: Readout-external decodability under progressively stronger acous￾tic controls. Each line is one model–corpus condition, with color denoting the system, solid circles CREMA-D, and dashed squares VESUS. Shown is held￾out Sdecoding accuracy for the raw states, after input…
Figure 6
Figure 6. Figure 6: It gives the descriptor-only reference Adesc, decoding after linear removal of the ten descriptors, and the paired drop in Sdecoding accuracy. D. Subspace–Descriptor Associations Table S18 reports, for each condition and subspace, the single descriptor best predicted f…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 18 canonical work pages

  1. [1]

    Qwen2.5-omni technical report,

    J. Xu, Z. Guo, J. He, H. Hu, T. He, S. Bai, K. Chen, J. Wang, Y . Fan, K. Dang, B. Zhang, X. Wang, Y . Chu, and J. Lin, “Qwen2.5-omni technical report,” 2025. [Online]. Available: https://arxiv.org/abs/2503.20215

    arXiv 2025

  2. [2]

    Qwen2-audio technical report,

    Y . Chu, J. Xu, Q. Yanget al., “Qwen2-audio technical report,” arXiv preprint arXiv:2407.10759, 2024

    arXiv 2024

  3. [3]

    Audio flamingo 3: Advancing audio intelligence with fully open large audio language models,

    A. Goel, S. Ghosh, J. Kim, S. Kumar, Z. Kong, S.-g. Lee, C.-H. H. Yang, R. Duraiswami, D. Manocha, R. Valle, and B. Catanzaro, “Audio flamingo 3: Advancing audio intelligence with fully open large audio language models,”arXiv preprint arXiv:2507.08128, 2025

    arXiv 2025

  4. [4]

    Kimi-audio technical report,

    Kimi Team, “Kimi-audio technical report,” 2025

    work page 2025

  5. [5]

    Phi- 4 technical report,

    M. Abdin, J. Aneja, H. Behl, S. Bubeck, R. Eldan, S. Gunasekar, M. Harrison, R. J. Hewett, M. Javaheripi, P. Kauffmannet al., “Phi- 4 technical report,”arXiv preprint arXiv:2412.08905, 2024

    arXiv 2024

  6. [6]

    Dynamic-superb: Towards a dynamic, collaborative, and comprehensive instruction-tuning bench- mark for speech,

    C.-y. Huang, K.-H. Lu, S.-H. Wang, C.-Y . Hsiao, C.-Y . Kuan, H. Wu, S. Arora, K.-W. Chang, J. Shi, Y . Penget al., “Dynamic-superb: Towards a dynamic, collaborative, and comprehensive instruction-tuning bench- mark for speech,” inICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 12 136–12 140

    work page 2024

  7. [7]

    Air-bench: Benchmarking large audio-language models via generative comprehension,

    Q. Yang, J. Xu, W. Liu, Y . Chu, Z. Jiang, X. Zhou, Y . Leng, Y . Lv, Z. Zhao, C. Zhouet al., “Air-bench: Benchmarking large audio-language models via generative comprehension,” inProceedings of the 62nd An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2024, pp. 1979–1998

    2024

  8. [8]

    Sd-eval: A benchmark dataset for spoken dialogue under- standing beyond words,

    J. Ao, Y . Wang, X. Tian, D. Chen, J. Zhang, L. Lu, Y . Wang, H. Li, and Z. Wu, “Sd-eval: A benchmark dataset for spoken dialogue under- standing beyond words,”Advances in Neural Information Processing Systems, vol. 37, pp. 56 898–56 918, 2024

    work page 2024

Show all 39 references
  1. [9]

    V ocal communication of emotion: A review of research paradigms,

    K. R. Scherer, “V ocal communication of emotion: A review of research paradigms,”Speech Communication, vol. 40, no. 1–2, pp. 227–256, 2003

    2003

  2. [10]

    Communication of emotions in vocal expression and music performance: Different channels, same code?

    P. N. Juslin and P. Laukka, “Communication of emotions in vocal expression and music performance: Different channels, same code?” Psychological bulletin, vol. 129, no. 5, p. 770, 2003

    2003

  3. [11]

    Crema-d: Crowd-sourced emotional multimodal actors dataset,

    H. Cao, D. G. Cooper, M. K. Keutmann, R. C. Gur, A. Nenkova, and R. Verma, “Crema-d: Crowd-sourced emotional multimodal actors dataset,”IEEE transactions on affective computing, vol. 5, no. 4, pp. 377–390, 2014

    2014

  4. [12]

    Calibrate before use: Improving few-shot performance of language models,

    T. Z. Zhao, E. Wallace, S. Feng, D. Klein, and S. Singh, “Calibrate before use: Improving few-shot performance of language models,” in Proceedings of the 38th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 139. PMLR, 2021, pp....

    2021

  5. [13]

    Batch calibration: Rethinking calibration for in-context learning and prompt engineering,

    H. Zhou, X. Wan, L. Proleev, D. Mincu, J. Chen, K. Heller, and S. Roy, “Batch calibration: Rethinking calibration for in-context learning and prompt engineering,” inInternational Conference on Learning Representations, 2024, arXiv:2309.17249

    2024 arXiv

  6. [14]

    Speechgpt: Empowering large language models with intrinsic cross- modal conversational abilities,

    D. Zhang, S. Li, X. Zhang, J. Zhan, P. Wang, Y . Zhou, and X. Qiu, “Speechgpt: Empowering large language models with intrinsic cross- modal conversational abilities,” inFindings of the Association for Computational Linguistics: EMNLP 2023, 2023, pp. 15 757–15 773

    2023

  7. [15]

    Salmonn: Towards generic hearing abilities for large language models,

    C. Tang, W. Yu, G. Sun, X. Chen, T. Tan, W. Li, L. Lu, Z. Ma, and C. Zhang, “Salmonn: Towards generic hearing abilities for large language models,” inInternational Conference on Learning Represen- tations, vol. 2024, 2024, pp. 16 607–16 629

    2024

  8. [16]

    Audio flamingo: A novel audio language model with few-shot learning and dialogue abilities,

    Z. Kong, A. Goel, R. Badlani, W. Ping, R. Valle, and B. Catanzaro, “Audio flamingo: A novel audio language model with few-shot learning and dialogue abilities,”arXiv preprint arXiv:2402.01831, 2024

    2024 arXiv

  9. [17]

    Do audio LLMs really LISTEN, or just transcribe? measuring lexical vs. acoustic emotion cues reliance,

    J. Chen, Z. Guo, J. Chun, P. Wang, A. Perrault, and M. Elsner, “Do audio LLMs really LISTEN, or just transcribe? measuring lexical vs. acoustic emotion cues reliance,” inProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics...

    2026

  10. [18]

    Large language models are not robust multiple choice selectors,

    C. Zheng, H. Zhou, F. Meng, J. Zhou, and M. Huang, “Large language models are not robust multiple choice selectors,” inInternational Con- ference on Learning Representations, 2024, spotlight; arXiv:2309.03882

    2024 arXiv

  11. [19]

    Answer-level calibration for free-form multiple choice ques- tion answering,

    S. Kumar, “Answer-level calibration for free-form multiple choice ques- tion answering,” inProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Dublin, Ireland: Association for Computational Linguistics, 2022, pp. 665–679

    2022

  12. [20]

    Hubert: Self-supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,”IEEE/ACM transactions on audio, speech, and language processing, vol. 29, pp. 3451–3460, 2021

    2021

  13. [21]

    Wavlm: Large-scale self-supervised pre- training for full stack speech processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiaoet al., “Wavlm: Large-scale self-supervised pre- training for full stack speech processing,”IEEE Journal of Selected Topics in Signal Processing, vol. 16, no. 6, pp. 1505–1518, 2022

    2022

  14. [22]

    Robust speech recognition via large-scale weak supervi- sion,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervi- sion,” inInternational conference on machine learning. PMLR, 2023, pp. 28 492–28 518

    2023

  15. [23]

    Layer-wise analysis of a self-supervised speech representation model,

    A. Pasad, J.-C. Chou, and K. Livescu, “Layer-wise analysis of a self-supervised speech representation model,” in2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE, 2021, pp. 914–921

    2021

  16. [24]

    Probing phoneme, language and speaker information in unsupervised speech representations,

    M. De Seyssel, M. Lavechin, Y . Adi, E. Dupoux, and G. Wisniewski, “Probing phoneme, language and speaker information in unsupervised speech representations,” inInterspeech 2022, 2022, pp. 1402–1406

    2022

  17. [25]

    Dawn of the transformer era in speech emotion recognition: Closing the valence gap,

    J. Wagner, A. Triantafyllopoulos, H. Wierstorf, M. Schmitt, F. Burkhardt, F. Eyben, and B. W. Schuller, “Dawn of the transformer era in speech emotion recognition: Closing the valence gap,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023

    2023

  18. [26]

    emotion2vec: Self-supervised pre-training for speech emotion repre- sentation,

    Z. Ma, Z. Zheng, J. Ye, J. Li, Z. Gao, S. Zhang, and X. Chen, “emotion2vec: Self-supervised pre-training for speech emotion repre- sentation,” inFindings of the Association for Computational Linguistics: ACL 2024, 2024, pp. 15 747–15 760

    2024

  19. [27]

    AudioLens: A closer look at auditory attribute perception of large audio-language models,

    C.-K. Yang, N. Ho, Y .-J. Lee, and H.-y. Lee, “AudioLens: A closer look at auditory attribute perception of large audio-language models,”arXiv preprint arXiv:2506.05140, 2025

    2025 arXiv

  20. [28]

    Probing classifiers: Promises, shortcomings, and ad- vances,

    Y . Belinkov, “Probing classifiers: Promises, shortcomings, and ad- vances,”Computational Linguistics, vol. 48, no. 1, pp. 207–219, 2022

    2022

  21. [29]

    Interpreting GPT: The logit lens,

    nostalgebraist, “Interpreting GPT: The logit lens,” 2020, less- Wrong post; https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/ interpreting-gpt-the-logit-lens

    2020

  22. [30]

    Eliciting latent predictions from trans- formers with the tuned lens,

    N. Belrose, I. Ostrovsky, L. McKinney, Z. Furman, L. Smith, D. Halawi, S. Biderman, and J. Steinhardt, “Eliciting latent predictions from trans- formers with the tuned lens,”arXiv preprint arXiv:2303.08112, 2023

    2023 arXiv

  23. [31]

    Investigating gender bias in language models using causal mediation analysis,

    J. Vig, S. Gehrmann, Y . Belinkov, S. Qian, D. Nevo, Y . Singer, and S. Shieber, “Investigating gender bias in language models using causal mediation analysis,”Advances in neural information processing systems, vol. 33, pp. 12 388–12 401, 2020

    2020

  24. [32]

    Causal abstractions of neural networks,

    A. Geiger, H. Lu, T. Icard, and C. Potts, “Causal abstractions of neural networks,” inAdvances in Neural Information Processing Systems, vol. 34, 2021, pp. 9574–9586

    2021

  25. [33]

    Locating and editing factual associations in gpt,

    K. Meng, D. Bau, A. Andonian, and Y . Belinkov, “Locating and editing factual associations in gpt,”Advances in neural information processing systems, vol. 35, pp. 17 359–17 372, 2022

    2022

  26. [34]

    Is this the subspace you are looking for? an interpretability illusion for subspace activation patching,

    A. Makelov, G. Lange, A. Geiger, and N. Nanda, “Is this the subspace you are looking for? an interpretability illusion for subspace activation patching,” arXiv preprint arXiv:2311.17030, 2023

    2023 arXiv

  27. [35]

    Bridging the knowledge-prediction gap in LLMs on multiple-choice questions,

    Y . Park, H. Pyun, and Y . Jo, “Bridging the knowledge-prediction gap in LLMs on multiple-choice questions,” inProc. International Conference on Machine Learning, 2026, arXiv:2509.23782

    2026 arXiv

  28. [36]

    AR&D: A framework for retrieving and describing concepts for interpreting AudioLLMs,

    T. F. Chowdhury, D. H. Ta, S. Pan, J. Stoddard, and Z. Liao, “AR&D: A framework for retrieving and describing concepts for interpreting AudioLLMs,” inProc. IEEE International Conference on Acoustics, Speech and Signal Processing, 2026, arXiv:2602.22253

    2026

  29. [37]

    Who wins the conflict? mechanistic interpretability of text bias in audio LLMs,

    H. Cho, S. Yoo, J. Jang, C. Kim, and J. S. Chung, “Who wins the conflict? mechanistic interpretability of text bias in audio LLMs,” 2026

    2026

  30. [38]

    VESUS: A crowd-annotated database to study emotion production and perception in spoken English,

    J. Sager, R. Shankar, J. Reinhold, and A. Venkataraman, “VESUS: A crowd-annotated database to study emotion production and perception in spoken English,” inInterspeech, 2019

    2019

  31. [39]

    A theory of usable information under computational constraints,

    Y . Xu, S. Zhao, J. Song, R. Stewart, and S. Ermon, “A theory of usable information under computational constraints,” inInternational Conference on Learning Representations (ICLR), 2020. 11 TABLE S1 THE FIVE EVALUATED SYSTEMS. PARAMETER COUNTS ARE TAKEN FROM THE PUBLIC MODEL C...

    2020

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.