AVLLMs encode audio semantics in middle layers but suppress them in final text outputs when audio conflicts with vision, due to training that largely inherits from vision-language base models.
Behind the scenes: Mechanistic interpretability of lora-adapted whis- per for speech emotion recognition
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
citation-role summary
background 1
citation-polarity summary
years
2026 2roles
background 1polarities
background 1representative citing papers
Pretrained audio-visual speech encoders adapted with LoRA improve multimodal voice activity projection for turn-taking prediction across multiple languages and a robot mediation corpus.
citing papers explorer
-
Do Audio-Visual Large Language Models Really See and Hear?
AVLLMs encode audio semantics in middle layers but suppress them in final text outputs when audio conflicts with vision, due to training that largely inherits from vision-language base models.
-
Multimodal Voice Activity Projection for Turn-Taking in Social Robots with Voice-Activity-Related Pretrained Encoders
Pretrained audio-visual speech encoders adapted with LoRA improve multimodal voice activity projection for turn-taking prediction across multiple languages and a robot mediation corpus.