InsideSSL analyzes self-supervised speech models layer-by-layer using entropy, curvature, robustness metrics, and a cross-layer Generative Compatibility Matrix, finding that training objectives induce distinct compression and geometric regimes that correlate with downstream task performance.
InsideSSL: Understanding Self-Supervised Speech Representations using a Model-Centric Perspective
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Self-supervised learning (SSL) models, such as Wav2Vec2, HuBERT, and WavLM, have become foundational across a wide range of speech and audio tasks. Despite their success, understanding their internal layer-wise dynamics remains an ongoing challenge. To address this, we propose a two-part model-centric framework called InsideSSL. First, we establish a task-agnostic analysis from three intrinsic per-layer perspectives: compression (entropy), geometry (curvature), and robustness to perturbations. We show that varying training objectives induce distinct regimes of acoustic compression and manifold unfolding. Second, we introduce the cross-layer Generative Compatibility Matrix (GCM) to evaluate functional transferability, exposing stable phonetic cores, identity volatility, and deep-layer semantic pruning. In addition to these evaluations, linear probing connects the model-centric perspective to downstream tasks, demonstrating how layer topology dictates phoneme, pitch, and speaker encoding.
fields
cs.SD 1years
2026 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
InsideSSL: Understanding Self-Supervised Speech Representations using a Model-Centric Perspective
InsideSSL analyzes self-supervised speech models layer-by-layer using entropy, curvature, robustness metrics, and a cross-layer Generative Compatibility Matrix, finding that training objectives induce distinct compression and geometric regimes that correlate with downstream task performance.