pith. sign in

arxiv: 2606.25459 · v1 · pith:D7C5B2SZnew · submitted 2026-06-24 · 💻 cs.CL

Probing in the Wild: A Case Study of Self-Supervised Speech Representations on Mandarin Sub-dialects with Unsupervised Articulatory Analysis

Shu Shang , Fuliang Weng , Zeqian Hu , Yaqian Zhou This is my paper

Pith reviewed 2026-06-25 21:02 UTC · model grok-4.3

classification 💻 cs.CL
keywords self-supervised speech modelsarticulatory featuresMandarin sub-dialectsunsupervised probingdialect variationphone recognitionrepresentation analysis
0
0 comments X

The pith

Self-supervised speech models show structured variation in how well they represent articulatory features across Mandarin sub-dialects.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tests whether a fully unlabeled pipeline can reveal how self-supervised speech models encode phonetic details when the input comes from real Mandarin sub-dialect recordings rather than curated data. Phone sequences are produced by a universal recognizer and converted into articulatory feature vectors so that each frame of dialect speech can be probed directly inside the model. Results indicate that features tied to clear acoustic cues remain easy to decode across dialects, while features that depend on finer spectral detail vary more, mainly because Beijing speech yields higher decodability than the other sub-dialects. The same pipeline also shows that these two groups of features exhibit different layer-wise dynamics inside the model.

Core claim

Articulatory feature decodability follows a structured pattern across Mandarin sub-dialects: acoustically salient features such as labiality and stridency remain comparatively stable, whereas features associated with finer spectral distinctions exhibit larger dialect-dependent variation driven primarily by elevated decodability for Beijing speech relative to other sub-dialects, with distinct representational dynamics visible across layers.

What carries the argument

Unsupervised articulatory probing pipeline that maps output from a language-agnostic universal phone recognizer to articulatory feature vectors for frame-level analysis on unlabeled dialect speech.

If this is right

  • Acoustically salient articulatory features maintain stable decodability across Mandarin sub-dialects.
  • Features linked to finer spectral distinctions display greater dialect-dependent variation.
  • Beijing speech produces elevated decodability compared with other Mandarin sub-dialects.
  • Stable and variable feature groups display distinct layer-wise representational dynamics.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same pipeline could be applied directly to other unlabeled dialect or language corpora without requiring new annotations.
  • Self-supervised models may encode prominent acoustic cues more reliably than subtle spectral distinctions when faced with natural variation.
  • Targeted fine-tuning on non-Beijing varieties might reduce the observed dialect asymmetry in finer-feature representations.

Load-bearing premise

The language-agnostic universal phone recognizer produces phone sequences that can be mapped to articulatory feature vectors with sufficient accuracy to support frame-level probing of dialect speech without manual annotation or dialect-specific adjustments.

What would settle it

Repeating the probing analysis on the same recordings but with manually verified phone labels instead of the universal recognizer output and finding that the reported stability-versus-variation pattern disappears or reverses.

Figures

Figures reproduced from arXiv: 2606.25459 by Fuliang Weng, Shu Shang, Yaqian Zhou, Zeqian Hu.

Figure 1
Figure 1. Figure 1: Overview of our unsupervised probing pipeline. In summary, our contributions are: • We adapt a language-agnostic pseudo-labeling pipeline for articulatory probing of unlabeled Mandarin sub￾dialects. arXiv:2606.25459v1 [cs.CL] 24 Jun 2026 [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Heatmap of Macro-F1 scores for articulatory feature decoding across eight Mandarin sub-dialects. Darker colors indicate higher classification performance. 3.2. Hierarchy of Representation The disparity between the Beijing sub-dialect and other vari￾eties is further visualized in [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Comparison of Beijing speech to the envelope of seven other Mandarin sub-dialects across phonological features. For each feature, the horizontal bar shows the range of scores among the other dialects, the blue point indicates their mean, and the orange point indicates Beijing. 12 transformer layers [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Layer-wise Macro-F1 scores for a stable feature (labial) and an unstable feature (nasal) comparing Beijing to the average of other Mandarin sub-dialects. Points indicate mean F1 per layer. Error bars represent standard deviation across non-Beijing sub-dialects. sub-dialects follow roughly the same pattern as robust features, performance for the Beijing sub-dialect is highly erratic. We observe dramatic per… view at source ↗
read the original abstract

While self-supervised speech models have achieved strong performance across speech tasks, relatively little is known about how their internal phonetic representations behave under fine-grained dialect variation. Existing probing studies typically rely on curated corpora with manual phonetic annotations, limiting their applicability to naturally occurring dialect speech. We present a case study of articulatory feature representations in a Mandarin self-supervised speech model using an entirely unlabeled probing pipeline. Phone sequences are generated using a language-agnostic universal phone recognizer and mapped to articulatory feature vectors, enabling frame-level probing without manual annotation. Our results reveal a structured pattern in articulatory feature decodability across Mandarin sub-dialects. Acoustically salient features such as labiality and stridency remain comparatively stable, whereas features associated with finer spectral distinctions exhibit larger dialect-dependent variation. This variation is driven primarily by elevated decodability for Beijing speech relative to other Mandarin sub-dialects. Layer-wise analyses further show distinct representational dynamics for these feature groups. These findings suggest that language-agnostic articulatory probing can be applied to real-world dialect corpora and that dialect sensitivity in self-supervised speech representations is unevenly distributed across articulatory dimensions.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 0 minor

Summary. The paper presents a case study of articulatory feature probing in a Mandarin self-supervised speech model on unlabeled sub-dialect corpora. It uses a language-agnostic universal phone recognizer to generate phone sequences, maps them to articulatory feature vectors, and performs frame-level probing without manual annotations or dialect-specific tuning. The central claim is that decodability shows a structured pattern: acoustically salient features (labiality, stridency) are stable across dialects while finer spectral features vary, with the variation driven primarily by elevated performance on Beijing speech; layer-wise analyses reveal distinct dynamics for these groups.

Significance. If the empirical patterns hold after validation, the work would demonstrate that language-agnostic articulatory probing can be applied to real-world unlabeled dialect data and that dialect sensitivity in SSL representations is uneven across articulatory dimensions rather than uniform.

major comments (2)
  1. [Methods / pipeline] Methods / pipeline description: the central claim depends on the untested assumption that the language-agnostic phone recognizer produces phone sequences accurate enough for reliable frame-level articulatory feature mapping on the target sub-dialect corpora. No phone error rates, confusion matrices, or cross-dialect validation results are supplied, so systematic recognizer bias (e.g., higher confusion on non-Beijing spectral distinctions) could produce the reported stable-vs-variable split and Beijing elevation as artifacts rather than properties of the SSL model.
  2. [Results] Results section: the abstract and headline findings supply no quantitative values (accuracies, deltas, error bars, statistical tests, or dataset sizes), making it impossible to assess effect sizes or whether the dialect-dependent variation is statistically supported.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive feedback on our manuscript. We address each major comment below and indicate planned revisions where appropriate.

read point-by-point responses

  1. Referee: [Methods / pipeline] Methods / pipeline description: the central claim depends on the untested assumption that the language-agnostic phone recognizer produces phone sequences accurate enough for reliable frame-level articulatory feature mapping on the target sub-dialect corpora. No phone error rates, confusion matrices, or cross-dialect validation results are supplied, so systematic recognizer bias (e.g., higher confusion on non-Beijing spectral distinctions) could produce the reported stable-vs-variable split and Beijing elevation as artifacts rather than properties of the SSL model.

    Authors: We acknowledge this is a valid concern and a genuine limitation of the current pipeline. Because the sub-dialect corpora are unlabeled, computing phone error rates or confusion matrices would require new manual annotation, which is outside the scope of the present study. We will add a dedicated limitations subsection that explicitly discusses the possibility of recognizer bias, references prior cross-lingual validation of the universal phone recognizer, and notes that any such bias would need to align precisely with acoustic salience to produce the observed stable-vs-variable split. This is a partial revision, as full empirical validation is not feasible without additional labeled data. revision: partial

  2. Referee: [Results] Results section: the abstract and headline findings supply no quantitative values (accuracies, deltas, error bars, statistical tests, or dataset sizes), making it impossible to assess effect sizes or whether the dialect-dependent variation is statistically supported.

    Authors: We agree that the abstract should be more self-contained. In the revised manuscript we will update the abstract to report key quantitative details, including approximate dataset sizes per sub-dialect, mean probing accuracies (with ranges) for the stable versus variable feature groups, and a brief mention of the statistical tests used. The main results section already contains these values with error bars and significance tests; the revision will ensure the headline claims are quantitatively grounded. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical probing results with no fitted predictions or self-referential derivations

full rationale

The paper describes an empirical probing pipeline that generates phone sequences from a language-agnostic recognizer, maps them to articulatory features, and measures decodability across sub-dialects. No equations, parameter fitting, or derivation steps are present that would reduce outputs to inputs by construction. The reported patterns (stable vs. variable features, Beijing elevation) are observational outcomes of the probing experiments rather than predictions forced by any fitted quantity or self-citation chain. The recognizer accuracy is an external methodological assumption whose validity is separate from circularity; it does not create a self-definitional loop or rename a known result. This is a standard non-circular empirical study.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim depends on the accuracy of the universal phone recognizer and the validity of the phone-to-articulatory mapping as a proxy for phonetic content in unlabeled dialect data.

axioms (1)
  • domain assumption The language-agnostic universal phone recognizer produces phone sequences that map reliably to articulatory feature vectors for frame-level analysis of dialect speech.
    Invoked as the foundation of the entirely unlabeled probing pipeline.

pith-pipeline@v0.9.1-grok · 5748 in / 1228 out tokens · 32387 ms · 2026-06-25T21:02:20.648694+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

28 extracted references · 2 canonical work pages · 1 internal anchor

  1. [1]

    Probing in the Wild: A Case Study of Self-Supervised Speech Representations on Mandarin Sub-dialects with Unsupervised Articulatory Analysis

    Introduction Self-supervised speech representation models have become the default backbone for modern speech systems by enabling mod- els to learn general-purpose acoustic representations from un- labeled audio, which are then transferred to downstream tasks. However, they remain opaque in how they encode and represent the wealth of phonetic information w...

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  2. [2]

    mandarin

    Methods 2.1. Dataset To study how SSL representations encode fine-grained Man- darin sub-dialect variation, we require a corpus containing mul- tiple closely related sub-dialects with sufficient speaker diver- sity and consistent recording conditions. We conduct our experiments on the KeSpeech corpus [17], a large-scale Mandarin speech dataset containing ...

  3. [3]

    Entering Tone

    Results 3.1. Heterogeneity of Representation Figure 2 shows a per-dialect and per-feature breakdown of the model’s articulatory representation ability. A clear boundary is drawn between the Beijing Mandarin sub-dialect and all other Mandarin sub-dialects. As a qualitative sanity check, we examine whether the prob- ing results recover known phonological pa...

    2088

  4. [4]

    Hierarchy in Sensitivities to Acoustic Features An examination of the probing performance reveals an asym- metry in how SSL models generalize in different acoustic prop- erties

    Discussion 4.1. Hierarchy in Sensitivities to Acoustic Features An examination of the probing performance reveals an asym- metry in how SSL models generalize in different acoustic prop- erties. For robustly encoded features (e.g.strident,labial), the model maintains a consistent baseline across dialects. However, fine-grained features (e.g.back,coronal) e...

  5. [5]

    First, linear probing only measures the linear decodability of representations

    Limitations and Future Directions Despite these findings, our approach has several limitations. First, linear probing only measures the linear decodability of representations. This may not accurately reflect how such rep- resentations are utilized by downstream tasks. Second, our cur- rent work examines only a single SSL model (wav2vec 2.0) trained on Man...

  6. [6]

    Conclusion This work utilizes a cross-dialectal unsupervised articulatory probing framework to dissect the phonetic representations of self-supervised speech models in unlabeled datasets. Our em- pirical analysis suggests a potential hierarchy in how SSL en- coders represent articulatory features: while SSL encoders suc- cessfully generalize in robust aco...

  7. [7]

    Self-Supervised Speech Representa- tion Learning: A Review,

    A. Mohamed, H.-y. Lee, L. Borgholt, J. D. Havtorn, J. Edin, C. Igel, K. Kirchhoff, S.-W. Li, K. Livescu, L. Maaløe, T. N. Sainath, and S. Watanabe, “Self-Supervised Speech Representa- tion Learning: A Review,”IEEE Journal of Selected Topics in Signal Processing, vol. 16, no. 6, pp. 1179–1210, 2022

    2022

  8. [8]

    Domain-Informed Probing of wav2vec 2.0 Embeddings for Pho- netic Features,

    P. Cormac English, J. D. Kelleher, and J. Carson-Berndsen, “Domain-Informed Probing of wav2vec 2.0 Embeddings for Pho- netic Features,” inProceedings of the 19th SIGMORPHON Work- shop on Computational Research in Phonetics, Phonology, and Morphology. Seattle, Washington: Association for Computa- tional Linguistics, 2022, pp. 83–91

    2022

  9. [9]

    Probing Self- supervised Speech Models for Phonetic and Phonemic Informa- tion: A Case Study in Aspiration,

    K. Martin, J. Gauthier, C. Breiss, and R. Levy, “Probing Self- supervised Speech Models for Phonetic and Phonemic Informa- tion: A Case Study in Aspiration,” inInterspeech 2023. ISCA, 2023, pp. 251–255

    2023

  10. [10]

    Orthogo- nality and isotropy of speaker and phonetic information in self- supervised speech representations,

    M. Mohamed, O. D. Liu, H. Tang, and S. Goldwater, “Orthogo- nality and isotropy of speaker and phonetic information in self- supervised speech representations,” inInterspeech 2024, 2024, pp. 3625–3629

    2024

  11. [11]

    Analyzing hidden representations in end-to-end automatic speech recognition systems,

    Y . Belinkov and J. R. Glass, “Analyzing hidden representations in end-to-end automatic speech recognition systems,” inAdvances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, I. Guyon, U. von Luxburg, S. Ben- gio, H. M. Wallach, R. Fergus, S. V . N. Vis...

    2017

  12. [12]

    SUPERB: Speech Processing Universal PERformance Benchmark,

    S. wen Yang, P.-H. Chi, Y .-S. Chuang, C.-I. J. Lai, K. Lakhotia, Y . Y . Lin, A. T. Liu, J. Shi, X. Chang, G.-T. Lin, T.-H. Huang, W.-C. Tseng, K. tik Lee, D.-R. Liu, Z. Huang, S. Dong, S.-W. Li, S. Watanabe, A. Mohamed, and H. yi Lee, “SUPERB: Speech Processing Universal PERformance Benchmark,” inInterspeech 2021, 2021, pp. 1194–1198

    2021

  13. [13]

    Echoes of Phonetics: Unveiling Relevant Acoustic Cues for ASR via Feature Attribution,

    D. Fucci, M. Gaido, M. Negri, M. Cettolo, and L. Bentivogli, “Echoes of Phonetics: Unveiling Relevant Acoustic Cues for ASR via Feature Attribution,” inInterspeech 2025. ISCA, 2025, pp. 206–210

    2025

  14. [14]

    Visualizing Automatic Speech Recognition – Means for a Better Understanding?

    K. Markert, R. Parracone, M. Kulakov, P. Sperl, C.-Y . Kao, and K. B ¨ottinger, “Visualizing Automatic Speech Recognition – Means for a Better Understanding?” in2021 ISCA Symposium on Security and Privacy in Speech Communication, 2021, pp. 14–20

    2021

  15. [15]

    What Do Self- Supervised Speech Models Know About Words?

    A. Pasad, C.-M. Chien, S. Settle, and K. Livescu, “What Do Self- Supervised Speech Models Know About Words?”Transactions of the Association for Computational Linguistics, vol. 12, pp. 372– 391, 2024

    2024

  16. [16]

    Similarity analysis of self-supervised speech representations,

    Y .-A. Chung, Y . Belinkov, and J. Glass, “Similarity analysis of self-supervised speech representations,” inICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021, pp. 3040–3044

    2021

  17. [17]

    Do Acoustic Word Embeddings Capture Phonologi- cal Similarity? An Empirical Study,

    B. M. Abdullah, M. Mosbach, I. Zaitova, B. M ¨obius, and D. Klakow, “Do Acoustic Word Embeddings Capture Phonologi- cal Similarity? An Empirical Study,” inInterspeech 2021, 2021, pp. 4194–4198

    2021

  18. [18]

    Sound analogies with phoneme embeddings,

    M. P. Silfverberg, L. Mao, and M. Hulden, “Sound analogies with phoneme embeddings,” inProceedings of the Society for Compu- tation in Linguistics (SCiL) 2018, G. Jarosz, B. O’Connor, and J. Pater, Eds., 2018, pp. 136–144

    2018

  19. [19]

    Darpa timit acoustic-phonetic continous speech corpus cd-rom. nist speech disc 1-1.1,

    J. S. Garofolo, L. F. Lamel, W. M. Fisher, J. G. Fiscus, and D. S. Pallett, “Darpa timit acoustic-phonetic continous speech corpus cd-rom. nist speech disc 1-1.1,”NASA STI/Recon technical report n, vol. 93, p. 27403, 1993

    1993

  20. [20]

    The Chinese dialects: phonology,

    J. Norman, “The Chinese dialects: phonology,”The Sino-Tibetan languages, vol. 3, no. 1, pp. 72–83, 2003

    2003

  21. [21]

    Probing phoneme, language and speaker information in unsupervised speech representations,

    M. de Seyssel, M. Lavechin, Y . Adi, E. Dupoux, and G. Wis- niewski, “Probing phoneme, language and speaker information in unsupervised speech representations,” inInterspeech 2022, 2022, pp. 1402–1406

    2022

  22. [22]

    wav2vec 2.0: A framework for self-supervised learning of speech representa- tions,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representa- tions,” inAdvances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Sys- tems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020

    2020

  23. [23]

    Kespeech: An open source speech dataset of mandarin and its eight subdialects,

    Z. Tang, D. Wang, Y . Xu, J. Sun, X. Lei, S. Zhao, C. Wen, X. Tan, C. Xie, S. Zhouet al., “Kespeech: An open source speech dataset of mandarin and its eight subdialects,” inThirty-fifth Conference on Neural Information Processing Systems Datasets and Bench- marks Track (Round 2), 2021

    2021

  24. [24]

    Universal phone recognition with a multilingual allo- phone system,

    X. Li, S. Dalmia, J. Li, M. Lee, P. Littell, J. Yao, A. Anas- tasopoulos, D. R. Mortensen, G. Neubig, A. W. Black, and F. Metze, “Universal phone recognition with a multilingual allo- phone system,” in2020 IEEE International Conference on Acous- tics, Speech and Signal Processing, ICASSP 2020, Barcelona, Spain, May 4-8, 2020. IEEE, 2020, pp. 8249–8253

    2020

  25. [25]

    PanPhon: A resource for mapping IPA segments to articulatory feature vectors,

    D. R. Mortensen, P. Littell, A. Bharadwaj, K. Goyal, C. Dyer, and L. Levin, “PanPhon: A resource for mapping IPA segments to articulatory feature vectors,” inProceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, Y . Matsumoto and R. Prasad, Eds. Osaka, Japan: The COLING 2016 Organizing Committee, Dec...

    2016

  26. [26]

    Wenetspeech: A 10000+ hours multi- domain mandarin corpus for speech recognition,

    B. Zhang, H. Lv, P. Guo, Q. Shao, C. Yang, L. Xie, X. Xu, H. Bu, X. Chen, C. Zenget al., “Wenetspeech: A 10000+ hours multi- domain mandarin corpus for speech recognition,” inICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 6182–6186

    2022

  27. [27]

    Layer-Wise Analysis of a Self-Supervised Speech Representation Model,

    A. Pasad, J.-C. Chou, and K. Livescu, “Layer-Wise Analysis of a Self-Supervised Speech Representation Model,”2021 IEEE Auto- matic Speech Recognition and Understanding Workshop (ASRU), pp. 914–921, 2021

    2021

  28. [28]

    Machine learning in python: Main developments and technology trends in data sci- ence, machine learning, and artificial intelligence,

    S. Raschka, J. Patterson, and C. Nolet, “Machine learning in python: Main developments and technology trends in data sci- ence, machine learning, and artificial intelligence,”arXiv preprint arXiv:2002.04803, 2020

    work page arXiv 2002